[ECC2] Duplicate DPs becoming hot bed

[hawkeye177]

IMPORTANT

Duplicate DPs - Submission of DPs to the server that have already been submitted. Typically done by the person who already submitted the DP.

People, this is becoming a problem. As many of you may know, the way the ECC2 system is set up is to by default automatically submit DPs found to the DP server at a predetermined frequency. This default process can be circumvented in a number of ways. Any number of these "manual submission" processes cause the submission of DPs to become very delicate and in some cases it fails to work properly altogether.

When this happens the DPs that have been submitted already get sent again. Hence the duplication. For a glimpse of the issue from a bandwidth standpoint, the first thing to keep in mind that this project is not being handled by a free university paid connection to the internet. It is being paid for out of the project leaders pocket.

Sending a DP value a second time never gets you anything but grief. Do it enough times, you'll be blacklisted from sending any!

Now, imagine that you have 100 Dps that you submit that are duplicates. The average size of a DP submission packet sent to the DP server is 150 bytes. That means that you send 15,000 bytes to the server every time you want to submit your 100 DPs. So what. That's not much is it? Well, keep in mind that it's measured in bytes, not bits as bandwidth on a broadband connection is. Think about this. If you use the default 5 minute per submission timing, that means you send the 15,000 bytes 12 times. That comes out to about 180K per hour.

Again. Still not a big hit right? Well, what if I told you there were at least 1200 people on the project right now. If 10% of those people also had those same 100 DPs duplicated, that means the bandwidth of 180K per hour is not multiplied 120 times. Or 21.6MB per hour. You also have to understand that this space is being handled by a router, the LAN where the DP server is located and other things.

And to be completely honest, 100 dupes is not really very high. We have seen at one point 4000 duplicated DPs sent in at a time. It's becoming even more of a problem as more and more people get involved in the project. So we are going to fix this problem or things could get ugly.

Here's the deal. Teams need to educate their members as to the proper handling of manual DP sumissions. If you do not understand, then ASK. Someone is almost always around to help on the IRC channel. Email works wonders too.

Which is the second thing we are all going to work on. We are all going to start checking Emails much more frequently at the email account used for submission. Things come up. Clients are found to be buggy. Etc. The ONLY way we can find you folks is via that email address. You MUST be diligent in checking for messages from us. The reasons why you must are very simple.

1. Duplicate DP submission is becoming a problem and we are working on ways to solve the issue before it becomes too big to handle.
2. If the issue does NOT get resolved we will activate code that is already in the DP server that will "ban" users that continually send in duplicates above an already determined "safe" level. We simply have to "flip the switch" and the bannage begins.

On another note, we have done some research and have determined that the manual submission page is a big part of the problem. But of course, the page itself is not the problem, the people that use it improperly are. And it's also interesting to note that the problem coincides with large team flushes that have started occuring.

We have been extremely patient up to now and have tried to teach people to solve the issue on their own, and not banning anyone. But this is reaching epidemic proportions. Please help us keep the project viable and long lived.

-The ECC2 staff

For more info and or comments please see this thread http://www.ecc2.com/forum/viewtopic.php?t=206

Thanks.

[nukes]

That sounds fair, they have to stop it before it kills the project, like it did in seti@home.

[PresterJohn]

my hunch is they're going to have to turn on the filter & start banning people. may not happen right away, but it's just a matter of time.

i don't think people are discplined enough to check their dupes. so ultimately, this will wind up hurting the DC since they will 'lose' people.

[eagle1]

Well.. Maybe this just happens to people with a huge herd.
Sometimes people send a dplist from one machine but they forget to delete it and send it again, etc.
But is is fair to educate our members in doing this!

One thing I always do is once I send a dplist, I change it's name.
For example:

Computer 1: /ecc2/dplist -----> When I sumbit this file I mv to
/ecc2/backup/dplist_1

Computer 2: the same process... etc.

This way I know if I send the file or not.!