too many connections on one module?

[kimor79 at sonic.net]

> On Thu, 2006-04-27 at 12:13 -0700, kimor79 at sonic.net wrote:
>> The second pass is the problem. On this pass the clients are syncing
>> about
>> 10 files from the previously excluded dir only. For some reason rsync
>> sessions take an excessively long time to complete (hours as opposed to
>> minutes). Eventually the connections pile up and load on the server has
>> gotten as high as 900!
>
> This previously excluded directory has a lot of big files, right?  In
> rsync's incremental transfer protocol, the sender uses a considerable
> amount of CPU time: it scans each file byte by byte, looking for regions
> matching the block hashes it got from the receiver.  This usually isn't
> a problem, but with 1000 clients at a time, I can see why the
> connections back up.
>
> Having the clients disable incremental transfer on the second pass by
> passing --whole-file will remove the CPU load but increase network
> traffic.
>

The previously excluded directory contains all the small/empty files.

I'll try the -W option (not worried about network traffic).

Thanks
Kimo