rsync hangs in cron (not SSH-problem)

Pascal Nobus pascal at nobus.be
Thu Jun 3 00:44:35 GMT 2004


>>This is the case
>>
>>- mounted Inetpub's windows-webserver on /mnt/web1 /mnt/web2, etc.
>>- rsync this to local dir:
>>	rsync -av --delete /mnt/web1 /mass/kuurne/day
>>	rsync -av --delete /mnt/web2 /mass/kuurne/day
>>	etc..
>>
>>- when logged in, everything works (I do see some errors about 
>>non-existing files, but rsync won't stop.
>>
>>
>>When used this command in cron
>>
>>00 01 * * * rsync -av --delete /mnt/web1 /mass/kuurne/day
>>00 02 * * * rsync -av --delete /mnt/web2 /mass/kuurne/day
>>etc..
>>
>>Rsync hangs, it doesn't finish!
>>
>>root      2036  5.7 10.4 27616 26704 ?       S    01:00   5:19 rsync -av 
>>--delete /mnt/web1 /mass/kuurne/day
>>root      2037  3.4 11.0 29028 28104 ?       S    01:00   3:09 rsync -av 
>>--delete /mnt/web1 /mass/kuurne/day
>>root      2048  3.1 11.0 29060 28132 ?       S    01:11   2:36 rsync -av 
>>--delete /mnt/web1 /mass/kuurne/day
>>root      2062 10.0  7.9 21304 20168 ?       S    02:00   3:19 rsync -av 
>>--delete /mnt/web4 /mass/kuurne/day
>>root      2064  4.9  8.2 22208 21056 ?       S    02:00   1:37 rsync -av 
>>--delete /mnt/web4 /mass/kuurne/day
>>root      2094  5.7  8.2 22252 21096 ?       S    02:05   1:34 rsync -av 
>>--delete /mnt/web4 /mass/kuurne/day
> 
> 
> It looks like you have 3 of each running concurrently.

The concurrent rsync processes seem to spawn itself (this also happens 
when rsync run from command line.



>   Uh, that memory's pretty full.  Once you start paging to disk you may
> never finish rsync.  Does the system become unresponsive?  Make sure
> there are no other cron jobs that might add memory pressure timed to run
> soon before (or concurrent to) the rsync.

The system doesn't become unresponsive (for the first day), but after 
some weeks it gets slower and slower (Saw once there hundreds of rsync 
processes)
The memory is just "filled" with buffered files, total off NOT 
rsync-prog's is about 10 MB (there's nothing else to do for this server)


>   You can also 'ls -al /proc/{pid-of-rsync}/fd' to see what files are 
> open.
>   Also, just to ensure that rsyncs don't run at the same time, consider 
> making just one cron job which is a script containing each rsync command 
> ending with semicolon (;), no backgrounding.  They will always run 
> serially, then.

Well, did that, but hangs again.
Strange thing I saw:
when starting up:
   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
29221 root      18   0 28184  27M  9484 S     9.7 11.0   6:26 rsync
29213 root      11   0 28112  27M  9476 S     6.3 11.0   4:25 rsync
29212 root      17   0 26712  26M   576 S    15.8 10.4   8:39 rsync

Two hours later:
29221 root       9   0 28172  27M  2860 S     0.0 11.0   7:01 rsync
29213 root       9   0 28112  27M  1528 S     0.0 11.0   4:49 rsync
29212 root       9   0 26724  26M   576 S     0.0 10.4   9:36 rsync

Seems like it's dead again?



Find out 1 thing:
If rsync is used without the -v option, it doesn't hang...
However I want to use this option, so I can see quick which files where 
changed, and that the copy was succesfull.







More information about the rsync mailing list