rsync hangs in cron (not SSH-problem)
Pascal Nobus
pascal at nobus.be
Thu Jun 3 00:44:35 GMT 2004
>>This is the case
>>
>>- mounted Inetpub's windows-webserver on /mnt/web1 /mnt/web2, etc.
>>- rsync this to local dir:
>> rsync -av --delete /mnt/web1 /mass/kuurne/day
>> rsync -av --delete /mnt/web2 /mass/kuurne/day
>> etc..
>>
>>- when logged in, everything works (I do see some errors about
>>non-existing files, but rsync won't stop.
>>
>>
>>When used this command in cron
>>
>>00 01 * * * rsync -av --delete /mnt/web1 /mass/kuurne/day
>>00 02 * * * rsync -av --delete /mnt/web2 /mass/kuurne/day
>>etc..
>>
>>Rsync hangs, it doesn't finish!
>>
>>root 2036 5.7 10.4 27616 26704 ? S 01:00 5:19 rsync -av
>>--delete /mnt/web1 /mass/kuurne/day
>>root 2037 3.4 11.0 29028 28104 ? S 01:00 3:09 rsync -av
>>--delete /mnt/web1 /mass/kuurne/day
>>root 2048 3.1 11.0 29060 28132 ? S 01:11 2:36 rsync -av
>>--delete /mnt/web1 /mass/kuurne/day
>>root 2062 10.0 7.9 21304 20168 ? S 02:00 3:19 rsync -av
>>--delete /mnt/web4 /mass/kuurne/day
>>root 2064 4.9 8.2 22208 21056 ? S 02:00 1:37 rsync -av
>>--delete /mnt/web4 /mass/kuurne/day
>>root 2094 5.7 8.2 22252 21096 ? S 02:05 1:34 rsync -av
>>--delete /mnt/web4 /mass/kuurne/day
>
>
> It looks like you have 3 of each running concurrently.
The concurrent rsync processes seem to spawn itself (this also happens
when rsync run from command line.
> Uh, that memory's pretty full. Once you start paging to disk you may
> never finish rsync. Does the system become unresponsive? Make sure
> there are no other cron jobs that might add memory pressure timed to run
> soon before (or concurrent to) the rsync.
The system doesn't become unresponsive (for the first day), but after
some weeks it gets slower and slower (Saw once there hundreds of rsync
processes)
The memory is just "filled" with buffered files, total off NOT
rsync-prog's is about 10 MB (there's nothing else to do for this server)
> You can also 'ls -al /proc/{pid-of-rsync}/fd' to see what files are
> open.
> Also, just to ensure that rsyncs don't run at the same time, consider
> making just one cron job which is a script containing each rsync command
> ending with semicolon (;), no backgrounding. They will always run
> serially, then.
Well, did that, but hangs again.
Strange thing I saw:
when starting up:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
29221 root 18 0 28184 27M 9484 S 9.7 11.0 6:26 rsync
29213 root 11 0 28112 27M 9476 S 6.3 11.0 4:25 rsync
29212 root 17 0 26712 26M 576 S 15.8 10.4 8:39 rsync
Two hours later:
29221 root 9 0 28172 27M 2860 S 0.0 11.0 7:01 rsync
29213 root 9 0 28112 27M 1528 S 0.0 11.0 4:49 rsync
29212 root 9 0 26724 26M 576 S 0.0 10.4 9:36 rsync
Seems like it's dead again?
Find out 1 thing:
If rsync is used without the -v option, it doesn't hang...
However I want to use this option, so I can see quick which files where
changed, and that the copy was succesfull.
More information about the rsync
mailing list