too many rsync process's?

[Matt McCutchen]

On 8/15/07, Robert McLain <infiniteitsystems at gmail.com> wrote:
> This is my first post here for a reason, rsync has run great on our system
> for over 2.5 years now, it seams. I just recently took over this contract
> from another IT person who decided to stop showing up to this location. I
> had the pleasure of maping out their network of 5 servers and 40
> workstations, with no info from anyone who works their (they just dont know
> anything). I just recently started checking out how cwrsync was setup on
> their system, as I needed a backup solution for a new server I setup.
> Currently they use cwrsync to backup a Redhat 8 server to a Windows 2k3 sp2
> server, I wanted to add another backup "job" for this new windows 2k3 server
> to the first 2k3 server. I did some reading, and to be honist with you, I am
> still not sure how to properly set this up. But that's not my issue.
> Everything was working fine untill last night. The normal backup from the
> Redhat to the 2k3 machines started at 9pm, around 10pm I noticed that the
> 2k3 server was extremely slugish, checking Task Man I noticed that there
> were 7 rsync.exe process's running. I can honistly say I dont know how many
> process's rsync usualy runs, but 7 seams a bit extreme. I undid the changes
> that I made, witch were not that many, and started the cwserver back up, and
> left for the night. I came in this morning to find over 10 rsync process's
> running. I am at a loss as to what I may have done. I currently have the
> cwserver stoped, untill I can find out what is going on. Any insight would
> be greatly apreciated.
>
> Specs:
>
> CWrsync Server: Windows Small Business Server 2003 SP2
>
> CWrsync Client: Redhat Linux 8
>
> Not sure what version I am running, like I said i'm a noob to rsync, but the
> setup log for cygwin says it was installed 9/23/2005. And I have to be an
> ass and assume that rsync was setup at the same time.
>
> I dont know if you need any more info, let me know and I would be glad to
> provide it.

To get a machine's rsync version, run "rsync --version" or (Windows)
"rsync.exe --version" at the command line.  If any of the rsyncs
involved are older than 2.6.9 (the current stable version), upgrading
them might help.

The clients each have some sort of cron job or scheduled task to push
their files to the backup server, right?

On the backup server, I would expect to see one process for the
background rsync daemon plus two processes (generator and receiver)
for each active connection.  It's conceivable that there could be 7
rsync processes on the backup server if three clients are pushing
backups at the same time.  However, if the processes seem to be
accumulating, it's more likely that each job is still around when the
next job starts because it either legitimately takes a long time or
(more likely) hangs.

To solve the problem, you need to know which client's job is hanging
and how.  Bring up the backup server again but disable all the
scheduled backup jobs.  Then test each client in isolation by running
the job manually from the client's command line.  If the job gets
stuck, kill any stuck processes and re-run the job with high verbosity
(pass -vvv options).  Look at the output and the backup server's log
file to try to figure out what happens when the job gets stuck.  (If
you need help interpreting the output, send it to the list.)

When rsync is running over an unreliable network connection, sometimes
one side quits and leaves the other side hung.  If this is what is
happening to you, the solution is to set a timeout in the backup
server's configuration file.

If you can't reproduce the problem by running individual backup jobs
manually, I'm not sure what to suggest.

Matt