URGENT: Tricky rsync problem

[tim.conway at philips.com]

I would recommend doing both moves in the same script, to keep them 
sequential.
rsync the server to the local staging directory, then rsync the local 
staging directory to the local destination.
Have the script save its pid in a file when it starts.
Before it writes its pid into that file, it reads the file (if it exists), 
and does a ps on that pid to see if the earlier run is still going, in 
which case it just exits, and will be tried again at the next 2-minute 
mark.
#!/bin/sh

ps -p`cat pidfile` |grep `basename $0` && exit 0

echo $$ >pidfile
rsync -avz server::module localstage
rsync -av localstage localdestination

Tim Conway
tim.conway at philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn, 
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), 
".\n" '
"There are some who call me.... Tim?"




"Craig A. Sharp" <csharp2a at adni.net>
Sent by: rsync-admin at lists.samba.org
03/25/2002 11:48 AM

 
        To:     <rsync at lists.samba.org>
        cc:     (bcc: Tim Conway/LMT/SC/PHILIPS)
        Subject:        URGENT:  Tricky rsync problem
        Classification: 



Hi all,
 
Here is the problem that I need to solve:
 
I have an ftp server that I wish to rsync the files from a directory on 
the remote server to a local server and then when the files are complete, 
move them to a public directory on the local server.
 
To move the files from the remote server, I successfully tried using the 
following command:
 
rsync -avz ftp.servername.com::roush/inbound/* /localserver/directory
 
I am running this command from the local server.  The command is correctly 
moving the files from the remote to the local server.  I am not sure I am 
using the correct switches but it is working. 
 
Here are the problems that I need to solve:
 
1. The rsync command must update the local files based on a cron entry 
every two minutes.  I can do this.  I need to be sure though, that rsync 
is updating the files with new data. 
 
In other words, if the file is still coming into the ftp server, I need 
rsync to move the additional data across to the local server until the 
file is complete.
 
2. Once the files exist on the local server, the files need to be moved 
into a public directory for local access, again via cron (the local rsync 
directory is separate from the public directory).  I envision running 
rsync again with new parameters for the local move. 
 
Here is the issue at this point.  I do not want to move the files to the 
public directory unless they are complete.  The end user (public 
directory) must never see an incomplete file.
 
I am a beginner at rsync and would appreciate any help with this problem.
 
Thanks,
 
Craig A. Sharp
Unix Systems Administrator
Roush Industries, Inc.