Cache file list in daemon mode?

Peter Scott Peter at PSDT.com
Thu Aug 2 16:30:53 MDT 2012


Hello.  I suspect that what I want to do is not possible with rsync, but 
this is the best place to double-check.

We are pushing files to a remote target that stores them on a very slow 
network file system.  There are also over a million files on the 
target.  Consequently, running rsync to push an update takes hours while 
the remote side enumerates and stats all those files.

I thought, that wouldn't be necessary if the remote side was running 
rsync in daemon mode, and that it only built its internal map of the 
files there once after startup, thereafter updating that in-memory list 
with every push it receives.  For that to work, there would have to be 
some flag I could set to promise rsyncd that no files in the target 
would be updated through any means other than rsyncd.  I looked for such 
an option, did some experimenting with write-only targets, straced the 
daemon and saw I wasn't getting anywhere.

Our alternatives are options like building a list of what has changed 
and copying only those things across.  There's some housekeeping and 
race condition avoidance in there that makes it more than a couple of 
simple commands and replicates some of what rsync knows how to do 
already.  By any chance is what I want to do possible with rsyncd, or 
some other tool?


More information about the rsync mailing list