Steroids for Rsync!
scottb at bxwa.com
Thu Mar 3 00:11:52 GMT 2005
I've been researching the state of 'file alteration monitoring'
technology on Linux. Famd uses dnotify to inefficently monitor a handful
of directories. The replacement for dnotify is being worked on in the
kenel and it's called inotify. If I understand it correctly and they get
it finished, it would be an awesome addition to rsync. With it, you
could run rsync to update a remote system (push mode) and keep it
up2date. With inotify efficiently feeding 'file opened for write was
just closed' notifications to rsync, it could efficiently and
continuously mirror an active file system. With enough bandwidth, the
time lag could be mere seconds.
My challenge is to mirror 300 gigs of half meg files on to three remote
filesystems, afap. Most of the files are 'write and leave for two
months' with some being 'write 4 times a day' so rsync should be perfect
if it can be told right after the file is created. We are installing
fiber to our building so this might be the perfect combination to come
up with a five second mirror performance.
Can rsync be set up with: here's the src dir, here's the remote dir, now
here's a relative list of files I want you to sync? Perhaps being fed
one at a time via a pipe. The pipe thing wouldn't be needed if rsync
used inotify directly.
Right now I'm using a homebrew, ftp utility over a T1 with hours of
back-log. I'm excited about the prospects.
More information about the rsync