Steroids for Rsync!
rsync at danny.teleologic.net
Sat Mar 5 17:01:02 GMT 2005
On Wednesday 02 March 2005 06:11 pm, Scott Becker wrote:
> I've been researching the state of 'file alteration monitoring'
> technology on Linux. Famd uses dnotify to inefficently monitor a handful
> of directories. The replacement for dnotify is being worked on in the
> kenel and it's called inotify. If I understand it correctly and they get
> it finished, it would be an awesome addition to rsync. With it, you
> could run rsync to update a remote system (push mode) and keep it
> up2date. With inotify efficiently feeding 'file opened for write was
> just closed' notifications to rsync, it could efficiently and
> continuously mirror an active file system. With enough bandwidth, the
> time lag could be mere seconds.
> My challenge is to mirror 300 gigs of half meg files on to three remote
> filesystems, afap. Most of the files are 'write and leave for two
> months' with some being 'write 4 times a day' so rsync should be perfect
> if it can be told right after the file is created. We are installing
> fiber to our building so this might be the perfect combination to come
> up with a five second mirror performance.
Have you looked into the existing solutions for this problem, along the
lines of network block devices used in a RAID-1? I used that setup
probably 5 years ago with a reliable network link, in order to keep 4
machines in sync. The ndb driver in Linux is much improved since then...
Using the mdutils, I'm pretty sure that the software RAID susbsystem (yeah,
I'm still assuming Linux) would handle disconnected remote drives, though
you might have to manually bring them back into sync after network
disconnections (probably with a script to monitor /proc/mdstat).
I agree that rsync is really cool, and that something like this would be
handy, but I'm inclined to agree with Eli in that this might be better
handled by having a daemon listen for changes and then send the changed
file(s) up to the destination server(s), probably using the rsync protocol.
This one-way sync thing might be even better handled by the RAID/nbd
solution - since that's really what nbd was designed for. :)
More information about the rsync