Renamed files and directories

[N.J. van der Horn (Nico)]

Thanks for your reply Wayne !
I have carried out several experiments the last couple of years 
regarding this.
In former posts other members directed me to all kind of solutions.

The Inotify functionality can be used to create an agent for the source 
machine.
It would send closed&modified pathnames to a queue.
This queue can be processed by uploading to the destination machine,
I prefer the backup action to be initiated from the central backup machine.
Every time a file changes, its name is send to the backup machine,
which on turn starts picking up that file using rsync.
Every 24hours a full sync is done to ensure nothing is missed.

But my biggest concern is the renamed files and directories.
Right now a renamed directory appears to rsync as being removed.
Then the new name is recognised as new and a full sync is carried out.
I am using the backup-dir option to create daily increments.
So the files are not really lost, but unknow to rsync.
I agree that the only solution is to keep a database.
Maybe the solution can make use of the increment directory,
to link these files hard in the current directory.

You probably noticed Henri's reply, he pointed to link-backup:

    This URL has a links to Link-Backup and LSync :
    http://connect.homeunix.com/lbackup/about#alternatives_to_lbackup

    The LSync project has integrated with Link-Backup if you are
    interested.
    otherwise just jump directly to the Link-Backup project.

It is a big python script, but not easy to integrate in my current setup.

At the moment i make backups of 18 hosts for a total of about 2 Terabyte.
These hosts are spread around Europe and most have just ADSL...
Normally the comple process takes about 2 to 3 hours,
including extra mirror to a second backup machine on colo.

But if the renamed directory disaster takes place, I have to manually 
sort it out,
then try to reconstruct the directory on the "current" tree....
When I am busy, sometimes I miss this, like i told before, it took weeks 
to get 40GB....

I use rsync now for several years and know it is a good tool, thanks for 
all efforts !

Regards, Nico


Wayne Davison schreef:
> On Mon, Feb 16, 2009 at 10:14:25AM +0100, N.J. van der Horn (Nico) wrote:
>   
>> What is the current status of both rename-patches ?
>> Are there alternative measures ?
>>     
>
> I'm not thrilled with how the rename patches work, especially since they
> disable incremental recursion.  As such, I'm hoping to change how they
> work before considering adding renaming handling to a released version.
>
> One option is making further enhancements to the db.diff patch that
> would let rsync leverage DB info on where files are and what their
> checksums are.  If the DB were kept updated via something like inotify,
> it would be an especially nice way to quickly find a file with the
> right size+mtime+checksum (or user-selected attributes).  For those
> without inotify, an upgrade script could be run, perhaps right before
> the transfer.  Or perhaps rsync could leverage one of the existing
> file-monitoring packages -- does something like beagle keep track of
> the file's checksum?
>
> ..wayne..
-------------- next part --------------
HTML attachment scrubbed and removed