[clug] using rsync for backups

[Kim Holburn]

Can't you run an rsync daemon on the backup device?  Then hopefully  
the filelist would be generated when the process started up.


On 2008/May/23, at 4:21 AM, Michael James wrote:

> From my experience backing up 3/4 of a terrabyte using rsync,
> I've had some thoughts for possible improvements.
>
>
>
> Using prepared filelists:
>
> A normal rsync job sets up a sender and a reciever process
> which each generate filelists. Then the lists are compared,
> a task list is created, and the 2 processes get to work
> transferring new and changed files.
>
> When the reciever is a backup, no changes happen between rsyncs.
> So the filelist is (theoretically at least) the same as it was last  
> time.
>
> Can rsync (or could it be modified to) use the pre-existing filelist?
> So the sender process could load the filelist from last time,
> prepare the task list and save the reciever all that preparation.
> For my 3/4 of a terrabyte system that's ~3 hours of work.
> Occasionally the reciever could re-scan the backup,
> just to make sure the filelist reflects reality.
>
>
>
> Coping with dormant branches by tarring, zipping and archiving:
>
> A lot of the space on my disks is taken up by files
> that mustn't be lost, but probably won't be needed again.
> As long as they can be easily recovered,
> making people wait a day to get them back wouldn't be a problem.
> So they are candidates for shipping to a tape-based data silo.
>
> So how about having a cron job that "find"s unread branches,
> tars and zips them up, puts the .tgz file into a separate archive  
> area,
> and replaces the head directory with a special link.
> The archive area is  rsync-ed to the silo without  --delete
> so once the .tgz file has been transferred
> it can be deleted from the source.
>
> I'd prefer to use tar as this would cut our use of inodes in the silo,
> which has also been an issue.
>
>
>
>
> So much for the backing up, any fool can backup, but what about  
> recovery?
>
> The source archive area is mounted using some fuse magic
> so users can  "ls -l"  it and even see the rsync sizes, dates
> and checksums.  Once they ask for any contents,
> the fuse magic sets about recovering the tgz file
> and unpacking it back into its original location.
> I don't know what happens to the requesting process
> during the wait.  It hangs?  It returns a warning?  An error?
>
>
>
> Any comments?
>
> -- 
> There is no perl one line hack
> that a page of java won't do more elegantly.
> --
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux

--
Kim Holburn
IT Network & Security Consultant
Ph: +39 06 855 4294  M: +39 3494957443
mailto:kim at holburn.net  aim://kimholburn
skype://kholburn - PGP Public Key on request

Democracy imposed from without is the severest form of tyranny.
                           -- Lloyd Biggle, Jr. Analog, Apr 1961