[Samba] Is Samba Shadowcopying can be used in Production Environement with more than 20 TB of data

Scott Lovenberg scott.lovenberg at gmail.com
Tue Feb 12 05:16:59 GMT 2008


On Feb 11, 2008 8:15 AM, Adam Tauno Williams <adamtaunowilliams at gmail.com>
wrote:

> > > We have something setup here (on a smaller scale) that might be
> > > useful. Our main file server rsync's with our backup server every
> > > hour (using hardlinks to keep snapshots). Since relatively little
> > > data changes between each sync, it is fairly fast (approx 5 minutes
> > > with no noticable slowdown for the clients) the backup server can
> > > then take as long as it likes to write to tape/etc without affecting
> > > the main server.
> > How well does this work on a live filesystem?
>
> Badly.  rsync is a really cool tool for transporting data;  but it
> should never be mistaken for a real backup tool.  It isn't one.  Active
> files will either be skipped or very likely trashed (on the backup copy)
> which isn't a backup at all.
>
> > Are collisions handled gracefully?
>
> It doesn't.
>
> > For example, what happens when a file
> > is in the process of being rsynced at the exact moment it is in the
> > process of being written to?
>
> You get junk.
>
> A real backup requires the applications (in this case, functionally, the
> Windows clients) to be quiescent (including having commited/fsync()'d
> pending writes),  rsync offers nothing at all to facilitate that and
> isn't even aware of it.
>
> It is probably better to LVM snapshot and rsync from the snapshot,  at
> least then you are rsync-ing a single point in time and not a 'rolling'
> filesystem.  But even that doesn't promise that files are in a
> consistent state.
>
> --


You could call sync right before snapshotting the LVM, and then mount the
LVM read only somewhere else to rsync against it.  A journaled file system
is a must - you can always fsck the backup as a mounted image before
finishing your backup.  This should mitigate the chances of corruption, but
by no means eliminate them, FWIW.


Mount options for ext3 which may be of interest (from man mount(8)):
*data=journal* / *data=ordered* / *data=writeback* Specifies the journalling
mode for file data. Metadata is always journaled. To use modes other than *
ordered* on the root file system, pass the mode to the kernel as boot
parameter, e.g. *rootflags=data=journal*. *journal* All data is committed
into the journal prior to being written into the main file system.
*ordered* This
is the default mode. All data is forced directly out to the main file system
prior to its metadata being committed to the journal. *writeback* Data
ordering is not preserved - data may be written into the main file system
after its metadata has been committed to the journal. This is rumoured to be
the highest-throughput option. It guarantees internal file system integrity,
however it can allow old data to appear in files after a crash and journal
recovery. *commit=**nrsec* Sync all data and metadata every *nrsec* seconds.
The default value is 5 seconds. Zero means default.
-- 
Peace and Blessings,
-Scott.

"Of course, that's just my opinion; I could be wrong"
-Dennis Miller


More information about the samba mailing list