Check for Open files during rsync

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rsync can in fact copy open files.  What it can't handle are files
that are actively being written to at the block level at the time that
rsync attempts to read them.  Rsync reads each block of the file as it
was at that particular instant meaning that the copy can end up with
parts of several versions of that file rather than a complete copy
from a particular point in time.  The most common examples of this
problem are with databases and virtual machine images.

This is due to the simple fact that rsync is a file based utility.
Any file based utility would have this problem on this type of file
including the simple cp.  This is why programs that use files of this
type generally include their own mechanisms for backups.  There isn't
really any way that rsync itself can work around this problem other
than possibly detecting that it happened but there are perfectly
acceptable solutions to it.  Here they are in my order of preference:

1. Do something to freeze the files at a particular point in time.
Since you mentioned fuser I will assume you are on Linux.  In Linux
the standard way to freeze the files is with an LVM2 snapshot.  Then
you rsync backup the snapshot instead of the live files.  Here is a
perl script I wrote to do an LVM snapshot on my MySQL server so that
it could be backed up:
http://sanitarium.net/unix_stuff/rspaghetti_backup/snapshot_scripts/snap-mysql.txt
Note that pretty much all of the details are hard coded into it.

2. Use the application's included backup functionality.  Depending on
the application this could be some kind of log that is append only
(rsync is fine with files being appended to) that you have to replay
to restore or it could be something that dumps the data out into a
separate file that you could then backup.  If we are going to use
MySQL as an example again this would be mysqldump or mysqlhotcopy.

3.  Force the application to stop writing to files while the backup
runs.  This generally means shutting down a service or a virtual
machine but sometimes they can just be operated in read only mode.  Of
course this is an intrusive option which is why it is the last one.

On 01/19/12 02:08, charith wrote:
> Hi All,
> 
> I saw on previous mail  thread under topic *"exclude open file in
> backup with rsync (2006)"* That Rsync can't handle open files and
> it sends all files without checking whether they are open.I want to
> know whether still Rsync have this problem or if their any solution
> for this. I really need to check open files and I want to sync
> without open files. if their any way do this using --exclude mail
> thread also explain about "*/you could run fuser on all the files
> and translate the results into exclude rules, but files might open
> and close after rsync starts/"* please anyone can explain this to
> me...
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8YVgIACgkQVKC1jlbQAQe3dQCfXGWvxFzzHwbNeWDScNQBQc9k
tYsAoNNPIq2cs0PlFt20Pv/+o3I/4IsR
=j9S6
-----END PGP SIGNATURE-----