Concern: rsync failing to find some attributes in a file transfer?
L. A. Walsh
rsync at tlinx.org
Sun Jul 27 10:01:37 MDT 2014
Kevin Korb wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> I wasn't objecting to the use of multiple file systems. I have a
> bunch of them too. I was objecting to the use of partitions to
> achieve multiple files systems. Logical volume management has been
> available for a long time and now we also have access to file systems
> that include such features.
I use the terms synonymously.
I'm doing the snapshots via lvm and rsync. Create the dynamic snapshot
vol once a day, then use rsync once a day to copy all the new files off
to another "fixed snap" that contains all of the files that changed that
I then set that up to provide "Previous Versions" in the properties
window on Win 7.
Fixed partitioning, I still use on my system drive for for boot & OS.
Makes for more reliability, though systemd wants everything in /usr and
/usr/share mounted at boot time along w/root, and that's causing a bit
of an annoyance -- a more lovely "gotcha", (my root and /usr are
separate partitions), they moved "mount" from /bin to /usr/bin and left
a 'mount' symlink to the new location on /usr.
Of course little thought was given to how one would mount /usr in order
to be able to access mount, but this seem typical of the thought going
into the systemd changes...
LV VG Attr LSize Pool Origin Data%
Backups Backups -wc-ao--- 10.91t
Home Data owc-aos-- 1.50t
Home-2014.06.25-03.07.08 Data -wc-ao--- 3.84g
Home-2014.07.03-03.07.03 Data -wc-ao--- 2.33g
Home-2014.07.07-03.07.03 Data -wc-ao--- 1.37g
Home-2014.07.09-03.07.03 Data -wc-ao--- 2.45g
Home-2014.07.11-03.07.03 Data -wc-ao--- 5.36g
Home-2014.07.13-03.07.04 Data -wc-ao--- 4.32g
Home-2014.07.15-03.07.03 Data -wc-ao--- 21.59g
Home-2014.07.17-03.07.03 Data -wc-ao--- 2.30g
Home-2014.07.18-03.07.03 Data -wc-ao--- 2.26g
Home-2014.07.19-03.07.03 Data -wc-ao--- 2.25g
Home-2014.07.20-03.07.04 Data -wc-ao--- 1.71g
Home-2014.07.21-03.07.03 Data -wc-ao--- 485.62g
Home-2014.07.25-11.10.31 Data -wi-ao--- 656.00m
Home-2014.07.25-11.14.30 Data swi-aos-- 1.50t Home 0.11
Home.diff Data -wi-ao--- 512.00g
Home3 Data -wc-ao--- 1.50t
Media Data -wc-a---- 7.28t
Share Data -wc-a---- 1.50t
Squid_Cache Data -wc-ao--- 128.00g
UsrShare Data -wc-ao--- 50.00g
Media_Back HnS -wi-a---- 8.00t
Share HnS -wi-ao--- 1.50t
Squid_Cache HnS -wi-a---- 128.00g
Sys HnS -wc-a---- 96.00g
Sysboot HnS -wc-a---- 4.00g
Sysvar HnS -wc-a---- 28.00g
UsrShare HnS -wi-a---- 50.00g
Win HnS -wi-a---- 1.00t
oHome HnS -wi-ao--- 1.00t
Media Media -wi-ao--- 7.28t
So in the above, all the dated Home partitions are "frozen snaps" that
only hold files changed on that day. The are not my backup solution,
but a convenience so I can use the "previous versions" feature in windows.
The last snap, will get used with the current base and the output sent to
"Home.diff", from there, the script computes the needed size, creates
it, throws xfs on it, and copies the data to it. Script also prunes
old snapshots keeping the last week, but going to every other day , then
every 3rd and then 4th.. and that's about as far as this goes back.
Daily backups using a tower of hanoi ordering are used for actual backup
It was the base vol & active snap writing diffs to a side partition where
I got the original errors -- since it is working on the whole partition,
it was running as root.
Does that give enough technical detail about my use case? ;-)
Oh, forgot the files at the end of the push
>>>> my $rcmd = [$Rsync]; push( @$rcmd, qw( --8-bit-output --acls
>>>> --archive --hard-links --human-readable --no-inc-recursive
>>>> --one-file-system --prune-empty-dirs --whole-file --xattrs ),
Should add : push @$rcmd, $OAsnap_lvh->fs_mp . "/.",
$bdiff_lvh->fs_mp . "/";
for src and dest (OA=Oldest active snap (the dated active home, above,
and the "diff"
dir for the base (home.diff).
>> Transferring with --compare-dest? I thought that the data was
>> being moved from one filesystem to another, that seldomly calls for
>> usage of --compare-dest.
Data from the source gets moved to the "diff" volume after comparing it
"base".... I only want to copy over the "diffs" for a given day.
>> It seems to me that the perl script being
>> used is meant for another purpose, and it's being used
>> inappropriately here. Why not just use rsync directly? That way
>> maybe we here on the mailing list can make sense of what's actually
>> happening. Otherwise take it up with the author of that script.
?!?! Um... that would be me. the compare-dest = to the mount point
base-lvh being the object...
> > Why not use rsync directly?
It has been running every day (w/occasionally belches fixed) for
over a year.
Involves creating and destroying snapshot partitions, creating the file
them... Would you want to type all that in by hand every day?
FWIW, to move my volume, I use "xfsdump|mbuffer|xfsrestore" -- it's
I use rsync for making partial copies -- like the separate snapshots for
of days going back.
More information about the rsync