Concern: rsync failing to find some attributes in a file transfer?

[L. A. Walsh]

Kevin Korb wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I wasn't objecting to the use of multiple file systems.  I have a
> bunch of them too.  I was objecting to the use of partitions to
> achieve multiple files systems.  Logical volume management has been
> available for a long time and now we also have access to file systems
> that include such features.
----
I use the terms synonymously.

I'm doing the snapshots via lvm and rsync.  Create the dynamic snapshot
vol once a day, then use rsync once a day to copy all the new files off
to another "fixed snap" that contains all of the files that changed that
day.

I then set that up to provide "Previous Versions" in the properties
window on Win 7.

Fixed partitioning, I still use on my system drive for for boot & OS.
Makes for more reliability, though systemd wants everything in /usr and
/usr/share mounted at boot time along w/root, and that's causing a bit
of an annoyance -- a more lovely "gotcha", (my root and /usr are
separate partitions), they moved "mount" from /bin to /usr/bin and left
a 'mount' symlink to the new location on /usr.

Of course little thought was given to how one would mount /usr in order
to be able to access mount, but this seem typical of the thought going
into the systemd changes...


>  lvs
  LV                       VG      Attr      LSize   Pool Origin Data%
  Backups                  Backups -wc-ao---  10.91t
  Home                     Data    owc-aos--   1.50t
  Home-2014.06.25-03.07.08 Data    -wc-ao---   3.84g
  Home-2014.07.03-03.07.03 Data    -wc-ao---   2.33g
  Home-2014.07.07-03.07.03 Data    -wc-ao---   1.37g
  Home-2014.07.09-03.07.03 Data    -wc-ao---   2.45g
  Home-2014.07.11-03.07.03 Data    -wc-ao---   5.36g
  Home-2014.07.13-03.07.04 Data    -wc-ao---   4.32g
  Home-2014.07.15-03.07.03 Data    -wc-ao---  21.59g
  Home-2014.07.17-03.07.03 Data    -wc-ao---   2.30g
  Home-2014.07.18-03.07.03 Data    -wc-ao---   2.26g
  Home-2014.07.19-03.07.03 Data    -wc-ao---   2.25g
  Home-2014.07.20-03.07.04 Data    -wc-ao---   1.71g
  Home-2014.07.21-03.07.03 Data    -wc-ao--- 485.62g
  Home-2014.07.25-11.10.31 Data    -wi-ao--- 656.00m
  Home-2014.07.25-11.14.30 Data    swi-aos--   1.50t      Home     0.11
  Home.diff                Data    -wi-ao--- 512.00g
  Home3                    Data    -wc-ao---   1.50t
  Media                    Data    -wc-a----   7.28t
  Share                    Data    -wc-a----   1.50t
  Squid_Cache              Data    -wc-ao--- 128.00g
  UsrShare                 Data    -wc-ao---  50.00g
  Media_Back               HnS     -wi-a----   8.00t
  Share                    HnS     -wi-ao---   1.50t
  Squid_Cache              HnS     -wi-a---- 128.00g
  Sys                      HnS     -wc-a----  96.00g
  Sysboot                  HnS     -wc-a----   4.00g
  Sysvar                   HnS     -wc-a----  28.00g
  UsrShare                 HnS     -wi-a----  50.00g
  Win                      HnS     -wi-a----   1.00t
  oHome                    HnS     -wi-ao---   1.00t
  Media                    Media   -wi-ao---   7.28t
---

So in the above, all the dated Home partitions are "frozen snaps" that
only hold files changed on that day.  The are not my backup solution,
but a convenience so I can use the "previous versions" feature in windows.

The last snap, will get used with the current base and the output sent to
"Home.diff", from there, the script computes the needed size, creates
it, throws xfs on it, and copies the data to it.  Script also prunes
old snapshots keeping the last week, but going to every other day , then
every 3rd and then 4th.. and that's about as far as this goes back.

Daily backups using a tower of hanoi ordering are used for actual backup
purposes.

It was the base vol & active snap writing diffs to a side partition where
I got the original errors -- since it is working on the whole partition,
it was running as root.

Does that give enough technical detail about my use case?  ;-)

Oh, forgot the files at the end of the push
>
>>>> my $rcmd = [$Rsync]; push( @$rcmd, qw( --8-bit-output --acls 
>>>> --archive --hard-links --human-readable --no-inc-recursive 
>>>> --one-file-system --prune-empty-dirs --whole-file --xattrs ), 
>>>> "--compare-dest=$base_lvh->fs_mp/.");
Should add :     push @$rcmd, $OAsnap_lvh->fs_mp . "/.", 
$bdiff_lvh->fs_mp . "/";
 
for src and dest (OA=Oldest active snap (the dated active home, above, 
and the "diff"
dir for the base (home.diff).

>> Transferring with --compare-dest? I thought that the data was
>> being moved from one filesystem to another, that seldomly calls for
>> usage of --compare-dest.  
Data from the source gets moved to the "diff" volume after comparing it 
against the
"base".... I only want to copy over the "diffs" for a given day.
>> It seems to me that the perl script being
>> used is meant for another purpose, and it's being used
>> inappropriately here. Why not just use rsync directly? That way
>> maybe we here on the mailing list can make sense of what's actually
>> happening. Otherwise take it up with the author of that script.
----
    ?!?!  Um... that would be me.  the compare-dest = to the mount point 
of the
base-lvh being the object...

> >    Why not use rsync directly?

    It has been running every day (w/occasionally belches fixed) for 
over a year.
Involves creating and destroying snapshot partitions, creating the file 
systems for
them...   Would you want to type all that in by hand every day?

FWIW, to move my volume, I use "xfsdump|mbuffer|xfsrestore" -- it's 
50-100 times
faster.

I use rsync for making partial copies -- like the separate snapshots for 
some number
of days going back.