include and exclude file

[Matt McCutchen]

On Sat, 2006-04-15 at 14:05 -0400, Marc Collin wrote:
> > Variant 1 (at each level, tell rsync not to stray from the path):
> >
> > + .kde/
> > - .*
> > + .kde/share/
> > - .kde/*
> > + .kde/share/apps/
> > - .kde/share/*
> > + .kde/share/apps/kmail/
> > + .kde/share/apps/kwallet/
> > - .kde/share/apps/*
> >
> > Variant 2 (exclude everything and then re-include subtrees):
> >
> > + .kde/
> > + .kde/share/
> > + .kde/share/apps/
> > + .kde/share/apps/kmail/***
> > + .kde/share/apps/kwallet/***
> > - .**
> 
> in the first variant you write 
> -.*
> in the second variant you write
> -.**
> 
> where to get information about *?
> the number of * seem to have an impact...

Yes, one * matches part of a single path component (no /), while **
matches anything.  That means .** will match .kde/share while .* will
not.

An example of three *: kmail/*** matches a directory kmail and its
contents, while kmail/ only matches the directory and kmail/** only
matches the contents.

I hope the following will explain why the variants differ:

Rsync traverses the filesystem recursively, checking at each step
whether a file or directory is excluded.  That means rsync needs to be
allowed into .kde/share/apps/kmail/ and needs to be allowed to transfer
the files inside, but it needs to be kept out of other dot-directories
and other areas of .kde .  Both variant 1 and variant 2 accomplish this,
but in different ways.

The final exclude rule in variant 2 excludes dotfiles and everything
inside them: for example, .viminfo, .gaim/, .gaim/prefs.xml
and .kde/share/config .  The fourth include rule overrides the exclude
rule by including .kde/share/apps/kmail and everything inside.  (Without
the ***, .kde/share/apps/kmail/foo would be excluded because the exclude
rule matches it.)

The first three include rules are needed to allow rsync to get
to .kde/share/apps/kmail at all.  Including .kde only lets rsync
in; .kde/Autostart is still excluded by the - .** rule.

The - .* rule in variant 1 excludes only _top-level_ dotfiles: that
means .viminfo and .gaim/ but not .gaim/prefs.xml or .kde/share/config .
Even though .gaim/prefs.xml is included, rsync never gets there
because .gaim/ is excluded, so - .* is enough to stop rsync from
transferring any of the dotfiles other than .kde/ .

So rsync enters .kde/.  We need to keep it out of subdirectories
of .kde/ other than .kde/share/; the - .* rule applies only at the top
level, so we need another exclude rule.  An exclude rule - .kde/* keeps
rsync out of other subdirectories, but an include rule + .kde/share
overrides it, allowing rsync into .kde/share/.  Again,
.kde/Autostart/foo is included, but rsync never gets there.  The same
reasoning is applied for a few more levels.

Does this make sense, I hope?

> > Give rsync the option --prune-empty-dirs.
> 
> rsync  version 2.6.6  protocol version 29
> don't seem to have this option

Right, --prune-empty-dirs was added in rsync 2.6.7; unfortunately,
distros seem to be slow in getting this version out.  Upgrade your
rsync.  The source package for 2.6.7 is here:
	http://rsync.samba.org/ftp/rsync/rsync-2.6.7.tar.gz

> i tried without prune...
> 
> rsync -a -r -v -p -t --del --include-from=include.txt 
> --exclude-from=exclude.txt 
> --filter=filters.txt /home/collinm /tmp/test_backup

Pass --filter=". filters.txt" with a period and a space before the
filename; quotes prevent your shell from splitting the argument in two.

Write a filter file that contains all the includes from your include
file (each preceded by +) and all the excludes from your exclude file
(each preceded by -).  Then use this filter file instead of the include
and exclude files, not in addition to them.  Using a single filter file
allows you to prioritize the includes and excludes however you want
instead of giving all the includes higher priority than all the excludes
or vice versa.

> > rsync --relative $BDIR/./ $BDIR/./.kde/share/apps/kmail/
> > $BDIR/./.kde/share/apps/kwallet/ $BACKUPDIR
> 
> this approach work fine and i not create a filter rule file.....
> this approchoad copy BDIR=/home/$USER
> /tmp/test_backup/home/joe
> with this technic can i have /tmp/test_backup/joe

Yes, if you upgrade your rsync.  Rsync 2.6.7 introduced the ./ syntax to
delimit the suffix of a source path that is recreated inside the
destination by --relative.  If you want /tmp/test_backup/joe , put
the ./ just before the joe in the source path.  Do something like this:

BDIR=/home/./$USER
rsync --relative $BDIR/ $BDIR/.kde/share/apps/kmail/ $BDIR/.kde/share/apps/kwallet/ $BACKUPDIR

-- 
Matt McCutchen
hashproduct at verizon.net
http://hashproduct.metaesthetics.net/