Include/Exclude problems

[Matt McCutchen]

On Sun, 2009-01-25 at 10:29 -0500, Mag Gam wrote:
> I am trying to rsync a very large filesystem which is about 3TB, but
> naturally I want to exclude a lot of things. However, I am really
> struggling with excluding directories.
> 
> SRC=/dasd/december/2008  #Notice there is no trailing slash
> TARGET=/backup/december/2008 #Notice there is no trailing slash
> 
> I want to exclude
> 
> /dasd/december/2008/Data
> /dasd/december/2008/Logs
> /dasd/december/2008/Catalogs
> 
> #NOTICE: I added a trailing slash for $SRC but nothing for $TARGET
> rsync --exclude Data/** --exclude Catalogs/**  --exclude Logs/**
> --numeric-ids --archive --delete --delete-excluded --human-readable
> --stats --verbose --progress $SRC/ $TARGET.0
> 
> For some reason Data Logs and Catalogs are still being synced.

Your rules should match all the contents of the Data, Logs, and Catalogs
dirs, so the only thing I can guess is that the shell is expanding the
whildcards.  To avoid that, quote the rules:

--exclude 'Data/**' --exclude 'Catalogs/**' --exclude 'Logs/**'

Still, those rules may not be quite what you want.  The
directories /dasd/december/2008/{Data,Logs,Catalogs} themselves are not
excluded; to exclude them, you can change the trailing ** on each rule
to *** or just remove it and rely on exclusion of the directory
short-circuiting the traversal of its contents (see the man page).

Also, the rules will match directories named Data, Logs, or Catalogs
anywhere under /dasd/december/2008 .  You probably want to anchor them
with a leading slash to match only the specific directories you listed.

With both of these changes, the rules would be:

--exclude /Data/ --exclude /Catalogs/ --exclude /Logs/

> BTW, I am using the hardlink method to keep snapshots. So before the
> rsync I am cp -al from 0 to 1 and removing the oldest

You might consider --link-dest=$TARGET.1 instead; it will save you a
step and avoid corrupting the attributes of old backups.

-- 
Matt