strip setuid/setgid bits on backup (was Re: small security-related rsync extension)
jw at pegasys.ws
Mon Jul 8 14:06:16 EST 2002
On Mon, Jul 08, 2002 at 08:52:29AM -0700, Eric Horst wrote:
> Hi, I'm new around here and thought I'd join the discussion. Hope that's
> > I'm inclined to push for more flexibility with:
> > --post-process=<exec_pathname>
> > Runs <exec_pathname> on the receiver just before rsync exits.
> > <exec_pathname> is passed a list of fully-qualified pathnames on
> > stdin (one per line) that have been created, modified or deleted by
> > this rsync instance.
> > Dan's patch would therefore be reduced to the single script:
> > #!/bin/bash
> > xargs chmod u-s
> The post-process is more flexible but one issue I'm currently living with
> is rsyncing millions of files. This solution would not be my first choice
> if I wanted to flip a permission bit on each file. Spewing 10 million
> filenames/dispositions into another script (especially the bad example of
> xargs which would fork up a storm) is mildly inefficient when you move
> into this scale. Especially for the case in question where a setuid bit
> need be changed and rsync has already done a stat and is dealing with the
> file and could easily do it immediately at much lower cost.
The fork bomb is just a product of the example. Hey, i made
it even worse in my modified example. If you were
processing that many files you wouldn't use xargs.
> Not to mention, is it a real long-term goal is to redesign rsync to deal
> with large numbers of files by not building the entire file list up front?
> If rsync is ever rewritten work directory by directory (or whatever)
> building small file lists instead of building the mega filelist then when
> do you run the post-process script? After each small batch of files? Or
> store up the disposition list till the end effectively building a huge
> filelist again?
The rewritten rsync would process depth-first and the
pipeline would be written to as it went.
> > This would also allow rsync users to post-process modified files in
> > whatever way they wish.
> I'm not an advocate for the original patch just weighing in on not liking
> the post-process idea as currently described.
There is some merit to the outboard post-processing. It
certainly is an interesting idea and therefore worth
discussion. The real drawback i see to it right now is the
fact that post-processing the files will modify them so they
are no longer in sync. If such processing needs to be done
it almost needs doing in a sort of ouboard filter.
Such a filter would read filenames (with action) from stdin
and then write to stdout those it acted on so that the
timestamps could be reset. Of course any modifications of
the files would adversly affect rsyncing them again. It
would cause lots of false positives.
I am beginning to think that this needs to be deferred until
the rsync stages are compartmentalized (if that ever
happens). Then there would be the capacity to insert such
filters at multiple points. Such filters could include gzip
processing, text-mode EOL conversion, charset conversion,
encryption, and many others. I can see where this leads and
i'm not sure i like it.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync