Is there a way to force rsync to be monothreaded (ie to don't fork)?

Vitorio Machado v.machado at permanence-informatique.fr
Thu Feb 21 20:18:14 GMT 2008


Thanks for those precisions Matt!

I've worked on another solution. The one of avoiding the unsafe Carbon 
calls :)
I've made a lot of modifications on the xattr preTiger patch. I'm just 
not releasing it yet as v0.4 because I would like to fix an issue 
first: It works when source volume is the root one, but fails to parse 
path when the source is on another volume (so instead of /path/to/file 
we have /Volumes/myVolume/path/to/file). Surprisingly, it works for 
destination on another volume than root. So, by the moment preTiger 
rsync can backup only root volume... Annoying.

Vitorio

Le 21 févr. 08, à 21:03, Matt McCutchen a écrit :

> On Thu, 2008-02-21 at 08:22 +0100, Vitorio wrote:
>> the questio is all in the subject: Is there a way to force rsync to
>> be monothreaded (ie to don't fork)?
>> The reason for this is that the Carbon API isn't fork-safe and
>> fonction calls I do to the pretiger resource fork randomly don't work
>> when rsync forks, what's annoying.
>
>> PS: I'm running rsync locally, not with a server. I don't know if
>> this is important nor if it's the cause of the fork.
>
> There are two forks on a local run: the sender forks off the generator,
> and the generator forks off the receiver.  You can eliminate the first
> fork by accessing either the source or the destination via ssh to
> localhost (or, equivalently, "support/lsh" in the rsync source tree).
>
> The second fork is a bigger problem, as rsync's mode of operation 
> relies
> on having separate generator and receiver processes.  There used to be 
> a
> maintained patch "threaded-receiver.diff" that made the generator and
> receiver two threads in the same process instead of two separate
> processes (Jamie may have been referring to this patch), but Wayne
> discontinued it on December 15, 2006.  The old version can still be
> obtained from the rsync CVS repository as follows:
>
> cvs -d :pserver:cvs at pserver.samba.org:/cvsroot co -r1.36 \
> 	rsync/patches/threaded-receiver.diff
>
> but I'm guessing that updating it for rsync 3.0.0 would be a lot of
> work.  And I don't know if using threads instead of processes will make
> the Carbon implementation happy.  I can think of three other potential
> options:
>
> (1) Look into having the forked-off receiver process re-"exec" itself 
> so
> it becomes an independent process.  You'll have to find a way to carry
> over the command-line arguments, filter rules, and first file-list
> chunk.
>
> (2) Move the Carbon code into a separate program, and have the 
> generator
> and receiver each start an instance of this program as a subprocess to
> do the actual work of accessing attributes.  This solution requires the
> least messing with the guts of rsync but may have a significant
> performance penalty.
>
> (3) Use a custom version of rsync with xattr abbreviation disabled.
> Split every rsync run into two, the first without -X and the second 
> with
> -X.  The first run will do the file transfers.  The second run will set
> the xattrs; since the receiver is neither handling abbreviated xattrs
> nor creating files, it won't make any xattr calls and the problem will
> be avoided.  Unfortunately, this solution is ugly for the user.
>
> Matt
>



More information about the rsync mailing list