rsync 2.5.6 globbing bug

[Alan Burlison]

As I said, I wasn't on the list so replying without cc'ing me isn't much 
use... although I'm now subscribed.

> You obviously don't know how globbing or the exclude list
> works.  Excluding like this is more efficient than globbing.
> In terms of CPU time, memory and the logic that requires the
> subdirs be skipped because they are dirs and we you aren't
> recursing.  On the other hand, efficiency at this
> particular point it the code is largely irrelevant.  What is
> relevant is sanity.

Actually I do know how globbing works, although I've already made clear 
I'm not an rsync expert.

I do entirely agree with your point about sanity over performance, and I 
think the current mysterious failure mode is not at all helpful.

>> The problem can be made to go away entirely with a relatively code minor 
>> change, which will be one less cause of the "rsync: connection unexpectedly 
>> closed" problems that people hit so often, and in addition I'm happy to 
>> submit a patch, so minimal work is involved for anyone else.
> 
> Eliminating the limit is not a minor code change.  This is
> such a rare case it isn't funny.  It is rare to have that
> many files in a directory.  It is even more rare for someone
> to want all the files in such a directory but not any
> subdirectories.

I disagree about it being a significant change, it involves making 
maxargs as passed to glob_expand and glob_expand_one into a int* rather 
than an int and reallocing as appropriate.  As I said, I'm happy to 
submit a patch, but I'm not going to waste my time if there isn't a 
reasonable chance of it being accepted.  I must say I'm slightly 
dismayed by the less-than-enthusiastic response I have received - I've 
identified a problem, root-cause it and even offered to submit a fix - 
what exactly is the problem?

As for 'rare', CPAN has several hunderd mirrors and they all have more 
than 1000 files in four directories - as CPAN grows, so will the 
problem.  It is not at all unusual to have directories with thousands of 
files in them - much work has been done in Solaris in this area to 
improve performance (particulary for /var/mail dirs for big ISPs)

-- 
Alan Burlison
--