Modifying Samba3 vfs opendir()

[James Peach]

On Fri, Jun 24, 2005 at 06:11:01PM -0700, Jeremy Allison wrote:
> Hi all,
> 
> 	I just helped a vendor with an interesting problem. The
> customer had one of these incredibly braindead applications that
> wanted to store more than 300,000 files in a directory, and their
> application deliberately does a findfirst() call with a name ending
> in a wildcard match.
> 
> Of course this completely killed performance...
> 
> The reason for this is that under POSIX there is no way to tell
> the kernel to only return a particular wildcard match on a directory
> scan - but this got me thinking.
> 
> Of course the problem could be easily fixed if the application could
> restructure its files into a more sensible directory heirarchy.
> 
> This can be simulated by leaving the braindead app. directory alone
> and then creating a sensible hierarchy populated by symlinks - the
> problem of course is that these applications have hardcoded paths.
> 
> What you really need is for opendir() to return a directory listing
> for a *different* directory (ie. the symlinked directory) when a
> particular wildcarded search is being done - that way you can detect
> a linear search of a 300,000 file directory and redirect it to the
> symlink shadow. As the pathnames returned from the readdir() are
> simple entries in the directory they're valid whether they came
> from the big directory or the symlink shadow.
> 
> "This is a job for a VFS module", I thought. Then I realized you
> can't write such a module as the POSIX opendir() call doesn't
> know about the wildcard mask. So am proposing to modify the
> VFS opendir to take a further two parameters - a "const char *"
> pointer to the mask within the directory to scan, and a uint16
> of DOS attributes to match (hidden, system etc.). The normal
> modules will ignore these extra parameters, but this would allow
> customized vfs modules like the one described above to get at
> the extra data to allow them to perform this task efficiently
> without hacking the core Samba code.

I'd prefer that this be a new VFS call, rather than an extension to an
existing one. Pushing the two different semantics into the same call
seems like a recipe for confusion. Also, it weakens the correlation
between VFS call and system calls, which will make performance
measurement more difficult.

-- 
James Peach | jpeach at sgi.com | SGI Australian Software Group
I don't speak for SGI.