Modifying Samba3 vfs opendir()

[Jeremy Allison]

Hi all,

	I just helped a vendor with an interesting problem. The
customer had one of these incredibly braindead applications that
wanted to store more than 300,000 files in a directory, and their
application deliberately does a findfirst() call with a name ending
in a wildcard match.

Of course this completely killed performance...

The reason for this is that under POSIX there is no way to tell
the kernel to only return a particular wildcard match on a directory
scan - but this got me thinking.

Of course the problem could be easily fixed if the application could
restructure its files into a more sensible directory heirarchy.

This can be simulated by leaving the braindead app. directory alone
and then creating a sensible hierarchy populated by symlinks - the
problem of course is that these applications have hardcoded paths.

What you really need is for opendir() to return a directory listing
for a *different* directory (ie. the symlinked directory) when a
particular wildcarded search is being done - that way you can detect
a linear search of a 300,000 file directory and redirect it to the
symlink shadow. As the pathnames returned from the readdir() are
simple entries in the directory they're valid whether they came
from the big directory or the symlink shadow.

"This is a job for a VFS module", I thought. Then I realized you
can't write such a module as the POSIX opendir() call doesn't
know about the wildcard mask. So am proposing to modify the
VFS opendir to take a further two parameters - a "const char *"
pointer to the mask within the directory to scan, and a uint16
of DOS attributes to match (hidden, system etc.). The normal
modules will ignore these extra parameters, but this would allow
customized vfs modules like the one described above to get at
the extra data to allow them to perform this task efficiently
without hacking the core Samba code.

This is already done correctly in the Samba4 code of course
where the VFS already abstracts out the Windows semantics, not
the POSIX ones, but for Samba3 it could help in these situations
a lot.

This will mean a revision number change, and is very easy to
add to the next release (ie. extra parameters we will by default
ignore) but I wanted to get some feedback.

Thanks,

	Jeremy.