Samba Speed Question

[Dave Dezinski]

Dave Collier-Brown wrote:
> | We've got a few NT40 servers that have more than 20000 files 
> | in a single directory, that don't perform this badly.  We'd 
> | like to replace these servers with Linux boxes using Samba, 
> | but unless we re-organize these files into multiple directories
> | (which means a bunch of custom software needs to be rewritten), 
> | this isn't possible.
> |
> | This definately is a problem that should be a looked into, this 
> | is not the first time I've seen someone complain about this.  
> 
> Yes, it's an artifact of the Unix algorihms:
> linear search and the reliable writes of the
> directory metadata.  The latter is adressed 
> in the  SGI logging filesystem, the Solaris 
> logging option, ReiserFS and in ext2fs.

What still doesn't make sense to me is that I saw no noticable 
improvement in speed using the ReiserFS with Samba vs ext2,
but did notice a difference in speed when cp'ing the files in the
ReiserFS vs ext2 filesystem.
 
> | Spliting the files up into multiple directories, is just a 
> | temporary workaround, not a solution.
> 
> Well, it's sort of a permanent workaround (;-))
> It adresses the linear-search problem nicely,
> which only a true logging fs (or caching) helps with.

I guess without having a filesystem that does some sort of directory
entry caching, there is no other way around this.  I really wish there 
was a better way to handle this, I understand that having 10000 or
more files in a directory is not that efficient, but it since it wasn't a 
problem in NT or in the previous Netware servers we had I couldn't
see why it would be a problem here.

> NT will run out of speed on very deep b-tree
> structures, which fortunately are rare, although
> programs generating filenames sometimes stumble into
> the "bad" part of the namespace. So would hashing
> filesystems, if anyone built them.
> 
> --dave

Thanks for the reply, looks like I'll have to look into changing some
of our applications to handle the spliting up of these files into multiple
directories if Samba is the way I'm going. :-)

Dave Dezinski