Status of 64-bit TDB

[John H Terpstra - Samba Team]

What is the status of updating TDB to 64-bit?

I have been monitoring a site that has experienced problems with
messages.tdb hitting the 4GB level.  The site runs an application that
sometime takes 7-10 days to complete.  After 2-3 days of operation the
message.tdb file will grow to approx. 3.5GB.

When tdbdump is run against this file it seems to be empty, except when
an application is processing files in deeply nested directories. When
that happens the messages.tdb, brlock.tdb and the locking.tdb file can
suddenly grow very large.  This typically happens when two workstations
are accessing the same directories.

Every few weeks this file hits the 4GB limit and then smbd crashes.
These files need to be deleted, and Samba restarted.  This of course
interrupts production work.

The files being processed can be in directories that are nested 10-15
deep.  There are 10,000-20,000 small (1kbyte-500kbyte) files per
directory. Admittedly, this is an extreme situation.

Some questions intrigue me:

a) What is progress towards a 64-bit implementation of tdb?  I know
there has been some recent work on this.

b) Is this generally considered necessary, or is it just a nice-to-have
possibility?

c) Is 64-bit tdb the solution to this problem?

d) Would it help if we could implement a method to shrink the size of
this file by way of a live (while running) housekeeping (cleanup) process?

e) Should we document limits beyond which it is recommended that Samba
should NOT be used?  If so, what should these be?


Cheers,
John T.