Forms, print queues and tdb

[Tim Potter]

Damian Ivereigh writes:

> > Are you sure it's tdb that is the bottleneck?  I've found you run
> > into limitations of the printing system before tdb.
> 
> I am pretty sure. Jeremy has labelled me a troublemaker because
> we stress test thing so much! On a medium loaded system, there

(-:

> might be 300 records in the printing.tdb file. We would have
> maybe 200 connections (i.e.  smbd's) running.  The clients are
> NT40 so they request a new view of the queue every 10 seconds
> or so. For each lookup, the file is traversed twice (one to
> count them, the next to actually read them). So multiplying
> that all up we get around 12,000 record reads (including the
> lock & unlock) every second! Doing an strace it actually never
> does any disk I/O, I think its the lock/unlock that kills
> things.

Eek - all those clients bashing away at a single tdb could really
slow things down.  I wonder if the length of a tdb could be
stored internally and accessed in constant time.  This would
necessitate per-queue tdbs though which would also reduce the
amount of contention as the locking for the 12,000 record reads
would be distributed over 100 files assuming the distribution of
print jobs is equal.

> As a test, I faked the print_queue_status() such that it just
> returned immediately as if there was an empty queue. And the
> load average went down from about 50 to around 2!

On the systems that we've load tested we've run in to problems
with lpq taking a couple of minutes to generate a queue listing
for a large printer.  Your problem sounds like the opposite -
Samba is choking on the length of the print queue.

> > Can you give some more information on your setup?  Which printing
> > backend, how many printers, how many jobs etc.
> 
> We are running multiple (about 300) RedHat linux 5.2, with 2.2.13
> kernel servers. The backend is lpr and probably around 100 printers on
> each of the bigger servers, processing around 5,000 jobs per day each.

Wow.  That's a lot of jobs.


Tim.