Forms, print queues and tdb

[Damian Ivereigh]

Hi Tim,

Tim Potter wrote:
> 
> Damian Ivereigh writes:
> 
> > I am noticing that there are several times where Samba is doing a
> > serial search through all the keys in a tdb file. The main reason
> > seems to be that because tdb uses hashes, you can only go to a certain
> > key, you can't do a subkey match.
> 
> Are you sure it's tdb that is the bottleneck?  I've found you run
> into limitations of the printing system before tdb.

I am pretty sure. Jeremy has labelled me a troublemaker because we
stress
test thing so much! On a medium loaded system, there might be 300
records
in the printing.tdb file. We would have maybe 200 connections (i.e.
smbd's) running.
The clients are NT40 so they request a new view of the queue every 10
seconds or so. For
each lookup, the file is traversed twice (one to count them, the next
to actually read
them). So multiplying that all up we get around 12,000 record reads
(including the lock & unlock)
every second! Doing an strace it actually never does any disk I/O, I
think its the lock/unlock
that kills things.

As a test, I faked the print_queue_status() such that it just returned 
immediately as if there was an empty queue. And the load average went
down
from about 50 to around 2!

It's a big resource drain!
 
> Can you give some more information on your setup?  Which printing
> backend, how many printers, how many jobs etc.

We are running multiple (about 300) RedHat linux 5.2, with 2.2.13
kernel servers. The backend is lpr and probably around 100 printers on
each of the bigger servers, processing around 5,000 jobs per day each.
 
> > The only solution I can see is to either have a seperate tdb file for
> > each print queue (and a seperate one for the forms), or to use some
> > method of indexing that gives a sorted order and therefore allows
> > partial matches - e.g. one of the b-tree algorithms. So how attached
> > is everyone to tdb? Can anyone think of a better solution? Is my brain
> > deranged?
> 
> I think you'd have a hard time convincing people to drop tdb.
> (-:  Storing per queue tdbs may be a good solution though.

I hear you on this one. Still I will see what I can find/think up.

Damian

-- 
Damian Ivereigh
CEPS Team Lead
http://wwwin-print.cisco.com
Desk: +61 2 8446 6344
Mob: +61 418 217 582