scalability of print_queue_update

[jra at dp.samba.org]

On Mon, Mar 10, 2003 at 07:10:10PM +1100, Martin Pool wrote:
> I've been testing injection of many jobs (thousands) into a print
> queue, and am noticing that appliance_head samba seems to spend heaps
> of time in print_queue_update, trying to reconcile the output of lpq
> with samba's database.

Yep. I've been working on this in app-head (our 'day' job :-).

> In particular, this is causing smbspool to give warnings because
> print_job_end is taking a long time to complete.  (>10s for tiny
> files.)  

Yep.

> One approach that was discussed a while ago is to have the lpd notify
> samba when jobs are completed, deleted, or changed.  It could either
> give all the details sufficient to update the record, or (much
> simpler) just send an smbcontrol message to rescan the queue.

That's hard. Modifying all lpd's out there will not be easy.

> Another approach, which I suppose is really just skirting the issue,
> would be to never run the update function while a client is waiting,
> but rather just at intervals when the timer expires.  Possibly a
> separate smbd process might be forked to do this.
> 
> Alternatively we might say that 40,000 jobs is a silly number to have
> queued. :-)
> 
> I think we can also improve the efficiency of the current code without
> restructuring it; I'll send a patch for that shortly.

Ok, what will this patch do ? This is a very irritating comment :-) "I have
a simple proof for this that unfortunately won't fit in this margin..." :-).

Please explain :-).

Jeremy.