s3-passdb: Keep caches coherent

[simo]

On Mon, 2011-08-22 at 08:59 +0200, Volker Lendecke wrote:
> On Mon, Aug 22, 2011 at 09:10:51AM +1000, Andrew Bartlett wrote:
> > On Sun, 2011-08-21 at 16:40 +0200, Simo Sorce wrote:
> > >     via  d713f9e s3-passdb: Only delete 1 entry from memcache.
> > >        via  99bb3ee s3-passdb: Remove always the user from getpwsid
> > > cache.
> > >        via  1152aa8 s3-passdb: Keep caches coherent 
> > 
> > Andreas and Simo,
> > 
> > It seems you have also tripped across the passdb cache - I noticed it
> > while working on the py_passdb layer, and wondered if you or anyone on
> > the list knew the full background?
> > 
> > In particular, I wondered if anyone had any numbers on how much the
> > passdb cache helps us, and if there is another way we could get that
> > benefit?  As you have noticed, keeping the cache coherent takes quite a
> > bit of effort.
> > 
> > In particular, we don't seem to have inter-processes cache coherency,
> > except for the pdb_delete_user() call. 
> > 
> > The only call that uses the cache, as far as I can tell, is
> > pdb_getsampwsid(), and it is only populated on pdb_getsampwnam()
> > 
> > I dug into the history of this cache, which has been moved around and
> > re-factored many times, but never managed to find it's original
> > incarnation.  Does anyone remember what it was added for?
> 
> Please keep up that particular cache or replace it
> with something that makes us query the passdb backend a LOT
> less frequently. Before you remove that cache, please
> test a full domain logon including a range of applications
> that do their own name lookups while running from autostart.
> This was the scenario I unfortunately had to support and
> thus had to add that particular cache.
> 
> I have a customer with a huge user database in a proprietary
> LDAP directory that is unfortunately very slow. The customer
> has already spent a LOT of money with the company providing
> that LDAP server, but it remains very, very slow compared to
> OpenLDAP. We can certainly stop supporting that kind of
> environment, but this would as certainly create customer
> specific patches that distros will have to support.
> 
> Alternatively, you may have hints how to tune those
> proprietary DSAs for decent performance. If so, please
> update me on that.

I have no plans to change it at this time.
But thanks for clarifying the situation that originated this code, it
will help in case we need some refactoring.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Principal Software Engineer at Red Hat, Inc. <simo at redhat.com>