access idmap cache directly from smbd

[simo]

On Sat, 2008-08-16 at 12:18 +0200, Volker Lendecke wrote:
> 
> 
> Hi, Jerry & Simo!
> 
> Attached find two patches that make smbd directly access the
> idmap cache.
> 
> It does widen the interface between smbd and the Samba
> implementation of winbind, but I think it is fully
> compatible with alternative implementations of libwbclient
> like the Likewise one.
> 
> It will speed up Samba operation, because it gets rid of a
> considerable amount of roundtrips to winbind, in particular
> together with the third patch that increases the default
> positive cache timeout to a week.
> 
> If this is not acceptable upstream, we need to discuss a
> separate tdb-based idmap cache for smbd only. I have
> customers where the LDAP server is running at its total
> limit, because they have "hide unreadable" set together with
> many files and Posix ACLs. With every user pressing "f5" in
> a directory, we look up hundreds of gids against LDAP, one
> after the other.  This is a significant performance problem
> in my situation, and I would like to get a relief upstream.

Hi Volker,

I find it very desirable to be able to read the cache directly from
smbd, but I think you should treat the cache as read-only from it. If
the mapping is not found you should always go to winbindd (if possible)
where an idmap backend could have different ideas on mapping than smbd,
or do some special logging or other actions that would be bypassed if
you store directly form smbd.
At most, storing a negative mapping could be accepted I guess.
But I would not allow it to set positive mappings IMO.

The reason is that, if I read the code correctly, if winbindd is
temporarily down and an entry is expired, you could end up with a unix
user sid (S-1-22-1-uid) or a unix group sid, and because of the 7 days
expiration you would be stuck with it instead of the right one.


On the cache timeout:

Also the 1 full week positive caching is probably too much for a
default, although I agree we should probably change the cache to a few
hours and not just 15 minutes. Mapping for weeks is almost the same as
mapping forever.

The reason for positive mappings with a time limit is that this way
admins that uses ldap or ad backends can change these mappings and
expect the change to reflect in the server in a reasonable time.

Because it is an exposed configuration option, of course any admin can
tune it to whatever value they want, even months or years, so I'd leave
longer periods as something to set as a local tuning.

Of course someone could argue that being a tunable then admins that
expect to change mappings could tune it down, but because the expiration
time is written in the cache they would still have to wait a full week
after they change the parameter to a lesser value, so by the time they
realize they have to change a mapping it would be just too much with a
default configuration and they would be forced to either stop the server
and wipe the cache or try to manipulate it manually somehow.
In contrast, if someone has performance problems and realize that this
is because of excessive lookups they can easily tune up the cache time
and they would see their change to be immediately applied as soon as the
entry expire (15 min to a few hours depending on what we settle down
to).

Now, thinking some more, we might decide to change the cache format and
store the set time instead of the expiration time, and always calculate
the sum when fetching instead of when storing. This would allow to
change the timeout at will without being stuck with previous decisions.
It would require us to handle a migration of the cache, but that
shouldn't be too difficult, this is something to ponder I think...

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Senior Software Engineer at Red Hat Inc. <simo at redhat.com>