http://git.samba.org/?p=vl/samba.git/.git;a=shortlog;h=idmap

[Simo]

On Thu, 2008-07-17 at 12:38 +0000, Volker.Lendeckea wrote:
> Hi!
> 
> $SUBJECT is an effort to simplify and streamline the idmap
> implementation in winbind.
> 
> Main changes:
> 
> The cache directly uses gencache, so you can watch and
> delete cached mappings with "net cache".
> 
> "idmap domains" is gone, the parent winbind figures out the
> list of trusted domains itself. If it becomes aware of a new
> trusted domain, it looks in smb.conf whether a special
> "idmap config" style configuration is around. If so, then it
> tells the winbind idmap child about that fact in the child
> request.domain_name field. The range parameter in idmap
> config settings is mandatory, the parent winbind uses it to
> direct the unix id to sid mappings correctly.
> 
> The default domain is set via "idmap backend", "idmap config
> foo : default = yes" is gone. It is highly recommended if
> not required that this backend is writable for not specially
> configured unknown trusted domains.
> 
> "idmap alloc backend" defaults to "idmap backend". "idmap
> alloc backend" is only required to set explicitly if you
> have a special unix id source, id mappings will always be
> written to the "idmap backend". "idmap alloc config : range"
> is gone, as this must match the "idmap [ug]id" valid for
> "idmap backend" anyway, we now only use that.

I have been taking a brief look at the patches, there a few things I
like very much.

Things I like:
- use of gencache, so that we have one less set of cache functions
- check of the cache in the main winbind daemon which removes one step
when we have a cache hit (this was long due and planned, thanks for
doing this)

Things I don't like or find controversial:
- I'd like an explanation of the reasons this patch cut back on some
options and functionality and not just the description of a new behavior
(so that people can understand why and not just what).

- the interface to query multiple ids at the same time (exp. multiple
SIDs to uid/gid) going away (I really do not like this one).
This interface was built as an optimization to observed performance
issues at login time. In some complex domain setups the PAC can contain
an enormous amount of SIDs (more than 2000 for a single user), when
trying to resolve these SIDs the 1 by 1 check become a real bottleneck
which can delay a single authentication by many seconds (and keeps
winbindd busy a lot too). The reason is that with cold caches we may get
as much as 2000 separate SID lookups against the domain controller (slow
LDAP/RPC calls) and if idmap_ad or idmap_ldap are used another 2000 sid
to uid/gid calls. This can take many seconds, and if multiple users are
logging in at the same time the situation becomes really bad.
The solution was implemented in 2 parts: 1) the idmap cache (with
negative caching to handle cases where idmap_ad (or other readonly
backend is used) and there is no mapping available), 2) allow winbind to
query multiple IDs at the same time so that a single LDAP call could be
made to solve all idmap mappings instead of 2000 separate calls.
The interface was later disabled by Jerry for concerns I cannot recall
at this moment, but I think it should be re-enabled and not definitely
eliminated.

- the patch is incompatible with 3.0.25 and 3.2.0 configurations not
only at the smb.conf level

- the patch allows only the default domain to be written to, as the
allocator backend is not allowed a different range from the default
backend, and uid2sid checks are bound to the id range anyway.
The original designed allowed multiple backends to be written to, so
that a single server could write to any backend (given appropriate
credentials).
The initial idea was to allow each backend to have it's own allocator so
that both the domain SID and the uid/gid range could be used to
determine the domain to query.
After Jerry's request a decision was made to 'simplify' the allocation
procedure and have only one single allocator.
Still I didn't want to loose the ability to allocate IDs both in the
local domain and in any trusted domain that allowed our server to add id
mappings, so domains were identified only by name and SID and uid ranges
were only optional and used as filters (mainly for read-only domains).
A scenario where this would be useful is a server joined to an AD domain
using tdb (or a local ldap) for local mappings and with one of the
trusted domains being a Samba domain that shared mappings via LDAP. In
this scenario (with a single allocator) you want to allocate mappings on
the ldap server so that ids are unique on all servers and yet save
mappings both in the local tdb (for local mappings) and on the LDAP
server when the mapping is about that trusted domain.
This was always a thing I didn't like much but it seemed to simplify
things.
I'd agree on a patch that removes the single allocator and instead
allows any backend that is not read only to have it's own allocator
functions. The only problem I see is that existing configuration would
break if a range is made mandatory and range filtering enforced when
UIDs are spread among multiple writable backends. But the simplification
would probably be worth the minor number of migrations and would also
speed up some lookups when many trusted domain are present.
Multiple allocation backends present a problem though, you cannot have a
simple "gimme a new id function", you now will need to always specify
for what domain you want it (think about passdb and group mapping for
things like BUILTIN\Administrators on a member server), so that will
require some thinking and some special casing I guess, but it is
entirely doable.

- the 'default' parameter going away if my own domain is not what I want
to be the default. If the samba server using idmap_rid is joined to a
'minor' domain in a forest or more complex setup and you want to keep
one LDAP server with company-wide mappings (where most other trusted
domains are stored), then you may want to have mappings, by default, to
go on the LDAP server that is referenced by trusted domains
configurations.
I guess here too we can settle on loosing this functionality and forcing
people to migrate their server configurations and idmapping storage
(except that sometimes this is not something admins have really the
power to do) and tell them they have to put everything on one LDAP
server, although I don't like forcing these choices on admins, therefore
I'd prefer to salvage these options if at all possible.

There are other things that have been changed where I really do not have
an opinion in favor or against they are just different because the code
have different premises and goals.

Simo.