Problems creating a Samba4 LDAP Backend

[simo]

On Fri, 2008-03-21 at 05:40 -0700, Howard Chu wrote:
> simo wrote:
> > On Wed, 2008-03-19 at 17:33 -0700, Howard Chu wrote:
> >> Searching on memberOf doesn't make a lot of sense to me, when you could simply
> >> read the group object directly. When is this actually a useful thing to do? An
> >> alternative would be to make the memberOf overlay intercept these filters and
> >> rewrite them in terms of member.
> >
> > Premise: here I am thinking beyond what AD is doing as I use the
> > memberOf concept in another project.
> >
> >> From my usage memberOf makes it very simple to find all the groups a
> > member is part of even if that membership derives from nested grouping.
> 
> Ah, an interesting point, but probably a separate discussion. Note that the 
> OpenLDAP memberOf overlay doesn't handle nested groups.

Interesting.
To be honest, if memberOf does not solve the nesting group problem, I
find it rather useless, and that may explain your comments below.

> > It's very clear that most of the time you have an identity and you want
> > to know what this Identity is part of, not the other way.
> 
> No, not clear at all. A very common application of "groups" is for things like 
> email lists. In that case, an MTA knows a specific group (the name of the 
> email list), and needs to know all of the members.

My bad, I should have specified that this is the case I meet/care in my
work. It is not, in general, the most common case, I agree.

> Other times (e.g. access control) you know an identity (current user) and want 
> to know if the identity belongs to a particular group (for an authorization 
> check). In that case, it is an equal amount of work to look in the user's 
> entry for a memberOf value as to look in the group entry for the particular 
> member value. In practice, because groups may be referenced frequently for 
> multiple users, the group entries will be hot in the server cache and so the 
> member lookup is actually cheaper.

This is true only if you do member checking by looking at groups. But if
you don't they won't, so I am not sure this really is a reason unless
there is another reason groups are frequently checked. But given the
fact that groups may have huge memberships, actually wasting a lot of
cache memory to keep around the members could also be a reason not to do
it that way. Of course this is just cheap talk of me, without numbers
related to a particular use case (and the use case here is more identity
management, not much MTAs or such, even if, in the end, the same
Directory can be used for many applications, so groups can actually be
really in cache already).

> The frequency with which the question "what groups do I belong to" is asked is 
> extremely low in most applications. The frequency with which the question "is 
> XX a member of this group" is asked is very high in most applications. E.g., 
> the libc initgroups() call needs to know what groups you belong to; that call 
> typically occurs only once at the beginning of a login session. After that the 
> result is essentially cached by the kernel. The result is cached in the kernel 
> because the subsequent "is XX a member of group YY" questions happen so 
> frequently as a process accesses system resources.

Well, initgroups() is a special case, it asks for all groups.
And I agree that the question: "is XX a member of group YY" is the most
common.

> > So you really want to do a single search on one entry, rather than a
> > huge search on the whole directory to find out (including local
> > calculation for nesting) what groups include that identity as member, by
> > parsing all groups one by one.
> 
> > It is as simple as that.
> 
> No, not simple at all.

Points of view :-)

> Yes, ideally you would like to be able to look in a single place and get the 
> answer to "what privileges does user X have" but that doesn't actually mean 
> what you're implying.

Checking privileges is a whole different business, you still have the
"is XX a member of group YY" to solve before that.

>  In particular, the "single place" you're looking isn't 
> necessarily that user's own entry. In most cases, that's the worst place to 
> use because users generally have full write privileges to their own data, and 
> the data comprising their set of privileges really belongs to the sysadmin, 
> not to the individual user.

Well obviously memberOf need to be protected from user manipulation.
Esp. in Identity Management applications user entries are usually very
well constrained as they hold other information about the user that the
user is not supposed to be able to modify. That's not a big problem, we
have ACIs just for that.

> Ideally, you write to the privilege set and read from the privilege set in the 
> same way, in the same place.

This may be theoretically a sound principle, but in this case I think it
is not as ideal as it sounds.

> As an administrator, this simple consistency makes life easier.

As long as the Admin knows how the mechanism work, I honestly see no
problem, and you can still just query the members of a group when
member/memberOf is in use.

> The memberOf concept is fundamentally broken if you 
> actually rely on it for privilege determination because it is one step removed 
> from how privileges are actually assigned by the sysadmin. (I.e., the sysadmin 
> doesn't assign membership privileges to a user by writing to the memberOf 
> attribute, therefore memberOf is not authoritative.)

I do not agree with the "fundamentally broken", I think that's
exaggeration. Esp. when memberOf solves the "unrolling nested
memberships" problem, it actually is, I'd say it is "fundamentally
better" than requiring clients to do it and for 2 reasons:

1.) The server choose what nested groups mean, if you want to change
that mechanism you are free to do it without having to care what legacy
clients may be doing. *** <-- This is *extremely* important IMHO ***
1.1.) You also do not have to beg client app implementers to fix bugs,
or change the code if you need to change something, and as group
membership can be critical it means you make it simpler for client apps
not to do the wrong thing.

2.) The operation is done only once, and is verifiable for the admin.
Nested groups can be very complicated to grok, but, by querying
memberOf, the Administrator can empirically check that what he did
actually lead to the expected results. So in this sense I see memberOf
as a real aid for admins.

Of course all this discussion is almost meaningless if you do not
support group nesting which is, for me, the whole point of using
memberOf.

Finally, while I agree that, technically, memberOf is not authoritative,
as what admins change is really the groups's "member" attribute, I fail
to see how this can be a problem unless you have a buggy implementation.
Memberships do not change often enough to consider the mechanism not
worth of trust imo, and there are many ways to implement the memberOf
attribute, you can even make it a virtual attribute generated by a
search, but this would add a substantial cost if you consider nested
groups, so you would very quick end up caching it aggressively which is
actually what calculating and storing it ends up being. If you think of
memberOf as a cached search that tells you the user's set of groups
(including nested...) I think it makes a lot of sense.

> > Now for what concern the Samba4 problem, I think we should be more
> > creative and first understand in which cases we might hit a problem with
> > plugins like memberOf. I am sure some of these cases are just normal
> > possible inconsistencies that can happen even in a normal AD server if
> > you do many modifications at the same time. For these cases we just have
> > to try not to make them more probable or problematic than what they are
> > now.
> > In other cases we might think of doing aggressive caching/prediction in
> > our internal transactions. It might require some more work, but it could
> > be a viable option, and also drive some more performance as dealing with
> > an external LDAP is necessarily slower.
> > Finally, if caching/prediction is not possible, we can think of writing
> > overlays/slapi plugins directly for the LDAP server of choice be it
> > OpenLDAP or Fedora Directory Server or anything else. This third option
> > would require some more work and will be server specific, and perhaps
> > involve some creative thinking wrt licensing, but it is certainly a
> > viable option we should not discard. After all, these LDAP servers have
> > a plugin system with defined APIs exactly to solve those problems that
> > cannot be solved merely by external interaction.
> 
> Agreed. And frankly, there's already an existence proof that this approach is 
> viable.

Indeed.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Senior Software Engineer at Red Hat Inc. <ssorce at redhat.com>