Making Winbind Efficient for 15,000 users

Mike Papper mike at digitalpipe.net
Sun Nov 18 19:37:28 GMT 2001


I am using the following components to monitor the set or users and groups
(and what users are in what groups) for a given NT PDC:

linux with nsswitch set to use winbind
samba with smbd, nmbd and winbind (samba 2.2.2)
C library system calls "getpwent" and "getgrent" which are similar to the
linux commands "getent passwd" and "getent group".

This allows me to get a list of all users, all groups and for each group,
the set of users int hat group as seen by the host linux machine. Since the
linux box is running winbind etc. the list of users on the system mirrors
those in a given NT PDC box (which in our case is a real NT box).

The problem occurs when there are, say 15,000 users and 250 groups. Each
call to these functions takes a long time. In many cases logging in as root
takes so long that we cannot login.

I am relying on winbind to provide a list of users and groups through the
linux system calls getpwent and getgrent. I am simulating the PDCs list of
users and groups in a SQL database. So I need to keep the DB consistent with
the PDC. To do this I have a C program that calls getpwent and getgrent
every time a root user logs in (because the root user requires consistent
up-to-date user-group info). Each time I do this it takes quite a while to
come back to me - and I think were hitting the PDC quite hard too. To solve
this problem I thought I would build an in-memory cache of the user/group DB
and periodically calling getgrent (for a specific group instead of all
groups) - possibly every 30 seconds or so - rather than getting the list of
all users and groups at one time, I periodically poll for a single group
(and its users).

Instead of polling the PDC continously, it would be much more efficient to
get events from the PDC when any of the user/group info was updated.What I
would then need is a way to get "events" from the PDC that tell me when
there is any of the following:
1) a user was added or removed from the system
2) a group was added or removed from the system
3) the set of users in a group changed

Anyone know of a way to do this? I think this requires some functions that
notify on these changes from the PDC.

Since I know not very much how samba really works, I am wondering if:
1) someone can explain how smbd et al communicates with the PDC - really, I
see 0 documentation on any of this
2) if there are samba API calls that do some sort of notify
3) if I can add new functions to linux to augment the getpwent/getgrent
calls for this kind of notify mechainsm
4) or if the NT PDC keeps a version number around and chnages this number
when the user/group status chnages - therby replacing the polling of users
and groups with polling of the version number.
5) what if the linux box was setup as a BDC, would it autmatically get
user/group updates from the PDC - and so hitting this BDC from winbind would
be more efficient than going over the wire to the real PDC?

----------------------
Also, barring any of the above, I have another, simple polling strategy to
keep my local user DB in sync with the PDC, heres what I do - I would love
to get any criticism etc. about this methodology:
1) 1 process that calls "get all groups" every 20 mins or so - this takes a
while
2) a process that calls "get all users in a given group" every 18 seconds?
(or slower or faster) for a single group. Cycle through all the groups over
a period of time.

By polling groups I can cover all users and avoid having to get a list of
all users (since there are many more users than groups). This assumes that
every user belongs to at least one group.

Ideally this kind of caching would be built into winbind...although I have
no idea of how to investigate this or to modify its code or to submit
changes.

Mike Papper
mike at bodaro.com
415-584-8449





More information about the samba-technical mailing list