winbind with large amount of users and groups

[Mike Papper]

We are experiencing a similar problem - we had 15,000 users and 225 groups
on a windows NT machine and were running samba 2.2.2 with winbind and a
setup in nsswitch to use winbind on a linux machine. Login as root to the
linux machine takes a long time. Doing a "getent passwd" takes a long time -
we see bursty output from this too.

Im am wondering if you can tell me what winbind does - if it simply talks to
the PDC for every request it gets (i.e., each time a program calls getpwent
or getgrent or someone issues "getent passwd" or "getent group") and gets a
whole new list of ALL users and groups (and a groups' users)? Or does it
poll the NT PDC periodically to keep info up to date? Could/would it arrange
with the PDC to receive notifies of user/group status changes (therby
reducing all kinds of load) - i.e., is thispart of the SMB protocol?

What if we setup our samba on linux to also be a BDC? - can it do this?
Would this mean that the PDC is then communicating "diffs" (i.e., changes of
user and group status) with the BDC and we would avoid the "polling" of the
above? Possibly the winbind could do this itself (so we dont have to setup
ourselves as a BDC). Any word on this?

BTW I am simulating the PDCs list of users and groups in a SQL database. So
I need to keep the DB consistent with the PDC. To do this I have a C program
that calls getpwent and getgrent every time a root user logs in (because the
root user requires consistent up-to-date user-group info). Each time I do
this it takes quite a while to come back to me - and I think were hitting
the PDC quite hard too. So I will be building an in-memory cache of the
user/group DB and periodically calling getgrent (for a specific group
instead of all groups) - possibly every 30 seconds or so.

I am wondering if you know of a way I can get only the user-group changes
without having to poll the PDC?

I am wondering if the code youre writing "for the 2.2.3 release" will do
this in-memory caching of all users for me? Maybe I dont have to write this
myself?

Any feedback is appreciated.

Mike Papper

----- Original Message -----
From: Jeremy Allison <jra at samba.org>
To: Yuval Hager <yuval at disksites.com>
Cc: <samba-technical at lists.samba.org>
Sent: Friday, November 16, 2001 7:21 PM
Subject: Re: winbind with large amount of users and groups


> On Fri, Nov 16, 2001 at 06:15:59PM +0200, Yuval Hager wrote:
> > Hi,
> >
> > I have just tested winbind (as a PAM library) in the following
> > configurations:
> > - an NT domain with 200 users and 1000 groups
> > - an NT domain with 3000 users and 200 groups.
> >
> > After installed (samba-2.2.2 w/o the winbind memleak fix) and added to
> > nsswitch.conf and the PAM, any trial to login to the machine, or even
issue
> > an `id' on a domain user would take 20 to 60 seconds.
> >
> > The winbind settings are the usual, I tried to chaged the "cache time"
> > settings but it seemed to had no real effect.. (I have previously seen
some
> > problems with the winbind cache, but I'm not sure about that yet).
> >
> > We have traced the line and found many queries onto the NT PDC. From a
first
> > look it looks very inefficient..
> > I haven't checked the code yet, but is this reasonable ? Did any of you
had
> > a successful installation on such amounts of users ? Is the search
merely
> > inefficient or does it "have" to be that way.. ?
>
> This is a known issue that we're working on. It will
> be fixed (ie. much faster :-) for the 2.2.3 release.
>
> I just finished the backport from HEAD into 2.2 of the
> new mem-leak-free winbindd code, and will now spend significant
> time optimising it.
>
> Jeremy.