Improving performance of winbindd_list_groups

[simo]

On Mon, 2008-04-07 at 11:27 -0700, Steven Danneman wrote:
> My current pain point involves winbindd_group.c:winbindd_list_groups()
> but I believe everything described here also applies to
> winbindd_user.c:winbindd_list_users().
> 
> We have a somewhat complicated domain topology setup including a primary
> domain with 2000 groups, a trusted domain with 10,000 groups, a trusted
> NT4 domain, and a trusted domain with a slow link.  When attempting a
> "wbinfo -g" command on a machine joined to our primary domain, the
> command is waiting ~60 seconds and then erroring out.  Upon
> investigation, there's no error, winbindd's response is just exceeding
> the max timeout that wbinfo will wait.  This is due to the groups from
> all trusted domains being sequentially enumerated by the main winbindd
> process in winbindd_list_groups().  Combined across all domains this
> sequential enumeration takes ~85 seconds with the longest domain taking
> 32 of those seconds.
> 
> This function seems ripe for optimization.  With all the existing
> winbindd asynchronous framework available, we shouldn't be sequentially
> enumerating groups from each domain, but instead querying them all
> simultaneously.  Thus, setting an upper time bound for the operation to
> be the slowest domain, not the combined time of all domains. 
> 
> My proposal is to rewrite winbindd_list_groups() (and subsequently
> winbindd_list_users()) to asynchronously enumerate groups from all known
> domains simultaneously using async_domain_request() and calling
> get_sam_group_entries() from the winbindd_child_dispatch_table.
> 
> As I get coding can anybody point out any obvious flaws in this plan?

No, and a few of us had a plan to actually convert it the way you
described at some point.

> Is there a reason this group enumeration must be done from the parent
> winbindd process or must be serial and not parallel?

Nope, no reason against it.

So if you can do it, it's great.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Senior Software Engineer at Red Hat Inc. <ssorce at redhat.com>