Problem with AD membership in an AD with more the 100.000 group (possible regression in 4.12?)

[Dr. Hansjörg Maurer]

Hi Andrew

Am 18.05.21 um 05:15 schrieb Andrew Bartlett:

> Ouch.  This isn't good.
>
> While of course you could locally patch to increase the limit,
yes, tried increasing it and this works
>   I'm
> really not sure what to do here, because the way the NDR tokens are
> handled, some of the behaviour is N^2, so large numbers are a really
> bad idea.
>
> Can you attach a debugger and break on that line?  (It might take some
> trial/error to find the right process).  It would be helpful to know
> exactly which structure can't be parsed.  If it is this one from
> winbind.idl:
>       NTSTATUS wbint_QueryGroupList(
> 	[out] wbint_Principals *groups
> 	);
here are the last two gdb cycles of
gdb --args /usr/sbin/winbindd --foreground --no-process-group -S
  with an breakpoint at
break librpc/ndr/ndr.c:1089
         ret = ndr_token_store(ndr, &ndr->array_size_list, p, size);

until the error occurs (cont 65533)



ndr_pull_array_size (ndr=ndr at entry=0x5555558cf020, 
p=p at entry=0x7fffdca09f78) at ../../librpc/ndr/ndr.c:1090
1090            if (ret == NDR_ERR_RANGE) {
(gdb)
ndr_pull_wbint_Principal (ndr=ndr at entry=0x5555558cf020, 
ndr_flags=ndr_flags at entry=512, r=0x7fffdca09f30)
     at librpc/gen_ndr/ndr_winbind.c:566
566                             NDR_CHECK(ndr_pull_array_size(ndr, 
&r->name));
(gdb) cont
Continuing.

Breakpoint 1, ndr_pull_array_size (ndr=ndr at entry=0x5555558cf020, 
p=p at entry=0x7fffdca09fc8)
     at ../../librpc/ndr/ndr.c:1089
1089            ret = ndr_token_store(ndr, &ndr->array_size_list, p, size);
(gdb) step
ndr_token_store (mem_ctx=mem_ctx at entry=0x5555558cf020, 
list=list at entry=0x5555558cf068,
     key=key at entry=0x7fffdca09fc8, value=31) at ../../librpc/ndr/ndr.c:979
979     {
(gdb)
980             if (list->tokens == NULL) {
(gdb)
987                     uint32_t alloc_count = 
talloc_array_length(list->tokens);
(gdb)
talloc_get_size (context=0x5555573f97b0) at ../../lib/talloc/talloc.c:2821
2821    {
(gdb)
2824            if (context == NULL) {
(gdb)
2828            tc = talloc_chunk_from_ptr(context);
(gdb)
talloc_get_size (context=0x5555573f97b0) at ../../lib/talloc/talloc.c:2830
2830            return tc->size;
(gdb)
ndr_token_store (mem_ctx=mem_ctx at entry=0x5555558cf020, 
list=list at entry=0x5555558cf068,
     key=key at entry=0x7fffdca09fc8, value=31) at ../../librpc/ndr/ndr.c:994
994                     if (list->count >= NDR_TOKEN_MAX_LIST_SIZE) {
(gdb)
997                     if (list->count == alloc_count) {
(gdb)
1014            list->tokens[list->count].key = key;
(gdb)
1015            list->tokens[list->count].value = value;
(gdb)
1016            list->count++;
(gdb)
1017            return NDR_ERR_SUCCESS;
(gdb)
ndr_pull_array_size (ndr=ndr at entry=0x5555558cf020, 
p=p at entry=0x7fffdca09fc8) at ../../librpc/ndr/ndr.c:1090
1090            if (ret == NDR_ERR_RANGE) {
(gdb)
ndr_pull_wbint_Principal (ndr=ndr at entry=0x5555558cf020, 
ndr_flags=ndr_flags at entry=512, r=0x7fffdca09f80)
     at librpc/gen_ndr/ndr_winbind.c:566
566                             NDR_CHECK(ndr_pull_array_size(ndr, 
&r->name));
(gdb)
567                             NDR_CHECK(ndr_pull_array_length(ndr, 
&r->name));
(gdb) cont
Continuing.

Breakpoint 1, ndr_pull_array_size (ndr=ndr at entry=0x5555558cf020, 
p=p at entry=0x7fffdca0a018)
     at ../../librpc/ndr/ndr.c:1089
1089            ret = ndr_token_store(ndr, &ndr->array_size_list, p, size);
(gdb) step
ndr_token_store (mem_ctx=mem_ctx at entry=0x5555558cf020, 
list=list at entry=0x5555558cf068,
     key=key at entry=0x7fffdca0a018, value=31) at ../../librpc/ndr/ndr.c:979
979     {
(gdb)
980             if (list->tokens == NULL) {
(gdb)
987                     uint32_t alloc_count = 
talloc_array_length(list->tokens);
(gdb)
talloc_get_size (context=0x5555573f97b0) at ../../lib/talloc/talloc.c:2821
2821    {
(gdb)
2824            if (context == NULL) {
(gdb)
2828            tc = talloc_chunk_from_ptr(context);
(gdb)
talloc_get_size (context=0x5555573f97b0) at ../../lib/talloc/talloc.c:2830
2830            return tc->size;
(gdb)
ndr_token_store (mem_ctx=mem_ctx at entry=0x5555558cf020, 
list=list at entry=0x5555558cf068,
     key=key at entry=0x7fffdca0a018, value=31) at ../../librpc/ndr/ndr.c:994
994                     if (list->count >= NDR_TOKEN_MAX_LIST_SIZE) {
(gdb)
1017            return NDR_ERR_SUCCESS;
(gdb)
ndr_pull_array_size (ndr=ndr at entry=0x5555558cf020, 
p=p at entry=0x7fffdca0a018) at ../../librpc/ndr/ndr.c:1090
1090            if (ret == NDR_ERR_RANGE) {
(gdb)
1091                    return ndr_pull_error(ndr, ret,
(gdb)
_ndr_pull_error (ndr=ndr at entry=0x5555558cf020, 
ndr_err=ndr_err at entry=NDR_ERR_RANGE,
     function=function at entry=0x7ffff7bcd650 <__FUNCTION__.9556> 
"ndr_pull_array_size",
     location=location at entry=0x7ffff7bcc431 "../../librpc/ndr/ndr.c:1093",
     format=format at entry=0x7ffff7bcce00 "More than %d NDR tokens stored 
for array_size")
     at ../../librpc/ndr/ndr.c:606
606     {
(gdb)
607             char *s=NULL;

>
> Then we have more hope of being able to modify the structure to be less
> resource-consuming.
>
> I wonder if changing to a different string type would help.  Thankfully
> this isn't a public protocol, so we can be flexible.
>
> Try the attached patch.  It uses an encoding that stores the strings as
> <num bytes><string> rather than <num bytes><pointer> .... <string>,
> which needs these 'tokens' to get between the stages.
the patch did not help,

Let me know if I should file a bug or can provide additional information

Regards

Hansjörg
>
> Andrew Bartlett