WARNING to those running Samba on OpenIndiana or other Illumos based systems with > 16 groups

Tue Apr 23 18:52:01 MDT 2013

On Tue, 2013-04-23 at 20:44 -0400, Ira Cooper wrote:
> "Hey Volker, is this familiar?"  (I've actually used this exact
> example in presentations!)
> 
> 
> I think this is the 1st or 2nd issue I tried to address.  It turns out
> there is a *MUCH* simpler fix.
> 
> 
> For "modern enough" Illumos/Solaris systems you can really "fix" this,
> for the most part.
> 
> 
> Put "set ngroups_max = 1024" in your /etc/system.  (On less modern
> systems you may have to use 128...)
> 
> 
> If your user is in over 1024 groups... Well.. Then you need a patch to
> cap it.  But in my environment, it doesn't happen.
> 
> 
> I suspect with a recompile that 1024 can be bumped, though I haven't
> researched it.
> 
> 
> Note on my 1st systems, I couldn't do this, so I clamped using
> NGROUPS_MAX as defined by POSIX.  That stopped the process death, but
> it didn't deal with the security issue, that users can't access files
> in some of the groups they should be in... (For me, a working system
> was more important, I didn't need all the groups.  I moved on.)

I should have made clear, ngroups_max = 1024 on these systems already.
That's the easy part, for which we have a PANIC in the code.

This issue is that changing ngroups_max is essentially untested in this
kernel.  Later Oracle Solaris releases allegedly already have the fix. 

I had this IRC conversation on #openindiana on irc.freenode.net on  09
Apr 2013:

(15:22:16) abartlet: anyone here deep enough into the kernel to comment
on setgroups() and if the group list must be sorted?
(15:22:39) abartlet: I've been chasing for some time a really odd issue,
which seems to be that if more than 16 groups are specified, they only
work if sorted...

(17:17:47) MarcelT: abartlet: it is a bug in illumos. The groups should
be sorted automatically, but they are not. Feel free to file a bug.

(17:26:13) abartlet: MarcelT: done: https://www.illumos.org/issues/3691
(17:26:28) abartlet: MarcelT: it sounds like it's been around for a
while then?

(18:35:21) abartlet: is there something about this that makes it
particularly fiendish to fix, or do folks just pretend the old 16 group
limit still exists?
(18:35:36) easye [~user at 213.33.70.157] entered the room.
(18:36:06) MarcelT: I do not remember details. I just know that there is
a bug related to (un)sorted groups
(18:36:17) MarcelT: IIRC it was fixed in Solaris 11 in 2011
(18:37:01) MarcelT: and probably backported to Solaris 10 too, but I
(18:37:08) MarcelT: 'am not sure about that
(18:37:41) abartlet: BTW, my background is that I'm an upstream dev on
Samba (which is one of the best ways to get lots of groups onto a box,
because of windows nested groups), working for a NAS vendor 
(18:39:15) MarcelT: ... and maybe I backported the fix to Amber Road
about a year ago :-)
(18:39:57) MarcelT: I did something related to >16 groups in amber road
and I backported a bunch o bugs there... :-)
(18:41:01) abartlet: so, I take it this isn't the kind of thing where I
grab the Solaris 11 git tree, and cherry-pick out a fix?
(18:41:31) ***abartlet assumes not, given Oracle's reputation, but
anyway...
(18:41:57) MarcelT: heh, do you have access to Solaris 11 sources? :-)
(18:42:10) MarcelT: BTW, the amber road stuff is here:
(18:42:12) MarcelT:
https://wikis.oracle.com/display/FishWorks/ak-2011.04.24.4.0+Release
+Notes
(18:42:27) MarcelT: 6199185 netname2user() code has a limit for the
number of groups
(18:42:33) MarcelT: 6949066 User can't belong to more than 16 groups.
Impacts AUTH_SYS authentication
(18:42:50) abartlet: MarcelT: that's different, I think...
(18:43:04) MarcelT: 7044547 kernel rpc should call KEY_GETCRED_3 and get
all available gids
(18:43:12) MarcelT: 7044600 keyserv dumps core when the remote procedure
KEY_GETCRED_3 is called
(18:43:20) MarcelT: 7044891 groups aren't always sorted in the
credential
(18:43:26) MarcelT: 7047829 AUTH_LOOPBACK corrupts data when > 32 groups
are available
(18:43:31) abartlet: that sounds more like it 7044891
(18:43:33) MarcelT: 7052192 Several parts of the kernel are inefficient
when using multiple groups
(18:43:40) MarcelT: 7052195 The backend can call netname2user with an
improperly sized array
(18:45:51) MarcelT: this is more-or-less the complete list of >16 groups
related fixes I backported to amber road
(18:46:09) MarcelT: but since I just backported them, I do not remember
details about the fix
(18:46:19) abartlet: ok, so how do I get a kernel with that in it to
test with?
(18:46:32) MarcelT: try Solaris 11 :-)
(18:46:40) MarcelT: or maybe Solaris 11.1
(18:47:08) MarcelT: or, try to fix it in illumos :-)
(18:47:28) abartlet: ahh, so you were doing this inside Oracle?
(18:47:30) MarcelT: you have a hint from the Sun bugs synopses above :-)
(18:47:38) MarcelT: sure :-)
(18:47:57) abartlet: sorry, don't know folks here (yet)
(18:48:06) MarcelT: no problem

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org