IDMAP, allocated rids and 3.0

Fri May 9 21:20:13 GMT 2003

Hi Andrew,
I just checked out the latest cvs of SAMBA_3_0. I see that the only current
winbindd idmap backend (the tdb one) does not support any algorithmic
mapping, and continues to use the high-water mark allocation for uids/gids.
This is bad for clusters where a SID could get assigned a different uid/gid
on each node in the cluster. I thought from reading the below message that
this was to be changed, but the only algorithmic allocation I could find was
in the passdb backend which is never used when samba is using winbind. So
does it make sense to have to change (or create anew) a winbindd backend
that first queries the idmap tdb, but failing that returns a uid/gid from
the algorithm used in passdb?

Matt Zinkevicius
Software Engineer
Network Storage Array Solutions
Hewlett-Packard

> -----Original Message-----
> From: Andrew Bartlett [mailto:abartlet at samba.org]
> Sent: Tuesday, April 29, 2003 12:02 AM
> To: Multiple recipients of list SAMBA-TECHNICAL
> Cc: Simo Sorce
> Subject: IDMAP, allocated rids and 3.0
> 
> 
> Some general notes on IDMAP - mostly stuff that Simo and I 
> agree on, but
> probably with a bit of an extra spin my way :-)
> 
> Firstly, some definitions:
> --------------------------
> RID - Relative Identifier
>  For a single domain, all users and groups share a single 'number
> space'.  A RID may represent either, but we can't tell what.
> 
> SID - A globally unique identifier for a user/group/domain.  
> The 'domain
> sid' with the RID appended is that user/group's SID.
> 
> uid - Unix UID
> gid - Unix GID
> 
> 'domain sid' The globally unique identifier for this domain, be it a
> domain hosted by a PDC, or the domain internal to a member server (or
> indeed standalone).
> 
> It is basically assumed that the Samba server can only represent a
> single 'domain sid' space.
> 
> 'free RID' - A RID that has been manually set by the admin, to a
> particular value.  
> 
> 'allocated rid'.- A RID that has been allocated by this system, for a
> user so as to get them in the system without external assistance from
> scripts.  This is what _nua is all about - Non Unix Accounts was all
> about adding these users 'on the fly' with no fuss.
> 
>  The RID algorithm:
>  -----------------
> Samba 2.2 used a RID algorithm of 
> USER RID = uid*2+1000
> GROUP RID = gid*2+1000+1
> 
> Samba 3.0 uses a RID algorithm of 
> USER RID = uid*2+'algorithmic rid base'
> GROUP RID = uid*2+'algorithmic rid base'+1
> 
> The algorithmic rid base is an smb.conf parameter, and is 
> used to ensure
> that no uid could map to particular low RIDs  (this helps migrations,
> see below).  (if you set it to 100000, then there is no need to worry
> about a collision with your 500 NT users you are about to import).
> 
> 'idmap uid' and 'idmap gid'
> These are smb.conf parameters, formally know an 'winbind uid' and
> 'winbind gid'.  This defines the range of users we can map 
> users/groups
> into.
> 
>  Allocated RID space
>  -------------------
> 
> Because of the algorithm above, we can implicitly define an 'allocate
> rid space'.  
> 
> For example:
> 
> idmap uid=1000-2000
> 
> 1000*2+1000=3000
> 2000*2+1000=5000
> 
> Therefore, for *even* rids, we can allocate them freely from 3000 to
> 5000, with the assumption that any real user with rid 3001 will have a
> real SID mapped from elsewhere, stored in a tdb or the like.
> 
> likewise for groups.  Note that the uid and gid range may not be the
> same, so we need to take care.  (perhaps they should be?).
> 
> Some constraints:
> -----------------
> 
> The first, and primary constraint is our commitment to 
> compatibility.  A
> Samba sever must be upgradeable from Samba 2.2 (and below!) operation,
> with at most smb.conf file changes.  
> 
> The second big constraint is that we cannot change the SID 
> being used to
> identify a current user - it might not affect the fileserver 
> itself, but
> if a file on a workstation is owned by that user, a change in SID will
> lock him/her out!
> 
> It is basically assumed that the Samba server can only represent a
> single 'domain sid' space - which might be the machine's own local
> domain, or the domain it is a DC for.
> 
> This means we have one 32-bit space to play with, and in 
> reality it gets
> much smaller than that, if you keep algorithmic rids.
> 
> You can only really allocate RIDs in the 'allocated rid 
> space' described
> above, or in the rids below 'algorithmic rid base'
> 
> Also, we may have a UID or GID space that is only 16 bits, 
> and there is
> a completely different game to be played in mapping all the other
> domains onto this space.
> 
> Finally, we want to be able to take over a NT4 domain 
> controller.  Here
> we have no control whatsoever as to who has what RID.  The 
> rids are just
> allocated in an increasing pattern.  
> 
> Discussion
> ----------
> 
> I've been working with idra on the new IDMAP code - those who 
> have been
> on #samba-technical will have noted the discussions...
> 
> To my mind, the idmap code is in general a 'good thing'.  It allows us
> to replace two current hacks that both very much work, and 
> are very ugly
> things indeed.  It also moves a pile of code out of winbind, 
> that recent
> research and events have shown to be unsuitable.  (The advent of
> 'foreign sids').
> 
> We are not quite in agreement about some technical design 
> details, but I
> hope we will make it.  Reading the next section might convince you why
> we don't want to ship with the current code...
> 
>  Samba 3.0 Hacks
>  ---------------
> 
> The first of these is _nua - the idea that accounts can exist in our
> SAM, but that you can never actually log into them.  These were
> basically a hack for machine accounts not in /etc/passwd.  
> However, they
> are also the predecessor to the idmap accounts appearing via winbind
> below:
> 
> The second is our current method of mapping from a SID to a 
> unix UID and
> from the unix UID to a SID again.  For remote users, with 
> winbind, this
> is pretty easy - we just lookup a tdb - but there are certain
> expectations about local users that we need to deal with..
> 
> This is the 'algorithmic rid' problem - the problem that in Samba 2.2
> all 'local' RIDs may be mapped to a UID or a GID by this method, and
> only this method.
> 
> This hurts 'vampire' installations - how do you convert from an NT4 DC
> to samba if you can't assign the rids freely?
> To solve this, I came up with a quick hack - instead of using an
> algorithm, do this:
> 
> uid -> getpwnam(uid) -> name -> pdb_getpwnam(name) -> SID
> 
> and 
> 
> SID -> pdb_getpwsid(SID) -> name -> getpwnam(name) -> uid.
> 
> And only if this fails, use the algorithm.
> 
> This works surprisingly well - with a few extra hacks like 
> 'algorithmic
> rid base' to keep existing 'algorithmic' RIDs out of the way.
> 
> This gets us 'free rids' very easily.  
> 
> However, it really is nasty - and doesn't really allow us to make the
> SAM the center of the world, exported via Winbind etc.  This makes
> migrations harder, as admins have to fuss with 'add user scripts' and
> the like.
> 
> IDMAP Problems/Solutions
> ------------------------
> 
> So, the idea with idmap is that we will assign a mapping into 
> a tdb, or
> LDAP and that we will use that mapping.  However, this 
> creates it's own
> problems - how do you co-ordinate that mapping?  The a new 
> mapping might
> be required on the BDC that the PDC has never heard of.   The 
> PDC might
> not be available at the moment, because the network is out.
> 
> So, we drop back to our old friend the algorithmic mapping - and use a
> hybrid:  
> 
> We use the 'algorithmic' mapping where possible - particularly for
> unknown uids/gids found on the filesystem.  
> 
> When you add an existing unix user to the passdb, they will take their
> alorithmic RID with them - as changing their SID may have serious
> consequences.
> 
> However, some users need a fixed mapping - things like 'root' 
> being 'rid
> 500', the administrator.  This is a fixed mapping in the 
> mapping table. 
> Likewise for migrations from NT that have already occurred 
> and relied on
> previous behavior - these admins will need to set up the 
> mapping (which
> should nicely sit on the same LDAP object already in use)
> 
> Finally, when we are asked to add a new user, we allocate a new RID in
> the passdb, in a RID space set aside for new users.  We then allocate
> them a uid via IDMAP.
> 
> For PDC/BDC sanity, we will fail operations that include a
> yet-to-be-allocated RID, in our allocated range, and that include a
> yet-to-be-allocated uid in the 'idmap uid' range.  A more 'pure'
> solution would allocate uid/RID in this case, but would impose a
> synchronization requirement on BDCs, which may be unacceptable.
> 
>  Winbind_pdb
>  -----------
> 
> This final class of users then needs to appear via nsswitch 
> somehow.  To
> do this, the winbind_pdb module is being developed.  It will 
> export the
> contents of the passdb to NSS.  However, we don't want 'root' 
> appearing
> twice - that would be nasty.  So while it reads the entire passdb, it
> will only expose those parts that are inside the 'idmap uid' 
> and 'idmap
> gid' range.
> 
>  Life without winbind
>  --------------------
> 
> There will also be the option for 'life without winbind' - the LDAP
> schema for IDMAP etc will be constructed such that nss_ldap 
> will be able
> to read these objects as standard users and groups.  (With some
> constraints).
> 
> Conclusion
> ----------
> 
> Despite it's horrors, the current Samba 3.0 code has shown itself to
> work, and work well.  Sites do run the current code, and while the
> design may be less-than-is-desired, it is pretty solid, and could
> certainly ship for a 3.0.0
> 
> IDMAP can certainly be dropped in at a later release, if need be, but
> our commitment to compatibility might require that certain 'less than
> pleasant' parts of the code-base remain as options, at the very least.
> 
> However, _nua accounts have been declared a HACK, and are likely to be
> pulled from Samba 3.0 until IDMAP is integrated.
> 
> Andrew Bartlett
> 
> -- 
> Andrew Bartlett                                 abartlet at pcug.org.au
> Manager, Authentication Subsystems, Samba Team  abartlet at samba.org
> Student Network Administrator, Hawker College   abartlet at hawkerc.net
> http://samba.org     http://build.samba.org     http://hawkerc.net
> 