Comprehensive re-write of the classicupgrade HowTo and other changes

[Andrew Bartlett]

On Fri, 2014-05-30 at 18:03 +0200, Marc Muehlfeld wrote:
> Hello Andrew,
> 
> Am 29.05.2014 23:38, schrieb Andrew Bartlett:
> >>> What we could do is permit a 'group' file to be put in the dbdir on the
> >>> new server, and somehow use that to get group memberships. 
> >>
> >> This sounds like a good idea. Then users don't have to replace
> >> /etc/group with the old file, if you do the upgrade on a new host.
> > 
> > Please file a wishlist bug for that,...
> 
> Done:
> https://bugzilla.samba.org/show_bug.cgi?id=10642
> 
> 
> 
> 
> 
> Am 29.05.2014 00:22, schrieb Andrew Bartlett:
> >>> One bug I noticed (and I would just fix, but I wanted to first
> >>> understand what made you say that) is:
> >>>
> >>> "When using the passdb backends smbpasswd or tdbsam, it is not
> >>> possible to automatically import groups from /etc/group during
> >>> the classicupgrade. This has to be done manually afterwards."
> >>>
> >>> This isn't meant to be the case.  It is meant to work, but it
> >>> uses the *new* systems /etc/group file, if you move between
> >>> systems, except when using ldap (where it forces the
> >>> ldapsam:trusted option, so that we don't use nss for groups).
> 
> I did some more testings and found the problem:
> 
> I had setup a fresh PDC for writing the HowTo about tdbsam/smbpasswd.
> After setting up and configuring the PDC, the private folder contains
> only these 3 files:
> /usr/local/samba.PDC/private/passdb.tdb
> /usr/local/samba.PDC/private/schannel_store.tdb
> /usr/local/samba.PDC/private/secrets.tdb
> 
> The group_mapping.tdb is located here:
> /usr/local/samba.PDC/var/locks/group_mapping.tdb
> 
> This was the cause, why the import did not find it. It was not in the
> private folder, that was set as dbdir during the classicupgrade! :-)

This is one reason why we have the --testparm option to classicupgrade.
The idea with this particular tool is to work out what the previous
layout was.  (The disadvantage is that without copying the files, there
is a much lower assurance that we won't write to the old files that
might be needed in a rollback).

--dbdir assumes you collected all the databases (that's why it isn't
--old-privatedir).  I do agree that the --help needs to list them, and
clarify this!

> *Problem 1*: I need a list of _all_ files, that need to exist in the
> dbdir folder. I already know: passdb.tdb/smbpasswd, group_mapping.tdb,
> account_policy.tdb. Which other files are tried to be read during the
> migration?

The long and not very useful answer is anything that the source3 passdb
code can touch.  

> Then I started digging around, why I haven't seen this, when writing the
> HowTo (even if the old HowTo version said, that there are problems, I
> tried having a look at it). Then I saw, why I haven't recognized, that
> the file is in a different folder:
> 
> I reused the same private folder again for every testing I had made for
> the HowTo. And this was the problem! Because as I said earlier, there
> were only 3 files in that folder. But after the first time
> classicupgrade run, I had in the source dbdir folder:
> -rw------- 1 root root  421888 30. Mai 17:50
> /usr/local/samba.PDC/private/account_policy.tdb
> -rw-r--r-- 1 root root     696 30. Mai 17:50
> /usr/local/samba.PDC/private/gencache_notrans.tdb
> -rw------- 1 root root     696 30. Mai 17:50
> /usr/local/samba.PDC/private/group_mapping.tdb
> -rw------- 1 root root  421888 26. Mai 18:58
> /usr/local/samba.PDC/private/passdb.tdb
> -rw------- 1 root root 1286144 30. Mai 17:50
> /usr/local/samba.PDC/private/sam.ldb
> -rw------- 1 root root     696 30. Mai 17:39
> /usr/local/samba.PDC/private/schannel_store.tdb
> -rw------- 1 root root  430080 30. Mai 17:50
> /usr/local/samba.PDC/private/secrets.tdb
> 
> The classicupgrade had created 4 additional files! And one of them was
> an empty group_mapping.tdb. And I tought it is the one containing the
> real mappings.
> 
> 
> *Problem 2*: Why does the import create the additional tdb files in the
> dbdir folder? I had expected that the source used for the import is not
> touched.

It is very hard to get our current code to only open these databases for
read.  Indeed, you also see this if you ldbsearch on a path that doesn't
exist. So it isn't surprising that it creates an empty file, sadly.

> I'll rewrite the HowTo regarding the group import soon.

Thanks!

Andrew Bartlett
-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba