Samba4 index errors, and upgrade challenges

Thu Aug 19 15:06:13 MDT 2010

  On 19/08/2010 16:50, oowolabi at qrios.com wrote:
> Hi, Matthieu.
>
> We at Qrios were referred to you by Stefan Metzmacher regarding our issues with samba-4(alpha-9), running on a RHEL 5.3 64-bit server, which we have currently deployed for a friendly company willing to try out open source domain services, in lieu of AD.
> In a nutshell, ADUC (and all other ldap browser tools we have tried to utilize) complains 'an operational error has occurred' (classic MSFT empty error message!), and is unable to enumerate the objects and directories in the domain. Strangely enough, most domain user objects are still searchable and modifiable to a large extent. (It cannot search and find all of them, though. )
> On viewing the samba logs, we see this little snippet when ADUC (and other tools) attempt to browse the tree:
>
> [Thu Aug 12 20:59:41 2010 WAT, 1 lib/ldb_wrap.c:68:ldb_wrap_debug()]
>> ldb: Invalid data for index CN=Esther O. Tewogbola,DC=skyebankzm,DC=net
> We have found several .ldb files in the /usr/local/samba/private/sam.ldb.d/ directory and discovered which one holds the errant index record. We have tried to delete the index and related ones (this index is for a user that was created badly) using samba-4's tdbtool (after exhaustively searching for any tool that can modify an .ldb file. Deleting the index does not solve the problem.
Well it's highly recommended to use ldbedit/ldbsearch/ldbmodify to 
modify ldb files and it's __very__ recommended to modify them under the 
control of the samdb (that is to say do something like ldbedit -H 
private/sam.ldb rather than ldbedit -H private/sam.ldb.d/DC=sambaorg, 
DC=corp.ldb).
> Stefan informs us you have had such index problems in the last few days and you have been successful in solving them. Can you please share with us what you have been able to do, so we can sanitize the database?
Well I had a couple of index pb last week but they were due to the fact 
that ldb wanted to reindex my provision after upgrade.
I made the following patch 2651c2f98841a3521b6893ae5158bbb81832b7ee in 
my upgradeprovsion-wip branch on 
http://gitweb.samba.org/?p=mat/samba.git;a=shortlog;h=refs/heads/upgradeprovision-wip.

But I'm pretty sure it won't work for you. My advice is to trash the 
index and to force ldb to recreate it.

If I were you here is what I would do:

1) Stop samba
2) Take a backup (or 2) of the samba provision
3) copy 1 backup somewhere else and modify the smb.conf to point to the 
folder. Ie if you put the provision in /usr/local/backupprovision, the 
file /usr/local/backupprovision/etc/smb.conf must have an entry private 
dir with the following content: /usr/local/backupprovision/private, and 
a lock dir with the following value  /usr/local/backupprovision (modify 
also the path for the sysvol and netlogon although not mandatory it's 
better that everything is coherent)
4) ldbedit -H /usr/local/backupprovision/private/sam.ldb -o modules:, it 
will open the file sam.ldb without loading the modules (otherwise you 
have the module loaded and it looks different)
5) Locate the entries @INDEXLIST remove all the IDXATTR entries, save 
and exit, this should force samdb to reindex the whole database
6) ldbedit -H /usr/local/backupprovision/private/sam.ldb, it will take 
some time as ldb is reindexing your provision (it can take up to 20 
minutes for a 20 000 users/contacts/computer provision)

Hopefully it should manage to remove the dirty index and rebuild it. If 
not well let me know !
After to check that every thing is ok you have to make a search on the 
user with a pb:

ldbsearch -H /usr/local/backupprovision/private/sam.ldb  -b "CN=Esther 
O. Tewogbola,DC=skyebankzm,DC=net"

If every thing is ok then copy the sam.ldb file and the sam.ldb.d folder 
back to the initial place.
> Also, we would like very much to be able to upgrade from alpha-9 to 12, and run samba-4 in at least a replicated (if not completely clustered) mode, in order to accommodate increased connections to the samba-4 service (more users).
I'm not 100% sure I understand your term of replicated/clustered. Do you 
speak about file system served by S4 in this case you won't gain much 
from using 2 or 3 samba4 servers as it didn't support the clustering 
mode (yet) nor ms-dfs for share different from sysvol and netlogon (this 
two are working with ms-dfs). If you speak about Directory services, yes 
it can help although I'm surprised that you have problems, how many 
users are in your AD ? In a normal mode the active directory server is 
used with burst in the morning (when everybody log in) and then it is 
pretty calm unless users are connecting all day long to tons and tons of 
servers (so that it will require a lot of verification for the AD).

Well in anycase the only good solution is to have replicated DCs
> What we have done in attempting to upgrade was to setup alpha-12 without provisioning, and then rsync -avHk the samba etc and private directories.
As I said the only good solution is to have replicated DCs, here what 
you are doing is duplicating the information of the 1st DC so you'll end 
with 2 server with the same server information, it's not too great as 
password are not replicated and client can get confused.
>   It seems to work, but the logs show that alpha-12 is not altogether happy with that. We've also tried to vampire from 9 to 12, without success(following the howto). Please, what works?
Well show us the log.
Once you fixed your index you can try upgradeprovision with from the git 
tree: upgradeprovsion -s /usr/local/backupprovision/etc/smb.conf, test 
it with a copy of your provision somewhere else, it should work (I've 
been able to upgrade my production which is an alpha3 updated to several 
milestone up to alpha9/10).

Then try to vampire with the help of the howto.

If needed send email to the samba-technical list or join irc channel on 
irc.freenode.org!

Cheers
Matthieu.
> Hope to hear from you soon.
> Very best regards,
>
> Sina Owolabi
> ------Original Message------
> From: Mosebolatan Adetoro
> To: Stefan Metzmacher
> Cc: Johannes Loxen
> Cc: samba at SerNet.DE
> Cc: oowolabi
> Subject: Re: Invalid data for index error [TT#65245]
> Sent: Aug 19, 2010 7:39 AM
>
> Hi Stefan,
>
> Thanks for this useful information!
>

-- 
Matthieu Patou
Samba Team        http://samba.org