[PATCH] [WIP] AD DC backup and restore tool

[Rowland Penny]

On Wed, 16 May 2018 21:28:46 +1200
Andrew Bartlett <abartlet at samba.org> wrote:

> On Wed, 2018-05-16 at 10:10 +0100, Rowland Penny wrote:
> > On Wed, 16 May 2018 20:30:16 +1200
> > Andrew Bartlett via samba-technical
> > <samba-technical at lists.samba.org> wrote:
> > 
> > > G'Day,
> > > 
> > > Just a heads up that Tim and I plan to finish up the backup tool
> > > soon. 
> > > Given the strong feedback so far it will include the restore tool,
> > > however it won't include the extended attributes (file
> > > permissions on the backup of the [netlogon share]).  
> > > 
> > > Handling the extended attributes turns out to be harder than you
> > > might expect, while 'just' a couple more options to a tar
> > > command, testing it runs up against the fake xattrs we use in our
> > > selftest environment, which are not visible to tar.  
> > > 
> > > Likewise, for the online backup, the ideal option would be to
> > > query these over SMB and store the NT ACL directly into the tar
> > > xattrs via the python API.  However this isn't available until
> > > python 3.x 
> > > 
> > > Finally, testing runs into the same issue, we can't just extract
> > > the files with tar because we need the xattrs put into the
> > > xattr.tdb.
> > > 
> > > As the client task was literally named 'samba_backup is not
> > > tested' I'm loathed to add a feature I can't test. 
> > > 
> > > Therefore, I will be proposing the tool matching the existing
> > > samba_backup for features, but more importantly with the critical
> > > locking bug addressed.  
> > > 
> > > For the overly curious, the current WIP patches are part of this
> > > tree:
> > > https://gitlab.com/catalyst-samba/samba/commits/aaron-backup2
> > > 
> > > Please let us know any further feedback we should be aware of when
> > > presenting these.
> > > 
> > > Thanks,
> > > 
> > > Andrew Bartlett
> > 
> > Hi Andrew, what does backing up another server online get you, that
> > just backing up the server the python code is running on doesn't ?
> 
> Quite a few things.  We generally find that DRS replication gives a
> much more reliable snapshot of the DB, without hidden faults lurking
> within the raw database. 

No, still don't understand this. Yes, I understand that some attributes
do not get replicated, but this is the same for all DCs, but if the DB
is damaged in some way, it is very probably damaged on all DCs including
the one being backed up online.

> 
> On the flip side, a file-based backup will get non-replicated
> attributes.  Each has their place. 
> 
> Tim will also be building on the basis of the online tool to provide a
> new domain-rename feature, which should be quite handy.

This is a very good idea, this topic comes up quite often.

> 
> > If the code has mostly been written by Aaron, why is the copyright
> > assigned to you ?
> 
> For internal corporate reasons.  While it looks strange, it is
> deliberate and legitimate. 

I didn't think otherwise, but I still think Aaron should be given
credit.
 
> 
> > As I said before, get the locking bug code into Samba and backport
> > it, we can discuss the python backup script then.
> 
> The inability to correctly lock the various databases safely in
> samba_backup is the major purpose of the new script.  However, now
> that we have needed to build it, we have tried to make it a proper
> part of samba-tool, built in a way we can support long-term and
> backed by a dazzling array of tests.

By your own admission, you cannot test it, the only valid test I can
think of is, can you backup a DC, remove the DB etc and then restore it
to a working state again.

> 
> Running tdbbackup -r won't change a shell script into one that locks
> the whole DB.  This is because a transaction lock (or at least a
> global read lock) needs to be taken out correctly over the DB while
> the databases are being copied.

I think what you are trying to point out here is, even with 'tdbbackup
-r', you only get a lock on the file being backed up, but the others
could change whilst the file is backed up.

> 
> (tdbbackup -r is being added to allow this snapshot.  It is a
> necessary but not alone a sufficient element of the process). 
> 
> Thanks,
> 
> Andrew Bartlett
> 

Rowland