[PATCH] [WIP] AD DC backup and restore tool

[Andrew Bartlett]

On Wed, 2018-05-16 at 10:10 +0100, Rowland Penny wrote:
> On Wed, 16 May 2018 20:30:16 +1200
> Andrew Bartlett via samba-technical <samba-technical at lists.samba.org>
> wrote:
> 
> > G'Day,
> > 
> > Just a heads up that Tim and I plan to finish up the backup tool soon.
> >  
> > Given the strong feedback so far it will include the restore tool,
> > however it won't include the extended attributes (file permissions on
> > the backup of the [netlogon share]).  
> > 
> > Handling the extended attributes turns out to be harder than you might
> > expect, while 'just' a couple more options to a tar command, testing
> > it runs up against the fake xattrs we use in our selftest environment,
> > which are not visible to tar.  
> > 
> > Likewise, for the online backup, the ideal option would be to query
> > these over SMB and store the NT ACL directly into the tar xattrs via
> > the python API.  However this isn't available until python 3.x 
> > 
> > Finally, testing runs into the same issue, we can't just extract the
> > files with tar because we need the xattrs put into the xattr.tdb.
> > 
> > As the client task was literally named 'samba_backup is not tested'
> > I'm loathed to add a feature I can't test. 
> > 
> > Therefore, I will be proposing the tool matching the existing
> > samba_backup for features, but more importantly with the critical
> > locking bug addressed.  
> > 
> > For the overly curious, the current WIP patches are part of this tree:
> > https://gitlab.com/catalyst-samba/samba/commits/aaron-backup2
> > 
> > Please let us know any further feedback we should be aware of when
> > presenting these.
> > 
> > Thanks,
> > 
> > Andrew Bartlett
> 
> Hi Andrew, what does backing up another server online get you, that just
> backing up the server the python code is running on doesn't ?

Quite a few things.  We generally find that DRS replication gives a
much more reliable snapshot of the DB, without hidden faults lurking
within the raw database. 

On the flip side, a file-based backup will get non-replicated
attributes.  Each has their place. 

Tim will also be building on the basis of the online tool to provide a
new domain-rename feature, which should be quite handy.

> If the code has mostly been written by Aaron, why is the copyright
> assigned to you ?

For internal corporate reasons.  While it looks strange, it is
deliberate and legitimate. 

> As I said before, get the locking bug code into Samba and backport it,
> we can discuss the python backup script then.

The inability to correctly lock the various databases safely in
samba_backup is the major purpose of the new script.  However, now that
we have needed to build it, we have tried to make it a proper part of
samba-tool, built in a way we can support long-term and backed by a
dazzling array of tests.

Running tdbbackup -r won't change a shell script into one that locks
the whole DB.  This is because a transaction lock (or at least a global
read lock) needs to be taken out correctly over the DB while the
databases are being copied.

(tdbbackup -r is being added to allow this snapshot.  It is a necessary
but not alone a sufficient element of the process). 

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba