samba-tool domain backup and xattrs

[Andrew Bartlett]

On Sat, 2018-04-07 at 21:30 +0100, Rowland Penny wrote:
> On Sun, 08 Apr 2018 07:57:21 +1200
> Andrew Bartlett <abartlet at samba.org> wrote:
> 
> > On Sat, 2018-04-07 at 20:32 +0100, Rowland Penny wrote:
> > > On Sun, 08 Apr 2018 07:02:30 +1200
> > > Andrew Bartlett <abartlet at samba.org> wrote:
> > > 
> > > > On Sat, 2018-04-07 at 12:01 +0100, Rowland Penny wrote:
> > > > > On Sat, 07 Apr 2018 09:18:21 +0000
> > > > > Github bot account via samba-technical
> > > > > <samba-technical at lists.samba.org> wrote:
> > > > > 
> > > > > > New comment by abartlet on Samba Github repository
> > > > > > 
> > > > > > https://github.com/samba-team/samba/pull/160#issuecomment-379455902
> > > > > > Comment:
> > > > > > Just a heads-up that a project to replace this with a tested
> > > > > > 'samba-tool domain backup' and 'samba-tool domain restore' is
> > > > > > on-going (a patch was posted, but metze wanted the restore
> > > > > > tool done at the same time so that is in progress), so while
> > > > > > we can merge this it likely won't stay around long.
> > > > > > 
> > > > > > The proposal is that those tools will then replace and remove
> > > > > > samba_backup.
> > > > > 
> > > > > I asked once before, but never got an answer, does the python
> > > > > 'tar' do this: tar --acls --xattrs --xattrs-include='*.*' -cjf
> > > > > 
> > > > > If it doesn't then the proposed samba-tool is useless (as is the
> > > > > current sh script).
> > > > 
> > > > As Metze set the requirement that the restore not just be an
> > > > un-tar but a process, a sysvolreset could be added at that stage.
> > > > 
> > > 
> > > Which if you have given 'Domain Admins' a gidNumber and/or added
> > > more GPOs will not work.
> > > 
> > > > Indications online are that the python 2.6 tar doesn't support
> > > > xattrs[1], but at least it wouldn't be a move backwards.
> > > > 
> > > 
> > > As far as I can see, tarfile doesn't support xattrs (or ACLs, come
> > > to that) at all.
> > > 
> > > In my opinion python-tarfile is not suitable for the task, but what
> > > do I know, I only have a working 'restore' bash script.
> > 
> > G'Day Rowland,
> > 
> > Thanks for the extra feedback on the requirements here.   
> > 
> > Full automated testing is part of the brief here, and your use cases
> > are very helpful data towards that.  Also we have set a pattern that
> > the backup should not succeed if a restore is not possible, so we can
> > watch out for that.
> 
> Just how are you going to test if a restore will succeed before
> actually carrying out the backup ? Do you have a time machine ? ;-)

By confirming prerequisites.  To restore per metze's set of
instructions we need a RID for the new server, so we will stash one
away at backup time for example.  

It is also pretty standard to practice a restore after a backup to
confirm it worked, that could either be in the tool or as part of the
administrative advice.  

> > 
> > Sadly for the existing script or variants thereof, there is a serious
> > issue with using tdbbackup on the files in sam.ldb.d, if a global lock
> > isn't taken then they can be out of sync when backed up.  That is why
> > the tool was re-written not evolved.
> 
> I take it you are referring to tdbbackup here, if so how do take a
> global lock, or is some other tool used instead ?

See Aaron's patch posted a few weeks ago for the required technique
involving a transaction lock taken from the main python process, and
tdbbackup -r (a newly developed option) running in a subprocess.  

It is quite tricky to get safe.

> > 
> > (And if it is best to put the sysvol share in a tarfile within the
> > tarfile then we can do that too).
> 
> I might be missing something here, but creating a tarball with
> something that doesn't understand ACLs and xattrs, then wrapping this
> inside another tarball with something that doesn't ACLs and xattrs,
> isn't going to make much difference. You still will not have the ACLs
> & xattrs.

Perhaps I misunderstand, wasn't that the point of the tar command you
posted?

Finally, I hope you can agree we all have the same goal here, that is
tested, workable backups and restoration.  I'm sorry if this larger
effort (the locked backup and the hard work in the restoration steps
metze requested) has stepped on any toes. 

I hope this clarifies things,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba