[SCM] Samba Shared Repository - branch master updated

Gary Lockyer gary at samba.org
Mon Aug 6 06:46:02 UTC 2018


The branch, master has been updated
       via  eb4161d selftest: offline backup restore target
       via  2800611 netcmd: domain backup offline command - offline test with ldapcmp
       via  2104818 tests: New offline backup tests with tweaks to old online classes
       via  f17d201 netcmd: domain backup offline command
       via  4f532cc netcmd: Improve domain backup targetdir checks
       via  0421737 tdb: test for readonly locks mode on tdbbackup command
       via  8f83933 tdb: adding readonly locks mode to tdbbackup tool
      from  dea788e dns scavenging: Add extra tests for custom filter

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit eb4161d701465800beca2400aed292baa91bd936
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Mon Jul 23 15:20:03 2018 +1200

    selftest: offline backup restore target
    
    This is a selftest target built from a restored offline backup.
    Other backup routines are modified to remove the assumption that every backup
    requires server and credentials arguments, since offline backup doesn't
    want them.  Also, prepare_dc_testenv now returns the generated ctx so we can
    run or re-run routines that require it later.
    
    Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
    
    Autobuild-User(master): Gary Lockyer <gary at samba.org>
    Autobuild-Date(master): Mon Aug  6 08:45:19 CEST 2018 on sn-devel-144

commit 2800611df3295d5ba4c82b2ecd0f8a77d6a5c658
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Tue May 1 11:10:31 2018 +1200

    netcmd: domain backup offline command - offline test with ldapcmp
    
    This test checks that when you do an offline backup and restore or untar it,
    the restored database is the same as the original.  Test is repeated for
    'mdb' and 'tdb' database backends.
    
    Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

commit 2104818ea60d51c54c9243ace5a28d2a087996f1
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Jul 12 14:34:56 2018 +1200

    tests: New offline backup tests with tweaks to old online classes
    
    Offline backups have a slightly different syntax, as they don't take the
    server or user-creds parameters. In the untar case, the offline backup
    will actually have the secrets present, so making asserting on this
    more flexible.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

commit f17d20123a4dc6b3b798456775c9449a2afd4921
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Tue Jun 26 13:47:42 2018 +1200

    netcmd: domain backup offline command
    
    Unlike the existing 'domain backup online' command, this command allows an
    admin to back up a local samba installation using the filesystem and the
    tdbbackup tool instead of using remote protocols.  It replaces samba_backup
    as that tool does not handle sam.ldb and secrets.ldb correctly.  Those two
    databases need to have transactions started on them before their downstream
    ldb and tdb files are backed up.
    
    Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

commit 4f532cc177cd1e95d8ccf8e69f50b315354df34c
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Jul 12 16:13:27 2018 +1200

    netcmd: Improve domain backup targetdir checks
    
    + Added check that specified targetdir is actually a directory (if it
    exists)
    + Deleted a redundant 'Creating targetdir' check that would never be hit
    + Move code into a separate function so we can reuse it for offline
    backups (which take a different set of parameters, but still have a
    targetdir)
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

commit 04217372f4114bd7583ba56a3860ebbe22ae0719
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Tue Jul 31 10:07:53 2018 +1200

    tdb: test for readonly locks mode on tdbbackup command
    
    Simple bash test for readonly locks on tdbbackup:
    1. Running tdbbackup on a database with and without readonly locks enabled.
    2. Dump both backups and original.
    3. Check all three dumps match.
    
    A binary sample_tdb.tdb file is included for the test because the existing
    sample tdbs in lib/tdb/test are either corrupt or empty.
    
    Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

commit 8f83933f6ea1edfdaff2da26ef264cc028b9e6a2
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Tue May 1 11:10:43 2018 +1200

    tdb: adding readonly locks mode to tdbbackup tool
    
    The netcmd 'domain backup offline' command will use the tdbbackup tool but
    require readonly locking of tdb databases, otherwise all database access would
    be blocked during a backup.  This patch adds the option.  A backup script
    should use this tool with the readonly locks option after taking a transaction
    lock on the target database.
    
    Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

-----------------------------------------------------------------------

Summary of changes:
 docs-xml/manpages/samba-tool.8.xml          |   5 +
 lib/tdb/test/sample_tdb.tdb                 | Bin 0 -> 8192 bytes
 lib/tdb/test/test_tdbbackup.sh              |  54 ++++++
 lib/tdb/tools/tdbbackup.c                   |  35 +++-
 lib/tdb/wscript                             |  11 ++
 python/samba/mdb_util.py                    |   7 +-
 python/samba/netcmd/domain_backup.py        | 257 ++++++++++++++++++++++++++--
 python/samba/tdb_util.py                    |  12 +-
 python/samba/tests/domain_backup.py         |  41 +++--
 python/samba/tests/domain_backup_offline.py | 117 +++++++++++++
 selftest/target/Samba.pm                    |   1 +
 selftest/target/Samba4.pm                   | 101 +++++++++--
 source4/scripting/bin/samba_backup          |  97 -----------
 source4/selftest/tests.py                   |  12 +-
 14 files changed, 592 insertions(+), 158 deletions(-)
 create mode 100644 lib/tdb/test/sample_tdb.tdb
 create mode 100755 lib/tdb/test/test_tdbbackup.sh
 create mode 100644 python/samba/tests/domain_backup_offline.py
 delete mode 100755 source4/scripting/bin/samba_backup


Changeset truncated at 500 lines:

diff --git a/docs-xml/manpages/samba-tool.8.xml b/docs-xml/manpages/samba-tool.8.xml
index e4b278e..2c043b9 100644
--- a/docs-xml/manpages/samba-tool.8.xml
+++ b/docs-xml/manpages/samba-tool.8.xml
@@ -297,6 +297,11 @@
 </refsect3>
 
 <refsect3>
+	<title>domain backup offline</title>
+	<para>Backup (with proper locking) local domain directories into a tar file.</para>
+</refsect3>
+
+<refsect3>
 	<title>domain backup online</title>
 	<para>Copy a running DC's current DB into a backup tar file.</para>
 </refsect3>
diff --git a/lib/tdb/test/sample_tdb.tdb b/lib/tdb/test/sample_tdb.tdb
new file mode 100644
index 0000000..a40e50c
Binary files /dev/null and b/lib/tdb/test/sample_tdb.tdb differ
diff --git a/lib/tdb/test/test_tdbbackup.sh b/lib/tdb/test/test_tdbbackup.sh
new file mode 100755
index 0000000..cf87921
--- /dev/null
+++ b/lib/tdb/test/test_tdbbackup.sh
@@ -0,0 +1,54 @@
+#!/bin/sh
+# Blackbox test for tdbbackup of given ldb or tdb database
+# Copyright (C) 2018 Andrew Bartlett <abartlet at samba.org>
+
+if [ $# -lt 1 ]; then
+	echo "Usage: $0 LDBFILE"
+	exit 1;
+fi
+
+LDBFILE=$1
+
+timestamp() {
+  date -u +'time: %Y-%m-%d %H:%M:%S.%6NZ' | sed 's/\..*NZ$/.000000Z/'
+}
+
+subunit_fail_test () {
+  timestamp
+  printf 'failure: %s [\n' "$1"
+  cat -
+  echo "]"
+}
+
+testit () {
+	name="$1"
+	shift
+	cmdline="$@"
+	timestamp
+	printf 'test: %s\n' "$1"
+	output=`$cmdline 2>&1`
+	status=$?
+	if [ x$status = x0 ]; then
+		timestamp
+		printf 'success: %s\n' "$name"
+	else
+		echo "$output" | subunit_fail_test "$name"
+	fi
+	return $status
+}
+
+$BINDIR/tdbdump $LDBFILE | sort > orig_dump
+
+testit "normal tdbbackup on tdb file" $BINDIR/tdbbackup $LDBFILE -s .bak
+$BINDIR/tdbdump $LDBFILE.bak | sort > bak_dump
+testit "cmp between tdbdumps of original and backup" cmp orig_dump bak_dump
+rm $LDBFILE.bak
+rm bak_dump
+
+testit "readonly tdbbackup on tdb file" $BINDIR/tdbbackup $LDBFILE -s .bak -r
+$BINDIR/tdbdump $LDBFILE.bak | sort > bak_dump
+testit "cmp between tdbdumps of original and back dbs" cmp orig_dump bak_dump
+rm $LDBFILE.bak
+rm bak_dump
+
+rm orig_dump
diff --git a/lib/tdb/tools/tdbbackup.c b/lib/tdb/tools/tdbbackup.c
index eb33e25..1125987 100644
--- a/lib/tdb/tools/tdbbackup.c
+++ b/lib/tdb/tools/tdbbackup.c
@@ -105,7 +105,7 @@ static int test_fn(TDB_CONTEXT *tdb, TDB_DATA key, TDB_DATA dbuf, void *state)
   this function is also used for restore
 */
 static int backup_tdb(const char *old_name, const char *new_name,
-		      int hash_size, int nolock)
+		      int hash_size, int nolock, bool readonly)
 {
 	TDB_CONTEXT *tdb;
 	TDB_CONTEXT *tdb_new;
@@ -145,8 +145,17 @@ static int backup_tdb(const char *old_name, const char *new_name,
 		return 1;
 	}
 
-	if (tdb_transaction_start(tdb) != 0) {
-		printf("Failed to start transaction on old tdb\n");
+	if (readonly) {
+		if (tdb_lockall_read(tdb) != 0) {
+			printf("Failed to obtain read only lock on old tdb\n");
+			tdb_close(tdb);
+			tdb_close(tdb_new);
+			unlink(tmp_name);
+			free(tmp_name);
+			return 1;
+		}
+	} else if (tdb_transaction_start(tdb) != 0) {
+		printf("Failed to start transaction on db\n");
 		tdb_close(tdb);
 		tdb_close(tdb_new);
 		unlink(tmp_name);
@@ -167,7 +176,15 @@ static int backup_tdb(const char *old_name, const char *new_name,
 	failed = 0;
 
 	/* traverse and copy */
-	count1 = tdb_traverse(tdb, copy_fn, (void *)tdb_new);
+	if (readonly) {
+		count1 = tdb_traverse_read(tdb,
+					   copy_fn,
+					   (void *)tdb_new);
+	} else {
+		count1 = tdb_traverse(tdb,
+				      copy_fn,
+				      (void *)tdb_new);
+	}
 	if (count1 < 0 || failed) {
 		fprintf(stderr,"failed to copy %s\n", old_name);
 		tdb_close(tdb);
@@ -251,7 +268,7 @@ static int verify_tdb(const char *fname, const char *bak_name)
 	/* count is < 0 means an error */
 	if (count < 0) {
 		printf("restoring %s\n", fname);
-		return backup_tdb(bak_name, fname, 0, 0);
+		return backup_tdb(bak_name, fname, 0, 0, 0);
 	}
 
 	printf("%s : %d records\n", fname, count);
@@ -282,6 +299,7 @@ static void usage(void)
 	printf("   -v            verify mode (restore if corrupt)\n");
 	printf("   -n hashsize   set the new hash size for the backup\n");
 	printf("   -l            open without locking to back up mutex dbs\n");
+	printf("   -r            open with read only locking\n");
 }
 
  int main(int argc, char *argv[])
@@ -292,11 +310,12 @@ static void usage(void)
 	int verify = 0;
 	int hashsize = 0;
 	int nolock = 0;
+	bool readonly = false;
 	const char *suffix = ".bak";
 
 	log_ctx.log_fn = tdb_log;
 
-	while ((c = getopt(argc, argv, "vhs:n:l")) != -1) {
+	while ((c = getopt(argc, argv, "vhs:n:lr")) != -1) {
 		switch (c) {
 		case 'h':
 			usage();
@@ -313,6 +332,8 @@ static void usage(void)
 		case 'l':
 			nolock = 1;
 			break;
+		case 'r':
+			readonly = true;
 		}
 	}
 
@@ -337,7 +358,7 @@ static void usage(void)
 		} else {
 			if (file_newer(fname, bak_name) &&
 			    backup_tdb(fname, bak_name, hashsize,
-				       nolock) != 0) {
+				       nolock, readonly) != 0) {
 				ret = 1;
 			}
 		}
diff --git a/lib/tdb/wscript b/lib/tdb/wscript
index fdf218c..414943a 100644
--- a/lib/tdb/wscript
+++ b/lib/tdb/wscript
@@ -213,6 +213,17 @@ def testonly(ctx):
         if not os.path.exists(link):
             os.symlink(os.path.abspath(os.path.join(env.cwd, 'test')), link)
 
+        sh_tests = ["test/test_tdbbackup.sh test/jenkins-be-hash.tdb"]
+
+        for sh_test in sh_tests:
+            cmd = "BINDIR={} {}".format(Utils.g_module.blddir, sh_test)
+            print("shell test: " + cmd)
+            ret = samba_utils.RUN_COMMAND(cmd)
+            if ret != 0:
+                print("%s sh test failed" % cmd)
+                ecode = ret
+                break
+
         for t in tdb1_unit_tests:
             f = "tdb1-" + t
             cmd = "cd " + testdir + " && " + os.path.abspath(os.path.join(Utils.g_module.blddir, f)) + " > test-output 2>&1"
diff --git a/python/samba/mdb_util.py b/python/samba/mdb_util.py
index 4fc6c3e..4dbff48 100644
--- a/python/samba/mdb_util.py
+++ b/python/samba/mdb_util.py
@@ -32,9 +32,6 @@ def mdb_copy(file1, file2):
             break
 
     mdb_copy_cmd = [toolpath, "-n", file1, "%s.copy.mdb" % file1]
-    status = subprocess.call(mdb_copy_cmd, close_fds=True, shell=False)
+    status = subprocess.check_call(mdb_copy_cmd, close_fds=True, shell=False)
 
-    if status == 0:
-        os.rename("%s.copy.mdb" % file1, file2)
-    else:
-        raise Exception("Error copying %d  %s" % (status, file1))
+    os.rename("%s.copy.mdb" % file1, file2)
diff --git a/python/samba/netcmd/domain_backup.py b/python/samba/netcmd/domain_backup.py
index cfd9796..05146c0 100644
--- a/python/samba/netcmd/domain_backup.py
+++ b/python/samba/netcmd/domain_backup.py
@@ -23,11 +23,12 @@ import logging
 import shutil
 import tempfile
 import samba
+import tdb
 import samba.getopt as options
 from samba.samdb import SamDB
 import ldb
 from samba import smb
-from samba.ntacls import backup_online, backup_restore
+from samba.ntacls import backup_online, backup_restore, backup_offline
 from samba.auth import system_session
 from samba.join import DCJoinContext, join_clone, DCCloneAndRenameContext
 from samba.dcerpc.security import dom_sid
@@ -45,6 +46,11 @@ from samba.provision import guess_names, determine_host_ip, determine_host_ip6
 from samba.provision.sambadns import (fill_dns_data_partitions,
                                       get_dnsadmins_sid,
                                       get_domainguid)
+from samba.tdb_util import tdb_copy
+from samba.mdb_util import mdb_copy
+import errno
+import tdb
+from subprocess import CalledProcessError
 
 
 # work out a SID (based on a free RID) to use when the domain gets restored.
@@ -139,6 +145,17 @@ def add_backup_marker(samdb, marker, value):
     samdb.modify(m)
 
 
+def check_targetdir(logger, targetdir):
+    if targetdir is None:
+        raise CommandError('Target directory required')
+
+    if not os.path.exists(targetdir):
+        logger.info('Creating targetdir %s...' % targetdir)
+        os.makedirs(targetdir)
+    elif not os.path.isdir(targetdir):
+        raise CommandError("%s is not a directory" % targetdir)
+
+
 def check_online_backup_args(logger, credopts, server, targetdir):
     # Make sure we have all the required args.
     u_p = {'user': credopts.creds.get_username(),
@@ -147,12 +164,8 @@ def check_online_backup_args(logger, credopts, server, targetdir):
         raise CommandError("Creds required.")
     if server is None:
         raise CommandError('Server required')
-    if targetdir is None:
-        raise CommandError('Target directory required')
 
-    if not os.path.exists(targetdir):
-        logger.info('Creating targetdir %s...' % targetdir)
-        os.makedirs(targetdir)
+    check_targetdir(logger, targetdir)
 
 
 # For '--no-secrets' backups, this sets the Administrator user's password to a
@@ -211,10 +224,6 @@ class cmd_domain_backup_online(samba.netcmd.Command):
         lp = sambaopts.get_loadparm()
         creds = credopts.get_credentials(lp)
 
-        if not os.path.exists(targetdir):
-            logger.info('Creating targetdir %s...' % targetdir)
-            os.makedirs(targetdir)
-
         tmpdir = tempfile.mkdtemp(dir=targetdir)
 
         # Run a clone join on the remote
@@ -769,8 +778,234 @@ class cmd_domain_backup_rename(samba.netcmd.Command):
         shutil.rmtree(tmpdir)
 
 
+class cmd_domain_backup_offline(samba.netcmd.Command):
+    '''Backup the local domain directories safely into a tar file.
+
+    Takes a backup copy of the current domain from the local files on disk,
+    with proper locking of the DB to ensure consistency. If the domain were to
+    undergo a catastrophic failure, then the backup file can be used to recover
+    the domain.
+
+    An offline backup differs to an online backup in the following ways:
+    - a backup can be created even if the DC isn't currently running.
+    - includes non-replicated attributes that an online backup wouldn't store.
+    - takes a copy of the raw database files, which has the risk that any
+      hidden problems in the DB are preserved in the backup.'''
+
+    synopsis = "%prog [options]"
+    takes_optiongroups = {
+        "sambaopts": options.SambaOptions,
+    }
+
+    takes_options = [
+        Option("--targetdir",
+               help="Output directory (required)",
+               type=str),
+    ]
+
+    backup_ext = '.bak-offline'
+
+    def offline_tdb_copy(self, path):
+        backup_path = path + self.backup_ext
+        try:
+            tdb_copy(path, backup_path, readonly=True)
+        except CalledProcessError as copy_err:
+            # If the copy didn't work, check if it was caused by an EINVAL
+            # error on opening the DB.  If so, it's a mutex locked database,
+            # which we can safely ignore.
+            try:
+                tdb.open(path)
+            except Exception as e:
+                if hasattr(e, 'errno') and e.errno == errno.EINVAL:
+                    return
+                raise e
+            raise copy_err
+        if not os.path.exists(backup_path):
+            s = "tdbbackup said backup succeeded but {} not found"
+            raise CommandError(s.format(backup_path))
+
+    def offline_mdb_copy(self, path):
+        mdb_copy(path, path + self.backup_ext)
+
+    # Secrets databases are a special case: a transaction must be started
+    # on the secrets.ldb file before backing up that file and secrets.tdb
+    def backup_secrets(self, private_dir, lp, logger):
+        secrets_path = os.path.join(private_dir, 'secrets')
+        secrets_obj = Ldb(secrets_path + '.ldb', lp=lp)
+        logger.info('Starting transaction on ' + secrets_path)
+        secrets_obj.transaction_start()
+        self.offline_tdb_copy(secrets_path + '.ldb')
+        self.offline_tdb_copy(secrets_path + '.tdb')
+        secrets_obj.transaction_cancel()
+
+    # sam.ldb must have a transaction started on it before backing up
+    # everything in sam.ldb.d with the appropriate backup function.
+    def backup_smb_dbs(self, private_dir, samdb, lp, logger):
+        # First, determine if DB backend is MDB.  Assume not unless there is a
+        # 'backendStore' attribute on @PARTITION containing the text 'mdb'
+        store_label = "backendStore"
+        res = samdb.search(base="@PARTITION", scope=ldb.SCOPE_BASE,
+                           attrs=[store_label])
+        mdb_backend = store_label in res[0] and res[0][store_label][0] == 'mdb'
+
+        sam_ldb_path = os.path.join(private_dir, 'sam.ldb')
+        copy_function = None
+        if mdb_backend:
+            logger.info('MDB backend detected.  Using mdb backup function.')
+            copy_function = self.offline_mdb_copy
+        else:
+            logger.info('Starting transaction on ' + sam_ldb_path)
+            copy_function = self.offline_tdb_copy
+            sam_obj = Ldb(sam_ldb_path, lp=lp)
+            sam_obj.transaction_start()
+
+        logger.info('   backing up ' + sam_ldb_path)
+        self.offline_tdb_copy(sam_ldb_path)
+        sam_ldb_d = sam_ldb_path + '.d'
+        for sam_file in os.listdir(sam_ldb_d):
+            sam_file = os.path.join(sam_ldb_d, sam_file)
+            if sam_file.endswith('.ldb'):
+                logger.info('   backing up locked/related file ' + sam_file)
+                copy_function(sam_file)
+            else:
+                logger.info('   copying locked/related file ' + sam_file)
+                shutil.copyfile(sam_file, sam_file + self.backup_ext)
+
+        if not mdb_backend:
+            sam_obj.transaction_cancel()
+
+    # Find where a path should go in the fixed backup archive structure.
+    def get_arc_path(self, path, conf_paths):
+        backup_dirs = {"private": conf_paths.private_dir,
+                       "statedir": conf_paths.state_dir,
+                       "etc": os.path.dirname(conf_paths.smbconf)}
+        matching_dirs = [(_, p) for (_, p) in backup_dirs.items() if
+                         path.startswith(p)]
+        arc_path, fs_path = matching_dirs[0]
+
+        # If more than one directory is a parent of this path, then at least
+        # one configured path is a subdir of another. Use closest match.
+        if len(matching_dirs) > 1:
+            arc_path, fs_path = max(matching_dirs, key=lambda (_, p): len(p))
+        arc_path += path[len(fs_path):]
+
+        return arc_path
+
+    def run(self, sambaopts=None, targetdir=None):
+
+        logger = logging.getLogger()
+        logger.setLevel(logging.DEBUG)
+        logger.addHandler(logging.StreamHandler(sys.stdout))
+
+        # Get the absolute paths of all the directories we're going to backup
+        lp = sambaopts.get_loadparm()
+
+        paths = samba.provision.provision_paths_from_lp(lp, lp.get('realm'))
+        if not (paths.samdb and os.path.exists(paths.samdb)):
+            raise CommandError('No sam.db found.  This backup ' +
+                               'tool is only for AD DCs')
+
+        check_targetdir(logger, targetdir)
+
+        samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+        sid = get_sid_for_restore(samdb)
+
+        backup_dirs = [paths.private_dir, paths.state_dir,
+                       os.path.dirname(paths.smbconf)]  # etc dir
+        logger.info('running backup on dirs: {}'.format(backup_dirs))
+
+        # Recursively get all file paths in the backup directories
+        all_files = []
+        for backup_dir in backup_dirs:
+            for (working_dir, _, filenames) in os.walk(backup_dir):
+                if working_dir.startswith(paths.sysvol):
+                    continue
+
+                for filename in filenames:
+                    if filename in all_files:
+                        continue
+
+                    # Assume existing backup files are from a previous backup.
+                    # Delete and ignore.
+                    if filename.endswith(self.backup_ext):
+                        os.remove(os.path.join(working_dir, filename))
+                        continue
+                    all_files.append(os.path.join(working_dir, filename))
+
+        # Backup secrets, sam.ldb and their downstream files
+        self.backup_secrets(paths.private_dir, lp, logger)
+        self.backup_smb_dbs(paths.private_dir, samdb, lp, logger)
+
+        # Open the new backed up samdb, flag it as backed up, and write
+        # the next SID so the restore tool can add objects.
+        # WARNING: Don't change this code unless you know what you're doing.
+        #          Writing to a .bak file only works because the DN being
+        #          written to happens to be top level.
+        samdb = SamDB(url=paths.samdb + self.backup_ext,
+                      session_info=system_session(), lp=lp)
+        time_str = get_timestamp()
+        add_backup_marker(samdb, "backupDate", time_str)
+        add_backup_marker(samdb, "sidForRestore", sid)
+
+        # Now handle all the LDB and TDB files that are not linked to
+        # anything else.  Use transactions for LDBs.
+        for path in all_files:
+            if not os.path.exists(path + self.backup_ext):
+                if path.endswith('.ldb'):
+                    logger.info('Starting transaction on solo db: ' + path)
+                    ldb_obj = Ldb(path, lp=lp)
+                    ldb_obj.transaction_start()
+                    logger.info('   running tdbbackup on the same file')
+                    self.offline_tdb_copy(path)
+                    ldb_obj.transaction_cancel()
+                elif path.endswith('.tdb'):
+                    logger.info('running tdbbackup on lone tdb file ' + path)
+                    self.offline_tdb_copy(path)
+
+        # Now make the backup tar file and add all
+        # backed up files and any other files to it.
+        temp_tar_dir = tempfile.mkdtemp(dir=targetdir,
+                                        prefix='INCOMPLETEsambabackupfile')
+        temp_tar_name = os.path.join(temp_tar_dir, "samba-backup.tar.bz2")
+        tar = tarfile.open(temp_tar_name, 'w:bz2')
+
+        logger.info('running offline ntacl backup of sysvol')
+        sysvol_tar_fn = 'sysvol.tar.gz'
+        sysvol_tar = os.path.join(temp_tar_dir, sysvol_tar_fn)
+        backup_offline(paths.sysvol, sysvol_tar, samdb, paths.smbconf)
+        tar.add(sysvol_tar, sysvol_tar_fn)
+        os.remove(sysvol_tar)
+
+        create_log_file(temp_tar_dir, lp, "offline", "localhost", True)
+        backup_fn = os.path.join(temp_tar_dir, "backup.txt")
+        tar.add(backup_fn, os.path.basename(backup_fn))
+        os.remove(backup_fn)
+
+        logger.info('building backup tar')
+        for path in all_files:
+            arc_path = self.get_arc_path(path, paths)


-- 
Samba Shared Repository



More information about the samba-cvs mailing list