[SCM] Samba Shared Repository - branch v4-13-test updated

Karolin Seeger kseeger at samba.org
Tue Jul 13 13:19:01 UTC 2021


The branch, v4-13-test has been updated
       via  b9b1d98af4c smbXsrv_{open,session,tcon}: protect smbXsrv_{open,session,tcon}_global_traverse_fn against invalid records
       via  7065f203a9f gensec_krb5: restore ipv6 support for kpasswd
       via  82e0f3e7997 netcmd: Use next_free_rid() function to calculate a SID for restoring a backup
       via  e5c3a675464 python/tests/dsdb: Add tests for RID allocation functions
       via  afad2fd9e24 dsdb: Add next_free_rid() function to allocate a RID without modifying the database
       via  b3d59842fd9 netcmd: Add tests for performing an offline backup immediately after joining a domain
       via  00444ac64f5 netcmd: Ignore rIDUsedPool attribute in offline domain backup test
       via  445fb770c77 netcmd: Fix error-checking condition
       via  303a0ecdd9d netcmd: Avoid database corruption by opting not to create database files during an offline domain backup
       via  54c353e9ad6 netcmd: Determine which files are to be copied for an offline domain backup
       via  4a68b1cb2dc netcmd: Add test for an offline backup of nested directories
       via  6569d0b9967 netcmd: Add test for an offline backup of a directory containing hardlinks
       via  d0bde5703b2 samba-tool: Give better error information when the 'domain backup restore' fails with a duplicate SID
       via  6e284db7877 samba-tool domain backup: Confirm the sidForRestore we will put into the backup is free
      from  b01c4526fef s3: smbd: Fix uninitialized memory read in process_symlink_open() when used with vfs_shadow_copy2().

https://git.samba.org/?p=samba.git;a=shortlog;h=v4-13-test


- Log -----------------------------------------------------------------
commit b9b1d98af4c7cd2326e12e1c3b734056663932d1
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Jul 5 17:17:30 2021 +0200

    smbXsrv_{open,session,tcon}: protect smbXsrv_{open,session,tcon}_global_traverse_fn against invalid records
    
    I saw systems with locking.tdb records being part of:
      ctdb catdb smbXsrv_tcon_global.tdb
    
    It's yet unknown how that happened, but we should not panic in srvsvc_*
    calls because the info0 pointer was NULL.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14752
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Volker Lendecke <vl at samba.org>
    
    Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
    Autobuild-Date(master): Tue Jul  6 11:08:43 UTC 2021 on sn-devel-184
    
    (cherry picked from commit 00bab5b3c821f272153a25ded9743460887a7907)
    
    Autobuild-User(v4-13-test): Karolin Seeger <kseeger at samba.org>
    Autobuild-Date(v4-13-test): Tue Jul 13 13:18:20 UTC 2021 on sn-devel-184

commit 7065f203a9fa0618e9a72043ec925eee7c7cdd01
Author: Stefan Metzmacher <metze at samba.org>
Date:   Fri Jul 2 09:37:25 2021 +0200

    gensec_krb5: restore ipv6 support for kpasswd
    
    We need to offer as much space we have in order to
    get the address out of tsocket_address_bsd_sockaddr().
    
    This fixes a regression in commit
    43c808f2ff907497dfff0988ff90a48fdcfc16ef.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14750
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 0388a8f33bdde49f1cc805a0291859203c1a52b4)

commit 82e0f3e79975ffdffd5afca77b6458a33488eff7
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Thu May 27 15:35:35 2021 +1200

    netcmd: Use next_free_rid() function to calculate a SID for restoring a backup
    
    This means we won't get errors if the DC doesn't have a rIDNextRID
    attribute, but we will still error if there is no RID Set or if all its
    pools are exhausted.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit 59d293b60608172ae61551c642d13d3b215924e4)

commit e5c3a675464208bffad08a0e923406c9a2d4b0a5
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Mon May 24 16:46:28 2021 +1200

    python/tests/dsdb: Add tests for RID allocation functions
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit 7c7cad81844950c3efe9a540a47b9d4e1ce1b2a1)

commit afad2fd9e2499f6ddacae9ddace22c81e9de7da0
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Mon May 24 12:59:59 2021 +1200

    dsdb: Add next_free_rid() function to allocate a RID without modifying the database
    
    If used to generate SIDs for objects, care should be taken, as the
    possibility for having duplicate objectSIDs can arise.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit cc98e03e7a0f2bf7a1ace2950fe6500f53640c1b)

commit b3d59842fd99c8d72dbc6f65259efad05bd5d897
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Mon May 24 14:58:40 2021 +1200

    netcmd: Add tests for performing an offline backup immediately after joining a domain
    
    This currently fails due to the DC not having a rIDNextRID attribute,
    which is required for the restore process.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit b7e6a1c5da7283c49586dc29f85ab19e0e57b0f6)

commit 00444ac64f5b9ece88a62100b9b98c6fe75f6a60
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Wed May 26 13:40:30 2021 +1200

    netcmd: Ignore rIDUsedPool attribute in offline domain backup test
    
    The RID Set of the newly created DC account has all its values
    initialised to zero. If the rIDUsedPool attribute was previously
    non-zero, then the restore process will cause its value to change.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit 658e5a6cc20b57f48477affd370fe25458178b92)

commit 445fb770c77b488c3ff7e426d985206967f5d825
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Mon May 24 16:40:55 2021 +1200

    netcmd: Fix error-checking condition
    
    This condition probably meant to check the argument of the most recently
    thrown exception, rather than the previous one again.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit e8c242bed19432d96e78dc345ab5f06422c5b104)

commit 303a0ecdd9d6c3072824380b4faa88cb0821f117
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Tue Mar 16 22:20:21 2021 +1300

    netcmd: Avoid database corruption by opting not to create database files during an offline domain backup
    
    If backup dirs contain hardlinks, the backup process could previously
    attempt to open an LMDB database already opened during the backup,
    causing it to be recreated as a new TDB database. This commit ensures
    that new database files are not created during this operation, and that
    the main SamDB database is not modified.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    (cherry picked from commit 4cf773591d49166b8c7ef8d637d7edfe755d48aa)

commit 54c353e9ad65a8a764963bddd6396dab4779bfe3
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Tue Mar 16 16:22:40 2021 +1300

    netcmd: Determine which files are to be copied for an offline domain backup
    
    The old behaviour attempted to check for and remove files with duplicate
    names, but did not do so due to a bug, and would have left undetermined
    which files were given priority when duplicate filenames were present.
    Now when hardlinks are present, only one instance of each file is
    chosen, with files in the private directory having priority. If one
    backup dir is nested inside another, the files contained in the nested
    directory are only added once. Additionally, the BIND DNS database is
    omitted from the backup.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    (cherry picked from commit 3723148e7aa7e6d4a48a1a38112f121f52b6ee6f)

commit 4a68b1cb2dc9b32f49423b96c1fd935f5886d057
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Thu Mar 18 10:52:52 2021 +1300

    netcmd: Add test for an offline backup of nested directories
    
    This test verifies that when performing an offline backup of a domain
    where one of the directories to be backed up is nested inside another,
    the contained files are only included once in the backup.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    (cherry picked from commit f994783f4279884ec4d2ee3e7db80fb7af267d1c)

commit 6569d0b99672e8752eb13704056094a729d929c5
Author: Joseph Sutton <josephsutton at catalyst.net.nz>
Date:   Tue Mar 16 16:13:05 2021 +1300

    netcmd: Add test for an offline backup of a directory containing hardlinks
    
    This test verifies that when performing an offline backup of a domain
    where the directories to be backed up contain hardlinks, only one
    instance of each file is backed up, and that files in the private
    directory take precedence.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
    
    Signed-off-by: Joseph Sutton <josephsutton at catalyst.net.nz>
    (cherry picked from commit 0e5738887524b467bfebcf657bcb00ed71827784)

commit d0bde5703b25fb34a0b672422c052909b1acbbef
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Fri Nov 13 15:26:07 2020 +1300

    samba-tool: Give better error information when the 'domain backup restore' fails with a duplicate SID
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14575
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
    
    Autobuild-User(master): Gary Lockyer <gary at samba.org>
    Autobuild-Date(master): Thu Nov 26 21:15:40 UTC 2020 on sn-devel-184
    
    (cherry picked from commit 8ad82ae66157c893a2b84d353ec4d9feb4815ede)

commit 6e284db78779aae20b49300ae304c5b643f31b7d
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Wed Nov 18 12:11:10 2020 +1300

    samba-tool domain backup: Confirm the sidForRestore we will put into the backup is free
    
    Otherwise the administrator might only find there is a problem once they
    attempt to restore the domain!
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=14575
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
    (cherry picked from commit 15609cb91986b3e29c5b1a3b6c69c04829f43eb4)

-----------------------------------------------------------------------

Summary of changes:
 python/samba/netcmd/domain_backup.py        | 173 +++++++++++-----
 python/samba/samdb.py                       | 105 ++++++++++
 python/samba/tests/domain_backup_offline.py | 162 ++++++++++++---
 python/samba/tests/dsdb.py                  | 305 +++++++++++++++++++++++++++-
 source3/smbd/smbXsrv_open.c                 |   9 +
 source3/smbd/smbXsrv_session.c              |   7 +
 source3/smbd/smbXsrv_tcon.c                 |   7 +
 source4/auth/gensec/gensec_krb5.c           |   6 +-
 source4/selftest/tests.py                   |   2 +-
 9 files changed, 692 insertions(+), 84 deletions(-)


Changeset truncated at 500 lines:

diff --git a/python/samba/netcmd/domain_backup.py b/python/samba/netcmd/domain_backup.py
index a3dc7fb454f..a629b31d70f 100644
--- a/python/samba/netcmd/domain_backup.py
+++ b/python/samba/netcmd/domain_backup.py
@@ -27,6 +27,7 @@ import tdb
 import samba.getopt as options
 from samba.samdb import SamDB, get_default_backend_store
 import ldb
+from ldb import LdbError
 from samba.samba3 import libsmb_samba_internal as libsmb
 from samba.samba3 import param as s3param
 from samba.ntacls import backup_online, backup_restore, backup_offline
@@ -60,53 +61,50 @@ from samba.ndr import ndr_pack
 # This ensures that the restored DC's SID won't clash with any other RIDs
 # already in use in the domain
 def get_sid_for_restore(samdb, logger):
-    # Find the DN of the RID set of the server
-    res = samdb.search(base=ldb.Dn(samdb, samdb.get_serverName()),
-                       scope=ldb.SCOPE_BASE, attrs=["serverReference"])
-    server_ref_dn = ldb.Dn(samdb, str(res[0]['serverReference'][0]))
-    res = samdb.search(base=server_ref_dn,
-                       scope=ldb.SCOPE_BASE,
-                       attrs=['rIDSetReferences'])
-    rid_set_dn = ldb.Dn(samdb, str(res[0]['rIDSetReferences'][0]))
-
-    # Get the alloc pools and next RID of the RID set
-    res = samdb.search(base=rid_set_dn,
-                       scope=ldb.SCOPE_SUBTREE,
-                       expression="(rIDNextRID=*)",
-                       attrs=['rIDAllocationPool',
-                              'rIDPreviousAllocationPool',
-                              'rIDNextRID'])
-
-    # Decode the bounds of the RID allocation pools
+    # Allocate a new RID without modifying the database. This should be safe,
+    # because we acquire the RID master role after creating an account using
+    # this RID during the restore process. Acquiring the RID master role
+    # creates a new RID pool which we will fetch RIDs from, so we shouldn't get
+    # duplicates.
     try:
-        rid = int(res[0].get('rIDNextRID')[0])
-    except IndexError:
-        logger.info("The RID pool for this DC is not initalized "
-                    "(e.g. it may be a fairly new DC).")
-        logger.info("To initialize it, create a temporary user on this DC "
-                    "(you can delete it later).")
-        raise CommandError("Cannot create backup - "
-                           "please initialize this DC's RID pool first.")
-
-    def split_val(num):
-        high = (0xFFFFFFFF00000000 & int(num)) >> 32
-        low = 0x00000000FFFFFFFF & int(num)
-        return low, high
-    pool_l, pool_h = split_val(res[0].get('rIDPreviousAllocationPool')[0])
-    npool_l, npool_h = split_val(res[0].get('rIDAllocationPool')[0])
-
-    # Calculate next RID based on pool bounds
-    if rid == npool_h:
-        raise CommandError('Out of RIDs, finished AllocPool')
-    if rid == pool_h:
-        if pool_h == npool_h:
-            raise CommandError('Out of RIDs, finished PrevAllocPool.')
-        rid = npool_l
-    else:
-        rid += 1
+        rid = samdb.next_free_rid()
+    except LdbError as err:
+        logger.info("A SID could not be allocated for restoring the domain. "
+                    "Either no RID Set was found on this DC, "
+                    "or the RID Set was not usable.")
+        logger.info("To initialise this DC's RID pools, obtain a RID Set from "
+                    "this domain's RID master, or run samba-tool dbcheck "
+                    "to fix the existing RID Set.")
+        raise CommandError("Cannot create backup", err)
 
     # Construct full SID
     sid = dom_sid(samdb.get_domain_sid())
+    sid_for_restore = str(sid) + '-' + str(rid)
+
+    # Confirm the SID is not already in use
+    try:
+        res = samdb.search(scope=ldb.SCOPE_BASE,
+                           base='<SID=%s>' % sid_for_restore,
+                           attrs=[],
+                           controls=['show_deleted:1',
+                                     'show_recycled:1'])
+        if len(res) != 1:
+            # This case makes no sense, but neither does a corrupt RID set
+            raise CommandError("Cannot create backup - "
+                               "this DC's RID pool is corrupt, "
+                               "the next SID (%s) appears to be in use." %
+                               sid_for_restore)
+        raise CommandError("Cannot create backup - "
+                           "this DC's RID pool is corrupt, "
+                           "the next SID %s points to existing object %s. "
+                           "Please run samba-tool dbcheck on the source DC." %
+                           (sid_for_restore, res[0].dn))
+    except ldb.LdbError as e:
+        (enum, emsg) = e.args
+        if enum != ldb.ERR_NO_SUCH_OBJECT:
+            # We want NO_SUCH_OBJECT, anything else is a serious issue
+            raise
+
     return str(sid) + '-' + str(rid)
 
 
@@ -278,7 +276,8 @@ class cmd_domain_backup_online(samba.netcmd.Command):
             shutil.rmtree(paths.sysvol)
 
             # Edit the downloaded sam.ldb to mark it as a backup
-            samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+            samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp,
+                          flags=ldb.FLG_DONT_CREATE_DB)
             time_str = get_timestamp()
             add_backup_marker(samdb, "backupDate", time_str)
             add_backup_marker(samdb, "sidForRestore", new_sid)
@@ -502,7 +501,8 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
         # open a DB connection to the restored DB
         private_dir = os.path.join(targetdir, 'private')
         samdb_path = os.path.join(private_dir, 'sam.ldb')
-        samdb = SamDB(url=samdb_path, session_info=system_session(), lp=lp)
+        samdb = SamDB(url=samdb_path, session_info=system_session(), lp=lp,
+                      flags=ldb.FLG_DONT_CREATE_DB)
         backup_type = self.get_backup_type(samdb)
 
         if site is None:
@@ -550,7 +550,50 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
                            attrs=['sidForRestore'])
         sid = res[0].get('sidForRestore')[0]
         logger.info('Creating account with SID: ' + str(sid))
-        ctx.join_add_objects(specified_sid=dom_sid(str(sid)))
+        try:
+            ctx.join_add_objects(specified_sid=dom_sid(str(sid)))
+        except LdbError as e:
+            (enum, emsg) = e.args
+            if enum != ldb.ERR_CONSTRAINT_VIOLATION:
+                raise
+
+            dup_res = []
+            try:
+                dup_res = samdb.search(base=ldb.Dn(samdb, "<SID=%s>" % sid),
+                                       scope=ldb.SCOPE_BASE,
+                                       attrs=['objectGUID'],
+                                       controls=["show_deleted:0",
+                                                 "show_recycled:0"])
+            except LdbError as dup_e:
+                (dup_enum, _) = dup_e.args
+                if dup_enum != ldb.ERR_NO_SUCH_OBJECT:
+                    raise
+
+            if (len(dup_res) != 1):
+                raise
+
+            objectguid = samdb.schema_format_value("objectGUID",
+                                                       dup_res[0]["objectGUID"][0])
+            objectguid = objectguid.decode('utf-8')
+            logger.error("The RID Pool on the source DC for the backup in %s "
+                         "may be corrupt "
+                         "or in conflict with SIDs already allocated "
+                         "in the domain. " % backup_file)
+            logger.error("Running 'samba-tool dbcheck' on the source "
+                         "DC (and obtaining a new backup) may correct the issue.")
+            logger.error("Alternatively please obtain a new backup "
+                         "against a different DC.")
+            logger.error("The SID we wish to use (%s) is recorded in "
+                         "@SAMBA_DSDB as the sidForRestore attribute."
+                         % sid)
+
+            raise CommandError("Domain restore failed because there "
+                               "is already an existing object (%s) "
+                               "with SID %s and objectGUID %s.  "
+                               "This conflicts with "
+                               "the new DC account we want to add "
+                               "for the restored domain.   " % (
+                                dup_res[0].dn, sid, objectguid))
 
         m = ldb.Message()
         m.dn = ldb.Dn(samdb, '@ROOTDSE')
@@ -568,7 +611,8 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
                                    host_ip, host_ip6, site)
 
         secrets_path = os.path.join(private_dir, 'secrets.ldb')
-        secrets_ldb = Ldb(secrets_path, session_info=system_session(), lp=lp)
+        secrets_ldb = Ldb(secrets_path, session_info=system_session(), lp=lp,
+                          flags=ldb.FLG_DONT_CREATE_DB)
         secretsdb_self_join(secrets_ldb, domain=ctx.domain_name,
                             realm=ctx.realm, dnsdomain=ctx.dnsdomain,
                             netbiosname=ctx.myname, domainsid=ctx.domsid,
@@ -860,7 +904,8 @@ class cmd_domain_backup_rename(samba.netcmd.Command):
 
         # connect to the local DB (making sure we use the new/renamed config)
         lp.load(paths.smbconf)
-        samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+        samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp,
+                      flags=ldb.FLG_DONT_CREATE_DB)
 
         # Edit the cloned sam.ldb to mark it as a backup
         time_str = get_timestamp()
@@ -948,7 +993,8 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
     # on the secrets.ldb file before backing up that file and secrets.tdb
     def backup_secrets(self, private_dir, lp, logger):
         secrets_path = os.path.join(private_dir, 'secrets')
-        secrets_obj = Ldb(secrets_path + '.ldb', lp=lp)
+        secrets_obj = Ldb(secrets_path + '.ldb', lp=lp,
+                          flags=ldb.FLG_DONT_CREATE_DB)
         logger.info('Starting transaction on ' + secrets_path)
         secrets_obj.transaction_start()
         self.offline_tdb_copy(secrets_path + '.ldb')
@@ -973,7 +1019,7 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
         else:
             logger.info('Starting transaction on ' + sam_ldb_path)
             copy_function = self.offline_tdb_copy
-            sam_obj = Ldb(sam_ldb_path, lp=lp)
+            sam_obj = Ldb(sam_ldb_path, lp=lp, flags=ldb.FLG_DONT_CREATE_DB)
             sam_obj.transaction_start()
 
         logger.info('   backing up ' + sam_ldb_path)
@@ -1025,9 +1071,14 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
 
         check_targetdir(logger, targetdir)
 
-        samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+        samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp,
+                      flags=ldb.FLG_RDONLY)
         sid = get_sid_for_restore(samdb, logger)
 
+        # Iterating over the directories in this specific order ensures that
+        # when the private directory contains hardlinks that are also contained
+        # in other directories to be backed up (such as in paths.binddns_dir),
+        # the hardlinks in the private directory take precedence.
         backup_dirs = [paths.private_dir, paths.state_dir,
                        os.path.dirname(paths.smbconf)]  # etc dir
         logger.info('running backup on dirs: {0}'.format(' '.join(backup_dirs)))
@@ -1040,22 +1091,31 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
                     continue
                 if working_dir.endswith('.sock') or '.sock/' in working_dir:
                     continue
+                # The BIND DNS database can be regenerated, so it doesn't need
+                # to be backed up.
+                if working_dir.startswith(os.path.join(paths.binddns_dir, 'dns')):
+                    continue
 
                 for filename in filenames:
-                    if filename in all_files:
+                    full_path = os.path.join(working_dir, filename)
+
+                    # Ignore files that have already been added. This prevents
+                    # duplicates if one backup dir is a subdirectory of another,
+                    # or if backup dirs contain hardlinks.
+                    if any(os.path.samefile(full_path, file) for file in all_files):
                         continue
 
                     # Assume existing backup files are from a previous backup.
                     # Delete and ignore.
                     if filename.endswith(self.backup_ext):
-                        os.remove(os.path.join(working_dir, filename))
+                        os.remove(full_path)
                         continue
 
                     # Sock files are autogenerated at runtime, ignore.
                     if filename.endswith('.sock'):
                         continue
 
-                    all_files.append(os.path.join(working_dir, filename))
+                    all_files.append(full_path)
 
         # Backup secrets, sam.ldb and their downstream files
         self.backup_secrets(paths.private_dir, lp, logger)
@@ -1067,7 +1127,8 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
         #          Writing to a .bak file only works because the DN being
         #          written to happens to be top level.
         samdb = SamDB(url=paths.samdb + self.backup_ext,
-                      session_info=system_session(), lp=lp)
+                      session_info=system_session(), lp=lp,
+                      flags=ldb.FLG_DONT_CREATE_DB)
         time_str = get_timestamp()
         add_backup_marker(samdb, "backupDate", time_str)
         add_backup_marker(samdb, "sidForRestore", sid)
@@ -1079,7 +1140,7 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
             if not os.path.exists(path + self.backup_ext):
                 if path.endswith('.ldb'):
                     logger.info('Starting transaction on solo db: ' + path)
-                    ldb_obj = Ldb(path, lp=lp)
+                    ldb_obj = Ldb(path, lp=lp, flags=ldb.FLG_DONT_CREATE_DB)
                     ldb_obj.transaction_start()
                     logger.info('   running tdbbackup on the same file')
                     self.offline_tdb_copy(path)
diff --git a/python/samba/samdb.py b/python/samba/samdb.py
index d903babb406..d13c5e3b7a2 100644
--- a/python/samba/samdb.py
+++ b/python/samba/samdb.py
@@ -1285,6 +1285,111 @@ schemaUpdateNow: 1
         '''return a new RID from the RID Pool on this DSA'''
         return dsdb._dsdb_allocate_rid(self)
 
+    def next_free_rid(self):
+        '''return the next free RID from the RID Pool on this DSA.
+
+        :note: This function is not intended for general use, and care must be
+            taken if it is used to generate objectSIDs. The returned RID is not
+            formally reserved for use, creating the possibility of duplicate
+            objectSIDs.
+        '''
+        rid, _ = self.free_rid_bounds()
+        return rid
+
+    def free_rid_bounds(self):
+        '''return the low and high bounds (inclusive) of RIDs that are
+            available for use in this DSA's current RID pool.
+
+        :note: This function is not intended for general use, and care must be
+            taken if it is used to generate objectSIDs. The returned range of
+            RIDs is not formally reserved for use, creating the possibility of
+            duplicate objectSIDs.
+        '''
+        # Get DN of this server's RID Set
+        server_name_dn = ldb.Dn(self, self.get_serverName())
+        res = self.search(base=server_name_dn,
+                          scope=ldb.SCOPE_BASE,
+                          attrs=["serverReference"])
+        try:
+            server_ref = res[0]["serverReference"]
+        except KeyError:
+            raise ldb.LdbError(
+                ldb.ERR_NO_SUCH_ATTRIBUTE,
+                "No RID Set DN - "
+                "Cannot find attribute serverReference of %s "
+                "to calculate reference dn" % server_name_dn) from None
+        server_ref_dn = ldb.Dn(self, server_ref[0].decode("utf-8"))
+
+        res = self.search(base=server_ref_dn,
+                          scope=ldb.SCOPE_BASE,
+                          attrs=["rIDSetReferences"])
+        try:
+            rid_set_refs = res[0]["rIDSetReferences"]
+        except KeyError:
+            raise ldb.LdbError(
+                ldb.ERR_NO_SUCH_ATTRIBUTE,
+                "No RID Set DN - "
+                "Cannot find attribute rIDSetReferences of %s "
+                "to calculate reference dn" % server_ref_dn) from None
+        rid_set_dn = ldb.Dn(self, rid_set_refs[0].decode("utf-8"))
+
+        # Get the alloc pools and next RID of this RID Set
+        res = self.search(base=rid_set_dn,
+                          scope=ldb.SCOPE_BASE,
+                          attrs=["rIDAllocationPool",
+                                 "rIDPreviousAllocationPool",
+                                 "rIDNextRID"])
+
+        uint32_max = 2**32 - 1
+        uint64_max = 2**64 - 1
+
+        try:
+            alloc_pool = int(res[0]["rIDAllocationPool"][0])
+        except KeyError:
+            alloc_pool = uint64_max
+        if alloc_pool == uint64_max:
+            raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+                               "Bad RID Set %s" % rid_set_dn)
+
+        try:
+            prev_pool = int(res[0]["rIDPreviousAllocationPool"][0])
+        except KeyError:
+            prev_pool = uint64_max
+        try:
+            next_rid = int(res[0]["rIDNextRID"][0])
+        except KeyError:
+            next_rid = uint32_max
+
+        # If we never used a pool, set up our first pool
+        if prev_pool == uint64_max or next_rid == uint32_max:
+            prev_pool = alloc_pool
+            next_rid = prev_pool & uint32_max
+
+        next_rid += 1
+
+        # Now check if our current pool is still usable
+        prev_pool_lo = prev_pool & uint32_max
+        prev_pool_hi = prev_pool >> 32
+        if next_rid > prev_pool_hi:
+            # We need a new pool, check if we already have a new one
+            # Otherwise we return an error code.
+            if alloc_pool == prev_pool:
+                raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+                                   "RID pools out of RIDs")
+
+            # Now use the new pool
+            prev_pool = alloc_pool
+            prev_pool_lo = prev_pool & uint32_max
+            prev_pool_hi = prev_pool >> 32
+            next_rid = prev_pool_lo
+
+        if next_rid < prev_pool_lo or next_rid > prev_pool_hi:
+            raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+                               "Bad RID chosen %d from range %d-%d" %
+                               (next_rid, prev_pool_lo, prev_pool_hi))
+
+        return next_rid, prev_pool_hi
+
     def normalize_dn_in_domain(self, dn):
         '''return a new DN expanded by adding the domain DN
 
diff --git a/python/samba/tests/domain_backup_offline.py b/python/samba/tests/domain_backup_offline.py
index 8b7209ec24d..21f42c6dab8 100644
--- a/python/samba/tests/domain_backup_offline.py
+++ b/python/samba/tests/domain_backup_offline.py
@@ -19,8 +19,12 @@ import tarfile
 import os
 import shutil
 import tempfile
-from samba.tests import BlackboxTestCase
+from samba.tests import BlackboxTestCase, BlackboxProcessError
 from samba.netcmd import CommandError
+from samba.param import LoadParm
+from samba.join import join_DC
+from samba.credentials import Credentials
+from samba.logger import get_samba_logger
 
 # The backup tests require that a completely clean LoadParm object gets used
 # for the restore. Otherwise the same global LP gets re-used, and the LP
@@ -31,6 +35,81 @@ from samba.netcmd import CommandError
 # so that we never inadvertently use .runcmd() by accident.
 class DomainBackupOfflineCmp(BlackboxTestCase):
 
+    def test_domain_backup_offline_nested_tdb(self):
+        self.nested_testcase('tdb')
+
+    def test_domain_backup_offline_nested_mdb(self):
+        self.nested_testcase('mdb')
+
+    def nested_testcase(self, backend):
+        self.prov_dir = self.provision(backend)
+        self.extract_dir = None
+
+        src = os.path.join(self.prov_dir, "private")
+        dst = os.path.join(self.prov_dir, "state", "private")
+
+        # Move private directory inside state directory
+        shutil.move(src, dst)
+
+        smbconf = os.path.join(self.prov_dir, "etc", "smb.conf")
+
+        # Update the conf file
+        lp = LoadParm(filename_for_non_global_lp=smbconf)
+        lp.set("private dir", dst)
+        lp.dump(False, smbconf)
+
+        backup_file = self.backup(self.prov_dir)
+
+        # Ensure each file is only present once in the tar file
+        tf = tarfile.open(backup_file)
+        names = tf.getnames()
+        self.assertEqual(len(names), len(set(names)))
+
+    def test_domain_backup_offline_join_restore_tdb(self):
+        self.join_restore_testcase('tdb')
+
+    def test_domain_backup_offline_join_restore_mdb(self):
+        self.join_restore_testcase('mdb')
+
+    def join_restore_testcase(self, backend):
+        self.prov_dir = self.join(backend)
+        self.extract_dir = None
+
+        try:
+            backup_file = self.backup(self.prov_dir)
+        except BlackboxProcessError as e:
+            self.fail(e)
+
+        self.extract_dir = self.restore(backup_file)
+
+    def test_domain_backup_offline_hard_link_tdb(self):
+        self.hard_link_testcase('tdb')
+
+    def test_domain_backup_offline_hard_link_mdb(self):
+        self.hard_link_testcase('mdb')
+
+    def hard_link_testcase(self, backend):
+        self.prov_dir = self.provision(backend)
+        self.extract_dir = None
+
+        # Create hard links in the private and state directories
+        os.link(os.path.join(self.prov_dir, "private", "krb5.conf"),
+                os.path.join(self.prov_dir, "state", "krb5.conf"))
+
+        backup_file = self.backup(self.prov_dir)
+
+        # Extract the backup
+        self.extract_dir = tempfile.mkdtemp(dir=self.tempdir)
+        tf = tarfile.open(backup_file)
+        tf.extractall(self.extract_dir)
+
+        # Ensure that the hard link in the private directory was backed up,
+        # while the one in the state directory was not.
+        self.assertTrue(os.path.exists(os.path.join(self.extract_dir,
+                                                    "private", "krb5.conf")))
+        self.assertFalse(os.path.exists(os.path.join(self.extract_dir,
+                                                    "statedir", "krb5.conf")))
+


-- 
Samba Shared Repository



More information about the samba-cvs mailing list