[SCM] Samba Shared Repository - branch master updated

Noel Power npower at samba.org
Tue Dec 15 15:33:01 UTC 2020


The branch, master has been updated
       via  7a077f152aa s3:smbd:trans2.c - add twrp to tmp smb_fname in smbd_do_qfsinfo
       via  9bf9b998088 dbcheck: clarify check_object userparams
       via  83371443898 dbcheck: check_object/userparams: use variable for clarity
       via  22447a51598 dbcheck: reduce useless use of str(attrname)
       via  0dd736ff334 dbcheck: better disambiguate 'attrs'
       via  206a028e1e8 dbcheck: split out attr calculations from check_object()
       via  43530f087de dbcheck: add a helper function for attr tracking
       via  b21287c2f49 dbcheck: do not add duplicate attrs for checking
       via  6b4ff458055 dbcheck: check_object() caches of lower case attr names
       via  25a94fa4743 dbcheck: make rIDSetReferences attr check case-insensitve
      from  6601b3ac544 bootstrap: Update distro list in README.md

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit 7a077f152aa3e1e09b4228ae24780fa613540a68
Author: Andrew Walker <awalker at ixsystems.com>
Date:   Tue Dec 8 10:36:10 2020 -0500

    s3:smbd:trans2.c - add twrp to tmp smb_fname in smbd_do_qfsinfo
    
    Preserve VSS-related timestamp in temporary smb_filename before
    calling vfs_stat_fn() in smbd_do_qfsinfo. Otherwise, we can fail
    here on smb2_getinfo requests if file does not exist outside of
    shadow copy path.
    
    Signed-off-by: Andrew Walker <awalker at ixsystems.com>
    Reviewed-by: Ralph Boehme <slow at samba.org>
    Reviewed-by: Noel Power <npower at samba.org>
    
    Autobuild-User(master): Noel Power <npower at samba.org>
    Autobuild-Date(master): Tue Dec 15 15:32:18 UTC 2020 on sn-devel-184

commit 9bf9b998088b181955994da2004bf459ef131037
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Dec 9 11:50:37 2020 +1300

    dbcheck: clarify check_object userparams
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 83371443898aacdc6033ccf20bc67d4033b96767
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Dec 9 11:38:48 2020 +1300

    dbcheck: check_object/userparams: use variable for clarity
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 22447a5159867729559ae47c6fc20159be8f4c25
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Dec 9 11:37:48 2020 +1300

    dbcheck: reduce useless use of str(attrname)
    
    it's already a string!
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 0dd736ff3343ce19f2fef3132285d4595aa54afb
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Dec 9 11:34:50 2020 +1300

    dbcheck: better disambiguate 'attrs'
    
    We had too many things called 'attrs'; now we have just one, but we
    don't want it to look like it is *the* one.
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 206a028e1e8c83fa1fc2a7b3bbe4a573eb315ff8
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Dec 4 13:17:24 2020 +1300

    dbcheck: split out attr calculations from check_object()
    
    check_object is too long!
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 43530f087deab648708508445d4dea160de46889
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Dec 4 13:10:49 2020 +1300

    dbcheck: add a helper function for attr tracking
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit b21287c2f4922abb76ead510680bd489c6cf3cf9
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Dec 4 13:06:25 2020 +1300

    dbcheck: do not add duplicate attrs for checking
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 6b4ff45805550216a47577a2ee2fa04be305fe61
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Dec 4 12:57:57 2020 +1300

    dbcheck: check_object() caches of lower case attr names
    
    The construct `'name' in map(str.lower, attrs)` is doubly inefficient,
    because not only is it running the lower() function too often, it is
    searching linearly in a temporary iterator for membership.
    
    So we make a set, and use that.
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

commit 25a94fa4743f4bda9924786775aba43bb8c58c0d
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Dec 4 13:56:56 2020 +1300

    dbcheck: make rIDSetReferences attr check case-insensitve
    
    Yes, it looks inefficient, but that's because it is just trying to fit
    in. Very soon we will fix it it properly.
    
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Noel Power <npower at samba.org>

-----------------------------------------------------------------------

Summary of changes:
 python/samba/dbchecker.py | 127 +++++++++++++++++++++++++++++-----------------
 source3/smbd/trans2.c     |   7 ++-
 2 files changed, 86 insertions(+), 48 deletions(-)


Changeset truncated at 500 lines:

diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 28b56edaafb..364dc9427d7 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -250,7 +250,7 @@ class dbcheck(object):
 
         for object in res:
             self.dn_set.add(str(object.dn))
-            error_count += self.check_object(object.dn, attrs=attrs)
+            error_count += self.check_object(object.dn, requested_attrs=attrs)
 
         if DN is None:
             error_count += self.check_rootdse()
@@ -2217,27 +2217,38 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
         raise KeyError
 
-    def check_object(self, dn, attrs=None):
-        '''check one object'''
-        if self.verbose:
-            self.report("Checking object %s" % dn)
-        if attrs is None:
+    def find_checkable_attrs(self, dn, requested_attrs):
+        """A helper function for check_object() that calculates the list of
+        attributes that need to be checked, and returns that as a list
+        in the original case, and a set normalised to lowercase (for
+        easy existence checks).
+        """
+        if requested_attrs is None:
             attrs = ['*']
         else:
-            # make a local copy to modify
-            attrs = list(attrs)
-        if "dn" in map(str.lower, attrs):
-            attrs.append("name")
-        if "distinguishedname" in map(str.lower, attrs):
-            attrs.append("name")
-        if str(dn.get_rdn_name()).lower() in map(str.lower, attrs):
+            attrs = list(requested_attrs)
+
+        lc_attrs = set(x.lower() for x in attrs)
+
+        def add_attr(a):
+            if a.lower() not in lc_attrs:
+                attrs.append(a)
+                lc_attrs.add(a.lower())
+
+        if ("dn" in lc_attrs or
+            "distinguishedname" in lc_attrs or
+            dn.get_rdn_name().lower() in lc_attrs):
             attrs.append("name")
-        if 'name' in map(str.lower, attrs):
-            attrs.append(dn.get_rdn_name())
-            attrs.append("isDeleted")
-            attrs.append("systemFlags")
+            lc_attrs.add('name')
+
+        if 'name' in lc_attrs:
+            for a in (dn.get_rdn_name(),
+                      "isDeleted",
+                      "systemFlags"):
+                add_attr(a)
+
         need_replPropertyMetaData = False
-        if '*' in attrs:
+        if '*' in lc_attrs:
             need_replPropertyMetaData = True
         else:
             for a in attrs:
@@ -2249,8 +2260,20 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                 need_replPropertyMetaData = True
                 break
         if need_replPropertyMetaData:
-            attrs.append("replPropertyMetaData")
-        attrs.append("objectGUID")
+            add_attr("replPropertyMetaData")
+
+        add_attr("objectGUID")
+
+        return attrs, lc_attrs
+
+    def check_object(self, dn, requested_attrs=None):
+        '''check one object'''
+        if self.verbose:
+            self.report("Checking object %s" % dn)
+
+        # search attrs are used to find the attributes, lc_attrs are
+        # used for existence checks
+        search_attrs, lc_attrs = self.find_checkable_attrs(dn, requested_attrs)
 
         try:
             sd_flags = 0
@@ -2267,7 +2290,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                                         "sd_flags:1:%d" % sd_flags,
                                         "reveal_internals:0",
                                     ],
-                                    attrs=attrs)
+                                    attrs=search_attrs)
         except ldb.LdbError as e10:
             (enum, estr) = e10.args
             if enum == ldb.ERR_NO_SUCH_OBJECT:
@@ -2302,14 +2325,14 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         repl_meta_data_val = None
 
         for attrname in obj:
-            if str(attrname).lower() == 'isdeleted':
+            if attrname.lower() == 'isdeleted':
                 if str(obj[attrname][0]) != "FALSE":
                     isDeleted = True
 
-            if str(attrname).lower() == 'systemflags':
+            if attrname.lower() == 'systemflags':
                 systemFlags = int(obj[attrname][0])
 
-            if str(attrname).lower() == 'replpropertymetadata':
+            if attrname.lower() == 'replpropertymetadata':
                 repl_meta_data_val = obj[attrname][0]
 
         if isDeleted and repl_meta_data_val:
@@ -2324,10 +2347,10 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
             if attrname == 'dn' or attrname == "distinguishedName":
                 continue
 
-            if str(attrname).lower() == 'objectclass':
+            if attrname.lower() == 'objectclass':
                 got_objectclass = True
 
-            if str(attrname).lower() == "name":
+            if attrname.lower() == "name":
                 if len(obj[attrname]) != 1:
                     error_count += 1
                     self.report("ERROR: Not fixing num_values(%d) for '%s' on '%s'" %
@@ -2335,7 +2358,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                 else:
                     name_val = str(obj[attrname][0])
 
-            if str(attrname).lower() == str(obj.dn.get_rdn_name()).lower():
+            if attrname.lower() == str(obj.dn.get_rdn_name()).lower():
                 object_rdn_attr = attrname
                 if len(obj[attrname]) != 1:
                     error_count += 1
@@ -2344,7 +2367,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                 else:
                     object_rdn_val = str(obj[attrname][0])
 
-            if str(attrname).lower() == 'replpropertymetadata':
+            if attrname.lower() == 'replpropertymetadata':
                 if self.has_replmetadata_zero_invocationid(dn, obj[attrname][0]):
                     error_count += 1
                     self.err_replmetadata_zero_invocationid(dn, attrname, obj[attrname][0])
@@ -2375,7 +2398,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
                 continue
 
-            if str(attrname).lower() == 'ntsecuritydescriptor':
+            if attrname.lower() == 'ntsecuritydescriptor':
                 (sd, sd_broken) = self.process_sd(dn, obj)
                 if sd_broken is not None:
                     self.err_wrong_sd(dn, sd, sd_broken)
@@ -2403,7 +2426,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                         continue
                 continue
 
-            if str(attrname).lower() == 'objectclass':
+            if attrname.lower() == 'objectclass':
                 normalised = self.samdb.dsdb_normalise_attributes(self.samdb_schema, attrname, obj[attrname])
                 # Do not consider the attribute incorrect if:
                 #  - The sorted (alphabetically) list is the same, inclding case
@@ -2422,37 +2445,49 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                     error_count += 1
                 continue
 
-            if str(attrname).lower() == 'userparameters':
-                if len(obj[attrname][0]) == 1 and obj[attrname][0][0] == b'\x20'[0]:
+            if attrname.lower() == 'userparameters':
+                userparams = obj[attrname][0]
+                if userparams == b' ':
                     error_count += 1
                     self.err_short_userParameters(obj, attrname, obj[attrname])
                     continue
 
-                elif obj[attrname][0][:16] == b'\x20\x00\x20\x00\x20\x00\x20\x00\x20\x00\x20\x00\x20\x00\x20\x00':
+                elif userparams[:16] == b'\x20\x00' * 8:
                     # This is the correct, normal prefix
                     continue
 
-                elif obj[attrname][0][:20] == b'IAAgACAAIAAgACAAIAAg':
+                elif userparams[:20] == b'IAAgACAAIAAgACAAIAAg':
                     # this is the typical prefix from a windows migration
                     error_count += 1
                     self.err_base64_userParameters(obj, attrname, obj[attrname])
                     continue
 
                 #43:00:00:00:74:00:00:00:78
-                elif obj[attrname][0][1] != b'\x00'[0] and obj[attrname][0][3] != b'\x00'[0] and obj[attrname][0][5] != b'\x00'[0] and obj[attrname][0][7] != b'\x00'[0] and obj[attrname][0][9] != b'\x00'[0]:
-                    # This is a prefix that is not in UTF-16 format for the space or munged dialback prefix
+                elif (userparams[1] != 0 and
+                      userparams[3] != 0 and
+                      userparams[5] != 0 and
+                      userparams[7] != 0 and
+                      userparams[9] != 0):
+                    # This is a prefix that is not in UTF-16 format
+                    # for the space or munged dialback prefix
                     error_count += 1
                     self.err_utf8_userParameters(obj, attrname, obj[attrname])
                     continue
 
-                elif len(obj[attrname][0]) % 2 != 0:
+                elif len(userparams) % 2 != 0:
                     # This is a value that isn't even in length
                     error_count += 1
                     self.err_odd_userParameters(obj, attrname)
                     continue
 
-                elif obj[attrname][0][1] == b'\x00'[0] and obj[attrname][0][2] == b'\x00'[0] and obj[attrname][0][3] == b'\x00'[0] and obj[attrname][0][4] != b'\x00'[0] and obj[attrname][0][5] == b'\x00'[0]:
-                    # This is a prefix that would happen if a SAMR-written value was replicated from a Samba 4.1 server to a working server
+                elif (userparams[1] == 0 and
+                      userparams[2] == 0 and
+                      userparams[3] == 0 and
+                      userparams[4] != 0 and
+                      userparams[5] == 0):
+                    # This is a prefix that would happen if a
+                    # SAMR-written value was replicated from a Samba
+                    # 4.1 server to a working server
                     error_count += 1
                     self.err_doubled_userParameters(obj, attrname, obj[attrname])
                     continue
@@ -2487,7 +2522,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
             if (not flag & dsdb.DS_FLAG_ATTR_NOT_REPLICATED
                 and not flag & dsdb.DS_FLAG_ATTR_IS_CONSTRUCTED
                 and not linkID):
-                set_attrs_seen.add(str(attrname).lower())
+                set_attrs_seen.add(attrname.lower())
 
             if syntax_oid in [dsdb.DSDB_SYNTAX_BINARY_DN, dsdb.DSDB_SYNTAX_OR_NAME,
                               dsdb.DSDB_SYNTAX_STRING_DN, ldb.SYNTAX_DN]:
@@ -2511,17 +2546,17 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                     error_count += 1
                     break
 
-            if str(attrname).lower() == "instancetype":
+            if attrname.lower() == "instancetype":
                 calculated_instancetype = self.calculate_instancetype(dn)
                 if len(obj["instanceType"]) != 1 or int(obj["instanceType"][0]) != calculated_instancetype:
                     error_count += 1
                     self.err_wrong_instancetype(obj, calculated_instancetype)
 
-        if not got_objectclass and ("*" in attrs or "objectclass" in map(str.lower, attrs)):
+        if not got_objectclass and ("*" in lc_attrs or "objectclass" in lc_attrs):
             error_count += 1
             self.err_missing_objectclass(dn)
 
-        if ("*" in attrs or "name" in map(str.lower, attrs)):
+        if ("*" in lc_attrs or "name" in lc_attrs):
             if name_val is None:
                 error_count += 1
                 self.report("ERROR: Not fixing missing 'name' on '%s'" % (str(obj.dn)))
@@ -2583,7 +2618,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                 self.fix_metadata(obj, att)
 
         if self.is_fsmo_role(dn):
-            if "fSMORoleOwner" not in obj and ("*" in attrs or "fsmoroleowner" in map(str.lower, attrs)):
+            if "fSMORoleOwner" not in obj and ("*" in lc_attrs or "fsmoroleowner" in lc_attrs):
                 self.err_no_fsmoRoleOwner(obj)
                 error_count += 1
 
@@ -2604,7 +2639,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
             else:
                 raise
 
-        if dn in self.deleted_objects_containers and '*' in attrs:
+        if dn in self.deleted_objects_containers and '*' in lc_attrs:
             if self.is_deleted_deleted_objects(obj):
                 self.err_deleted_deleted_objects(obj)
                 error_count += 1
@@ -2632,7 +2667,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
         if dn == self.server_ref_dn:
             # Check we have a valid RID Set
-            if "*" in attrs or "rIDSetReferences" in attrs:
+            if "*" in lc_attrs or "ridsetreferences" in lc_attrs:
                 if "rIDSetReferences" not in obj:
                     # NO RID SET reference
                     # We are RID master, allocate it.
diff --git a/source3/smbd/trans2.c b/source3/smbd/trans2.c
index f6e261ed658..5dac5f0f265 100644
--- a/source3/smbd/trans2.c
+++ b/source3/smbd/trans2.c
@@ -3616,8 +3616,11 @@ NTSTATUS smbd_do_qfsinfo(struct smbXsrv_connection *xconn,
 
 	DEBUG(3,("smbd_do_qfsinfo: level = %d\n", info_level));
 
-	ZERO_STRUCT(smb_fname);
-	smb_fname.base_name = discard_const_p(char, filename);
+	smb_fname = (struct smb_filename) {
+		.base_name = discard_const_p(char, filename),
+		.flags = fname ? fname->flags : 0,
+		.twrp = fname ? fname->twrp : 0,
+	};
 
 	if(info_level != SMB_FS_QUOTA_INFORMATION
 	   && SMB_VFS_STAT(conn, &smb_fname) != 0) {


-- 
Samba Shared Repository



More information about the samba-cvs mailing list