[SCM] Samba Shared Repository - branch v4-10-test updated

Karolin Seeger kseeger at samba.org
Tue Apr 2 13:53:03 UTC 2019


The branch, v4-10-test has been updated
       via  af05bf7911e libcli: permit larger values of DataLength in SMB2_ENCRYPTION_CAPABILITIES of negotiate response
       via  815be52b600 lib: Make fd_load work for non-regular files
       via  e769bd66089 s4/messaging: Fix undefined reference in linking libMESSAGING-samba4.so
       via  0e94b0a3144 s4:dlz make b9_has_soa check dc=@ node
       via  7921aa6365b dlz: Add test to ensure there are writable zones
       via  78b72ff339d regfio tests: Update comment style to match README.Coding
       via  722c5b32548 regfio: Update code near recent changes to match README.Coding
       via  d4ef858ec11 regfio: Improve handling of malformed registry hive files
       via  bf6a8517820 regfio: Add trivial unit test
       via  cfffac0fc9f regfio: Use correct function names in debug information
       via  3f278c3f911 Fix typos in "valid"
       via  76b38e19d3c py/logger: use python 2.6 compatible arguments
       via  f52ebe258ba py/uptodateness: use 2.6 compatible dictionary construction
       via  9b8398ecbbd py/kcc_utils: py2.6 compatibility
       via  75b6e02a8e6 py/graph: use 2.6 compatible check for set membership
       via  868356cf365 acl_read: Fix regression caused by db15fcfa899e1fe4d6994f68ceb299921b8aa6f1 for empty lists
       via  286b80cb7a3 ldb: cmocka test for empty attributes bug
       via  10a390e8975 dbcheck: use the str() value of the "name" attribute
       via  6602a77b649 dbcheck: don't check expired tombstone objects by default anymore
       via  4b658a5a396 blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
       via  40b6af9c000 blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
       via  b292ef1d9f6 dbcheck: add --selftest-check-expired-tombstones cmdline option
       via  178fad24f2c python/samba/netcmd: provide SUPPRESS_HELP via Option class
       via  89fb9d0a81b dbcheck: detect the change after deletion bug
       via  4f0b554b955 blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
       via  caf0caba4e4 dbcheck: add find_repl_attid() helper function
       via  a47b27b2c96 dbcheck: don't remove dangling one-way links on already deleted objects
       via  0c2f7224e07 dbcheck: don't move already deleted objects to LostAndFound
       via  95f5b9f246a dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
       via  8736fb5eb4e dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
       via  3e539f756ac dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
       via  4fc17804088 blackbox/dbcheck-links.sh: reproduce lost deleted object problem
       via  8d1241dac6b blackbox/*.sh: pass -u to 'diff'
       via  18f4167198a selftest: force running with TZ=UTC
       via  2d4820f0a8e s3:waf: Fix the detection of makdev() macro on Linux
       via  a21e9754c74 s3:tests: Add test for smbstatus and smbstatus --resolve_uids
       via  79d3de4de41 selftest: Add smbstatus to testhelper
       via  b866bdbe4fa s3:utils: Add 'smbstatus -L --resolve-uids' to show usernames
       via  084d2f1bc4f s3:utils: Use C99 initializer for poptOption in smbstatus
      from  c8e8d97959d s3:lib: Fix the debug message for adding cache entries.

https://git.samba.org/?p=samba.git;a=shortlog;h=v4-10-test


- Log -----------------------------------------------------------------
commit af05bf7911eed71315843c5333cc5bb6b6a06ec3
Author: Philipp Gesang <philipp.gesang at intra2net.com>
Date:   Thu Feb 14 10:17:28 2019 +0100

    libcli: permit larger values of DataLength in SMB2_ENCRYPTION_CAPABILITIES of negotiate response
    
    Certain Netapp versions are sending SMB2_ENCRYPTION_CAPABILITIES
    structures containing DataLength field that includes the padding
    [0]. Microsoft has since clarified that only values smaller than
    the size are considered invalid [1].
    
    While parsing the NegotiateContext it is ensured that DataLength
    does not exceed the message bounds. Also, the value is not
    actually used anywhere outside the validation. Thus values
    greater than the actual data size are safe to use. This patch
    makes Samba fail only on values that are too small for the (fixed
    size) payload.
    
    [0] https://lists.samba.org/archive/samba/2019-February/221139.html
    [1] https://lists.samba.org/archive/cifs-protocol/2019-March/003210.html
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13869
    
    Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
    Reviewed-by: Ralph Böhme <slow at samba.org>
    Reviewed-by: Jeremy Allison <jra at samba.org>
    
    Autobuild-User(master): Jeremy Allison <jra at samba.org>
    Autobuild-Date(master): Sun Mar 31 01:11:09 UTC 2019 on sn-devel-144
    
    (cherry picked from commit 865b7b0c7d2ba7fa0a045586d1e83a72028a0864)
    
    Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
    Autobuild-Date(v4-10-test): Tue Apr  2 13:52:02 UTC 2019 on sn-devel-144

commit 815be52b60026f9101a66ed8b276a581ec83df7d
Author: Volker Lendecke <vl at samba.org>
Date:   Thu Feb 21 18:37:08 2019 +0100

    lib: Make fd_load work for non-regular files
    
    Follow-up to
    
    https://lists.samba.org/archive/samba/2018-September/217992.html
    
    and following. This also fixes a small and very theoretical race: Between the
    fstat and the read call the file size might change. This would make us fail on
    potentially legitimate files.
    
    This is more complex and probably slower, but looking at the use cases I don't
    think the speed matters.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13859
    
    Signed-off-by: Volker Lendecke <vl at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Tue Mar 26 04:43:40 UTC 2019 on sn-devel-144
    
    (cherry picked from commit ac487bf4d04c9771ada1ca7eeb9dac4e5fe34185)

commit e769bd6608951e80bcc3fac7b4808194f22bbd36
Author: Anoop C S <anoopcs at redhat.com>
Date:   Thu Dec 27 18:19:42 2018 +0530

    s4/messaging: Fix undefined reference in linking libMESSAGING-samba4.so
    
    Early check for DEVELOPER or ENABLE_SELFTEST configure options inside
    messaging_handlers.c leaves us with the following undefined reference
    linkage error:
    
    [1315/3712] Linking bin/default/source4/lib/messaging/libMESSAGING-samba4.so
    /usr/bin/ld: source4/lib/messaging/messaging.c.4.o: in function
    						`imessaging_init_internal':
    /root/samba.git/bin/default/../../source4/lib/messaging/messaging.c:472:
    		undefined reference to `imessaging_register_extra_handlers'
    collect2: error: ld returned 1 exit status
    
    This happened due to failure in including "includes.h" before checking
    the above mentioned configure options.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13854
    
    Signed-off-by: Anoop C S <anoopcs at redhat.com>
    Reviewed-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 08ba013a2b8b2cf9fc17fdcb3d107e1434709036)

commit 0e94b0a3144385e551d25ecb55cda2cb51fe398f
Author: Michael Saxl <mike at mwsys.mine.bz>
Date:   Thu Mar 21 18:22:38 2019 +0100

    s4:dlz make b9_has_soa check dc=@ node
    
    the zone node does not hold the dnsRecord values, so for the zone level
    the node dc=@,dc=zonename has to be queried
    
    regression introduced with 28e2a518ff32, BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13841
    
    Signed-off-by: Michael Saxl <mike at mwsys.mine.bz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 526c6d0be53d97beb38f82a3619d8710fefb4091)

commit 7921aa6365ba8128f47dfa136a1f7945feec6c6f
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Thu Mar 21 16:12:26 2019 +1300

    dlz: Add test to ensure there are writable zones
    
    This is currently broken since 28e2a518ff32
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13841
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 6c62e05466917d9454d67eb2dd77e303e612c8a9)

commit 78b72ff339da26769ec8b5915eb145bdaeea42fb
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Wed Mar 20 17:33:46 2019 +1300

    regfio tests: Update comment style to match README.Coding
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit 68c0fc4335d0c3c526a38481538a33290be6d58a)

commit 722c5b3254818ae3a033de0d702196795f47d7e9
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Wed Mar 20 17:32:39 2019 +1300

    regfio: Update code near recent changes to match README.Coding
    
    This file long predates our current code conventions.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit acbf103fcaa4150a57bfbab2450e36b5b39e399b)

commit d4ef858ec112149f5ab5504d6367aa60d52446f5
Author: Michael Hanselmann <public at hansmi.ch>
Date:   Sun Mar 17 13:49:20 2019 +0100

    regfio: Improve handling of malformed registry hive files
    
    * next_record: A malformed file can lead to an endless loop.
    * regfio_rootkey: Supplying a malformed registry hive file to the
      registry hive I/O code can lead to out-of-bounds reads.
    
    Test cases are included. Both issues resolved have been identified using
    AddressSanitizer.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Michael Hanselmann <public at hansmi.ch>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 601afd690346087fbd53819dba9b1afa81560064)

commit bf6a85178205b777274136cd2c2ace62a8f1a17d
Author: Michael Hanselmann <public at hansmi.ch>
Date:   Tue Mar 19 00:47:52 2019 +0100

    regfio: Add trivial unit test
    
    An upcoming commit will resolve two cases of insufficient handling of
    mangled registry hive files and will include unit tests.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Michael Hanselmann <public at hansmi.ch>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 9b2cb845b23cd1c91ab3b5ea8ad791b18b3ab733)

commit cfffac0fc9f1e6b40ad9bb18b12590ba1e6eabda
Author: Michael Hanselmann <public at hansmi.ch>
Date:   Sun Mar 17 16:20:47 2019 +0100

    regfio: Use correct function names in debug information
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Michael Hanselmann <public at hansmi.ch>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit aa6b355858a0d8b77bf49384e5329642add1a5ff)

commit 3f278c3f91101dfc46057d5da499dad6d65f3e22
Author: Michael Hanselmann <public at hansmi.ch>
Date:   Sun Mar 17 13:04:52 2019 +0100

    Fix typos in "valid"
    
    s/vald/valid/
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840
    
    Signed-off-by: Michael Hanselmann <public at hansmi.ch>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 305346d360d3c13fd315c1af27b037f46fd10650)

commit 76b38e19d3c922662fc3a4df6e334243ada8a9e0
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Mar 27 09:47:56 2019 +1300

    py/logger: use python 2.6 compatible arguments
    
    In 2.6 stream is a positional argument; 2.7+ it is also a keyword
    argument.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13837
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit f52ebe258ba5566e52cd6180343d0847c69cac04
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Mar 20 12:12:34 2019 +1300

    py/uptodateness: use 2.6 compatible dictionary construction
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13837
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 9b8398ecbbd8abafd7621b5e334bced7b3216e41
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Mar 22 15:24:47 2019 +1300

    py/kcc_utils: py2.6 compatibility
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13837
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 75b6e02a8e62dc65d4dfbf3be1e7a15329c08b80
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Mar 20 12:02:09 2019 +1300

    py/graph: use 2.6 compatible check for set membership
    
    It is better this way anyway.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13837
    Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Wed Mar 20 06:36:05 UTC 2019 on sn-devel-144
    
    (cherry picked from commit c0aca17a4c9ec06f0127d5c972f3fa979a87a77f)

commit 868356cf36566636cfce5ee28c1a38055dd38ce8
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Mar 13 10:52:19 2019 +1300

    acl_read: Fix regression caused by db15fcfa899e1fe4d6994f68ceb299921b8aa6f1 for empty lists
    
    The original code never dereferenced attrs and only added "*" if attrs
    was NULL (not if attrs[0] was NULL).
    
    This causes significant performance issues with the new paged_results
    module introduced for 4.10 as the initial GUID search requests no
    attributes. This GUID search turns into a search for "*" and ends up
    allocating memory for the entire database.
    
    This never appears to cause changes in the final result set, only
    intermediate processing.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13836
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Fri Mar 29 18:37:29 UTC 2019 on sn-devel-144
    
    (cherry picked from commit a2b1970a37836e46d6c9eb6bda9bd20185de96ce)

commit 286b80cb7a3b7da8bb381786521931c14fed3383
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date:   Mon Mar 25 13:13:33 2019 +1300

    ldb: cmocka test for empty attributes bug
    
    Cmocka test exposing LDB bug where a request with an empty attributes
    list returns a response containing all attributes.  The bug is in the
    ACL module and will be fixed in the next commit.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13836
    
    Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 24efa3ca5399d5cf538c3be504014a954685f1ed)

commit 10a390e89753bc2180a9326cd142763dedc55825
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 19 13:05:16 2019 +0100

    dbcheck: use the str() value of the "name" attribute
    
    We do the same with the rdn attribute value
    and we need the same logic on both in order to
    check they are the same.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Noel Power <npower at samba.org>
    (cherry picked from commit dd6f0dad218ec1d5aa38ea8aa6848ec81035cb3f)

commit 6602a77b649faae38cce43b98f5f142120916597
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 11:41:01 2019 +0100

    dbcheck: don't check expired tombstone objects by default anymore
    
    These will be removed anyway and any change on them risks to
    be an originating update that causes replication problems.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Thu Mar 14 03:12:27 UTC 2019 on sn-devel-144
    
    (cherry picked from commit a2c5f8cf41c2dfdc4f122e8427d1dfeabb6ba311)

commit 4b658a5a396bf88cad6948bb83819ea32de3daab
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 11:38:22 2019 +0100

    blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit b096a3117ed9249fd6f65f3221a26c88efbba3b8)

commit 40b6af9c000db6c83dbd44a78911e24b08344170
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 11:04:33 2019 +0100

    blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
    
    These tests operate on provision dumps created long ago, they still
    want to run tests on deleted objects, when the next commits remove
    processing expired tombstone objects in dbcheck.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 5fccc4e9044d2e57be33471f5e6b9be7cc37ac3a)

commit b292ef1d9f6c3e7006a7b47e9c836c745cb825db
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 11:02:18 2019 +0100

    dbcheck: add --selftest-check-expired-tombstones cmdline option
    
    This will be used by dbcheck tests which operate on static/old provision
    dumps in the following commits.
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 6f9c5ed8de47bb98e21e8064d8e90f963f2f71ca)

commit 178fad24f2c59f10ba07aa23182e586bf4461fa0
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 10:25:40 2019 +0100

    python/samba/netcmd: provide SUPPRESS_HELP via Option class
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit b61d580fb7dba8ff94e9e98c958e324865cd2f1d)

commit 89fb9d0a81bbb736f74e2cc19c578ae2028f7d09
Author: Stefan Metzmacher <metze at samba.org>
Date:   Thu Feb 28 18:22:18 2019 +0100

    dbcheck: detect the change after deletion bug
    
    Old versions of 'samba-tool dbcheck' could reanimate
    deleted objects, when running at the same time as the
    tombstone garbage collection.
    
    When the (deleted) parent of a deleted object
    (with the DISALLOW_MOVE_ON_DELETE bit in systemFlags),
    is removed before the object itself, dbcheck moved
    it in the LostAndFound[Config] subtree of the partition
    as an originating change. That means that the object
    will be in tombstone state again for 180 days on the local
    DC. And other DCs fail to replicate the object as
    it's already removed completely there and the replication
    only gives the name and lastKnownParent attributes, because
    all other attributes should already be known to the other DC.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit a1658b306d85452407388b91a745078c9c1f7dc7)

commit 4f0b554b9552a326fe07a69fdd98aa8e5c826e79
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Mar 11 23:14:02 2019 +0100

    blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 1ccc21a34d295be3bb2ab481a5918003eae88bf4)

commit caf0caba4e42657abd8f908d7fb201fa20726cb1
Author: Stefan Metzmacher <metze at samba.org>
Date:   Thu Feb 28 18:16:27 2019 +0100

    dbcheck: add find_repl_attid() helper function
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 598e38d2a5e0832429ba65b4e55bf7127618f894)

commit a47b27b2c96a7823f5283b3a523c8eed71653e02
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Feb 25 15:35:22 2019 +0100

    dbcheck: don't remove dangling one-way links on already deleted objects
    
    This would typically happen when the garbage collection
    removed a parent object before a child object (both with
    the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
    while dbcheck is running at the same time as the garbage collection.
    In this case the lastKnownParent attributes points a non existing
    object.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit e388e599495b6d7c38b8b6966332e27f8b958783)

commit 0c2f7224e075538fc0fd70e4405cafc82f037909
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Feb 25 15:35:22 2019 +0100

    dbcheck: don't move already deleted objects to LostAndFound
    
    This would typically happen when the garbage collection
    removed a parent object before a child object (both with
    the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
    while dbcheck is running at the same time as the garbage collection.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 6d50ee74920c39cdb18b427bfaaf200775bf2d73)

commit 95f5b9f246a46e5233108251a4122809aa0f1220
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Feb 25 15:09:36 2019 +0100

    dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 9afcd5331ce567bd80d35175f8e4e21c506e9347)

commit 8736fb5eb4e4bf2afcd52451f5250a47e515201c
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Mar 11 22:45:46 2019 +0100

    dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
    
    We should never do originating updates on deleted objects.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 07a8326746f0c444eedf3860b178fc29d84e8d16)

commit 3e539f756ac0d59e5b62ae752ec825900903fc10
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Mar 11 22:38:38 2019 +0100

    dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
    
    We need a way to rename an object without updating the replication meta
    data.
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 3e8a435d27da899d0e3dab7cbc0a1c738067eba3)

commit 4fc1780408803ca77b4c5544b7e0503a7f4aba6d
Author: Stefan Metzmacher <metze at samba.org>
Date:   Mon Mar 11 14:52:57 2019 +0100

    blackbox/dbcheck-links.sh: reproduce lost deleted object problem
    
    When a parent object is removed during the tombstone garbage collection
    before a child object and samba-tool dbcheck runs at the same time, the
    following can happen:
    
    - If the object child had DISALLOW_MOVE_ON_DELETE in systemFlags,
      samba-tool dbcheck moves the object under the LostAndFound[Config]
      object (as an originating update!)
    - The lastKnownParent attribute is removed (as an originating update!)
    
    These originating updates cause the object to have an extended time
    as tombstone. And these changes are replicated to other DCs,
    which very likely already removed the object completely!
    
    This means the destination DC of replication has no chance to handle
    the object it gets from the source DC with just 2 attributes (name, lastKnownParent).
    
    The destination logs something like:
    
      No objectClass found in replPropertyMetaData
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 5357f591accffbf8c62335c308b985811b66f0b5)

commit 8d1241dac6bdfec953f3778ff006c85add85bd2b
Author: Stefan Metzmacher <metze at samba.org>
Date:   Tue Mar 12 10:36:49 2019 +0100

    blackbox/*.sh: pass -u to 'diff'
    
    This is what we work with every day...
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 8ba6f1c895ee9b6b592578f21e7f79ed36236bef)

commit 18f4167198a666876dace140c33bb19f3eeacb9e
Author: Stefan Metzmacher <metze at samba.org>
Date:   Wed Feb 27 08:22:09 2019 +0100

    selftest: force running with TZ=UTC
    
    Signed-off-by: Stefan Metzmacher <metze at samba.org>
    Reviewed-by: Andreas Schneider <asn at samba.org>
    
    Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
    Autobuild-Date(master): Wed Feb 27 11:24:59 UTC 2019 on sn-devel-144
    
    (cherry picked from commit 4f307f2302b0fe8fd0fc6379eb8e6491faf8520c)

commit 2d4820f0a8eb480f02576bed167da230f0e3824a
Author: Andreas Schneider <asn at samba.org>
Date:   Thu Mar 21 11:55:46 2019 +0100

    s3:waf: Fix the detection of makdev() macro on Linux
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13853
    
    Signed-off-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Jeremy Allison <jra at samba.org>
    (cherry picked from commit eace58b539a382c61edd7c2be6fdfab31114719f)

commit a21e9754c74c92d47ce805c0b747a9da2c043cec
Author: Andreas Schneider <asn at samba.org>
Date:   Wed Mar 13 12:00:27 2019 +0100

    s3:tests: Add test for smbstatus and smbstatus --resolve_uids
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13793
    
    Signed-off-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Wed Mar 27 14:33:35 UTC 2019 on sn-devel-144
    
    (cherry picked from commit 2e7f4b1d3701b9da32e03dcee1095711945f22b8)

commit 79d3de4de41f6cce09917f071057d54cfaa28a40
Author: Andreas Schneider <asn at samba.org>
Date:   Wed Mar 13 12:00:11 2019 +0100

    selftest: Add smbstatus to testhelper
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13793
    
    Signed-off-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit 6106b8a053e06699c332fd1a6d7636f550422cc7)

commit b866bdbe4fa80a17887701ac62c1b955973c9145
Author: Andreas Schneider <asn at samba.org>
Date:   Mon Feb 18 14:11:32 2019 +0100

    s3:utils: Add 'smbstatus -L --resolve-uids' to show usernames
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13793
    
    Signed-off-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    (cherry picked from commit ffad76ae260ac33926db87c61aede5b66d9b09e6)

commit 084d2f1bc4fb5ee82706bb2960ba9ab20e2c4e87
Author: Andreas Schneider <asn at samba.org>
Date:   Tue Jan 8 12:21:36 2019 +0100

    s3:utils: Use C99 initializer for poptOption in smbstatus
    
    Signed-off-by: Andreas Schneider <asn at samba.org>
    Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
    (cherry picked from commit 83d25ca9d90897925a4431dd55e68c78244900b3)

-----------------------------------------------------------------------

Summary of changes:
 lib/util/util_file.c                               |  63 +++-
 libcli/smb/smbXcli_base.c                          |   2 +-
 python/samba/dbchecker.py                          | 228 +++++++++++--
 python/samba/graph.py                              |   2 +-
 python/samba/kcc/kcc_utils.py                      |   4 +-
 python/samba/logger.py                             |   2 +-
 python/samba/netcmd/__init__.py                    |   1 +
 python/samba/netcmd/dbcheck.py                     |  13 +-
 python/samba/uptodateness.py                       |   2 +-
 selftest/selftest.pl                               |   3 +
 selftest/selftesthelpers.py                        |   1 +
 selftest/tests.py                                  |   2 +
 source3/registry/regfio.c                          |  29 +-
 source3/registry/tests/test_regfio.c               | 184 +++++++++++
 source3/script/tests/test_smbstatus.sh             | 155 +++++++++
 source3/selftest/tests.py                          |   6 +-
 source3/torture/torture.c                          |   2 +-
 source3/utils/status.c                             | 135 +++++++-
 source3/wscript                                    |   3 +
 source3/wscript_build                              |   6 +
 source4/dns_server/dlz_bind9.c                     |  11 +
 source4/dsdb/common/tests/dsdb.c                   |  93 ++++++
 source4/dsdb/samdb/ldb_modules/acl_read.c          |   3 -
 source4/dsdb/samdb/ldb_modules/repl_meta_data.c    |   7 +
 source4/lib/messaging/messaging_handlers.c         |   4 +-
 ...cted-dbcheck-link-output-lost-deleted-user1.txt |   9 +
 ...cted-dbcheck-link-output-lost-deleted-user2.txt |   8 +
 ...cted-dbcheck-link-output-lost-deleted-user3.txt |  19 ++
 source4/selftest/tests.py                          |   3 +
 source4/torture/dns/dlz_bind9.c                    |   7 +
 source4/torture/wscript_build                      |   9 +
 testdata/samba3/regfio_corrupt_hbin1.dat           | Bin 0 -> 5120 bytes
 testdata/samba3/regfio_corrupt_lf_subkeys.dat      | Bin 0 -> 5120 bytes
 testprogs/blackbox/dbcheck-links.sh                | 356 ++++++++++++++++++++-
 testprogs/blackbox/dbcheck-oldrelease.sh           |  42 +--
 testprogs/blackbox/tombstones-expunge.sh           |  14 +-
 36 files changed, 1308 insertions(+), 120 deletions(-)
 create mode 100644 source3/registry/tests/test_regfio.c
 create mode 100755 source3/script/tests/test_smbstatus.sh
 create mode 100644 source4/dsdb/common/tests/dsdb.c
 create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user1.txt
 create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user2.txt
 create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user3.txt
 create mode 100644 testdata/samba3/regfio_corrupt_hbin1.dat
 create mode 100644 testdata/samba3/regfio_corrupt_lf_subkeys.dat


Changeset truncated at 500 lines:

diff --git a/lib/util/util_file.c b/lib/util/util_file.c
index 926eda240f6..29c0be91b6b 100644
--- a/lib/util/util_file.c
+++ b/lib/util/util_file.c
@@ -168,30 +168,63 @@ load a file into memory from a fd.
 **/
 _PUBLIC_ char *fd_load(int fd, size_t *psize, size_t maxsize, TALLOC_CTX *mem_ctx)
 {
-	struct stat sbuf;
-	char *p;
-	size_t size;
+	FILE *file;
+	char *p = NULL;
+	size_t size = 0;
+	size_t chunk = 1024;
+	int err;
+
+	if (maxsize == 0) {
+		maxsize = SIZE_MAX;
+	}
 
-	if (fstat(fd, &sbuf) != 0) return NULL;
+	file = fdopen(fd, "r");
+	if (file == NULL) {
+		return NULL;
+	}
 
-	size = sbuf.st_size;
+	while (size < maxsize) {
+		size_t newbufsize;
+		size_t nread;
 
-	if (maxsize) {
-		size = MIN(size, maxsize);
-	}
+		chunk = MIN(chunk, (maxsize - size));
 
-	p = (char *)talloc_size(mem_ctx, size+1);
-	if (!p) return NULL;
+		newbufsize = size + (chunk+1); /* chunk+1 can't overflow */
+		if (newbufsize < size) {
+			goto fail; /* overflow */
+		}
 
-	if (read(fd, p, size) != size) {
-		talloc_free(p);
-		return NULL;
+		p = talloc_realloc(mem_ctx, p, char, newbufsize);
+		if (p == NULL) {
+			goto fail;
+		}
+
+		nread = fread(p+size, 1, chunk, file);
+		size += nread;
+
+		if (nread != chunk) {
+			break;
+		}
 	}
-	p[size] = 0;
 
-	if (psize) *psize = size;
+	err = ferror(file);
+	if (err != 0) {
+		goto fail;
+	}
 
+	p[size] = '\0';
+
+	if (psize != NULL) {
+		*psize = size;
+	}
+
+	fclose(file);
 	return p;
+
+fail:
+	TALLOC_FREE(p);
+	fclose(file);
+	return NULL;
 }
 
 /**
diff --git a/libcli/smb/smbXcli_base.c b/libcli/smb/smbXcli_base.c
index a237bf17d0a..a8c73be445a 100644
--- a/libcli/smb/smbXcli_base.c
+++ b/libcli/smb/smbXcli_base.c
@@ -5064,7 +5064,7 @@ static void smbXcli_negprot_smb2_done(struct tevent_req *subreq)
 			return;
 		}
 
-		if (cipher->data.length != (2 + 2 * cipher_count)) {
+		if (cipher->data.length < (2 + 2 * cipher_count)) {
 			tevent_req_nterror(req,
 					NT_STATUS_INVALID_NETWORK_RESPONSE);
 			return;
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index bf999ddaab9..d341983738a 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -60,7 +60,8 @@ class dbcheck(object):
 
     def __init__(self, samdb, samdb_schema=None, verbose=False, fix=False,
                  yes=False, quiet=False, in_transaction=False,
-                 reset_well_known_acls=False):
+                 reset_well_known_acls=False,
+                 check_expired_tombstones=False):
         self.samdb = samdb
         self.dict_oid_name = None
         self.samdb_schema = (samdb_schema or samdb)
@@ -107,6 +108,8 @@ class dbcheck(object):
         self.fix_doubled_userparameters = False
         self.fix_sid_rid_set_conflict = False
         self.reset_well_known_acls = reset_well_known_acls
+        self.check_expired_tombstones = check_expired_tombstones
+        self.expired_tombstones = 0
         self.reset_all_well_known_acls = False
         self.in_transaction = in_transaction
         self.infrastructure_dn = ldb.Dn(samdb, "CN=Infrastructure," + samdb.domain_dn())
@@ -120,6 +123,7 @@ class dbcheck(object):
         self.fix_missing_deleted_objects = False
         self.fix_replica_locations = False
         self.fix_missing_rid_set_master = False
+        self.fix_changes_after_deletion_bug = False
 
         self.dn_set = set()
         self.link_id_cache = {}
@@ -208,6 +212,14 @@ class dbcheck(object):
         else:
             self.rid_set_dn = None
 
+        ntds_service_dn = "CN=Directory Service,CN=Windows NT,CN=Services,%s" % \
+                          self.samdb.get_config_basedn().get_linearized()
+        res = samdb.search(base=ntds_service_dn,
+                           scope=ldb.SCOPE_BASE,
+                           expression="(objectClass=nTDSService)",
+                           attrs=["tombstoneLifetime"])
+        self.tombstoneLifetime = int(res[0]["tombstoneLifetime"][0])
+
         self.compatibleFeatures = []
         self.requiredFeatures = []
 
@@ -244,6 +256,13 @@ class dbcheck(object):
         if DN is None:
             error_count += self.check_rootdse()
 
+        if self.expired_tombstones > 0:
+            self.report("NOTICE: found %d expired tombstones, "
+                        "'samba' will remove them daily, "
+                        "'samba-tool domain tombstones expunge' "
+                        "would do that immediately." % (
+                        self.expired_tombstones))
+
         if error_count != 0 and not self.fix:
             self.report("Please use --fix to fix these errors")
 
@@ -569,6 +588,19 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
     def err_missing_target_dn_or_GUID(self, dn, attrname, val, dsdb_dn):
         """handle a missing target DN (if specified, GUID form can't be found,
         and otherwise DN string form can't be found)"""
+
+        # Don't change anything if the object itself is deleted
+        if str(dn).find('\\0ADEL') != -1:
+            # We don't bump the error count as Samba produces these
+            # in normal operation
+            self.report("WARNING: no target object found for GUID "
+                        "component link %s in deleted object "
+                        "%s - %s" % (attrname, dn, val))
+            self.report("Not removing dangling one-way "
+                        "link on deleted object "
+                        "(tombstone garbage collection in progress?)")
+            return 0
+
         # check if its a backlink
         linkID, _ = self.get_attr_linkID_and_reverse_name(attrname)
         if (linkID & 1 == 0) and str(dsdb_dn).find('\\0ADEL') == -1:
@@ -878,7 +910,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         else:
             self.samdb.transaction_cancel()
 
-    def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val):
+    def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val, controls):
         '''handle a wrong dn'''
 
         new_rdn = ldb.Dn(self.samdb, str(new_dn))
@@ -895,7 +927,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
             self.report("Not renaming %s to %s" % (obj.dn, new_dn))
             return
 
-        if self.do_rename(obj.dn, new_rdn, new_parent, ["show_recycled:1", "relax:0"],
+        if self.do_rename(obj.dn, new_rdn, new_parent, controls,
                           "Failed to rename object %s into %s" % (obj.dn, new_dn)):
             self.report("Renamed %s into %s" % (obj.dn, new_dn))
 
@@ -1478,6 +1510,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
         return error_count
 
+    def find_repl_attid(self, repl, attid):
+        for o in repl.ctr.array:
+            if o.attid == attid:
+                return o
+
+        return None
+
     def get_originating_time(self, val, attid):
         '''Read metadata properties and return the originating time for
            a given attributeId.
@@ -1486,11 +1525,9 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         '''
 
         repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, val)
-
-        for o in repl.ctr.array:
-            if o.attid == attid:
-                return o.originating_change_time
-
+        o = self.find_repl_attid(repl, attid)
+        if o is not None:
+            return o.originating_change_time
         return 0
 
     def process_metadata(self, dn, val):
@@ -1740,6 +1777,132 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
             self.report("Fixed attribute '%s' of '%s'\n" % (sd_attr, dn))
         self.samdb.set_session_info(self.system_session_info)
 
+    def is_expired_tombstone(self, dn, repl_val):
+        if self.check_expired_tombstones:
+            # This is not the default, it's just
+            # used to keep dbcheck tests work with
+            # old static provision dumps
+            return False
+
+        repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+        isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+        delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+        current_time = time.time()
+
+        tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+        delta = current_time - delete_time
+        if delta <= tombstone_delta:
+            return False
+
+        self.report("SKIPING: object %s is an expired tombstone" % dn)
+        self.report("isDeleted: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+                    isDeleted.attid,
+                    isDeleted.version,
+                    isDeleted.originating_invocation_id,
+                    isDeleted.originating_usn,
+                    isDeleted.local_usn,
+                    time.ctime(samba.nttime2unix(isDeleted.originating_change_time))))
+        self.expired_tombstones += 1
+        return True
+
+    def find_changes_after_deletion(self, repl_val):
+        repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+        isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+        delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+
+        tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+        found = []
+        for o in repl.ctr.array:
+            if o.attid == drsuapi.DRSUAPI_ATTID_isDeleted:
+                continue
+
+            if o.local_usn <= isDeleted.local_usn:
+                continue
+
+            if o.originating_change_time <= isDeleted.originating_change_time:
+                continue
+
+            change_time = samba.nttime2unix(o.originating_change_time)
+
+            delta = change_time - delete_time
+            if delta <= tombstone_delta:
+                continue
+
+            # If the modification happened after the tombstone lifetime
+            # has passed, we have a bug as the object might be deleted
+            # already on other DCs and won't be able to replicate
+            # back
+            found.append(o)
+
+        return found, isDeleted
+
+    def has_changes_after_deletion(self, dn, repl_val):
+        found, isDeleted = self.find_changes_after_deletion(repl_val)
+        if len(found) == 0:
+            return False
+
+        def report_attid(o):
+            try:
+                attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+            except KeyError:
+                attname = "<unknown:0x%x08x>" % o.attid
+
+            self.report("%s: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+                        attname, o.attid, o.version,
+                        o.originating_invocation_id,
+                        o.originating_usn,
+                        o.local_usn,
+                        time.ctime(samba.nttime2unix(o.originating_change_time))))
+
+        self.report("ERROR: object %s, has changes after deletion" % dn)
+        report_attid(isDeleted)
+        for o in found:
+            report_attid(o)
+
+        return True
+
+    def err_changes_after_deletion(self, dn, repl_val):
+        found, isDeleted = self.find_changes_after_deletion(repl_val)
+
+        in_schema_nc = dn.is_child_of(self.schema_dn)
+        rdn_attr = dn.get_rdn_name()
+        rdn_attid = self.samdb_schema.get_attid_from_lDAPDisplayName(rdn_attr,
+                                                     is_schema_nc=in_schema_nc)
+
+        unexpected = []
+        for o in found:
+            if o.attid == rdn_attid:
+                continue
+            if o.attid == drsuapi.DRSUAPI_ATTID_name:
+                continue
+            if o.attid == drsuapi.DRSUAPI_ATTID_lastKnownParent:
+                continue
+            try:
+                attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+            except KeyError:
+                attname = "<unknown:0x%x08x>" % o.attid
+            unexpected.append(attname)
+
+        if len(unexpected) > 0:
+            self.report('Unexpeted attributes: %s' % ",".join(unexpected))
+            self.report('Not fixing changes after deletion bug')
+            return
+
+        if not self.confirm_all('Delete broken tombstone object %s deleted %s days ago?' % (
+                                dn, self.tombstoneLifetime), 'fix_changes_after_deletion_bug'):
+            self.report('Not fixing changes after deletion bug')
+            return
+
+        if self.do_delete(dn, ["relax:0"],
+                          "Failed to remove DN %s" % dn):
+            self.report("Removed DN %s" % dn)
+
     def has_replmetadata_zero_invocationid(self, dn, repl_meta_data):
         repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob,
                           repl_meta_data)
@@ -2088,7 +2251,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         error_count = 0
         set_attrs_from_md = set()
         set_attrs_seen = set()
-        got_repl_property_meta_data = False
         got_objectclass = False
 
         nc_dn = self.samdb.get_nc_root(obj.dn)
@@ -2105,6 +2267,26 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         name_val = None
         isDeleted = False
         systemFlags = 0
+        repl_meta_data_val = None
+
+        for attrname in obj:
+            if str(attrname).lower() == 'isdeleted':
+                if str(obj[attrname][0]) != "FALSE":
+                    isDeleted = True
+
+            if str(attrname).lower() == 'systemflags':
+                systemFlags = int(obj[attrname][0])
+
+            if str(attrname).lower() == 'replpropertymetadata':
+                repl_meta_data_val = obj[attrname][0]
+
+        if isDeleted and repl_meta_data_val:
+            if self.has_changes_after_deletion(dn, repl_meta_data_val):
+                error_count += 1
+                self.err_changes_after_deletion(dn, repl_meta_data_val)
+                return error_count
+            if self.is_expired_tombstone(dn, repl_meta_data_val):
+                return error_count
 
         for attrname in obj:
             if attrname == 'dn' or attrname == "distinguishedName":
@@ -2119,7 +2301,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                     self.report("ERROR: Not fixing num_values(%d) for '%s' on '%s'" %
                                 (len(obj[attrname]), attrname, str(obj.dn)))
                 else:
-                    name_val = obj[attrname][0]
+                    name_val = str(obj[attrname][0])
 
             if str(attrname).lower() == str(obj.dn.get_rdn_name()).lower():
                 object_rdn_attr = attrname
@@ -2130,13 +2312,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                 else:
                     object_rdn_val = str(obj[attrname][0])
 
-            if str(attrname).lower() == 'isdeleted':
-                if str(obj[attrname][0]) != "FALSE":
-                    isDeleted = True
-
-            if str(attrname).lower() == 'systemflags':
-                systemFlags = int(obj[attrname][0])
-
             if str(attrname).lower() == 'replpropertymetadata':
                 if self.has_replmetadata_zero_invocationid(dn, obj[attrname][0]):
                     error_count += 1
@@ -2166,7 +2341,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                         self.report("ERROR: Not fixing incorrect initial attributeID in '%s' on '%s', it should be objectClass" %
                                     (attrname, str(dn)))
 
-                got_repl_property_meta_data = True
                 continue
 
             if str(attrname).lower() == 'ntsecuritydescriptor':
@@ -2325,9 +2499,11 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
         if name_val is not None:
             parent_dn = None
+            controls = ["show_recycled:1", "relax:0"]
             if isDeleted:
                 if not (systemFlags & samba.dsdb.SYSTEM_FLAG_DISALLOW_MOVE_ON_DELETE):
                     parent_dn = deleted_objects_dn
+                controls += ["local_oid:%s:1" % dsdb.DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME]
             if parent_dn is None:
                 parent_dn = obj.dn.parent()
             expected_dn = ldb.Dn(self.samdb, "RDN=RDN,%s" % (parent_dn))
@@ -2338,19 +2514,20 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
 
             if expected_dn != obj.dn:
                 error_count += 1
-                self.err_wrong_dn(obj, expected_dn, object_rdn_attr, object_rdn_val, name_val)
+                self.err_wrong_dn(obj, expected_dn, object_rdn_attr,
+                        object_rdn_val, name_val, controls)
             elif obj.dn.get_rdn_value() != object_rdn_val:
                 error_count += 1
                 self.report("ERROR: Not fixing %s=%r on '%s'" % (object_rdn_attr, object_rdn_val, str(obj.dn)))
 
         show_dn = True
-        if got_repl_property_meta_data:
+        if repl_meta_data_val:
             if obj.dn == deleted_objects_dn:
                 isDeletedAttId = 131120
                 # It's 29/12/9999 at 23:59:59 UTC as specified in MS-ADTS 7.1.1.4.2 Deleted Objects Container
 
                 expectedTimeDo = 2650466015990000000
-                originating = self.get_originating_time(obj["replPropertyMetaData"][0], isDeletedAttId)
+                originating = self.get_originating_time(repl_meta_data_val, isDeletedAttId)
                 if originating != expectedTimeDo:
                     if self.confirm_all("Fix isDeleted originating_change_time on '%s'" % str(dn), 'fix_time_metadata'):
                         nmsg = ldb.Message()
@@ -2385,8 +2562,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
         except ldb.LdbError as e11:
             (enum, estr) = e11.args
             if enum == ldb.ERR_NO_SUCH_OBJECT:
-                self.err_missing_parent(obj)
-                error_count += 1
+                if isDeleted:
+                    self.report("WARNING: parent object not found for %s" % (obj.dn))
+                    self.report("Not moving to LostAndFound "
+                                "(tombstone garbage collection in progress?)")
+                else:
+                    self.err_missing_parent(obj)
+                    error_count += 1
             else:
                 raise
 
diff --git a/python/samba/graph.py b/python/samba/graph.py
index 0a0aecd6631..c8d5f9230d0 100644
--- a/python/samba/graph.py
+++ b/python/samba/graph.py
@@ -88,7 +88,7 @@ def shorten_vertex_names(vertices, suffix=',...', aggressive=False):
         try:
             while True:
                 c = set(x[i] for x in vlist)
-                if len(c) > 1 or c == {'*'}:
+                if len(c) > 1 or '*' in c:
                     break
                 i -= 1
         except IndexError:
diff --git a/python/samba/kcc/kcc_utils.py b/python/samba/kcc/kcc_utils.py
index 81d381abd99..ef4d706c8e2 100644
--- a/python/samba/kcc/kcc_utils.py
+++ b/python/samba/kcc/kcc_utils.py
@@ -31,7 +31,7 @@ from samba.dcerpc import (
 )
 from samba.common import dsdb_Dn
 from samba.ndr import ndr_unpack, ndr_pack
-from collections import Counter
+from collections import defaultdict
 
 
 class KCCError(Exception):


-- 
Samba Shared Repository



More information about the samba-cvs mailing list