Samba Shared Repository - branch master updated

[Douglas Bagnall]

One of my recent commits (maybe 882ad16138ced8e8f010b1ffaa44b31e10eb9e1c 
or a24ba4ea22a99b278cc03a0b484f514aaa8520c8) had a line so long in the 
diff that it broke the will of samba-cvs at lists.samba.org to send on the 
autobuild message.

What it would have said is something like the following, but with the 
too-large diff also included.

d

-----8<------------------


The branch, master has been updated
       via 18f44f3ba46 (origin/master, origin/HEAD) selftest:ndrdump: 
adjust xattr_NTACL test for ACE coda
       via 909a2af9548 libcli/security: rm unused sec_ace_copy()
       via 882ad16138c pytest: compare Samba vs Windows SDDL security 
descriptors
       via 741ab485b0c libcli/security: test helper script extracts fuzz 
SDDL
       via 536b9a0c920 libcli/security: move Windows test script to 
windows subdir
       via 641981c4ad8 libcli/security: rewrite SDDL tests for running 
on Windows
       via f58372cca5b python/colour: add a colour diff helper
       via a24ba4ea22a libcli/security: add some test more Windows SDDL 
test strings
       via 3c300790958 libcli/security: script to turn token/descriptor 
pairs into sddl
       via 55cc6b52c02 libcli/security: remove unused tmp_ctx in 
calculate_inherited_from_parent()
       via c1621d34922 libcli/security: avoid leak in 
calculate_inherited_from_parent()
       via 26722348b93 libcli/security: used sec_ace_object() in sddl 
encoding
       via c73034cf7c4 librpc/security.idl: adjust size calculations for 
upcoming ace types
       via 47edd41bc90 libcli/security: sddl ACL decode avoids early 
splitting on parenthesis
       via 5a45a434c7d s4/ntfvfs/pvfs_acl: initialise ACEs to zeroes
       via 31dadff39ac s4/torture: initialise ACE structs to zero
       via 4a1848a1f55 pytest/sddl: tests for more invalid or weird cases
       via a8a0ae89c5d pytest/sddl: add option for case-insensitive tests
       via 5b9f42b391a pytest/sddl: environment option for exporting as 
fuzz seeds
       via 77967274535 pytest/sddl: replace/export tests based on 
environment vars
       via 59a2d741070 pytests: add helper to grab a directory from 
environment
       via 43e017d3b45 libcli/security/pysecurity: use NULL to mean NULL 
in C
       via 4504324ea4c lib/util: strhex_to_data_blob checks talloc
       via 86f67f59eaf (gitlab-samba/master) s3: libsmb: Add a missing 
return statement in the timeout case.



commit 18f44f3ba46d0f178b8ac5bc2ebf7df30ec0b16d
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Tue Aug 22 10:14:50 2023 +1200

     selftest:ndrdump: adjust xattr_NTACL test for ACE coda

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

     Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
     Autobuild-Date(master): Thu Aug 24 03:47:08 UTC 2023 on atb-devel-224

commit 909a2af9548fe47305fa32c82d564e7cc3175271
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 18 16:04:51 2023 +1200

     libcli/security: rm unused sec_ace_copy()

     Unused since 2014.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 882ad16138ced8e8f010b1ffaa44b31e10eb9e1c
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Aug 9 14:15:27 2023 +1200

     pytest: compare Samba vs Windows SDDL security descriptors

     Can Samba understand Windows security descriptors? Does it parse SDDL
     the same way?

     Here we test on over 7000 SDDL/descriptor pairs and find the answer
     is pleasing. In later commits we will add more tests using different
     classes of ACE.

     The test cases are derived from fuzz seeds, exported to Windows via
     the script in the last commit, with the Windows descriptor bytes found
     using libcli/security/tests/windows/windows-sddl-test.py.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 741ab485b0c5d18a8c3cb0de3b09d003ba58d667
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 11 22:21:41 2023 +1200

     libcli/security: test helper script extracts fuzz SDDL

     This allows us to try the fuzz seeds as SDDL on Windows, then test
     that Samba matches Windows' security descriptors in the cases where
     the SDDL compiles. This will find SDDL edge cases that might otherwise
     be missed.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 536b9a0c9209fc7e12cf7a4d0c1a24d3f5906d2a
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 18 10:37:47 2023 +1200

     libcli/security: move Windows test script to windows subdir

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 641981c4ad8bd3f031d8a23a76e70212dcef1924
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 4 11:52:17 2023 +1200

     libcli/security: rewrite SDDL tests for running on Windows

     This script never worked well because it had to shell out to
     Powershell, which never worked well due to syntax conflicts and
     Powershell's specialness. The attempted ctypes version did not work,
     due to the difficulty in expressing things like "relative
     PSECURITY_DESCRIPTOR" in ctypes.

     It turns out that pywin32 is easy to install and use, and we can
     extract the NDR bytes which is far more useful than just testing if
     the SDDL parses.

     On Windows:

     1. install Python from python.org
     2. run `pip install pywin32`
     3. copy e.g. libcli/security/tests/data/conditional-aces.txt to Windows
     4. run `python windows-sddl-tests.py conditional-aces.txt`
     5. add `--help` to see how to export descriptor bytes.

     The default output is a whole lot of multi-coloured text, indicating
     what failed and what didn't.

     With --export-json it writes a JSON file mapping SDDL strings to NDR
     byte sequences, which can be used to compare with Samba's attempts. If
     you are only interested in --export-json, you might also like --quiet.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit f58372cca5b59a5d4ed653ef53e69ea756940a68
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Thu Aug 17 14:20:12 2023 +1200

     python/colour: add a colour diff helper

     Sometimes colour can help show what is different between two strings.

     This is roughly the same as

     `git diff --no-index --color-words=. <a> <b>`.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit a24ba4ea22a99b278cc03a0b484f514aaa8520c8
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 4 14:15:19 2023 +1200

     libcli/security: add some test more Windows SDDL test strings

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 3c3007909580ada71ecf262eb673d9001478e4fb
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Wed Jul 26 17:50:22 2023 +1200

     libcli/security: script to turn token/descriptor pairs into sddl

     We fuzz security descriptors in a couple of different ways, and this
     maps seeds from one form into the other. The SDDL examples can also be
     used in Windows tests.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 55cc6b52c02cff0b1aeb86044de931bda0d535ab
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 18 14:37:35 2023 +1200

     libcli/security: remove unused tmp_ctx in 
calculate_inherited_from_parent()

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit c1621d34922265f664e26d03f2b049c6bf1b1c21
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Jul 21 14:35:45 2023 +1200

     libcli/security: avoid leak in calculate_inherited_from_parent()

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 26722348b938ce14f54716ff7ab4d02d94c7f5c6
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Jul 21 16:58:45 2023 +1200

     libcli/security: used sec_ace_object() in sddl encoding

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit c73034cf7c4392f5d3505319948bc84634c20fa5
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Thu Jul 13 21:31:50 2023 +1200

     librpc/security.idl: adjust size calculations for upcoming ace types

     Soon we will get Conditional ACEs and Resource Attribute ACES, each of
     which have trailing bytes at the end of the ACE. Here's a diagram:

                   ____      The ACE size field may indicate a size bigger
       .type      /    |     than the known parts, even when you take
       .flags    /     |     rounding to a multiple of four into account.
       .size  --'      |     This extra data is meaningful in some ACEs.
       .access_mask    |
       .trustee (sid) _|  <- known data ends here.
                       :
        "coda"      ___:  <- the trailing part, Zero size unless the size
                             field points beyond the end of the known data.
                             Probably empty for ordinary ACE types.

     Until now we have thrown away these extra bytes, because they have no
     meaning in the ACE types we recognise. But with conditional and
     resource attribute ACEs we need to catch and process these bytes, so
     we add an extra field for that.

     Thus we can drop the manually written ndr_pull_security_ace() that
     discarded the trailing bytes, because we just allow it to be pulled
     into an unused blob. In the very common case, the blob will be empty.

     Microsoft does not use a common name across different ACE types to
     describe this end-data -- "coda" is a Samba term.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 47edd41bc903ccd95eb368e405c5cdda65de4332
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Jul 21 16:51:53 2023 +1200

     libcli/security: sddl ACL decode avoids early splitting on parenthesis

     Soon we will have Conditional ACEs and Resource Attribute ACEs. It is
     expected --indeed mandatory-- that the SDDL representations of these
     ACEs will contain parentheses, so we can't use '(' and ')' to decide
     where ACEs stop and start.

     This means shifting where we make a mutable copy of the SDDL string
     from per-ACE to per-ACL, and allowing sddl_decode_ace() to decide when
     its ACE is finished.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 5a45a434c7d8cc3d313e695de54efbefbdc9d2a2
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Tue Aug 22 14:13:44 2023 +1200

     s4/ntfvfs/pvfs_acl: initialise ACEs to zeroes

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 31dadff39ac70559d65c90092bb3cbe4d5e275cb
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Tue Aug 22 14:12:46 2023 +1200

     s4/torture: initialise ACE structs to zero

     Because soon these structs will have more members, which are typically
     going to be zero.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 4a1848a1f557ac02439a40c2786eb63c9d08def3
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Apr 28 10:46:27 2023 +1200

     pytest/sddl: tests for more invalid or weird cases

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit a8a0ae89c5d78f8661bab98d0494010628308364
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 18 13:10:02 2023 +1200

     pytest/sddl: add option for case-insensitive tests

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 5b9f42b391a4d9bf9d150383f041dddd54af3491
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 18 13:09:13 2023 +1200

     pytest/sddl: environment option for exporting as fuzz seeds

     We want realistic examples for sddl fuzzing seeds, and we want
     realistic examples for sddl tests, so hopefully we only need to get
     it right once.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 77967274535c1180413619aa7d33602a82e34a5f
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Aug 11 11:08:00 2023 +1200

     pytest/sddl: replace/export tests based on environment vars

     We had a stupid system involving test functions with side-effects,
     that needed to be enabled by editing the file. Now you get the same
     effects by setting environment variables, the names of which you can
     only learn by reading the file closely.

     This works better because some stuff needs to happen at class-time
     rather than instance-time.

     Also the environment variables specify the import and export locations.

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 59a2d7410705eaa5315e4ea11325cce74ce4067e
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Jul 7 16:12:19 2023 +1200

     pytests: add helper to grab a directory from environment

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 43e017d3b45e5aef1305b86facf765b8c11dc5a0
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Fri Apr 14 13:58:15 2023 +1200

     libcli/security/pysecurity: use NULL to mean NULL in C

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 4504324ea4c8d9c2f29bc64479221b87837da4ce
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date:   Thu Aug 10 14:26:11 2023 +1200

     lib/util: strhex_to_data_blob checks talloc

     Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
     Reviewed-by: Andrew Bartlett <abartlet at samba.org>