[SCM] Samba Shared Repository - branch master updated

Andrew Bartlett abartlet at samba.org
Thu Dec 14 11:35:02 UTC 2017


The branch, master has been updated
       via  964bc8d markdown: Rename ms_markdown.py -> ms_schema_markdown.py
       via  4f20416 provision: Use the official MS 2008R2 schema by default
       via  1daba6f schema: 2008R2 AD schema attributes and classes
       via  8019c76 schema: 2016 AD schema attributes and classes
       via  8519f98 provision: RODC revision level should be at 2
       via  1978838 selftest: Add basic test for schema upgrade
       via  6bdbcb1 domain.py: Auto-patch the diffs for the adprep schemaupgrade
       via  5db10e0 domain.py: Add a base dir option for schema upgrades
       via  c870c34 schema: Some 2012 objects were missing systemflags
       via  c22d022 upgradeprovision: Change test to always use 2008 R2 schema
       via  ea9cde92 domain.py: Add base-schema option to samba-tool provision
       via  1f60f5b schema: Add option of specifying the base schema for a provision
       via  f4d9b79 selftest: Fix upgradeprovision test by importing new objects for schema 45
       via  d157f97 2008R2: Missing flags on optional features container for objectVersion 45
       via  ff98bf9 2008R2: Missing extended rights for objectVersion 45
       via  d67f706 schema: Re-work extended rights handling in provision (prep for 2012R2)
       via  d6e0f43 provision: Make clarifying header an LDIF comment in extended-rights.ldif
       via  e8b200f provision: Align displayName of Property Sets with MS-ADTS 3.1.1.2.3.3
       via  d44c811 provision: Fill in a nicer displayName for Extended Rights
       via  b9f0fbd provision: Fill in validAccesses in extended-rights.ldif for Property Sets
       via  7657168 provision: Fill in validAccesses in extended-rights.ldif for Validated Writes
       via  9840ee7 provision: Fill in validAccesses in extended-rights.ldif for Control Access Rights
       via  593a845 provision: Align extended-rights.ldif with the adprep LDIF for 2012R2
       via  6721052 provision: Reformat appliesTo in Extended Rights into LDIF
       via  7fad489 provision: Remove section numbers from extended rights, replace with dn
       via  7bc9c20 provision: Import extended rights schema from MS-ADTS v47.0
       via  9327c5a domain.py: Add a schemaupgrade option to apply missing 2008R2 schema
       via  f9059c7 domain.py: Make schemaupgrade option work regardless of config
       via  580e6ba domain.py: Add schema upgrade option to samba-tool
       via  2650e92 schema: Allow schemaUpdateNow to refresh schema during a transaction
       via  d66cbca adprep: Add the LDF data needed to upgrade to 2012R2 schema
       via  d9c6f47 objectclass: Ensure that backlinks are not replicated
       via  3257c7f ms_schema: Properly handle base64 encoded attributes
       via  0f6e52a schema: 2012 and 2012 R2 AD schema attributes and classes
       via  ed6a3dd ms_schema: Allow for CN=X and DC=X replacements
       via  f4286f3 typo: Change case to match DN
       via  07f094f flags.h: Introduce the 2016 function level constant
       via  4ea7aa9 ldb: Show the last successful DN when failing to parse LDIF
      from  e8b801d WHATSNEW: document the removal of 'auth methods', 'map untrusted to domain' and 'profile acls'

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit 964bc8d19aa695f6c5188ab1a941127a259c0bc8
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Thu Nov 23 17:06:53 2017 +1300

    markdown: Rename ms_markdown.py -> ms_schema_markdown.py
    
    We also reduce the scope of the import so that python-markdown is only
    required if interacting with 2012 code.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>
    
    Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
    Autobuild-Date(master): Thu Dec 14 12:34:04 CET 2017 on sn-devel-144

commit 4f20416b383f11d6f7d30616696a5fdf5b6057a3
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Nov 20 17:10:25 2017 +1300

    provision: Use the official MS 2008R2 schema by default
    
    This fixes us to have the official adminDescription etc.  While both schema were provided by
    Microsoft this is a better quality one, but still under the same licence.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 1daba6f25541dab6f3d888431d00eb61544382d8
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Nov 20 15:45:41 2017 +1300

    schema: 2008R2 AD schema attributes and classes
    
    Obtained under the Open Protocols Specifications licence from
    https://www.microsoft.com/en-us/download/details.aspx?id=23782
    
    These are more complete than the version we have had in the tree until now.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 8019c76b5681a1a86b410fdd6bf0a1447266cfb8
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Nov 20 15:18:41 2017 +1300

    schema: 2016 AD schema attributes and classes
    
    Obtained under the Open Protocols Specifications licence from
    https://www.microsoft.com/en-us/download/details.aspx?id=23782
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 8519f98677dd28b8ed4091bf266652b870cff4a8
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Aug 16 16:02:32 2017 +1200

    provision: RODC revision level should be at 2
    
    This number had been mistakenly updated alongside the standard forest
    updates revision. This version number appears to be independent of the
    other revision levels.
    
    Also add the change to a new .ldf file, which can be used to apply
    the schema change to an existing Samba 4.7 (or earlier) instance.
    Update the provision/upgrade test to do just this (otherwise it
    complains about differences between a new provision and an older Samba
    4.0.0 instance).
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 197883838f8ecb027e9d0375ff5238aec1567a42
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Fri Oct 6 16:30:40 2017 +1300

    selftest: Add basic test for schema upgrade
    
    This tests that we can provision using both the 2008 and 2012 schema,
    that we can upgrade a 2008 Samba instance to use the 2012 schema, and
    that when we do that the result (more or less) matches a straight
    2012 provision.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 6bdbcb1d4c577b019f74f027e5a753583f34bbf8
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Nov 1 11:53:29 2017 +1300

    domain.py: Auto-patch the diffs for the adprep schemaupgrade
    
    This creates a temporary directory where the markdown is parsed and the
    diffs are then applied.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 5db10e066279e8b2c917cad16908c66d36cbfde3
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Nov 1 10:48:36 2017 +1300

    domain.py: Add a base dir option for schema upgrades
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit c870c34df7fe1d4391543e6701a1398dce42c7e5
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Sep 27 14:51:25 2017 +1300

    schema: Some 2012 objects were missing systemflags
    
    The adprep LDIF files were adding the systemFlags, but they weren't
    present in the 2012 schema files. This is not just a Microsoft
    documentation problem - the difference was present when doing a provision
    of a 2012 Windows server vs using Adprep.exe to upgrade an older Windows
    server.
    
    Samba might as well use the correct systemFlags right from the start.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit c22d022ceafebe0c30a3947154016e51baf2d5b3
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Oct 5 10:01:27 2017 +1300

    upgradeprovision: Change test to always use 2008 R2 schema
    
    This tool (and the corresponding test) is designed to migrate a Samba DC
    from a pre-4.0.0 release up to a more recent schema (i.e. Windows 2008R2).
    
    Going further than 2008R2 turns this test into a bit of a nightmare. We
    now have a better adprep/'samba-tool domain schemaupgrade' option for
    upgrading from 2008R2 to a more recent schema.
    
    It seems to make most sense to leave this tests just running against
    2008R2 schema provisions and add new tests to migrate from 2008R2 to
    2012R2.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit ea9cde92fb57d6b65581b0fb48b8f3f253cadc55
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Oct 5 09:53:28 2017 +1300

    domain.py: Add base-schema option to samba-tool provision
    
    Allow a different base-schema to be used when provisioning a new domain.
    This allows us to test the new 2012 schema without committing Samba to
    using it by default.
    
    If, in future, we change the default to use the 2012 schema, some
    existing Samba tests (like upgradeprovision) rely on the 2012 schema.
    So making the base-schema optional allows these tests to continue using
    the older schema.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 1f60f5b51a8f510461f218ee1a5fc2ebbc9ac625
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Wed Oct 4 12:30:59 2017 +1300

    schema: Add option of specifying the base schema for a provision
    
    Add the ability to override the base schema files being used for the
    new provision, e.g. instead of using the default supported schema,
    the code can now potentially specify an older or newer schema to use.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit f4d9b797e22a4cade3752930483bfc7a5a955338
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Tue Dec 12 15:26:35 2017 +1300

    selftest: Fix upgradeprovision test by importing new objects for schema 45
    
    The recent schema changes mean that the upgradeprovision test starts
    failing. This is because it's using an old 4.0.0 schema (that doesn't
    have these schema changes), but it's comparing it against a fresh
    provision (which does have the changes). We can avoid this failure by
    using the 'samba-tool domain schemaupgrade' to bring the old 4.0.0 schema
    in line with a fresh provision. Note that the 'upgradeprovision --full'
    test doesn't need this change as it seems to more aggressively copy over
    any schema differences with a fresh provision.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit d157f9752bb3590ce74634db96d5c36b84ce792e
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Tue Dec 12 15:20:26 2017 +1300

    2008R2: Missing flags on optional features container for objectVersion 45
    
    To match Windows 2008R2, this should have the same flags as the
    recycle bin enabled feature.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>

commit ff98bf96e9b24242893dc0fe9e1f2fa64d261d30
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Wed Dec 13 15:03:57 2017 +1300

    2008R2: Missing extended rights for objectVersion 45
    
    We appear to have been missing some extended rights from 2008R2. These were
    added in samba by the extended-rights.ldif
    
    On Windows this was in Sch45.ldf (triggered by adprep schema updates).
    
    We add these changes to adprep/samba-4.7-missing-for-schema-45.ldif,
    which can be used to apply the changes to an existing Samba instance.
    
    This is not extracted from the Sch45.ldf file provided by Microsoft
    but is instead extracted using ldapcmp against a Samba install running
    the new extended-rights.ldif.
    
    Finally, these schema changes mean that the upgradeprovision test starts
    failing. This is because it's using an old 4.0.0 schema (that doesn't
    have these schema changes), but it's comparing it against a fresh
    provision (which does have the changes). We can avoid this failure by
    using the 'samba-tool domain schemaupgrade' to bring the old 4.0.0 schema
    in line with a fresh provision. Note that the 'upgradeprovision --full'
    test doesn't need this change as it seems to more aggressively copy over
    any schema differences with a fresh provision.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>

commit d67f706b34d3bae05c7155092aa29d7e1148e7e6
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 14:42:55 2017 +1300

    schema: Re-work extended rights handling in provision (prep for 2012R2)
    
    Add the changes needed to provision a 2012 DC (mostly this just affects
    the Extended Rights objects) by moving to the new extended-rights.ldif
    
    The localizationDisplayId is not documented in MS-ATDS so these values
    are moved to provision_configuation_modify.ldif and applied after the
    display-specifiers.ldif
    
    We don't enable the 2012R2 mode yet. The ${INC2012} variable
    just gets replaced with '#' so the lines get commented out and not
    applied.
    
    This approach allows us to support provisioning both a 2008R2 DC or
    a 2012R2 DC (so that we can test we can upgrade a 2008 DC to 2012).
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit d6e0f43ab98fff300ffdc7a888bbc84f74e580f0
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 14:50:39 2017 +1300

    provision: Make clarifying header an LDIF comment in extended-rights.ldif
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit e8b200fad365298e57ca4b8fa7451e06451e7b0c
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 13:35:25 2017 +1300

    provision: Align displayName of Property Sets with MS-ADTS 3.1.1.2.3.3
    
    This gives some better names than what the CN of the object was.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit d44c811a8ca92347f29855909e1effc0c2c6abbd
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 13:26:53 2017 +1300

    provision: Fill in a nicer displayName for Extended Rights
    
    We replace all the hyphens with a space.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit b9f0fbdeaa8571f3a3f382fee609402b4dddcbd8
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 12:35:45 2017 +1300

    provision: Fill in validAccesses in extended-rights.ldif for Property Sets
    
    A Property Right has the value of RIGHT_DS_READ_PROPERTY|RIGHT_DS_WRITE_PROPERTY which is
    48 (0x30) per 5.1.3.2 Access Rights.
    
    The property Sets are listed in MS-ATDS 3.1.1.2.3.3 and can also be found by looking
    at the attributeSecurityGuid on the schema objects.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 7657168e753c2c9de03e4a06fc89a1ff962e2f14
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 12:26:04 2017 +1300

    provision: Fill in validAccesses in extended-rights.ldif for Validated Writes
    
    MS-ATDS 5.1.3.2.2 Validated Writes specifies the value of RIGHT_DS_WRITE_PROPERTY_EXTENDED which is
    8 (0x08) per 5.1.3.2 Access Rights.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 9840ee76fbb2e52b2ddb36c9342eb9a7faeacda6
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 12:22:05 2017 +1300

    provision: Fill in validAccesses in extended-rights.ldif for Control Access Rights
    
    MS-ATDS 5.1.3.2.1 Control Access Rights specifies the value of RIGHT_DS_CONTROL_ACCESS which is
    256 (0x100) per 5.1.3.2 Access Rights.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 593a8456a8f194b1e0dd4fe4a6524375af5dc696
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 11:57:35 2017 +1300

    provision: Align extended-rights.ldif with the adprep LDIF for 2012R2
    
    This removes the additional rights for 2016 and flags the 2012R2 changes to allow
    the same file to be used to produce a 2008R2 or 2012R2 domain
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 67210522160ebc429b4eabc5f4e36d2677e145bb
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 11:09:51 2017 +1300

    provision: Reformat appliesTo in Extended Rights into LDIF
    
    We remove comments about Schema 45 and earlier as this is the base
    level that Samba supports.  A future commit will move to a
    machine-parsable flag for the 2012 schema and remove the 2016 elements.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 7fad4896f61f833d29a49ca20766abfbbd7874b4
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 10:51:32 2017 +1300

    provision: Remove section numbers from extended rights, replace with dn
    
    This makes this file more like LDIF so we can process it automatically as well as
    use it as a text document.
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 7bc9c20037c908a2211d92ddb960325a45972969
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 10:09:55 2017 +1300

    provision: Import extended rights schema from MS-ADTS v47.0
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

commit 9327c5a35e760619060cf87b00ce39e32d54c319
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Oct 5 16:16:30 2017 +1300

    domain.py: Add a schemaupgrade option to apply missing 2008R2 schema
    
    We've identified some cases where we've gotten our implementation of the
    2008R2 schema wrong. We can fix these up for new provisions going
    forward, but it'd be nice to have some way of fixing up the schema on
    existing DCs.
    
    A lot of what we're missing is already documented in Microsoft's
    Sch45.ldf file:
    https://technet.microsoft.com/en-us/library/dd378890(v=ws.10).aspx
    
    Unfortunately we can't just apply the Sch45.ldf file using the existing
    'samba-tool domain schema-upgrade' option because:
    - We have got some of the Sch45.ldf changes, just not all of them.
    - We already say the Samba schema objectVersion is 47 (2008R2), so
      there's no way to tell if the Samba instance does or doesn't have the
      missing changes (apart from querying each change).
    
    We may want to add this to dbcheck eventually, but the simplest
    implementation option for now is to extend the new schemaupgrade command
    to allow us to specify a particular .LDF file to apply.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit f9059c7c1b83935dcd4b3bb645c926979c26a207
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Thu Oct 5 15:43:53 2017 +1300

    domain.py: Make schemaupgrade option work regardless of config
    
    Currently the 'samba-tool domain schemaupgrade' command will only work
    if the Samba config has the non-default option 'dsdb:schema update
    allowed = yes'. The whole point of running this samba-tool option is to
    upgrade the schema, so it would seem to make sense to bypass the setting
    temporarily, in order to apply the schema updates successfully.
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 580e6babaf93a9a88e993527f0731408a0f2d9bf
Author: Tim Beale <timbeale at catalyst.net.nz>
Date:   Wed Oct 4 12:30:59 2017 +1300

    domain.py: Add schema upgrade option to samba-tool
    
    Microsoft has published the Schema updates that its Adprep.exe tool
    applies when it upgrades a 2008R2 schema to 2012R2.
    
    This patch adds an option to samba-tool to go through these update files
    and apply each change one by one. Along the way we need to make a few
    changes to the LDIF operations, e.g. change 'ntdsschemaadd' to 'add' and
    so on.
    
    The bulk of the changes involve parsing the .ldif file and separating
    out each update into a separate operation.
    
    There are a couple of errors that we've chosen to ignore:
    - Trying to set isDefunct for an object we don't know about.
    - Trying to set a value for an attribute OID that we don't know about
      (we may need to fix this in future, but it'll require some help from
       Microsoft about what the OIDs actually are).
    
    To try to make life easier, I've added a ldif_schema_update helper
    class. This provides convenient access of the DN the change applies to
    and other such details (whether it's setting isDefunct, etc).
    
    Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 2650e9258b88228544148f5254dee7958819f6eb
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Fri Aug 18 13:59:30 2017 +1200

    schema: Allow schemaUpdateNow to refresh schema during a transaction
    
    When we upgrade a schema from 2008R2 to 2012R2, we want to apply all the
    changes in a single transaction - if we can't apply all the updates then
    we don't want to be left with a schema halfway in between the two.
    
    However, as we apply each LDIF update, we also want to refresh the
    schema. There are 2 reasons for this:
    1. The adprep .LDIF files provided by Microsoft have some writes to
    schemaUpdateNow in them.
    2. Microsoft uses attribute OIDs in their adprep .LDIF files, which
    Samba doesn't handle so well. However, we can replace the OIDs with the
    attribute's ldapDisplayName and they work fine. But to do this, we need
    to query the schema to map the OID to attribute name. And to query the
    schema successfully, the schema needs to be refreshed after the new
    attribute object has been added.
    
    Basically this patch avoids bailing out during the dsdb_schema_refresh()
    if we are writing schemaUpdateNow as part of a larger transaction.
    
    Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
    
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit d66cbca4e1c4d68a40fb117efc3e7e357690fc5d
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Tue Oct 3 10:01:30 2017 +1300

    adprep: Add the LDF data needed to upgrade to 2012R2 schema
    
    This patch adds the LDF files corresponding to the changes that the
    Windows Adprep.exe tool makes when upgrading a AD schema to Windows
    2012R2.
    
    This is based on information Microsoft has made public on github
    (Schema-Updates.md - see the README.txt for more details).
    
    The LDF files 48-56 are for upgrading to Windows Server 2012, and 57-69
    are for Windows Server 2012 R2.
    
    Unfortunately, the raw LDF information from Microsoft wasn't enough to
    get the schema working. The .diff files contain changes we needed to
    make on top of the raw LDF content from Microsoft.
    
    The basic steps to regenerate the .LDF files are documented in the
    README.txt file. The files used to generate the .LDF files are in the
    WindowsServerDocs/ sub-directory. (The .LDF generation is done at runtime
    during provision).
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit d9c6f47851311838ba6a67f8effc7a4e398e12ca
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Tue Sep 5 16:03:04 2017 +1200

    objectclass: Ensure that backlinks are not replicated
    
    Adprep schema adds backlinks, but they do not have the NOT_REPLICATED
    bit. We need to force this in locally to ensure we have it.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 3257c7f60fea45ade3f761dd566aa181417b638c
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Fri Aug 18 13:46:57 2017 +1200

    ms_schema: Properly handle base64 encoded attributes
    
    There used to be a special case for omobjectclass, but now there is just
    generic handling for such attributes.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 0f6e52a268c9812e12602031247c88cceb9dec62
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Mon Sep 12 17:07:02 2016 +1200

    schema: 2012 and 2012 R2 AD schema attributes and classes
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit ed6a3ddb2ac127e086af748710750320d46f13d4
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Mon Sep 19 13:52:54 2016 +1200

    ms_schema: Allow for CN=X and DC=X replacements
    
    These occur in the newer 2012 and 2016 schemas.
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit f4286f3516c3e14ebdd16758cadc4ed4c0afce10
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Wed Aug 2 12:52:22 2017 +1200

    typo: Change case to match DN
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 07f094f69fa91f7f363ca892cd2a640a76c90a94
Author: Garming Sam <garming at catalyst.net.nz>
Date:   Tue Aug 15 15:17:34 2017 +1200

    flags.h: Introduce the 2016 function level constant
    
    Signed-off-by: Garming Sam <garming at catalyst.net.nz>
    Reviewed-by: Andrew Bartlett <abartlet at samba.org>

commit 4ea7aa9265199e515d8f08ef849b69cfa3ee1955
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Mon Dec 11 15:57:30 2017 +1300

    ldb: Show the last successful DN when failing to parse LDIF
    
    Signed-off-by: Andrew Bartlett <abartlet at samba.org>
    Reviewed-by: Garming Sam <garming at catalyst.net.nz>

-----------------------------------------------------------------------

Summary of changes:
 lib/ldb/pyldb.c                                    |    24 +-
 libds/common/flags.h                               |     1 +
 python/samba/ms_schema.py                          |    52 +-
 python/samba/ms_schema_markdown.py                 |    71 +
 python/samba/netcmd/domain.py                      |   309 +-
 python/samba/provision/__init__.py                 |    18 +-
 python/samba/schema.py                             |    41 +-
 python/samba/upgradehelpers.py                     |     4 +-
 selftest/tests.py                                  |     5 +
 source4/dsdb/pydsdb.c                              |     1 +
 source4/dsdb/samdb/ldb_modules/objectclass.c       |     6 +
 source4/dsdb/samdb/ldb_modules/rootdse.c           |    16 +
 source4/dsdb/samdb/ldb_modules/schema_load.c       |    12 +-
 source4/scripting/bin/samba_upgradeprovision       |     2 +-
 .../AD_DS_Attributes__Windows_Server_2012_R2.ldf   | 30374 ++++++++++++++++++
 .../AD_DS_Attributes__Windows_Server_2016.ldf      | 30912 +++++++++++++++++++
 .../AD_DS_Classes__Windows_Server_2012_R2.ldf      |  8875 ++++++
 .../AD_DS_Classes__Windows_Server_2016.ldf         |  9031 ++++++
 ...ttributes_for_AD_DS__Windows_Server_2008_R2.ldf | 26925 ++++++++++++++++
 .../Attributes_for_AD_DS__Windows_Server_2012.ldf  | 29357 ++++++++++++++++++
 .../Classes_for_AD_DS__Windows_Server_2008_R2.ldf  |  7934 +++++
 .../Classes_for_AD_DS__Windows_Server_2012.ldf     |  8624 ++++++
 source4/setup/adprep/README.txt                    |    23 +
 source4/setup/adprep/WindowsServerDocs/LICENSE     |   395 +
 .../setup/adprep/WindowsServerDocs/LICENSE-CODE    |    17 +
 .../setup/adprep/WindowsServerDocs/Sch49.ldf.diff  |    30 +
 .../setup/adprep/WindowsServerDocs/Sch50.ldf.diff  |   107 +
 .../setup/adprep/WindowsServerDocs/Sch51.ldf.diff  |   225 +
 .../setup/adprep/WindowsServerDocs/Sch57.ldf.diff  |   105 +
 .../setup/adprep/WindowsServerDocs/Sch59.ldf.diff  |    26 +
 .../adprep/WindowsServerDocs/Schema-Updates.md     |  4583 +++
 source4/setup/adprep/fix-forest-rev.ldf            |     6 +
 .../adprep/samba-4.7-missing-for-schema45.ldif     |   112 +
 source4/setup/extended-rights.ldif                 |   835 +
 source4/setup/provision_configuration.ldif         |   682 +-
 source4/setup/provision_configuration_modify.ldif  |   503 +
 source4/setup/provision_schema_basedn_modify.ldif  |     2 +-
 source4/setup/provision_users.ldif                 |     4 +-
 source4/setup/wscript_build                        |     3 +
 testprogs/blackbox/schemaupgrade.sh                |   122 +
 testprogs/blackbox/upgradeprovision-oldrelease.sh  |     8 +-
 41 files changed, 159669 insertions(+), 713 deletions(-)
 create mode 100644 python/samba/ms_schema_markdown.py
 create mode 100644 source4/setup/ad-schema/AD_DS_Attributes__Windows_Server_2012_R2.ldf
 create mode 100644 source4/setup/ad-schema/AD_DS_Attributes__Windows_Server_2016.ldf
 create mode 100644 source4/setup/ad-schema/AD_DS_Classes__Windows_Server_2012_R2.ldf
 create mode 100644 source4/setup/ad-schema/AD_DS_Classes__Windows_Server_2016.ldf
 create mode 100644 source4/setup/ad-schema/Attributes_for_AD_DS__Windows_Server_2008_R2.ldf
 create mode 100644 source4/setup/ad-schema/Attributes_for_AD_DS__Windows_Server_2012.ldf
 create mode 100644 source4/setup/ad-schema/Classes_for_AD_DS__Windows_Server_2008_R2.ldf
 create mode 100644 source4/setup/ad-schema/Classes_for_AD_DS__Windows_Server_2012.ldf
 create mode 100644 source4/setup/adprep/README.txt
 create mode 100644 source4/setup/adprep/WindowsServerDocs/LICENSE
 create mode 100644 source4/setup/adprep/WindowsServerDocs/LICENSE-CODE
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Sch49.ldf.diff
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Sch50.ldf.diff
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Sch51.ldf.diff
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Sch57.ldf.diff
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Sch59.ldf.diff
 create mode 100644 source4/setup/adprep/WindowsServerDocs/Schema-Updates.md
 create mode 100644 source4/setup/adprep/fix-forest-rev.ldf
 create mode 100644 source4/setup/adprep/samba-4.7-missing-for-schema45.ldif
 create mode 100644 source4/setup/extended-rights.ldif
 create mode 100755 testprogs/blackbox/schemaupgrade.sh


Changeset truncated at 500 lines:

diff --git a/lib/ldb/pyldb.c b/lib/ldb/pyldb.c
index e61b5b6..04b3f1b 100644
--- a/lib/ldb/pyldb.c
+++ b/lib/ldb/pyldb.c
@@ -1669,6 +1669,7 @@ static PyObject *py_ldb_parse_ldif(PyLdbObject *self, PyObject *args)
 	PyObject *list, *ret;
 	struct ldb_ldif *ldif;
 	const char *s;
+	struct ldb_dn *last_dn = NULL;
 
 	TALLOC_CTX *mem_ctx;
 
@@ -1686,8 +1687,29 @@ static PyObject *py_ldb_parse_ldif(PyLdbObject *self, PyObject *args)
 		talloc_steal(mem_ctx, ldif);
 		if (ldif) {
 			PyList_Append(list, ldb_ldif_to_pyobject(ldif));
+			last_dn = ldif->msg->dn;
 		} else {
-			PyErr_SetString(PyExc_ValueError, "unable to parse ldif string");
+			const char *last_dn_str = NULL;
+			const char *err_string = NULL;
+			if (last_dn == NULL) {
+				PyErr_SetString(PyExc_ValueError,
+						"unable to parse LDIF "
+						"string at first chunk");
+				talloc_free(mem_ctx);
+				return NULL;
+			}
+
+			last_dn_str
+				= ldb_dn_get_linearized(last_dn);
+
+			err_string
+				= talloc_asprintf(mem_ctx,
+						  "unable to parse ldif "
+						  "string AFTER %s",
+						  last_dn_str);
+
+			PyErr_SetString(PyExc_ValueError,
+					err_string);
 			talloc_free(mem_ctx);
 			return NULL;
 		}
diff --git a/libds/common/flags.h b/libds/common/flags.h
index 88b93cb..d431bd5 100644
--- a/libds/common/flags.h
+++ b/libds/common/flags.h
@@ -190,6 +190,7 @@
 #define DS_DOMAIN_FUNCTION_2008_R2	4
 #define DS_DOMAIN_FUNCTION_2012 	5
 #define DS_DOMAIN_FUNCTION_2012_R2	6
+#define DS_DOMAIN_FUNCTION_2016		7
 
 /* sa->systemFlags on attributes */
 #define DS_FLAG_ATTR_NOT_REPLICATED    0x00000001
diff --git a/python/samba/ms_schema.py b/python/samba/ms_schema.py
index 245ce3f..a8c9363 100644
--- a/python/samba/ms_schema.py
+++ b/python/samba/ms_schema.py
@@ -162,6 +162,10 @@ def fix_dn(dn):
         dn = dn.replace("\n ", "")
         dn = dn.replace(" ", "")
         return dn.replace("CN=Schema,CN=Configuration,<RootDomainDN>", "${SCHEMADN}")
+    elif dn.endswith("DC=X"):
+        return dn.replace("CN=Schema,CN=Configuration,DC=X", "${SCHEMADN}")
+    elif dn.endswith("CN=X"):
+        return dn.replace("CN=Schema,CN=Configuration,CN=X", "${SCHEMADN}")
     else:
         return dn
 
@@ -194,7 +198,7 @@ def __write_ldif_one(entry):
         else:
             vl = l[1]
 
-        if l[0].lower() == 'omobjectclass':
+        if l[2]:
             out.append("%s:: %s" % (l[0], l[1]))
             continue
 
@@ -211,8 +215,15 @@ def __transform_entry(entry, objectClass):
     entry = [l.split(":", 1) for l in entry]
 
     cn = ""
+    skip_dn = skip_objectclass = skip_admin_description = skip_admin_display_name = False
 
     for l in entry:
+        if l[1].startswith(': '):
+            l.append(True)
+            l[1] = l[1][2:]
+        else:
+            l.append(False)
+
         key = l[0].lower()
         l[1] = l[1].lstrip()
         l[1] = l[1].rstrip()
@@ -231,25 +242,42 @@ def __transform_entry(entry, objectClass):
             l[1] = __convert_bitfield(key, l[1])
 
         if key == "omobjectclass":
-            l[1] = oMObjectClassBER[l[1].strip()]
+            if not l[2]:
+                l[1] = oMObjectClassBER[l[1].strip()]
+                l[2] = True
 
         if isinstance(l[1], str):
             l[1] = fix_dn(l[1])
 
+        if key == 'dn':
+            skip_dn = True
+            dn = l[1]
+
+        if key == 'objectclass':
+            skip_objectclass = True
+        elif key == 'admindisplayname':
+            skip_admin_display_name = True
+        elif key == 'admindescription':
+            skip_admin_description = True
 
     assert(cn)
-    entry.insert(0, ["dn", "CN=%s,${SCHEMADN}" % cn])
-    entry.insert(1, ["objectClass", ["top", objectClass]])
-    entry.insert(2, ["cn", cn])
-    entry.insert(2, ["objectGUID", str(uuid.uuid4())])
-    entry.insert(2, ["adminDescription", cn])
-    entry.insert(2, ["adminDisplayName", cn])
 
-    for l in entry:
-        key = l[0].lower()
+    header = []
+    if not skip_dn:
+        header.append(["dn", "CN=%s,${SCHEMADN}" % cn, False])
+    else:
+        header.append(["dn", dn, False])
+
+    if not skip_objectclass:
+        header.append(["objectClass", ["top", objectClass], False])
+    if not skip_admin_description:
+        header.append(["adminDescription", cn, False])
+    if not skip_admin_display_name:
+        header.append(["adminDisplayName", cn, False])
+
+    header.append(["objectGUID", str(uuid.uuid4()), False])
 
-        if key == "cn":
-            entry.remove(l)
+    entry = header + [x for x in entry if x[0].lower() not in {'dn', 'changetype', 'objectcategory'}]
 
     return entry
 
diff --git a/python/samba/ms_schema_markdown.py b/python/samba/ms_schema_markdown.py
new file mode 100644
index 0000000..c695f8b
--- /dev/null
+++ b/python/samba/ms_schema_markdown.py
@@ -0,0 +1,71 @@
+# Create schema.ldif from Github markdown
+#
+# Each LDF section in the markdown file then gets written to a corresponding
+# .LDF output file.
+#
+# Copyright (C) Andrew Bartlett 2017
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Generate LDIF from Github documentation."""
+
+import re
+import os
+import markdown
+import xml.etree.ElementTree as ET
+
+def innertext(tag):
+    return (tag.text or '') + \
+            ''.join(innertext(e) for e in tag) + \
+            (tag.tail or '')
+
+def read_ms_markdown(in_file, out_folder):
+    """Read Github documentation-derived schema files."""
+
+    with open(in_file) as update_file:
+        # Remove any comments from the raw LDF files
+        html = markdown.markdown(re.sub(r'(?m)^# .*\n?', '', update_file.read()),
+                                 output_format='xhtml')
+
+    tree = ET.fromstring('<root>' + html + '</root>')
+
+    ldf = None
+    try:
+        for node in tree:
+            if node.tag == 'h3':
+                if ldf is not None:
+                    ldf.close()
+
+                out_path = os.path.join(out_folder, innertext(node).strip())
+                ldf = open(out_path, 'w')
+            elif node.tag == 'p' and ldf is not None:
+                ldf.write(innertext(node).replace('```', '') + '\n')
+    finally:
+        if ldf is not None:
+            ldf.close()
+
+if __name__ == '__main__':
+    import sys
+
+    out_folder = ''
+
+    if len(sys.argv) == 0:
+        print >>sys.stderr, "Usage: %s <Schema-Update.md> [<output folder>]" % (sys.argv[0])
+        sys.exit(1)
+
+    in_file = sys.argv[1]
+    if len(sys.argv) > 2:
+        out_folder = sys.argv[2]
+
+    read_ms_markdown(in_file, out_folder)
diff --git a/python/samba/netcmd/domain.py b/python/samba/netcmd/domain.py
index f54b404..6f6ef61 100644
--- a/python/samba/netcmd/domain.py
+++ b/python/samba/netcmd/domain.py
@@ -33,6 +33,7 @@ import tempfile
 import logging
 import subprocess
 import time
+import shutil
 from samba import ntstatus
 from samba import NTSTATUSError
 from samba import werror
@@ -85,7 +86,8 @@ from samba.dsdb import (
 from samba.provision import (
     provision,
     ProvisioningError,
-    DEFAULT_MIN_PWD_LENGTH
+    DEFAULT_MIN_PWD_LENGTH,
+    setup_path
     )
 
 from samba.provision.common import (
@@ -232,6 +234,10 @@ class cmd_domain_provision(Command):
                 choices=["2000", "2003", "2008", "2008_R2"],
                 help="The domain and forest function level (2000 | 2003 | 2008 | 2008_R2 - always native). Default is (Windows) 2008_R2 Native.",
                 default="2008_R2"),
+         Option("--base-schema", type="choice", metavar="BASE-SCHEMA",
+                choices=["2008_R2", "2008_R2_old", "2012", "2012_R2"],
+                help="The base schema files to use. Default is (Windows) 2008_R2.",
+                default="2008_R2"),
          Option("--next-rid", type="int", metavar="NEXTRID", default=1000,
                 help="The initial nextRid value (only needed for upgrades).  Default is 1000."),
          Option("--partitions-only",
@@ -309,7 +315,8 @@ class cmd_domain_provision(Command):
             ldap_backend_nosync=None,
             ldap_backend_extra_port=None,
             ldap_backend_forced_uri=None,
-            ldap_dryrun_mode=None):
+            ldap_dryrun_mode=None,
+            base_schema=None):
 
         self.logger = self.get_logger("provision")
         if quiet:
@@ -477,7 +484,8 @@ class cmd_domain_provision(Command):
                   use_rfc2307=use_rfc2307, skip_sysvolacl=False,
                   ldap_backend_extra_port=ldap_backend_extra_port,
                   ldap_backend_forced_uri=ldap_backend_forced_uri,
-                  nosync=ldap_backend_nosync, ldap_dryrun_mode=ldap_dryrun_mode)
+                  nosync=ldap_backend_nosync, ldap_dryrun_mode=ldap_dryrun_mode,
+                  base_schema=base_schema)
 
         except ProvisioningError, e:
             raise CommandError("Provision failed", e)
@@ -3852,6 +3860,300 @@ class cmd_domain_tombstones(SuperCommand):
     subcommands = {}
     subcommands["expunge"] = cmd_domain_tombstones_expunge()
 
+class ldif_schema_update:
+    """Helper class for applying LDIF schema updates"""
+
+    def __init__(self):
+        self.is_defunct = False
+        self.unknown_oid = None
+        self.dn = None
+        self.ldif = ""
+
+    def _ldap_schemaUpdateNow(self, samdb):
+        ldif = """
+dn:
+changetype: modify
+add: schemaUpdateNow
+schemaUpdateNow: 1
+"""
+        samdb.modify_ldif(ldif)
+
+    def can_ignore_failure(self, error):
+        """Checks if we can safely ignore failure to apply an LDIF update"""
+        (num, errstr) = error.args
+
+        # Microsoft has marked objects as defunct that Samba doesn't know about
+        if num == ldb.ERR_NO_SUCH_OBJECT and self.is_defunct:
+            print("Defunct object %s doesn't exist, skipping" % self.dn)
+            return True
+        elif self.unknown_oid is not None:
+            print("Skipping unknown OID %s for object %s" %(self.unknown_oid, self.dn))
+            return True
+
+        return False
+
+    def apply(self, samdb):
+        """Applies a single LDIF update to the schema"""
+
+        try:
+            samdb.modify_ldif(self.ldif, controls=['relax:0'])
+        except ldb.LdbError as e:
+            if self.can_ignore_failure(e):
+                return 0
+            else:
+                print("Exception: %s" % e)
+                print("Encountered while trying to apply the following LDIF")
+                print("----------------------------------------------------")
+                print("%s" % self.ldif)
+
+                raise
+
+        # REFRESH AFTER EVERY CHANGE
+        # Otherwise the OID-to-attribute mapping in _apply_updates_in_file()
+        # won't work, because it can't lookup the new OID in the schema
+        self._ldap_schemaUpdateNow(samdb)
+
+        return 1
+
+class cmd_domain_schema_upgrade(Command):
+    """Domain schema upgrading"""
+
+    synopsis = "%prog [options]"
+
+    takes_optiongroups = {
+        "sambaopts": options.SambaOptions,
+        "versionopts": options.VersionOptions,
+        "credopts": options.CredentialsOptions,
+    }
+
+    takes_options = [
+        Option("-H", "--URL", help="LDB URL for database or target server", type=str,
+               metavar="URL", dest="H"),
+        Option("--quiet", help="Be quiet", action="store_true"),
+        Option("--verbose", help="Be verbose", action="store_true"),
+        Option("--schema", type="choice", metavar="SCHEMA",
+               choices=["2012", "2012_R2"],
+               help="The schema file to upgrade to. Default is (Windows) 2012_R2.",
+               default="2012_R2"),
+        Option("--ldf-file", type=str, default=None,
+                help="Just apply the schema updates in the adprep/.LDF file(s) specified"),
+        Option("--base-dir", type=str, default=None,
+               help="Location of ldf files Default is ${SETUPDIR}/adprep.")
+    ]
+
+    def _apply_updates_in_file(self, samdb, ldif_file):
+        """
+        Applies a series of updates specified in an .LDIF file. The .LDIF file
+        is based on the adprep Schema updates provided by Microsoft.
+        """
+        count = 0
+        ldif_op = ldif_schema_update()
+
+        # parse the file line by line and work out each update operation to apply
+        for line in ldif_file:
+
+            line = line.rstrip()
+
+            # the operations in the .LDIF file are separated by blank lines. If
+            # we hit a blank line, try to apply the update we've parsed so far
+            if line == '':
+
+                # keep going if we haven't parsed anything yet
+                if ldif_op.ldif == '':
+                    continue
+
+                # Apply the individual change
+                count += ldif_op.apply(samdb)
+
+                # start storing the next operation from scratch again
+                ldif_op = ldif_schema_update()
+                continue
+
+            # replace the placeholder domain name in the .ldif file with the real domain
+            if line.upper().endswith('DC=X'):
+                line = line[:-len('DC=X')] + str(samdb.get_default_basedn())
+            elif line.upper().endswith('CN=X'):
+                line = line[:-len('CN=X')] + str(samdb.get_default_basedn())
+
+            values = line.split(':')
+
+            if values[0].lower() == 'dn':
+                ldif_op.dn = values[1].strip()
+
+            # replace the Windows-specific operation with the Samba one
+            if values[0].lower() == 'changetype':
+                line = line.lower().replace(': ntdsschemaadd',
+                                            ': add')
+                line = line.lower().replace(': ntdsschemamodify',
+                                            ': modify')
+
+            if values[0].lower() in ['rdnattid', 'subclassof',
+                                     'systemposssuperiors',
+                                     'systemmaycontain',
+                                     'systemauxiliaryclass']:
+                _, value = values
+
+                # The Microsoft updates contain some OIDs we don't recognize.
+                # Query the DB to see if we can work out the OID this update is
+                # referring to. If we find a match, then replace the OID with
+                # the ldapDisplayname
+                if '.' in value:
+                    res = samdb.search(base=samdb.get_schema_basedn(),
+                                       expression="(|(attributeId=%s)(governsId=%s))" %
+                                       (value, value),
+                                       attrs=['ldapDisplayName'])
+
+                    if len(res) != 1:
+                        ldif_op.unknown_oid = value
+                    else:
+                        display_name = res[0]['ldapDisplayName'][0]
+                        line = line.replace(value, ' ' + display_name)
+
+            # Microsoft has marked objects as defunct that Samba doesn't know about
+            if values[0].lower() == 'isdefunct' and values[1].strip().lower() == 'true':
+                ldif_op.is_defunct = True
+
+            # Samba has added the showInAdvancedViewOnly attribute to all objects,
+            # so rather than doing an add, we need to do a replace
+            if values[0].lower() == 'add' and values[1].strip().lower() == 'showinadvancedviewonly':
+                line = 'replace: showInAdvancedViewOnly'
+
+            # Add the line to the current LDIF operation (including the newline
+            # we stripped off at the start of the loop)
+            ldif_op.ldif += line + '\n'
+
+        return count
+
+
+    def _apply_update(self, samdb, update_file, base_dir):
+        """Wrapper function for parsing an LDIF file and applying the updates"""
+
+        print("Applying %s updates..." % update_file)
+
+        ldif_file = None
+        try:
+            ldif_file = open(os.path.join(base_dir, update_file))
+
+            count = self._apply_updates_in_file(samdb, ldif_file)
+
+        finally:
+            if ldif_file:
+                ldif_file.close()
+
+        print("%u changes applied" % count)
+
+        return count
+
+    def run(self, **kwargs):
+        from samba.ms_schema_markdown import read_ms_markdown
+        from samba.schema import Schema
+
+        updates_allowed_overriden = False
+        sambaopts = kwargs.get("sambaopts")
+        credopts = kwargs.get("credopts")
+        versionpts = kwargs.get("versionopts")
+        lp = sambaopts.get_loadparm()
+        creds = credopts.get_credentials(lp)
+        H = kwargs.get("H")
+        target_schema = kwargs.get("schema")
+        ldf_files = kwargs.get("ldf_file")
+        base_dir = kwargs.get("base_dir")
+
+        temp_folder = None
+
+        samdb = SamDB(url=H, session_info=system_session(), credentials=creds, lp=lp)
+
+        # we're not going to get far if the config doesn't allow schema updates
+        if lp.get("dsdb:schema update allowed") is None:
+            lp.set("dsdb:schema update allowed", "yes")
+            print("Temporarily overriding 'dsdb:schema update allowed' setting")
+            updates_allowed_overriden = True
+
+        # if specific LDIF files were specified, just apply them
+        if ldf_files:
+            schema_updates = ldf_files.split(",")
+        else:
+            schema_updates = []
+
+            # work out the version of the target schema we're upgrading to
+            end = Schema.get_version(target_schema)


-- 
Samba Shared Repository



More information about the samba-cvs mailing list