olc-conversion
Oliver Liebel
oliver at itc.li
Sat Jan 17 00:40:13 GMT 2009
hi andrew,
the olc-conversion works now:
for a standalone dc and
n - dcs in multimaster-replication.
all files and diffs are attached, please take a look on it
and tell me if its okay for you.
following, a short description of the changes/additions
i made:
we have now:
--ol-olc=[yes/no] # trigger olc-conversion, needs:
--ol-slaptest=</path to slaptest binary>
if wrong path is given or slaptest doesnt exist, the script will exit
with a sys.exit
Warning: ("Path to slaptest-Binary does not exist.")
if both options are chosen,
the slapd.conf will be generated as normal and is
then converted to olc (dir: ../private/ldap/slapd.d)
in case of olc-conversion, the access to cn=config is made via
cn=samba-admin,cn=samba (write) for administration and
cn=replicator,cn=samba (read) for replication purposes.
the slapd.conf should be removed after olc-setup.
(remove-line exists in provision.py and is commented out).
as it "protects" the admin to ran in trouble using both confs (olc and
static).
if olc AND mmr are chosen,
the olc-conversion will be setup like described above,
and a small olc_seed.ldif for initial load of the other masters with slapadd
will be generated in ../private/ldap/olc_seed.ldif. but its also
possible to setup the other masters using the same provision-backend
params as on the first master. final provision can then be skipped
on the secondary dcs; after starting their slapds, they will suck out
all contexts from the first master.
what else:
i also added some new output to the post- provision-backend-messages,
depending on the chosen options to make some olc/mmr-depending params
more clear.
i added several new files to the template-system
and cleaned up / modified some of the existing (all attached).
i had to clean up the bdb-file "setup/DB_CONFIG" (no comments "#" anymore),
otherwise we will ran into the error:
"mods check (olcDbConfig: value #0 provided more than once)"
during config-replication.
it was also necessary to create a small workaround (3 lines), in case
the syncprov overlay (and directory) of olcDatabase={0}config were
not created properly during conversion.
i also added a small filter to the config-replication
(!(olcDatabase={0}config)) to avoid running into trouble
if someone tries to edit the config-syncrepl-statements;
i made the experience that slapd dislike that sometimes.
i cleaned up (expand: tabs to whsp) the complete section
from:
# generate serverids, ldap-urls...
to
# olc = yes ?
and some other lines in provision.py.diff
at last theres one thing i am not really happy with:
in case of mmr-setup with olc-conversion, the olcServerIDs
contains some trailing garbage characters after
conversion is done. i tried several things to get the urls "cleaned up";
but the chars still remains. although it doesnt seem to have
an impact on the replication,
i have filed it to openldap-technical.
thats it for now.
i think, i got me a beer or two this eve....
greetings
oliver
Andrew Bartlett schrieb:
> On Wed, 2009-01-07 at 08:17 +1100, Andrew Bartlett wrote:
>
>> On Tue, 2009-01-06 at 13:24 +0100, Oliver Liebel wrote:
>>
>>> hi andrew,
>>>
>>> what do you think about the following idea
>>> to integrate the openldap online-configuration (olc) into the
>>> provision-backend script:
>>>
>>> --ol-olc=[yes/no] # activating openldap-online-configuration [choice]
>>>
>> That sounds good. While I would prefer not to have the option (less
>> options mean less things to get wrong and test), it would mean that we
>> can't test even the generation of the configuration on hosts without a
>> slaptest binary.
>>
>
> Hmm, too many double-negatives in that sentence. Let me try again:
>
> Yes, please include the option.
>
>
>>> --ol-olc-slaptest=[path/to/slaptest/binary] # using given slaptest
>>> binary for offline-conversion [path]
>>>
>> We should just make this --ol-slaptest=
>>
>> We need to use it anyway to guess the right module configuration (this
>> is currently done in the test scripts (../selftest/target/Samba4.pm)
>>
>>
>>> if both directives are used, the generated slapd.conf will be converted
>>> after creation into the
>>> olc-format (using hardcoded directory ../private/ldap/slapd.d/)
>>>
>> Great.
>>
>>
>>> if both directives above, and the directive --olc-mmr-urls=... are set,
>>> then an olc-mmr-seed.ldif will be generated additionally, to setup the
>>> other mmr-servers (using slapadd).
>>>
>> Thanks,
>>
>> Andrew Bartlett
>>
>>
-------------- next part --------------
--- setup/provision-backend.org 2009-01-06 13:01:01.000000000 +0100
+++ setup/provision-backend 2009-01-15 23:41:00.000000000 +0100
@@ -66,7 +66,11 @@
help="Set target directory")
parser.add_option("--ol-mmr-urls", type="string", metavar="LDAPSERVER",
help="List of LDAP-URLS [ ldap://<FQDN>:port/ (where port != 389) ] separated with whitespaces for use with OpenLDAP-MMR")
-
+parser.add_option("--ol-olc", type="choice", metavar="OPENLDAP-OLC",
+ help="To setup OpenLDAP-Backend with Online-Configuration [slapd.d] choose 'yes'",
+ choices=["yes", "no"])
+parser.add_option("--ol-slaptest", type="string", metavar="SLAPTEST-PATH",
+ help="Path to slaptest-binary [Only for use with --ol-olc='yes'")
opts = parser.parse_args()[0]
@@ -103,5 +107,7 @@
root=opts.root, serverrole=server_role,
ldap_backend_type=opts.ldap_backend_type,
ldap_backend_port=opts.ldap_backend_port,
- ol_mmr_urls=opts.ol_mmr_urls)
+ ol_mmr_urls=opts.ol_mmr_urls,
+ ol_olc=opts.ol_olc,
+ ol_slaptest=opts.ol_slaptest)
-------------- next part --------------
--- scripting/python/samba/provision.org 2009-01-06 12:59:15.000000000 +0100
+++ scripting/python/samba/provision.py 2009-01-17 00:16:04.000000000 +0100
@@ -26,6 +26,7 @@
from base64 import b64encode
import os
+import sys
import pwd
import grp
import time
@@ -76,9 +77,14 @@
self.memberofconf = None
self.fedoradsinf = None
self.fedoradspartitions = None
- self.olmmron = None
- self.olmmrserveridsconf = None
- self.olmmrsyncreplconf = None
+ self.olmmron = None
+ self.olmmrserveridsconf = None
+ self.olmmrsyncreplconf = None
+ self.olcdir = None
+ self.olslaptest = None
+ self.olcseedldif = None
+ self.olcsyncprovdir = None
+ self.olcsyncprovfile = None
class ProvisionNames(object):
@@ -251,6 +257,14 @@
"mmr_serverids.conf")
paths.olmmrsyncreplconf = os.path.join(paths.ldapdir,
"mmr_syncrepl.conf")
+ paths.olcdir = os.path.join(paths.ldapdir,
+ "slapd.d")
+ paths.olcseedldif = os.path.join(paths.ldapdir,
+ "olc_seed.ldif")
+ paths.olcsyncprovdir = os.path.join(paths.olcdir,
+ "cn=config/olcDatabase={0}config")
+ paths.olcsyncprovfile = os.path.join(paths.olcsyncprovdir,
+ "olcOverlay={0}syncprov.ldif")
paths.hklm = "hklm.ldb"
paths.hkcr = "hkcr.ldb"
paths.hkcu = "hkcu.ldb"
@@ -1160,7 +1174,7 @@
rootdn=None, domaindn=None, schemadn=None, configdn=None,
domain=None, hostname=None, adminpass=None, root=None, serverrole=None,
ldap_backend_type=None, ldap_backend_port=None,
- ol_mmr_urls=None):
+ ol_mmr_urls=None,ol_olc=None,ol_slaptest=None):
def setup_path(file):
return os.path.join(setup_dir, file)
@@ -1184,6 +1198,9 @@
make_smbconf(smbconf, setup_path, hostname, domain, realm, serverrole,
targetdir)
+ if not os.path.exists(ol_slaptest):
+ sys.exit("Warning: Path to slaptest-Binary does not exist.")
+
lp = param.LoadParm()
lp.load(smbconf)
@@ -1276,52 +1293,94 @@
{ "LINK_ATTRS" : refint_attributes})
# generate serverids, ldap-urls and syncrepl-blocks for mmr hosts
- mmr_on_config = ""
- mmr_replicator_acl = ""
- mmr_serverids_config = ""
+ mmr_on_config = ""
+ mmr_replicator_acl = ""
+ mmr_serverids_config = ""
mmr_syncrepl_schema_config = ""
- mmr_syncrepl_config_config = ""
- mmr_syncrepl_user_config = ""
-
- if ol_mmr_urls is not None:
+ mmr_syncrepl_config_config = ""
+ mmr_syncrepl_user_config = ""
+
+ if ol_mmr_urls is not None:
# For now, make these equal
mmr_pass = adminpass
- url_list=filter(None,ol_mmr_urls.split(' '))
+ url_list=filter(None,ol_mmr_urls.split(' '))
if (len(url_list) == 1):
url_list=filter(None,ol_mmr_urls.split(','))
- mmr_on_config = "MirrorMode On"
- mmr_replicator_acl = " by dn=cn=replicator,cn=samba read"
- serverid=0
- for url in url_list:
- serverid=serverid+1
- mmr_serverids_config += read_and_sub_file(setup_path("mmr_serverids.conf"),
- { "SERVERID" : str(serverid),
- "LDAPSERVER" : url })
+ mmr_on_config = "MirrorMode On"
+ mmr_replicator_acl = " by dn=cn=replicator,cn=samba read"
+ serverid=0
+ for url in url_list:
+ serverid=serverid+1
+ mmr_serverids_config += read_and_sub_file(setup_path("mmr_serverids.conf"),
+ { "SERVERID" : str(serverid),
+ "LDAPSERVER" : url })
rid=serverid*10
- rid=rid+1
- mmr_syncrepl_schema_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
- { "RID" : str(rid),
- "MMRDN": names.schemadn,
- "LDAPSERVER" : url,
+ rid=rid+1
+ mmr_syncrepl_schema_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
+ { "RID" : str(rid),
+ "MMRDN": names.schemadn,
+ "LDAPSERVER" : url,
"MMR_PASSWORD": mmr_pass})
- rid=rid+1
- mmr_syncrepl_config_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
- { "RID" : str(rid),
- "MMRDN": names.configdn,
- "LDAPSERVER" : url,
+ rid=rid+1
+ mmr_syncrepl_config_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
+ { "RID" : str(rid),
+ "MMRDN": names.configdn,
+ "LDAPSERVER" : url,
"MMR_PASSWORD": mmr_pass})
- rid=rid+1
- mmr_syncrepl_user_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
- { "RID" : str(rid),
- "MMRDN": names.domaindn,
- "LDAPSERVER" : url,
+ rid=rid+1
+ mmr_syncrepl_user_config += read_and_sub_file(setup_path("mmr_syncrepl.conf"),
+ { "RID" : str(rid),
+ "MMRDN": names.domaindn,
+ "LDAPSERVER" : url,
"MMR_PASSWORD": mmr_pass })
+ # olc = yes?
+ olc_config_pass = ""
+ olc_config_acl = ""
+ olc_syncrepl_config = ""
+ olc_mmr_config = ""
+ if ol_olc == "yes":
+ olc_config_pass += read_and_sub_file(setup_path("olc_pass.conf"),
+ { "OLC_PW": adminpass })
+ olc_config_acl += read_and_sub_file(setup_path("olc_acl.conf"),{})
+
+ # if olc = yes + mmr = yes, generate cn=config-replication directives
+ # and olc_seed.lif for the other mmr-servers
+ if ol_olc == "yes" and ol_mmr_urls is not None:
+ serverid=0
+ olc_serverids_config = ""
+ olc_syncrepl_config = ""
+ olc_syncrepl_seed_config = ""
+ olc_mmr_config = ""
+ olc_mmr_config += read_and_sub_file(setup_path("olc_mmr.conf"),{})
+ rid=1000
+ for url in url_list:
+ serverid=serverid+1
+ olc_serverids_config += read_and_sub_file(setup_path("olc_serverid.conf"),
+ { "SERVERID" : str(serverid),
+ "LDAPSERVER" : url })
+
+ rid=rid+1
+ olc_syncrepl_config += read_and_sub_file(setup_path("olc_syncrepl.conf"),
+ { "RID" : str(rid),
+ "LDAPSERVER" : url,
+ "MMR_PASSWORD": adminpass})
+
+ olc_syncrepl_seed_config += read_and_sub_file(setup_path("olc_syncrepl_seed.conf"),
+ { "RID" : str(rid),
+ "LDAPSERVER" : url})
+
+ setup_file(setup_path("olc_seed.ldif"), paths.olcseedldif,
+ {"OLC_SERVER_ID_CONF": olc_serverids_config,
+ "OLC_PW": adminpass,
+ "OLC_SYNCREPL_CONF": olc_syncrepl_seed_config})
+
+ # end olc
setup_file(setup_path("slapd.conf"), paths.slapdconf,
{"DNSDOMAIN": names.dnsdomain,
@@ -1336,8 +1395,12 @@
"MMR_SYNCREPL_SCHEMA_CONFIG": mmr_syncrepl_schema_config,
"MMR_SYNCREPL_CONFIG_CONFIG": mmr_syncrepl_config_config,
"MMR_SYNCREPL_USER_CONFIG": mmr_syncrepl_user_config,
+ "OLC_CONFIG_PASS": olc_config_pass,
+ "OLC_SYNCREPL_CONFIG": olc_syncrepl_config,
+ "OLC_CONFIG_ACL": olc_config_acl,
+ "OLC_MMR_CONFIG": olc_mmr_config,
"REFINT_CONFIG": refint_config})
- setup_file(setup_path("modules.conf"), paths.modulesconf,
+ setup_file(setup_path("modules.conf"), paths.modulesconf,
{"REALM": names.realm})
setup_db_config(setup_path, os.path.join(paths.ldapdir, "db", "user"))
@@ -1356,16 +1419,15 @@
{"LDAPADMINPASS_B64": b64encode(adminpass),
"UUID": str(uuid.uuid4()),
"LDAPTIME": timestring(int(time.time()))} )
-
- if ol_mmr_urls is not None:
- setup_file(setup_path("cn=replicator.ldif"),
+
+ if ol_mmr_urls is not None:
+ setup_file(setup_path("cn=replicator.ldif"),
os.path.join(paths.ldapdir, "db", "samba", "cn=samba", "cn=replicator.ldif"),
{"MMR_PASSWORD_B64": b64encode(mmr_pass),
"UUID": str(uuid.uuid4()),
"LDAPTIME": timestring(int(time.time()))} )
-
mapping = "schema-map-openldap-2.3"
backend_schema = "backend-schema.schema"
@@ -1375,7 +1437,12 @@
else:
server_port_string = ""
- slapdcommand="Start slapd with: slapd -f " + paths.ldapdir + "/slapd.conf -h " + ldapi_uri + server_port_string
+ if ol_olc != "yes":
+ slapdcommand="Start slapd with: slapd -f " + paths.ldapdir + "/slapd.conf -h " + ldapi_uri + server_port_string
+ else:
+ slapdcommand="Start slapd with: slapd -F " + paths.olcdir + " " + ldapi_uri + server_port_string
+ if ol_olc == "yes" and ol_mmr_urls is not None:
+ slapdcommand="Start slapd with: slapd -F " + paths.olcdir + " -H ldap://<FQHN>:<PORT>"
ldapuser = "--username=samba-admin"
@@ -1397,7 +1464,35 @@
message("LDAP admin password: %s" % adminpass)
message(slapdcommand)
- message("Run provision with: --ldap-backend=ldapi --ldap-backend-type=" + ldap_backend_type + " --password=" + adminpass + " " + ldapuser)
+
+ if ol_mmr_urls is None:
+ message("Run provision with: --ldap-backend=ldapi --ldap-backend-type=" + ldap_backend_type + " --password=" + adminpass + " " + ldapuser)
+ else:
+ message("Run provision with: --ldap-backend=""ldap://<FQHN>:<PORT>"" --ldap-backend-type=" + ldap_backend_type + " --password=" + adminpass + " " + ldapuser)
+
+
+ # if --ol-olc=yes, generate online-configuration in ../private/ldap/slapd.d
+ if os.path.exists(ol_slaptest):
+ if ol_olc == "yes" and ol_slaptest is not None:
+ if not os.path.isdir(paths.olcdir):
+ os.makedirs(paths.olcdir, 0770)
+ if ol_olc == "yes" and ol_slaptest is not None:
+ paths.olslaptest = str(ol_slaptest)
+ olc_command = paths.olslaptest + " -f" + paths.slapdconf + " -F" + paths.olcdir + " >/dev/null 2>&1"
+ os.system(olc_command)
+ # os.remove(paths.slapdconf)
+ # use line below for debugging during olc-conversion with slaptest
+ # olc_command = paths.olslaptest + " -f" + paths.slapdconf + " -F" + paths.olcdir
+
+
+ # workaround, if overlay syncprov is was not created properly during conversion to cn=config.
+ # otherwise, cn=config won't be replicated
+ if not os.path.exists(paths.olcsyncprovdir):
+ os.makedirs(paths.olcsyncprovdir, 0770)
+ setup_file(setup_path("olcOverlay={0}syncprov.ldif"),
+ os.path.join(paths.olcsyncprovdir, "olcOverlay={0}syncprov.ldif"), {})
+
+
def create_phpldapadmin_config(path, setup_path, ldapi_uri):
"""Create a PHP LDAP admin configuration file.
-------------- next part --------------
loglevel 0
### needed for initial content load ###
sizelimit unlimited
### Multimaster-ServerIDs and URLs ###
${MMR_SERVERIDS_CONFIG}
include ${LDAPDIR}/backend-schema.schema
pidfile ${LDAPDIR}/slapd.pid
argsfile ${LDAPDIR}/slapd.args
sasl-realm ${DNSDOMAIN}
#authz-regexp
# uid=([^,]*),cn=${DNSDOMAIN},cn=digest-md5,cn=auth
# ldap:///${DOMAINDN}??sub?(samAccountName=\$1)
#authz-regexp
# uid=([^,]*),cn=([^,]*),cn=digest-md5,cn=auth
# ldap:///${DOMAINDN}??sub?(samAccountName=\$1)
authz-regexp
uid=([^,]*),cn=([^,]*),cn=digest-md5,cn=auth
ldap:///cn=samba??one?(cn=\$1)
authz-regexp
uid=([^,]*),cn=([^,]*),cn=ntlm,cn=auth
ldap:///cn=samba??one?(cn=\$1)
access to dn.base=""
by dn=cn=samba-admin,cn=samba manage
by anonymous read
by * read
access to dn.subtree="cn=samba"
by anonymous auth
access to dn.subtree="${DOMAINDN}"
by dn=cn=samba-admin,cn=samba manage${REPLICATOR_ACL}
by dn=cn=manager manage
by * none
password-hash {CLEARTEXT}
include ${LDAPDIR}/modules.conf
defaultsearchbase ${DOMAINDN}
rootdn cn=Manager
overlay deref
${REFINT_CONFIG}
${MEMBEROF_CONFIG}
database ldif
suffix cn=Samba
directory ${LDAPDIR}/db/samba
rootdn cn=Manager,cn=Samba
########################################
## olc - configuration ###
${OLC_CONFIG_PASS}
${OLC_SYNCREPL_CONFIG}
${OLC_MMR_CONFIG}
${OLC_CONFIG_ACL}
########################################
### cn=schema ###
database hdb
suffix ${SCHEMADN}
rootdn cn=Manager,${SCHEMADN}
directory ${LDAPDIR}/db/schema
index objectClass eq
index samAccountName eq
index name eq
index objectCategory eq
index lDAPDisplayName eq
index subClassOf eq
index cn eq
index entryUUID,entryCSN eq
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
overlay syncprov
syncprov-sessionlog 100
syncprov-checkpoint 100 10
### Multimaster-Replication of cn=schema Subcontext ###
${MMR_SYNCREPL_SCHEMA_CONFIG}
${MIRRORMODE}
#########################################
### cn=config ###
database hdb
suffix ${CONFIGDN}
rootdn cn=Manager,${CONFIGDN}
directory ${LDAPDIR}/db/config
index objectClass eq
index samAccountName eq
index name eq
index objectSid eq
index objectCategory eq
index nCName eq
index subClassOf eq
index dnsRoot eq
index nETBIOSName eq
index cn eq
index entryUUID,entryCSN eq
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
overlay syncprov
syncprov-sessionlog 100
syncprov-checkpoint 100 10
### Multimaster-Replication of cn=config Subcontext ###
${MMR_SYNCREPL_CONFIG_CONFIG}
${MIRRORMODE}
########################################
### cn=users /base-dn ###
database hdb
suffix ${DOMAINDN}
rootdn cn=Manager,${DOMAINDN}
directory ${LDAPDIR}/db/user
index objectClass eq
index samAccountName eq
index name eq
index objectSid eq
index objectCategory eq
index member eq
index uidNumber eq
index gidNumber eq
index nCName eq
index lDAPDisplayName eq
index subClassOf eq
index dnsRoot eq
index nETBIOSName eq
index cn eq
index entryUUID,entryCSN eq
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
overlay syncprov
syncprov-sessionlog 100
syncprov-checkpoint 100 10
### Multimaster-Replication of cn=user/base-dn context ###
${MMR_SYNCREPL_USER_CONFIG}
${MIRRORMODE}
-------------- next part --------------
dn: olcOverlay={0}syncprov
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
entryUUID: 41df5aca-785a-102d-9077-999999999999
creatorsName: cn=config
createTimestamp: 20090116201111Z
entryCSN: 20090116201111.111111Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20090116201111Z
-------------- next part --------------
set_cachesize 0 524288 0
set_lg_regionmax 104857
set_lg_max 1048576
set_lg_bsize 209715
set_lg_dir ${LDAPDBDIR}/bdb-logs
set_tmp_dir ${LDAPDBDIR}/tmp
-------------- next part --------------
ServerID ${SERVERID} "${LDAPSERVER}"
-------------- next part --------------
# Generated from template mmr_syncrepl.conf
syncrepl rid=${RID}
provider="${LDAPSERVER}"
searchbase="${MMRDN}"
type=refreshAndPersist
retry="10 +"
bindmethod=sasl
saslmech=DIGEST-MD5
authcid="replicator"
credentials="${MMR_PASSWORD}"
-------------- next part --------------
access to dn.sub="cn=config"
by dn="cn=samba-admin,cn=samba" write
by dn="cn=replicator,cn=samba" read
-------------- next part --------------
overlay syncprov
MirrorMode on
-------------- next part --------------
database config
rootdn cn=config
-------------- next part --------------
dn: cn=config
objectClass: olcGlobal
cn: config
${OLC_SERVER_ID_CONF}
dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=config
olcRootPW: ${OLC_PW}
${OLC_SYNCREPL_CONF}olcMirrorMode: TRUE
dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config
objectClass: olcSyncProvConfig
olcOverlay: syncprov
-------------- next part --------------
olcServerID: ${SERVERID} "${LDAPSERVER}"
-------------- next part --------------
# Generated from template olc_syncrepl.conf
syncrepl rid=${RID}
provider="${LDAPSERVER}"
searchbase="cn=config"
filter="(!(olcDatabase={0}config))"
type=refreshAndPersist
retry="10 +"
bindmethod=sasl
saslmech=DIGEST-MD5
authcid="replicator"
credentials="${MMR_PASSWORD}"
-------------- next part --------------
olcSyncRepl: rid=${RID} provider="${LDAPSERVER}"
binddn="cn=config" bindmethod=sasl saslmech=DIGEST-MD5
authcid="replicator" credentials="linux"
searchbase="cn=config" filter="(!(olcDatabase={0}config))"
type=refreshAndPersist retry="10 +"
More information about the samba-technical
mailing list