[PATCH] DNS scavenging in the AD DC

Gary Lockyer gary at catalyst.net.nz
Thu Jul 12 01:09:30 UTC 2018


And again this time, without disabling the scavenging of deleted objects.

Gary

On 12/07/18 11:47, Gary Lockyer via samba-technical wrote:
> Updated patch set:
> * adds a smb.conf variable dns_zone_scavenging to control the feature
>   off by default.
> * tidying up some other code issues
> 
> CI currently running
> https://gitlab.com/samba-team/devel/samba/pipelines/25594715
> 
> Gary.
> 
> On 11/07/18 03:37, Stefan Metzmacher via samba-technical wrote:
>> Am 10.07.2018 um 11:35 schrieb Andrew Bartlett via samba-technical:
>>> On Tue, 2018-07-10 at 09:22 +0200, Stefan Metzmacher wrote:
>>>> Am 10.07.2018 um 07:21 schrieb Andrew Bartlett via samba-technical:
>>>>> Attached here is the latest iteration of Aaron's DNS scavenging patch
>>>>> series, as cleaned up by Gary and myself.
>>>>>
>>>>> While there is much more I would like to see done (dbcheck rules to
>>>>> handle the existing records, command-line tools to change the
>>>>> settings), at this time this is a useful improvement and finally
>>>>> creates static and dynamic records correctly.
>>>>
>>>> Do you have a strategy how to detect broken records, older versions
>>>> have added?
>>>
>>> I don't have a good plan on that yet.
>>>
>>>> Can we do some magic using 110 as magic?
>>>>
>>>> git grep 110 source4/dns_server/
>>>> source4/dns_server/dns_utils.c: uint32_t dwSerial = 110;
>>>> source4/dns_server/pydns.c:     static const int serial = 110;
>>>> source4/dns_server/pydns.c:     static const int serial = 110;
>>>
>>> I'm still trying to track down what BIND9_DLZ is using.  
>>>
>>>> And change that value in the fixing patchset?
>>>
>>> Sure, I can at least do that. 
>>>
>>>> I'd really like to avoid to force a manual cleanup of this
>>>> to administrators.
>>>
>>> Understood. 
>>>
>>>> And we also have to make sure that we don't delete records
>>>> in existing setups, which where supposed to be static!
>>>
>>> Sure, but this is off by default anyway.  
>>>
>>> Should we just disable it at the smb.conf level as well until we sort
>>> out a more complete plan?
>>
>> Yes, please.
>>
>>>>> Aside from the WHATSNEW it is reviewed by Gary and myself, and the
>>>>> tests have been run against Windows and the windows static record
>>>>> behaviour has been clarified by Microsoft. 
>>>>>
>>>>> https://gitlab.com/samba-team/samba/merge_requests/26
>>>>>
>>>>> CI: https://gitlab.com/catalyst-samba/samba/pipelines/25444977
>>>
>>> Hmm, CI was unhappy.  I'll investigate. 
>>>
>>>>> Please review and push!
>>>>
>>>> Can you please add bug references to
>>>> https://bugzilla.samba.org/show_bug.cgi?id=10812
>>>> and
>>>> https://bugzilla.samba.org/show_bug.cgi?id=12451
>>>
>>> I did add both, I guess you want them on more patches?  (Easily done,
>>> tomorrow). 
>>
>> I didn't noticed the related url on the commits which fix the specific
>> bug.
>>
>> metze
>>
>>
-------------- next part --------------
From d43a9cb837a5aff7b609e9c9fb3cafc05ada96b5 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 11 Jul 2018 16:30:38 +1200
Subject: [PATCH 01/19] smb.conf: add dns_zone_scavenging

Add parameter dns_zone_scavenging to control dns zone scavenging.
Scavenging is disabled by default, as due to
https://bugzilla.samba.org/show_bug.cgi?id=12451 the ageing properties of
existing DNS entries are incorrect.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
 docs-xml/smbdotconf/domain/dnszonescavenging.xml | 23 +++++++++++++++++++++++
 lib/param/loadparm.c                             |  1 +
 source3/param/loadparm.c                         |  1 +
 3 files changed, 25 insertions(+)
 create mode 100644 docs-xml/smbdotconf/domain/dnszonescavenging.xml

diff --git a/docs-xml/smbdotconf/domain/dnszonescavenging.xml b/docs-xml/smbdotconf/domain/dnszonescavenging.xml
new file mode 100644
index 0000000..80ec144
--- /dev/null
+++ b/docs-xml/smbdotconf/domain/dnszonescavenging.xml
@@ -0,0 +1,23 @@
+<samba:parameter name="dns zone scavenging"
+                 context="G"
+                 type="boolean"
+                 xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+	<para>
+	When enabled (the default is disabled) unused dynamic dns records are
+	periodically removed.
+	</para>
+	<warning><para>
+	This option should not be enabled for installations created with
+	versions of samba before 4.9. Doing this will result in the loss of
+	static DNS entries. This is due to a bug in previous versions
+	of samba (BUG 12451) which marked dynamic DNS records as static and
+	static records as dynamic.
+	</para></warning>
+	<note><para>
+	If one record for a DNS name is static (non-aging) then no other record
+	for that DNS name will be scavenged.
+	</para></note>
+</description>
+<value type="default">no</value>
+</samba:parameter>
diff --git a/lib/param/loadparm.c b/lib/param/loadparm.c
index 75ee668..1debcff 100644
--- a/lib/param/loadparm.c
+++ b/lib/param/loadparm.c
@@ -2778,6 +2778,7 @@ struct loadparm_context *loadparm_init(TALLOC_CTX *mem_ctx)
 	lpcfg_do_global_parameter(lp_ctx, "nsupdate command", "/usr/bin/nsupdate -g");
 
         lpcfg_do_global_parameter(lp_ctx, "allow dns updates", "secure only");
+	lpcfg_do_global_parameter(lp_ctx, "dns zone scavenging", "False");
         lpcfg_do_global_parameter(lp_ctx, "dns forwarder", "");
 
 	lpcfg_do_global_parameter(lp_ctx, "algorithmic rid base", "1000");
diff --git a/source3/param/loadparm.c b/source3/param/loadparm.c
index 5f646d6..291ba57 100644
--- a/source3/param/loadparm.c
+++ b/source3/param/loadparm.c
@@ -895,6 +895,7 @@ static void init_globals(struct loadparm_context *lp_ctx, bool reinit_globals)
 	Globals._preferred_master = Auto;
 
 	Globals.allow_dns_updates = DNS_UPDATE_SIGNED;
+	Globals.dns_zone_scavenging = false;
 
 	lpcfg_string_set(Globals.ctx, &Globals.ntp_signd_socket_directory,
 			 get_dyn_NTP_SIGND_SOCKET_DIR());
-- 
2.7.4


From b8171661e5e2d047212b3de9f6a311f52db3200d Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Wed, 9 May 2018 18:02:28 +1200
Subject: [PATCH 02/19] dns: record aging tests

First basic DNS record aging tests.  These check that we can
turn aging on and off, and that timestamps are written on DNS
add and update calls, but not RPC calls.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py           | 212 ++++++++++++++++++++++++++++++++++--
 python/samba/tests/dns_base.py      |  10 +-
 selftest/knownfail.d/dns            |  16 +++
 selftest/knownfail.d/dns-scavenging |  12 ++
 4 files changed, 236 insertions(+), 14 deletions(-)
 create mode 100644 selftest/knownfail.d/dns-scavenging

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 4082b53..083dc46 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -16,6 +16,12 @@
 #
 
 from __future__ import print_function
+
+from samba import dsdb
+from samba.ndr import ndr_unpack, ndr_pack
+from samba.samdb import SamDB
+from samba.auth import system_session
+import ldb
 import os
 import sys
 import struct
@@ -652,11 +658,9 @@ class TestComplexQueries(DNSTest):
         r.rr_class = dns.DNS_QCLASS_IN
         r.ttl = 900
         r.length = 0xffff
-        rdata = value
-        r.rdata = rdata
-        updates = [r]
+        r.rdata = value
         p.nscount = 1
-        p.nsrecs = updates
+        p.nsrecs = [r]
         (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
@@ -884,11 +888,22 @@ class TestZones(DNSTest):
         self.timeout = timeout
 
         self.zone = "test.lan"
-        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" % (self.server_ip),
+        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" %\
+                                            (self.server_ip),
                                             self.lp, self.creds)
 
+        self.samdb = SamDB(url="ldap://" + self.server_ip,
+                           lp = self.get_loadparm(),
+                           session_info=system_session(),
+                           credentials=self.creds)
+
+        self.zone_dn = "DC=" + self.zone +\
+                       ",CN=MicrosoftDNS,DC=DomainDNSZones," +\
+                       str(self.samdb.get_default_basedn())
+
     def tearDown(self):
         super(TestZones, self).tearDown()
+
         try:
             self.delete_zone(self.zone)
         except RuntimeError as e:
@@ -896,15 +911,18 @@ class TestZones(DNSTest):
             if num != werror.WERR_DNS_ERROR_ZONE_DOES_NOT_EXIST:
                 raise
 
-    def create_zone(self, zone):
+    def create_zone(self, zone, aging_enabled=False):
         zone_create = dnsserver.DNS_RPC_ZONE_CREATE_INFO_LONGHORN()
         zone_create.pszZoneName = zone
         zone_create.dwZoneType = dnsp.DNS_ZONE_TYPE_PRIMARY
-        zone_create.fAllowUpdate = dnsp.DNS_ZONE_UPDATE_SECURE
-        zone_create.fAging = 0
+        zone_create.fAging = int(aging_enabled)
         zone_create.dwDpFlags = dnsserver.DNS_DP_DOMAIN_DEFAULT
+        zone_create.fDsIntegrated = 1
+        zone_create.fLoadExisting = 1
+        zone_create.fAllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE
         try:
-            self.rpc_conn.DnssrvOperation2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+            client_version = dnsserver.DNS_CLIENT_VERSION_LONGHORN
+            self.rpc_conn.DnssrvOperation2(client_version,
                                            0,
                                            self.server_ip,
                                            None,
@@ -915,6 +933,182 @@ class TestZones(DNSTest):
         except WERRORError as e:
             self.fail(str(e))
 
+    def set_params(self, **kwargs):
+        zone = kwargs.pop('zone', None)
+        for key,val in kwargs.items():
+            name_param = dnsserver.DNS_RPC_NAME_AND_PARAM()
+            name_param.dwParam = val
+            name_param.pszNodeName = key
+
+            client_version = dnsserver.DNS_CLIENT_VERSION_LONGHORN
+            nap_type = dnsserver.DNSSRV_TYPEID_NAME_AND_PARAM
+            try:
+                self.rpc_conn.DnssrvOperation2(client_version, 0, self.server, zone,
+                                               0, 'ResetDwordProperty', nap_type,
+                                               name_param)
+            except WERRORError as e:
+                self.fail(str(e))
+
+    def ldap_modify_dnsrecs(self, name, func):
+        dn = 'DC={},{}'.format(name, self.zone_dn)
+        dns_recs = self.ldap_get_dns_records(name)
+        for rec in dns_recs:
+            func(rec)
+        update_dict = {'dn':dn, 'dnsRecord':[ndr_pack(r) for r in dns_recs]}
+        self.samdb.modify(ldb.Message.from_dict(self.samdb,
+                                                update_dict,
+                                                ldb.FLAG_MOD_REPLACE))
+
+    def dns_update_record(self, prefix, txt):
+        p = self.make_txt_update(prefix, txt, self.zone)
+        (code, response) = self.dns_transaction_udp(p, host=self.server_ip)
+        self.assert_dns_rcode_equals(code, dns.DNS_RCODE_OK)
+        recs = self.ldap_get_dns_records(prefix)
+        recs = [r for r in recs if r.data.str == txt]
+        self.assertEqual(len(recs), 1)
+        return recs[0]
+
+    def ldap_get_records(self, name):
+        dn = 'DC={},{}'.format(name, self.zone_dn)
+        expr = "(&(objectClass=dnsNode)(name={}))".format(name)
+        return self.samdb.search(base=dn, scope=ldb.SCOPE_SUBTREE,
+                                 expression=expr, attrs=["*"])
+
+    def ldap_get_dns_records(self, name):
+        records = self.ldap_get_records(name)
+        return [ndr_unpack(dnsp.DnssrvRpcRecord, r)
+                for r in records[0].get('dnsRecord')]
+
+    def ldap_get_zone_settings(self):
+        records = self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_BASE,
+                   expression="(&(objectClass=dnsZone)"+\
+                                "(name={}))".format(self.zone),
+                                    attrs=["dNSProperty"])
+        self.assertEqual(len(records), 1)
+        props = [ndr_unpack(dnsp.DnsProperty, r)
+                 for r in records[0].get('dNSProperty')]
+
+        #We have no choice but to repeat these here.
+        zone_prop_ids = {0x00: "EMPTY",
+                         0x01: "TYPE",
+                         0x02: "ALLOW_UPDATE",
+                         0x08: "SECURE_TIME",
+                         0x10: "NOREFRESH_INTERVAL",
+                         0x11: "SCAVENGING_SERVERS",
+                         0x12: "AGING_ENABLED_TIME",
+                         0x20: "REFRESH_INTERVAL",
+                         0x40: "AGING_STATE",
+                         0x80: "DELETED_FROM_HOSTNAME",
+                         0x81: "MASTER_SERVERS",
+                         0x82: "AUTO_NS_SERVERS",
+                         0x83: "DCPROMO_CONVERT",
+                         0x90: "SCAVENGING_SERVERS_DA",
+                         0x91: "MASTER_SERVERS_DA",
+                         0x92: "NS_SERVERS_DA",
+                         0x100: "NODE_DBFLAGS"}
+        return {zone_prop_ids[p.id].lower(): p.data for p in props}
+
+    def set_aging(self, enable=False):
+        self.create_zone(self.zone, aging_enabled=enable)
+        self.set_params(NoRefreshInterval=1, RefreshInterval=1,
+                        Aging=int(bool(enable)), zone=self.zone,
+                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+
+    def test_set_aging(self, enable=True, name='agingtest', txt=['test txt']):
+        self.set_aging(enable=True)
+        settings = self.ldap_get_zone_settings()
+        self.assertTrue(settings['aging_state'] is not None)
+        self.assertTrue(settings['aging_state'])
+
+        rec = self.dns_update_record('agingtest', ['test txt'])
+        self.assertNotEqual(rec.dwTimeStamp, 0)
+
+    def test_set_aging_disabled(self):
+        self.set_aging(enable=False)
+        settings = self.ldap_get_zone_settings()
+        self.assertTrue(settings['aging_state'] is not None)
+        self.assertFalse(settings['aging_state'])
+
+        rec = self.dns_update_record('agingtest', ['test txt'])
+        self.assertNotEqual(rec.dwTimeStamp, 0)
+
+    def test_aging_update(self, enable=True):
+        name, txt = 'agingtest', ['test txt']
+        self.set_aging(enable=True)
+        before_mod = self.dns_update_record(name, txt)
+        if not enable:
+            self.set_params(zone=self.zone, Aging=0)
+        dec = 2
+        def mod_ts(rec):
+            rec.dwTimeStamp -= dec
+        self.ldap_modify_dnsrecs(name, mod_ts)
+        after_mod = self.ldap_get_dns_records(name)
+        self.assertEqual(len(after_mod), 1)
+        after_mod = after_mod[0]
+        self.assertEqual(after_mod.dwTimeStamp,
+                         before_mod.dwTimeStamp - dec)
+        after_update = self.dns_update_record(name, txt)
+        after_should_equal = before_mod if enable else after_mod
+        self.assertEqual(after_should_equal.dwTimeStamp,
+                         after_update.dwTimeStamp)
+
+    def test_aging_update_disabled(self):
+        self.test_aging_update(enable=False)
+
+    def test_aging_refresh(self):
+        name,txt = 'agingtest', ['test txt']
+        self.create_zone(self.zone, aging_enabled=True)
+        interval = 10
+        self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
+                        Aging=1, zone=self.zone,
+                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+        before_mod = self.dns_update_record(name, txt)
+        def mod_ts(rec):
+            rec.dwTimeStamp -= interval/2
+        self.ldap_modify_dnsrecs(name, mod_ts)
+        update_during_norefresh = self.dns_update_record(name, txt)
+        def mod_ts(rec):
+            rec.dwTimeStamp -= interval + interval/2
+        self.ldap_modify_dnsrecs(name, mod_ts)
+        update_during_refresh = self.dns_update_record(name, txt)
+        self.assertEqual(update_during_norefresh.dwTimeStamp,
+                         before_mod.dwTimeStamp - interval/2)
+        self.assertEqual(update_during_refresh.dwTimeStamp,
+                         before_mod.dwTimeStamp)
+
+    def test_rpc_add_no_timestamp(self):
+        name,txt = 'agingtest', ['test txt']
+        self.set_aging(enable=True)
+        rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
+        rec_buf.rec = TXTRecord(txt)
+        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                                          0, self.server_ip,
+                                          self.zone, name, rec_buf, None)
+        recs = self.ldap_get_dns_records(name)
+        self.assertEqual(len(recs), 1)
+        self.assertEqual(recs[0].dwTimeStamp, 0)
+
+    def test_basic_scavenging(self):
+        self.create_zone(self.zone, aging_enabled=True)
+        interval = 1
+        self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
+                        zone=self.zone, Aging=1,
+                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+        name, txt = 'agingtest', ['test txt']
+        rec = self.dns_update_record(name,txt)
+        rec = self.dns_update_record(name+'2',txt)
+        def mod_ts(rec):
+            rec.dwTimeStamp -= interval*5
+        self.ldap_modify_dnsrecs(name, mod_ts)
+        dsdb._scavenge_dns_records(self.samdb)
+
+        recs = self.ldap_get_dns_records(name)
+        self.assertEqual(len(recs), 1)
+        self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TOMBSTONE)
+        recs = self.ldap_get_dns_records(name+'2')
+        self.assertEqual(len(recs), 1)
+        self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TXT)
+
     def delete_zone(self, zone):
         self.rpc_conn.DnssrvOperation2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
                                        0,
diff --git a/python/samba/tests/dns_base.py b/python/samba/tests/dns_base.py
index 10c7a0a..0709bbe 100644
--- a/python/samba/tests/dns_base.py
+++ b/python/samba/tests/dns_base.py
@@ -166,18 +166,18 @@ class DNSTest(TestCaseInTempDir):
 
         return (response, recv_packet[2:])
 
-    def make_txt_update(self, prefix, txt_array):
+    def make_txt_update(self, prefix, txt_array, domain=None):
         p = self.make_name_packet(dns.DNS_OPCODE_UPDATE)
         updates = []
 
-        name = self.get_dns_domain()
+        name = domain or self.get_dns_domain()
         u = self.make_name_question(name, dns.DNS_QTYPE_SOA, dns.DNS_QCLASS_IN)
         updates.append(u)
         self.finish_name_packet(p, updates)
 
         updates = []
         r = dns.res_rec()
-        r.name = "%s.%s" % (prefix, self.get_dns_domain())
+        r.name = "%s.%s" % (prefix, name)
         r.rr_type = dns.DNS_QTYPE_TXT
         r.rr_class = dns.DNS_QCLASS_IN
         r.ttl = 900
@@ -190,8 +190,8 @@ class DNSTest(TestCaseInTempDir):
 
         return p
 
-    def check_query_txt(self, prefix, txt_array):
-        name = "%s.%s" % (prefix, self.get_dns_domain())
+    def check_query_txt(self, prefix, txt_array, zone=None):
+        name = "%s.%s" % (prefix, zone or self.get_dns_domain())
         p = self.make_name_packet(dns.DNS_OPCODE_QUERY)
         questions = []
 
diff --git a/selftest/knownfail.d/dns b/selftest/knownfail.d/dns
index cb30032..9fa35a2 100644
--- a/selftest/knownfail.d/dns
+++ b/selftest/knownfail.d/dns
@@ -33,7 +33,23 @@ samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_padding_rpc_to_dns\(ro
 samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_slash_rpc_to_dns\(rodc:local\)
 samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_two_rpc_to_dns\(rodc:local\)
 samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_txt_rpc_to_dns\(rodc:local\)
+
+samba.tests.dns.__main__.TestZones.test_set_aging_disabled
+
 samba.tests.dns.__main__.TestZones.test_soa_query\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_set_aging\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_refresh\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_basic_scavenging\(rodc:local\)
+
+samba.tests.dns.__main__.TestZones.test_set_aging\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_refresh\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_basic_scavenging\(vampire_dc:local\)
+
 samba.tests.dns.__main__.TestComplexQueries.test_cname_two_chain\(vampire_dc:local\)
 samba.tests.dns.__main__.TestComplexQueries.test_one_a_query\(vampire_dc:local\)
 samba.tests.dns.__main__.TestSimpleQueries.test_one_a_query\(vampire_dc:local\)
diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
new file mode 100644
index 0000000..3c4cdc9
--- /dev/null
+++ b/selftest/knownfail.d/dns-scavenging
@@ -0,0 +1,12 @@
+#
+# Tests added for the dns scavenging changes
+#
+# Will be removed once the tests are implemented.
+#
+samba.tests.dns.__main__.TestZones.test_aging_refresh\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_basic_scavenging\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_set_aging\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(vampire_dc:local\)
-- 
2.7.4


From bdbf2485879c2ea52ca6eea51e6b93f02f4c7597 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Wed, 30 May 2018 18:56:16 +1200
Subject: [PATCH 03/19] rpc dns: setting timestamp to 0 on RPC processed
 records

All records created by RPC DNS server calls should have timestamp set to 0
according to [MS-DNSP]

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12451
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett<aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 source4/rpc_server/dnsserver/dnsdb.c | 29 +++++------------------------
 1 file changed, 5 insertions(+), 24 deletions(-)

diff --git a/source4/rpc_server/dnsserver/dnsdb.c b/source4/rpc_server/dnsserver/dnsdb.c
index 81a2d20..252d39f 100644
--- a/source4/rpc_server/dnsserver/dnsdb.c
+++ b/source4/rpc_server/dnsserver/dnsdb.c
@@ -258,11 +258,6 @@ static unsigned int dnsserver_update_soa(TALLOC_CTX *mem_ctx,
 	struct ldb_message_element *el;
 	enum ndr_err_code ndr_err;
 	int ret, i, serial = -1;
-	NTTIME t;
-
-	unix_to_nt_time(&t, time(NULL));
-	t /= 10*1000*1000; /* convert to seconds (NT time is in 100ns units) */
-	t /= 3600;         /* convert to hours */
 
 	ret = ldb_search(samdb, mem_ctx, &res, z->zone_dn, LDB_SCOPE_ONELEVEL, attrs,
 			"(&(objectClass=dnsNode)(name=@))");
@@ -285,7 +280,7 @@ static unsigned int dnsserver_update_soa(TALLOC_CTX *mem_ctx,
 		if (rec.wType == DNS_TYPE_SOA) {
 			serial = rec.data.soa.serial + 1;
 			rec.dwSerial = serial;
-			rec.dwTimeStamp = (uint32_t)t;
+			rec.dwTimeStamp = 0;
 			rec.data.soa.serial = serial;
 
 			ndr_err = ndr_push_struct_blob(&el->values[i], mem_ctx, &rec,
@@ -403,7 +398,6 @@ WERROR dnsserver_db_add_record(TALLOC_CTX *mem_ctx,
 	struct ldb_message_element *el;
 	struct ldb_dn *dn;
 	enum ndr_err_code ndr_err;
-	NTTIME t;
 	int ret, i;
 	int serial;
 	WERROR werr;
@@ -431,12 +425,8 @@ WERROR dnsserver_db_add_record(TALLOC_CTX *mem_ctx,
 		return WERR_INTERNAL_DB_ERROR;
 	}
 
-	unix_to_nt_time(&t, time(NULL));
-	t /= 10*1000*1000; /* convert to seconds (NT time is in 100ns units) */
-	t /= 3600;         /* convert to hours */
-
 	rec->dwSerial = serial;
-	rec->dwTimeStamp = t;
+	rec->dwTimeStamp = 0;
 
 	ret = ldb_search(samdb, mem_ctx, &res, z->zone_dn, LDB_SCOPE_ONELEVEL, attrs,
 			"(&(objectClass=dnsNode)(name=%s))",
@@ -524,7 +514,6 @@ WERROR dnsserver_db_update_record(TALLOC_CTX *mem_ctx,
 	struct dnsp_DnssrvRpcRecord *arec = NULL, *drec = NULL;
 	struct ldb_message_element *el;
 	enum ndr_err_code ndr_err;
-	NTTIME t;
 	int ret, i;
 	int serial;
 	WERROR werr;
@@ -540,10 +529,7 @@ WERROR dnsserver_db_update_record(TALLOC_CTX *mem_ctx,
 		return werr;
 	}
 
-	unix_to_nt_time(&t, time(NULL));
-	t /= 10*1000*1000;
-
-	arec->dwTimeStamp = t;
+	arec->dwTimeStamp = 0;
 
 	ret = ldb_search(samdb, mem_ctx, &res, z->zone_dn, LDB_SCOPE_ONELEVEL, attrs,
 			"(&(objectClass=dnsNode)(name=%s)(!(dNSTombstoned=TRUE)))",
@@ -886,7 +872,6 @@ WERROR dnsserver_db_create_zone(struct ldb_context *samdb,
 	struct dnsp_DnssrvRpcRecord *dns_rec;
 	struct dnsp_soa soa;
 	char *tmpstr, *server_fqdn, *soa_email;
-	NTTIME t;
 
 	/* We only support primary zones for now */
 	if (zone->zoneinfo->dwZoneType != DNS_ZONE_TYPE_PRIMARY) {
@@ -947,10 +932,6 @@ WERROR dnsserver_db_create_zone(struct ldb_context *samdb,
 	W_ERROR_HAVE_NO_MEMORY_AND_FREE(soa_email, tmp_ctx);
 	talloc_free(tmpstr);
 
-	unix_to_nt_time(&t, time(NULL));
-	t /= 10*1000*1000; /* convert to seconds (NT time is in 100ns units) */
-	t /= 3600;         /* convert to hours */
-
 	/* SOA Record - values same as defined in provision/sambadns.py */
 	soa.serial = 1;
 	soa.refresh = 900;
@@ -964,7 +945,7 @@ WERROR dnsserver_db_create_zone(struct ldb_context *samdb,
 	dns_rec[0].rank = DNS_RANK_ZONE;
 	dns_rec[0].dwSerial = soa.serial;
 	dns_rec[0].dwTtlSeconds = 3600;
-	dns_rec[0].dwTimeStamp = (uint32_t)t;
+	dns_rec[0].dwTimeStamp = 0;
 	dns_rec[0].data.soa = soa;
 
 	/* NS Record */
@@ -972,7 +953,7 @@ WERROR dnsserver_db_create_zone(struct ldb_context *samdb,
 	dns_rec[1].rank = DNS_RANK_ZONE;
 	dns_rec[1].dwSerial = soa.serial;
 	dns_rec[1].dwTtlSeconds = 3600;
-	dns_rec[1].dwTimeStamp = (uint32_t)t;
+	dns_rec[1].dwTimeStamp = 0;
 	dns_rec[1].data.ns = server_fqdn;
 
 	/* Add @ Record */
-- 
2.7.4


From a8c2bae0380c3b30432a7adf38ef57df4091a079 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 10 Jul 2018 13:37:18 +1200
Subject: [PATCH 04/19] dns: Reformat DNS with clang-format

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 lib/ldb-samba/ldb_matching_rules.c              |  2 +-
 source4/dns_server/dlz_bind9.c                  | 19 ++++++++++++++-----
 source4/dns_server/dns_update.c                 |  1 -
 source4/dsdb/kcc/kcc_periodic.c                 |  4 ++--
 source4/rpc_server/dnsserver/dcerpc_dnsserver.c |  3 ++-
 source4/rpc_server/dnsserver/dnsdata.c          | 10 ++++++----
 source4/rpc_server/dnsserver/dnsdb.c            |  6 +++---
 source4/rpc_server/dnsserver/dnsserver.h        |  8 ++++----
 source4/rpc_server/dnsserver/dnsutils.c         | 15 ++++++++-------
 9 files changed, 40 insertions(+), 28 deletions(-)

diff --git a/lib/ldb-samba/ldb_matching_rules.c b/lib/ldb-samba/ldb_matching_rules.c
index 063a5d3..5999943 100644
--- a/lib/ldb-samba/ldb_matching_rules.c
+++ b/lib/ldb-samba/ldb_matching_rules.c
@@ -448,7 +448,7 @@ static int dsdb_match_for_expunge(struct ldb_context *ldb,
 int ldb_register_samba_matching_rules(struct ldb_context *ldb)
 {
 	struct ldb_extended_match_rule *transitive_eval = NULL,
-		*match_for_expunge = NULL;
+				       *match_for_expunge = NULL;
 	int ret;
 
 	transitive_eval = talloc_zero(ldb, struct ldb_extended_match_rule);
diff --git a/source4/dns_server/dlz_bind9.c b/source4/dns_server/dlz_bind9.c
index ac785f0..0463c6e 100644
--- a/source4/dns_server/dlz_bind9.c
+++ b/source4/dns_server/dlz_bind9.c
@@ -1488,15 +1488,18 @@ static bool dns_name_equal(const char *name1, const char *name2)
 {
 	size_t len1 = strlen(name1);
 	size_t len2 = strlen(name2);
-	if (name1[len1-1] == '.') len1--;
-	if (name2[len2-1] == '.') len2--;
+	if (name1[len1 - 1] == '.') {
+		len1--;
+	}
+	if (name2[len2 - 1] == '.') {
+		len2--;
+	}
 	if (len1 != len2) {
 		return false;
 	}
 	return strncasecmp_m(name1, name2, len1) == 0;
 }
 
-
 /*
   see if two dns records match
  */
@@ -1648,8 +1651,14 @@ _PUBLIC_ isc_result_t dlz_addrdataset(const char *name, const char *rdatastr, vo
 	}
 
 	unix_to_nt_time(&t, time(NULL));
-	t /= 10*1000*1000; /* convert to seconds (NT time is in 100ns units) */
-	t /= 3600;         /* convert to hours */
+	/*
+	 * convert to seconds (NT time is in 100ns units)
+	 */
+	t /= 10 * 1000 * 1000;
+	/*
+	 * convert to hours
+	 */
+	t /= 3600;
 
 	rec->rank        = DNS_RANK_ZONE;
 	rec->dwTimeStamp = (uint32_t)t;
diff --git a/source4/dns_server/dns_update.c b/source4/dns_server/dns_update.c
index a48f27b..ac3c3e1 100644
--- a/source4/dns_server/dns_update.c
+++ b/source4/dns_server/dns_update.c
@@ -610,7 +610,6 @@ static WERROR handle_one_update(struct dns_server *dns,
 						struct dnsp_DnssrvRpcRecord);
 			W_ERROR_HAVE_NO_MEMORY(ns_rec);
 
-
 			werror = dns_rr_to_dnsp(ns_rec, update, ns_rec);
 			W_ERROR_NOT_OK_RETURN(werror);
 
diff --git a/source4/dsdb/kcc/kcc_periodic.c b/source4/dsdb/kcc/kcc_periodic.c
index fa19ba7..855b82b 100644
--- a/source4/dsdb/kcc/kcc_periodic.c
+++ b/source4/dsdb/kcc/kcc_periodic.c
@@ -601,8 +601,8 @@ WERROR kccsrv_periodic_schedule(struct kccsrv_service *service, uint32_t next_in
 static NTSTATUS kccsrv_check_deleted(struct kccsrv_service *s, TALLOC_CTX *mem_ctx)
 {
 	time_t current_time = time(NULL);
-	time_t interval = lpcfg_parm_int(s->task->lp_ctx, NULL, "kccsrv",
-					 "check_deleted_interval", 86400);
+	time_t interval = lpcfg_parm_int(
+	    s->task->lp_ctx, NULL, "kccsrv", "check_deleted_interval", 86400);
 	uint32_t tombstoneLifetime;
 	int ret;
 	unsigned int num_objects_removed = 0;
diff --git a/source4/rpc_server/dnsserver/dcerpc_dnsserver.c b/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
index b9ed3dd..c342cca 100644
--- a/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
+++ b/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
@@ -1517,7 +1517,8 @@ static WERROR dnsserver_operate_zone(struct dnsserver_state *dsstate,
 		}
 
 		/* Ignore property resets */
-		if (strcasecmp(r->NameAndParam->pszNodeName, "AllowUpdate") == 0) {
+		if (strcasecmp(r->NameAndParam->pszNodeName, "AllowUpdate") ==
+		    0) {
 			return WERR_OK;
 		}
 		valid_operation = true;
diff --git a/source4/rpc_server/dnsserver/dnsdata.c b/source4/rpc_server/dnsserver/dnsdata.c
index a7b8e74..6889cc3 100644
--- a/source4/rpc_server/dnsserver/dnsdata.c
+++ b/source4/rpc_server/dnsserver/dnsdata.c
@@ -1119,21 +1119,23 @@ int dns_name_compare(const struct ldb_message **m1, const struct ldb_message **m
 	return strcasecmp(ptr1, ptr2);
 }
 
-
 bool dns_name_equal(const char *name1, const char *name2)
 {
 	size_t len1 = strlen(name1);
 	size_t len2 = strlen(name2);
 
-	if (len1 > 0 && name1[len1-1] == '.') len1--;
-	if (len2 > 0 && name2[len2-1] == '.') len2--;
+	if (len1 > 0 && name1[len1 - 1] == '.') {
+		len1--;
+	}
+	if (len2 > 0 && name2[len2 - 1] == '.') {
+		len2--;
+	}
 	if (len1 != len2) {
 		return false;
 	}
 	return strncasecmp(name1, name2, len1) == 0;
 }
 
-
 bool dns_record_match(struct dnsp_DnssrvRpcRecord *rec1, struct dnsp_DnssrvRpcRecord *rec2)
 {
 	bool status;
diff --git a/source4/rpc_server/dnsserver/dnsdb.c b/source4/rpc_server/dnsserver/dnsdb.c
index 252d39f..caf3079 100644
--- a/source4/rpc_server/dnsserver/dnsdb.c
+++ b/source4/rpc_server/dnsserver/dnsdb.c
@@ -86,7 +86,7 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 						struct dnsserver_partition *p)
 {
 	TALLOC_CTX *tmp_ctx;
-	const char * const attrs[] = {"name", NULL};
+	const char *const attrs[] = {"name", NULL};
 	struct ldb_dn *dn;
 	struct ldb_result *res;
 	struct dnsserver_zone *zones, *z;
@@ -122,8 +122,8 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 		}
 
 		z->partition = p;
-		name = talloc_strdup(z,
-				ldb_msg_find_attr_as_string(res->msgs[i], "name", NULL));
+		name = talloc_strdup(
+		    z, ldb_msg_find_attr_as_string(res->msgs[i], "name", NULL));
 		if (strcmp(name, "..TrustAnchors") == 0) {
 			talloc_free(z);
 			continue;
diff --git a/source4/rpc_server/dnsserver/dnsserver.h b/source4/rpc_server/dnsserver/dnsserver.h
index 0b08f08..a30d4ee 100644
--- a/source4/rpc_server/dnsserver/dnsserver.h
+++ b/source4/rpc_server/dnsserver/dnsserver.h
@@ -248,10 +248,10 @@ WERROR dnsserver_db_update_record(TALLOC_CTX *mem_ctx,
 					struct DNS_RPC_RECORD *add_record,
 					struct DNS_RPC_RECORD *del_record);
 WERROR dnsserver_db_delete_record(TALLOC_CTX *mem_ctx,
-					struct ldb_context *samdb,
-					struct dnsserver_zone *z,
-					const char *node_name,
-					struct DNS_RPC_RECORD *del_record);
+				  struct ldb_context *samdb,
+				  struct dnsserver_zone *z,
+				  const char *node_name,
+				  struct DNS_RPC_RECORD *del_record);
 WERROR dnsserver_db_create_zone(struct ldb_context *samdb,
 				struct dnsserver_partition *partitions,
 				struct dnsserver_zone *z,
diff --git a/source4/rpc_server/dnsserver/dnsutils.c b/source4/rpc_server/dnsserver/dnsutils.c
index 72b47f7..f0d7e28 100644
--- a/source4/rpc_server/dnsserver/dnsutils.c
+++ b/source4/rpc_server/dnsserver/dnsutils.c
@@ -172,15 +172,16 @@ struct dnsserver_serverinfo *dnsserver_init_serverinfo(TALLOC_CTX *mem_ctx,
 	serverinfo->dwRpcProtocol = 5;
 	serverinfo->dwNameCheckFlag = DNS_ALLOW_MULTIBYTE_NAMES;
 	serverinfo->cAddressAnswerLimit = 0;
-	serverinfo->dwRecursionRetry = 3 /* seconds (default) */;
-	serverinfo->dwRecursionTimeout = 8 /* seconds (default) */;
-	serverinfo->dwMaxCacheTtl = 0x00015180; /* 1 day (default) */;
-	serverinfo->dwDsPollingInterval = 0xB4; /* 3 minutes (default) */;
-	serverinfo->dwLocalNetPriorityNetMask = 0x000000FF;;
+	serverinfo->dwRecursionRetry = 3;       /* seconds (default) */
+	serverinfo->dwRecursionTimeout = 8;     /* seconds (default) */
+	serverinfo->dwMaxCacheTtl = 0x00015180; /* 1 day (default) */
+	serverinfo->dwDsPollingInterval = 0xB4; /* 3 minutes (default) */
+	serverinfo->dwLocalNetPriorityNetMask = 0x000000FF;
 
 	serverinfo->dwScavengingInterval = 0;
-	serverinfo->dwDefaultRefreshInterval = 0xA8; /* 7 days in hours */;
-	serverinfo->dwDefaultNoRefreshInterval = 0xA8; /* 7 days in hours */;;
+	serverinfo->dwDefaultRefreshInterval = 0xA8;   /* 7 days in hours */
+	serverinfo->dwDefaultNoRefreshInterval = 0xA8; /* 7 days in hours */
+
 	serverinfo->dwLastScavengeTime = 0;
 
 	serverinfo->fAutoReverseZones = 0;
-- 
2.7.4


From 0dc37b61fd019b6a48130210db6e045dc5b36a70 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 3 Jul 2018 15:33:06 +1200
Subject: [PATCH 05/19] rpc dns: reading zone properties from LDB

Reading zone properties from LDB on server connection initialisation, instead
of them being volatile fields.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 source4/rpc_server/dnsserver/dnsdb.c     | 33 ++++++++++++++++---
 source4/rpc_server/dnsserver/dnsserver.h |  2 ++
 source4/rpc_server/dnsserver/dnsutils.c  | 54 ++++++++++++++++++++++++++++++++
 3 files changed, 84 insertions(+), 5 deletions(-)

diff --git a/source4/rpc_server/dnsserver/dnsdb.c b/source4/rpc_server/dnsserver/dnsdb.c
index caf3079..cdd6c02 100644
--- a/source4/rpc_server/dnsserver/dnsdb.c
+++ b/source4/rpc_server/dnsserver/dnsdb.c
@@ -86,11 +86,11 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 						struct dnsserver_partition *p)
 {
 	TALLOC_CTX *tmp_ctx;
-	const char *const attrs[] = {"name", NULL};
+	const char * const attrs[] = {"name", "dNSProperty", NULL};
 	struct ldb_dn *dn;
 	struct ldb_result *res;
 	struct dnsserver_zone *zones, *z;
-	int i, ret;
+	int i, j, ret;
 
 	tmp_ctx = talloc_new(mem_ctx);
 	if (tmp_ctx == NULL) {
@@ -116,14 +116,18 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 	zones = NULL;
 	for(i=0; i<res->count; i++) {
 		char *name;
+		struct ldb_message_element *element = NULL;
+		struct dnsp_DnsProperty *props = NULL;
+		enum ndr_err_code err;
 		z = talloc_zero(mem_ctx, struct dnsserver_zone);
 		if (z == NULL) {
 			goto failed;
 		}
 
 		z->partition = p;
-		name = talloc_strdup(
-		    z, ldb_msg_find_attr_as_string(res->msgs[i], "name", NULL));
+		name = talloc_strdup(z,
+				ldb_msg_find_attr_as_string(res->msgs[i],
+							    "name", NULL));
 		if (strcmp(name, "..TrustAnchors") == 0) {
 			talloc_free(z);
 			continue;
@@ -138,8 +142,27 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 
 		DLIST_ADD_END(zones, z);
 		DEBUG(2, ("dnsserver: Found DNS zone %s\n", z->name));
-	}
 
+		element = ldb_msg_find_element(res->msgs[i], "dNSProperty");
+		if(element != NULL){
+			props = talloc_zero_array(mem_ctx,
+						  struct dnsp_DnsProperty,
+						  element->num_values);
+			for (j = 0; j < element->num_values; j++ ) {
+				err = ndr_pull_struct_blob(
+					&(element->values[j]),
+					mem_ctx,
+					&props[j],
+					(ndr_pull_flags_fn_t)
+						ndr_pull_dnsp_DnsProperty);
+				if (!NDR_ERR_CODE_IS_SUCCESS(err)){
+					goto failed;
+				}
+			}
+			z->tmp_props = props;
+			z->num_props = element->num_values;
+		}
+	}
 	return zones;
 
 failed:
diff --git a/source4/rpc_server/dnsserver/dnsserver.h b/source4/rpc_server/dnsserver/dnsserver.h
index a30d4ee..83dccf5 100644
--- a/source4/rpc_server/dnsserver/dnsserver.h
+++ b/source4/rpc_server/dnsserver/dnsserver.h
@@ -164,6 +164,8 @@ struct dnsserver_zone {
 	const char *name;
 	struct ldb_dn *zone_dn;
 	struct dnsserver_zoneinfo *zoneinfo;
+	struct dnsp_DnsProperty *tmp_props;
+	int32_t num_props;
 };
 
 
diff --git a/source4/rpc_server/dnsserver/dnsutils.c b/source4/rpc_server/dnsserver/dnsutils.c
index f0d7e28..5eb95f8 100644
--- a/source4/rpc_server/dnsserver/dnsutils.c
+++ b/source4/rpc_server/dnsserver/dnsutils.c
@@ -214,6 +214,8 @@ struct dnsserver_zoneinfo *dnsserver_init_zoneinfo(struct dnsserver_zone *zone,
 	const char *revzone = "in-addr.arpa";
 	const char *revzone6 = "ip6.arpa";
 	int len1, len2;
+	union dnsPropertyData *prop = NULL;
+	int i=0;
 
 	zoneinfo = talloc_zero(zone, struct dnsserver_zoneinfo);
 	if (zoneinfo == NULL) {
@@ -280,6 +282,58 @@ struct dnsserver_zoneinfo *dnsserver_init_zoneinfo(struct dnsserver_zone *zone,
 	zoneinfo->dwLastXfrAttempt = 0;
 	zoneinfo->dwLastXfrResult = 0;
 
+	for(i=0; i<zone->num_props; i++){
+		prop=&(zone->tmp_props[i].data);
+		switch (zone->tmp_props[i].id) {
+		case DSPROPERTY_ZONE_TYPE:
+			zoneinfo->dwZoneType =
+				prop->zone_type;
+			break;
+		case DSPROPERTY_ZONE_ALLOW_UPDATE:
+			zoneinfo->fAllowUpdate =
+				prop->allow_update_flag;
+			break;
+		case DSPROPERTY_ZONE_NOREFRESH_INTERVAL:
+			zoneinfo->dwNoRefreshInterval =
+				prop->norefresh_hours;
+			break;
+		case DSPROPERTY_ZONE_REFRESH_INTERVAL:
+			zoneinfo->dwRefreshInterval =
+				prop->refresh_hours;
+			break;
+		case DSPROPERTY_ZONE_AGING_STATE:
+			zoneinfo->fAging =
+				prop->aging_enabled;
+			break;
+		case DSPROPERTY_ZONE_SCAVENGING_SERVERS:
+			zoneinfo->aipScavengeServers->AddrCount =
+				prop->servers.addrCount;
+			zoneinfo->aipScavengeServers->AddrArray =
+				prop->servers.addr;
+			break;
+		case DSPROPERTY_ZONE_AGING_ENABLED_TIME:
+			zoneinfo->dwAvailForScavengeTime =
+				prop->next_scavenging_cycle_hours;
+			break;
+		case DSPROPERTY_ZONE_MASTER_SERVERS:
+			zoneinfo->aipLocalMasters->AddrCount =
+				prop->master_servers.addrCount;
+			zoneinfo->aipLocalMasters->AddrArray =
+				prop->master_servers.addr;
+			break;
+		case DSPROPERTY_ZONE_EMPTY:
+		case DSPROPERTY_ZONE_SECURE_TIME:
+		case DSPROPERTY_ZONE_DELETED_FROM_HOSTNAME:
+		case DSPROPERTY_ZONE_AUTO_NS_SERVERS:
+		case DSPROPERTY_ZONE_DCPROMO_CONVERT:
+		case DSPROPERTY_ZONE_SCAVENGING_SERVERS_DA:
+		case DSPROPERTY_ZONE_MASTER_SERVERS_DA:
+		case DSPROPERTY_ZONE_NS_SERVERS_DA:
+		case DSPROPERTY_ZONE_NODE_DBFLAGS:
+			break;
+		}
+	}
+
 	return zoneinfo;
 }
 
-- 
2.7.4


From fe3c49c675166d97beffa6076afd44bd4c2fb11d Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 3 Jul 2018 15:34:32 +1200
Subject: [PATCH 06/19] rpc dns: reset dword aging related zone properties

This allows a user to set zone properties relevant to DNS record aging over RPC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py                       |   4 +
 selftest/knownfail.d/dns                        |   3 +-
 selftest/knownfail.d/dns-scavenging             |   3 +-
 source4/rpc_server/dnsserver/dcerpc_dnsserver.c |  10 +-
 source4/rpc_server/dnsserver/dnsdb.c            | 123 ++++++++++++++++++++++++
 source4/rpc_server/dnsserver/dnsserver.h        |   3 +
 6 files changed, 137 insertions(+), 9 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 083dc46..722b75c 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1040,6 +1040,7 @@ class TestZones(DNSTest):
             self.set_params(zone=self.zone, Aging=0)
         dec = 2
         def mod_ts(rec):
+            self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= dec
         self.ldap_modify_dnsrecs(name, mod_ts)
         after_mod = self.ldap_get_dns_records(name)
@@ -1064,10 +1065,12 @@ class TestZones(DNSTest):
                         AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
         before_mod = self.dns_update_record(name, txt)
         def mod_ts(rec):
+            self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= interval/2
         self.ldap_modify_dnsrecs(name, mod_ts)
         update_during_norefresh = self.dns_update_record(name, txt)
         def mod_ts(rec):
+            self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= interval + interval/2
         self.ldap_modify_dnsrecs(name, mod_ts)
         update_during_refresh = self.dns_update_record(name, txt)
@@ -1098,6 +1101,7 @@ class TestZones(DNSTest):
         rec = self.dns_update_record(name,txt)
         rec = self.dns_update_record(name+'2',txt)
         def mod_ts(rec):
+            self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= interval*5
         self.ldap_modify_dnsrecs(name, mod_ts)
         dsdb._scavenge_dns_records(self.samdb)
diff --git a/selftest/knownfail.d/dns b/selftest/knownfail.d/dns
index 9fa35a2..4beeabf 100644
--- a/selftest/knownfail.d/dns
+++ b/selftest/knownfail.d/dns
@@ -34,7 +34,8 @@ samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_slash_rpc_to_dns\(rodc
 samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_two_rpc_to_dns\(rodc:local\)
 samba.tests.dns.__main__.TestRPCRoundtrip.test_update_add_txt_rpc_to_dns\(rodc:local\)
 
-samba.tests.dns.__main__.TestZones.test_set_aging_disabled
+samba.tests.dns.__main__.TestZones.test_set_aging_disabled\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_set_aging_disabled\(vampire_dc:local\)
 
 samba.tests.dns.__main__.TestZones.test_soa_query\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_set_aging\(rodc:local\)
diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
index 3c4cdc9..715e145 100644
--- a/selftest/knownfail.d/dns-scavenging
+++ b/selftest/knownfail.d/dns-scavenging
@@ -7,6 +7,5 @@ samba.tests.dns.__main__.TestZones.test_aging_refresh\(fl2003dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_update\(fl2003dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(fl2003dc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(fl2003dc:local\)
 samba.tests.dns.__main__.TestZones.test_set_aging\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_set_aging_disabled\(fl2003dc:local\)
diff --git a/source4/rpc_server/dnsserver/dcerpc_dnsserver.c b/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
index c342cca..b42d7c5 100644
--- a/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
+++ b/source4/rpc_server/dnsserver/dcerpc_dnsserver.c
@@ -1512,16 +1512,14 @@ static WERROR dnsserver_operate_zone(struct dnsserver_state *dsstate,
 	bool valid_operation = false;
 
 	if (strcasecmp(operation, "ResetDwordProperty") == 0) {
+
 		if (typeid != DNSSRV_TYPEID_NAME_AND_PARAM) {
 			return WERR_DNS_ERROR_INVALID_PROPERTY;
 		}
 
-		/* Ignore property resets */
-		if (strcasecmp(r->NameAndParam->pszNodeName, "AllowUpdate") ==
-		    0) {
-			return WERR_OK;
-		}
-		valid_operation = true;
+		return dnsserver_db_do_reset_dword(dsstate->samdb, z,
+					           r->NameAndParam);
+
 	} else if (strcasecmp(operation, "ZoneTypeReset") == 0) {
 		valid_operation = true;
 	} else if (strcasecmp(operation, "PauseZone") == 0) {
diff --git a/source4/rpc_server/dnsserver/dnsdb.c b/source4/rpc_server/dnsserver/dnsdb.c
index cdd6c02..350e29a 100644
--- a/source4/rpc_server/dnsserver/dnsdb.c
+++ b/source4/rpc_server/dnsserver/dnsdb.c
@@ -733,6 +733,129 @@ static bool dnsserver_db_msg_add_dnsproperty(TALLOC_CTX *mem_ctx,
 	return true;
 }
 
+WERROR dnsserver_db_do_reset_dword(struct ldb_context *samdb,
+				   struct dnsserver_zone *z,
+				   struct DNS_RPC_NAME_AND_PARAM *n_p)
+{
+	struct ldb_message_element *element = NULL;
+	struct dnsp_DnsProperty *prop = NULL;
+	enum ndr_err_code err;
+	TALLOC_CTX *tmp_ctx = NULL;
+	const char * const attrs[] = {"dNSProperty", NULL};
+	struct ldb_result *res = NULL;
+	int i, ret, prop_id;
+
+	if (strcasecmp(n_p->pszNodeName, "Aging") == 0) {
+		z->zoneinfo->fAging = n_p->dwParam;
+		prop_id = DSPROPERTY_ZONE_AGING_STATE;
+	} else if (strcasecmp(n_p->pszNodeName, "RefreshInterval") == 0) {
+		z->zoneinfo->dwRefreshInterval = n_p->dwParam;
+		prop_id = DSPROPERTY_ZONE_REFRESH_INTERVAL;
+	} else if (strcasecmp(n_p->pszNodeName, "NoRefreshInterval") == 0) {
+		z->zoneinfo->dwNoRefreshInterval = n_p->dwParam;
+		prop_id = DSPROPERTY_ZONE_NOREFRESH_INTERVAL;
+	} else if (strcasecmp(n_p->pszNodeName, "AllowUpdate") == 0) {
+		z->zoneinfo->fAllowUpdate = n_p->dwParam;
+		prop_id = DSPROPERTY_ZONE_ALLOW_UPDATE;
+	} else {
+		return WERR_UNKNOWN_PROPERTY;
+	}
+
+	tmp_ctx = talloc_new(NULL);
+	if (tmp_ctx == NULL) {
+		return WERR_NOT_ENOUGH_MEMORY;
+	}
+
+	ret = ldb_search(samdb, tmp_ctx, &res, z->zone_dn, LDB_SCOPE_BASE,
+			 attrs, "(objectClass=dnsZone)");
+	if (ret != LDB_SUCCESS) {
+		DBG_ERR("dnsserver: no zone: %s\n",
+			ldb_dn_get_linearized(z->zone_dn));
+		TALLOC_FREE(tmp_ctx);
+		return WERR_INTERNAL_DB_ERROR;
+	}
+
+	if (res->count != 1) {
+		DBG_ERR("dnsserver: duplicate zone: %s\n",
+			ldb_dn_get_linearized(z->zone_dn));
+		TALLOC_FREE(tmp_ctx);
+		return WERR_GEN_FAILURE;
+	}
+
+	element = ldb_msg_find_element(res->msgs[0], "dNSProperty");
+	if (element == NULL) {
+		DBG_ERR("dnsserver: zone %s has no properties.\n",
+			ldb_dn_get_linearized(z->zone_dn));
+		TALLOC_FREE(tmp_ctx);
+		return WERR_INTERNAL_DB_ERROR;
+	}
+
+	for (i = 0; i < element->num_values; i++) {
+		prop = talloc_zero(element, struct dnsp_DnsProperty);
+		if (prop == NULL) {
+			TALLOC_FREE(tmp_ctx);
+			return WERR_NOT_ENOUGH_MEMORY;
+		}
+		err = ndr_pull_struct_blob(
+			&(element->values[i]),
+			tmp_ctx,
+			prop,
+			(ndr_pull_flags_fn_t)ndr_pull_dnsp_DnsProperty);
+		if (!NDR_ERR_CODE_IS_SUCCESS(err)){
+			DBG_ERR("dnsserver: couldn't PULL dns property id "
+				"%d in zone %s\n",
+				prop->id,
+				ldb_dn_get_linearized(z->zone_dn));
+			TALLOC_FREE(tmp_ctx);
+			return WERR_INTERNAL_DB_ERROR;
+		}
+
+		if (prop->id == prop_id) {
+			switch (prop_id) {
+			case DSPROPERTY_ZONE_AGING_STATE:
+				prop->data.aging_enabled = n_p->dwParam;
+				break;
+			case DSPROPERTY_ZONE_NOREFRESH_INTERVAL:
+				prop->data.norefresh_hours = n_p->dwParam;
+				break;
+			case DSPROPERTY_ZONE_REFRESH_INTERVAL:
+				prop->data.refresh_hours = n_p->dwParam;
+				break;
+			case DSPROPERTY_ZONE_ALLOW_UPDATE:
+				prop->data.allow_update_flag = n_p->dwParam;
+				break;
+			}
+
+			err = ndr_push_struct_blob(
+				&(element->values[i]),
+				tmp_ctx,
+				prop,
+				(ndr_push_flags_fn_t)ndr_push_dnsp_DnsProperty);
+			if (!NDR_ERR_CODE_IS_SUCCESS(err)){
+				DBG_ERR("dnsserver: couldn't PUSH dns prop id "
+					"%d in zone %s\n",
+					prop->id,
+					ldb_dn_get_linearized(z->zone_dn));
+				TALLOC_FREE(tmp_ctx);
+				return WERR_INTERNAL_DB_ERROR;
+			}
+		}
+	}
+
+	element->flags = LDB_FLAG_MOD_REPLACE;
+	ret = ldb_modify(samdb, res->msgs[0]);
+	if (ret != LDB_SUCCESS) {
+		TALLOC_FREE(tmp_ctx);
+		DBG_ERR("dnsserver: Failed to modify zone %s prop %s: %s\n",
+			z->name,
+			n_p->pszNodeName,
+			ldb_errstring(samdb));
+		return WERR_INTERNAL_DB_ERROR;
+	}
+	TALLOC_FREE(tmp_ctx);
+
+	return WERR_OK;
+}
 
 /* Create dnsZone record to database and set security descriptor */
 static WERROR dnsserver_db_do_create_zone(TALLOC_CTX *tmp_ctx,
diff --git a/source4/rpc_server/dnsserver/dnsserver.h b/source4/rpc_server/dnsserver/dnsserver.h
index 83dccf5..6948fb5 100644
--- a/source4/rpc_server/dnsserver/dnsserver.h
+++ b/source4/rpc_server/dnsserver/dnsserver.h
@@ -249,6 +249,9 @@ WERROR dnsserver_db_update_record(TALLOC_CTX *mem_ctx,
 					const char *node_name,
 					struct DNS_RPC_RECORD *add_record,
 					struct DNS_RPC_RECORD *del_record);
+WERROR dnsserver_db_do_reset_dword(struct ldb_context *samdb,
+					struct dnsserver_zone *z,
+					struct DNS_RPC_NAME_AND_PARAM *n_p);
 WERROR dnsserver_db_delete_record(TALLOC_CTX *mem_ctx,
 				  struct ldb_context *samdb,
 				  struct dnsserver_zone *z,
-- 
2.7.4


From 908541eed4040a52ec6d09978c85e8a0c02ccd4a Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 5 Jun 2018 17:12:44 +1200
Subject: [PATCH 07/19] dns: moving name_equal func into common

This function is duplicated in the BIND9 and RPC DNS servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 source4/dns_server/dlz_bind9.c           | 21 +--------------------
 source4/dns_server/dns_server.h          |  1 -
 source4/dns_server/dnsserver_common.c    | 20 ++++++++++++++++++++
 source4/dns_server/dnsserver_common.h    |  1 +
 source4/rpc_server/dnsserver/dnsdata.c   | 17 -----------------
 source4/rpc_server/dnsserver/dnsserver.h |  1 -
 6 files changed, 22 insertions(+), 39 deletions(-)

diff --git a/source4/dns_server/dlz_bind9.c b/source4/dns_server/dlz_bind9.c
index 0463c6e..e55d73b 100644
--- a/source4/dns_server/dlz_bind9.c
+++ b/source4/dns_server/dlz_bind9.c
@@ -38,7 +38,7 @@
 #include <popt.h>
 #include "lib/util/dlinklist.h"
 #include "dlz_minimal.h"
-#include "dns_server/dnsserver_common.h"
+#include "dnsserver_common.h"
 
 struct b9_options {
 	const char *url;
@@ -1482,25 +1482,6 @@ _PUBLIC_ isc_boolean_t dlz_ssumatch(const char *signer, const char *name, const
 }
 
 /*
-  see if two DNS names are the same
- */
-static bool dns_name_equal(const char *name1, const char *name2)
-{
-	size_t len1 = strlen(name1);
-	size_t len2 = strlen(name2);
-	if (name1[len1 - 1] == '.') {
-		len1--;
-	}
-	if (name2[len2 - 1] == '.') {
-		len2--;
-	}
-	if (len1 != len2) {
-		return false;
-	}
-	return strncasecmp_m(name1, name2, len1) == 0;
-}
-
-/*
   see if two dns records match
  */
 static bool b9_record_match(struct dlz_bind9_data *state,
diff --git a/source4/dns_server/dns_server.h b/source4/dns_server/dns_server.h
index 05ef302..48dd5ff 100644
--- a/source4/dns_server/dns_server.h
+++ b/source4/dns_server/dns_server.h
@@ -82,7 +82,6 @@ WERROR dns_server_process_update(struct dns_server *dns,
 				 struct dns_res_rec **updates,    uint16_t *update_count,
 				 struct dns_res_rec **additional, uint16_t *arcount);
 
-bool dns_name_equal(const char *name1, const char *name2);
 bool dns_records_match(struct dnsp_DnssrvRpcRecord *rec1,
 		       struct dnsp_DnssrvRpcRecord *rec2);
 bool dns_authoritative_for_zone(struct dns_server *dns,
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 6c7ab80..20eaf12 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -1047,3 +1047,23 @@ NTSTATUS dns_common_zones(struct ldb_context *samdb,
 	TALLOC_FREE(frame);
 	return NT_STATUS_OK;
 }
+
+/*
+  see if two DNS names are the same
+ */
+bool dns_name_equal(const char *name1, const char *name2)
+{
+	size_t len1 = strlen(name1);
+	size_t len2 = strlen(name2);
+
+	if (len1 > 0 && name1[len1 - 1] == '.') {
+		len1--;
+	}
+	if (len2 > 0 && name2[len2 - 1] == '.') {
+		len2--;
+	}
+	if (len1 != len2) {
+		return false;
+	}
+	return strncasecmp(name1, name2, len1) == 0;
+}
diff --git a/source4/dns_server/dnsserver_common.h b/source4/dns_server/dnsserver_common.h
index f2be44f..e37c7b8 100644
--- a/source4/dns_server/dnsserver_common.h
+++ b/source4/dns_server/dnsserver_common.h
@@ -68,6 +68,7 @@ WERROR dns_common_name2dn(struct ldb_context *samdb,
 			  TALLOC_CTX *mem_ctx,
 			  const char *name,
 			  struct ldb_dn **_dn);
+bool dns_name_equal(const char *name1, const char *name2);
 
 /*
  * For this routine, base_dn is generally NULL.  The exception comes
diff --git a/source4/rpc_server/dnsserver/dnsdata.c b/source4/rpc_server/dnsserver/dnsdata.c
index 6889cc3..59e29f0 100644
--- a/source4/rpc_server/dnsserver/dnsdata.c
+++ b/source4/rpc_server/dnsserver/dnsdata.c
@@ -1119,23 +1119,6 @@ int dns_name_compare(const struct ldb_message **m1, const struct ldb_message **m
 	return strcasecmp(ptr1, ptr2);
 }
 
-bool dns_name_equal(const char *name1, const char *name2)
-{
-	size_t len1 = strlen(name1);
-	size_t len2 = strlen(name2);
-
-	if (len1 > 0 && name1[len1 - 1] == '.') {
-		len1--;
-	}
-	if (len2 > 0 && name2[len2 - 1] == '.') {
-		len2--;
-	}
-	if (len1 != len2) {
-		return false;
-	}
-	return strncasecmp(name1, name2, len1) == 0;
-}
-
 bool dns_record_match(struct dnsp_DnssrvRpcRecord *rec1, struct dnsp_DnssrvRpcRecord *rec2)
 {
 	bool status;
diff --git a/source4/rpc_server/dnsserver/dnsserver.h b/source4/rpc_server/dnsserver/dnsserver.h
index 6948fb5..93f1d72 100644
--- a/source4/rpc_server/dnsserver/dnsserver.h
+++ b/source4/rpc_server/dnsserver/dnsserver.h
@@ -190,7 +190,6 @@ char *dns_split_node_name(TALLOC_CTX *mem_ctx, const char *node_name, const char
 
 int dns_name_compare(const struct ldb_message **m1, const struct ldb_message **m2,
 			char *search_name);
-bool dns_name_equal(const char *name1, const char *name2);
 bool dns_record_match(struct dnsp_DnssrvRpcRecord *rec1, struct dnsp_DnssrvRpcRecord *rec2);
 
 void dnsp_to_dns_copy(TALLOC_CTX *mem_ctx, struct dnsp_DnssrvRpcRecord *dnsp,
-- 
2.7.4


From 67bfbba55d6862e9449309890db0003058ea94b5 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Mon, 2 Jul 2018 13:43:33 +1200
Subject: [PATCH 08/19] dns: server side implementation of record aging

Code for retrieving aging properties from a zone and using them for timestamp
setting logic during processing of DNS requests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py               |   1 +
 selftest/knownfail.d/dns-scavenging     |   5 --
 source4/dns_server/dns_update.c         |  10 ++-
 source4/dns_server/dnsserver_common.c   | 123 ++++++++++++++++++++++++++++++++
 source4/dns_server/dnsserver_common.h   |   6 ++
 source4/rpc_server/dnsserver/dnsutils.c |   9 ++-
 6 files changed, 145 insertions(+), 9 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 722b75c..800ce57 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1104,6 +1104,7 @@ class TestZones(DNSTest):
             self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= interval*5
         self.ldap_modify_dnsrecs(name, mod_ts)
+        self.assertTrue(callable(getattr(dsdb, '_scavenge_dns_records', None)))
         dsdb._scavenge_dns_records(self.samdb)
 
         recs = self.ldap_get_dns_records(name)
diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
index 715e145..fe71f17 100644
--- a/selftest/knownfail.d/dns-scavenging
+++ b/selftest/knownfail.d/dns-scavenging
@@ -3,9 +3,4 @@
 #
 # Will be removed once the tests are implemented.
 #
-samba.tests.dns.__main__.TestZones.test_aging_refresh\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_aging_update\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(fl2003dc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_set_aging\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_set_aging_disabled\(fl2003dc:local\)
diff --git a/source4/dns_server/dns_update.c b/source4/dns_server/dns_update.c
index ac3c3e1..ebed449 100644
--- a/source4/dns_server/dns_update.c
+++ b/source4/dns_server/dns_update.c
@@ -300,6 +300,7 @@ static WERROR dns_rr_to_dnsp(TALLOC_CTX *mem_ctx,
 			     struct dnsp_DnssrvRpcRecord *r)
 {
 	enum ndr_err_code ndr_err;
+	NTTIME t;
 
 	if (rrec->rr_type == DNS_QTYPE_ALL) {
 		return DNS_ERR(FORMAT_ERROR);
@@ -310,6 +311,10 @@ static WERROR dns_rr_to_dnsp(TALLOC_CTX *mem_ctx,
 	r->wType = (enum dns_record_type) rrec->rr_type;
 	r->dwTtlSeconds = rrec->ttl;
 	r->rank = DNS_RANK_ZONE;
+	unix_to_nt_time(&t, time(NULL));
+	t /= 10 * 1000 * 1000;
+	t /= 3600;
+	r->dwTimeStamp = t;
 
 	/* If we get QCLASS_ANY, we're done here */
 	if (rrec->rr_class == DNS_QCLASS_ANY) {
@@ -535,7 +540,10 @@ static WERROR handle_one_update(struct dns_server *dns,
 				continue;
 			}
 
-			recs[i] = recs[rcount];
+			recs[i].data = recs[rcount].data;
+			recs[i].wType = recs[rcount].wType;
+			recs[i].dwTtlSeconds = recs[rcount].dwTtlSeconds;
+			recs[i].rank = recs[rcount].rank;
 
 			werror = dns_replace_records(dns, mem_ctx, dn,
 						     needs_add, recs, rcount);
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 20eaf12..2551240 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -31,6 +31,7 @@
 #include "dsdb/samdb/samdb.h"
 #include "dsdb/common/util.h"
 #include "dns_server/dnsserver_common.h"
+#include "rpc_server/dnsserver/dnsserver.h"
 #include "lib/util/dlinklist.h"
 
 #undef DBGC_CLASS
@@ -723,6 +724,93 @@ static WERROR check_name_list(TALLOC_CTX *mem_ctx, uint16_t rec_count,
 	return WERR_OK;
 }
 
+WERROR dns_get_zone_properties(struct ldb_context *samdb,
+			       TALLOC_CTX *mem_ctx,
+			       struct ldb_dn *zone_dn,
+			       struct dnsserver_zoneinfo *zoneinfo)
+{
+
+	int ret, i;
+	struct dnsp_DnsProperty *prop = NULL;
+	struct ldb_message_element *element = NULL;
+	const char *const attrs[] = {"dNSProperty", NULL};
+	struct ldb_result *res = NULL;
+	enum ndr_err_code err;
+
+	ret = ldb_search(samdb,
+			 mem_ctx,
+			 &res,
+			 zone_dn,
+			 LDB_SCOPE_BASE,
+			 attrs,
+			 "(objectClass=dnsZone)");
+	if (ret != LDB_SUCCESS) {
+		DBG_ERR("dnsserver: Failed to find DNS zone: %s\n",
+			ldb_dn_get_linearized(zone_dn));
+		return DNS_ERR(SERVER_FAILURE);
+	}
+
+	element = ldb_msg_find_element(res->msgs[0], "dNSProperty");
+	if (element == NULL) {
+		return DNS_ERR(NOTZONE);
+	}
+
+	for (i = 0; i < element->num_values; i++) {
+		prop = talloc_zero(mem_ctx, struct dnsp_DnsProperty);
+		if (prop == NULL) {
+			return WERR_NOT_ENOUGH_MEMORY;
+		}
+		err = ndr_pull_struct_blob(
+		    &(element->values[i]),
+		    mem_ctx,
+		    prop,
+		    (ndr_pull_flags_fn_t)ndr_pull_dnsp_DnsProperty);
+		if (!NDR_ERR_CODE_IS_SUCCESS(err)) {
+			return DNS_ERR(SERVER_FAILURE);
+		}
+
+		switch (prop->id) {
+		case DSPROPERTY_ZONE_AGING_STATE:
+			zoneinfo->fAging = prop->data.aging_enabled;
+			break;
+		case DSPROPERTY_ZONE_NOREFRESH_INTERVAL:
+			zoneinfo->dwNoRefreshInterval =
+			    prop->data.norefresh_hours;
+			break;
+		case DSPROPERTY_ZONE_REFRESH_INTERVAL:
+			zoneinfo->dwRefreshInterval = prop->data.refresh_hours;
+			break;
+		case DSPROPERTY_ZONE_ALLOW_UPDATE:
+			zoneinfo->fAllowUpdate = prop->data.allow_update_flag;
+			break;
+		case DSPROPERTY_ZONE_AGING_ENABLED_TIME:
+			zoneinfo->dwAvailForScavengeTime =
+			    prop->data.next_scavenging_cycle_hours;
+			break;
+		case DSPROPERTY_ZONE_SCAVENGING_SERVERS:
+			zoneinfo->aipScavengeServers->AddrCount =
+			    prop->data.servers.addrCount;
+			zoneinfo->aipScavengeServers->AddrArray =
+			    prop->data.servers.addr;
+			break;
+		case DSPROPERTY_ZONE_EMPTY:
+		case DSPROPERTY_ZONE_TYPE:
+		case DSPROPERTY_ZONE_SECURE_TIME:
+		case DSPROPERTY_ZONE_DELETED_FROM_HOSTNAME:
+		case DSPROPERTY_ZONE_MASTER_SERVERS:
+		case DSPROPERTY_ZONE_AUTO_NS_SERVERS:
+		case DSPROPERTY_ZONE_DCPROMO_CONVERT:
+		case DSPROPERTY_ZONE_SCAVENGING_SERVERS_DA:
+		case DSPROPERTY_ZONE_MASTER_SERVERS_DA:
+		case DSPROPERTY_ZONE_NS_SERVERS_DA:
+		case DSPROPERTY_ZONE_NODE_DBFLAGS:
+			break;
+		}
+	}
+
+	return WERR_OK;
+}
+
 WERROR dns_common_replace(struct ldb_context *samdb,
 			  TALLOC_CTX *mem_ctx,
 			  struct ldb_dn *dn,
@@ -738,12 +826,37 @@ WERROR dns_common_replace(struct ldb_context *samdb,
 	struct ldb_message *msg = NULL;
 	bool was_tombstoned = false;
 	bool become_tombstoned = false;
+	struct ldb_dn *zone_dn = NULL;
+	struct dnsserver_zoneinfo *zoneinfo = NULL;
+	NTTIME t;
 
 	msg = ldb_msg_new(mem_ctx);
 	W_ERROR_HAVE_NO_MEMORY(msg);
 
 	msg->dn = dn;
 
+	zone_dn = ldb_dn_copy(mem_ctx, dn);
+	if (zone_dn == NULL) {
+		return WERR_NOT_ENOUGH_MEMORY;
+	}
+	if (!ldb_dn_remove_child_components(zone_dn, 1)) {
+		return DNS_ERR(SERVER_FAILURE);
+	}
+	zoneinfo = talloc(mem_ctx, struct dnsserver_zoneinfo);
+	if (zoneinfo == NULL) {
+		return WERR_NOT_ENOUGH_MEMORY;
+	}
+	werr = dns_get_zone_properties(samdb, mem_ctx, zone_dn, zoneinfo);
+	if (W_ERROR_EQUAL(DNS_ERR(NOTZONE), werr)) {
+		/*
+		 * We only got zoneinfo for aging so if we didn't find any
+		 * properties then just disable aging and keep going.
+		 */
+		zoneinfo->fAging = 0;
+	} else if (!W_ERROR_IS_OK(werr)) {
+		return werr;
+	}
+
 	werr = check_name_list(mem_ctx, rec_count, records);
 	if (!W_ERROR_IS_OK(werr)) {
 		return werr;
@@ -781,6 +894,16 @@ WERROR dns_common_replace(struct ldb_context *samdb,
 			continue;
 		}
 
+		if (zoneinfo->fAging == 1 && records[i].dwTimeStamp != 0) {
+			unix_to_nt_time(&t, time(NULL));
+			t /= 10 * 1000 * 1000;
+			t /= 3600;
+			if (t - records[i].dwTimeStamp >
+			    zoneinfo->dwNoRefreshInterval) {
+				records[i].dwTimeStamp = t;
+			}
+		}
+
 		records[i].dwSerial = serial;
 		ndr_err = ndr_push_struct_blob(v, el->values, &records[i],
 				(ndr_push_flags_fn_t)ndr_push_dnsp_DnssrvRpcRecord);
diff --git a/source4/dns_server/dnsserver_common.h b/source4/dns_server/dnsserver_common.h
index e37c7b8..9067e22 100644
--- a/source4/dns_server/dnsserver_common.h
+++ b/source4/dns_server/dnsserver_common.h
@@ -19,6 +19,8 @@
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
 
+#include "rpc_server/dnsserver/dnsserver.h"
+
 #ifndef __DNSSERVER_COMMON_H__
 #define __DNSSERVER_COMMON_H__
 
@@ -55,6 +57,10 @@ WERROR dns_common_wildcard_lookup(struct ldb_context *samdb,
 WERROR dns_name_check(TALLOC_CTX *mem_ctx,
 		      size_t len,
 		      const char *name);
+WERROR dns_get_zone_properties(struct ldb_context *samdb,
+			       TALLOC_CTX *mem_ctx,
+			       struct ldb_dn *zone_dn,
+			       struct dnsserver_zoneinfo *zoneinfo);
 WERROR dns_common_replace(struct ldb_context *samdb,
 			  TALLOC_CTX *mem_ctx,
 			  struct ldb_dn *dn,
diff --git a/source4/rpc_server/dnsserver/dnsutils.c b/source4/rpc_server/dnsserver/dnsutils.c
index 5eb95f8..a1c7490 100644
--- a/source4/rpc_server/dnsserver/dnsutils.c
+++ b/source4/rpc_server/dnsserver/dnsutils.c
@@ -178,9 +178,12 @@ struct dnsserver_serverinfo *dnsserver_init_serverinfo(TALLOC_CTX *mem_ctx,
 	serverinfo->dwDsPollingInterval = 0xB4; /* 3 minutes (default) */
 	serverinfo->dwLocalNetPriorityNetMask = 0x000000FF;
 
-	serverinfo->dwScavengingInterval = 0;
-	serverinfo->dwDefaultRefreshInterval = 0xA8;   /* 7 days in hours */
-	serverinfo->dwDefaultNoRefreshInterval = 0xA8; /* 7 days in hours */
+	serverinfo->dwScavengingInterval = lpcfg_parm_int(
+	    lp_ctx, NULL, "dnsserver", "ScavengingInterval", 24 * 7);
+	serverinfo->dwDefaultRefreshInterval = lpcfg_parm_int(
+	    lp_ctx, NULL, "dnsserver", "DefaultRefreshInterval", 24 * 3);
+	serverinfo->dwDefaultNoRefreshInterval = lpcfg_parm_int(
+	    lp_ctx, NULL, "dnsserver", "DefaultNoRefreshInterval", 24 * 3);
 
 	serverinfo->dwLastScavengeTime = 0;
 
-- 
2.7.4


From 9563babcc6d958953332ab274df6174fb5dcad65 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Mon, 2 Jul 2018 13:48:06 +1200
Subject: [PATCH 09/19] dns: custom match rule for DNS records to be tombstoned

A custom match rule for records to be tombstoned by the scavenging process.
Needed because DNS records are a multi-valued attribute on name records, so
without a custom match rule we'd have entire zones into memory to search for
expired records.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
 lib/ldb-samba/ldb_matching_rules.c  | 136 +++++++++++++++++++++++++++++++++++-
 lib/ldb-samba/ldb_matching_rules.h  |   1 +
 python/samba/tests/dns.py           |  46 ++++++++++++
 selftest/knownfail.d/dns            |   3 +
 selftest/knownfail.d/dns-scavenging |   1 +
 source4/setup/schema_samba4.ldif    |   1 +
 6 files changed, 187 insertions(+), 1 deletion(-)

diff --git a/lib/ldb-samba/ldb_matching_rules.c b/lib/ldb-samba/ldb_matching_rules.c
index 5999943..2aaaeb7 100644
--- a/lib/ldb-samba/ldb_matching_rules.c
+++ b/lib/ldb-samba/ldb_matching_rules.c
@@ -26,6 +26,7 @@
 #include "ldb_matching_rules.h"
 #include "libcli/security/security.h"
 #include "dsdb/common/util.h"
+#include "librpc/gen_ndr/ndr_dnsp.h"
 
 static int ldb_eval_transitive_filter_helper(TALLOC_CTX *mem_ctx,
 					     struct ldb_context *ldb,
@@ -328,6 +329,125 @@ static int ldb_comparator_trans(struct ldb_context *ldb,
 
 
 /*
+ * This rule provides match of a dns object with expired records.
+ *
+ * This allows a search filter such as:
+ *
+ * dnsRecord:1.3.6.1.4.1.7165.4.5.3:=131139216000000000
+ *
+ * This allows the caller to find records that should become a DNS
+ * tomestone, despite that information being deep within an NDR packed
+ * object
+ */
+static int dsdb_match_for_dns_to_tombstone_time(struct ldb_context *ldb,
+						const char *oid,
+						const struct ldb_message *msg,
+						const char *attribute_to_match,
+						const struct ldb_val *value_to_match,
+						bool *matched)
+{
+	TALLOC_CTX *tmp_ctx;
+	unsigned int i;
+	struct ldb_message_element *el = NULL;
+	struct auth_session_info *session_info = NULL;
+	uint64_t tombstone_time;
+	struct dnsp_DnssrvRpcRecord *rec = NULL;
+	enum ndr_err_code err;
+	*matched = false;
+
+	/* Needs to be dnsRecord, no match otherwise */
+	if (ldb_attr_cmp(attribute_to_match, "dnsRecord") != 0) {
+		return LDB_SUCCESS;
+	}
+
+	el = ldb_msg_find_element(msg, attribute_to_match);
+	if (el == NULL) {
+		return LDB_SUCCESS;
+	}
+
+	session_info = talloc_get_type(ldb_get_opaque(ldb, "sessionInfo"),
+				       struct auth_session_info);
+	if (session_info == NULL) {
+		return ldb_oom(ldb);
+	}
+	if (security_session_user_level(session_info, NULL)
+		!= SECURITY_SYSTEM) {
+
+		DBG_ERR("unauthorised access\n");
+		return LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS;
+	}
+
+	/* Just check we don't allow the caller to fill our stack */
+	if (value_to_match->length >= 64) {
+		DBG_ERR("Invalid timestamp passed\n");
+		return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+	} else {
+		char *p = NULL;
+		char s[value_to_match->length+1];
+		memcpy(s, value_to_match->data, value_to_match->length);
+		s[value_to_match->length] = 0;
+		if (s[0] == '\0' || s[0] == '-') {
+			DBG_ERR("Empty timestamp passed\n");
+			return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+		}
+		tombstone_time = strtoull(s, &p, 10);
+		if (p == NULL || p == s || *p != '\0' ||
+		    tombstone_time == ULLONG_MAX) {
+			DBG_ERR("Invalid timestamp string passed\n");
+			return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+		}
+	}
+
+	tmp_ctx = talloc_new(ldb);
+	if (tmp_ctx == NULL) {
+		return ldb_oom(ldb);
+	}
+
+	for (i = 0; i < el->num_values; i++) {
+		rec = talloc_zero(tmp_ctx, struct dnsp_DnssrvRpcRecord);
+		if (rec == NULL) {
+			TALLOC_FREE(tmp_ctx);
+			return ldb_oom(ldb);
+		}
+		err = ndr_pull_struct_blob(
+			&(el->values[i]),
+			tmp_ctx,
+			rec,
+			(ndr_pull_flags_fn_t)ndr_pull_dnsp_DnssrvRpcRecord);
+		if (!NDR_ERR_CODE_IS_SUCCESS(err)){
+			DBG_ERR("Failed to pull dns rec blob.\n");
+			TALLOC_FREE(tmp_ctx);
+			return LDB_ERR_OPERATIONS_ERROR;
+		}
+
+		if (rec->wType == DNS_TYPE_SOA || rec->wType == DNS_TYPE_NS) {
+			TALLOC_FREE(tmp_ctx);
+			continue;
+		}
+
+		if (rec->wType == DNS_TYPE_TOMBSTONE) {
+			TALLOC_FREE(tmp_ctx);
+			continue;
+		}
+		if (rec->dwTimeStamp == 0) {
+			TALLOC_FREE(tmp_ctx);
+			continue;
+		}
+		if (rec->dwTimeStamp > tombstone_time) {
+			TALLOC_FREE(tmp_ctx);
+			continue;
+		}
+
+		*matched = true;
+		break;
+	}
+
+	TALLOC_FREE(tmp_ctx);
+	return LDB_SUCCESS;
+}
+
+
+/*
  * This rule provides match of a link attribute against a 'should be expunged' criteria
  *
  * This allows a search filter such as:
@@ -448,7 +568,8 @@ static int dsdb_match_for_expunge(struct ldb_context *ldb,
 int ldb_register_samba_matching_rules(struct ldb_context *ldb)
 {
 	struct ldb_extended_match_rule *transitive_eval = NULL,
-				       *match_for_expunge = NULL;
+		*match_for_expunge = NULL,
+		*match_for_dns_to_tombstone_time = NULL;
 	int ret;
 
 	transitive_eval = talloc_zero(ldb, struct ldb_extended_match_rule);
@@ -469,5 +590,18 @@ int ldb_register_samba_matching_rules(struct ldb_context *ldb)
 		return ret;
 	}
 
+	match_for_dns_to_tombstone_time = talloc_zero(
+		ldb,
+		struct ldb_extended_match_rule);
+	match_for_dns_to_tombstone_time->oid = DSDB_MATCH_FOR_DNS_TO_TOMBSTONE_TIME;
+	match_for_dns_to_tombstone_time->callback
+		= dsdb_match_for_dns_to_tombstone_time;
+	ret = ldb_register_extended_match_rule(ldb,
+					       match_for_dns_to_tombstone_time);
+	if (ret != LDB_SUCCESS) {
+		TALLOC_FREE(match_for_dns_to_tombstone_time);
+		return ret;
+	}
+
 	return LDB_SUCCESS;
 }
diff --git a/lib/ldb-samba/ldb_matching_rules.h b/lib/ldb-samba/ldb_matching_rules.h
index 421e1ce..28c4e3d 100644
--- a/lib/ldb-samba/ldb_matching_rules.h
+++ b/lib/ldb-samba/ldb_matching_rules.h
@@ -25,5 +25,6 @@
 /* This rule provides recursive search of a link attribute */
 #define SAMBA_LDAP_MATCH_RULE_TRANSITIVE_EVAL	"1.2.840.113556.1.4.1941"
 #define DSDB_MATCH_FOR_EXPUNGE	"1.3.6.1.4.1.7165.4.5.2"
+#define DSDB_MATCH_FOR_DNS_TO_TOMBSTONE_TIME "1.3.6.1.4.1.7165.4.5.3"
 
 #endif /* _LDB_MATCHING_RULES_H_ */
diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 800ce57..5250556 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1091,6 +1091,52 @@ class TestZones(DNSTest):
         self.assertEqual(len(recs), 1)
         self.assertEqual(recs[0].dwTimeStamp, 0)
 
+    def test_dns_tombstone_custom_match_rule(self):
+        name,txt = 'agingtest', ['test txt']
+        name2,txt2 = 'agingtest2', ['test txt2']
+        name3,txt3 = 'agingtest3', ['test txt3']
+        self.create_zone(self.zone, aging_enabled=True)
+        interval = 10
+        self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
+                        Aging=1, zone=self.zone,
+                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+
+        self.dns_update_record(name, txt),
+
+        self.dns_update_record(name2, txt),
+        self.dns_update_record(name2, txt2),
+
+        self.dns_update_record(name3, txt),
+        self.dns_update_record(name3, txt2),
+        last_update = self.dns_update_record(name3, txt3)
+
+        # Modify txt1 of the first 2 names
+        def mod_ts(rec):
+            if rec.data.str == txt:
+                rec.dwTimeStamp -= 2
+        self.ldap_modify_dnsrecs(name, mod_ts)
+        self.ldap_modify_dnsrecs(name2, mod_ts)
+
+        recs = self.ldap_get_dns_records(name3)
+        expr = "(dnsRecord:1.3.6.1.4.1.7165.4.5.3:={})"
+        expr = expr.format(int(last_update.dwTimeStamp)-1)
+        try:
+            res = self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_SUBTREE,
+                                    expression=expr, attrs=["*"])
+        except ldb.LdbError as e:
+            self.fail(str(e))
+        updated_names = {str(r.get('name')) for r in res}
+        self.assertEqual(updated_names, set([name, name2]))
+
+    def test_dns_tombstone_custom_match_rule_fail(self):
+        self.create_zone(self.zone, aging_enabled=True)
+
+        # The check here is that this does not blow up on silly input
+        expr = "(dnsProperty:1.3.6.1.4.1.7165.4.5.3:=1)"
+        res = self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_SUBTREE,
+                                expression=expr, attrs=["*"])
+        self.assertEquals(len(res), 0)
+
     def test_basic_scavenging(self):
         self.create_zone(self.zone, aging_enabled=True)
         interval = 1
diff --git a/selftest/knownfail.d/dns b/selftest/knownfail.d/dns
index 4beeabf..5a3e456 100644
--- a/selftest/knownfail.d/dns
+++ b/selftest/knownfail.d/dns
@@ -44,12 +44,15 @@ samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_refresh\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule_fail\(rodc:local\)
 
 samba.tests.dns.__main__.TestZones.test_set_aging\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_update\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_refresh\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(vampire_dc:local\)
 
 samba.tests.dns.__main__.TestComplexQueries.test_cname_two_chain\(vampire_dc:local\)
 samba.tests.dns.__main__.TestComplexQueries.test_one_a_query\(vampire_dc:local\)
diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
index fe71f17..8de3101 100644
--- a/selftest/knownfail.d/dns-scavenging
+++ b/selftest/knownfail.d/dns-scavenging
@@ -4,3 +4,4 @@
 # Will be removed once the tests are implemented.
 #
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(fl2003dc:local\)
+samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(fl2003dc:local\)
diff --git a/source4/setup/schema_samba4.ldif b/source4/setup/schema_samba4.ldif
index 5b26dc0..648f049 100644
--- a/source4/setup/schema_samba4.ldif
+++ b/source4/setup/schema_samba4.ldif
@@ -246,6 +246,7 @@
 # ldap extended matches
 #Allocated: SAMBA_LDAP_MATCH_ALWAYS_FALSE 1.3.6.1.4.1.7165.4.5.1
 #Allocated: DSDB_MATCH_FOR_EXPUNGE 1.3.6.1.4.1.7165.4.5.2
+#Allocated: DSDB_MATCH_FOR_DNS_TO_TOMBSTONE_TIME 1.3.6.1.4.1.7165.4.5.3
 
 
 #Allocated: (middleName) attributeID: 1.3.6.1.4.1.7165.4.255.1
-- 
2.7.4


From c3e40e130236610eb47efca8cb14fbd66492f7ed Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 10 Jul 2018 13:23:42 +1200
Subject: [PATCH 10/19] dns: Use ldb.SCOPE_SUBTREE in ldap_get_records()
 routine in tests/dns.py

DNS records have the odd property that the DN can be reliably determined by the
name only, so we do not need a subtree search.

However by using a subtree search under the zone we can without
trapping exceptions confirm if the record exists or not in the tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
---
 python/samba/tests/dns.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 5250556..c0f71db 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -969,9 +969,11 @@ class TestZones(DNSTest):
         return recs[0]
 
     def ldap_get_records(self, name):
-        dn = 'DC={},{}'.format(name, self.zone_dn)
+        # The use of SCOPE_SUBTREE here avoids raising an exception in the
+        # 0 results case for a test below.
+
         expr = "(&(objectClass=dnsNode)(name={}))".format(name)
-        return self.samdb.search(base=dn, scope=ldb.SCOPE_SUBTREE,
+        return self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_SUBTREE,
                                  expression=expr, attrs=["*"])
 
     def ldap_get_dns_records(self, name):
-- 
2.7.4


From 0a326b74ed6c3e0406a0f62b2e2948a8a777d7c8 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Fri, 1 Jun 2018 16:07:46 +1200
Subject: [PATCH 11/19] dns: dns record scavenging function (without task)

DNS record scavenging function with testing.  The logic of the custom match rule
in previous commit is inverted so that calculations using zone properties can
be taken out of the function's inner loop. Periodic task to come.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py               |  49 +++-
 selftest/knownfail.d/dns-scavenging     |   2 -
 source4/dsdb/kcc/kcc_service.h          |   1 +
 source4/dsdb/kcc/scavenge_dns_records.c | 441 ++++++++++++++++++++++++++++++++
 source4/dsdb/kcc/scavenge_dns_records.h |  48 ++++
 source4/dsdb/pydsdb.c                   |  73 ++++++
 source4/dsdb/wscript_build              |  10 +-
 source4/rpc_server/dnsserver/dnsdb.c    |   2 +-
 8 files changed, 617 insertions(+), 9 deletions(-)
 create mode 100644 source4/dsdb/kcc/scavenge_dns_records.c
 create mode 100644 source4/dsdb/kcc/scavenge_dns_records.h

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index c0f71db..faf4c52 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1094,6 +1094,11 @@ class TestZones(DNSTest):
         self.assertEqual(recs[0].dwTimeStamp, 0)
 
     def test_dns_tombstone_custom_match_rule(self):
+        lp = self.get_loadparm()
+        self.samdb = SamDB(url = lp.samdb_url(), lp = lp,
+                           session_info=system_session(),
+                           credentials=self.creds)
+
         name,txt = 'agingtest', ['test txt']
         name2,txt2 = 'agingtest2', ['test txt2']
         name3,txt3 = 'agingtest3', ['test txt3']
@@ -1140,27 +1145,63 @@ class TestZones(DNSTest):
         self.assertEquals(len(res), 0)
 
     def test_basic_scavenging(self):
+        lp = self.get_loadparm()
+        self.samdb = SamDB(url = lp.samdb_url(), lp = lp,
+                           session_info=system_session(),
+                           credentials=self.creds)
+
         self.create_zone(self.zone, aging_enabled=True)
         interval = 1
         self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
                         zone=self.zone, Aging=1,
                         AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
         name, txt = 'agingtest', ['test txt']
-        rec = self.dns_update_record(name,txt)
-        rec = self.dns_update_record(name+'2',txt)
+        name2, txt2 = 'agingtest2', ['test txt2']
+        name3, txt3 = 'agingtest3', ['test txt3']
+        self.dns_update_record(name,txt)
+        self.dns_update_record(name2,txt)
+        self.dns_update_record(name2,txt2)
+        self.dns_update_record(name3,txt)
+        self.dns_update_record(name3,txt2)
+        last_add = self.dns_update_record(name3,txt3)
+
         def mod_ts(rec):
             self.assertTrue(rec.dwTimeStamp > 0)
-            rec.dwTimeStamp -= interval*5
+            if rec.data.str == txt:
+                rec.dwTimeStamp -= interval*5
         self.ldap_modify_dnsrecs(name, mod_ts)
+        self.ldap_modify_dnsrecs(name2, mod_ts)
+        self.ldap_modify_dnsrecs(name3, mod_ts)
         self.assertTrue(callable(getattr(dsdb, '_scavenge_dns_records', None)))
         dsdb._scavenge_dns_records(self.samdb)
 
         recs = self.ldap_get_dns_records(name)
         self.assertEqual(len(recs), 1)
         self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TOMBSTONE)
-        recs = self.ldap_get_dns_records(name+'2')
+
+        recs = self.ldap_get_dns_records(name2)
         self.assertEqual(len(recs), 1)
         self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TXT)
+        self.assertEqual(recs[0].data.str, txt2)
+
+        recs = self.ldap_get_dns_records(name3)
+        self.assertEqual(len(recs), 2)
+        txts = {str(r.data.str) for r in recs}
+        self.assertEqual(txts, {str(txt2),str(txt3)})
+        self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TXT)
+        self.assertEqual(recs[1].wType, dnsp.DNS_TYPE_TXT)
+
+        for make_it_work in [False, True]:
+            inc = -1 if make_it_work else 1
+            def mod_ts(rec):
+                rec.data = (last_add.dwTimeStamp - 24*14) + inc
+            self.ldap_modify_dnsrecs(name, mod_ts)
+            dsdb._dns_delete_tombstones(self.samdb)
+            recs = self.ldap_get_records(name)
+            if make_it_work:
+                self.assertEqual(len(recs), 0)
+            else:
+                self.assertEqual(len(recs), 1)
 
     def delete_zone(self, zone):
         self.rpc_conn.DnssrvOperation2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
index 8de3101..86c0fa8 100644
--- a/selftest/knownfail.d/dns-scavenging
+++ b/selftest/knownfail.d/dns-scavenging
@@ -3,5 +3,3 @@
 #
 # Will be removed once the tests are implemented.
 #
-samba.tests.dns.__main__.TestZones.test_basic_scavenging\(fl2003dc:local\)
-samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(fl2003dc:local\)
diff --git a/source4/dsdb/kcc/kcc_service.h b/source4/dsdb/kcc/kcc_service.h
index b62fb12..a456903 100644
--- a/source4/dsdb/kcc/kcc_service.h
+++ b/source4/dsdb/kcc/kcc_service.h
@@ -91,6 +91,7 @@ struct kccsrv_service {
 struct kcc_connection_list;
 
 #include "dsdb/kcc/garbage_collect_tombstones.h"
+#include "dsdb/kcc/scavenge_dns_records.h"
 #include "dsdb/kcc/kcc_service_proto.h"
 
 #endif /* _DSDB_REPL_KCC_SERVICE_H_ */
diff --git a/source4/dsdb/kcc/scavenge_dns_records.c b/source4/dsdb/kcc/scavenge_dns_records.c
new file mode 100644
index 0000000..2f4f482
--- /dev/null
+++ b/source4/dsdb/kcc/scavenge_dns_records.c
@@ -0,0 +1,441 @@
+/*
+   Unix SMB/CIFS implementation.
+
+   DNS tombstoning routines
+
+   Copyright (C) Andrew Bartlett <abartlet at samba.org> 2018
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+*/
+
+#include "includes.h"
+#include <ldb_errors.h>
+#include "../lib/util/dlinklist.h"
+#include "librpc/gen_ndr/ndr_misc.h"
+#include "librpc/gen_ndr/ndr_drsuapi.h"
+#include "librpc/gen_ndr/ndr_drsblobs.h"
+#include "param/param.h"
+#include "lib/util/dlinklist.h"
+#include "ldb.h"
+#include "dsdb/kcc/scavenge_dns_records.h"
+#include "lib/ldb-samba/ldb_matching_rules.h"
+#include "lib/util/time.h"
+#include "dns_server/dnsserver_common.h"
+#include "librpc/gen_ndr/ndr_dnsp.h"
+#include "param/param.h"
+
+#include "librpc/gen_ndr/ndr_misc.h"
+#include "librpc/gen_ndr/ndr_drsuapi.h"
+#include "librpc/gen_ndr/ndr_drsblobs.h"
+
+/*
+ * Copy only non-expired dns records from one message element to another.
+ */
+NTSTATUS copy_current_records(TALLOC_CTX *mem_ctx,
+			      struct ldb_message_element *old_el,
+			      struct ldb_message_element *el,
+			      NTTIME t)
+{
+	unsigned int i, num_kept = 0;
+	struct dnsp_DnssrvRpcRecord *recs = NULL;
+	enum ndr_err_code ndr_err;
+	TALLOC_CTX *tmp_ctx = talloc_new(NULL);
+
+	if (tmp_ctx == NULL) {
+		return NT_STATUS_NO_MEMORY;
+	}
+
+	recs = talloc_zero_array(
+	    tmp_ctx, struct dnsp_DnssrvRpcRecord, el->num_values);
+	if (recs == NULL) {
+		TALLOC_FREE(tmp_ctx);
+		return NT_STATUS_NO_MEMORY;
+	}
+
+	for (i = 0; i < el->num_values; i++) {
+		ndr_err = ndr_pull_struct_blob(
+		    &(old_el->values[i]),
+		    tmp_ctx,
+		    &(recs[num_kept]),
+		    (ndr_pull_flags_fn_t)ndr_pull_dnsp_DnssrvRpcRecord);
+		if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+			TALLOC_FREE(tmp_ctx);
+			DBG_ERR("Failed to pull dns rec blob.\n");
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+		if (recs[num_kept].dwTimeStamp > t ||
+		    recs[num_kept].dwTimeStamp == 0) {
+			num_kept++;
+		}
+	}
+
+	if (num_kept == el->num_values) {
+		TALLOC_FREE(tmp_ctx);
+		return NT_STATUS_OK;
+	}
+
+	el->values = talloc_zero_array(mem_ctx, struct ldb_val, num_kept);
+	if (el->values == NULL) {
+		TALLOC_FREE(tmp_ctx);
+		return NT_STATUS_NO_MEMORY;
+	}
+	el->num_values = num_kept;
+	for (i = 0; i < el->num_values; i++) {
+		ndr_err = ndr_push_struct_blob(
+		    &(el->values[i]),
+		    mem_ctx,
+		    &(recs[i]),
+		    (ndr_push_flags_fn_t)ndr_push_dnsp_DnssrvRpcRecord);
+		if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+			TALLOC_FREE(tmp_ctx);
+			DBG_ERR("Failed to push dnsp_DnssrvRpcRecord\n");
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+	}
+
+	TALLOC_FREE(tmp_ctx);
+	return NT_STATUS_OK;
+}
+
+/*
+ * Check all records in a zone and tombstone them if they're expired.
+ */
+NTSTATUS dns_tombstone_records_zone(TALLOC_CTX *mem_ctx,
+				    struct ldb_context *samdb,
+				    struct dns_server_zone *zone,
+				    struct ldb_val *true_struct,
+				    struct ldb_val *tombstone_blob,
+				    NTTIME t,
+				    char **error_string)
+{
+	WERROR werr;
+	NTSTATUS status;
+	unsigned int i;
+	struct dnsserver_zoneinfo *zi = NULL;
+	struct ldb_result *res = NULL;
+	struct ldb_message_element *el = NULL;
+	struct ldb_message_element *tombstone_el = NULL;
+	struct ldb_message_element *old_el = NULL;
+	int ret;
+	struct GUID guid;
+	struct GUID_txt_buf buf_guid;
+	const char *attrs[] = {"dnsRecord",
+			       "dNSTombstoned",
+			       "objectGUID",
+			       NULL};
+
+	*error_string = NULL;
+
+	/* Get NoRefreshInterval and RefreshInterval from zone properties.*/
+	zi = talloc(mem_ctx, struct dnsserver_zoneinfo);
+	if (zi == NULL) {
+		return NT_STATUS_NO_MEMORY;
+	}
+	werr = dns_get_zone_properties(samdb, mem_ctx, zone->dn, zi);
+	if (W_ERROR_EQUAL(DNS_ERR(NOTZONE), werr)) {
+		return NT_STATUS_PROPSET_NOT_FOUND;
+	} else if (!W_ERROR_IS_OK(werr)) {
+		return NT_STATUS_INTERNAL_ERROR;
+	}
+
+	/* Subtract them from current time to get the earliest possible.
+	 * timestamp allowed for a non-expired DNS record. */
+	t -= zi->dwNoRefreshInterval + zi->dwRefreshInterval;
+
+	/* Custom match gets dns records in the zone with dwTimeStamp < t. */
+	ret = ldb_search(samdb,
+			 mem_ctx,
+			 &res,
+			 zone->dn,
+			 LDB_SCOPE_SUBTREE,
+			 attrs,
+			 "(&(objectClass=dnsNode)"
+			 "(&(!(dnsTombstoned=TRUE))"
+			 "(dnsRecord:" DSDB_MATCH_FOR_DNS_TO_TOMBSTONE_TIME
+			 ":=%lu)))",
+			 t);
+	if (ret != LDB_SUCCESS) {
+		*error_string = talloc_asprintf(mem_ctx,
+						"Failed to search for dns "
+						"objects in zone %s: %s",
+						ldb_dn_get_linearized(zone->dn),
+						ldb_errstring(samdb));
+		return NT_STATUS_INTERNAL_ERROR;
+	}
+
+	/*
+	 * Do a constrained update on each expired DNS node. To do a constrained
+	 * update we leave the dnsRecord element as is, and just change the flag
+	 * to MOD_DELETE, then add a new element with the changes we want.  LDB
+	 * will run the deletion first, and bail out if a binary comparison
+	 * between the attribute we pass and the one in the database shows a
+	 * change.  This prevents race conditions.
+	 */
+	for (i = 0; i < res->count; i++) {
+		old_el = ldb_msg_find_element(res->msgs[i], "dnsRecord");
+		old_el->flags = LDB_FLAG_MOD_DELETE;
+
+		ret = ldb_msg_add_empty(
+		    res->msgs[i], "dnsRecord", LDB_FLAG_MOD_ADD, &el);
+		if (ret != LDB_SUCCESS) {
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+
+		el->num_values = old_el->num_values;
+		status = copy_current_records(mem_ctx, old_el, el, t);
+
+		if (!NT_STATUS_IS_OK(status)) {
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+
+		/* If nothing was expired, do nothing. */
+		if (el->num_values == old_el->num_values &&
+		    el->num_values != 0) {
+			continue;
+		}
+
+		el->flags = LDB_FLAG_MOD_ADD;
+
+		/* If everything was expired, we tombstone the node. */
+		if (el->num_values == 0) {
+			el->values = tombstone_blob;
+			el->num_values = 1;
+
+			tombstone_el = ldb_msg_find_element(res->msgs[i],
+						  "dnsTombstoned");
+			if (tombstone_el == NULL) {
+				ret = ldb_msg_add_value(res->msgs[i],
+							"dnsTombstoned",
+							true_struct,
+							&tombstone_el);
+				if (ret != LDB_SUCCESS) {
+					return NT_STATUS_INTERNAL_ERROR;
+				}
+				tombstone_el->flags = LDB_FLAG_MOD_ADD;
+			} else {
+				tombstone_el->flags = LDB_FLAG_MOD_REPLACE;
+				tombstone_el->values = true_struct;
+			}
+			tombstone_el->num_values = 1;
+		} else {
+			/*
+			 * Do not change the status of dnsTombstoned
+			 * if we found any live records
+			 */
+			ldb_msg_remove_attr(res->msgs[i],
+					    "dnsTombstoned");
+		}
+
+		/* Set DN to the GUID in case the object was moved. */
+		el = ldb_msg_find_element(res->msgs[i], "objectGUID");
+		if (el == NULL) {
+			*error_string =
+			    talloc_asprintf(mem_ctx,
+					    "record has no objectGUID "
+					    "in zone %s",
+					    ldb_dn_get_linearized(zone->dn));
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+
+		status = GUID_from_ndr_blob(el->values, &guid);
+		if (!NT_STATUS_IS_OK(status)) {
+			*error_string =
+			    discard_const_p(char, "Error: Invalid GUID.\n");
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+
+		GUID_buf_string(&guid, &buf_guid);
+		res->msgs[i]->dn =
+		    ldb_dn_new_fmt(mem_ctx, samdb, "<GUID=%s>", buf_guid.buf);
+
+		/* Remove the GUID so we're not trying to modify it. */
+		ldb_msg_remove_attr(res->msgs[i], "objectGUID");
+
+		ret = ldb_modify(samdb, res->msgs[i]);
+		if (ret != LDB_SUCCESS) {
+			*error_string =
+			    talloc_asprintf(mem_ctx,
+					    "Failed to modify dns record "
+					    "in zone %s: %s",
+					    ldb_dn_get_linearized(zone->dn),
+					    ldb_errstring(samdb));
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+	}
+
+	return NT_STATUS_OK;
+}
+
+/*
+ * Tombstone all expired DNS records.
+ */
+NTSTATUS dns_tombstone_records(TALLOC_CTX *mem_ctx,
+			       struct ldb_context *samdb,
+			       char **error_string)
+{
+	struct dns_server_zone *zones = NULL;
+	struct dns_server_zone *z = NULL;
+	NTSTATUS ret;
+	struct dnsp_DnssrvRpcRecord tombstone_struct;
+	struct ldb_val tombstone_blob;
+	struct ldb_val true_struct;
+	NTTIME t;
+	enum ndr_err_code ndr_err;
+	TALLOC_CTX *tmp_ctx = NULL;
+	uint8_t true_str[4] = "TRUE";
+
+	unix_to_nt_time(&t, time(NULL));
+	t /= 10 * 1000 * 1000;
+	t /= 3600;
+
+	tombstone_struct = (struct dnsp_DnssrvRpcRecord){
+	    .wType = DNS_TYPE_TOMBSTONE, .data = {.timestamp = t}};
+
+	true_struct = (struct ldb_val){.data = true_str, .length = 4};
+
+	ndr_err = ndr_push_struct_blob(
+	    &tombstone_blob,
+	    mem_ctx,
+	    &tombstone_struct,
+	    (ndr_push_flags_fn_t)ndr_push_dnsp_DnssrvRpcRecord);
+	if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+		*error_string = discard_const_p(char,
+						"Failed to push "
+						"dnsp_DnssrvRpcRecord\n");
+		return NT_STATUS_INTERNAL_ERROR;
+	}
+
+	dns_common_zones(samdb, mem_ctx, NULL, &zones);
+	for (z = zones; z; z = z->next) {
+		tmp_ctx = talloc_new(NULL);
+		ret = dns_tombstone_records_zone(tmp_ctx,
+						 samdb,
+						 z,
+						 &true_struct,
+						 &tombstone_blob,
+						 t,
+						 error_string);
+		TALLOC_FREE(tmp_ctx);
+		if (NT_STATUS_EQUAL(ret, NT_STATUS_PROPSET_NOT_FOUND)) {
+			continue;
+		} else if (!NT_STATUS_IS_OK(ret)) {
+			return ret;
+		}
+	}
+	return NT_STATUS_OK;
+}
+
+/*
+ * Delete all DNS tombstones that have been around for longer than the server
+ * property 'DsTombstoneInterval' which we store in smb.conf
+ */
+NTSTATUS dns_delete_tombstones(TALLOC_CTX *mem_ctx,
+			       struct ldb_context *samdb,
+			       char **error_string)
+{
+	struct dns_server_zone *zones = NULL;
+	struct dns_server_zone *z = NULL;
+	int ret, i;
+	NTTIME current_time;
+	enum ndr_err_code ndr_err;
+	struct ldb_result *res = NULL;
+	int tombstone_time;
+	TALLOC_CTX *tmp_ctx = NULL;
+	struct loadparm_context *lp_ctx = NULL;
+	struct ldb_message_element *el = NULL;
+	struct dnsp_DnssrvRpcRecord *rec = NULL;
+	const char *attrs[] = {"dnsRecord", "dNSTombstoned", NULL};
+	rec = talloc_zero(mem_ctx, struct dnsp_DnssrvRpcRecord);
+
+	unix_to_nt_time(&current_time, time(NULL));
+	current_time /= 10 * 1000 * 1000;
+	current_time /= 3600;
+
+	lp_ctx = (struct loadparm_context *)ldb_get_opaque(samdb, "loadparm");
+	tombstone_time =
+	    current_time -
+	    lpcfg_parm_int(
+		lp_ctx, NULL, "dnsserver", "dns_tombstone_interval", 24 * 14);
+
+	dns_common_zones(samdb, mem_ctx, NULL, &zones);
+	for (z = zones; z; z = z->next) {
+		tmp_ctx = talloc_new(NULL);
+		if (tmp_ctx == NULL) {
+			return NT_STATUS_NO_MEMORY;
+		}
+
+		/*
+		 * This can load a very large set, but on the
+		 * assumption that the number of tombstones is
+		 * relatively small compared with the number of active
+		 * records, and that this is an indexed lookup, this
+		 * should be OK.  We can make a match rule if
+		 * returning the set of tombstones becomes an issue.
+		 */
+
+		ret = ldb_search(samdb,
+				 tmp_ctx,
+				 &res,
+				 z->dn,
+				 LDB_SCOPE_SUBTREE,
+				 attrs,
+				 "(&(objectClass=dnsNode)(dNSTombstoned=TRUE))");
+
+		if (ret != LDB_SUCCESS) {
+			TALLOC_FREE(tmp_ctx);
+			*error_string =
+			    talloc_asprintf(mem_ctx,
+					    "Failed to "
+					    "search for tombstoned "
+					    "dns objects in zone %s: %s",
+					    ldb_dn_get_linearized(z->dn),
+					    ldb_errstring(samdb));
+			return NT_STATUS_INTERNAL_ERROR;
+		}
+
+		for (i = 0; i < res->count; i++) {
+			el = ldb_msg_find_element(res->msgs[i], "dnsRecord");
+			ndr_err = ndr_pull_struct_blob(
+			    el->values,
+			    tmp_ctx,
+			    rec,
+			    (ndr_pull_flags_fn_t)ndr_pull_dnsp_DnssrvRpcRecord);
+			if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+				TALLOC_FREE(tmp_ctx);
+				DBG_ERR("Failed to pull dns rec blob.\n");
+				return NT_STATUS_INTERNAL_ERROR;
+			}
+
+			if (rec->wType != DNS_TYPE_TOMBSTONE) {
+				continue;
+			}
+
+			if (rec->data.timestamp > tombstone_time) {
+				continue;
+			}
+
+			ret = dsdb_delete(samdb, res->msgs[i]->dn, 0);
+			if (ret != LDB_ERR_NO_SUCH_OBJECT &&
+			    ret != LDB_SUCCESS) {
+				TALLOC_FREE(tmp_ctx);
+				DBG_ERR("Failed to delete dns node \n");
+				return NT_STATUS_INTERNAL_ERROR;
+			}
+		}
+
+		TALLOC_FREE(tmp_ctx);
+	}
+	return NT_STATUS_OK;
+}
diff --git a/source4/dsdb/kcc/scavenge_dns_records.h b/source4/dsdb/kcc/scavenge_dns_records.h
new file mode 100644
index 0000000..799fedb
--- /dev/null
+++ b/source4/dsdb/kcc/scavenge_dns_records.h
@@ -0,0 +1,48 @@
+/*
+   Unix SMB/CIFS implementation.
+
+   DNS tombstoning routines
+
+   Copyright (C) Andrew Bartlett <abartlet at samba.org> 2018
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+*/
+#include "param/param.h"
+#include "dsdb/samdb/samdb.h"
+#include "dsdb/common/util.h"
+#include "dns_server/dnsserver_common.h"
+
+NTSTATUS dns_tombstone_records(TALLOC_CTX *mem_ctx,
+			       struct ldb_context *samdb,
+			       char **error_string);
+
+NTSTATUS dns_delete_tombstones(TALLOC_CTX *mem_ctx,
+			       struct ldb_context *samdb,
+			       char **error_string);
+NTSTATUS remove_expired_records(TALLOC_CTX *mem_ctx,
+				struct ldb_message_element *el,
+				NTTIME t);
+NTSTATUS dns_tombstone_records_zone(TALLOC_CTX *mem_ctx,
+				    struct ldb_context *samdb,
+				    struct dns_server_zone *zone,
+				    struct ldb_val *true_struct,
+				    struct ldb_val *tombstone_blob,
+				    NTTIME t,
+				    char **error_string);
+
+NTSTATUS copy_current_records(TALLOC_CTX *mem_ctx,
+			      struct ldb_message_element *old_el,
+			      struct ldb_message_element *el,
+			      NTTIME t);
diff --git a/source4/dsdb/pydsdb.c b/source4/dsdb/pydsdb.c
index a25f341..62d2a91 100644
--- a/source4/dsdb/pydsdb.c
+++ b/source4/dsdb/pydsdb.c
@@ -32,6 +32,7 @@
 #include "param/pyparam.h"
 #include "lib/util/dlinklist.h"
 #include "dsdb/kcc/garbage_collect_tombstones.h"
+#include "dsdb/kcc/scavenge_dns_records.h"
 
 
 /* FIXME: These should be in a header file somewhere */
@@ -1150,6 +1151,74 @@ static PyObject *py_dsdb_allocate_rid(PyObject *self, PyObject *args)
 	return PyInt_FromLong(rid);
 }
 
+static PyObject *py_dns_delete_tombstones(PyObject *self, PyObject *args)
+{
+	PyObject *py_ldb;
+	NTSTATUS status;
+	struct ldb_context *ldb = NULL;
+	TALLOC_CTX *mem_ctx = NULL;
+	char *error_string = NULL;
+
+	if (!PyArg_ParseTuple(args, "O", &py_ldb)) {
+		return NULL;
+	}
+	PyErr_LDB_OR_RAISE(py_ldb, ldb);
+
+	mem_ctx = talloc_new(ldb);
+	if (mem_ctx == NULL) {
+		return PyErr_NoMemory();
+	}
+
+	status = dns_delete_tombstones(mem_ctx, ldb, &error_string);
+
+	if (!NT_STATUS_IS_OK(status)) {
+		if (error_string) {
+			PyErr_Format(PyExc_RuntimeError, "%s", error_string);
+		} else {
+			PyErr_SetNTSTATUS(status);
+		}
+		TALLOC_FREE(mem_ctx);
+		return NULL;
+	}
+
+	TALLOC_FREE(mem_ctx);
+	Py_RETURN_NONE;
+}
+
+static PyObject *py_scavenge_dns_records(PyObject *self, PyObject *args)
+{
+	PyObject *py_ldb;
+	NTSTATUS status;
+	struct ldb_context *ldb = NULL;
+	TALLOC_CTX *mem_ctx = NULL;
+	char *error_string = NULL;
+
+	if (!PyArg_ParseTuple(args, "O", &py_ldb)) {
+		return NULL;
+	}
+	PyErr_LDB_OR_RAISE(py_ldb, ldb);
+
+	mem_ctx = talloc_new(ldb);
+	if (mem_ctx == NULL) {
+		return PyErr_NoMemory();
+	}
+
+	status = dns_tombstone_records(mem_ctx, ldb, &error_string);
+
+	if (!NT_STATUS_IS_OK(status)) {
+		if (error_string) {
+			PyErr_Format(PyExc_RuntimeError, "%s", error_string);
+		} else {
+			PyErr_SetNTSTATUS(status);
+		}
+		TALLOC_FREE(mem_ctx);
+		return NULL;
+	}
+
+	TALLOC_FREE(mem_ctx);
+	Py_RETURN_NONE;
+}
+
 static PyObject *py_dsdb_garbage_collect_tombstones(PyObject *self, PyObject *args)
 {
 	PyObject *py_ldb, *py_list_dn;
@@ -1383,6 +1452,10 @@ static PyMethodDef py_dsdb_methods[] = {
 	{ "_dsdb_garbage_collect_tombstones", (PyCFunction)py_dsdb_garbage_collect_tombstones, METH_VARARGS,
 		"_dsdb_kcc_check_deleted(samdb, [dn], current_time, tombstone_lifetime)"
 		" -> (num_objects_expunged, num_links_expunged)" },
+	{ "_scavenge_dns_records", (PyCFunction)py_scavenge_dns_records,
+		METH_VARARGS, NULL},
+	{ "_dns_delete_tombstones", (PyCFunction)py_dns_delete_tombstones,
+		METH_VARARGS, NULL},
 	{ "_dsdb_create_own_rid_set", (PyCFunction)py_dsdb_create_own_rid_set, METH_VARARGS,
 		"_dsdb_create_own_rid_set(samdb)"
 		" -> None" },
diff --git a/source4/dsdb/wscript_build b/source4/dsdb/wscript_build
index e1f1e61..be99e99 100644
--- a/source4/dsdb/wscript_build
+++ b/source4/dsdb/wscript_build
@@ -42,12 +42,17 @@ bld.SAMBA_LIBRARY('dsdb_garbage_collect_tombstones',
                   deps='samdb RPC_NDR_DRSUAPI',
                   private_library=True)
 
+bld.SAMBA_LIBRARY('scavenge_dns_records',
+                  source='kcc/scavenge_dns_records.c',
+                  deps='samdb RPC_NDR_DRSUAPI dnsserver_common',
+                  private_library=True)
+
 bld.SAMBA_MODULE('service_kcc',
 	source='kcc/kcc_service.c kcc/kcc_connection.c kcc/kcc_periodic.c kcc/kcc_drs_replica_info.c',
 	autoproto='kcc/kcc_service_proto.h',
 	subsystem='service',
 	init_function='server_service_kcc_init',
-	deps='samdb process_model RPC_NDR_IRPC RPC_NDR_DRSUAPI UTIL_RUNCMD dsdb_garbage_collect_tombstones',
+	deps='samdb process_model RPC_NDR_IRPC RPC_NDR_DRSUAPI UTIL_RUNCMD dsdb_garbage_collect_tombstones scavenge_dns_records',
 	internal_module=False,
 	enabled=bld.AD_DC_BUILD_IS_ENABLED()
 	)
@@ -71,6 +76,7 @@ for env in bld.gen_python_environments():
 		# the dependency on dcerpc here is because gensec
 		# depends on dcerpc but the waf circular dependency finder
 		# removes it so we end up with unresolved symbols.
-		deps='samdb %s dcerpc com_err %s %s dsdb_garbage_collect_tombstones' % (pyldb_util, pyrpc_util, pyparam_util),
+		deps='samdb %s dcerpc com_err %s %s dsdb_garbage_collect_tombstones scavenge_dns_records' %\
+			 (pyldb_util, pyrpc_util, pyparam_util),
 		realname='samba/dsdb.so'
 		)
diff --git a/source4/rpc_server/dnsserver/dnsdb.c b/source4/rpc_server/dnsserver/dnsdb.c
index 350e29a..899c7ec 100644
--- a/source4/rpc_server/dnsserver/dnsdb.c
+++ b/source4/rpc_server/dnsserver/dnsdb.c
@@ -145,7 +145,7 @@ struct dnsserver_zone *dnsserver_db_enumerate_zones(TALLOC_CTX *mem_ctx,
 
 		element = ldb_msg_find_element(res->msgs[i], "dNSProperty");
 		if(element != NULL){
-			props = talloc_zero_array(mem_ctx,
+			props = talloc_zero_array(tmp_ctx,
 						  struct dnsp_DnsProperty,
 						  element->num_values);
 			for (j = 0; j < element->num_values; j++ ) {
-- 
2.7.4


From 4b5fd69cae8d7ab8e9bc2e19e9642811affaa45d Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 29 May 2018 15:50:19 +1200
Subject: [PATCH 12/19] dns+kcc: adding dns scavenging to kcc periodic run

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Pair-Programmed-With: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 selftest/knownfail.d/dns-scavenging |  5 ---
 source4/dsdb/kcc/kcc_periodic.c     | 75 +++++++++++++++++++++++++++++++++++++
 source4/dsdb/kcc/kcc_service.h      |  4 ++
 3 files changed, 79 insertions(+), 5 deletions(-)
 delete mode 100644 selftest/knownfail.d/dns-scavenging

diff --git a/selftest/knownfail.d/dns-scavenging b/selftest/knownfail.d/dns-scavenging
deleted file mode 100644
index 86c0fa8..0000000
--- a/selftest/knownfail.d/dns-scavenging
+++ /dev/null
@@ -1,5 +0,0 @@
-#
-# Tests added for the dns scavenging changes
-#
-# Will be removed once the tests are implemented.
-#
diff --git a/source4/dsdb/kcc/kcc_periodic.c b/source4/dsdb/kcc/kcc_periodic.c
index 855b82b..3bfd134 100644
--- a/source4/dsdb/kcc/kcc_periodic.c
+++ b/source4/dsdb/kcc/kcc_periodic.c
@@ -596,6 +596,76 @@ WERROR kccsrv_periodic_schedule(struct kccsrv_service *service, uint32_t next_in
 }
 
 /*
+ * check to see if any dns entries need scavenging
+ */
+static NTSTATUS kccsrv_dns_zone_scavenging(
+	struct kccsrv_service *s,
+	TALLOC_CTX *mem_ctx)
+{
+
+	time_t current_time = time(NULL);
+	time_t dns_scavenge_interval;
+	time_t dns_collection_interval;
+
+	NTSTATUS status;
+	char *error_string = NULL;
+
+	/*
+	 * Only perform zone scavenging if it's been enabled.
+	 */
+	if (!lpcfg_dns_zone_scavenging(s->task->lp_ctx)) {
+		return NT_STATUS_OK;
+	}
+
+	dns_scavenge_interval = lpcfg_parm_int(s->task->lp_ctx,
+					       NULL,
+					       "dnsserver",
+					       "scavenging_interval",
+					       2 * 60 * 60);
+	dns_collection_interval =
+	    lpcfg_parm_int(s->task->lp_ctx,
+			   NULL,
+			   "dnsserver",
+			   "tombstone_collection_interval",
+			   24 * 60 * 60);
+	if ((current_time - s->last_dns_scavenge) > dns_scavenge_interval) {
+		s->last_dns_scavenge = current_time;
+		status = dns_tombstone_records(mem_ctx, s->samdb,
+					       &error_string);
+		if (!NT_STATUS_IS_OK(status)) {
+			const char *err = NULL;
+			if (error_string != NULL) {
+				err = error_string;
+			} else {
+				err = nt_errstr(status);
+			}
+			DBG_ERR("DNS record scavenging process failed: %s",
+				err);
+			return status;
+		}
+	}
+
+	if ((current_time - s->last_dns_tombstone_collection) >
+	    dns_collection_interval) {
+		s->last_dns_tombstone_collection = current_time;
+		status = dns_delete_tombstones(mem_ctx, s->samdb,
+					       &error_string);
+		if (!NT_STATUS_IS_OK(status)) {
+			const char *err = NULL;
+			if (error_string != NULL) {
+				err = error_string;
+			} else {
+				err = nt_errstr(status);
+			}
+			DBG_ERR("DNS tombstone deletion failed: %s", err);
+			return status;
+		}
+	}
+
+	return NT_STATUS_OK;
+}
+
+/*
   check to see if any deleted objects need scavenging
  */
 static NTSTATUS kccsrv_check_deleted(struct kccsrv_service *s, TALLOC_CTX *mem_ctx)
@@ -665,6 +735,11 @@ static void kccsrv_periodic_run(struct kccsrv_service *service)
 	if (!NT_STATUS_IS_OK(status)) {
 		DEBUG(0,("kccsrv_check_deleted failed - %s\n", nt_errstr(status)));
 	}
+	status = kccsrv_dns_zone_scavenging(service, mem_ctx);
+	if (!NT_STATUS_IS_OK(status)) {
+		DBG_ERR("kccsrv_dns_zone_scavenging failed - %s\n",
+			nt_errstr(status));
+	}
 	talloc_free(mem_ctx);
 }
 
diff --git a/source4/dsdb/kcc/kcc_service.h b/source4/dsdb/kcc/kcc_service.h
index a456903..00122ab 100644
--- a/source4/dsdb/kcc/kcc_service.h
+++ b/source4/dsdb/kcc/kcc_service.h
@@ -80,6 +80,10 @@ struct kccsrv_service {
 
 	time_t last_deleted_check;
 
+	time_t last_dns_scavenge;
+
+	time_t last_dns_tombstone_collection;
+
 	time_t last_full_scan_deleted_check;
 
 	bool am_rodc;
-- 
2.7.4


From f6000aaa89e496deb72c5f1dca7d8e1abf613e04 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Tue, 10 Jul 2018 13:14:18 +1200
Subject: [PATCH 13/19] dns: update tool changed for scavenging

Now that scavenging is implemented, the DNS update tool needs to be changed so
that it always updates every name required by the DC.  Otherwise, the records
might be scavenged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/blackbox/samba_dnsupdate.py | 18 +++++++++++++-----
 selftest/knownfail.d/dns                       |  2 ++
 source4/scripting/bin/samba_dnsupdate          | 10 +---------
 3 files changed, 16 insertions(+), 14 deletions(-)

diff --git a/python/samba/tests/blackbox/samba_dnsupdate.py b/python/samba/tests/blackbox/samba_dnsupdate.py
index e6cad3b..c4f14b3 100644
--- a/python/samba/tests/blackbox/samba_dnsupdate.py
+++ b/python/samba/tests/blackbox/samba_dnsupdate.py
@@ -23,6 +23,7 @@ from samba.credentials import Credentials
 from samba.auth import system_session
 from samba.samdb import SamDB
 import ldb
+import shutil, os
 
 class SambaDnsUpdateTests(samba.tests.BlackboxTestCase):
     """Blackbox test case for samba_dnsupdate."""
@@ -37,7 +38,10 @@ class SambaDnsUpdateTests(samba.tests.BlackboxTestCase):
             pass
 
     def test_samba_dnsupate_no_change(self):
-        out = self.check_output("samba_dnsupdate --verbose")
+        try:
+            out = self.check_output("samba_dnsupdate --verbose")
+        except samba.tests.BlackboxProcessError as e:
+            self.fail("Error calling samba_dnsupdate: %s" % e)
         self.assertTrue("No DNS updates needed" in out, out)
 
     def test_samba_dnsupate_set_ip(self):
@@ -83,6 +87,9 @@ class SambaDnsUpdateTests(samba.tests.BlackboxTestCase):
         self.creds = Credentials()
         self.creds.guess(self.lp)
         self.session = system_session()
+        uc_fn = self.lp.private_path('dns_update_cache')
+        tmp_uc = uc_fn + '_tmp'
+        shutil.copyfile(uc_fn, tmp_uc)
 
         self.samdb = SamDB(session_info=self.session,
                            credentials=self.creds,
@@ -97,13 +104,14 @@ class SambaDnsUpdateTests(samba.tests.BlackboxTestCase):
             self.samdb.get_config_basedn()),
             ldb.FLAG_MOD_ADD, "siteList")
 
-        out = self.check_output("samba_dnsupdate --verbose")
-        self.assertTrue("No DNS updates needed" in out, out)
+        dns_c = "samba_dnsupdate --verbose --use-file={}".format(tmp_uc)
+        out = self.check_output(dns_c)
+        self.assertFalse(site_name.lower() in out, out)
 
         self.samdb.modify(m)
 
-        out = self.check_output("samba_dnsupdate --verbose --use-samba-tool"
-                                " --rpc-server-ip={}".format(self.server_ip))
+        shutil.copyfile(uc_fn, tmp_uc)
+        out = self.check_output(dns_c)
 
         self.assertFalse("No DNS updates needed" in out, out)
         self.assertTrue(site_name.lower() in out, out)
diff --git a/selftest/knownfail.d/dns b/selftest/knownfail.d/dns
index 5a3e456..7ae19f6 100644
--- a/selftest/knownfail.d/dns
+++ b/selftest/knownfail.d/dns
@@ -65,3 +65,5 @@ samba.tests.dns.__main__.TestSimpleQueries.test_qtype_all_query\(rodc:local\)
 
 # The SOA override should not pass against the RODC, it must not overstamp
 samba.tests.dns.__main__.TestSimpleQueries.test_one_SOA_query\(rodc:local\)
+.*samba.tests.blackbox.samba_dnsupdate.SambaDnsUpdateTests.test_samba_dnsupate_set_ip
+.*samba.tests.blackbox.samba_dnsupdate.SambaDnsUpdateTests.test_samba_dnsupate_no_change
diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index 2d3fede..071cebe 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -847,15 +847,7 @@ for d in dns_list:
         rebuild_cache = True
         if opts.verbose:
             print "need cache add: %s" % d
-    if opts.all_names:
-        update_list.append(d)
-        if opts.verbose:
-            print "force update: %s" % d
-    elif not check_dns_name(d):
-        update_list.append(d)
-        if opts.verbose:
-            print "need update: %s" % d
-
+    update_list.append(d)
 
 for c in cache_list:
     found = False
-- 
2.7.4


From f42a7c8d8a823d2f9a0cfce14a08c44a03929cf9 Mon Sep 17 00:00:00 2001
From: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Thu, 7 Jun 2018 16:51:37 +1200
Subject: [PATCH 14/19] dns: static records

Modifies bind9 and internal dns to match windows static records behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py             | 36 +++++++++++++++++++++++++++++++++
 selftest/knownfail.d/dns              |  4 ++++
 source4/dns_server/dlz_bind9.c        | 20 +++++++++---------
 source4/dns_server/dns_update.c       | 38 ++++++++++++++++++++++++-----------
 source4/dns_server/dnsserver_common.c | 17 ++++++++++++++++
 source4/dns_server/dnsserver_common.h |  2 ++
 6 files changed, 94 insertions(+), 23 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index faf4c52..508d49f 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1093,6 +1093,42 @@ class TestZones(DNSTest):
         self.assertEqual(len(recs), 1)
         self.assertEqual(recs[0].dwTimeStamp, 0)
 
+    def test_static_record_dynamic_update(self):
+        name,txt = 'agingtest', ['test txt']
+        txt2 = ['test txt2']
+        self.set_aging(enable=True)
+        rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
+        rec_buf.rec = TXTRecord(txt)
+        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                                          0, self.server_ip,
+                                          self.zone, name, rec_buf, None)
+
+        rec2 = self.dns_update_record(name, txt2)
+        self.assertEqual(rec2.dwTimeStamp, 0)
+
+    def test_dynamic_record_static_update(self):
+        name,txt = 'agingtest', ['test txt']
+        txt2 = ['test txt2']
+        txt3 = ['test txt3']
+        self.set_aging(enable=True)
+
+        self.dns_update_record(name, txt)
+
+        rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
+        rec_buf.rec = TXTRecord(txt2)
+        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                                          0, self.server_ip,
+                                          self.zone, name, rec_buf, None)
+
+        self.dns_update_record(name, txt3)
+
+        recs = self.ldap_get_dns_records(name)
+        # Put in dict because ldap recs might be out of order
+        recs = {str(r.data.str):r for r in recs}
+        self.assertNotEqual(recs[str(txt)].dwTimeStamp, 0)
+        self.assertEqual(recs[str(txt2)].dwTimeStamp, 0)
+        self.assertEqual(recs[str(txt3)].dwTimeStamp, 0)
+
     def test_dns_tombstone_custom_match_rule(self):
         lp = self.get_loadparm()
         self.samdb = SamDB(url = lp.samdb_url(), lp = lp,
diff --git a/selftest/knownfail.d/dns b/selftest/knownfail.d/dns
index 7ae19f6..99b0f1d 100644
--- a/selftest/knownfail.d/dns
+++ b/selftest/knownfail.d/dns
@@ -46,6 +46,8 @@ samba.tests.dns.__main__.TestZones.test_rpc_add_no_timestamp\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(rodc:local\)
 samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule_fail\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_dynamic_record_static_update\(rodc:local\)
+samba.tests.dns.__main__.TestZones.test_static_record_dynamic_update\(rodc:local\)
 
 samba.tests.dns.__main__.TestZones.test_set_aging\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_aging_update\(vampire_dc:local\)
@@ -53,6 +55,8 @@ samba.tests.dns.__main__.TestZones.test_aging_update_disabled\(vampire_dc:local\
 samba.tests.dns.__main__.TestZones.test_aging_refresh\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_basic_scavenging\(vampire_dc:local\)
 samba.tests.dns.__main__.TestZones.test_dns_tombstone_custom_match_rule\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_dynamic_record_static_update\(vampire_dc:local\)
+samba.tests.dns.__main__.TestZones.test_static_record_dynamic_update\(vampire_dc:local\)
 
 samba.tests.dns.__main__.TestComplexQueries.test_cname_two_chain\(vampire_dc:local\)
 samba.tests.dns.__main__.TestComplexQueries.test_one_a_query\(vampire_dc:local\)
diff --git a/source4/dns_server/dlz_bind9.c b/source4/dns_server/dlz_bind9.c
index e55d73b..5f9a71d 100644
--- a/source4/dns_server/dlz_bind9.c
+++ b/source4/dns_server/dlz_bind9.c
@@ -1631,18 +1631,7 @@ _PUBLIC_ isc_result_t dlz_addrdataset(const char *name, const char *rdatastr, vo
 		return ISC_R_NOMEMORY;
 	}
 
-	unix_to_nt_time(&t, time(NULL));
-	/*
-	 * convert to seconds (NT time is in 100ns units)
-	 */
-	t /= 10 * 1000 * 1000;
-	/*
-	 * convert to hours
-	 */
-	t /= 3600;
-
 	rec->rank        = DNS_RANK_ZONE;
-	rec->dwTimeStamp = (uint32_t)t;
 
 	if (!b9_parse(state, rdatastr, rec)) {
 		state->log(ISC_LOG_INFO, "samba_dlz: failed to parse rdataset '%s'", rdatastr);
@@ -1704,6 +1693,15 @@ _PUBLIC_ isc_result_t dlz_addrdataset(const char *name, const char *rdatastr, vo
 			return ISC_R_NOMEMORY;
 		}
 		num_recs++;
+
+		if (dns_name_is_static(recs, num_recs)) {
+			rec->dwTimeStamp = 0;
+		} else {
+			unix_to_nt_time(&t, time(NULL));
+			t /= 10 * 1000 * 1000; /* convert to seconds */
+			t /= 3600;	     /* convert to hours */
+			rec->dwTimeStamp = (uint32_t)t;
+		}
 	}
 
 	recs[i] = *rec;
diff --git a/source4/dns_server/dns_update.c b/source4/dns_server/dns_update.c
index ebed449..f986661 100644
--- a/source4/dns_server/dns_update.c
+++ b/source4/dns_server/dns_update.c
@@ -38,7 +38,8 @@
 
 static WERROR dns_rr_to_dnsp(TALLOC_CTX *mem_ctx,
 			     const struct dns_res_rec *rrec,
-			     struct dnsp_DnssrvRpcRecord *r);
+			     struct dnsp_DnssrvRpcRecord *r,
+			     bool name_is_static);
 
 static WERROR check_one_prerequisite(struct dns_server *dns,
 				     TALLOC_CTX *mem_ctx,
@@ -181,7 +182,7 @@ static WERROR check_one_prerequisite(struct dns_server *dns,
 	rec = talloc_zero(mem_ctx, struct dnsp_DnssrvRpcRecord);
 	W_ERROR_HAVE_NO_MEMORY(rec);
 
-	werror = dns_rr_to_dnsp(rec, pr, rec);
+	werror = dns_rr_to_dnsp(rec, pr, rec, dns_name_is_static(ans, acount));
 	W_ERROR_NOT_OK_RETURN(werror);
 
 	for (i = 0; i < acount; i++) {
@@ -297,7 +298,8 @@ static WERROR update_prescan(const struct dns_name_question *zone,
 
 static WERROR dns_rr_to_dnsp(TALLOC_CTX *mem_ctx,
 			     const struct dns_res_rec *rrec,
-			     struct dnsp_DnssrvRpcRecord *r)
+			     struct dnsp_DnssrvRpcRecord *r,
+			     bool name_is_static)
 {
 	enum ndr_err_code ndr_err;
 	NTTIME t;
@@ -311,10 +313,14 @@ static WERROR dns_rr_to_dnsp(TALLOC_CTX *mem_ctx,
 	r->wType = (enum dns_record_type) rrec->rr_type;
 	r->dwTtlSeconds = rrec->ttl;
 	r->rank = DNS_RANK_ZONE;
-	unix_to_nt_time(&t, time(NULL));
-	t /= 10 * 1000 * 1000;
-	t /= 3600;
-	r->dwTimeStamp = t;
+	if (name_is_static) {
+		r->dwTimeStamp = 0;
+	} else {
+		unix_to_nt_time(&t, time(NULL));
+		t /= 10 * 1000 * 1000;
+		t /= 3600;
+		r->dwTimeStamp = t;
+	}
 
 	/* If we get QCLASS_ANY, we're done here */
 	if (rrec->rr_class == DNS_QCLASS_ANY) {
@@ -390,6 +396,7 @@ static WERROR handle_one_update(struct dns_server *dns,
 	WERROR werror;
 	bool tombstoned = false;
 	bool needs_add = false;
+	bool name_is_static;
 
 	DEBUG(2, ("Looking at record: \n"));
 	if (DEBUGLVL(2)) {
@@ -432,6 +439,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 		first = rcount;
 	}
 
+	name_is_static = dns_name_is_static(recs, rcount);
+
 	if (update->rr_class == zone->question_class) {
 		if (update->rr_type == DNS_QTYPE_CNAME) {
 			/*
@@ -456,7 +465,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 					struct dnsp_DnssrvRpcRecord, rcount + 1);
 			W_ERROR_HAVE_NO_MEMORY(recs);
 
-			werror = dns_rr_to_dnsp(recs, update, &recs[rcount]);
+			werror = dns_rr_to_dnsp(
+			    recs, update, &recs[rcount], name_is_static);
 			W_ERROR_NOT_OK_RETURN(werror);
 			rcount += 1;
 
@@ -508,7 +518,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 				return WERR_OK;
 			}
 
-			werror = dns_rr_to_dnsp(mem_ctx, update, &recs[i]);
+			werror = dns_rr_to_dnsp(
+			    mem_ctx, update, &recs[i], name_is_static);
 			W_ERROR_NOT_OK_RETURN(werror);
 
 			for (i++; i < rcount; i++) {
@@ -532,7 +543,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 				struct dnsp_DnssrvRpcRecord, rcount+1);
 		W_ERROR_HAVE_NO_MEMORY(recs);
 
-		werror = dns_rr_to_dnsp(recs, update, &recs[rcount]);
+		werror =
+		    dns_rr_to_dnsp(recs, update, &recs[rcount], name_is_static);
 		W_ERROR_NOT_OK_RETURN(werror);
 
 		for (i = first; i < rcount; i++) {
@@ -618,7 +630,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 						struct dnsp_DnssrvRpcRecord);
 			W_ERROR_HAVE_NO_MEMORY(ns_rec);
 
-			werror = dns_rr_to_dnsp(ns_rec, update, ns_rec);
+			werror = dns_rr_to_dnsp(
+			    ns_rec, update, ns_rec, name_is_static);
 			W_ERROR_NOT_OK_RETURN(werror);
 
 			for (i = first; i < rcount; i++) {
@@ -635,7 +648,8 @@ static WERROR handle_one_update(struct dns_server *dns,
 		del_rec = talloc(mem_ctx, struct dnsp_DnssrvRpcRecord);
 		W_ERROR_HAVE_NO_MEMORY(del_rec);
 
-		werror = dns_rr_to_dnsp(del_rec, update, del_rec);
+		werror =
+		    dns_rr_to_dnsp(del_rec, update, del_rec, name_is_static);
 		W_ERROR_NOT_OK_RETURN(werror);
 
 		for (i = first; i < rcount; i++) {
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 2551240..2a49370 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -724,6 +724,23 @@ static WERROR check_name_list(TALLOC_CTX *mem_ctx, uint16_t rec_count,
 	return WERR_OK;
 }
 
+bool dns_name_is_static(struct dnsp_DnssrvRpcRecord *records,
+			uint16_t rec_count)
+{
+	int i = 0;
+	for (i = 0; i < rec_count; i++) {
+		if (records[i].wType == DNS_TYPE_TOMBSTONE) {
+			continue;
+		}
+
+		if (records[i].wType == DNS_TYPE_SOA ||
+		    records[i].dwTimeStamp == 0) {
+			return true;
+		}
+	}
+	return false;
+}
+
 WERROR dns_get_zone_properties(struct ldb_context *samdb,
 			       TALLOC_CTX *mem_ctx,
 			       struct ldb_dn *zone_dn,
diff --git a/source4/dns_server/dnsserver_common.h b/source4/dns_server/dnsserver_common.h
index 9067e22..380f61b 100644
--- a/source4/dns_server/dnsserver_common.h
+++ b/source4/dns_server/dnsserver_common.h
@@ -61,6 +61,8 @@ WERROR dns_get_zone_properties(struct ldb_context *samdb,
 			       TALLOC_CTX *mem_ctx,
 			       struct ldb_dn *zone_dn,
 			       struct dnsserver_zoneinfo *zoneinfo);
+bool dns_name_is_static(struct dnsp_DnssrvRpcRecord *records,
+			uint16_t rec_count);
 WERROR dns_common_replace(struct ldb_context *samdb,
 			  TALLOC_CTX *mem_ctx,
 			  struct ldb_dn *dn,
-- 
2.7.4


From e33387779ef517810bc26f34120b0fc30a06cc3d Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 2 Jul 2018 16:47:16 +1200
Subject: [PATCH 15/19] tests dns: fix rpc null byte test

Fix update_add_null_char_rpc_to_dns so that the test matches the name.
It was not passing the embedded null to the rpc call.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 508d49f..5114711 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -1435,27 +1435,36 @@ class TestRPCRoundtrip(DNSTest):
                              dnsp.DNS_TYPE_TXT, '"NULL" "NULL"'))
 
     def test_update_add_null_char_rpc_to_dns(self):
-        prefix, txt = 'nulltextrec', ['NULL\x00BYTE']
-        prefix = 'rpc' + prefix
+        prefix = 'rpcnulltextrec'
         name = "%s.%s" % (prefix, self.get_dns_domain())
 
-        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT, '"NULL"')
+        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT, '"NULL\x00BYTE"')
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
 
         try:
-           self.check_query_txt(prefix, ['NULL'])
+            self.check_query_txt(prefix, ['NULL'])
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     def test_update_add_hex_char_txt_record(self):
         "test adding records works"
-- 
2.7.4


From 0eb229843c76f966609c4c1143afcad4040a79f9 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 2 Jul 2018 16:51:00 +1200
Subject: [PATCH 16/19] tests dns: dns.py remove flake8 warnings

Remove flake8 warnings from the code, this highlighted the issue with
test_update_add_null_char_rpc_to_dns fixed in the preceding commit.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns.py | 513 +++++++++++++++++++++++++++++++---------------
 1 file changed, 346 insertions(+), 167 deletions(-)

diff --git a/python/samba/tests/dns.py b/python/samba/tests/dns.py
index 5114711..6771e3b 100644
--- a/python/samba/tests/dns.py
+++ b/python/samba/tests/dns.py
@@ -25,10 +25,9 @@ import ldb
 import os
 import sys
 import struct
-import random
 import socket
 import samba.ndr as ndr
-from samba import credentials, param
+from samba import credentials
 from samba.dcerpc import dns, dnsp, dnsserver
 from samba.netcmd.dns import TXTRecord, dns_record_match, data_to_dns_record
 from samba.tests.subunitrun import SubunitOptions, TestProgram
@@ -67,6 +66,7 @@ server_name = args[0]
 server_ip = args[1]
 creds.set_krb_forwardable(credentials.NO_KRB_FORWARDABLE)
 
+
 class TestSimpleQueries(DNSTest):
     def setUp(self):
         super(TestSimpleQueries, self).setUp()
@@ -88,7 +88,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
@@ -106,12 +107,14 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
-        self.assertEquals(response.answers[0].rdata.mname.upper(),
-                          ("%s.%s" % (self.server, self.get_dns_domain())).upper())
+        self.assertEquals(
+            response.answers[0].rdata.mname.upper(),
+            ("%s.%s" % (self.server, self.get_dns_domain())).upper())
 
     def test_one_a_query_tcp(self):
         "create a query packet containing one query record via TCP"
@@ -124,7 +127,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_tcp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_tcp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
@@ -142,7 +146,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 0)
@@ -156,7 +161,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXDOMAIN)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 0)
@@ -176,7 +182,8 @@ class TestSimpleQueries(DNSTest):
 
         self.finish_name_packet(p, questions)
         try:
-            (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_FORMERR)
         except socket.timeout:
             # Windows chooses not to respond to incorrectly formatted queries.
@@ -196,7 +203,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
 
         num_answers = 1
         dc_ipv6 = os.getenv('SERVER_IPV6')
@@ -217,12 +225,16 @@ class TestSimpleQueries(DNSTest):
         questions = []
 
         name = "%s.%s" % (self.server, self.get_dns_domain())
-        q = self.make_name_question(name, dns.DNS_QTYPE_ALL, dns.DNS_QCLASS_NONE)
+        q = self.make_name_question(
+            name,
+            dns.DNS_QTYPE_ALL,
+            dns.DNS_QCLASS_NONE)
         questions.append(q)
 
         self.finish_name_packet(p, questions)
         try:
-            (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NOTIMP)
         except socket.timeout:
             # Windows chooses not to respond to incorrectly formatted queries.
@@ -241,7 +253,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         # We don't get SOA records for single hosts
@@ -259,7 +272,8 @@ class TestSimpleQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
@@ -291,7 +305,8 @@ class TestDNSUpdates(DNSTest):
 
         self.finish_name_packet(p, updates)
         try:
-            (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_FORMERR)
         except socket.timeout:
             # Windows chooses not to respond to incorrectly formatted queries.
@@ -310,7 +325,8 @@ class TestDNSUpdates(DNSTest):
         updates.append(u)
 
         self.finish_name_packet(p, updates)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NOTIMP)
 
     def test_update_prereq_with_non_null_ttl(self):
@@ -337,7 +353,8 @@ class TestDNSUpdates(DNSTest):
         p.answers = prereqs
 
         try:
-            (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_FORMERR)
         except socket.timeout:
             # Windows chooses not to respond to incorrectly formatted queries.
@@ -369,7 +386,8 @@ class TestDNSUpdates(DNSTest):
         p.ancount = len(prereqs)
         p.answers = prereqs
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXRRSET)
 
     def test_update_prereq_nonexisting_name(self):
@@ -395,14 +413,16 @@ class TestDNSUpdates(DNSTest):
         p.ancount = len(prereqs)
         p.answers = prereqs
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXRRSET)
 
     def test_update_add_txt_record(self):
         "test adding records works"
         prefix, txt = 'textrec', ['"This is a test"']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
 
@@ -434,7 +454,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # Now check the record is around
@@ -444,7 +465,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # Now delete the record
@@ -470,7 +492,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # And finally check it's gone
@@ -481,7 +504,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXDOMAIN)
 
     def test_readd_record(self):
@@ -512,7 +536,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # Now check the record is around
@@ -522,7 +547,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # Now delete the record
@@ -548,7 +574,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # check it's gone
@@ -559,7 +586,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXDOMAIN)
 
         # recreate the record
@@ -585,7 +613,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         # Now check the record is around
@@ -595,7 +624,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
     def test_update_add_mx_record(self):
@@ -624,7 +654,8 @@ class TestDNSUpdates(DNSTest):
         p.nscount = len(updates)
         p.nsrecs = updates
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         p = self.make_name_packet(dns.DNS_OPCODE_QUERY)
@@ -635,7 +666,8 @@ class TestDNSUpdates(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assertEqual(response.ancount, 1)
         ans = response.answers[0]
@@ -661,7 +693,8 @@ class TestComplexQueries(DNSTest):
         r.rdata = value
         p.nscount = 1
         p.nsrecs = [r]
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
     def setUp(self):
@@ -689,12 +722,15 @@ class TestComplexQueries(DNSTest):
 
             # Check the record
             name = "cname_test.%s" % self.get_dns_domain()
-            q = self.make_name_question(name, dns.DNS_QTYPE_A, dns.DNS_QCLASS_IN)
+            q = self.make_name_question(name,
+                                        dns.DNS_QTYPE_A,
+                                        dns.DNS_QCLASS_IN)
             print("asking for ", q.name)
             questions.append(q)
 
             self.finish_name_packet(p, questions)
-            (response, response_packet) = self.dns_transaction_udp(p, host=self.server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=self.server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
             self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
             self.assertEquals(response.ancount, 2)
@@ -712,7 +748,9 @@ class TestComplexQueries(DNSTest):
 
             name = self.get_dns_domain()
 
-            u = self.make_name_question(name, dns.DNS_QTYPE_SOA, dns.DNS_QCLASS_IN)
+            u = self.make_name_question(name,
+                                        dns.DNS_QTYPE_SOA,
+                                        dns.DNS_QCLASS_IN)
             updates.append(u)
             self.finish_name_packet(p, updates)
 
@@ -728,7 +766,8 @@ class TestComplexQueries(DNSTest):
             p.nscount = len(updates)
             p.nsrecs = updates
 
-            (response, response_packet) = self.dns_transaction_udp(p, host=self.server_ip)
+            (response, response_packet) =\
+                self.dns_transaction_udp(p, host=self.server_ip)
             self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
     def test_cname_two_chain(self):
@@ -746,7 +785,8 @@ class TestComplexQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 3)
@@ -767,7 +807,7 @@ class TestComplexQueries(DNSTest):
         name0 = "cnamedotprefix0.%s" % self.get_dns_domain()
         try:
             self.make_dns_update(name0, "", dns.DNS_QTYPE_CNAME)
-        except AssertionError as e:
+        except AssertionError:
             pass
         else:
             self.fail("Successfully added empty CNAME, which is invalid.")
@@ -787,7 +827,8 @@ class TestComplexQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
 
@@ -803,6 +844,7 @@ class TestComplexQueries(DNSTest):
         self.assertEquals(response.answers[1].name, name2)
         self.assertEquals(response.answers[1].rdata, name0)
 
+
 class TestInvalidQueries(DNSTest):
     def setUp(self):
         super(TestInvalidQueries, self).setUp()
@@ -814,7 +856,8 @@ class TestInvalidQueries(DNSTest):
         self.timeout = timeout
 
     def test_one_a_query(self):
-        "send 0 bytes follows by create a query packet containing one query record"
+        """send 0 bytes follows by create a query packet
+           containing one query record"""
 
         s = None
         try:
@@ -834,7 +877,8 @@ class TestInvalidQueries(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=self.server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=self.server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
@@ -860,7 +904,7 @@ class TestInvalidQueries(DNSTest):
             send_packet = ndr.ndr_pack(p)
             s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
             s.settimeout(timeout)
-            host=self.server_ip
+            host = self.server_ip
             s.connect((host, 53))
             tcp_packet = struct.pack('!H', len(send_packet))
             tcp_packet += send_packet
@@ -877,6 +921,7 @@ class TestInvalidQueries(DNSTest):
             if s is not None:
                 s.close()
 
+
 class TestZones(DNSTest):
     def setUp(self):
         super(TestZones, self).setUp()
@@ -888,12 +933,12 @@ class TestZones(DNSTest):
         self.timeout = timeout
 
         self.zone = "test.lan"
-        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" %\
+        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" %
                                             (self.server_ip),
                                             self.lp, self.creds)
 
         self.samdb = SamDB(url="ldap://" + self.server_ip,
-                           lp = self.get_loadparm(),
+                           lp=self.get_loadparm(),
                            session_info=system_session(),
                            credentials=self.creds)
 
@@ -935,7 +980,7 @@ class TestZones(DNSTest):
 
     def set_params(self, **kwargs):
         zone = kwargs.pop('zone', None)
-        for key,val in kwargs.items():
+        for key, val in kwargs.items():
             name_param = dnsserver.DNS_RPC_NAME_AND_PARAM()
             name_param.dwParam = val
             name_param.pszNodeName = key
@@ -943,8 +988,13 @@ class TestZones(DNSTest):
             client_version = dnsserver.DNS_CLIENT_VERSION_LONGHORN
             nap_type = dnsserver.DNSSRV_TYPEID_NAME_AND_PARAM
             try:
-                self.rpc_conn.DnssrvOperation2(client_version, 0, self.server, zone,
-                                               0, 'ResetDwordProperty', nap_type,
+                self.rpc_conn.DnssrvOperation2(client_version,
+                                               0,
+                                               self.server,
+                                               zone,
+                                               0,
+                                               'ResetDwordProperty',
+                                               nap_type,
                                                name_param)
             except WERRORError as e:
                 self.fail(str(e))
@@ -954,7 +1004,7 @@ class TestZones(DNSTest):
         dns_recs = self.ldap_get_dns_records(name)
         for rec in dns_recs:
             func(rec)
-        update_dict = {'dn':dn, 'dnsRecord':[ndr_pack(r) for r in dns_recs]}
+        update_dict = {'dn': dn, 'dnsRecord': [ndr_pack(r) for r in dns_recs]}
         self.samdb.modify(ldb.Message.from_dict(self.samdb,
                                                 update_dict,
                                                 ldb.FLAG_MOD_REPLACE))
@@ -983,14 +1033,14 @@ class TestZones(DNSTest):
 
     def ldap_get_zone_settings(self):
         records = self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_BASE,
-                   expression="(&(objectClass=dnsZone)"+\
+                   expression="(&(objectClass=dnsZone)" +
                                 "(name={}))".format(self.zone),
                                     attrs=["dNSProperty"])
         self.assertEqual(len(records), 1)
         props = [ndr_unpack(dnsp.DnsProperty, r)
                  for r in records[0].get('dNSProperty')]
 
-        #We have no choice but to repeat these here.
+        # We have no choice but to repeat these here.
         zone_prop_ids = {0x00: "EMPTY",
                          0x01: "TYPE",
                          0x02: "ALLOW_UPDATE",
@@ -1014,7 +1064,7 @@ class TestZones(DNSTest):
         self.create_zone(self.zone, aging_enabled=enable)
         self.set_params(NoRefreshInterval=1, RefreshInterval=1,
                         Aging=int(bool(enable)), zone=self.zone,
-                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+                        AllowUpdate=dnsp.DNS_ZONE_UPDATE_UNSECURE)
 
     def test_set_aging(self, enable=True, name='agingtest', txt=['test txt']):
         self.set_aging(enable=True)
@@ -1041,6 +1091,7 @@ class TestZones(DNSTest):
         if not enable:
             self.set_params(zone=self.zone, Aging=0)
         dec = 2
+
         def mod_ts(rec):
             self.assertTrue(rec.dwTimeStamp > 0)
             rec.dwTimeStamp -= dec
@@ -1059,55 +1110,67 @@ class TestZones(DNSTest):
         self.test_aging_update(enable=False)
 
     def test_aging_refresh(self):
-        name,txt = 'agingtest', ['test txt']
+        name, txt = 'agingtest', ['test txt']
         self.create_zone(self.zone, aging_enabled=True)
         interval = 10
         self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
                         Aging=1, zone=self.zone,
-                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+                        AllowUpdate=dnsp.DNS_ZONE_UPDATE_UNSECURE)
         before_mod = self.dns_update_record(name, txt)
+
         def mod_ts(rec):
             self.assertTrue(rec.dwTimeStamp > 0)
-            rec.dwTimeStamp -= interval/2
+            rec.dwTimeStamp -= interval / 2
         self.ldap_modify_dnsrecs(name, mod_ts)
         update_during_norefresh = self.dns_update_record(name, txt)
+
         def mod_ts(rec):
             self.assertTrue(rec.dwTimeStamp > 0)
-            rec.dwTimeStamp -= interval + interval/2
+            rec.dwTimeStamp -= interval + interval / 2
         self.ldap_modify_dnsrecs(name, mod_ts)
         update_during_refresh = self.dns_update_record(name, txt)
         self.assertEqual(update_during_norefresh.dwTimeStamp,
-                         before_mod.dwTimeStamp - interval/2)
+                         before_mod.dwTimeStamp - interval / 2)
         self.assertEqual(update_during_refresh.dwTimeStamp,
                          before_mod.dwTimeStamp)
 
     def test_rpc_add_no_timestamp(self):
-        name,txt = 'agingtest', ['test txt']
+        name, txt = 'agingtest', ['test txt']
         self.set_aging(enable=True)
         rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         rec_buf.rec = TXTRecord(txt)
-        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                          0, self.server_ip,
-                                          self.zone, name, rec_buf, None)
+        self.rpc_conn.DnssrvUpdateRecord2(
+            dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+            0,
+            self.server_ip,
+            self.zone,
+            name,
+            rec_buf,
+            None)
         recs = self.ldap_get_dns_records(name)
         self.assertEqual(len(recs), 1)
         self.assertEqual(recs[0].dwTimeStamp, 0)
 
     def test_static_record_dynamic_update(self):
-        name,txt = 'agingtest', ['test txt']
+        name, txt = 'agingtest', ['test txt']
         txt2 = ['test txt2']
         self.set_aging(enable=True)
         rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         rec_buf.rec = TXTRecord(txt)
-        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                          0, self.server_ip,
-                                          self.zone, name, rec_buf, None)
+        self.rpc_conn.DnssrvUpdateRecord2(
+            dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+            0,
+            self.server_ip,
+            self.zone,
+            name,
+            rec_buf,
+            None)
 
         rec2 = self.dns_update_record(name, txt2)
         self.assertEqual(rec2.dwTimeStamp, 0)
 
     def test_dynamic_record_static_update(self):
-        name,txt = 'agingtest', ['test txt']
+        name, txt = 'agingtest', ['test txt']
         txt2 = ['test txt2']
         txt3 = ['test txt3']
         self.set_aging(enable=True)
@@ -1116,33 +1179,38 @@ class TestZones(DNSTest):
 
         rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         rec_buf.rec = TXTRecord(txt2)
-        self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                          0, self.server_ip,
-                                          self.zone, name, rec_buf, None)
+        self.rpc_conn.DnssrvUpdateRecord2(
+            dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+            0,
+            self.server_ip,
+            self.zone,
+            name,
+            rec_buf,
+            None)
 
         self.dns_update_record(name, txt3)
 
         recs = self.ldap_get_dns_records(name)
         # Put in dict because ldap recs might be out of order
-        recs = {str(r.data.str):r for r in recs}
+        recs = {str(r.data.str): r for r in recs}
         self.assertNotEqual(recs[str(txt)].dwTimeStamp, 0)
         self.assertEqual(recs[str(txt2)].dwTimeStamp, 0)
         self.assertEqual(recs[str(txt3)].dwTimeStamp, 0)
 
     def test_dns_tombstone_custom_match_rule(self):
         lp = self.get_loadparm()
-        self.samdb = SamDB(url = lp.samdb_url(), lp = lp,
+        self.samdb = SamDB(url=lp.samdb_url(), lp=lp,
                            session_info=system_session(),
                            credentials=self.creds)
 
-        name,txt = 'agingtest', ['test txt']
-        name2,txt2 = 'agingtest2', ['test txt2']
-        name3,txt3 = 'agingtest3', ['test txt3']
+        name, txt = 'agingtest', ['test txt']
+        name2, txt2 = 'agingtest2', ['test txt2']
+        name3, txt3 = 'agingtest3', ['test txt3']
         self.create_zone(self.zone, aging_enabled=True)
         interval = 10
         self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
                         Aging=1, zone=self.zone,
-                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+                        AllowUpdate=dnsp.DNS_ZONE_UPDATE_UNSECURE)
 
         self.dns_update_record(name, txt),
 
@@ -1160,9 +1228,9 @@ class TestZones(DNSTest):
         self.ldap_modify_dnsrecs(name, mod_ts)
         self.ldap_modify_dnsrecs(name2, mod_ts)
 
-        recs = self.ldap_get_dns_records(name3)
+        self.ldap_get_dns_records(name3)
         expr = "(dnsRecord:1.3.6.1.4.1.7165.4.5.3:={})"
-        expr = expr.format(int(last_update.dwTimeStamp)-1)
+        expr = expr.format(int(last_update.dwTimeStamp) - 1)
         try:
             res = self.samdb.search(base=self.zone_dn, scope=ldb.SCOPE_SUBTREE,
                                     expression=expr, attrs=["*"])
@@ -1182,7 +1250,7 @@ class TestZones(DNSTest):
 
     def test_basic_scavenging(self):
         lp = self.get_loadparm()
-        self.samdb = SamDB(url = lp.samdb_url(), lp = lp,
+        self.samdb = SamDB(url=lp.samdb_url(), lp=lp,
                            session_info=system_session(),
                            credentials=self.creds)
 
@@ -1190,21 +1258,21 @@ class TestZones(DNSTest):
         interval = 1
         self.set_params(NoRefreshInterval=interval, RefreshInterval=interval,
                         zone=self.zone, Aging=1,
-                        AllowUpdate = dnsp.DNS_ZONE_UPDATE_UNSECURE)
+                        AllowUpdate=dnsp.DNS_ZONE_UPDATE_UNSECURE)
         name, txt = 'agingtest', ['test txt']
         name2, txt2 = 'agingtest2', ['test txt2']
         name3, txt3 = 'agingtest3', ['test txt3']
-        self.dns_update_record(name,txt)
-        self.dns_update_record(name2,txt)
-        self.dns_update_record(name2,txt2)
-        self.dns_update_record(name3,txt)
-        self.dns_update_record(name3,txt2)
-        last_add = self.dns_update_record(name3,txt3)
+        self.dns_update_record(name, txt)
+        self.dns_update_record(name2, txt)
+        self.dns_update_record(name2, txt2)
+        self.dns_update_record(name3, txt)
+        self.dns_update_record(name3, txt2)
+        last_add = self.dns_update_record(name3, txt3)
 
         def mod_ts(rec):
             self.assertTrue(rec.dwTimeStamp > 0)
             if rec.data.str == txt:
-                rec.dwTimeStamp -= interval*5
+                rec.dwTimeStamp -= interval * 5
         self.ldap_modify_dnsrecs(name, mod_ts)
         self.ldap_modify_dnsrecs(name2, mod_ts)
         self.ldap_modify_dnsrecs(name3, mod_ts)
@@ -1223,14 +1291,15 @@ class TestZones(DNSTest):
         recs = self.ldap_get_dns_records(name3)
         self.assertEqual(len(recs), 2)
         txts = {str(r.data.str) for r in recs}
-        self.assertEqual(txts, {str(txt2),str(txt3)})
+        self.assertEqual(txts, {str(txt2), str(txt3)})
         self.assertEqual(recs[0].wType, dnsp.DNS_TYPE_TXT)
         self.assertEqual(recs[1].wType, dnsp.DNS_TYPE_TXT)
 
         for make_it_work in [False, True]:
             inc = -1 if make_it_work else 1
+
             def mod_ts(rec):
-                rec.data = (last_add.dwTimeStamp - 24*14) + inc
+                rec.data = (last_add.dwTimeStamp - 24 * 14) + inc
             self.ldap_modify_dnsrecs(name, mod_ts)
             dsdb._dns_delete_tombstones(self.samdb)
             recs = self.ldap_get_records(name)
@@ -1258,25 +1327,29 @@ class TestZones(DNSTest):
         questions.append(q)
         self.finish_name_packet(p, questions)
 
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         # Windows returns OK while BIND logically seems to return NXDOMAIN
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXDOMAIN)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 0)
 
         self.create_zone(zone)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 1)
         self.assertEquals(response.answers[0].rr_type, dns.DNS_QTYPE_SOA)
 
         self.delete_zone(zone)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_NXDOMAIN)
         self.assert_dns_opcode_equals(response, dns.DNS_OPCODE_QUERY)
         self.assertEquals(response.ancount, 0)
 
+
 class TestRPCRoundtrip(DNSTest):
     def setUp(self):
         super(TestRPCRoundtrip, self).setUp()
@@ -1285,8 +1358,10 @@ class TestRPCRoundtrip(DNSTest):
         self.server_ip = server_ip
         self.lp = lp
         self.creds = creds
-        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" % (self.server_ip),
-                                            self.lp, self.creds)
+        self.rpc_conn = dnsserver.dnsserver("ncacn_ip_tcp:%s[sign]" %
+                                            (self.server_ip),
+                                            self.lp,
+                                            self.creds)
 
     def tearDown(self):
         super(TestRPCRoundtrip, self).tearDown()
@@ -1300,63 +1375,94 @@ class TestRPCRoundtrip(DNSTest):
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
+
         except WERRORError as e:
             self.fail(str(e))
 
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     def test_update_add_null_padded_txt_record(self):
         "test adding records works"
         prefix, txt = 'pad1textrec', ['"This is a test"', '', '']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
-        self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
-                             self.get_dns_domain(),
-                             "%s.%s" % (prefix, self.get_dns_domain()),
-                             dnsp.DNS_TYPE_TXT, '"\\"This is a test\\"" "" ""'))
+        self.assertIsNotNone(
+            dns_record_match(self.rpc_conn,
+                self.server_ip,
+                self.get_dns_domain(),
+                "%s.%s" % (prefix, self.get_dns_domain()),
+                dnsp.DNS_TYPE_TXT,
+                '"\\"This is a test\\"" "" ""'))
 
         prefix, txt = 'pad2textrec', ['"This is a test"', '', '', 'more text']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
-        self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
-                             self.get_dns_domain(),
-                             "%s.%s" % (prefix, self.get_dns_domain()),
-                             dnsp.DNS_TYPE_TXT, '"\\"This is a test\\"" "" "" "more text"'))
+        self.assertIsNotNone(
+            dns_record_match(
+                self.rpc_conn,
+                self.server_ip,
+                self.get_dns_domain(),
+                "%s.%s" % (prefix, self.get_dns_domain()),
+                dnsp.DNS_TYPE_TXT,
+                '"\\"This is a test\\"" "" "" "more text"'))
 
         prefix, txt = 'pad3textrec', ['', '', '"This is a test"']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
-        self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
-                             self.get_dns_domain(),
-                             "%s.%s" % (prefix, self.get_dns_domain()),
-                             dnsp.DNS_TYPE_TXT, '"" "" "\\"This is a test\\""'))
+        self.assertIsNotNone(
+            dns_record_match(
+                self.rpc_conn,
+                self.server_ip,
+                self.get_dns_domain(),
+                "%s.%s" % (prefix, self.get_dns_domain()),
+                dnsp.DNS_TYPE_TXT,
+                '"" "" "\\"This is a test\\""'))
 
     def test_update_add_padding_rpc_to_dns(self):
         prefix, txt = 'pad1textrec', ['"This is a test"', '', '']
         prefix = 'rpc' + prefix
         name = "%s.%s" % (prefix, self.get_dns_domain())
 
-        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT, '"\\"This is a test\\"" "" ""')
+        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT,
+                                 '"\\"This is a test\\"" "" ""')
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
@@ -1364,21 +1470,32 @@ class TestRPCRoundtrip(DNSTest):
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
         prefix, txt = 'pad2textrec', ['"This is a test"', '', '', 'more text']
         prefix = 'rpc' + prefix
         name = "%s.%s" % (prefix, self.get_dns_domain())
 
-        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT, '"\\"This is a test\\"" "" "" "more text"')
+        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT,
+                                 '"\\"This is a test\\"" "" "" "more text"')
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
@@ -1386,37 +1503,54 @@ class TestRPCRoundtrip(DNSTest):
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
         prefix, txt = 'pad3textrec', ['', '', '"This is a test"']
         prefix = 'rpc' + prefix
         name = "%s.%s" % (prefix, self.get_dns_domain())
 
-        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT, '"" "" "\\"This is a test\\""')
+        rec = data_to_dns_record(dnsp.DNS_TYPE_TXT,
+                                 '"" "" "\\"This is a test\\""')
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
         except WERRORError as e:
             self.fail(str(e))
 
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     # Test is incomplete due to strlen against txt records
     def test_update_add_null_char_txt_record(self):
         "test adding records works"
         prefix, txt = 'nulltextrec', ['NULL\x00BYTE']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, ['NULL'])
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1426,7 +1560,8 @@ class TestRPCRoundtrip(DNSTest):
 
         prefix, txt = 'nulltextrec2', ['NULL\x00BYTE', 'NULL\x00BYTE']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, ['NULL', 'NULL'])
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1470,7 +1605,8 @@ class TestRPCRoundtrip(DNSTest):
         "test adding records works"
         prefix, txt = 'hextextrec', ['HIGH\xFFBYTE']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1487,25 +1623,36 @@ class TestRPCRoundtrip(DNSTest):
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
 
         try:
-           self.check_query_txt(prefix, txt)
+            self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     def test_update_add_slash_txt_record(self):
         "test adding records works"
         prefix, txt = 'slashtextrec', ['Th\\=is=is a test']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1525,9 +1672,14 @@ class TestRPCRoundtrip(DNSTest):
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
@@ -1536,16 +1688,22 @@ class TestRPCRoundtrip(DNSTest):
             self.check_query_txt(prefix, txt)
 
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     def test_update_add_two_txt_records(self):
         "test adding two txt records works"
         prefix, txt = 'textrec2', ['"This is a test"',
                                    '"and this is a test, too"']
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1566,9 +1724,14 @@ class TestRPCRoundtrip(DNSTest):
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
 
         except WERRORError as e:
             self.fail(str(e))
@@ -1576,15 +1739,21 @@ class TestRPCRoundtrip(DNSTest):
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
     def test_update_add_empty_txt_records(self):
         "test adding two txt records works"
         prefix, txt = 'emptytextrec', []
         p = self.make_txt_update(prefix, txt)
-        (response, response_packet) = self.dns_transaction_udp(p, host=server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.check_query_txt(prefix, txt)
         self.assertIsNotNone(dns_record_match(self.rpc_conn, self.server_ip,
@@ -1601,17 +1770,27 @@ class TestRPCRoundtrip(DNSTest):
         add_rec_buf = dnsserver.DNS_RPC_RECORD_BUF()
         add_rec_buf.rec = rec
         try:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                     0, self.server_ip, self.get_dns_domain(),
-                                     name, add_rec_buf, None)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                add_rec_buf,
+                None)
         except WERRORError as e:
             self.fail(str(e))
 
         try:
             self.check_query_txt(prefix, txt)
         finally:
-            self.rpc_conn.DnssrvUpdateRecord2(dnsserver.DNS_CLIENT_VERSION_LONGHORN,
-                                              0, self.server_ip, self.get_dns_domain(),
-                                              name, None, add_rec_buf)
+            self.rpc_conn.DnssrvUpdateRecord2(
+                dnsserver.DNS_CLIENT_VERSION_LONGHORN,
+                0,
+                self.server_ip,
+                self.get_dns_domain(),
+                name,
+                None,
+                add_rec_buf)
 
 TestProgram(module=__name__, opts=subunitopts)
-- 
2.7.4


From 8c27a88dde72c7a70039983c2ab9cec642a1a566 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 3 Jul 2018 17:03:38 +1200
Subject: [PATCH 17/19] tests dns: dns_base.py remove flake8 warnings

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dns_base.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/python/samba/tests/dns_base.py b/python/samba/tests/dns_base.py
index 0709bbe..5abd6bd 100644
--- a/python/samba/tests/dns_base.py
+++ b/python/samba/tests/dns_base.py
@@ -28,6 +28,7 @@ import socket
 import uuid
 import time
 
+
 class DNSTest(TestCaseInTempDir):
 
     def setUp(self):
@@ -199,7 +200,8 @@ class DNSTest(TestCaseInTempDir):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, host=self.server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, host=self.server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
         self.assertEquals(response.ancount, 1)
         self.assertEquals(response.answers[0].rdata.txt.str, txt_array)
@@ -244,7 +246,7 @@ class DNSTKeyTest(DNSTest):
         rdata = dns.tkey_record()
         rdata.algorithm = "gss-tsig"
         rdata.inception = int(time.time())
-        rdata.expiration = int(time.time()) + 60*60
+        rdata.expiration = int(time.time()) + 60 * 60
         rdata.mode = dns.DNS_TKEY_MODE_GSSAPI
         rdata.error = 0
         rdata.other_size = 0
@@ -271,7 +273,8 @@ class DNSTKeyTest(DNSTest):
         p.arcount = 1
         p.additional = additional
 
-        (response, response_packet) = self.dns_transaction_tcp(p, self.server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_tcp(p, self.server_ip)
         self.assert_dns_rcode_equals(response, dns.DNS_RCODE_OK)
 
         tkey_record = response.answers[0].rdata
@@ -395,7 +398,8 @@ class DNSTKeyTest(DNSTest):
         questions.append(q)
 
         self.finish_name_packet(p, questions)
-        (response, response_packet) = self.dns_transaction_udp(p, self.server_ip)
+        (response, response_packet) =\
+            self.dns_transaction_udp(p, self.server_ip)
         return response.operation & 0x000F
 
     def make_update_request(self, delete=False):
-- 
2.7.4


From 012b4dd95744719526125a5bfd46e8661b637994 Mon Sep 17 00:00:00 2001
From: Bob Campbell <bobcampbell at catalyst.net.nz>
Date: Fri, 9 Dec 2016 09:13:11 +1300
Subject: [PATCH 18/19] python/tests: check setting values on dnsRecord
 attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12451

Signed-off-by: Bob Campbell <bobcampbell at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 python/samba/tests/dcerpc/dnsserver.py | 34 ++++++++++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/python/samba/tests/dcerpc/dnsserver.py b/python/samba/tests/dcerpc/dnsserver.py
index 93d4478..53e1abd 100644
--- a/python/samba/tests/dcerpc/dnsserver.py
+++ b/python/samba/tests/dcerpc/dnsserver.py
@@ -212,7 +212,7 @@ class DnsserverTests(RpcInterfaceTestCase):
 
         zone_dn = None
         for zone in zones:
-            if zone_name in str(zone.dn):
+            if "DC=%s," % zone_name in str(zone.dn):
                 zone_dn = zone.dn
                 break
 
@@ -225,7 +225,8 @@ class DnsserverTests(RpcInterfaceTestCase):
 
         for old_packed_record in records:
             if record_name in str(old_packed_record.dn):
-                return (old_packed_record.dn, ndr_unpack(dnsp.DnssrvRpcRecord, old_packed_record["dnsRecord"][0]))
+                rec = ndr_unpack(dnsp.DnssrvRpcRecord, old_packed_record["dnsRecord"][0])
+                return (old_packed_record.dn, rec)
 
     def test_duplicate_matching(self):
         """
@@ -323,6 +324,35 @@ class DnsserverTests(RpcInterfaceTestCase):
                 self.assert_num_records(self.custom_zone, "testrecord", record_type_str)
                 self.delete_record(self.custom_zone, "testrecord", record_type_str, record_str)
 
+    def check_params(self, wDataLength, rank, flags, dwTtlSeconds, dwReserved, data,
+                     wType, dwTimeStamp=0, zone="zone", rec_name="testrecord"):
+        res = self.get_record_from_db(zone, rec_name)
+        self.assertIsNotNone(res, "Expected record %s but was not found over LDAP." % data)
+        (rec_dn, rec) = res
+        self.assertEqual(wDataLength, rec.wDataLength, "Unexpected data length for record %s. Got %s, expected %s." % (data, rec.wDataLength, wDataLength))
+        self.assertEqual(rank, rec.rank, "Unexpected rank for record %s. Got %s, expected %s." % (data, rec.rank, rank))
+        self.assertEqual(flags, rec.flags, "Unexpected flags for record %s. Got %s, expected %s." % (data, rec.flags, flags))
+        self.assertEqual(dwTtlSeconds, rec.dwTtlSeconds, "Unexpected time to live for record %s. Got %s, expected %s." % (data, rec.dwTtlSeconds, dwTtlSeconds))
+        self.assertEqual(dwReserved, rec.dwReserved, "Unexpected dwReserved for record %s. Got %s, expected %s." % (data, rec.dwReserved, dwReserved))
+        self.assertEqual(data.lower(), rec.data.lower(), "Unexpected data for record %s. Got %s, expected %s." % (data, rec.data.lower(), data.lower()))
+        self.assertEqual(wType, rec.wType, "Unexpected wType for record %s. Got %s, expected %s." % (data, rec.wType, wType))
+        self.assertEqual(dwTimeStamp, rec.dwTimeStamp, "Unexpected timestamp for record %s. Got %s, expected %s." % (data, rec.dwTimeStamp, dwTimeStamp))
+
+    def test_record_params(self):
+        """
+        Make sure that, when we add records to the database,
+        they're added with reasonable parameters.
+        """
+        self.add_record(self.custom_zone, "testrecord", "A", "192.168.50.50")
+        self.check_params(4, 240, 0, 900, 0, "192.168.50.50", 1)
+        self.delete_record(self.custom_zone, "testrecord", "A", "192.168.50.50")
+        self.add_record(self.custom_zone, "testrecord", "AAAA", "AAAA:AAAA::")
+        self.check_params(16, 240, 0, 900, 0, "AAAA:AAAA:0000:0000:0000:0000:0000:0000", 28)
+        self.delete_record(self.custom_zone, "testrecord", "AAAA", "AAAA:AAAA::")
+        self.add_record(self.custom_zone, "testrecord", "CNAME", "cnamedest")
+        self.check_params(13, 240, 0, 900, 0, "cnamedest", 5)
+        self.delete_record(self.custom_zone, "testrecord", "CNAME", "cnamedest")
+
     def test_reject_invalid_commands(self):
         """
         Make sure that we can't add a variety of invalid records,
-- 
2.7.4


From 04ce5dd9ae8699e19db7b3b966c7b72b6aa8784f Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Tue, 10 Jul 2018 17:13:48 +1200
Subject: [PATCH 19/19] WHATSNEW: Add entry for "Dynamic DNS record scavenging
 support"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10812

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
 WHATSNEW.txt | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/WHATSNEW.txt b/WHATSNEW.txt
index 6795e0f..fb3b498 100644
--- a/WHATSNEW.txt
+++ b/WHATSNEW.txt
@@ -218,6 +218,23 @@ Samba developers now have pre-commit testing available in GitLab,
 giving reviewers confidence that the submitted patches pass a full CI
 before being submitted to the Samba Team's own autobuild system.
 
+Dynamic DNS record scavenging support
+-------------------------------------
+
+It is now possible to enable scavenging of DNS Zones to remove DNS
+records that were dynamically created and have not been touched in
+some time.
+
+This support should however only be enabled on new zones or new
+installations.  Sadly old Samba versions suffer from BUG 12451 and
+mark dynamic DNS records as static and static records as dynamic.
+While a dbcheck rule may be able to find these in the future,
+currently a reliable test has not been devised.
+
+Finally, there is not currently a command-line tool to enable this
+feature, currently it should be enabled from the DNS Manager tool from
+Windows. Also the feature needs to have been enabled by setting the smb.conf
+parameter dns_zone_scavenging to yes
 
 REMOVED FEATURES
 ================
-- 
2.7.4

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180712/b0baff5d/signature-0001.sig>


More information about the samba-technical mailing list