samba_dnsupdate timeouts (was Re: [PATCH] python indent bugfix in dns_hub.py)

[Garming Sam]

I actually noticed something like this when I was working on the RODC a
while back. I could never reliably reproduce it, but there were
definitely occasions when samba_dnsupdate would get into a strange
state. I presume that this is the answer to that mystery.

Cheers,

Garming

On 7/02/19 10:37 PM, Stefan Metzmacher via samba-technical wrote:
> BTW: there's at least one additional problem I noticed while cleaning up
> my patchset to wait for the dns_update_cache file to be filled.
>
> On an RODC samba_dnsupdate calls DsrUpdateReadOnlyServerDnsRecords via
> IRPC to the local winbindd, which calls
> DsrUpdateReadOnlyServerDnsRecords via netlogon to the RWDC.
> The netlogon server calls dnsupdate_RODC via IRPC to the dnsupdate
> task on the RWDC, which calls samba_dnsupdate with a temporary config
> on behalf of the RODC.
>
> Currently samba_dnsupdate (on the RODC) constantly recreates its irpc
> handle (and the messaging context and the dgram socket). This causes
> problems when winbindd tries to send back the result to samba_dnsupdate,
> as winbindd's messaging context caches connected dgram sockets per
> target pid for 1 second. As the target (samba_dnsupdate) constantly
> recreates its socket, winbindd very likely hits ECONNREFUSED when
> the socket is recreated multiple times within 1 second.
> As a result samba_dnsupdate hits a 10 second irpc timeout, so the
> whole samba_dnsupdate hits the 20 second timeout on the RODC.
>
> The solution to this is to cache the irpc handle in samba_dnsupdate.
>
>> I'll raise a bug and backport the autobuild change to 4.10.
> I'm currently testing the attached additional patches here:
> https://gitlab.com/samba-team/devel/samba/pipelines/46503335
> and
> https://gitlab.com/samba-team/devel/samba/pipelines/46503395
>
> Thanks!
> metze
>
>> On 5/02/19 8:54 PM, Jeremy Allison wrote:
>>> On Tue, Feb 05, 2019 at 08:44:08AM +0100, Volker Lendecke wrote:
>>>> On Tue, Feb 05, 2019 at 09:21:58AM +0200, Isaac Boukris via samba-technical wrote:
>>>>> Hi,
>>>>>
>>>>> On Tue, Feb 5, 2019 at 6:28 AM Tim Beale <timbeale at catalyst.net.nz> wrote:
>>>>>> Yeah, it looks like changing the process model was enough to push the CI
>>>>>> runners over the edge fairly reliably.
>>>>> FYI, I just tried to rebase merge request !200 on master and
>>>>> build_samba_ad_dc_2 failed on a somewhat different dns error:
>>>>> https://gitlab.com/samba-team/devel/samba/-/jobs/156308654
>>>> I'll work on a patchset to restore the pre-async-dns/dns_hub behaviour
>>>> and remove the dns_hub again. We need to find a different way to
>>>> approach async DNS and properly test it. The current approach has
>>>> proven to be the wrong way.
>>> Please don't do that. It appears to be a python / resource limit
>>> issue on the gitlab-CI runners. Removing dns_hub isn't going
>>> to make *any* difference here, we're going to run into this
>>> again and again until we find a way to break up the tests or
>>> get more resources on gitlab.
>>>
>>> Revert frenzy is never a good move. Don't do it please, it's
>>> a waste of time.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20190215/719195ed/signature.sig>