Error injection in the MSRPC code in Samba as an AD

[Jeremy Allison]

On Fri, Aug 25, 2023 at 09:02:43AM -0700, Richard Sharpe via samba-technical wrote:
>Hi folks (and Jeremy :-),
>
>We have seen a bunch of issues around winbindd and issues caused by
>errors from Windows domain controllers or delays because the DCs are
>far away or slow.
>
>To try to create more robust code I have embarked on a project to
>allow the injection of errors into the Samba AD code when handling
>MSRPC requests.
>
>We tried other approaches, like using the Linux networking code to
>randomly drop packets or inject delays, but this was not very useful.
>We also tried adding error injection into the winbind code when
>handling responses, but that creates issues around packaging. So, it
>seemed like it would be easier to add it to the AD code and then
>provision test systems with the correct setups.
>
>The changes occur in a couple of areas:
>
>1. In PIDL I have modified the generated code to parse an smb.conf
>parameter to do error injection during the RPC library setup and then
>to handle the specific requests when functions are called (see below
>for more details of the smb.conf params.)
>
>2. In librpc/rpc/dcesrv_core.c et al, allow delays to be inserted into
>RPC responses by using some tevent magic.
>
>The sort of smb.conf parameters I am thinking of are:
>
>lsarpc error inject = lsa_LookupSids error NT_STATUS_RPC_CALL_FAILED 10
>
>which says to inject the specified error into lsa_LookupSids every
>tenth call, or
>
>lsarpc error inject = lsa_LookupSids delay 30000 10
>
>inject a delay of 30,000 milliseconds into every tenth call.
>
>Having said all that, there seems to be willingness among my managers
>to try to upstream this code.
>
>So, is this of interest to anyone?
>
>Should I post examples of the changes (to 4.19.0rc1 but they seem to
>apply to master).
>
>Is anyone interested in providing feedback?

Fantastic work Richard. I've already seen a preliminary
version of this patch, sorry I haven't had time to respond
(yet:-).

My only suggestion would be to bracket the changes in a --with-ad-error-injection
at configure time. People able to use this should be able to
build a custom Samba (IMHO).

If you can see any value in having this enabled for real
customers though I'm willing to be wrong (as usual :-).

Cheers,

	Jeremy.