Error injection in the MSRPC code in Samba as an AD

[Richard Sharpe]

On Fri, Aug 25, 2023 at 9:40 AM Jeremy Allison <jra at samba.org> wrote:
>
> On Fri, Aug 25, 2023 at 09:02:43AM -0700, Richard Sharpe via samba-technical wrote:
> >Hi folks (and Jeremy :-),
> >
> >We have seen a bunch of issues around winbindd and issues caused by
> >errors from Windows domain controllers or delays because the DCs are
> >far away or slow.
> >
> >To try to create more robust code I have embarked on a project to
> >allow the injection of errors into the Samba AD code when handling
> >MSRPC requests.
> >
> >We tried other approaches, like using the Linux networking code to
> >randomly drop packets or inject delays, but this was not very useful.
> >We also tried adding error injection into the winbind code when
> >handling responses, but that creates issues around packaging. So, it
> >seemed like it would be easier to add it to the AD code and then
> >provision test systems with the correct setups.
> >
> >The changes occur in a couple of areas:
> >
> >1. In PIDL I have modified the generated code to parse an smb.conf
> >parameter to do error injection during the RPC library setup and then
> >to handle the specific requests when functions are called (see below
> >for more details of the smb.conf params.)
> >
> >2. In librpc/rpc/dcesrv_core.c et al, allow delays to be inserted into
> >RPC responses by using some tevent magic.
> >
> >The sort of smb.conf parameters I am thinking of are:
> >
> >lsarpc error inject = lsa_LookupSids error NT_STATUS_RPC_CALL_FAILED 10
> >
> >which says to inject the specified error into lsa_LookupSids every
> >tenth call, or
> >
> >lsarpc error inject = lsa_LookupSids delay 30000 10
> >
> >inject a delay of 30,000 milliseconds into every tenth call.
> >
> >Having said all that, there seems to be willingness among my managers
> >to try to upstream this code.
> >
> >So, is this of interest to anyone?
> >
> >Should I post examples of the changes (to 4.19.0rc1 but they seem to
> >apply to master).
> >
> >Is anyone interested in providing feedback?
>
> Fantastic work Richard. I've already seen a preliminary
> version of this patch, sorry I haven't had time to respond
> (yet:-).
>
> My only suggestion would be to bracket the changes in a --with-ad-error-injection
> at configure time. People able to use this should be able to
> build a custom Samba (IMHO).

D'oh. Of course this is needed.

> If you can see any value in having this enabled for real
> customers though I'm willing to be wrong (as usual :-).
>
> Cheers,
>
>         Jeremy.



-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)(传说杜康是酒的发明者)