Summary of flapping tests this year

[Douglas Bagnall]

On 22/02/19 4:56 PM, Scott Lovenberg wrote:

>>
> 
> If I’m understanding correctly, it also seems that the majority of these flapping tests are due to race conditions or of some sort of asynchronous nature either within the actual code or the test framework itself?
> 

That is a reasonable bet. The tests are full of waits and time-outs, and
the server does a lot more than these periodic tests so the load is
unpredictable.

Over the last year we have also found a few cases where our Python
bindings failed to initialise variables, which naturally leads to
indeterminacy. But I can't tell you if any flapping tests have got
better as a result -- you tend not to notice them when they're gone.

Douglas