Flaky tests in CI (12% (now 3%) chance of failure)

I’ve just run (Linux Release) CI tests 50 times in a row and out of 50 runs 6 failed with following failures:

5x (Fix pending https://github.com/dotnet/iot/pull/945):

    System.Device.Gpio.Tests.RaspberryPiDriverTests.InputPullResistorsWork [FAIL]
      Assert.Equal() Failure
      Expected: Low
      Actual:   High
      Stack Trace:
        /_/src/System.Device.Gpio.Tests/RaspberryPiDriverTests.Linux.cs(34,0): at System.Device.Gpio.Tests.RaspberryPiDriverTests.InputPullResistorsWork()

1x:

    System.Device.Gpio.Tests.SysFsDriverTests.AddCallbackFallingEdgeNotDetectedTest [FAIL]
      Assert.False() Failure
      Expected: False
      Actual:   True
      Stack Trace:
        /_/src/System.Device.Gpio.Tests/GpioControllerTestBase.cs(176,0): at System.Device.Gpio.Tests.GpioControllerTestBase.AddCallbackFallingEdgeNotDetectedTest()

cc: @pgrawehr @joperezr

We should try to understand the failures and if this is by design then retry or remove the test

[krwq]

After merging pull up/down PR here are the stats.

3/100 failed with all 3 failures being:

    System.Device.Gpio.Tests.SysFsDriverTests.AddCallbackFallingEdgeNotDetectedTest [FAIL]
      Assert.False() Failure
      Expected: False
      Actual:   True
      Stack Trace:
        /_/src/System.Device.Gpio.Tests/GpioControllerTestBase.cs(176,0): at System.Device.Gpio.Tests.GpioControllerTestBase.AddCallbackFallingEdgeNotDetectedTest()

[pgrawehr]

I agree for now. I was already trying to improve this test (when I was doing the performance analysis) but could only improve it, but apparently not fully fix it. It seems it has to do with some pending interrupts at the time the event callback is set up, but the picture is not so clear.