NSubstitute.ReceivedCalls() returning wrong value using AutoFixture.AutoNSubstitute

We have a lot of tests using ‘Class.ReceivedCalls()’ and ‘var tmp = Class.Received(int).Property;’ to check the call count. In v3 of AutoFixture it correctly reported the call count, especially of Properties, which it doesn’t do anymore. The call count of methods seems to be still ok. Given that we have the following code:

    public interface IRunSpeed
    {
        int Speed { get; }
    }

    public class GetToDaChoppa
    {
        private readonly IRunSpeed _runSpeed;

        public GetToDaChoppa(IRunSpeed runSpeed)
        {
            _runSpeed = runSpeed ?? throw new ArgumentNullException(nameof(runSpeed));
        }

        public void DoItNow()
        {
            var runningSpeed = _runSpeed.Speed;
        }
    }

And given that we have the following tests:

        [Fact]
        public void DoItNow_WithOutAutoNSubstitute()
        {
            // Arrange
            var runSpeed = Substitute.For<IRunSpeed>();
            runSpeed.Speed.Returns(2);

            var sut = new GetToDaChoppa(runSpeed);

            //Act
            sut.DoItNow();

            //Assert
            var tmp = runSpeed.Received(1).Speed;
            Assert.Single(runSpeed.ReceivedCalls());
        }

        [Theory, AutoNSubstituteData]
        public void DoItNow_UsingAutoNSubstitute(
            [Frozen] IRunSpeed runSpeed,
            GetToDaChoppa sut)
        {
            // Arrange
            runSpeed.Speed.Returns(49);

            //Act
            sut.DoItNow();

            //Assert
            var tmp = runSpeed.Received(1).Speed;
            Assert.Single(runSpeed.ReceivedCalls());

My AutoNSubstituteDataAttribute looks like this:

        public class AutoNSubstituteDataAttribute : AutoDataAttribute
        {
            public AutoNSubstituteDataAttribute()
                : base(() => new Fixture()
                           .Customize(new AutoNSubstituteCustomization()))
            {
            }
        }

The first test ‘DoItNow_WithOutAutoNSubstitute’ is working fine. But the second test ‘DoItNow_UsingAutoNSubstitute’ returns 2 for ‘ios.Received(1).Speed;’. It also returns 2 for ‘runSpeed.ReceivedCalls();’. Because of this we currently cannot upgrade our Solutions to v4 as we instantly have >1000 failing tests per solution. Any guidance on what the problem might be or where to look for the fix?

[zvirja]

Thanks for firing the issue. Actually, this issue has nothing to do with AutoFixture, while we are affected by it as well 😕

This issue is mostly caused by xUnit and if you rewrite the test as following, it will pass:

[Theory, AutoNSubstituteData]
public void DoItNow_UsingAutoNSubstitute_CreateManually(IFixture fixture)
{
    // Arrange
    var runSpeed = fixture.Freeze<IRunSpeed>();
    var sut = fixture.Create<GetToDaChoppa>();
    runSpeed.Speed.Returns(49);

    //Act
    sut.DoItNow();

    //Assert
    var tmp = runSpeed.Received(1).Speed;
    Assert.Single(runSpeed.ReceivedCalls());
}

When you run tests, xUnit tries to format the test name for you. If it finds that type is not known and doesn’t have ToString() overload, it uses the structural inspection and recursively fetches the property values. If you check the test name, you’ll see the following:

As you might notice, xUnit fetched the Speed property value to nicely show you the test argument. As this is a usual invocation, NSubstitute counted that as a call.

It’s quite strange that you didn’t have this issue with v3, as nothing changed in that regards. I’ve just tested xUnit2 + AutoFixture v3 and still got the issue. Probably, you have upgraded the xUnit as well.

---

As for a workaround, I have good and bad news. The good news is that this will be resolved in the next major version of NSubsitute, as it started to override the ToString() method to return a proxy id. As result, xUnit doesn’t touch the properties anymore:

The bad news is that NSubsitute v4 has not been released yet.

I could suggest you either:

• postpone migration till NSubsitute v4 is released;
• use the latest master source code of NSubsitute, make a local build and use your own NSubsitute.dll instead of NuGet package. After v4 is released, you can switch to the NuGet package.
• if you updated xUnit version as well (as this might explain why you didn’t have this issue before), revert that change and use the previous version of xUnit.

I apologize for the inconvenience, but, unfortunately, we can do nothing on AutoFixture side to fix this issue.

[zvirja]

@dklinger Thanks for the detailed scenario. It’s indeed quite tricky and it’s hard to somehow work around it. From AutoFixture and NSubsitute perspective there no difference whether code is called somewhere deeply inside xUnit or in the test body.

Usually, as a workaround you can use clear feature of NSubstitute:

runSpeed.ClearReceivedCalls();

This code should be run at the prologue of each test where you verify the exact number of calls. It works fine if you have a few tests, however obviously, if thousands of tests are affected, it will not help much 😅

It’s a pity that NSubsitute + xUnit2 + AutoFixture integration doesn’t work well and suffers from this kind of issues. AutoFixture product was designed to simplify the life, rather than to make it a nightmare 😕 Hope guys from xUnit will advice you a quick way to solve the issue for entire project.

Let me know if you believe we can do something from our side to improve the situation.