MemberAutoData only uses first entry from the supplied enumerable

In AutoFixture.xUnit2, the MemberAutoData attribute only causes one test to be generated regardless of the size of the supplied data. You can see this in the test AutoFixture.Xunit2.UnitTest.Scenario.MemberAutoDataSuppliesDataSpecimens.

A similar test with MemberData and a member supplying a similar but “full” enumerable will generate two test cases:

[Theory, MemberData(nameof(StringData))]
public void MemberDataSuppliesDataSpecimens(string s1, string s2, MyClass myClass)
{
    Assert.Equal("foo", s1);
    Assert.NotNull(s2);
    Assert.NotNull(myClass);
}

public static IEnumerable<object[]> StringData
{
    get
    {
        yield return new object[] { "foo", "bar", new MyClass() };
        yield return new object[] { "foo", "baz", new MyClass() };
    }
}

[Ergamon]

The problem is that the implementation uses the CompositeDataAttribute base class. This one tries to merge the data from MemberDataAttribute and AutoDataAttribute, but the later always returns one row. So in your example MemberDataAttribute returns 2 rows, AutoDataAttribute only 1. The code merges this to the smallest number found, so only 1 row is generated.

I tried my best understanding of the AutoFixture source code and generated a (hopefully) working implementation:

  using System;
  using System.Collections.Generic;
  using System.Linq;
  using System.Reflection;
  using System.Threading;
  using AutoFixture;
  using AutoFixture.Kernel;
  using AutoFixture.Xunit2;
  using Xunit;
  using Xunit.Sdk;

  public class MemberAutoDataAttribute : DataAttribute
  {
    private readonly Lazy<IFixture> fixture;
    private readonly MemberDataAttribute memberDataAttribute;

    public MemberAutoDataAttribute(string memberName, params object[] parameters)
      : this(memberName, parameters, () => new Fixture())
    {
    }

    protected MemberAutoDataAttribute(string memberName, object[] parameters, Func<IFixture> fixtureFactory)
    {
      if (fixtureFactory == null)
      {
        throw new ArgumentNullException(nameof(fixtureFactory));
      }
      
      memberDataAttribute = new MemberDataAttribute(memberName, parameters);
      fixture = new Lazy<IFixture>(fixtureFactory, LazyThreadSafetyMode.PublicationOnly);
    }

    public override IEnumerable<object[]> GetData(MethodInfo testMethod)
    {
      if (testMethod == null)
      {
        throw new ArgumentNullException(nameof(testMethod));
      }

      var memberData = memberDataAttribute.GetData(testMethod);

      using (var enumerator = memberData.GetEnumerator())
      {
        if (enumerator.MoveNext())
        {
          var specimens = GetSpecimens(testMethod.GetParameters(), enumerator.Current.Length).ToArray();

          do
          {
            yield return enumerator.Current.Concat(specimens).ToArray();
          } while (enumerator.MoveNext());
        }
      }
    }

    private IEnumerable<object> GetSpecimens(IEnumerable<ParameterInfo> parameters, int skip)
    {
      foreach (var parameter in parameters.Skip(skip))
      {
        CustomizeFixture(parameter);

        yield return Resolve(parameter);
      }
    }

    private void CustomizeFixture(ParameterInfo p)
    {
      var customizeAttributes = p.GetCustomAttributes()
          .OfType<IParameterCustomizationSource>()
          .OrderBy(x => x, new CustomizeAttributeComparer());

      foreach (var ca in customizeAttributes)
      {
        var c = ca.GetCustomization(p);
        fixture.Value.Customize(c);
      }
    }

    private object Resolve(ParameterInfo p)
    {
      var context = new SpecimenContext(fixture.Value);

      return context.Resolve(p);
    }

    private class CustomizeAttributeComparer : Comparer<IParameterCustomizationSource>
    {
      public override int Compare(IParameterCustomizationSource x, IParameterCustomizationSource y)
      {
        var xfrozen = x is FrozenAttribute;
        var yfrozen = y is FrozenAttribute;

        if (xfrozen && !yfrozen)
        {
          return 1;
        }

        if (yfrozen && !xfrozen)
        {
          return -1;
        }

        return 0;
      }
    }
  }

So basically I iterate over the results from MemberDataAttribute while adding everytime the data from AutoDataAttribute.

The CustomizeFixture and the Resolve methods are copies from AutoDataAttribute. The CustomizeAttributeComparer class is a direct copy from the source code as the original class is internal.

The protected constructor is there to provide the possibility to inherit, e.g. while testing I made my own MemberAutoNSubstituteDataAttribute.

[thomOrbelius]

@Ergamon thank you for the workaround, it really helped me out.

Also we created a ClassAutoDataAttribute as well, with the slight modification of using ClassDataAttribute instead of MemberDataAttribute.