@RunWith(Enclosed.class) and its @BeforeClass

ℹ️ preliminary note: this issue might not be an Junit issue at all but a question about how it is designed and probably an error in gradle / IntelliJ

💁 we actually ran into some problems with Enclosed tests, a BeforeClass method in the enclosing class:

package test;

import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.experimental.runners.Enclosed;
import org.junit.runner.RunWith;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;

@RunWith(Enclosed.class)
public class TestSomething {
  private static Object someSetupNeeded;

  @BeforeClass
  public static void runBeforeEnclosedSuite() {
    someSetupNeeded = new SomeObject();
  }

  @Category(testcategory.Fast.class) // yes, not slow
  public static class InnerTestClass {
    @Before
    public void setUp() {
      boltStreams.toString(); // NPE on gradle console but not in intelliJ nor when running enclosed test class soley
    }

    @Test
    public void testSomeObject() {
      assertThat(someSetupNeeded, notNullValue());
    }
  } 
}

<strike> with

package testcategory;
public interface Fast {
}

</strike> <strike>

package testcategory;
public interface Slow {
}

</strike> <strike> with build.gradle settings

test {
    useJUnit {
            excludeCategories 'testcategory.Slow'
    }
}

</strike>

when running ./gradlew clean test some of us gets NPE in the inner before class while others don’t. it is ok if you run the test with ./gradlew clean test --test test.TestSomething None of us have any problems running the outer TestSomething class in IntelliJ.

Just to complete: All of us have the same problem, that a single inner test class (TestSomething.InnerTestClass) run in IntelliJ will fail -> for me, an Intelli problem not running a correct suite or building it’s own but not using JUnit test filter. So the result is the same as for those who get the error on console gradlew test : beforeClass of the encapsulating class is not executed when running an inner test class

❓ no1: what is the designed behaviour of this <strike>combination</strike> of Enclosed test suits <strike>and Category('s / filters)</strike>? ❓ no2: why is this not reproducable for all environment using the same build tool version and junit?

used:

• gradle 2.13 (but also tested with gradle 2.14.1) (both, using same wrapper checked in in project)
• IntelliJ CE 2016 (both)
• developer1: Sun Java 8u91 (error as described above) ❓
• developer2: Sun Java 8u102 (no error on console) ❓ ❗ ❓

P.S: topic someway related: #656

P.S: this is not a category related issue, it is reproducable this the same code above without categories

[stefanbirkner]

This is a Gradle issue. I had a look at the Enclosed runners code and tested with IntelliJ, Gradle and Maven. Only Gradle has the problem. I think its a problem of Gradle’s test discovery. Please use gradle/gradle#2320 for further information about this problem.

[kcooney]

This is a Gradle issue.