java.lang.IllegalStateException: Was not expecting to find existing database transaction on current strand when setting database when running multiple flow unit tests

full stack trace:

java.lang.IllegalStateException: Was not expecting to find existing database transaction on current strand when setting database: Thread[main,5,main], org.jetbrains.exposed.sql.Transaction@4a0ee0ba

	at net.corda.node.utilities.StrandLocalTransactionManager.<init>(DatabaseSupport.kt:130)
	at net.corda.node.utilities.DatabaseSupportKt$configureDatabase$database$1.invoke(DatabaseSupport.kt:54)
	at net.corda.node.utilities.DatabaseSupportKt$configureDatabase$database$1.invoke(DatabaseSupport.kt)
	at org.jetbrains.exposed.sql.Database$Companion.connect(Database.kt:51)
	at net.corda.node.utilities.DatabaseSupportKt.configureDatabase(DatabaseSupport.kt:54)
	at net.corda.node.internal.AbstractNode.initialiseDatabasePersistence(AbstractNode.kt:338)
	at net.corda.node.internal.AbstractNode.start(AbstractNode.kt:214)
	at net.corda.testing.node.MockNetwork$MockNode.start(MockNode.kt:212)
	at net.corda.testing.node.MockNetwork$MockNode.start(MockNode.kt:128)
	at net.corda.testing.node.MockNetwork.createNode(MockNode.kt:283)
	at net.corda.testing.node.MockNetwork.createNode(MockNode.kt:256)
	at net.corda.testing.node.MockNetwork.createNode$default(MockNode.kt:254)
	at net.corda.testing.node.MockNetwork.createSomeNodes(MockNode.kt:351)
	at net.corda.testing.node.MockNetwork.createSomeNodes$default(MockNode.kt:344)
	at com.r3cev.voltron.flow.LOCAmendFinaleFlowTester.setup(LOCAmendFinaleFlowTester.kt:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)


java.lang.IllegalArgumentException: Failed requirement.

	at net.corda.testing.node.MockNetwork.stopNodes(MockNode.kt:384)
	at com.r3cev.voltron.flow.LOCAmendFinaleFlowTester.tearDown(LOCAmendFinaleFlowTester.kt:210)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

It was reproduced in some of the Voltron unit tests. Jose is looking into it…

[josecoll]

This error manifests itself when code inadvertently causes creation of a new transaction outside of an explicitly demarcation transaction boundary.

This problem often manifests itself’s in JUnit tests (and more so in Java than Kotlin due to the different setup code used in each language). You will observe that individual JUnit tests execute successfully when ran individually, but fail when executed as a complete suite.

The immediate workaround is to ensure that all calling code to setup services in the @Before setUp() function of JUnits is included within explicit transaction boundaries. For example:

    @Before
    public void setUp() {
        Properties dataSourceProps = makeTestDataSourceProperties(SecureHash.randomSHA256().toString());
        Pair<Closeable, Database> dataSourceAndDatabase = configureDatabase(dataSourceProps);
        dataSource = dataSourceAndDatabase.getFirst();
        database = dataSourceAndDatabase.getSecond();

        Set<MappedSchema> customSchemas = new HashSet<>(Arrays.asList(DummyLinearStateSchemaV1.INSTANCE));
        HibernateConfiguration hibernateConfig = new HibernateConfiguration(new NodeSchemaService(customSchemas));
        transaction(database,
                    statement -> { services = new MockServices(getMEGA_CORP_KEY()) {
                        @NotNull
                        @Override
                        public VaultService getVaultService() {
                            return makeVaultService(dataSourceProps, hibernateConfig);
                        }

                        @Override
                        public VaultQueryService getVaultQueryService() {
                            return new HibernateVaultQueryImpl(hibernateConfig, getVaultService().getUpdatesPublisher());
                        }

                        @Override
                        public void recordTransactions(@NotNull Iterable<SignedTransaction> txs) {
                            for (SignedTransaction stx : txs) {
                                getStorageService().getValidatedTransactions().addTransaction(stx);
                            }

                            Stream<WireTransaction> wtxn = StreamSupport.stream(txs.spliterator(), false).map(txn -> txn.getTx());
                            getVaultService().notifyAll(wtxn.collect(Collectors.toList()));
                        }
                    };
            vaultSvc = services.getVaultService();
            vaultQuerySvc = services.getVaultQueryService();

            return services;
        });
    }

Note the calls to get the vaultSvc and vaultQuerySvc Mock Services must be within the transaction boundary. Simply by moving those two call outside of the transaction boundary will cause the java.lang.IllegalStateException: Was not expecting to find existing database transaction on current strand (because internally these services create a new transaction if one does not already exist - and hold on to them.).

Code changes associated with this Issue will ensure that code fails fast if it is not within a transaction but should be (ie. will no longer automatically create a new transaction).

[joeldudley]

Fixed.