BadReaderLockException on concurrent read transactions

Opening two concurrent read txns gives an error. Here’s a unit test to reproduce (for TxnTest.java):

  @Test
  public void readOnlyConcurrentTxnAllowedInReadOnlyEnv() {
    env.openDbi(DB_1, MDB_CREATE);
    final Env<ByteBuffer> roEnv = create().open(path, MDB_NOSUBDIR,
            MDB_RDONLY_ENV);
    Txn<ByteBuffer> txn1 = roEnv.txnRead();
    Txn<ByteBuffer> txn2 = roEnv.txnRead();
    assertThat(txn1, is(notNullValue()));
    assertThat(txn2, is(notNullValue()));
    assertThat(txn1, is(not(sameInstance(txn2))));
    assertThat(txn1.getId(), is(not(txn2.getId())));
    txn1.close();
    txn2.close();
  }

Stacktrace:

org.lmdbjava.Txn$BadReaderLockException: Invalid reuse of reader locktable slot (-30783)

    at org.lmdbjava.ResultCodeMapper.<clinit>(ResultCodeMapper.java:54)
    at org.lmdbjava.Env$Builder.open(Env.java:369)
    at org.lmdbjava.TxnTest.before(TxnTest.java:81)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)


Process finished with exit code 255

[benalexau]

I have just reverted of most of the Txn thread guard work.

Upon further reflection and working on my own LmdbJava-reliant applications in recent days, I can see that ThreadLocal-enforced single transactions per thread will impose unreasonable restrictions in certain cases. There’s a lot of flexibility provided by MDB_NOTLS if users know what they’re doing (cf @phraktle’s wrapper code), and as such it seems unwise to block advanced usage via a ThreadLocal in lower-level types such as Env.

[benalexau]

My primary concerns are code complexity (thus maintenance effort) and blocking paths for advanced users who determine they require a different model for Txn management (eg their own reuse approaches, or multiple Txns per thread, or just the default native library managed handle reuse etc).

I am not opposed to adding a strategy interface to LmdbJava to simplify alternate transaction handle allocators. For example, it would receive callbacks on Env creation, Txn acquisition and Txn close. If we had this users would be able to use standard Env methods to acquire and dispose of transactions without having to know about handle recycling internals. The applicable strategy could just be set in the Env.Builder. I would not mind if a ThreadLocalTxnAllocator was shipped with LmdbJava, beside the default NativeLibTxnAllocator (just quick example names). Does this seem a reasonable way forward?