Updating a node from different threads

Hello, I am using neo4j ogm 2.0.0-M02 (HTTP driver) with neo4j server enterprise 2.3.2. I do some tests and it looks that a deadlock issue happens when I try to update one node from several threads. My node entity is the following:

@NodeEntity public class Node { @GraphId Long id; String oid; int name;

public Long getId() { return id; }
public void setId(Long id) { this.id = id; }
public String getOid() { return oid; }
public void setOid(String oid) { this.oid = oid; }
public int getName() { return name; }
public void setName(int name) { this.name = name; }

}

After saving one instance of my Node, I send a cypher request with this code:

Transaction tx = session.beginTransaction(); session.query(“match(n: Node { oid: ‘100’}) set n.name = 10”, Collections.EMPTY_MAP); tx.commit(); tx.close();

Everything works properly. But if I run more than 2 concurrent threads, each thread executing the code above, a deadlock happens and the execution stays stuck until reaching the transactions timeout. Executing the same query without transaction eliminates this issue. Could someone explain me this behavior ?

Vincent [ (via the Neo4j ML)]

[vince-bickers]

@vincentmooser I’ve had a chance to look at this now in more detail. A simple example should help explain what’s happening.

Scenario Assume you have just 1 connection available and 2 threads. Each of these threads wants to update the same node within its own transaction. Here’s a scenario to explain what is happening.

Thread 1 asks for and gets a transaction from Neo4j Thread 2 asks for and gets a transaction from Neo4j Thread 2 updates the object Thread 1 tries to update the object, but it can’t because Thread 2’s transaction still has a lock on it.

So Thread 1 waits for the lock to be released. Thread 2 would like to commit now, But, because Thread 1 is still holding onto the only connection, it cannot. Thread 1 will continue to wait for the lock to be released, and Thread 2 will wait for the connection to be released. So at this point, neither thread is able to make progress.

Eventually, one of the transactions will timeout. At this point, one of two things can happen, depending on which transaction it was.

• If Thread 2’s transaction timed out, the lock will be released by Neo4j. Thread 1 will now be able to proceed, and update the object, releasing the connection. If Thread 2 tries to commit, it will get a 404. Thread 1 will be able to get the connection and commit.
• If Thread 1’s transaction timed out, it will release the connection. Thread 2 can now commit. If Thread 1 tries to commit, it will get a 404.

In this scenario, usually one of the threads will commit. However there is a slight chance that both transactions could timeout, in which case neither will be able to commit of course.

Is this a bug? We don’t think this is a bug in the OGM because you can reproduce this exact same problem just using the Neo4j browser and Postman.

From the documentation, it would seem that when a transaction has a lock on an object and another transaction attempts to also lock it, a deadlock exception should be thrown. However we don’t seem to be getting any such exception at the Http level. I will raise a query with the Neo4j dev team about this.

Resolution If all your threads could be trying to update the same object at the same time, you must ensure that the number of threads you use does not exceed the http connection pool size. (This is now configurable as of 2.0)

[vince-bickers]

@vincentmooser Although the order of operations will be undefined, I believe both updates should succeed without timing out.

You mention you get this behaviour when you try to run with more than 2 threads. My guess is that you are seeing this behaviour due to connection starvation, caused by your version of the OGM library not releasing connection resources appropriately.

A number of fixes have been made regarding closing connections in the latest snapshot version of the OGM, as well as improvements to connection pooling for the OGM HttpClient. Please could you rerun your tests using the latest snapshot version, 2.0.0-SNAPSHOT, and verify whether the problem still exists.