RESTEasy Reactive: Lock contention between callback processing and resume()

Description

I was running some performance tests on Resteasy reactive and picked up some lock contention (about 11% of all contention during my load test) which I think may not be needed:

Its this code here on org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext:

 synchronized void onComplete(Throwable throwable) {
        if (completionCallbacks != null) {
            for (CompletionCallback callback : completionCallbacks) {
                callback.onComplete(throwable);
            }
        }
 }
@Override
    public synchronized void registerCompletionCallback(CompletionCallback callback) {
        if (completionCallbacks == null) {
            completionCallbacks = new ArrayList<>();
        }
        completionCallbacks.add(callback);
    }

I am assuming the onComplete is contending with the resume() calls which also synchronize on the object.

Implementation ideas

In order to resolve this, and as a by-product avoid state issues of callbacks being added after they may have been processed, I suggest using an explicit lock and state flag as opposed to locking on (this):

private final Lock stateManagementLock = new ReentrantLock(); // Prevent race conditions on callbacks without needing to lock on (this)
private boolean completionCallbacksInvoked = false;

void onComplete(Throwable throwable) {
        try {
            stateManagementLock.lock();
            if (completionCallbacks != null) {
                for (CompletionCallback callback : completionCallbacks) {
                    callback.onComplete(throwable);
                }
                completionCallbacks = null;
            }
            completionCallbacksInvoked = true;
        } finally {
            stateManagementLock.unlock();
        }
    }

    @Override
    public void registerCompletionCallback(CompletionCallback callback) {
        try {
            stateManagementLock.lock();
            if (completionCallbacksInvoked) {
                throw new IllegalStateException("Completion callbacks cannot be registered after the context has been closed");
            }
            if (completionCallbacks == null) {
                completionCallbacks = new ArrayList<>();
            }
            completionCallbacks.add(callback);
        } finally {
            stateManagementLock.unlock();
        }
 }

[bcluap]

Very reasonable. Cannot see why one would ever go from worker back to eventloop on purpose

On Fri, Mar 19, 2021 at 9:26 AM Georgios Andrianakis < @.***> wrote:

As I am certain that the @NonBlocking -> @Blocking -> @NonBlocking dance will cause problems, here is what I propose:

We do add support for @NonBlocking on filters, but all the @NonBlocking filters must be the first ones to execute - otherwise fail.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/15732#issuecomment-802615743, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGMB23KJT6QSKJT77IW6S3TTEL4D3ANCNFSM4ZGZUMYQ .

[geoand]

https://github.com/quarkusio/quarkus/pull/15882 should take care of it