Performance decrease w/Quarkus 1.5.0, Mongo Extension, Mutiny

It seems that there is a considerable throughput performance hit (approx 40%) when upgrading one microservice from an older stack using Quarkus to a newer one.

Older stack:

• Quarkus 1.2.1
• JDK 8 (running in OpenJ9)
• MongoDB extension (with Completion Stages)
• SOAP Calls (but reproducible with HTTP REST, without SOAP)

Newer stack:

• Quarkus 1.5.0
• JDK 11 (running in OpenJ9)
• MongoDB extension (with Mutiny)
• SOAP Calls (but reproducible with HTTP REST, without SOAP)

The microservice uses MongoDB for caching. Steps:

• Querying the DB for a key
  • If the key is found, return the value of this key from the MongoDB.
• If the key is not found, make an HTTP request
• Save the result to MongoDB. The result is saved as a model, using a Jackson codec.
• Return the result to the caller.

Our older stack was using RxJava, and was transforming the CompletionStages to Singles. The new stack uses Mutiny, and also exposes Mutiny Interfaces. In both cases, we use RestEasy interfaces, we are not using Reactive Routes.

On a performance load test lasting for approximately 30 minutes each, I have seen the following differences in performance:

The old stack’s CPU load is something like the following

The new stack’s behaviour is totally different, however.

It seems observed that the CPU consumption went up by 40%, thus losing throughput in the process. Both tests were conducted on a microservice using the same amount of RAM and CPU limits.

Tests already done to debug this:

• Switching to OpenJDK11 image. Tested with both the default garbage collector and the old one (-XX:UseParallelGC). The results were very similar (decreased performance).
• Switching the old implementation using Quarkus 1.2.1 to Java 11 (both OpenJ9). The performance was great (same as the one we had with the old implementation using Java 8)

[csotiriou]

Hello,

Just to add to this issue, I switched to the synchronous client in MongoDB, and I still faced problems. I decided to make another test:

• Use the synchronous MongoDB driver, but return it using Uni.createFrom().item(response) (just to expose Uni interfaces, for testing)
• Use the synchronous MongoDB driver again, but encapsulate it in RxJava’s Single, to simulate as much of the codebase as the one we had before.

I was surprised to see that using RxJava, without any other change in the code, my performance returned back to normal.

I am led to believe that perhaps Mutiny itself (regardless of how it’s used inside the Mongo extension) is the real issue here.

[geoand]

That’s awesome, thanks!