Kryo serialising logger during finality flow

We have been getting the following error

"FiberTimedScheduler-Same thread scheduler" java.lang.OutOfMemoryError: GC overhead limit exceeded

During the finality flow while transferring 100 token states. After a lengthy investigation we found the the Kyro serializer was trying to serialize log4j which contains a rather large LMAX disruptor ring buffer.

The screen shot above shows the logger on the stack and about the be serialised. This started to happen when we upgraded from Corda OS 4.0 to 4.4.

Viewing this in visual VM you can see that the garbage collector is trying to kick in and collect, but it appears to be blocked.

We had this issue a couple of years ago and I think @rick-r3 fixed it.

Here’s the dependency list

corda_release_group = 'net.corda'
corda_release_version = '4.4'
corda_finance_version = '4.0'
corda_gradle_plugins_version = '5.0.6'
kotlin_version = '1.2.71'
junit_version = '4.12'
quasar_version = '0.7.12_r3'
spring_boot_version = '2.0.2.RELEASE'
spring_boot_gradle_plugin_version = '2.0.2.RELEASE'
slf4j_version = '1.7.25'
log4j_version = '2.9.1'
rxjava_version = '1.3.8'
hamkrest_version = '1.4.2.2'
mockito_kotlin_version = '1.5.0'
braid_version = '4.4.0'
vertx_version = '3.7.0'
jacoco_version = '0.8.1'
license_gradle_plugin_version = '0.14.0'assertj_version = '3.11.1'
jolokia_agent_version = '1.6.0'
postgres_driver_version = '42.2.5'
corda_platform_version = '4'

This issue will greatly affect performance of corda OS, but will only be noticeable if you are moving lots of states around.

Let me know if you need any more info Thanks Mark

[Bartman250]

basically make sure you have the same logger dependancies as corda - i.e slf4j etc. If you have different dependancies then it will try and deserialise the wrong logger and blow up. Apologies for lack of write up - been too busy !

[r3jirabot]

Automatically created Jira issue: CORDA-3821