Whitelists are not loaded correctly in driver based tests
See original GitHub issueThe problem
After an upgrade to Corda 4.3 from 4.0, my driver based tests started to fail because the nodes were unable to find classes to be serialized on the whitelist. However, they are present on a whitelist published as a META-INF.service
in the CorDapp.
An investigation into the problem revealed that, when running a node via the driver based test framework, all CorDapps are loaded with an empty list of whitelists. The problem is in the function findWhitelists()
in the file JarScanningCordappLoader.kt
. It extends the classpath with the jar of the CorDapp that has been deployed on the node and then proceeds to create a list of instances of all classes implementing SerializationWhitelist
that can be found on the extended classpath, filtering out the instances whose class does not come from the deployed CorDapp jar.
The code seems to hinge on the assumption that all SerializationWhitelist
services on the classpath will actually be loaded, but this is not true. The documentation of ServiceLoader
includes the following paragraph:
If a particular concrete provider class is named in more than one configuration file, or is named in the same configuration file more than once, then the duplicates are ignored. The configuration file naming a particular provider need not be in the same jar file or other distribution unit as the provider itself. The provider must be accessible from the same class loader that was initially queried to locate the configuration file; note that this is not necessarily the class loader from which the file was actually loaded.
Since the driver framework starts nodes with a classpath that includes the build paths of the project, and because parent classloaders are visited first, ServiceLoader
will instantiate the whitelists from the wrong location, and the instances in the deployed jar will never be created because they have the same package and class names.
This happens regardless of the value of startNodesInProcess
in the driver parameters.
To reproduce in Corda 4.3
The bug is reproduced here: https://github.com/ulrikrasmussen/reproduce-whitelist-bug
Run the driver based test and observe that it fails. However, the whitelist class is actually instantiated, and I have inserted logging code in the constructor which tells you the class location of the instantiated whitelist and the stack trace. This is printed to stdout on the nodes:
[INFO] 12:39:35,356 [main] cordapp.JarScanningCordappLoader. - Looking for CorDapps in [/home/ulrik/deon/dev/corda-whitelist-bug/cordapp-template-kotlin/workflows/build/node-driver/20200131-113853.736/BankA/cordapps]
[INFO] 12:39:35,376 [main] cordapp.JarScanningCordappLoader. - Loading CorDapps from RestrictedURL(url=file:/home/ulrik/deon/dev/corda-whitelist-bug/cordapp-template-kotlin/workflows/build/node-driver/20200131-113853.736/BankA/cordapps/workflows-0.1.jar, rootPackageName=null), RestrictedURL(url=file:/home/ulrik/deon/dev/corda-whitelist-bug/cordapp-template-kotlin/workflows/build/no
de-driver/20200131-113853.736/BankA/cordapps/contracts-0.1.jar, rootPackageName=null)
[INFO] 12:39:35,411 [main] cordapp.JarScanningCordappLoader. - Scanning CorDapp in file:/home/ulrik/deon/dev/corda-whitelist-bug/cordapp-template-kotlin/workflows/build/node-driver/20200131-113853.736/BankA/cordapps/workflows-0.1.jar
[INFO] 12:39:35,556 [main] flows.MyWhitelist. - Whitelist instantiated.
location=file:/home/ulrik/deon/dev/corda-whitelist-bug/cordapp-template-kotlin/workflows/build/classes/kotlin/main/
stacktrace:
java.lang.Thread.getStackTrace(Thread.java:1559)
com.template.flows.MyWhitelist.<init>(MyWhitelist.kt:13)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
java.lang.Class.newInstance(Class.java:442)
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
java.util.ServiceLoader$1.next(ServiceLoader.java:480)
kotlin.collections.CollectionsKt___CollectionsKt.toCollection(_Collections.kt:1062)
kotlin.collections.CollectionsKt___CollectionsKt.toMutableList(_Collections.kt:1095)
kotlin.collections.CollectionsKt___CollectionsKt.toList(_Collections.kt:1086)
net.corda.node.internal.cordapp.JarScanningCordappLoader.findWhitelists(JarScanningCordappLoader.kt:272)
net.corda.node.internal.cordapp.JarScanningCordappLoader.toCordapp(JarScanningCordappLoader.kt:147)
net.corda.node.internal.cordapp.JarScanningCordappLoader.loadCordapps(JarScanningCordappLoader.kt:106)
[... snip ...]
Note that the location of the instantiated class is from the build output, and not the CorDapp jar. Only this instance is created.
Further digging: Why did it work in Corda 4.0?
AMQPSerializationScheme
used to also invoke the ServiceLoader
to find whitelists, which would skip verification of the class location and load the whitelist from the build directory anyway. This has probably masked the issue, and is why the problem did not occur in Corda 4.0. The behavior was removed in #5112, although the description in this PR does not indicate that this change of behavior was intentional.
Is it fixed in Corda 4.4?
Only partially. Tested at version 1380779a9c.
The easiest way to test this was to apply the patch in the bottom of this issue to the simm-valuation-demo and observe the location of the instantiated whitelist in stdout
when running the driver based test. I ran the driver based test with startNodesInProcess
equal to true
and false
, respectively.:
startNodesInProcess = false
: The loaded whitelist is the correct one from the deployed jar, so this case seems to have been fixed.startNodesInProcess = true
: The incorrect whitelist from the build output is loaded. Also, custom serializers cannot be found, so the test fails for other reasons before it can get to the whitelist issue. This is probably a related bug that also has to be fixed.
Debugging Patch:
diff --git a/samples/simm-valuation-demo/contracts-states/src/main/kotlin/net/corda/vega/plugin/SimmContractsPluginRegistry.kt b/samples/simm-valuation-demo/contracts-states/src/main/kotlin/net/corda/vega/plugin/SimmContractsPluginRegistry.kt
index 1e6abdfd0f..e4f7c2fd72 100644
--- a/samples/simm-valuation-demo/contracts-states/src/main/kotlin/net/corda/vega/plugin/SimmContractsPluginRegistry.kt
+++ b/samples/simm-valuation-demo/contracts-states/src/main/kotlin/net/corda/vega/plugin/SimmContractsPluginRegistry.kt
@@ -11,10 +11,13 @@ import com.opengamma.strata.market.param.CurrencyParameterSensitivities
import com.opengamma.strata.market.param.CurrencyParameterSensitivity
import com.opengamma.strata.market.param.TenorDateParameterMetadata
import com.opengamma.strata.market.param.ParameterMetadata
+import net.corda.core.internal.location
import net.corda.core.serialization.SerializationWhitelist
import net.corda.vega.analytics.CordaMarketData
import net.corda.vega.analytics.InitialMarginTriple
+import org.slf4j.LoggerFactory
+private val log = LoggerFactory.getLogger(SimmContractsPluginRegistry::class.java)
/**
* [SimmService] is the object that makes available the flows and services for the Simm agreement / evaluation flow.
* It is loaded via discovery - see [SerializationWhitelist].
@@ -22,6 +25,11 @@ import net.corda.vega.analytics.InitialMarginTriple
* It is loaded via discovery see [WebServerPluginRegistry].
*/
class SimmContractsPluginRegistry : SerializationWhitelist {
+ init {
+ val cp = SimmContractsPluginRegistry::class.java.location
+ log.info("Whitelist instantiated.\n\tlocation=$cp\n\tstacktrace:\n\t\t${Thread.currentThread().stackTrace.joinToString("\n\t\t")}")
+ }
+
override val whitelist = listOf(
MultiCurrencyAmount::class.java,
Ordering.natural<Comparable<Any>>().javaClass,
Issue Analytics
- State:
- Created 4 years ago
- Comments:13 (13 by maintainers)
Filed as https://r3-cev.atlassian.net/browse/CORDA-3566.
Thanks for the excellent investigation / analysis.
Hi @ulrikrasmussen, I think I fixed this in CORDA-3643, which should be included in Corda 4.5 😃 .