Incompatible BouncyCastle FIPS changes in bookkeeper 4.14

Describe the bug In bookkeeper # 2631, the default BouncyCastle was changed from non-fips into fips version. But the default version of BouncyCastle in Pulsar is the non-fips one(aimed to make it compatible with the old version of Pulsar).

Bouncy Castle provides both FIPS and non-FIPS version, but in a JVM, it can not include both of the 2 versions(non-Fips and Fips), and we have to exclude the current version before including the other. This make the backward compatible a little hard, and that’s why Pulsar has to involve individual module for Bouncy Castle.

Pulsar excluded the dependencies of bookkeeper-server’s BouncyCastle in Pulsar’s pom file, and Pulsar only includes the non-fips version, but the bookkeeper-server still wants to use the hard-coded fips version in bookkeeper # 2631

And if we want to start BookKeeper with TLS enabled through Pulsar’s binary, it will meet the following error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/bouncycastle/jcajce/provider/BouncyCastleFipsProvider
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:315)
	at org.apache.bookkeeper.common.util.ReflectionUtils.forName(ReflectionUtils.java:49)
	at org.apache.bookkeeper.tls.SecurityProviderFactoryFactory.getSecurityProviderFactory(SecurityProviderFactoryFactory.java:39)
	at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:129)
	at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:52)
	at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304)
	at org.apache.bookkeeper.server.Main.doMain(Main.java:226)
	at org.apache.bookkeeper.server.Main.main(Main.java:208)
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 9 more

To Reproduce Steps to reproduce the behavior:

1. using pulsar 2.8.0,
2. start bookkeeper through bin/pulsar bookie, with TLS enabled.

Expected behavior By using bin/pulsar bookie, BookKeeper server should able to start success. BK should not hard-coded fips version for Bouncy Castle, it should have an option to use non-fips version to make it compatible.

Additional context We may need to provide the fix in the BookKeeper side first, and then change the dependency of bookkeeper version in Pulsar.

[tisonkun]

@codelipenghui @hangc0276 is this ticket still relevant?

It’s weird when I filter release/blocker and see this issue targets to 2.8.1 and stale for a long time.

At least we should correct the metadata as @eolivelli mentioned https://github.com/apache/pulsar/issues/10937#issuecomment-926598889

[tisonkun]

Closed as completed. I don’t think it’s still relevant. For integration tests, a new issue is better.