SSL/TLS Configuration for Zookeeper,BookKeeper and Pulsar

hello ,

i want to add ssl/tls configuration to zookeeper using the zookeeper.conf , so first step is modify the the args in the statefulset.yaml file by deleting bin/generate-zookeeper-config.sh conf/zookeeper.conf && cat conf/zookeeper.conf && from the image ,

args: - > bin/wait-for-all-dns.sh && bin/wait-for-my-dns.sh && bin/apply-config-from-env.py conf/zookeeper.conf && bin/apply-config-from-env.py conf/pulsar_env.sh && bin/pulsar zookeeper

if i want to provide zookeeper.conf externally via a configmap , how do i load so that i can add all the ssl configurtion into that file.

the error when starting zookeeper POD with above configuration is ,

> Traceback (most recent call last):
>   File "bin/apply-config-from-env.py", line 44, in <module>
>     for line in open(conf_filename):
> IOError: [Errno 2] No such file or directory: 'conf/pulsar_env.sh

Thanks,

[hari819]

@sijie @skyrocknroll @rounak11 , did not have time to prepare a document when i completed enabling TLS in February.Hope the following config is helpful bin/pulsar(sh file) and conf files,

ZooKeeper:

elif [ $COMMAND == "zookeeper" ]; then
    PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"zookeeper.log"}
    ZK_OPTS=" -Dzookeeper.4lw.commands.whitelist=* -Dzookeeper.snapshot.trust.empty=true -Djava.security.auth.login.config=conf/zk_jaas.conf -Dzookeeper.requireClientAuthScheme=sasl -Dzookeeper.sasl.client=true -Dzookeeper.sasl.clientconfig=Client -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory -Dzookeeper.ssl.keyStore.location=keys/KeyStore.jks -Dzookeeper.ssl.keyStore.password=keys/jkspassword -Dzookeeper.ssl.trustStore.location=keys/TrustStore.jks -Dzookeeper.ssl.trustStore.password=keys/jkspassword  -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.client.secure=true -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.authProvider.2=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.authProvider.3=org.apache.zookeeper.server.auth.SASLAuthenticationProvider"

zookeeper.conf:

secureClientPort=2281

quorum.auth.enableSasl=true
quorum.auth.learnerRequireSasl=true
quorum.auth.serverRequireSasl=true
quorum.auth.learner.saslLoginContext=QuorumLearner
quorum.auth.server.saslLoginContext=QuorumServer

requireClientAuthScheme=sasl
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
authProvider.2=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
authProvider.3=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

Bookkeeper:

elif [ $COMMAND == "bookie" ]; then
    PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"bookkeeper.log"}
    # Pass BOOKIE_EXTRA_OPTS option defined in pulsar_env.sh
    BOOKIE_EXTRA_OPTS=" -Djavax.net.debug=all -Djavax.net.debug=ssl:handshake:verbose -Dzookeeper.client.secure=true -Djava.security.auth.login.config=conf/bk_jaas.conf"
    OPTS="$OPTS $BOOKIE_EXTRA_OPTS"
    exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.bookkeeper.proto.BookieServer --conf $PULSAR_BOOKKEEPER_CONF $@

bookkeeper.conf:

` tlsProvider=OpenSSL

tlsProviderFactoryClass=org.apache.bookkeeper.tls.TLSContextFactory

tlsClientAuthentication=true

tlsEnabledProtocols=TLSv1.2

tlsKeyStoreType=JKS

tlsKeyStore=bookie.keystore.jks

tlsKeyStorePasswordPath=bookie.keystore.passwd

tlsTrustStoreType=JKS

tlsTrustStore=bookie.truststore.jks

tlsTrustStorePasswordPath=bookie.truststore.passwd

clientTrustStore=client.truststore.jks clientTrustStorePasswordPath=client.truststore.passwd clientKeyStore=client.keystore.jks clientKeyStorePasswordPath=client.keystore.passwd`

Pulsar(Broker):

if [ $COMMAND == "broker" ]; then PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"pulsar-broker.log"} exec $JAVA $OPTS -Djavax.net.debug=all -Djavax.net.debug=ssl:handshake:verbose -Djava.security.auth.login.config=bk_jaas.conf $ASPECTJ_AGENT -Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.pulsar.PulsarBrokerStarter --broker-conf $PULSAR_BROKER_CONF $@

broker.conf

`tlsEnabled=true

tlsCertRefreshCheckDurationSec=300

tlsCertificateFilePath=tls.crt.pem

tlsKeyFilePath=tls.key.pem

tlsTrustCertsFilePath=ca.cert.pem

tlsAllowInsecureConnection=false

tlsProtocols=TLSv1.2,TLSv1.1

tlsRequireTrustedClientCertOnConnect=false

authenticationEnabled=true

authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls

authorizationEnabled=false

authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider

brokerClientTlsEnabled=true brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls brokerClientAuthenticationParameters=tlsCertFile:tls.crt.pem,tlsKeyFile:tls.key.pem brokerClientTrustCertsFilePath=ca.cert.pem

bookkeeperTLSProviderFactoryClass=org.apache.bookkeeper.tls.TLSContextFactory

bookkeeperTLSClientAuthentication=true

bookkeeperTLSKeyFileType=JKS

bookkeeperTLSTrustCertTypes=JKS

bookkeeperTLSKeyStorePasswordPath=bookie.keystore.passwd

bookkeeperTLSTrustStorePasswordPath=bookie.truststore.passwd`

[sijie]

@rounak11 I have fixed the links. Please check them again.