websocket when ssl tsl is enabled, error reported: Error during handshake

Describe the bug

websocket when ssl tsl is enabled, error reported: Error during handshake

---

To Reproduce

Steps to reproduce the behavior:

1. config：broker.conf, client.conf, proxy.conf, standalone.conf, websocket.conf

2. run：bin/pulsar standalone

3. pulsar-client is good：bin/pulsar-client produce my-topic --messages “hello-pulsar”

4. python3 test.py: Handshake status 500 Server Error

---

Expected behavior

I was expecting the same thing as pulsar-client, that I could connect, but I didn’t expect a handshake failure

---

doc

https://pulsar.apache.org/docs/en/security-tls-transport/

https://pulsar.apache.org/docs/en/client-libraries-websocket/#python

---

Screenshots

Desktop (please complete the following information):

• OS: linux centos v4.18.0-240.el8.x86_64
• java: v1.8.0_31, 64-Bit
• pulsar: v2.8.1
• py: v3.9.6

---

Additional context

test2.py

import websocket, base64, json, ssl

# https://websocket-client.readthedocs.io/en/latest/faq.html#what-else-can-i-do-with-sslopts
# https://pulsar.apache.org/docs/en/client-libraries-websocket/#python


# my_context = ssl.create_default_context()
# my_context.load_verify_locations('/root/my-ca4/ca-cert.pem')

ws = websocket.WebSocket(sslopt={"ca_cert_path": "/root/my-ca4/ca-cert.pem", "cert_reqs": ssl.CERT_NONE, "check_hostname": False})

print('0')

ws.connect("wss://139.198.15.174:8443/ws/v2/producer/persistent/public/default/my-topic", timeout=None)

print('1')

# # Send one message as JSON
ws.send(json.dumps({
    'payload' : base64.b64encode('Hello World'),
    'properties': {
        'key1' : 'value1',
        'key2' : 'value2'
     },
    'context' : 5
}))

print('2')

Key configuration:

standalone.conf

configurationStoreServers=

brokerServicePort=6650

# Port to use to server HTTP request

webServicePort=8080

webServicePortTls=8443
brokerServicePortTls=6651


### --- TLS --- ###
# Deprecated - Use webServicePortTls and brokerServicePortTls instead
tlsEnabled=true

# Tls cert refresh duration in seconds (set 0 to check on every new connection)
tlsCertRefreshCheckDurationSec=300

# Path for the TLS certificate file
tlsCertificateFilePath=/root/my-ca4/server-cert.pem

# Path for the TLS private key file
tlsKeyFilePath=/root/my-ca4/serverKey-pk8.pem

# Path for the trusted TLS certificate file.
# This cert is used to verify that any certs presented by connecting clients
# are signed by a certificate authority. If this verification
# fails, then the certs are untrusted and the connections are dropped.
tlsTrustCertsFilePath=/root/my-ca4/ca-cert.pem

# Accept untrusted TLS certificate from client.
# If true, a client with a cert which cannot be verified with the
# 'tlsTrustCertsFilePath' cert will allowed to connect to the server,
# though the cert will not be used for client authentication.
tlsAllowInsecureConnection=false

# authentication.
tlsRequireTrustedClientCertOnConnect=false

client.conf

useTls=true

# URL for Pulsar REST API (for admin operations)
# For TLS:
webServiceUrl=https://localhost:8443/
# webServiceUrl=http://localhost:8080/

# URL for Pulsar Binary Protocol (for produce and consume operations)
# For TLS:
brokerServiceUrl=pulsar+ssl://localhost:6651/
# brokerServiceUrl=pulsar://localhost:6650/

tlsTrustCertsFilePath=/root/my-ca4/ca-cert.pem

# Enable TLS with KeyStore type configuration in broker.
useKeyStoreTls=false

websocket.conf

### --- TLS --- ###

# Deprecated CentTlsEnabled use webServicePortTls and brokerClientTlsEnabled instead
tlsEnabled=true

# Accept untrusted TLS certificate from client
tlsAllowInsecureConnection=false

# Path for the TLS certificate file
tlsCertificateFilePath=/root/my-ca4/client-cert.pem

# Path for the TLS private key file
tlsKeyFilePath=/root/my-ca4/clientKey-pk8.pem

# Path for the trusted TLS certificate file
tlsTrustCertsFilePath=/root/my-ca4/ca-cert.pem

# Specify whether Client certificates are required for TLS
# Reject the Connection if the Client Certificate is not trusted.
tlsRequireTrustedClientCertOnConnect=false

[xiasf]

❤ Thank you very much, Already solved, It’s a configuration problem, but also because the documentation is not clear enough

These points solved my problem:

1. bin/pulsar standalone will not be used websocket.conf

2. ws client Trusting CA root certificates and set not to verify the host name

ws = websocket.WebSocket(sslopt={'ca_certs': '/root/my-ca6/certs/ca.cert.pem', "check_hostname": False})

1. Note the difference between server certificate and client certificate issuance: -extensions server_cert or -extensions usr_cert `
2. tlsRequireTrustedClientCertOnConnect=true If enabled, Configuration required:

client.conf

authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
authParams={"tlsCertFile":"/root/my-ca6/admin.cert.pem","tlsKeyFile":"/root/my-ca6/admin.key-pk8.pem"}

standalone.conf

brokerClientTlsEnabled=true
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
brokerClientAuthenticationParameters={"tlsCertFile":"/root/my-ca6/admin.cert.pem","tlsKeyFile":"/root/my-ca6/admin.key-pk8.pem"}
brokerClientTrustCertsFilePath=/root/my-ca6/certs/ca.cert.pem

Related documents:

https://pulsar.apache.org/docs/en/next/security-tls-authentication/

https://github.com/apache/pulsar/issues/5598

https://github.com/apache/pulsar/issues/12313

[xiasf]

@Shoothzj Yes, and the brokerClientAuthenticationParameters configuration is not mentioned when tlsEnabled=true is enabled.

I also found a problem: configuring authenticationEnabled=true as documented caused the startup to fail because useTls=true was also required in functions_worker.yml, which was not mentioned in the documentation. I tried several times without success, until I saw https://github.com/apache/pulsar/issues/7287 and solved it.

Most of the problems I encountered were solved by google and issues. The documentation is not perfect and if you can’t follow it, it’s not very newbie friendly, especially for people with a non-java technology stack. Also, where can I develop my own pulsar client in other languages?