Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

livy ssl issue

See original GitHub issue

Apache Airflow version: 2.0.1

Kubernetes version (if you are using kubernetes) (use kubectl version): 1.17.6

OS (e.g. from /etc/os-release): oracle linux 7.9
Kernel (e.g. uname -a): Linux airflow-worker-1 4.14.35-2047.502.4.el7uek.x86_64 #2 SMP Mon Apr 5 13:21:44 PDT 2021 x86_64 GNU/Linux

What happened:

Got the following trying to user the livy operator. I have added the certs to the image and ran update-ca-certs. I can use the curl command from the worker pod and the response is as expected. But pyopenssl is not able to verify the cert. I’ve changed the actual server name.

[2021-05-12 20:40:04,826] {taskinstance.py:1455} ERROR - HTTPSConnectionPool(host='host.example.com', port=8998): Max retries exceeded with url: /batches (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
    cnx.do_handshake()
  File "/home/airflow/.local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 1934, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/home/airflow/.local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 1671, in _raise_ssl_error
    _raise_current_error()
  File "/home/airflow/.local/lib/python3.8/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

here is trhe output from curl running in the pod.

airflow@airflow-worker-1:/opt/airflow$ curl -I https://host.example.com:8998/
HTTP/1.1 401 Unauthorized
WWW-Authenticate: MAPR-Negotiate
WWW-Authenticate: Basic realm="WebLogin"
Set-Cookie: hadoop.auth=; Path=/; Domain=.example.com; Expires=Thu, 01-Jan-1970 00:00:00 GMT; Secure; HttpOnly
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 519
Server: Jetty(9.4.22.v20191022)

What you expected to happen:

No ssl error

Issue Analytics

State:
Created 2 years ago
Comments:11 (6 by maintainers)

Top GitHub Comments

1reaction

cocampbecommented, May 13, 2021

Passing it the certs directory worked.

1reaction

potiukcommented, May 13, 2021

The operator. Look at the Livy Operator. It is based on the HttpOperator and it has ‘extra_options’ field. You can pass there the same kind of dictionary as for the HttpOperator/Hook. One of the values might be ‘verify’. verify might be False/True or it also can be a pth to the root certificate used to verify the server side

Top Results From Across the Web

SSL error for Livy - Cloudera Community - 282311

we got the below error while submiting spark jobs via Livy REST API using python script - 282311.

Enable SSL support for Livy Endpoints · Issue #377 - GitHub

When trying to add an SSL'd Livy server Endpoint the following error is thrown: TypeError: unorderable types: NoneType() >= int() Works well ...

Enabling HTTPS with Apache Livy ... - AWS Documentation

Run the following script as an Amazon EMR step. This script modifies /etc/livy/conf/livy.conf to activate SSL.

[LIVY-329] Fix small bugs when enabling SSL - Cloudera Open ...

1. Livy server url is exposed as HTTP URL even Https is enabled, so we should handle this. 2. Livy server SSL keystore...

Livy Interpreter for Apache Zeppelin

Property Default Description zeppelin.livy.url http://localhost:8998 URL where livy server is running zeppelin.livy.spark.sql.maxResult 1000 Max number of Spark SQL result t... zeppelin.livy.session.create_timeout 120 Timeout in seconds for...