Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unstable ClearML Client

See original GitHub issue

Dear ClearML Community,

When trying to create new task, sometimes it can be created directly but more often get exception like this:

Retrying (Retry(total=239, connect=239, read=240, redirect=240, status=240)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f6dff7eaf50>, 'Connection to <HOST_IP> timed out. (connect timeout=3.0)')': /auth.login

Usually, after several above exceptions, then the client able to create the new task.

Is this expected? How can we smooth the connection between ClearML servers and client servers?

Issue Analytics

State:
Created 2 years ago
Comments:5 (3 by maintainers)

Top GitHub Comments

1reaction

bmartinncommented, May 12, 2021

It is definitely a hack, but it will increase the timeout for the first authentication call:

from clearml.backend_api import Session
Session._session_initial_timeout = (15., 30.)

0reactions

hadyan-tvlkcommented, May 12, 2021

Noted @bmartinn,

Is there any way to increase the timeout ya from the client side? i assume we need more time to connect with server

Top Results From Across the Web

Stop Using Kubernetes for ML-Ops - ClearML

When things are simple, everything works. But since ML / DL code is unstable and packaging sometimes takes time, we will need to...

clearml-session - PyPI

clearml -session is a utility for launching detachable remote interactive sessions (MacOS, Windows, Linux). tl;dr. CLI to launch remote sessions for ...

Lei Wang on LinkedIn: #innovation #artificialintelligence ...

Here is a new blog post describing a lightweight implementation of Deep Lake, leveraging #clearml. What do you think is going to happen...

Answers - Springer Link

(ii) it can draw on expertise accumulated in solving customers' problems worldwide. ... (iii) But demand will only clear Ml at price P2...

Introducing MLOps - ITlligenze.com

When faced with unseen data, be it a photo or a customer, the ML model uses what ... Can be unstable—small changes in...