question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Kubernetes Configuration Watcher failed to connect and stopped watching for changes

See original GitHub issue

We are using watcher pod to watch on secret/configmap changes. From time to time it raises connection errors and watcher pod stopped watch for changes, it didn’t try even to reconnect.

samples errors :

  • Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210)
  • java.net.ConnectException: Failed to connect to /***:443 at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:265) *java.io.EOFException: null at okio.RealBufferedSource.require(RealBufferedSource.java:65)

And always we see WatchConnectionManager : Exec Failure

Currently I monitor errors in the watcher and restart the watcher when received errors, but we must get a normal solution. Any idea how can it be fixed? or even workaround? code changes? can I catch this errors and trigger a reconnect?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:13 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
dmitrypotenkocommented, Sep 1, 2022

As far as I can see you are moving away from fabric8 client so I am going to deploy the watcher with kubernetes official client and see if issue is gone. (And I could not reproduce the issue locally 😄)

0reactions
ryanjbaxtercommented, Aug 12, 2022

Once you have a reliable way of reproducing the problem let me know and I will take a look

Read more comments on GitHub >

github_iconTop Results From Across the Web

Spring boot fails to start when config reload is enabled
It starts if you set management.endpoint.restart.enabled=true. The message tells you that it can't load a RestartEndpoint bean.
Read more >
Kubernetes API Concepts
The Kubernetes API is a resource-based (RESTful) programmatic interface provided via HTTP. It supports retrieving, creating, updating, ...
Read more >
Spring Cloud Kubernetes
The Spring Cloud Kubernetes Config project makes Kubernetes ConfigMap instances available during application bootstrapping and triggers hot reloading of beans ...
Read more >
Enable Watcher | Elasticsearch Service Documentation | Elastic
Watcher lets you take action based on changes in your data. ... you can use the console Elasticsearch settings editor for your deployment....
Read more >
Troubleshooting - Harness.io Docs
Harness error messages, causes, and solutions. ... Artifact Server, etc, because of network issues like port changes and proxy settings.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found