Kubernetes Configuration Watcher failed to connect and stopped watching for changes
See original GitHub issueWe are using watcher pod to watch on secret/configmap changes. From time to time it raises connection errors and watcher pod stopped watch for changes, it didn’t try even to reconnect.
samples errors :
- Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210)
- java.net.ConnectException: Failed to connect to /***:443 at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:265) *java.io.EOFException: null at okio.RealBufferedSource.require(RealBufferedSource.java:65)
And always we see WatchConnectionManager : Exec Failure
Currently I monitor errors in the watcher and restart the watcher when received errors, but we must get a normal solution. Any idea how can it be fixed? or even workaround? code changes? can I catch this errors and trigger a reconnect?
Issue Analytics
- State:
- Created 3 years ago
- Comments:13 (6 by maintainers)
Top Results From Across the Web
Spring boot fails to start when config reload is enabled
It starts if you set management.endpoint.restart.enabled=true. The message tells you that it can't load a RestartEndpoint bean.
Read more >Kubernetes API Concepts
The Kubernetes API is a resource-based (RESTful) programmatic interface provided via HTTP. It supports retrieving, creating, updating, ...
Read more >Spring Cloud Kubernetes
The Spring Cloud Kubernetes Config project makes Kubernetes ConfigMap instances available during application bootstrapping and triggers hot reloading of beans ...
Read more >Enable Watcher | Elasticsearch Service Documentation | Elastic
Watcher lets you take action based on changes in your data. ... you can use the console Elasticsearch settings editor for your deployment....
Read more >Troubleshooting - Harness.io Docs
Harness error messages, causes, and solutions. ... Artifact Server, etc, because of network issues like port changes and proxy settings.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
As far as I can see you are moving away from fabric8 client so I am going to deploy the watcher with kubernetes official client and see if issue is gone. (And I could not reproduce the issue locally 😄)
Once you have a reliable way of reproducing the problem let me know and I will take a look