Ambassador crashes when it fails to retrieve server APIs
See original GitHub issueDescribe the bug The ambassador pod goes into crash-loop when it fails to connect to Kubernetes API server before eventually coming up.
+ kubewatch --namespace pavan --sync 'python3 /ambassador/post_update.py' --warmup-delay 10s secrets services
AMBASSADOR: diagd running
AMBASSADOR: waiting
PIDS: 30:ambex 31:diagd 49:kubewatch
2019/04/08 23:32:32 unable to retrieve the complete list of server APIs: custom.api.pavan/v1alpha1: the server is currently unable to handle the request
AMBASSADOR: kubewatch exited with status 1
AMBASSADOR: shutting down (1)
NAME READY STATUS RESTARTS AGE
gateway-7d9d88db4f-nhkqn 1/1 Running 4 49m
Ambassador deployment is named gateway
in our setup.
To Reproduce Deploy the ambassador and the service responding to the custom Kubernetes APIs together. If the service is not yet up when ambassador starts watching the namespace, it crashes.
Expected behavior Ambassador should come up without crashing/restarts or there should be a way to configure it to not watch a particular service.
Versions
- Ambassador: 0.53.1
- Kubernetes environment: Google Kubernetes Engine
- Version: 1.12.5
Additional context
The custom API service is designed to respond to kubectl
requests and is not configured with any ambassador related annotations.
Note that the same setup works fine with versions < 0.50+.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:7
- Comments:17 (4 by maintainers)
Top Results From Across the Web
Ambassador crashes when it fails to retrieve server APIs #1408
Describe the bug The ambassador pod goes into crash-loop when it fails to connect to Kubernetes API server before eventually coming up.
Read more >Debugging | Edge Stack
First, check the Ambassador Edge Stack Deployment with the following: kubectl get -n ambassador deployments · Check that the “desired” number of Pods...
Read more >issue in installing ambassador over kubernetes - Server Fault
The Ambassador readiness probe is failing, causing Kubernetes to restart the pod, and putting you into a crash loop.
Read more >KB2159286 - FIX: The Remote Registry service crashes when ...
Cause. This issue occurs because the SQL Server Performance Counter uses an incorrect method to call the RegQueryValueEx API to access a registry...
Read more >Failing to see how ambassador pattern enhances modularity ...
To consider my example, lets say that db crashed and restarts, thus get assigned to a different IP. db-ambassador would have to be...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
It works as a workaround for me to delete the error apiservice
But it would be good to tolerate such error in ambassador.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.