Dockerfile generation fails with `Cluster not found for context 'k8s.KubeConfigModels.Context' in kubeconfig`

If I generate a new operator using the template, I get the following build failure:

Generating Dockerfile
Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> k8s.Exceptions.KubeConfigException: Cluster not found for context `k8s.KubeConfigModels.Context` in kubeconfig
   at k8s.KubernetesClientConfiguration.SetClusterDetails(K8SConfiguration k8SConfig, Context activeContext)
   at k8s.KubernetesClientConfiguration.InitializeContext(K8SConfiguration k8SConfig, String currentContext)
   at k8s.KubernetesClientConfiguration.GetKubernetesClientConfiguration(String currentContext, String masterUrl, K8SConfiguration k8SConfig)
   at k8s.KubernetesClientConfiguration.BuildConfigFromConfigObject(K8SConfiguration k8SConfig, String currentContext, String masterUrl)
   at k8s.KubernetesClientConfiguration.BuildDefaultConfig()
   at DotnetKubernetesClient.KubernetesClient..ctor()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ServiceProvider.GetService(Type serviceType)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.System.IServiceProvider.GetService(Type serviceType)
   at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.FindMatchedConstructor[TModel](ConstructorInfo[] constructors, IServiceProvider services, Boolean throwIfNoParameterTypeRegistered)
   at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.ApplyImpl[TModel](ConventionContext context)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.Apply(ConventionContext context)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Builder.McMaster.Extensions.CommandLineUtils.Conventions.IConventionBuilder.AddConvention(IConvention convention)
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseConstructorInjection(IConventionBuilder builder, IServiceProvider additionalServices)
   at KubeOps.Operator.HostExtensions.RunOperatorAsync(IHost host, String[] args)
   at CloudFormationStackOperator.Program.Main(String[] args) in <redacted>
   at CloudFormationStackOperator.Program.<Main>(String[] args)

[buehler]

I see… I don’t get the point of your suggestion clearly, do you mind explaining which executables you mean? I use the kubernetes client quite heavily in the operator sdk, so there is no replacing it. I had built a software once that used kubectl, but this is error prone as hell when using it over sub-processes.

When we have a look at the method that builds the kube config out of “default”: https://github.com/kubernetes-client/csharp/blob/master/src/KubernetesClient/KubernetesClientConfiguration.ConfigFile.cs#L47-L71

Is any of this “if” 's true on your machine, or can you think of any reason why it does use the wrong config?

The steps are:

1. Check if KUBECONFIG environment var is set, if yes, use this file (or multiple files) as config
2. Check if the <user home>/.kube/config file exists, if yes, use this file
3. Check if the running app is “in - cluster” (meaning is KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT set, and does the service account token as well as the root ca exist on the machine), if yes, build the kube config inside the cluster
4. use a default location of http://localhost:8080

I’m asking this, because - despite the usage of docker and stuff - I’ve got a classical “works on my machine” problem. I cannot reproduce the error.

BUT, what you could do: as soon as you added the operator stuff with the AddKubernetesOperator, you may remove the transient binding for IKubernetesClient and add it with your specific config:

services.AddKubernetesOperator();
services.RemoveAll(typeof(IKubernetesClient));
// use your specific config here
services.AddTransient(_ => new KubernetesClient(KubernetesClientConfiguration.BuildConfigFromConfigFile()));

[buehler]

All right. This issue is even a non-issue for the kubernetes client that I wrote. This logic is provided by the base kubernetes client over at https://github.com/kubernetes-client/csharp. I guess, this is expected behaviour.

If a cluster is not present or cannot be connected, the logic should fail.