controller is binding to a priviliged port 443

/kind bug

What steps did you take and what happened: I install Kfserving on openshift 4.3 with the steps in guides, and in the end, found that kfserving pod cannot be up.

The logs as below:

oc logs kfserving-controller-manager-0 -c manager
{"level":"info","ts":1598011267.176977,"logger":"entrypoint","msg":"Setting up client for manager"}
{"level":"info","ts":1598011267.177478,"logger":"entrypoint","msg":"Setting up manager"}
{"level":"info","ts":1598011270.674518,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"127.0.0.1:8080"}
{"level":"info","ts":1598011270.674906,"logger":"entrypoint","msg":"Registering Components."}
{"level":"info","ts":1598011270.6749208,"logger":"entrypoint","msg":"Setting up scheme"}
{"level":"info","ts":1598011270.6750262,"logger":"entrypoint","msg":"Setting up Knative scheme"}
{"level":"info","ts":1598011270.6751091,"logger":"entrypoint","msg":"Setting up Istio schemes"}
{"level":"info","ts":1598011270.6751914,"logger":"entrypoint","msg":"Setting up controller"}
{"level":"info","ts":1598011270.7700293,"logger":"entrypoint","msg":"setting up webhook server"}
{"level":"info","ts":1598011270.7700856,"logger":"entrypoint","msg":"registering webhooks to the webhook server"}
{"level":"info","ts":1598011270.7701104,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-pods"}
{"level":"info","ts":1598011270.7701356,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-inferenceservices"}
{"level":"info","ts":1598011270.7701447,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-inferenceservices"}
{"level":"info","ts":1598011270.7701523,"logger":"entrypoint","msg":"Starting the Cmd."}
{"level":"info","ts":1598011270.7705138,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"kfserving-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1598011270.770591,"logger":"controller-runtime.webhook.webhooks","msg":"starting webhook server"}
{"level":"info","ts":1598011270.7705498,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1598011270.7714868,"logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}
{"level":"info","ts":1598011270.7718337,"logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}
{"level":"error","ts":1598011270.771932,"logger":"entrypoint","msg":"unable to run the manager","error":"listen tcp :443: bind: permission denied","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\nmain.main\n\t/go/src/github.com/kubeflow/kfserving/cmd/manager/main.go:100\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203"}

I can workaround it with adding anyuid scc to kfserving namespaces.

oc adm policy add-scc-to-group anyuid system:serviceaccounts:kfserving-system

What did you expect to happen: installs kfserving successfully w/o extra scc. Maybe use other unprivileged port, like 9443.

Anything else you would like to add:

Environment:

• Istio Version: 1.6.8
• Knative Version: 0.13.3
• KFServing Version: 0.4.0
• Kubeflow version:
• Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
• Minikube version:
• Kubernetes version: (use kubectl version):

oc version
Client Version: 4.3.10-202003300415-6a90d0a
Server Version: 4.3.23
Kubernetes Version: v1.16.2+18cfcc9

• OS (e.g. from /etc/os-release):

[yuzisun]

@robin2008 thanks for the issue! I think there are a few places need to fix https://github.com/kubeflow/kfserving/blob/465a4e6a7499b9be05de64f7fd2c8f4e98cbd0f3/config/webhook/service.yaml#L9 https://github.com/kubeflow/kfserving/blob/465a4e6a7499b9be05de64f7fd2c8f4e98cbd0f3/config/webhook/service.yaml#L9

When constructing the controller manager we need to set the webhook port like 9443 https://github.com/kubeflow/kfserving/blob/master/cmd/manager/main.go#L65

mgr, err := manager.New(cfg, manager.Options{MetricsBindAddress: metricsAddr, Port: 9443})

[yuzisun]

And in my test, released versions’ port cannot be changed because we need rebuild manager but released version just use images from “gcr.io/kfserving/kfserving-controller:$TAG”. So I can only change the target port from /config/webhook/service.yaml and main.go and then make deploy-dev to make it work(after code merged make deploy should work too) @yuzisun Do you think we need change /install/${TAG}/kfserving.yaml and rebuild gcr.io/kfserving/kfserving-controller:$TAG for all release versions?

No, releases are immutable. Once this is fixed it will be available for the upcoming 0.5 release.