Unable to use KFServing on Kubeflow

/kind bug

What steps did you take and what happened: I was trying to use KFServing for Model Prediction as described here.

MODEL_NAME=flowers-sample
INPUT_PATH=@./input.json
INGRESS_GATEWAY=istio-ingressgateway
CLUSTER_IP=$(kubectl -n istio-system get service $INGRESS_GATEWAY -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
SERVICE_HOSTNAME=$(kubectl get inferenceservice ${MODEL_NAME} -o jsonpath='{.status.url}' | cut -d "/" -f 3)

I realize that CLUSTER_IP & SERVICE_HOSTNAME are blank. On checking the JSON file, it can be figured out that the jsonpath specified for CLUSTER_IP has been modified to spec.clusterIP, whereas the SERVICE_HOSTNAME continues to remain blank leading to failure of the steps that follow.

What did you expect to happen: The commands to execute successfully & output a model prediction.

Anything else you would like to add: I tried to hardcode the SERVICE_HOSTNAME to flowers-sample.default.example.com but that led to the connection time out error.

Environment:

• Istio Version:
• Knative Version:
• KFServing Version:
• Kubeflow version: v1.0.2
• Kfdef: k8s_istio
• Minikube version:
• Kubernetes version: (use kubectl version): Client Version: 1.18 Server Version: 1.15
• OS (e.g. from /etc/os-release): Ubuntu 18.04

[yuzisun]

@Manvi07 Looks like it is not able to create the istio virtual service due to some networking issue on your cluster.

@yuzisun any suggestions on how should I go about the problem?

@Manvi07 this smells like firewall issue on your cluster, in this case knative is calling istio webhook istio-galley in istio-system namespace to create the virtual service but timed out. It is best to check your firewall setup.

[yuzisun]

@Manvi07 if you remove the annotation "sidecar.istio.io/inject":"true" on both autoscaler and activator deployment, the pod should then come up.