External IP for istio-ingressgateway in pending status

/kind bug

What steps did you take and what happened:

I have installed standalone kfserving on minikube. I am trying out InferenceService using Tensorflow with the flower sample. I can see that the services are running fine, but external IP in pending status. I am kind of new to istio and kfserving. I went through a couple of issues and troubleshooting guide. I couldn’t find a solution for this. Any document or reference is appreciated.

NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                      AGE
istio-ingressgateway   LoadBalancer   10.103.199.41   <pending>     15021:31726/TCP,80:31658/TCP,443:32372/TCP,15443:30133/TCP   15h

$ kubectl get ksvc,configuration,revision,route
NAME                                                           URL                                                           LATESTCREATED                            LATESTREADY                              READY   REASON
service.serving.knative.dev/flowers-sample-predictor-default   http://flowers-sample-predictor-default.default.example.com   flowers-sample-predictor-default-dg749   flowers-sample-predictor-default-dg749   True

NAME                                                                 LATESTCREATED                            LATESTREADY                              READY   REASON
configuration.serving.knative.dev/flowers-sample-predictor-default   flowers-sample-predictor-default-dg749   flowers-sample-predictor-default-dg749   True

NAME                                                                  CONFIG NAME                        K8S SERVICE NAME                         GENERATION   READY   REASON
revision.serving.knative.dev/flowers-sample-predictor-default-dg749   flowers-sample-predictor-default   flowers-sample-predictor-default-dg749   1            True

NAME                                                         URL                                                           READY   REASON
route.serving.knative.dev/flowers-sample-predictor-default   http://flowers-sample-predictor-default.default.example.com   True

$ kubectl get svc istio-ingressgateway -n istio-system -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"istio-ingressgateway","install.operator.istio.io/owning-resource":"installed-state","install.operator.istio.io/owning-resource-namespace":"istio-system","istio":"ingressgateway","operator.istio.io/component":"IngressGateways","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.6.7","release":"istio"},"name":"istio-ingressgateway","namespace":"istio-system"},"spec":{"ports":[{"name":"status-port","port":15021,"targetPort":15021},{"name":"http2","port":80,"targetPort":8080},{"name":"https","port":443,"targetPort":8443},{"name":"tls","port":15443,"targetPort":15443}],"selector":{"app":"istio-ingressgateway","istio":"ingressgateway"},"type":"LoadBalancer"}}
  creationTimestamp: "2020-08-08T04:41:02Z"
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: installed-state
    install.operator.istio.io/owning-resource-namespace: istio-system
    istio: ingressgateway
    operator.istio.io/component: IngressGateways
    operator.istio.io/managed: Reconcile
    operator.istio.io/version: 1.6.7
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
  resourceVersion: "2085"
  selfLink: /api/v1/namespaces/istio-system/services/istio-ingressgateway
  uid: debd85d0-d8ae-4dda-a4f4-3a64b549852b
spec:
  clusterIP: 10.103.199.41
  externalTrafficPolicy: Cluster
  ports:
  - name: status-port
    nodePort: 31726
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    nodePort: 31658
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 32372
    port: 443
    protocol: TCP
    targetPort: 8443
  - name: tls
    nodePort: 30133
    port: 15443
    protocol: TCP
    targetPort: 15443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer: {}

Anything else you would like to add: After the installation Injection details are as follows. Inference service is deployed in default namespace.

kubectl get namespace -L istio-injection
NAME                   STATUS   AGE   ISTIO-INJECTION
cert-manager           Active   15h
default                Active   16h enabled
istio-system           Active   15h   disabled
kfserving-system       Active   15h   disabled
knative-serving        Active   15h
kube-node-lease        Active   16h
kube-public            Active   16h
kube-system            Active   16h
kubernetes-dashboard   Active   15h

Environment:

• Istio Version: 1.6.7
• Knative Version: v0.16.0
• KFServing Version: v0.4.0
• Kubeflow version: No kubeflow. Standalone installation of kfserving
• Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
• Minikube version: v1.12.2
• Kubernetes version: (use kubectl version): v1.17.9
• OS (e.g. from /etc/os-release): OS X

[yuzisun]

@ShilpaGopal you can check the istio ingress gateway doc https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control. Load balancer is usually provided by cloud and for minikube the best is to use nodePort.

export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export INGRESS_HOST=$(minikube ip)

[ShilpaGopal]

@yuzisun I tried with node port, but it’s resulting in a connection refused.

curl -v -H "Host: flowers-sample.default.example.com" http://127.0.0.1:31658/v1/models/flowers-sample:predict -d $INPUT_PATH
*   Trying 127.0.0.1:31658...
* Connection failed
* connect to 127.0.0.1 port 31658 failed: Connection refused
* Failed to connect to 127.0.0.1 port 31658: Connection refused
* Closing connection 0
curl: (7) Failed to connect to 127.0.0.1 port 31658: Connection refused

I am not sure what I am missing here. Accessing via gateway results in a connection refused. But I tried port forwarding from the service. At least with that, I am able to access the service endpoint.

Kubectl port-forward service/flowers-sample-predictor-default-5b5nm-private 8000:80