Predict request neither returns nor there is a timeout

/kind bug

What steps did you take and what happened: I am implementing serving for CRNN model. This model uses TensorFlow predict method. Prediction works fine without kfserving. When I implement kfserving.KFmodel, predict request neither returns nor there is a time out. The payload is less than 1mb

{
"roi":'base64(bytearray(image.png))'
}

I also have a preprocess method implemented, which does the base 64 decode, converts the image byte array to tensor, resizing, grayscale and padding.

What did you expect to happen: Any suggestion to resolve this issue is appreciated. Is there a reason why there is no time out for the request?

Anything else you would like to add:

Environment:

• Istio Version: NA
• Knative Version: NA
• KFServing Version:0.4.0 python sdk
• Kubeflow version:NA
• Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]: NA
• Minikube version:NA
• Kubernetes version: (use kubectl version):NA
• OS (e.g. from /etc/os-release): os x

[yuzisun]

@ShilpaGopal I wondering if multi-processing is causing some weirdness in your server, but if you deploy it on KFServing the timeout is enforced on sidecar regardless of how model server behaves. Regarding using transformer and tfserving, can you mock out the call to tfserving by overwriting predict on transformer? in that way you can test transformer separately.

[k8s-ci-robot]

@yuzisun: Closing this issue.

In response to this:

thanks @ShilpaGopal! let us know your findings.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.