Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KServe CloudEvents infinite-loop using Knative Eventing Broker

See original GitHub issue

/kind bug

What steps did you take and what happened: We’re using Knative eventing Broker and Trigger with a filter for triggering the PredictHandler handler in a custom model server using KServe python library. Currently, KServe python library sends the response CloudEvent with the same attributes from the original CloudEvent and there is no option to filter the response CloudEvent to avoid the infinite-loop of CloudEvents being sent to KServe from the Broker.

What did you expect to happen: We expect to at least change the response CloudEvent source field to KServe instance name or some env variable.

Anything else you would like to add: Line of problematic code: https://github.com/kserve/kserve/blob/464308defb7245110a3dbf255629d7f418bb9947/python/kserve/kserve/handlers/http.py#L99

Environment:

Istio Version: 1.11.4
Knative Version: 1.0
KFServing Version: 0.7
Kubeflow version: N/A
Kfdef: N/A
Minikube version: N/A
Kubernetes version: (use kubectl version): 1.21.2
OS (e.g. from /etc/os-release): Amazon Linux 2

Issue Analytics

State:
Created 2 years ago
Comments:9 (6 by maintainers)

Top GitHub Comments

1reaction

markwintercommented, Dec 6, 2021

@shahar3012 I made a PR here https://github.com/kserve/kserve/pull/1934

1reaction

markwintercommented, Dec 2, 2021

@shahar3012 Including the KFModel name in the source seems like a good idea. I’ll also make type source configurable with env vars. And probably an env var to merge attributes if desired (except id, timestamp, source, type)

I’ll make some changes, fix the tests and create a PR soon.

Top Results From Across the Web

Knative Eventing overview

Knative Eventing is a collection of APIs that enable you to use an ... You can use a trigger to consume events from...

Inference Logger - KServe Documentation Website

Create a sklearn predictor with the logger url pointing to the Knative eventing multi-tenant broker in knative-eventing namespace. Old Schema New Schema.

Chapter 4. Knative Eventing - O'Reilly

Use the Knative Eventing Broker and Trigger Custom Resources to allow for CloudEvent attribute filtering. The recipe is as follows: Inject the default...

How We Built an ML inference Platform with Knative - YouTube

How We Built an ML inference Platform with Knative - Dan Sun, Bloomberg LP & Animesh Singh, IBMDeploying and scaling machine learning(ML)- ...

A step-by-step guide to Knative eventing - Opensource.com

You can define triggers to use any filter from your cloud events. The broker receives events from the trigger and sends the events...