question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KServe CloudEvents infinite-loop using Knative Eventing Broker

See original GitHub issue

/kind bug

What steps did you take and what happened: We’re using Knative eventing Broker and Trigger with a filter for triggering the PredictHandler handler in a custom model server using KServe python library. Currently, KServe python library sends the response CloudEvent with the same attributes from the original CloudEvent and there is no option to filter the response CloudEvent to avoid the infinite-loop of CloudEvents being sent to KServe from the Broker.

What did you expect to happen: We expect to at least change the response CloudEvent source field to KServe instance name or some env variable.

Anything else you would like to add: Line of problematic code: https://github.com/kserve/kserve/blob/464308defb7245110a3dbf255629d7f418bb9947/python/kserve/kserve/handlers/http.py#L99

Environment:

  • Istio Version: 1.11.4
  • Knative Version: 1.0
  • KFServing Version: 0.7
  • Kubeflow version: N/A
  • Kfdef: N/A
  • Minikube version: N/A
  • Kubernetes version: (use kubectl version): 1.21.2
  • OS (e.g. from /etc/os-release): Amazon Linux 2

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
markwintercommented, Dec 6, 2021
1reaction
markwintercommented, Dec 2, 2021

@shahar3012 Including the KFModel name in the source seems like a good idea. I’ll also make type source configurable with env vars. And probably an env var to merge attributes if desired (except id, timestamp, source, type)

I’ll make some changes, fix the tests and create a PR soon.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Knative Eventing overview
Knative Eventing is a collection of APIs that enable you to use an ... You can use a trigger to consume events from...
Read more >
Inference Logger - KServe Documentation Website
Create a sklearn predictor with the logger url pointing to the Knative eventing multi-tenant broker in knative-eventing namespace. Old Schema New Schema.
Read more >
Chapter 4. Knative Eventing - O'Reilly
Use the Knative Eventing Broker and Trigger Custom Resources to allow for CloudEvent attribute filtering. The recipe is as follows: Inject the default...
Read more >
How We Built an ML inference Platform with Knative - YouTube
How We Built an ML inference Platform with Knative - Dan Sun, Bloomberg LP & Animesh Singh, IBMDeploying and scaling machine learning(ML)- ...
Read more >
A step-by-step guide to Knative eventing - Opensource.com
You can define triggers to use any filter from your cloud events. The broker receives events from the trigger and sends the events...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found