question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can request the Triton Inference Server by using the 'request package' in python rather than 'client library'?

See original GitHub issue

Can request the Triton Inference Server by using the ‘request package’ in python rather than ‘client library’?

like:

import request
res = requests.post(url, params=params)
print(res.text)

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
deadeyegoodwincommented, Oct 14, 2020

Yes, you can format and send your own HTTP requests, you just need to follow the HTTP/REST protocol https://github.com/triton-inference-server/server/blob/master/docs/inference_protocols.md.

Note that you will likely have lower performance than if you use the client library unless you take advantage of the binary data extension.

0reactions
thinkindscommented, May 12, 2021

This is already documented in the protocol.

The $request_input JSON describes an input to the model. If the input is batched, the shape and data must represent the full shape and contents of the entire batch.

Source: https://github.com/kubeflow/kfserving/blob/master/docs/predict-api/v2/required_api.md#request-input

I must admit I’ve read this, but didn’t understand it. Maybe an example would help a lot.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Triton Inference Server: The Basics and a Quick Tutorial
Learn about the NVIDIA Triton Inference Server, its key features, models and model repositories, client libraries, and get started with a quick tutorial....
Read more >
pyotritonclient - PyPI
A Pyodide python http client library and utilities for communicating with Triton Inference Server (based on tritonclient from NVIDIA).
Read more >
Deploying Diverse AI Model Categories from Public Model ...
The figure shows a generic workflow of a client application interaction with the Triton Inference Server Figure 3. Workflow of client ...
Read more >
Serving TensorRT Models with NVIDIA Triton Inference Server
The speed of a single client-server inference request depends on the ... More information about Triton server's performance can be found in ...
Read more >
Serving Predictions with NVIDIA Triton | Vertex AI
This page describes how to serve prediction requests with NVIDIA Triton inference server by using Vertex AI Prediction. NVIDIA Triton inference server ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found