Triton inference

Hello,

I’m trying to use Triton inf server (tritonserver:21.04-py3) with trt plan from your project. Model - retinaface

Your TRT backend works perfect. docker run -p 18081:18080 -d --gpus 0 -e LOG_LEVEL=INFO -e PYTHONUNBUFFERED=0 -e NUM_WORKERS=1 -e INFERENCE_BACKEND=trt -e FORCE_FP16=True -e DET_NAME=retinaface_r50_v1 -e DET_THRESH=0.6 -e REC_NAME=glint360k_r100FC_1.0 -e REC_IGNORE=False -e REC_BATCH_SIZE=64 -e GA_NAME=genderage_v1 -e GA_IGNORE=False -e KEEP_ALL=True -e MAX_SIZE=1024,780 -e DEF_RETURN_FACE_DATA=True -e DEF_EXTRACT_EMBEDDING=True -e DEF_EXTRACT_GA=True -e DEF_API_VER='1'

But if I try to transfer your processing from https://github.com/SthPhoenix/InsightFace-REST/blob/master/src/api_trt/modules/model_zoo/detectors/retinaface.py#L268 like postprocessing Triton’s result.

dw and dh just empty list for stride16

Would you recommend something?

Triton conf triton_model_config.zip

Jupyter notebook triton_test.zip

[SthPhoenix]

Thanks, I will wait for such updates if it intersects with your interests.

close

I’m definitely interested in such updates, though it might take a while )

[gulldan]

1. I have noticed that Triton for some reason states that model is fp32, but if you compare actual performance of fp32 and fp16 models with Triton perf client difference is obvious.
2. I haven’t tested it yet since it requires lot of changes to source code, but since now Triton supports python backend and DALI preprocessing it’s really worth a try.
3. It’ll be not trivial to put all face detection/recognition pipeline into Triton, but it’s very promising, especially considering that some parts of pipeline could be replaced with c++.

Thanks, I will wait for such updates if it intersects with your interests.

close