segfault at ... error 6 in libc-2.27.so
See original GitHub issueDescription Docker container crashes with segfault.
Triton Information
I’m using Triton container nvcr.io/nvidia/tritonserver:20.09-py3
To Reproduce Start Triton server with model with following configuration:
name: 'Model'
platform: 'tensorflow_savedmodel'
input [
{
name: 'input_layer'
data_type: TYPE_UINT8
dims: [ -1, -1, -1, 3 ]
}
]
output [
{
name: 'tf_op_layer_Cast_1'
data_type: TYPE_UINT8
dims: [ -1, -1, -1, 3 ]
}
]
instance_group {
kind: KIND_GPU
count: 1
}
Run two or more tasks constantly requesting inference using Triton client with cuda shared memory. One task may succeed, but another will cause segfault.
Expected behavior All tasks works correctly.
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (4 by maintainers)
Top Results From Across the Web
Segfault Error 6 in libc-2.27.so on Import Certificate · Issue #148
Successfully generated a new private key. ... Successfully verified PIN. Successfully generated a new self signed certificate. ... trying to connect ...
Read more >Bug #1809438 “systemd-resolved segfaults” - Launchpad Bugs
I regularly (once almost every hour) get segfaults resported in systemd-resolved. The syslog is not always the same:.
Read more >Segmentation fault after installing the glibc 2.7 - Stack Overflow
Typing ls, ll, vim etc normal command, it all come with error Segmentation fault . I have searched the Internet, and found two...
Read more >986427 – segfault error 4 at libc-2.17.so - Red Hat Bugzilla
Sshd generates the exact same crash, though not all the times, i.e. it works fine when I ssh directly into the machine, but...
Read more >Debian Bug report logs - #893320 libc6: libc-2.27.so causes ...
Debian Bug report logs - #893320 libc6: libc-2.27.so causes segfaults in multiple programs. version graph. Package: libc6; Maintainer for libc6 ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@jbkyang-nvi I’ll give a try
21.06
and come back later. @CoderHam Here’s Triton Server log. Running tasks creates cuda shared memory regions and reuse them between inference requests, at the end script destroys created regions. Unfortunately I can’t provide code and models to reproduce this bug due to NDA.Glad it worked for you. If you see similar issues with our recent releases or have general suggestions on how to handle such cases better in triton, feel free to open another GitHub issue.