Object Detection worker dies

I am deploying a FasterRCNN model with 2 workers, and when I POST an image to the inference API as follows:

curl -X POST http://localhost:8080/predictions/faster-rcnn -T image.png

The server responds with this error:

{
  "code": 507,
  "type": "InternalServerException",
  "message": "Worker died."
}

I first thought that this issue was related to memory usage, but I am increasing the Java VM memory by setting this options to the ts config file: vmargs=-Xmx12g and I am still getting this 507 error.

The server logs are:

2020-04-29 06:57:02,007 [INFO ] epollEventLoopGroup-4-2 org.pytorch.serve.wlm.WorkerThread - 9001 Worker disconnected. WORKER_MODEL_LOADED
2020-04-29 06:57:02,057 [DEBUG] W-9001-faster-rcnn_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133)
        at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432)
        at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Can someone help me?

Also I want to thank Pytorch team for this awesome work.

Environment details:

• Windows using docker image pytorch/torchserve:0.1-cpu

Log file: ts_log.log

[Guillem96]

It Is not trained with coco I am using a custom dataset. Maybe the issue is related to Windows an Docker for windows.

I will try to run the code on a GCP compute engine with ubuntu and see what happens

[Guillem96]

@harshbafna

Upload as txt because GitHub is not allowing .py

Model handler: handler.txt Model file: model.txt

Model archive command:

$ torch-model-archiver \
    --model-name faster_rcnn \
    --serialized-file faster_rcnn.pt \
    --handler handler.py \
    --model-file model.py

TS config file:

# Server properties
inference_address=http://0.0.0.0:8080
management_address=http://0.0.0.0:8081
job_queue_size=100

# Model archives location
load_models=all
default_workers_per_model=1
model_store=/image-asset-server/model-store

# Java VM arguments
vmargs=-Xmx12g