Could not get models from the server - Error: Request failed with status code 500.

My actions before raising this issue

• Read/searched the docs
• Searched past issues

Expected Behaviour

I was expecting to see a model that is currently running.

Current Behaviour

After deploying the TensorFlow model, when I go to http://localhost:8080/models I see the error message as below.

Could not get models from the server
Error: Request failed with status code 500.
"<h1>Server Error (500)</h1>".

Possible Solution

Still searching.

Your Environment

• Git hash commit (git log -1):

Author: Dmitry Kruchinin <[e-mail]>
Date:   Thu Sep 3 13:59:59 2020 +0300

    Cypress test for issue 1568. (#2106)
    
    Co-authored-by: Dmitry Kruchinin <[e-mail]>

• Docker version :

Docker version 19.03.12, build 48a66213fe

• Are you using Docker Swarm or Kubernetes? No
• Operating System and version:

Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.7 LTS
Release:	16.04
Codename:	xenial

• Kernel

4.15.0-112-generic

• Docker ps

CONTAINER ID        IMAGE                                          COMMAND                  CREATED             STATUS                    PORTS                            NAMES
844d53c73bb3        nginx:stable-alpine                            "/docker-entrypoint.…"   34 minutes ago      Up 34 minutes             0.0.0.0:8080->80/tcp             cvat_proxy
cba3a870d17b        cvat/ui                                        "/docker-entrypoint.…"   34 minutes ago      Up 34 minutes             80/tcp                           cvat_ui
d75ba5df87b0        cvat/server                                    "/usr/bin/supervisord"   34 minutes ago      Up 34 minutes             8080/tcp, 8443/tcp               cvat
3110cba77a41        redis:4.0-alpine                               "docker-entrypoint.s…"   34 minutes ago      Up 34 minutes             6379/tcp                         cvat_redis
aed39d784a81        quay.io/nuclio/dashboard:1.4.8-amd64           "sh -c ./runner.sh"      34 minutes ago      Up 34 minutes             80/tcp, 0.0.0.0:8070->8070/tcp   nuclio
6d2d0b0fcdb0        postgres:10-alpine                             "docker-entrypoint.s…"   34 minutes ago      Up 34 minutes             5432/tcp                         cvat_db
eb242020f1a8        cvat/tf.faster_rcnn_inception_v2_coco:latest   "processor"              35 minutes ago      Up 35 minutes (healthy)   0.0.0.0:42561->8080/tcp          nuclio-nuclio-tf.faster_rcnn_inception_v2_coco
ae3f4109232a        alpine:3.11                                    "/bin/sh -c '/bin/sl…"   4 hours ago         Up 4 hours                                                 nuclio-local-storage-reader
e33a707a4c6e        nuclio/processor-test:latest                   "processor"              10 hours ago        Up 10 hours (healthy)     0.0.0.0:36521->8080/tcp          nuclio-nuclio-test

• Server deployment code: https://github.com/opencv/cvat/tree/develop/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio
• Server deployment command:

./nuctl deploy --project-name cvat_test --path cvat/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio --volume `pwd`/cvat/serverless/openvino/common:/opt/nuclio/common --platform local

• Server deployment log:

20.09.04 18:47:52.471                     nuctl (I) Deploying function {"name": ""}
20.09.04 18:47:52.471                     nuctl (I) Building {"versionInfo": "Label: 1.4.17, Git commit: 278c7a4fb23a93973d16d87dbaaad87823e9644f, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""}
20.09.04 18:47:52.702                     nuctl (I) Cleaning up before deployment
20.09.04 18:47:52.749                     nuctl (I) Staging files and preparing base images
20.09.04 18:47:52.750                     nuctl (I) Building processor image {"imageName": "cvat/tf.faster_rcnn_inception_v2_coco:latest"}
20.09.04 18:47:52.750     nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.4.17-amd64"}
20.09.04 18:47:56.570     nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/uhttpc:0.0.1-amd64"}
20.09.04 18:48:00.607            nuctl.platform (I) Building docker image {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest"}
20.09.04 18:48:01.002            nuctl.platform (I) Pushing docker image into registry {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest", "registry": ""}
20.09.04 18:48:01.002            nuctl.platform (I) Docker image was successfully built and pushed into docker registry {"image": "cvat/tf.faster_rcnn_inception_v2_coco:latest"}
20.09.04 18:48:01.002                     nuctl (I) Build complete {"result": {"Image":"cvat/tf.faster_rcnn_inception_v2_coco:latest","UpdatedFunctionConfig":{"metadata":{"name":"tf.faster_rcnn_inception_v2_coco","namespace":"nuclio","labels":{"nuclio.io/project-name":"cvat_test"},"annotations":{"framework":"tensorflow","name":"Faster RCNN via Tensorflow","spec":"[\n  { \"id\": 1, \"name\": \"person\" },\n  { \"id\": 2, \"name\": \"bicycle\" },\n  { \"id\": 3, \"name\": \"car\" },\n  { \"id\": 4, \"name\": \"motorcycle\" },\n  { \"id\": 5, \"name\": \"airplane\" },\n  { \"id\": 6, \"name\": \"bus\" },\n  { \"id\": 7, \"name\": \"train\" },\n  { \"id\": 8, \"name\": \"truck\" },\n  { \"id\": 9, \"name\": \"boat\" },\n  { \"id\":10, \"name\": \"traffic_light\" },\n  { \"id\":11, \"name\": \"fire_hydrant\" },\n  { \"id\":13, \"name\": \"stop_sign\" },\n  { \"id\":14, \"name\": \"parking_meter\" },\n  { \"id\":15, \"name\": \"bench\" },\n  { \"id\":16, \"name\": \"bird\" },\n  { \"id\":17, \"name\": \"cat\" },\n  { \"id\":18, \"name\": \"dog\" },\n  { \"id\":19, \"name\": \"horse\" },\n  { \"id\":20, \"name\": \"sheep\" },\n  { \"id\":21, \"name\": \"cow\" },\n  { \"id\":22, \"name\": \"elephant\" },\n  { \"id\":23, \"name\": \"bear\" },\n  { \"id\":24, \"name\": \"zebra\" },\n  { \"id\":25, \"name\": \"giraffe\" },\n  { \"id\":27, \"name\": \"backpack\" },\n  { \"id\":28, \"name\": \"umbrella\" },\n  { \"id\":31, \"name\": \"handbag\" },\n  { \"id\":32, \"name\": \"tie\" },\n  { \"id\":33, \"name\": \"suitcase\" },\n  { \"id\":34, \"name\": \"frisbee\" },\n  { \"id\":35, \"name\": \"skis\" },\n  { \"id\":36, \"name\": \"snowboard\" },\n  { \"id\":37, \"name\": \"sports_ball\" },\n  { \"id\":38, \"name\": \"kite\" },\n  { \"id\":39, \"name\": \"baseball_bat\" },\n  { \"id\":40, \"name\": \"baseball_glove\" },\n  { \"id\":41, \"name\": \"skateboard\" },\n  { \"id\":42, \"name\": \"surfboard\" },\n  { \"id\":43, \"name\": \"tennis_racket\" },\n  { \"id\":44, \"name\": \"bottle\" },\n  { \"id\":46, \"name\": \"wine_glass\" },\n  { \"id\":47, \"name\": \"cup\" },\n  { \"id\":48, \"name\": \"fork\" },\n  { \"id\":49, \"name\": \"knife\" },\n  { \"id\":50, \"name\": \"spoon\" },\n  { \"id\":51, \"name\": \"bowl\" },\n  { \"id\":52, \"name\": \"banana\" },\n  { \"id\":53, \"name\": \"apple\" },\n  { \"id\":54, \"name\": \"sandwich\" },\n  { \"id\":55, \"name\": \"orange\" },\n  { \"id\":56, \"name\": \"broccoli\" },\n  { \"id\":57, \"name\": \"carrot\" },\n  { \"id\":58, \"name\": \"hot_dog\" },\n  { \"id\":59, \"name\": \"pizza\" },\n  { \"id\":60, \"name\": \"donut\" },\n  { \"id\":61, \"name\": \"cake\" },\n  { \"id\":62, \"name\": \"chair\" },\n  { \"id\":63, \"name\": \"couch\" },\n  { \"id\":64, \"name\": \"potted_plant\" },\n  { \"id\":65, \"name\": \"bed\" },\n  { \"id\":67, \"name\": \"dining_table\" },\n  { \"id\":70, \"name\": \"toilet\" },\n  { \"id\":72, \"name\": \"tv\" },\n  { \"id\":73, \"name\": \"laptop\" },\n  { \"id\":74, \"name\": \"mouse\" },\n  { \"id\":75, \"name\": \"remote\" },\n  { \"id\":76, \"name\": \"keyboard\" },\n  { \"id\":77, \"name\": \"cell_phone\" },\n  { \"id\":78, \"name\": \"microwave\" },\n  { \"id\":79, \"name\": \"oven\" },\n  { \"id\":80, \"name\": \"toaster\" },\n  { \"id\":81, \"name\": \"sink\" },\n  { \"id\":83, \"name\": \"refrigerator\" },\n  { \"id\":84, \"name\": \"book\" },\n  { \"id\":85, \"name\": \"clock\" },\n  { \"id\":86, \"name\": \"vase\" },\n  { \"id\":87, \"name\": \"scissors\" },\n  { \"id\":88, \"name\": \"teddy_bear\" },\n  { \"id\":89, \"name\": \"hair_drier\" },\n  { \"id\":90, \"name\": \"toothbrush\" }\n]\n","type":"detector"}},"spec":{"description":"Faster RCNN from Tensorflow Object Detection API","handler":"main:handler","runtime":"python:3.6","resources":{},"image":"cvat/tf.faster_rcnn_inception_v2_coco:latest","targetCPU":75,"triggers":{"myHttpTrigger":{"class":"","kind":"http","name":"","maxWorkers":2,"workerAvailabilityTimeoutMilliseconds":10000,"attributes":{"maxRequestBodySize":33554432}}},"volumes":[{"volume":{"name":"volume-1","hostPath":{"path":"/mnt/hdd/projects/cvat/serverless/openvino/common"}},"volumeMount":{"name":"volume-1","mountPath":"/opt/nuclio/common"}}],"build":{"image":"cvat/tf.faster_rcnn_inception_v2_coco","baseImage":"tensorflow/tensorflow:2.1.1","directives":{"postCopy":[{"kind":"RUN","value":"curl -O http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz"},{"kind":"RUN","value":"tar -xzf faster_rcnn_inception_v2_coco_2018_01_28.tar.gz \u0026\u0026 rm faster_rcnn_inception_v2_coco_2018_01_28.tar.gz"},{"kind":"RUN","value":"ln -s faster_rcnn_inception_v2_coco_2018_01_28 faster_rcnn"},{"kind":"RUN","value":"pip install pillow pyyaml"}],"preCopy":[{"kind":"RUN","value":"apt install curl"},{"kind":"WORKDIR","value":"/opt/nuclio"}]},"codeEntryType":"image"},"platform":{"attributes":{"restartPolicy":{"maximumRetryCount":3,"name":"always"}}},"readinessTimeoutSeconds":60,"eventTimeout":"30s"}}}}
20.09.04 18:48:02.165            nuctl.platform (I) Waiting for function to be ready {"timeout": 60}
20.09.04 18:48:03.501                     nuctl (I) Function deploy complete {"functionName": "tf.faster_rcnn_inception_v2_coco", "httpPort": 42561}

Please help me to find a solution. 😃

[veer5551]

Followed the same procedure on a Linux system and everything was smooth. Models are deployed correctly and I am able to use them as well.

Need to check if this is an issue of Nuclio + WSL in Windows 10 combination problem. Maybe I’ll report the issue there. If anyone here has got it working, please share the procedure/guide.

Alternatively, is it possible to deploy the models via the Nuclio UI Dashboard by loading the appropriate YAML and model handler? Tried doing it but got some function invocation error.

Thanks a lot!

[beep-love]

Yeah. It was the same issue. Mine worked with v 1.5.7.