Couldn't create nuclio functions

Hi there,

I am new to cvat and am trying to follow instructions at https://github.com/opencv/cvat/blob/develop/cvat/apps/documentation/installation.md#semi-automatic-and-automatic-annotation to setup automatic annotation with nuclio.

The project in nuclio was created without any problem but I got errors creating the functions. Here is what I see:

(base) seki@xubuntu-20:~/src/cvat$ nuctl deploy --project-name cvat     --path serverless/openvino/dextr/nuclio     --volume `pwd`/serverless/openvino/common:/opt/nuclio/common     --platform local
20.08.23 16:05:46.718                     nuctl (I) Deploying function {"name": ""}
20.08.23 16:05:46.718                     nuctl (I) Building {"versionInfo": "Label: 1.4.17, Git commit: 278c7a4fb23a93973d16d87dbaaad87823e9644f, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""}
20.08.23 16:05:46.960                     nuctl (I) Cleaning up before deployment
20.08.23 16:05:47.050                     nuctl (I) Staging files and preparing base images
20.08.23 16:05:47.051                     nuctl (I) Building processor image {"imageName": "cvat/openvino.dextr:latest"}
20.08.23 16:05:47.051     nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.4.17-amd64"}
20.08.23 16:05:48.333     nuctl.platform.docker (W) Docker command outputted to stderr - this may result in errors {"workingDir": "/tmp/nuclio-build-479579805/staging", "cmd": "docker build --network host --force-rm -t nuclio-onbuild-bt1egf28hvo5093nhgs0 -f /tmp/nuclio-build-479579805/staging/Dockerfile.onbuild  --build-arg NUCLIO_BUILD_LOCAL_HANDLER_DIR=handler --build-arg NUCLIO_LABEL=1.4.17 --build-arg NUCLIO_ARCH=amd64  .", "stderr": "unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n"}
20.08.23 16:05:48.334                     nuctl (W) Create function failed, setting function status {"err": "Failed to build processor image", "errVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\n\nCall stack:\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\nFailed to build onbuild image\n    .../pkg/containerimagebuilderpusher/docker.go:233\nFailed to copy objects from onbuild\n    .../pkg/containerimagebuilderpusher/docker.go:194\nFailed to build image artifacts\n    .../pkg/containerimagebuilderpusher/docker.go:51\nFailed to build processor image\n    /nuclio/pkg/processor/build/builder.go:250\nFailed to build processor image", "errCauses": [{"error": "Failed to build image artifacts", "errorVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\n\nCall stack:\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\nFailed to build onbuild image\n    .../pkg/containerimagebuilderpusher/docker.go:233\nFailed to copy objects from onbuild\n    .../pkg/containerimagebuilderpusher/docker.go:194\nFailed to build image artifacts\n    .../pkg/containerimagebuilderpusher/docker.go:51\nFailed to build image artifacts", "errorCauses": [{"error": "Failed to copy objects from onbuild", "errorVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\n\nCall stack:\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\nFailed to build onbuild image\n    .../pkg/containerimagebuilderpusher/docker.go:233\nFailed to copy objects from onbuild\n    .../pkg/containerimagebuilderpusher/docker.go:194\nFailed to copy objects from onbuild", "errorCauses": [{"error": "Failed to build onbuild image", "errorVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\n\nCall stack:\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\nFailed to build onbuild image\n    .../pkg/containerimagebuilderpusher/docker.go:233\nFailed to build onbuild image", "errorCauses": [{"error": "stdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n", "errorVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\n\nCall stack:\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n\n    /nuclio/pkg/cmdrunner/cmdrunner.go:124\nstdout:\n\nstderr:\nunable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory\n", "errorCauses": [{"error": "exit status 1"}]}]}]}]}]}

Error - exit status 1
    /nuclio/pkg/cmdrunner/cmdrunner.go:124

Call stack:
stdout:

stderr:
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/nuclio-build-479579805: no such file or directory

    /nuclio/pkg/cmdrunner/cmdrunner.go:124
Failed to build onbuild image
    .../pkg/containerimagebuilderpusher/docker.go:233
Failed to copy objects from onbuild
    .../pkg/containerimagebuilderpusher/docker.go:194
Failed to build image artifacts
    .../pkg/containerimagebuilderpusher/docker.go:51
Failed to build processor image
    /nuclio/pkg/processor/build/builder.go:250

I assume we are missing files under the /tmp/nuclio-build-xxx folder but I don’t see anything like that in the /tmp folder and I made sure I do have the permission to create a new folder within /tmp.

Has anyone seen this problem before? Any advice is greatly appreciated.

Thanks a lot

[kiteklan]

@kiteklan , could you please post your error message please? Let’s try to investigate the issue together.

I had workaround the problem but seems like cvat serverless+nuclio has an issue with all kernel versions running on hyper-v , I had solved it by installing on a proxmox cpu type “host” so that the virtual machine will use all the cpu assets. I was very hopefull from hyper-v since I was going to use cvat on a double core 24processor with 32gb ram , insisted trying. I am preparing all the errors now , also shared on https://github.com/openvinotoolkit/cvat/issues/3109

[nmanovic]

@kiteklan , could you please post your error message please? Let’s try to investigate the issue together.