Running serverless function yolov3 fails, installing automatic annotation

[il-nietos]

My actions before raising this issue

• [x ] Read/searched the docs
• [ x] Searched past issues

I followed the steps in https://opencv.github.io/cvat/docs/administration/advanced/installation_automatic_annotation/,
and everything works (the nucleo dahsboard is up and available through localhost, the openvino-dextr is running), until attempting to run the yolo-v3-tf with the following command:

nuctl deploy --project-name cvat \
  --path serverless/openvino/omz/public/yolo-v3-tf/nuclio \
  --volume `pwd`/serverless/common:/opt/nuclio/common \
  --platform local

Expected Behaviour

Function yolo-v3-tf created succesfully and running on the nucleo dashboard.

Current Behaviour

Running the yolo-v3-tf command returns the following error:

executor failed running [/bin/bash -xo pipefail -c /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo]: exit code: 1
(full error log provided at the end).

Your Environment

• Git hash commit (git log -1):
• Docker version docker version (e.g. Docker 17.0.05): Docker v4.16.2
• Are you using Docker Swarm or Kubernetes? no
• Operating System and version (e.g. Linux, Windows, MacOS): MacOS 13.0
• Code example or link to GitHub repo or gist to reproduce problem: -
• Other diagnostic information / logs:
  error log

#9 [ 5/11] RUN /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo
#9 sha256:25f8733bb21b2145e790e989f7756bc9a391b75a29219bf6173963f571c3b218
#9 0.052 + /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo
#9 3.338 ========= Converting yolo-v3-tf to IR (FP32)
#9 3.338 Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=tf --data_type=FP32 --output_dir=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/FP32 --model_name=yolo-v3-tf '--input_shape=[1,416,416,3]' --input=input_1 '--scale_values=input_1[255]' --reverse_input_channels --transformations_config=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/yolo-v3.json --input_model=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/yolo-v3.pb
#9 3.338 
#9 5.482 2023-03-07 13:53:41.794320: F tensorflow/core/lib/monitoring/sampler.cc:42] Check failed: bucket_limits_[i] > bucket_limits_[i - 1] (inf vs. inf)
#9 5.482 qemu: uncaught target signal 6 (Aborted) - core dumped
#9 5.497 FAILED:
#9 5.497 yolo-v3-tf
#9 ERROR: executor failed running [/bin/bash -xo pipefail -c /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo]: exit code: 1
------
 > [ 5/11] RUN /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo:
------
executor failed running [/bin/bash -xo pipefail -c /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo]: exit code: 1

    /nuclio/pkg/cmdrunner/shellrunner.go:96
Failed to build
    /nuclio/pkg/dockerclient/shell.go:117
Failed to build docker image
    .../pkg/containerimagebuilderpusher/docker.go:54
Failed to build processor image
    /nuclio/pkg/processor/build/builder.go:263
Failed to deploy function
    ...//nuclio/pkg/platform/abstract/platform.go:197
(base) i.nietosvaara@ilonas-MBP cvat % xcode-select --install 

[SpecLad]

This looks like tensorflow/tensorflow#52845.

[SpecLad]

Note that even though the bug report is in the TensorFlow project, the underlying problem is with Docker Desktop. Specifically, it uses an old version of QEMU, which does not emulate AVX instructions. Once Docker Desktop updates to QEMU 7.2 or newer, this issue should resolve itself.

[nmanovic]

@il-nietos , it should be resolved. I will close the issue.

[il-nietos]

I am still getting the same error (after updating Docker, pulling any changes to the repo and rebuilding the docker containers). Am I missing something here?

Output:

#8 [ 5/11] RUN /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo
#8 sha256:25f8733bb21b2145e790e989f7756bc9a391b75a29219bf6173963f571c3b218
#8 0.077 + /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo
#8 3.593 ========= Converting yolo-v3-tf to IR (FP32)
#8 3.593 Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=tf --data_type=FP32 --output_dir=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/FP32 --model_name=yolo-v3-tf '--input_shape=[1,416,416,3]' --input=input_1 '--scale_values=input_1[255]' --reverse_input_channels --transformations_config=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/yolo-v3.json --input_model=/opt/nuclio/open_model_zoo/public/yolo-v3-tf/yolo-v3.pb
#8 3.593 
#8 6.119 2023-04-26 08:19:12.667703: F tensorflow/core/lib/monitoring/sampler.cc:42] Check failed: bucket_limits_[i] > bucket_limits_[i - 1] (inf vs. inf)
#8 6.120 qemu: uncaught target signal 6 (Aborted) - core dumped
#8 6.136 FAILED:
#8 6.136 yolo-v3-tf
#8 ERROR: executor failed running [/bin/bash -xo pipefail -c /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo]: exit code: 1
------
 > [ 5/11] RUN /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo:
------
executor failed running [/bin/bash -xo pipefail -c /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/converter.py --name yolo-v3-tf --precisions FP32 -d /opt/nuclio/open_model_zoo -o /opt/nuclio/open_model_zoo]: exit code: 1

    /nuclio/pkg/cmdrunner/shellrunner.go:96
Failed to build
    /nuclio/pkg/dockerclient/shell.go:117
Failed to build docker image
    .../pkg/containerimagebuilderpusher/docker.go:54
Failed to build processor image
    /nuclio/pkg/processor/build/builder.go:263
Failed to deploy function
    ...//nuclio/pkg/platform/abstract/platform.go:197