problem with checkov installation on ubuntu azure virtual machine scale set

We have setup an virtual machine scale set on Azure, running with Ubuntu 20.04-lts, to be used by the self hosted pipeline agent. cat /etc/os-release

VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Python Version is 3.8.10

We have installed Checkov with pip3 install checkov and the last line of output shows:

Successfully installed ... checkov-2.0.1092 ...

Within the log we see one error and some warning as shown below:

ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
Installing collected packages: decorator, ply, jsonpath-ng, policyuniverse, websocket-client, docker, pyyaml, detect-secrets, click, update-checker, contextlib2, schema, python-dateutil, jmespath, botocore, s3transfer, boto3, click-option-group, soupsieve, beautifulsoup4, policy-sentry, cached-property, importlib-metadata, markdown, cloudsplaining, typing-extensions, tqdm, configargparse, semantic-version, pyparsing, packaging, dockerfile-parse, cachetools, charset-normalizer, aiomultiprocess, pycparser, cffi, pycares, aiodns, dpath, tabulate, smmap, gitdb, GitPython, deep-merge, lark, bc-python-hcl2, argcomplete, termcolor, wcwidth, prettytable, types-toml, setuptools, packageurl-python, toml, types-setuptools, cyclonedx-python-lib, frozenlist, aiosignal, async-timeout, multidict, yarl, aiohttp, regex, pycep-parser, networkx, junit-xml, checkov
WARNING: The script jsonpath_ng is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script wsdump is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The scripts detect-secrets and detect-secrets-hook are installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script policy_sentry is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script markdown_py is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script cloudsplaining is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script tqdm is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script normalizer is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script tabulate is installed in '/home/AzDevOps/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed GitPython-3.1.27 aiodns-3.0.0 aiohttp-3.8.1 aiomultiprocess-0.9.0 aiosignal-1.2.0 argcomplete-2.0.0 async-timeout-4.0.2 bc-python-hcl2-0.3.39 beautifulsoup4-4.11.1 boto3-1.22.2 botocore-1.25.2 cached-property-1.5.2 cachetools-5.0.0 cffi-1.15.0 charset-normalizer-2.0.12 checkov-2.0.1092 click-8.1.2 click-option-group-0.5.3 cloudsplaining-0.5.0 configargparse-1.5.3 contextlib2-21.6.0 cyclonedx-python-lib-0.12.3 decorator-5.1.1 deep-merge-0.0.4 detect-secrets-1.2.0 docker-5.0.3 dockerfile-parse-1.2.0 dpath-1.5.0 frozenlist-1.3.0 gitdb-4.0.9 importlib-metadata-4.11.3 jmespath-1.0.0 jsonpath-ng-1.5.3 junit-xml-1.9 lark-1.1.2 markdown-3.3.6 multidict-6.0.2 networkx-2.8 packageurl-python-0.9.9 packaging-21.3 ply-3.11 policy-sentry-0.12.3 policyuniverse-1.5.0.20220426 prettytable-3.2.0 pycares-4.1.2 pycep-parser-0.3.4 pycparser-2.21 pyparsing-3.0.8 python-dateutil-2.8.2 pyyaml-6.0 regex-2022.4.24 s3transfer-0.5.2 schema-0.7.5 semantic-version-2.9.0 setuptools-62.1.0 smmap-5.0.0 soupsieve-2.3.2.post1 tabulate-0.8.9 termcolor-1.1.0 toml-0.10.2 tqdm-4.64.0 types-setuptools-57.4.14 types-toml-0.10.6 typing-extensions-4.2.0 update-checker-0.18.0 wcwidth-0.2.5 websocket-client-1.3.2 yarl-1.7.2

Running the same installation procedure on the “Azure Pipeline” with Agents

• ubuntu-20.04
• ubuntu-latest

works fine. The shown error of the installation log above does also existis on the “Azure Pipeline”, but the warnings are not

ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
Installing collected packages: tabulate, junit-xml, termcolor, networkx, configargparse, setuptools, packageurl-python, types-setuptools, types-toml, toml, cyclonedx-python-lib, websocket-client, docker, policyuniverse, dpath, deep-merge, smmap, gitdb, GitPython, pyyaml, jmespath, update-checker, frozenlist, multidict, yarl, async-timeout, charset-normalizer, aiosignal, aiohttp, decorator, ply, jsonpath-ng, wcwidth, prettytable, lark, regex, typing-extensions, pycep-parser, soupsieve, beautifulsoup4, click, contextlib2, schema, policy-sentry, click-option-group, cached-property, python-dateutil, botocore, s3transfer, boto3, importlib-metadata, markdown, cloudsplaining, semantic-version, aiomultiprocess, bc-python-hcl2, dockerfile-parse, cachetools, pycparser, cffi, pycares, aiodns, tqdm, detect-secrets, checkov
Successfully installed GitPython-3.1.27 aiodns-3.0.0 aiohttp-3.8.1 aiomultiprocess-0.9.0 aiosignal-1.2.0 async-timeout-4.0.2 bc-python-hcl2-0.3.39 beautifulsoup4-4.11.1 boto3-1.22.2 botocore-1.25.2 cached-property-1.5.2 cachetools-5.0.0 cffi-1.15.0 charset-normalizer-2.0.12 checkov-2.0.1092 click-8.1.2 click-option-group-0.5.3 cloudsplaining-0.5.0 configargparse-1.5.3 contextlib2-21.6.0 cyclonedx-python-lib-0.12.3 decorator-5.1.1 deep-merge-0.0.4 detect-secrets-1.2.0 docker-5.0.3 dockerfile-parse-1.2.0 dpath-1.5.0 frozenlist-1.3.0 gitdb-4.0.9 importlib-metadata-4.11.3 jmespath-1.0.0 jsonpath-ng-1.5.3 junit-xml-1.9 lark-1.1.2 markdown-3.3.6 multidict-6.0.2 networkx-2.8 packageurl-python-0.9.9 ply-3.11 policy-sentry-0.12.3 policyuniverse-1.5.0.20220426 prettytable-3.2.0 pycares-4.1.2 pycep-parser-0.3.4 pycparser-2.21 python-dateutil-2.8.2 pyyaml-6.0 regex-2022.4.24 s3transfer-0.5.2 schema-0.7.5 semantic-version-2.9.0 setuptools-62.1.0 smmap-5.0.0 soupsieve-2.3.2.post1 tabulate-0.8.9 termcolor-1.1.0 toml-0.10.2 tqdm-4.64.0 types-setuptools-57.4.14 types-toml-0.10.6 typing-extensions-4.2.0 update-checker-0.18.0 wcwidth-0.2.5 websocket-client-1.3.2 yarl-1.7.2

Please let us know, what we need to do, to get checkov running ov virtual machine scale set to. thanks

[gruebel]

hey @Joerg-L looks all fine to me, the warning is quite interesting, because the libs we are leveraging for some functionality have their own CLI capabilities, that’s why this warning pops up. It is probably related to AzDevOps being a special user, which is used to install checkov, if you change this to be installed by the root user or leveraging sudo then you won’t see them, because they will be globally installed for every user. And your approach with extending the PATH env is also a good solution.

[stale[bot]]

Closing issue due to inactivity. If you feel this is in error, please re-open, or reach out to the community via slack: https://slack.bridgecrew.io Thanks!