question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Updating python3 version on dvcorg/cml-py3:runner

See original GitHub issue

I have some projects that require python 3.8 and I tried two options below:

  1. install python3.8 inside your dvcorg/cml-py3:runner image, although I’m being unable to mount volumes on the image. I don’t know how to explore the reason behind this. I’m basically running docker run --rm -v scripts:/opt dvcorg/cml-py3:runner ./opt/test.sh although my answer is ..."exec: \"./opt/test.sh\": stat ./opt/test.sh: no such file or directory": unknown.. And the content of opt/test.sh is apt update && sudo apt install python3.8. Note: here I was not capable of finding which operating system the docker is using. Can I help to document and maintaining your docker images? Where is the repository generating them?

  2. run a container of my own that runs my training, like this at github actions:

train-test:
    name: Train and report
    needs: [python-test]
    runs-on: [ubuntu-latest]
    container: docker://dvcorg/cml-py3:runner

    steps:
      - uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Create virtualenv
        shell: bash
        run: |
          pip install --no-cache-dir pipenv==2020.8.13
          pipenv install --dev

      - name: cml_run
        shell: bash
        env:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
        run: |
          # run-cache and reproduce pipeline
          dvc pull ml/input/data/training/creditcard.csv.dvc
          dvc repro

Note: I had to upgrade DVC version as the original docker version was below 1.0 and requiring Dvcfile.

Continuing with my explanation: dvc repro in the code above calls a docker command like this docker run --rm -u ${CURRENT_UID}:${CURRENT_UID} -v ${PWD}/ml:/opt/m ${DOCKER_IMAGE_NAME} train --project_name ${project-name} --input_dir /opt/${DATA_FILE} from a Makefile… The problem here is the volume mounting. On the logs, it shows that it built the volume, although the container code can’t find the files that are supposed to be in that volume.

If it helps I have an open example here: https://github.com/ricoms/credit-fraud-dealing-with-imbalanced-datasets-mlops

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
0x2b3bfa0commented, May 11, 2021

@ricoms & @diegobit, we’ve released new image tags with updated package versions: docker://dvcorg/cml:0-dvc2-base1-gpu includes Ubuntu 20.04, Python 3.8, CUDA 11.0.3 and CuDNN 8. 🎉

1reaction
DavidGOrtegacommented, Dec 4, 2020

👋 @ricoms

please replace

container: docker://dvcorg/cml-py3:runner

with

container: docker://dvcorg/cml-py3:latest

Actually not sure what is that runner tag 🤔 Might be a stale version. Did you find it in a tutorial?

Can I help to document and maintaining your docker images?

Absolutely!

Where is the repository generating them?

This one! the Dockerfiles are in the docker folder and Github actions inside .github/workflows are building the images.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to Upgrade Python to 3.9 { Windows, macOS and Linux}
brew install python3. If you already have a version of Python 3 installed, upgrade the package with the brew upgrade command:
Read more >
How to Update Python
Here's an easy tutorial on how to update Python. ... Allows you to use two different versions of Python 3 at the same...
Read more >
How to Install Python 3 on Mac and Update the Version with ...
You can update your version by installing a new version, making it your global default, and optionally uninstalling the old version.
Read more >
keeping python 3 up to date on a mac - Apple Stack Exchange
to update python run brew update in the Terminal (this will update Homebrew) and then brew upgrade python3 if a new version of...
Read more >
How to upgrade to Python 3.8 on Ubuntu 18.04 LTS - isw blog
Many ubuntu users are facing problems during upgrading python to the latest version. Use this simple steps to upgrade to Python 3.8.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found