Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Improve docker build scripts

See original GitHub issue

🚀 Feature

Currently, we can build and publish dockers using Circle CI and it works more or less well, however, PR’s modifying Dockerfile can not build docker images to ensure PR correctness (related to the way how the PR is checkout on Circle CI).

The idea is

refactor all build_all.sh scripts (docker/main/build_all.sh, docker/hvd/build_all.sh and docker/msdp/build_all.sh) into a single shell script that could build a single docker image.
use github actions to build only on PR a single docker image and introduce a single yml file for that (to see what can be done).
- publishing will use existing scripts
add a test inside docker folder as a python script to check docker image versions for : torch, ignite ; can import cv2 ; maybe something else.

To Do list:

refactor build_all.sh into build.sh
run docker image build for PR on GA
run tests on built image. Tests
- For all images
  - can import torch and its version == required one
  - can import ignite and its version == required one
- for all -vision images
  - can import opencv without driver issue
- for all horovod images
  - can import horovod and its version == required one
- for all msdp images
  - can import deepspeed and its version == required one

cc @fco-dv

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:12 (2 by maintainers)

Top GitHub Comments

1reaction

trsvchncommented, Mar 1, 2021

@vfdev-5 Oh, no “manually” means using a kind workaround with manual events

https://docs.github.com/en/actions/reference/events-that-trigger-workflows#manual-events

I’ve found this solution

https://github.community/t/trigger-an-action-upon-completion-of-another-action/17642/2

1reaction

trsvchncommented, Mar 1, 2021

@vfdev-5 Yes, I am going to start from this:

create gtihub actions
add a test inside docker folder as a python script to check docker image versions for : torch, ignite ; can import cv2 ; maybe something else.

Top Results From Across the Web

Best practices for writing Dockerfiles - Docker Documentation

Best practices for writing Dockerfiles. This document covers recommended best practices and methods for building efficient images. Docker builds images ...

Tips for optimizing Docker builds - CircleCI

Docker images should remain as lean as possible. This helps with portability, shorter build times, reduced complexity, and smaller file sizes.

A small script for building, tagging and pushing Docker images

AWS ECR. The docker-build --ecr command attempts to create and push to ECR repositories if the following environment variables are present:.

Optimizing Your Dockerfile - Ginkgo Bioworks

How We Made our Docker Builds Three Times Faster ... yarn run v1.19.1 $ react-scripts build Creating an optimized production build.

Docker Tutorials: Image - Optimize faster builds and smaller ...

Multi-stage builds are a method of organizing a Dockerfile to minimize the size of the final container, improve run time performance, ...