[CI] Discussion about Docker and CI

I open this issue to talk about CI Goal is to optimize/improve the way we build docker images.

I’ve already tested a lot before open this issue.

Today a full build take 1H

Why is this so long ?

• Repeated build of the same dependencies ( OZW )
• Installation of the same npm packages again and again
• Our dockerfile is not cache efficient

---

Layers caching

First problem I encountered is circleci. For using layers caching we must pay for it ( I don’t really understand why to be honest ). Fir that reason, all tests and implementation will be done on Github Workflows.

Dependencies

Open Zwave

To avoid building openzwave from source there is some simple solution => Multi Stage build

We take compiled binary from an already builded openzwave image. We are lucky because openzwave build their own and we can get real version tag ( not commit like today )

So OZW can now be added in your image like this:

# OpenZwave version 1.6.1520 
FROM chrisns/openzwave:alpine-1.6.1520 as ozw

# Just copy conf and binaries
COPY --from=ozw /usr/local/include/openzwave /usr/local/include/openzwave
COPY --from=ozw /openzwave/libopenzwave.so* /lib/
COPY --from=ozw /openzwave/config /usr/local/etc/openzwave

And voilà , OZW in our images without building it ( Several minutes saved here 😃 )

node_modules

Also named by me “the tricky part” ( a lot of fail 😅 )

Even in active project like Gladys, npm deps doesn’t change so much, why node_modules is not cached by docker when building ?

Simple answer, because we run npm install on the whole base code ! Cache is invalidated on each build.

Simple solution is to copy only package.json and lock file and run npm install on it, that way cache will work if no modification is commited , cache will be used by docker

After implementation, I’m able to build docker images in only 5 minutes ( for all platforms ) with change in codebase ( server and/or front )

---

There is some work to do but I need feedback from @GladysAssistant/core to know if I continue on this way or not.

PS: DockerFile for testing => https://github.com/VonOx/GLADYS_CI/blob/master/docker/Dockerfile.buildx STEP 2 is not easy to maintain I know that 😉 .

[VonOx]

Yes this is tricky because in Gladys each service have their own package.

But I found a solution 😃

Full build with test and no cache take 1H15 Cached build + test take 17 minutes

Docker stage ( only building image ) 2 minutes, I can’t do better

Now I need to test generated images. PR incomming

[Pierre-Gilles]

Got it, nice!