Automate Katib Releases

Currently, to make Katib releases we have to follow this manual process: https://github.com/kubeflow/katib/tree/master/docs/release

We run make release command, build and publish the release Docker images locally, and publish Katib SDK version. Since we build docker images locally, our release images don’t support multi OS arch: https://hub.docker.com/layers/kubeflowkatib/katib-controller/v0.14.0/images/sha256-51ca80d6005010ff08853a5f7231158cb695ea899b623200076cbc01509fc0b5?context=repo.

The release process should be automated. For example, we can utilise GitHub Actions to make Katib releases.

cc @tenzen-y @johnugeorge

---

Love this feature? Give it a 👍 We prioritize the features with the most 👍

[johnugeorge]

@anencore94 @tenzen-y Currently, we are not using self hosted runners. We need to review this sometime if we can use self hosted runners in AWS

[tenzen-y]

We can build multiplatform images using the default amd64 runner. Actually, we publish multi-platform images for every commit like this. Probably, we don’t need arm64 runners for the multi-platform build.

Sure, But building an arm-image in amd64-runner would be much slower since it uses some kind of virtualizer like QEMU to build arm-image. So if we could prepare arm64 runner, then it would be better. However, if it is not affordable, then yes I agree with to build it with amd64-runner. @tenzen-y

@anencore94 I see. That’s a great idea, I agree with your idea. It makes speed up building time if we could prepare arm64 runners. Maybe, docker build create --append command and remote build instance help us.

Using multiple native nodes provide better support for more complicated cases that are not handled by QEMU and generally have better performance. You can add additional nodes to the builder instance using the --append flag.

Assuming contexts node-amd64 and node-arm64 exist in docker context ls;

 docker buildx create --use --name mybuild node-amd64
 docker buildx create --append --name mybuild node-arm64
 docker buildx build --platform linux/amd64,linux/arm64 .

https://docs.docker.com/build/building/multi-platform/#building-multi-platform-images

The Buildx remote driver allows for more complex custom build workloads, allowing you to connect to externally managed BuildKit instances. This is useful for scenarios that require manual management of the BuildKit daemon, or where a BuildKit daemon is exposed from another source.

docker buildx create \
  --name remote-unix \
  --driver remote \
  unix://$HOME/buildkitd.sock

https://docs.docker.com/build/drivers/remote/