ARM64 support in docker images

Description

ARM64 support in docker images

Use case / motivation

In the recent years ARM based cpu popularity hugely increased. The arm based cpus are offering better price to performance ratio and laptops outstanding performance. It’s getting popular to run ARM nodes in your Kubernetes clusterss. If you want to run airflow on arm node in kubernetes the docker image wont start it wailt fails with standard_init_linux.go:219: exec user process caused: exec format error.

ARM based cpu in the database world like AWS Graviton or in the consumer world like Apple M1.

Related Issues

In #14796 @potiuk confirmed there is no office support for ARM64 support in docker images

Update: ARM is supported for development now. The support for released imges is coming.

[potiuk]

The 2.3.0 now officially supports ARM64 and AMD64:

Closing this one 😃.

[potiuk]

For example this one:

https://github.com/apache/airflow/pkgs/container/airflow%2Fmain%2Fprod%2Fpython3.7/19115034?tag=latest

Generally all the “dev” images of our are following the scheme:

• ghcr.io/apache/airflow/main/prod/pythonX.Y
• ghcr.io/apache/airflow/main/ci/pythonX.Y

Following the naming convention we have for those: https://github.com/apache/airflow/blob/main/IMAGES.rst#naming-conventions

You can always retag the latest image to a regular airflow one if you want to use some more familiar names:

docker pull ghcr.io/apache/airflow/main/prod/python3.7
docker tag ghcr.io/apache/airflow/main/prod/python3.7 apache/airflow:latest

Note that X.Y >= 3.7 because we dropped Python 3.6 in main and it is not refreshed any more.

We only build and refresh the “latest” image for multi-platform. They are automatically refreshing after successful main build - so the images often reflect the latest “main” (unless we are in a period of fixing some main failures because of dependencies update - then they might be few commits behind).

Those are the images that breeze uses, thus the naming convention is different than the “release” images which are published on GitHub. We do not publish multi-platform images there yet.

We have also “per-commit” images that are used CI - but for now those are single-platform (amd64) only. The naming convention there is:

• ghcr.io/apache/airflow/main/ci/pythonX.Y:<COMMIT_SHA>
• ghcr.io/apache/airflow/main/prod/pythonX.Y:<COMIMT_SHA>

I proposed to start releasing the experimental multi-platform images to DockerHub as well (we could do it for one of the betas/rcs of 2.3.0) - the tooling we have is ready for it - but I have not received any answers yet 😃 - feel free to comment if you think it is a good idea: https://lists.apache.org/thread/pqhks390dkso9x668gbnvjq6k6wv8h9h