[Proposal] Supporting arch other than x86_64 or amd64 (with ideas)

Hello? I recently created a simple script that builds multi-platform supported docker image of shadowbox (or outline-server). During building the script, I got some clean ways to build multi-platform shadowbox image even there is directory called third_party which is platform-sensitive.

Using Docker BuildKit’s $TARGETPLATFORM

REF: Here is direct reference to the part of my current multiarch builder(not impl.) script which uses $TARGETPLATFORM.

Thanks to Docker BuildKit’s TARGETPLATFORM argument in Dockerfile, we can use it in COPY command inside of Dockerfile.

Like following snippet:

ARG TARGETPLATFORM
RUN mkdir -p third_party
COPY third_party/outline-ss-server/${TARGETPLATFORM} third_party/outline-ss-server
COPY third_party/prometheus/${TARGETPLATFORM} third_party/prometheus

Note that ARG has scoping issue, so we need to put it after FROM keyword.

Following to above, we can expect paths like third_party/${TARGETPLATFORM}:

• third_party/linux/arm64
• third_party/linux/amd64
• third_party/linux/arm/v7
• third_party/linux/arm/v6
• …

Finally, we can change current third_party directory structure into (only considering docker implementation):

- `${TARGETPLATFORM}`
  - outline-ss-server (binary)
  - prometheus (binary)

// respect current directory structures
- outline-ss-server
  - LICENSE
  - METADATA
- prometheus
  - LICENSE
  - METADATA
- shellcheck (keep original as it is not required at build process)

I agree that this implementation is not completed and should not be used as only works for docker.

Using $(uname -m)

EDIT: Now I have almost complete implementation of this (link to #issuecomment-1021256958).

Or if we target both docker way and non-docker way building, I think it’s reasonable to change directory structure into:

- $([[ "$(uname -s)" == "Darwin" ]] && echo "macos" || echo "linux")
  - $(uname -m) # For ARM, it should be arm64 if Apple based, otherwise aarch64
    - outline-ss-server
    - prometheus

: '
- linux
  - x86_64
  - aarch64
  - aarch32? (I could not check it)
- macos
  - x86_64
  - arm64
'

• StackOverflow: Answer of Differences between arm64 and aarch64

Also, modify the build script.

We still need additional changes for docker builds.

# Install third_party dependencies
readonly OS="$([[ "$(uname)" == "Darwin" ]] && echo "macos" || echo "linux")"
## Addition: check arch
readonly ARCH="$(uname -m)"
readonly BIN_DIR="${OUT_DIR}/bin"
mkdir -p "${BIN_DIR}"
cp "${ROOT_DIR}/third_party/prometheus/${OS}/${ARCH}/prometheus" "${BIN_DIR}/"
cp "${ROOT_DIR}/third_party/outline-ss-server/${OS}/${ARCH}/outline-ss-server" "${BIN_DIR}/"

I believe we can make even more simple by modifying build script located in /src/shadowbox/server/build.action.sh.

Ref

• My way to build multi-platform outline-server image (using docker buildx): https://github.com/seia-soto/outline-server-multiarch

---

I am not an expert to the development and want to hear your opinion to this proposal! 👏

[seia-soto]

Congrats! 🥳

Now I have a test suite that verifies the ways above work.

add-multiarch-support

This way uses pure $(uname -m) command to pick up valid third party binaries. Still, I needed to remap the value into Docker-recognizable text like below but it’s cleaner than add-multiarch-support-with-armv6:

# Install third_party dependencies
if [[ "$(uname)" == "Darwin" ]]; then
  readonly OS="macos"
  readonly ARCH="x86_64"
else
  readonly OS="linux"
  readonly ARCH="$(uname -m)"
fi

readonly BIN_DIR="${OUT_DIR}/bin"

mkdir -p "${BIN_DIR}"
cp "${ROOT_DIR}/third_party/prometheus/${OS}/prometheus_${ARCH}" "${BIN_DIR}/prometheus"
cp "${ROOT_DIR}/third_party/outline-ss-server/${OS}/outline-ss-server_${ARCH}" "${BIN_DIR}/outline-ss-server"

As above code describes, we just put value from $(uname -m) directly into third_party directory.

Tests

• x86_64: https://github.com/seia-soto/outline-server/runs/4946703360
• aarch64: https://github.com/seia-soto/outline-server/runs/4946703387
• armv7l: https://github.com/seia-soto/outline-server/runs/4946703426

add-multiarch-support-with-armv6 (dirtier but more coverage)

Like above add-multiarch-support but adds armv6 compatibility by adding some customizable parameters on build scripts. For brief diff, I added remap_arch functions and allowed customized value:

# Detect and set architecture for general users to build without installing emulator.
remap_arch() {
  local ARCH="${1}" AMD64="${2:-amd64}" ARM64="${3:-arm64}" ARMv7="${4:-armv7}" ARMv6="${5:-armv6}"

  [[ "${ARCH}" == *"amd64"* || "${ARCH}" == *"x86_64"* ]] && ARCH="${AMD64}"
  [[ "${ARCH}" == *"arm64"* || "${ARCH}" == *"aarch64"* ]] && ARCH="${ARM64}"
  [[ "${ARCH}" == *"v7"* ]] && ARCH="${ARMv7}"
  [[ "${ARCH}" == *"v6"* ]] && ARCH="${ARMv6}"

  echo "${ARCH}"
}

# Specify the target platform with `$SB_PLATFORM`.
[[ -z "${SB_PLATFORM:-}" ]] && SB_PLATFORM="$(remap_arch "$(uname -m)" linux/amd64 linux/arm64 linux/arm/v7 linux/arm/v6)"

I know that’s a bit dirty. However, in this way, we can extend the coverage even there are many variants in $(uname -m) output.

Tests

• x86_64: https://github.com/seia-soto/outline-server/runs/4946927874
• aarch64: https://github.com/seia-soto/outline-server/runs/4946927902
• armv7: https://github.com/seia-soto/outline-server/runs/4946927923
• armv6: https://github.com/seia-soto/outline-server/runs/4946927942

[Blackhatfrance]

Thanks @seia-soto