libatomic.so.1: cannot open shared object file: No such file or directory

I don’t think this a bug with caxa (it depends on one’s point of view!), yet I think it is an issue that caxa users will come across. For one, I am still looking for a convenient solution (that didn’t require me to compile Node.js from source).

I have created a caxa-based executable for my app, for ARM v7. Then, when trying to run it in a Docker container for the ARM platform, I got the following error regarding a missing shared library:

$ docker run -it --platform linux/arm/v7 -v ~/Downloads:/mnt/Downloads debian:10 /bin/bash

root@1f0afeb707aa:/# /mnt/Downloads/my-armv7-caxa-app
... node_modules/.bin/node: error while loading shared libraries: 
    libatomic.so.1: cannot open shared object file: No such file or directory

I believe that the problem is that, when creating the caxa executable, I was using a standard Node.js installation that uses shared / dynamically linked libraries. Then caxa bundled in the base Node.js executable, but not the shared libraries. For the libatomic.so.1 library in particular, the error above can be avoided if the end users of my app install the library before running the caxa-based executable:

apt-get update && apt-get install -y libatomic1

However, at least my use case for caxa is to simplify end users’ life by avoiding pre-requisites like installing Node.js (#20), “just download this executable and run it”, and if I had to ask end users to install shared libraries before running the caxa executable, it would spoil the experience.

I assume that the solution is to use a fully statically compiled version of Node.js (including libatomic.so.1) when creating the caxa executable. Where to find that though? For all architectures supported by caxa: x64, ARM v6, ARM v7, ARM 64. I gather that the standard Node.js builds offered for download are dynamically linked: https://nodejs.org/en/download/

[pdcastro]

Other notes:

• make -j4, instead of just make, specifies 4 compilation jobs in parallel, significantly reducing the overall compilation time if you have at least that many CPU cores. I’ve found a suggestion for make -j$(nproc), where nproc returns the number of CPU cores in the machine. Note that it will also cause the machine’s cooling fan to run at maximum speed, and if it is anything like mine, passers-by may confuse it with a jet engine. ✈️

• Using debian (or anything other than alpine?) as the Dockerfile base image for compiling Node.js statically appears to be a bad idea, because debian uses glibc and the compilation will produce warning messages such as:

b_sock.c:(.text+0x271): warning: Using 'gethostbyname' in statically linked applications
  requires at runtime the shared libraries from the glibc version used for linking

That sounds like users would not only have to install shared libraries, but also match the version used during Node.js compilation. If so, it would be much worse than a dynamically linked Node.js binary! Googling it, I found this other Node.js issue and comment:

https://github.com/nodejs/help/issues/1863#issuecomment-482852205 […] static linking to glibc doesn’t work in general, google around for reasons. You need to link to a libc like musl if you want to use --fully-static.

Well that’s good to know! alpine uses musl, so alpine sounds like the way to go.

[pdcastro]

I kept running into new issues. It turned into whack-a-mole.

After whacking enough moles and waiting enough hours, 😃 I can share some early, encouraging results.

With very similar Dockerfiles as the one from StackOverflow (linked in an earlier comment), I’ve got static builds of Node.js v12 (a version I wanted to use) and also “accidentally” the very latest Node.js v17.0.0-pre:

Dockerfile for Node.js v12

FROM alpine:3.11.3
RUN apk add git python gcc g++ linux-headers make
WORKDIR /usr/src/app
ENV NODE_VERSION=v12.22.3
RUN git clone https://github.com/nodejs/node && cd node && git checkout ${NODE_VERSION}
RUN cd node && ./configure --fully-static --enable-static
RUN cd node && make

Dockerfile for Node.js' master branch (v17.0.0-pre on 06 July 2021)

FROM alpine:3.11.3
RUN apk add git python3 gcc g++ linux-headers make
WORKDIR /usr/src/app
RUN git clone https://github.com/nodejs/node
RUN cd node && ./configure --fully-static --enable-static
RUN cd node && make

Note that the two Dockerfiles use different versions of Python and checkout different branches of Node.js. They were built with Docker v20.10.7, a command line similar to:

docker build -t node12-armv7-static-alpine --platform linux/arm/v7 .

In both cases, an ARM node binary was produced that avoided the libatomic.so.1 error:

$ docker run -it -v ~/Downloads:/mnt/Downloads --platform linux/arm/v7 debian:10 /bin/bash

root@fa428ae3d7da:~# ls -l /mnt/Downloads/node*
-rwxr-xr-x 1 root root 75159308 Jul  6 23:30 /mnt/Downloads/node17-arm-v7-built-in-alpine-qemu
-rwxr-xr-x 1 root root 44600464 Jul  6 13:03 /mnt/Downloads/node12-arm-v7-built-in-alpine-qemu
-rwxr-xr-x 1 root root 41230580 Jul  6 14:18 /mnt/Downloads/node12-arm-v7-copied-from-arm32v7-node-12

root@fa428ae3d7da:~# /mnt/Downloads/node12-arm-v7-built-in-alpine-qemu --version
v12.22.3

root@fa428ae3d7da:~# /mnt/Downloads/node17-arm-v7-built-in-alpine-qemu --version
v17.0.0-pre

root@fa428ae3d7da:~# /mnt/Downloads/node12-arm-v7-copied-from-arm32v7-node-12 --version
 error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory

Above, the Node.js binary that produced the libatomic.so.1 error was the dynamically linked Node.js binary copied from the “official” arm32v7/node:12 image.

Note also the file sizes in the ls -l output above. The statically compiled Node.js v12 binary is just over ~3MB larger than the dynamically linked one. I am OK with that. I have not compared the dynamic vs static size difference for other versions of Node.js yet.

I haven’t yet tested using the statically compiled Node.js versions with caxa, but the results above are encouraging. 😃