Short and Exciting Journey of M1 Build Agent Configuration

Back in November 2020 Apple’s M1 chip was introduced and as the end users moved forward to M1 based Macs it became mandatory to build applications that are compatible with the new technology.

The M1 chip has incredible improvements and features but I won’t cover them in this post.There are many resources on the internet covering this and I encourage you to explore them. In this post I will cover several challenges I tackled while setting up an M1 build. I will share tips which might be useful and save you time in the future.

My journey started when I was assigned a task of preparing a native M1 build agent for Lightrun’s Python and Java agents. Not to be confused with a CI agent. Lightrun’s server side offering is an OS native agent that integrates with the language runtime. 

I started with research on the implementation of the M1 build agent and I decided to use Scaleway’s Apple silicon M1 as a service. 

The onboarding process went pretty smooth and was straight forward. Within several clicks I had an up and running on the M1 MAC mini instance that was ready for use.

The next step was connecting the build agent to the Azure devops pipeline. There’s a wonderful blog that describes the process of Azure pipeline installation as a step-by-step guide for a self hosted  agent on M1. You can find this guide here. 

I want to bring to your attention the fact that at the time of this writing the azure pipeline agent is NOT supported on the M1 architecture and that is why we need to use Rosetta 2. Rosetta  Lets a Mac with Apple silicon execute apps built for an Intel processor as if they were native applications.  For those who want to stay updated with the Azure Pipeline Agent support for Apple M1 there is an open discussion on github, available here.

The note above is important since you might end up with a pipeline that invokes a bash script and it will utilize Rosetta 2 which in turn produces artifacts that target amd64 architectures and not arm64. It could lead to unintended consequences where you expected to compile the application targeting arm64 and instead the result is amd64. The good news is that you can work around it by adding the “arch -arm64” argument before invoking the script. E.g.:

         - script: |
             arch -arm64 /<PATH>/<TO>/<SCRIPT>/someScript.sh
            displayName: 'running script on M1 build agent'

In case you’re unsure whether your artifact is indeed an arm64 artifact. You can test this using the file command or the activity monitor tool.. Below are examples of both:

• The following file command shows arm64 output:

file /<PATH-TO-YOUR-FILE>/Object.so 
Object.so: Mach-O 64-bit dynamically linked shared library arm64

• This file command shows amd64 output:

file /<PATH-TO-YOUR-FILE>/Object.so 
Object.so: Mach-O 64-bit dynamically linked shared library x86_64

When running the activity monitor you can inspect the “Kind” column to see the architecture of the running binary. Intel means amd64 and Apple means arm64:

At this point I had the impression that the heavy lifting of this project is over. Unfortunately, this was not the case…

Installing python versions

When installing specific versions of Python you should be aware of these tracked Python issues:

https://bugs.python.org/issue41100

https://github.com/python/cpython/pull/22855

The issues briefly discuss the support of macOS 11 and Apple Silicon. The interesting detail is the content of this related message:

https://bugs.python.org/msg382939

Are there plans to backport PR 22855 to any branches older than 3.9?

The plan is to also support 3.8 on Big Sur and Apple Silicon as 3.8 is still in bugfix mode. There are no plans to backport support to 3.7 and 3.6 which are in the security-fix-only phase of the release cycles.

In my case I successfully installed versions – 3.6.15 , 3.7.12 , 3.8.11 , 3.9.6 using pyenv.

As for 3.6 and 3.7 I encountered errors when I tried to install 3.6.14 and 3.7.11 but those errors disappeared on 3.6.15 and 3.7.12.

In order to obfuscate scripts I used PyArmor https://pyarmor.readthedocs.io/en/latest/# . If you are a user of  this tool you should be aware that starting with PyArmor version 7.3.3 onwards a fix was added to circumvent an issue with obfuscated scripts on M1.

TL;DR

To recap, my overall experience of making a build agent from an M1 machine was fascinating and it’s just the beginning. The most impactful recommendation I can offer is to pay attention to the output and make sure your binaries match expectations (arm64/amd64). I would also like to highlight the challenges of Apple’s security restrictions.

If you had similar experiences I’d love to hear from you.

If you like reading this blog and you want to get familiar of what we are doing in Lightrun in the context of changing the future of connect developers to their live applications you more then welcome visit our blogs here – https://lightrun.com/blog/