Compiling Crystal on ARM

I was recently tasked to get crystal compiled on ARM for testing CNF Conformance which is an open source test suite to demonstrate conformance and implementation of Cloud Native Network Functions for vendors running Kubernetes.

I don't consider myself a code developer but am familiar with the basic development tools and building packages from source.

These are the steps outlined on Ubuntu Linux 18.04 LTS I got Crystal compiled in order to run crystal builds on ARM.

Pre-Requirements

• 1 x amd64 host running Ubuntu 18.04 LTS
• 1 x arm64 host running Ubuntu 18.04 LTS
• Internet Access for both.

Prepare your ARM Host

Install Development Package Dependencies and Other Pre-Req's

• You'll need to install gcc, make and other development packages required if running Debian/Ubuntu, run the following on your ARM host (you may already have these installed; it will skip any already present):
  • apt install gcc llvm-7 graphviz libevent-dev g++ libpcre3-dev libpcre++-dev libgc-dev libyaml-dev libxml2-dev libcrypto++-dev libssl-dev
  • The above command should install all the other required dependencies for each package.
  • Now run the following and save the output somewhere as you will need it for a future step:
  • gcc -dumpmachine
  • On our machine, our output was aarch64-linux-gnu, your output may differ but take note of this as we have to cross compile on x86 in the next steps.
  • Go ahead and download the crystal code from git:
  • git clone https://github.com/crystal-lang/crystal.git
  • Check out the tagged version you need or build from the master branch. In my case, we needed v0.33.0 so I did the following:
  • cd crystal; git checkout 0.33.0

Compiling Crystal

Compile on x86 First

• Before you can compile on ARM, you need to compile on x86.
  • If you never compiled crystal on your x86 host, you likely need the development packages mentioned in the previous steps. Run the following on your x86 to grab most of the required development packages and libraries (you may already have these installed, so it will skip any already present):
  • apt install gcc llvm-7 graphviz libevent-dev g++ libpcre3-dev libpcre++-dev libgc-dev libyaml-dev libxml2-dev libcrypto++-dev libssl-dev
  • Download the crystal code from git:
  • git clone https://github.com/crystal-lang/crystal.git
  • Like in our previous steps on the ARM host, we went with Crystal v0.33.0
  • cd crystal; git checkout 0.33.0
  • Before we can run our build, we need to make crystal or it will complain about missing files on our compile.
  • Now we can compile crystal to obtain our crystal.o file to use on the ARM host. Run the following, take note of your gcc-dumpmachine previously as it's needed for the --target in the command:
  • ./bin/crystal build src/compiler/crystal.cr --cross-compile --target=aarch64-linux-gnu -D without_openssl -D without_zlib --release
  • Take note we compiled using ./bin/crystal which uses the crystal source code wrapper script. If you already have crystal installed, using the installed crystal command will fail.
  • This should produce a crystal.o file now you can copy to your target ARM host and take note of the command it provides for you to run next to build crystal. We'll use this command on our ARM host compile.

Compile on ARM

• Now that you got a crystal.o file, do the following on your ARM host.
  • Copy the crystal.o file into your checked out crystal repo where you already checked out the 0.33.0 version to compile.
  • And the command in the output of your crystal.o compile on x86, ours was the following (Note: Your command may differ depending on machine, do not use this command below):
  • cc 'crystal.o' -o 'crystal' -rdynamic ./crystal/src/llvm/ext/llvm_ext.o '/usr/bin/llvm-config-7 --libs --system-libs --ldflags 2> /dev/null' -lstdc++ -lpcre -lm /usr/local/depot/crystal-0.33.0-1/bin/../lib/crystal/lib/libgc.a -lpthread /home/user/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/local/depot/crystal-0.33.0-1/bin/../lib/crystal/lib -L/usr/lib -L/usr/local/lib
  • You should now have a crystal binary in your checked out repo ./crystal directory but you're not finished, you'll have to use the wrapper that's located in ./crystal/bin/ and it automatically looks in .build so do the following steps:
  • mkdir .build; mv crystal .build
  • Now add the crystal wrapper to your user path where pwd is your ./crystal checked out repo location:
  • export OLDPATH=$PATH; export PATH=$PATH:$(pwd)/bin
  • If you intend to keep this path, add it to your users .bash_profile configs if bash is your default shell.
  • You can verify your crystal build and version by checking it's version:
  • crystal version

Crystal Needs Shards

• Shards doesn't need to be cross compiled but here are the steps we ran to get them to compile as the normal make to build would not compile on our ARM host.
  • Grab the latest source code for shards:
  • git clone https://github.com/crystal-lang/shards.git
  • cd into the directory:
  • cd shards
  • The latest version would not work on ARM so we went with the last stable release 0.8.1:
  • git checkout v0.8.1
  • And now we can compile shards:
  • crystal build src/shards.cr -o /usr/lib/crystal/bin/shards --release
  • This will create a shards binary in your repo checkout. We created a ./shards/bin directory to move the binary into and added that to our path as well where pwd is ./shards of your shards git repo checkout location.
  • export OLDPATH=$PATH; export PATH=$PATH;$(pwd)/bin

And there you have it, a working crystal build environment on ARM.

And to give credit where due, a few sources used to get this going for me were found at https://danilafe.com/blog/crystal_on_arm/ and this Russian post for some of the shards build at http://constxife.ru/blog/2018/10/16/rpi-crystal.html (which you may need to run through Google Translate) which were both great resources to complete this task.