Remote building NixOS packages using QEMU

2020.05.17

Nix

NixOS

Embedded

In a previous post I wrote about a method for remote building aarch64 packages for NixOS using a cloud ARM server, so that you can escape very long build times on the Raspberry Pi. This post does something similar, but using a faster x86_64 NixOS system, using QEMU binary wrapping.

This method is much cheaper and readily available than building on an expensive cloud server which costs $1 per hour.

Admittedly, the build is significantly slower using QEMU than it would be natively. Perhaps this can be improved by using a cross-compiler.

Generate an SSH key¶

On the Raspberry Pi, generate an ssh key with ssh-keygen. Copy the content of the public key to the host builder.

Host setup¶

Add the following to configuration.nix:

Declare that we want QEMU binary wrapping.

{
  boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
}

Add an SSH key to root

{
  users.users.root.openssh.authorizedKeys.keys = [
    "ssh-rsa...YOURPUBKEY...pi@nixos"
  ];
}

After doing this, you must reboot, so that the kernel can be configured for the new binary format.

Setup on Raspberry Pi¶

Add and entry to /root/.ssh/config with your build host's IP:

Host desktop
     HostName 192.168.1.73
     User root
     IdentityFile /root/.ssh/desktop

Test that you have a passwordless remote login:

$ sudo ssh desktop whoami
root
$ sudo ssh desktop nix-store --version
nix-store (Nix) 2.3.4

Add the remote builder to configuration.nix.

{
 nix.buildMachines = [
   {
      hostName = "desktop";
      systems = ["x86_64-linux" "aarch64-linux"];
      maxJobs = 1;
      speedFactor = 2;
      supportedFeatures = [ "nixos-test" "benchmark" "big-parallel" "kvm" ];
      mandatoryFeatures = [ ];
    }
  ];
}

Build on Raspberry Pi¶

Build, disabling local jobs:

sudo nixos-rebuild switch -I nixpkgs=. -j0

eipi.xyz

Remote building NixOS packages using QEMU

contents

Generate an SSH key¶

Host setup¶

Setup on Raspberry Pi¶

Build on Raspberry Pi¶

References¶