irdest/nix

Nix packaging for irdest components

From the Nix Package Manager Guide:

Nix is a purely functional package manager. This means that it treats packages like values in purely functional programming languages such as Haskell — they are built by functions that don’t have side-effects, and they never change after they have been built. Nix stores packages in the Nix store, usually the directory /nix/store, where each package has its own unique subdirectory such as

/nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1/

where b6gvzjyb2pg0… is a unique identifier for the package that captures all its dependencies (it’s a cryptographic hash of the package’s build dependency graph). This enables many powerful features.

Packages

The default.nix file describes all irdest-specific packages that are not packaged in nixpkgs. Some of them are:

irdest-rust

$ nix-build nix -A irdest-rust

A combined derivation that builds the whole rust workspace (equivalent to running cargo build --release in the repository root). It is packaged with nearsk. The Rust dependencies are read from ../Cargo.lock.

The web client (which connects to a local irdest instance) based on emberjs. It is packaged with yarn2nix. The JavaScript dependencies are read from ../emberweb/yarn.lock.

irdest-website

$ nix-build nix -A irdest-website

The website served on irde.st. It is built using the Hugo static site generator.

Using nix-shell for development

You can build Rust components in a Nix development shell, which will make all required native dependencies available:

$ nix-shell nix -A irdest-rust.builtDependencies
$ cargo build

External dependencies

All dependencies that are not described in this repository are managed using Niv. Niv provides a convenient command line interface to update and add dependencies. The details will be saved in nix/sources.json.

For example: To update the nixpkgs version to the latest commit on the master branch in the upstream repository, run niv update nixpkgs --owner NixOS --branch master in the repository root.

To use dependencies added using Niv in the Nix packaging, use the sources attribute set (obtained via import ./sources.nix) in default.nix. For example the naersk repository is available as sources.naersk.

Continous Integration

These packages are also used to drive GitLab CI jobs. This means you can locally reproduce the exact environment of the CI job by running the nix-build command with the same parameters used by CI.

Dynamic Child Pipelines

Because the description language used by GitLab CI is heavily limited, some GitLab pipelines are generated as Dynamic Child Pipelines. This means a gitlab-ci.yml will be generated by Nix code based on the repository contents.

Rust unit tests

The unittests pipeline will contain one GitLab CI job for each Rust unit test (for all crates in the Rust workspace). This is achieved by building the corresponding test binaries (cargo build --tests) and running each of them with the --list parameter. The output is parsed to a Nix attribute set to provide a Nix derivation (~= package) for each test.

You can view the generated pipeline:

$ nix-build nix/ci/rust.nix -A pipeline
/nix/store/a5hv156ww6mcwnwd4zxslyqhci7ab5xc-rust-gitlab-ci.yml

$ jq . result
{
  "alexandria-crypto::aes::key_is_key": {
    "artifacts": {
      "paths": [
        "result/*"
      ],
      "when": "always"
    },
    "needs": [],
    "script": [
      "nix-build nix/ci/rust.nix -A tests.alexandria.crypto::aes::key_is_key",
      "exit $(cat result/status)"
    ],
    "stage": "build",
    "tags": [
      "irdest-nix"
    ]
  },
  "alexandria-crypto::aes_encrypt_decrypt": {
    [...]
  },
  [...]
}

$

You can run single Rust unit tests using Nix, in the same environment used by the CI jobs:

$ nix-build nix/ci/rust.nix -A tests.alexandria.crypto::aes::key_is_key
[...]
/nix/store/l511d6zk22ar2j76wp1m2v489gln2v21-alexandria-crypto-aes-key_is_key

$ ls result
result log

$