<!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->
<!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->
- [ ] Tested using sandboxing ([nix.useSandbox](https://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option `sandbox` in [`nix.conf`](https://nixos.org/nix/manual/#sec-conf-file) on non-NixOS linux)
- Built on platform(s)
- Built on platform(s)
- [ ] NixOS
- [ ] x86_64-linux
- [ ] macOS
- [ ] aarch64-linux
- [ ] other Linux distributions
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- [ ] For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/#sec-conf-file))
- [ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- [ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- [ ] Tested compilation of all pkgs that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review wip"`
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review wip"`
- [ ] Tested execution of all binary files (usually in `./result/bin/`)
- [ ] Tested execution of all binary files (usually in `./result/bin/`)
## Build wrapped Firefox with extensions and policies {#build-wrapped-firefox-with-extensions-and-policies}
## Build wrapped Firefox with extensions and policies {#build-wrapped-firefox-with-extensions-and-policies}
The `wrapFirefox` function allows to pass policies, preferences and extension that are available to firefox. With the help of `fetchFirefoxAddon` this allows build a firefox version that already comes with addons pre-installed:
The `wrapFirefox` function allows to pass policies, preferences and extension that are available to Firefox. With the help of `fetchFirefoxAddon` this allows build a Firefox version that already comes with addons pre-installed:
```nix
```nix
{
{
myFirefox = wrapFirefox firefox-unwrapped {
# Nix firefox addons only work with the firefox-esr package.
myFirefox = wrapFirefox firefox-esr-unwrapped {
nixExtensions = [
nixExtensions = [
(fetchFirefoxAddon {
(fetchFirefoxAddon {
name = "ublock"; # Has to be unique!
name = "ublock"; # Has to be unique!
@ -45,5 +46,7 @@ or type into the Firefox url bar: `about:policies#documentation`.
Nix installed addons do not have a valid signature, which is why signature verification is disabled. This does not compromise security because downloaded addons are checksumed and manual addons can't be installed. Also make sure that the `name` field of fetchFirefoxAddon is unique. If you remove an addon from the nixExtensions array, rebuild and start Firefox the removed addon will be completly removed with all of its settings.
Nix installed addons do not have a valid signature, which is why signature verification is disabled. This does not compromise security because downloaded addons are checksumed and manual addons can't be installed. Also make sure that the `name` field of fetchFirefoxAddon is unique. If you remove an addon from the nixExtensions array, rebuild and start Firefox the removed addon will be completly removed with all of its settings.
## Troubleshooting {#sec-firefox-troubleshooting}
## Troubleshooting {#sec-firefox-troubleshooting}
If addons are marked as broken or the signature is invalid, make sure you have Firefox ESR installed. Normal Firefox does not provide the ability anymore to disable signature verification for addons thus nix addons get disabled by the normal Firefox binary.
If addons do not appear installed although they have been defined in your nix configuration file reset the local addon state of your Firefox profile by clicking `help -> restart with addons disabled -> restart -> refresh firefox`. This can happen if you switch from manual addon mode to nix addon mode and then back to manual mode and then again to nix addon mode.
If addons do not appear installed although they have been defined in your nix configuration file reset the local addon state of your Firefox profile by clicking `help -> restart with addons disabled -> restart -> refresh firefox`. This can happen if you switch from manual addon mode to nix addon mode and then back to manual mode and then again to nix addon mode.
$ git diff -a > nixpkgs/pkgs/the/package/0001-changes.patch
```
```
If a patch is available online but does not cleanly apply, it can be modified in some fixed ways by using additional optional arguments for `fetchpatch`:
If a patch is available online but does not cleanly apply, it can be modified in some fixed ways by using additional optional arguments for `fetchpatch`:
@ -537,9 +537,34 @@ Note that because the checksum is computed after applying these effects, using o
Tests are important to ensure quality and make reviews and automatic updates easy.
Tests are important to ensure quality and make reviews and automatic updates easy.
Nix package tests are a lightweight alternative to [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests). They can be used to create simple integration tests for packages while the module tests are used to test services or programs with a graphical user interface on a NixOS VM. Unittests that are included in the source code of a package should be executed in the `checkPhase`.
* [NixOS **module tests**](https://nixos.org/manual/nixos/stable/#sec-nixos-tests), which spawn one or more NixOS VMs. They exercise both NixOS modules and the packaged programs used within them. For example, a NixOS module test can start a web server VM running the `nginx` module, and a client VM running `curl` or a graphical `firefox`, and test that they can talk to each other and display the correct content.
* Nix **package tests** are a lightweight alternative to NixOS module tests. They should be used to create simple integration tests for packages, but cannot test NixOS services, and some programs with graphical user interfaces may also be difficult to test with them.
* The **`checkPhase` of a package**, which should execute the unit tests that are included in the source code of a package.
Here in the nixpkgs manual we describe mostly _package tests_; for _module tests_ head over to the corresponding [section in the NixOS manual](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
- [Multiple tests for nanopb](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/nanopb/default.nix)
- [Multiple tests for nanopb](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/nanopb/default.nix)
### Linking NixOS module tests to a package {#ssec-nixos-tests-linking}
Like [package tests](#ssec-package-tests-writing) as shown above, [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests) can also be linked to a package, so that the tests can be easily run when changing the related package.
For example, assuming we're packaging `nginx`, we can link its module test via `passthru.tests`:
Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of Erlang and any other interpreters, and then use your normal build tools. As an example with Elixir:
Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of Erlang and any other interpreters, and then use your normal build tools. As an example with Elixir:
@ -33,6 +33,7 @@ The recommended way of defining a derivation for a Coq library, is to use the `c
* `mlPlugin` (optional, defaults to `false`). Some extensions (plugins) might require OCaml and sometimes other OCaml packages. Standard dependencies can be added by setting the current option to `true`. For a finer grain control, the `coq.ocamlPackages` attribute can be used in `extraBuildInputs` to depend on the same package set Coq was built against.
* `mlPlugin` (optional, defaults to `false`). Some extensions (plugins) might require OCaml and sometimes other OCaml packages. Standard dependencies can be added by setting the current option to `true`. For a finer grain control, the `coq.ocamlPackages` attribute can be used in `extraBuildInputs` to depend on the same package set Coq was built against.
* `useDune2ifVersion` (optional, default to `(x: false)` uses Dune2 to build the package if the provided predicate evaluates to true on the version, e.g. `useDune2if = versions.isGe "1.1"` will use dune if the version of the package is greater or equal to `"1.1"`,
* `useDune2ifVersion` (optional, default to `(x: false)` uses Dune2 to build the package if the provided predicate evaluates to true on the version, e.g. `useDune2if = versions.isGe "1.1"` will use dune if the version of the package is greater or equal to `"1.1"`,
* `useDune2` (optional, defaults to `false`) uses Dune2 to build the package if set to true, the presence of this attribute overrides the behavior of the previous one.
* `useDune2` (optional, defaults to `false`) uses Dune2 to build the package if set to true, the presence of this attribute overrides the behavior of the previous one.
* `opam-name` (optional, defaults to `coq-` followed by the value of `pname`), name of the Dune package to build.
* `enableParallelBuilding` (optional, defaults to `true`), since it is activated by default, we provide a way to disable it.
* `enableParallelBuilding` (optional, defaults to `true`), since it is activated by default, we provide a way to disable it.
* `extraInstallFlags` (optional), allows to extend `installFlags` which initializes the variable `COQMF_COQLIB` so as to install in the proper subdirectory. Indeed Coq libraries should be installed in `$(out)/lib/coq/${coq.coq-version}/user-contrib/`. Such directories are automatically added to the `$COQPATH` environment variable by the hook defined in the Coq derivation.
* `extraInstallFlags` (optional), allows to extend `installFlags` which initializes the variable `COQMF_COQLIB` so as to install in the proper subdirectory. Indeed Coq libraries should be installed in `$(out)/lib/coq/${coq.coq-version}/user-contrib/`. Such directories are automatically added to the `$COQPATH` environment variable by the hook defined in the Coq derivation.
* `setCOQBIN` (optional, defaults to `true`), by default, the environment variable `$COQBIN` is set to the current Coq's binary, but one can disable this behavior by setting it to `false`,
* `setCOQBIN` (optional, defaults to `true`), by default, the environment variable `$COQBIN` is set to the current Coq's binary, but one can disable this behavior by setting it to `false`,
@ -13,6 +13,7 @@ In the following is an example expression using `buildGoModule`, the following a
- `vendorSha256`: is the hash of the output of the intermediate fetcher derivation. `vendorSha256` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorSha256 = null;`
- `vendorSha256`: is the hash of the output of the intermediate fetcher derivation. `vendorSha256` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorSha256 = null;`
- `runVend`: runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build.
- `runVend`: runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build.
- `proxyVendor`: Fetches (go mod download) and proxies the vendor directory. This is useful if any dependency has case-insensitive conflicts which will produce platform dependant `vendorSha256` checksums.
```nix
```nix
pet = buildGoModule rec {
pet = buildGoModule rec {
@ -112,16 +113,6 @@ done
Both `buildGoModule` and `buildGoPackage` can be tweaked to behave slightly differently, if the following attributes are used:
Both `buildGoModule` and `buildGoPackage` can be tweaked to behave slightly differently, if the following attributes are used:
### `buildFlagsArray` and `buildFlags`: {#ex-goBuildFlags-noarray}
These attributes set build flags supported by `go build`. We recommend using `buildFlagsArray`.
```nix
buildFlagsArray = [
"-tags=release"
];
```
### `ldflags` {#var-go-ldflags}
### `ldflags` {#var-go-ldflags}
Arguments to pass to the Go linker tool via the `-ldflags` argument of `go build`. The most common use case for this argument is to make the resulting executable aware of its own version. For example:
Arguments to pass to the Go linker tool via the `-ldflags` argument of `go build`. The most common use case for this argument is to make the resulting executable aware of its own version. For example:
@ -134,6 +125,21 @@ Arguments to pass to the Go linker tool via the `-ldflags` argument of `go build
];
];
```
```
### `tags` {#var-go-tags}
Arguments to pass to the Go via the `-tags` argument of `go build`. For example:
This contains instructions on how to package javascript applications. For instructions on how to add a cli package from npm please consult the #node.js section
The various tools available will be listed in the [tools-overview](#javascript-tools-overview). Some general principles for packaging will follow. Finally some tool specific instructions will be given.
## Getting unstuck / finding code examples
If you find you are lacking inspiration for packing javascript applications, the links below might prove useful.
Searching online for prior art can be helpful if you are running into solved problems.
### Github
- Searching Nix files for `mkYarnPackage`: <https://github.com/search?q=mkYarnPackage+language%3ANix&type=code>
- Searching just `flake.nix` files for `mkYarnPackage`: <https://github.com/search?q=mkYarnPackage+filename%3Aflake.nix&type=code>
### Gitlab
- Searching Nix files for `mkYarnPackage`: <https://gitlab.com/search?scope=blobs&search=mkYarnPackage+extension%3Anix>
- Searching just `flake.nix` files for `mkYarnPackage`: <https://gitlab.com/search?scope=blobs&search=mkYarnPackage+filename%3Aflake.nix>
## Tools overview {#javascript-tools-overview}
## General principles {#javascript-general-principles}
The following principles are given in order of importance with potential exceptions.
### Try to use the same node version used upstream {#javascript-upstream-node-version}
It is often not documented which node version is used upstream, but if it is, try to use the same version when packaging.
This can be a problem if upstream is using the latest and greatest and you are trying to use an earlier version of node. Some cryptic errors regarding V8 may appear.
An exception to this:
### Try to respect the package manager originally used by upstream (and use the upstream lock file) {#javascript-upstream-package-manager}
A lock file (package-lock.json, yarn.lock...) is supposed to make reproducible installations of node_modules for each tool.
Guidelines of package managers, recommend to commit those lock files to the repos. If a particular lock file is present, it is a strong indication of which package manager is used upstream.
It's better to try to use a nix tool that understand the lock file. Using a different tool might give you hard to understand error because different packages have been installed. An example of problems that could arise can be found [here](https://github.com/NixOS/nixpkgs/pull/126629). Upstream uses npm, but this is an attempt to package it with yarn2nix (that uses yarn.lock)
Using a different tool forces to commit a lock file to the repository. Those files are fairly large, so when packaging for nixpkgs, this approach does not scale well.
Exceptions to this rule are:
- when you encounter one of the bugs from a nix tool. In each of the tool specific instructions, known problems will be detailed. If you have a problem with a particular tool, then it's best to try another tool, even if this means you will have to recreate a lock file and commit it to nixpkgs. In general yarn2nix has less known problems and so a simple search in nixpkgs will reveal many yarn.lock files committed
- Some lock files contain particular version of a package that has been pulled off npm for some reason. In that case, you can recreate upstream lock (by removing the original and `npm install`, `yarn`, ...) and commit this to nixpkgs.
- The only tool that supports workspaces (a feature of npm that helps manage sub-directories with different package.json from a single top level package.json) is yarn2nix. If upstream has workspaces you should try yarn2nix.
### Try to use upstream package.json {#javascript-upstream-package-json}
Exceptions to this rule are
- Sometimes the upstream repo assumes some dependencies be installed globally. In that case you can add them manually to the upstream package.json (`yarn add xxx` or `npm install xxx`, ...). Dependencies that are installed locally can be executed with `npx` for cli tools. (e.g. `npx postcss ...`, this is how you can call those dependencies in the phases).
- Sometimes there is a version conflict between some dependency requirements. In that case you can fix a version (by removing the `^`).
- Sometimes the script defined in the package.json does not work as is. Some scripts for example use cli tools that might not be available, or cd in directory with a different package.json (for workspaces notably). In that case, it's perfectly fine to look at what the particular script is doing and break this down in the phases. In the build script you can see `build:*` calling in turns several other build scripts like `build:ui` or `build:server`. If one of those fails, you can try to separate those into:
```Shell
yarn build:ui
yarn build:server
# OR
npm run build:ui
npm run build:server
```
when you need to override a package.json. It's nice to use the one from the upstream src and do some explicit override. Here is an example.
you will still need to commit the modified version of the lock files, but at least the overrides are explicit for everyone to see.
### Using node_modules directly {#javascript-using-node_modules}
each tool has an abstraction to just build the node_modules (dependencies) directory. you can always use the stdenv.mkDerivation with the node_modules to build the package (symlink the node_modules directory and then use the package build command). the node_modules abstraction can be also used to build some web framework frontends. For an example of this see how [plausible](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/web-apps/plausible/default.nix) is built. mkYarnModules to make the derivation containing node_modules. Then when building the frontend you can just symlink the node_modules directory
The `pkgs/development/node-packages` folder contains a generated collection of
[NPM packages](https://npmjs.com/) that can be installed with the Nix package
manager.
As a rule of thumb, the package set should only provide _end user_ software
packages, such as command-line utilities. Libraries should only be added to the
package set if there is a non-NPM package that requires it.
When it is desired to use NPM libraries in a development project, use the
`node2nix` generator directly on the `package.json` configuration file of the
project.
The package set provides support for the official stable Node.js versions.
The latest stable LTS release in `nodePackages`, as well as the latest stable
Current release in `nodePackages_latest`.
If your package uses native addons, you need to examine what kind of native
build system it uses. Here are some examples:
- `node-gyp`
- `node-gyp-builder`
- `node-pre-gyp`
After you have identified the correct system, you need to override your package
expression while adding in build system as a build input. For example, `dat`
requires `node-gyp-build`, so [we override](https://github.com/NixOS/nixpkgs/blob/32f5e5da4a1b3f0595527f5195ac3a91451e9b56/pkgs/development/node-packages/default.nix#L37-L40) its expression in [`default.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix):
you will need to generate a nix expression for the dependencies
- don't forget the `-l package-lock.json` if there is a lock file
- Most probably you will need the `--development` to include the `devDependencies`
so the command will most likely be
`node2nix --development -l package-lock.json`
[link to the doc in the repo](https://github.com/svanderburg/node2nix)
#### Pitfalls {#javascript-node2nix-pitfalls}
- if upstream package.json does not have a "version" attribute, node2nix will crash. You will need to add it like shown in [the package.json section](#javascript-upstream-package-json)
- node2nix has some [bugs](https://github.com/svanderburg/node2nix/issues/238). related to working with lock files from npm distributed with nodejs-16_x
- node2nix does not like missing packages from npm. If you see something like `Cannot resolve version: vue-loader-v16@undefined` then you might want to try another tool. The package might have been pulled off of npm.
this will by default try to generate a binary. For package only generating static assets (Svelte, Vue, React...), you will need to explicitly override the build step with your instructions. It's important to use the `--offline` flag. For example if you script is `"build": "something"` in package.json use
```nix
buildPhase = ''
yarn build --offline
'';
```
The dist phase is also trying to build a binary, the only way to override it is with
```nix
distPhase = "true";
```
the configure phase can sometimes fail because it tries to be too clever.
this will generate a derivation including the node_modules. If you have to build a derivation for an integrated web framework (rails, phoenix..), this is probably the easiest way. [Plausible](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/web-apps/plausible/default.nix#L39) offers a good example of how to do this.
#### Overriding dependency behavior
In the `mkYarnPackage` record the property `pkgConfig` can be used to override packages when you encounter problems building.
For instance, say your package is throwing errors when trying to invoke node-sass: `ENOENT: no such file or directory, scandir '/build/source/node_modules/node-sass/vendor'`
To fix this we will specify different versions of build inputs to use, as well as some post install steps to get the software built the way we want:
```nix
mkYarnPackage rec {
pkgConfig = {
node-sass = {
buildInputs = with final;[ python libsass pkg-config ];
postInstall = ''
LIBSASS_EXT=auto yarn --offline run build
rm build/config.gypi
'';
};
};
}
```
#### Pitfalls {#javascript-yarn2nix-pitfalls}
- if version is missing from upstream package.json, yarn will silently install nothing. In that case, you will need to override package.json as shown in the [package.json section](#javascript-upstream-package-json)
- having trouble with node-gyp? Try adding these lines to the `yarnPreBuild` steps:
- The `echo 9` steps comes from this answer: <https://stackoverflow.com/a/49139496>
- Exporting the headers in `npm_config_nodedir` comes from this issue: <https://github.com/nodejs/node-gyp/issues/1191#issuecomment-301243919>
## Outside of nixpkgs {#javascript-outside-nixpkgs}
There are some other options available that can't be used inside nixpkgs. Those other options are written in nix. Importing them in nixpkgs will require moving the source code into nixpkgs. Using [Import From Derivation](https://nixos.wiki/wiki/Import_From_Derivation) is not allowed in hydra at present. If you are packaging something outside nixpkgs, those can be considered
### npmlock2nix {#javascript-npmlock2nix}
[npmlock2nix](https://github.com/nix-community/npmlock2nix) aims at building node_modules without code generation. It hasn't reached v1 yet, the api might be subject to change.
#### Pitfalls {#javascript-npmlock2nix-pitfalls}
- there are some [problems with npm v7](https://github.com/tweag/npmlock2nix/issues/45).
[nix-npm-buildpackage](https://github.com/serokell/nix-npm-buildpackage) aims at building node_modules without code generation. It hasn't reached v1 yet, the api might change. It supports both package-lock.json and yarn.lock.
@ -139,11 +139,9 @@ the whitelist maintainers/scripts/luarocks-packages.csv and updated by running m
[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock).
[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock).
The automation only goes so far though and some packages need to be customized.
The automation only goes so far though and some packages need to be customized.
These customizations go in `pkgs/development/lua-modules/overrides.nix`.
These customizations go in `pkgs/development/lua-modules/overrides.nix`.
For instance if the rockspec defines `external_dependencies`, these need to be manually added in its rockspec file then it won't work.
For instance if the rockspec defines `external_dependencies`, these need to be manually added to the overrides.nix.
You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`.
You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`.
Nix rely on luarocks to install lua packages, basically it runs:
`luarocks make --deps-mode=none --tree $out`
#### Packaging a library manually {#packaging-a-library-manually}
#### Packaging a library manually {#packaging-a-library-manually}
@ -161,8 +159,8 @@ are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua
### Lua interpreters {#lua-interpreters}
### Lua interpreters {#lua-interpreters}
Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as
Versions 5.1, 5.2, 5.3 and 5.4 of the lua interpreter are available as
respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too.
respectively `lua5_1`, `lua5_2`, `lua5_3` and `lua5_4`. Luajit is available too.
The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`.
The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`.
#### Attributes on lua interpreters packages {#attributes-on-lua-interpreters-packages}
#### Attributes on lua interpreters packages {#attributes-on-lua-interpreters-packages}
The `pkgs/development/node-packages` folder contains a generated collection of
[NPM packages](https://npmjs.com/) that can be installed with the Nix package
manager.
As a rule of thumb, the package set should only provide *end user* software
packages, such as command-line utilities. Libraries should only be added to the
package set if there is a non-NPM package that requires it.
When it is desired to use NPM libraries in a development project, use the
`node2nix` generator directly on the `package.json` configuration file of the
project.
The package set provides support for the official stable Node.js versions.
The latest stable LTS release in `nodePackages`, as well as the latest stable
Current release in `nodePackages_latest`.
If your package uses native addons, you need to examine what kind of native
build system it uses. Here are some examples:
* `node-gyp`
* `node-gyp-builder`
* `node-pre-gyp`
After you have identified the correct system, you need to override your package
expression while adding in build system as a build input. For example, `dat`
requires `node-gyp-build`, so [we override](https://github.com/NixOS/nixpkgs/blob/32f5e5da4a1b3f0595527f5195ac3a91451e9b56/pkgs/development/node-packages/default.nix#L37-L40) its expression in [`default.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix):
## Adding new plugins to nixpkgs {#adding-new-plugins-to-nixpkgs}
## Adding new plugins to nixpkgs {#adding-new-plugins-to-nixpkgs}
Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]@[gitref]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
@ -114,6 +114,10 @@ For details, see [Licenses](#sec-meta-license).
A list of the maintainers of this Nix expression. Maintainers are defined in [`nixpkgs/maintainers/maintainer-list.nix`](https://github.com/NixOS/nixpkgs/blob/master/maintainers/maintainer-list.nix). There is no restriction to becoming a maintainer, just add yourself to that list in a separate commit titled “maintainers: add alice”, and reference maintainers with `maintainers = with lib.maintainers; [ alice bob ]`.
A list of the maintainers of this Nix expression. Maintainers are defined in [`nixpkgs/maintainers/maintainer-list.nix`](https://github.com/NixOS/nixpkgs/blob/master/maintainers/maintainer-list.nix). There is no restriction to becoming a maintainer, just add yourself to that list in a separate commit titled “maintainers: add alice”, and reference maintainers with `maintainers = with lib.maintainers; [ alice bob ]`.
### `mainProgram` {#var-meta-mainProgram}
The name of the main binary for the package. This effects the binary `nix run` executes and falls back to the name of the package. Example: `"rg"`
### `priority` {#var-meta-priority}
### `priority` {#var-meta-priority}
The *priority* of the package, used by `nix-env` to resolve file name conflicts between packages. See the Nix manual page for `nix-env` for details. Example: `"10"` (a low-priority package).
The *priority* of the package, used by `nix-env` to resolve file name conflicts between packages. See the Nix manual page for `nix-env` for details. Example: `"10"` (a low-priority package).
@ -134,7 +138,28 @@ Attribute Set `lib.platforms` defines [various common lists](https://github.com/
This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations.
This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations.
:::
:::
An attribute set with as values tests. A test is a derivation, which builds successfully when the test passes, and fails to build otherwise. A derivation that is a test needs to have `meta.timeout` defined.
An attribute set with tests as values. A test is a derivation that builds when the test passes and fails to build otherwise.
You can run these tests with:
```ShellSession
$ cd path/to/nixpkgs
$ nix-build -A your-package.tests
```
#### Package tests
Tests that are part of the source package are often executed in the `installCheckPhase`.
Prefer `passthru.tests` for tests that are introduced in nixpkgs because:
* `passthru.tests` tests the 'real' package, independently from the environment in which it was built
* we can run `passthru.tests` independently
* `installCheckPhase` adds overhead to each build
For more on how to write and run package tests, see <xreflinkend="sec-package-tests"/>.
#### NixOS tests
The NixOS tests are available as `nixosTests` in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
The NixOS tests are available as `nixosTests` in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
@ -148,6 +173,8 @@ The NixOS tests are available as `nixosTests` in parameters of derivations. For
}
}
```
```
NixOS tests run in a VM, so they are slower than regular package tests. For more information see [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
### `timeout` {#var-meta-timeout}
### `timeout` {#var-meta-timeout}
A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, it can fail due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in `nixpkgs`.
A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, it can fail due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in `nixpkgs`.
@ -714,6 +714,8 @@ to `~/.gdbinit`. GDB will then be able to find debug information installed via `
The installCheck phase checks whether the package was installed correctly by running its test suite against the installed directories. The default `installCheck` calls `make installcheck`.
The installCheck phase checks whether the package was installed correctly by running its test suite against the installed directories. The default `installCheck` calls `make installcheck`.
It is often better to add tests that are not part of the source distribution to `passthru.tests` (see <xreflinkend="var-meta-tests"/>). This avoids adding overhead to every build and enables us to run them independently.
#### Variables controlling the installCheck phase {#variables-controlling-the-installcheck-phase}
#### Variables controlling the installCheck phase {#variables-controlling-the-installcheck-phase}
mkOptionDefault=mkOverride1500;# priority of option defaults
mkOptionDefault=mkOverride1500;# priority of option defaults
mkDefault=mkOverride1000;# used in config sections of non-user modules to set a default
mkDefault=mkOverride1000;# used in config sections of non-user modules to set a default
mkImageMediaOverride=mkOverride60;# image media profiles can be derived by inclusion into host config, hence needing to override host config, but do allow user to mkForce
mkForce=mkOverride50;
mkForce=mkOverride50;
mkVMOverride=mkOverride10;# used by ‘nixos-rebuild build-vm’
mkVMOverride=mkOverride10;# used by ‘nixos-rebuild build-vm’
# On Hackage every package description shows a category "Distributions" which
# lists a "NixOS" version.
# This script uploads a csv to hackage which will update the displayed versions
# based on the current versions in nixpkgs. This happens with a simple http
# request.
# For authorization you just need to have any valid hackage account. This
# script uses the `username` and `password-command` field from your
# ~/.cabal/config file.
# e.g. username: maralorn
# password-command: pass hackage.haskell.org (this can be any command, but not an arbitrary shell expression. Like cabal we only read the first output line and ignore the rest.)
# Those fields are specified under `upload` on the `cabal` man page.
package_list="$(nix-build -A haskell.package-list)/nixos-hackage-packages.csv"
username=$(grep "^username:" ~/.cabal/config | sed "s/^username: //")
password_command=$(grep "^password-command:" ~/.cabal/config | sed "s/^password-command: //")
# Building a NixOS (Live) ISO {#sec-building-image}
Building a NixOS CD is as easy as configuring your own computer. The idea is to use another module which will replace your `configuration.nix` to configure the system that would be installed on the CD.
Default CD/DVD configurations are available inside `nixos/modules/installer/cd-dvd`
Default live installer configurations are available inside `nixos/modules/installer/cd-dvd`.
For building other system images, [nixos-generators] is a good place to start looking at.
You have two options:
- Use any of those default configurations as is
- Combine them with (any of) your host config(s)
System images, such as the live installer ones, know how to enforce configuration settings
on wich they immediately depend in order to work correctly.
However, if you are confident, you can opt to override those
$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix
$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix
```
```
Before burning your CD/DVD, you can check the content of the image by mounting anywhere like suggested by the following command:
To check the content of an ISO image, mount it like so:
```ShellSession
```ShellSession
# mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen>
# mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso
```
```
If you want to customize your NixOS CD in more detail, or generate other kinds of images, you might want to check out [nixos-generators](https://github.com/nix-community/nixos-generators). This can also be a good starting point when you want to use Nix to build a 'minimal' image that doesn't include a NixOS installation.
@ -30,7 +30,7 @@ In addition to numerous new and upgraded packages, this release has the followin
- Python optimizations were disabled again. Builds with optimizations enabled are not reproducible. Optimizations can now be enabled with an option.
- Python optimizations were disabled again. Builds with optimizations enabled are not reproducible. Optimizations can now be enabled with an option.
- The linux_latest kernel was updated to the 5.12 series. It currently is not officially supported for use with the zfs filesystem. If you use zfs, you should use a different kernel version (either the LTS kernel, or track a specific one).
- The linux_latest kernel was updated to the 5.13 series. It currently is not officially supported for use with the zfs filesystem. If you use zfs, you should use a different kernel version (either the LTS kernel, or track a specific one).
@ -11,6 +11,8 @@ In addition to numerous new and upgraded packages, this release has the followin
- `python3` now defaults to Python 3.9, updated from Python 3.8.
- `python3` now defaults to Python 3.9, updated from Python 3.8.
- PostgreSQL now defaults to major version 13.
## New Services {#sec-release-21.11-new-services}
## New Services {#sec-release-21.11-new-services}
- [btrbk](https://digint.ch/btrbk/index.html), a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations. Available as [services.btrbk](options.html#opt-services.brtbk.instances).
- [btrbk](https://digint.ch/btrbk/index.html), a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations. Available as [services.btrbk](options.html#opt-services.brtbk.instances).
@ -37,9 +39,86 @@ pt-services.clipcat.enable).
- [buildkite-agent-metrics](https://github.com/buildkite/buildkite-agent-metrics), a command-line tool for collecting Buildkite agent metrics, now has a Prometheus exporter available as [services.prometheus.exporters.buildkite-agent](#opt-services.prometheus.exporters.buildkite-agent.enable).
- [buildkite-agent-metrics](https://github.com/buildkite/buildkite-agent-metrics), a command-line tool for collecting Buildkite agent metrics, now has a Prometheus exporter available as [services.prometheus.exporters.buildkite-agent](#opt-services.prometheus.exporters.buildkite-agent.enable).
- [influxdb-exporter](https://github.com/prometheus/influxdb_exporter) a Prometheus exporter that exports metrics received on an InfluxDB compatible endpoint is now available as [services.prometheus.exporters.influxdb](#opt-services.prometheus.exporters.influxdb.enable).
- [mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord), a discord puppeting bridge for matrix. Available as [services.mx-puppet-discord](#opt-services.mx-puppet-discord.enable).
- [MeshCentral](https://www.meshcommander.com/meshcentral2/overview), a remote administration service ("TeamViewer but self-hosted and with more features") is now available with a package and a module: [services.meshcentral.enable](#opt-services.meshcentral.enable)
- [moonraker](https://github.com/Arksine/moonraker), an API web server for Klipper.
Available as [moonraker](#opt-services.moonraker.enable).
- [influxdb2](https://github.com/influxdata/influxdb), a Scalable datastore for metrics, events, and real-time analytics. Available as [services.influxdb2](#opt-services.influxdb2.enable).
- [isso](https://posativ.org/isso/), a commenting server similar to Disqus.
Available as [isso](#opt-services.isso.enable)
* [navidrome](https://www.navidrome.org/), a personal music streaming server with
subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable).
- [fluidd](https://docs.fluidd.xyz/), a Klipper web interface for managing 3d printers using moonraker. Available as [fluidd](#opt-services.fluidd.enable).
- [sx](https://github.com/earnestly/sx), a simple alternative to both xinit and startx for starting a Xorg server. Available as [services.xserver.displayManager.sx](#opt-services.xserver.displayManager.sx.enable)
- [postfixadmin](https://postfixadmin.sourceforge.io/), a web based virtual user administration interface for Postfix mail servers. Available as [postfixadmin](#opt-services.postfixadmin.enable).
and advises all users to use `paperless-ng` instead.
Users can use the `services.paperless-ng` module as a replacement while noting the following incompatibilities:
- `services.paperless.ocrLanguages` has no replacement. Users should migrate to [`services.paperless-ng.extraConfig`](options.html#opt-services.paperless-ng.extraConfig) instead:
PAPERLESS_OCR_LANGUAGE = "deu+eng+jpn"; # German & English & Japanse
};
}
```
- If you previously specified `PAPERLESS_CONSUME_MAIL_*` settings in
`services.paperless.extraConfig` you should remove those options now. You
now *must* define those settings in the admin interface of paperless-ng.
- Option `services.paperless.manage` no longer exists.
Use the script at `${services.paperless-ng.dataDir}/paperless-ng-manage` instead.
Note that this script only exists after the `paperless-ng` service has been
started at least once.
- After switching to the new system configuration you should run the Django
management command to reindex your documents and optionally create a user,
if you don't have one already.
To do so, enter the data directory (the value of
`services.paperless-ng.dataDir`, `/var/lib/paperless` by default), switch
to the paperless user and execute the management command like below:
```
$ cd /var/lib/paperless
$ su paperless -s /bin/sh
$ ./paperless-ng-manage document_index reindex
# if not already done create a user account, paperless-ng requires a login
$ ./paperless-ng-manage createsuperuser
Username (leave blank to use 'paperless'): my-user-name
Email address: me@example.com
Password: **********
Password (again): **********
Superuser created successfully.
```
- The `staticjinja` package has been upgraded from 1.0.4 to 4.1.0
- Firefox v91 does not support addons with invalid signature anymore. Firefox ESR needs to be used for nix addon support.
- The `erigon` ethereum node has moved to a new database format in `2021-05-04`, and requires a full resync
- The `erigon` ethereum node has moved it's database location in `2021-08-03`, users upgrading must manually move their chaindata (see [release notes](https://github.com/ledgerwatch/erigon/releases/tag/v2021.08.03)).
- `services.geoip-updater` was broken and has been replaced by [services.geoipupdate](options.html#opt-services.geoipupdate.enable).
- `services.geoip-updater` was broken and has been replaced by [services.geoipupdate](options.html#opt-services.geoipupdate.enable).
@ -140,6 +219,47 @@ pt-services.clipcat.enable).
- the `mingw-64` package has been upgraded from 6.0.0 to 9.0.0
- the `mingw-64` package has been upgraded from 6.0.0 to 9.0.0
- `tt-rss` was upgraded to the commit on 2021-06-21, which has breaking changes. If you use `services.tt-rss.extraConfig` you should migrate to the `putenv`-style configuration. See [this Discourse post](https://community.tt-rss.org/t/rip-config-php-hello-classes-config-php/4337) in the tt-rss forums for more details.
- The following Visual Studio Code extensions were renamed to keep the naming convention uniform.
- `services.uptimed` now uses `/var/lib/uptimed` as its stateDirectory instead of `/var/spool/uptimed`. Make sure to move all files to the new directory.
- Deprecated package aliases in `emacs.pkgs.*` have been removed. These aliases were remnants of the old Emacs package infrastructure. We now use exact upstream names wherever possible.
- `programs.neovim.runtime` switched to a `linkFarm` internally, making it impossible to use wildcards in the `source` argument.
- The `openrazer` and `openrazer-daemon` packages as well as the `hardware.openrazer` module now require users to be members of the `openrazer` group instead of `plugdev`. With this change, users no longer need be granted the entire set of `plugdev` group permissions, which can include permissions other than those required by `openrazer`. This is desirable from a security point of view. The setting [`harware.openrazer.users`](options.html#opt-services.hardware.openrazer.users) can be used to add users to the `openrazer` group.
- The fontconfig service's dpi option has been removed.
Fontconfig should use Xft settings by default so there's no need to override one value in multiple places.
The user can set DPI via ~/.Xresources properly, or at the system level per monitor, or as a last resort at the system level with `services.xserver.dpi`.
- The `yambar` package has been split into `yambar` and `yambar-wayland`, corresponding to the xorg and wayland backend respectively. Please switch to `yambar-wayland` if you are on wayland.
- The `services.minio` module gained an additional option `consoleAddress`, that
configures the address and port the web UI is listening, it defaults to `:9001`.
To be able to access the web UI this port needs to be opened in the firewall.
- The `varnish` package was upgraded from 6.3.x to 6.5.x. `varnish60` for the last LTS release is also still available.
- The `kubernetes` package was upgraded to 1.22. The `kubernetes.apiserver.kubeletHttps` option was removed and HTTPS is always used.
- The attribute `linuxPackages_latest_hardened` was dropped because the hardened patches
lag behind the upstream kernel which made version bumps harder. If you want to use
a hardened kernel, please pin it explicitly with a versioned attribute such as
`linuxPackages_5_10_hardened`.
- The `nomad` package now defaults to a 1.1.x release instead of 1.0.x
- If `exfat` is included in `boot.supportedFilesystems` and when using kernel 5.7
or later, the `exfatprogs` user-space utilities are used instead of `exfat`.
- The `todoman` package was upgraded from 3.9.0 to 4.0.0. This introduces breaking changes in the [configuration file](https://todoman.readthedocs.io/en/stable/configure.html#configuration-file) format.
## Other Notable Changes {#sec-release-21.11-notable-changes}
## Other Notable Changes {#sec-release-21.11-notable-changes}
- The setting [`services.openssh.logLevel`](options.html#opt-services.openssh.logLevel) `"VERBOSE"``"INFO"`. This brings NixOS in line with upstream and other Linux distributions, and reduces log spam on servers due to bruteforcing botnets.
- The setting [`services.openssh.logLevel`](options.html#opt-services.openssh.logLevel) `"VERBOSE"``"INFO"`. This brings NixOS in line with upstream and other Linux distributions, and reduces log spam on servers due to bruteforcing botnets.
@ -150,6 +270,8 @@ pt-services.clipcat.enable).
- `python3` now defaults to Python 3.9. Python 3.9 introduces many deprecation warnings, please look at the [What's New In Python 3.9 post](https://docs.python.org/3/whatsnew/3.9.html) for more information.
- `python3` now defaults to Python 3.9. Python 3.9 introduces many deprecation warnings, please look at the [What's New In Python 3.9 post](https://docs.python.org/3/whatsnew/3.9.html) for more information.
- `qtile` hase been updated from '0.16.0' to '0.18.0', please check [qtile changelog](https://github.com/qtile/qtile/blob/master/CHANGELOG) for changes.
- The `claws-mail` package now references the new GTK+ 3 release branch, major version 4. To use the GTK+ 2 releases, one can install the `claws-mail-gtk2` package.
- The `claws-mail` package now references the new GTK+ 3 release branch, major version 4. To use the GTK+ 2 releases, one can install the `claws-mail-gtk2` package.
- The wordpress module provides a new interface which allows to use different webservers with the new option [`services.wordpress.webserver`](options.html#opt-services.wordpress.webserver). Currently `httpd` and `nginx` are supported. The definitions of wordpress sites should now be set in [`services.wordpress.sites`](options.html#opt-services.wordpress.sites).
- The wordpress module provides a new interface which allows to use different webservers with the new option [`services.wordpress.webserver`](options.html#opt-services.wordpress.webserver). Currently `httpd` and `nginx` are supported. The definitions of wordpress sites should now be set in [`services.wordpress.sites`](options.html#opt-services.wordpress.sites).
@ -179,3 +301,13 @@ pt-services.clipcat.enable).
- NSS modules which should be queried after `resolved`, `files` and
- NSS modules which should be queried after `resolved`, `files` and
`myhostname`, but before `dns` should use the default priority
`myhostname`, but before `dns` should use the default priority
- NSS modules which should come after `dns` should use mkAfter.
- NSS modules which should come after `dns` should use mkAfter.
- The [networking.wireless.iwd](options.html#opt-networking.wireless.iwd.enable) module has a new [networking.wireless.iwd.settings](options.html#opt-networking.wireless.iwd.settings) option.
- The [services.syncoid.enable](options.html#opt-services.syncoid.enable) module now properly drops ZFS permissions after usage. Before it delegated permissions to whole pools instead of datasets and didn't clean up after execution. You can manually look this up for your pools by running `zfs allow your-pool-name` and use `zfs unallow syncoid your-pool-name` to clean this up.
- Zfs: `latestCompatibleLinuxPackages` is now exported on the zfs package. One can use `boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;` to always track the latest compatible kernel with a given version of zfs.
- Nginx will use the value of `sslTrustedCertificate` if provided for a virtual host, even if `enableACME` is set. This is useful for providers not using the same certificate to sign OCSP responses and server certificates.
- `lib.formats.yaml`'s `generate` will not generate JSON anymore, but instead use more of the YAML-specific syntax.