nixos/doc: Document the activation script

This may be helpful to new module developers, curious users, and people
who just need a reference without having to look at the implementation

nixos/doc/manual/development/activation-script.section.md

@@ -0,0 +1,72 @@
+# Activation script {#sec-activation-script}
+
+The activation script is a bash script called to activate the new
+configuration which resides in a NixOS system in `$out/activate`. Since its
+contents depend on your system configuration, the contents may differ.
+This chapter explains how the script works in general and some common NixOS
+snippets. Please be aware that the script is executed on every boot and system
+switch, so tasks that can be performed in other places should be performed
+there (for example letting a directory of a service be created by systemd using
+mechanisms like `StateDirectory`, `CacheDirectory`, ... or if that's not
+possible using `preStart` of the service).
+
+Activation scripts are defined as snippets using
+[](#opt-system.activationScripts). They can either be a simple multiline string
+or an attribute set that can depend on other snippets. The builder for the
+activation script will take these dependencies into account and order the
+snippets accordingly. As a simple example:
+
+```nix
+system.activationScripts.my-activation-script = {
+  deps = [ "etc" ];
+  # supportsDryActivation = true;
+  text = ''
+    echo "Hallo i bims"
+  '';
+};
+```
+
+This example creates an activation script snippet that is run after the `etc`
+snippet. The special variable `supportsDryActivation` can be set so the snippet
+is also run when `nixos-rebuild dry-activate` is run. To differentiate between
+real and dry activation, the `$NIXOS_ACTION` environment variable can be
+read which is set to `dry-activate` when a dry activation is done.
+
+An activation script can write to special files instructing
+`switch-to-configuration` to restart/reload units. The script will take these
+requests into account and will incorperate the unit configuration as described
+above. This means that the activation script will "fake" a modified unit file
+and `switch-to-configuration` will act accordingly. By doing so, configuration
+like [systemd.services.\<name\>.restartIfChanged](#opt-systemd.services) is
+respected. Since the activation script is run **after** services are already
+stopped, [systemd.services.\<name\>.stopIfChanged](#opt-systemd.services)
+cannot be taken into account anymore and the unit is always restarted instead
+of being stopped and started afterwards.
+
+The files that can be written to are `/run/nixos/activation-restart-list` and
+`/run/nixos/activation-reload-list` with their respective counterparts for
+dry activation being `/run/nixos/dry-activation-restart-list` and
+`/run/nixos/dry-activation-reload-list`. Those files can contain
+newline-separated lists of unit names where duplicates are being ignored. These
+files are not create automatically and activation scripts must take the
+possiblility into account that they have to create them first.
+
+## NixOS snippets {#sec-activation-script-nixos-snippets}
+
+There are some snippets NixOS enables by default because disabling them would
+most likely break you system. This section lists a few of them and what they
+do:
+
+- `binsh` creates `/bin/sh` which points to the runtime shell
+- `etc` sets up the contents of `/etc`, this includes systemd units and
+  excludes `/etc/passwd`, `/etc/group`, and `/etc/shadow` (which are managed by
+  the `users` snippet)
+- `hostname` sets the system's hostname in the kernel (not in `/etc`)
+- `modprobe` sets the path to the `modprobe` binary for module auto-loading
+- `nix` prepares the nix store and adds a default initial channel
+- `specialfs` is responsible for mounting filesystems like `/proc` and `sys`
+- `users` creates and removes users and groups by managing `/etc/passwd`,
+  `/etc/group` and `/etc/shadow`. This also creates home directories
+- `usrbinenv` creates `/usr/bin/env`
+- `var` creates some directories in `/var` that are not service-specific
+- `wrappers` creates setuid wrappers like `ping` and `sudo`

nixos/doc/manual/development/development.xml

@@ -12,6 +12,7 @@
  <xi:include href="../from_md/development/sources.chapter.xml" />
  <xi:include href="../from_md/development/writing-modules.chapter.xml" />
  <xi:include href="../from_md/development/building-parts.chapter.xml" />
+ <xi:include href="../from_md/development/what-happens-during-a-system-switch.chapter.xml" />
  <xi:include href="../from_md/development/writing-documentation.chapter.xml" />
  <xi:include href="../from_md/development/building-nixos.chapter.xml" />
  <xi:include href="../from_md/development/nixos-tests.chapter.xml" />

nixos/doc/manual/development/unit-handling.section.md

@@ -0,0 +1,57 @@
+# Unit handling {#sec-unit-handling}
+
+To figure out what units need to be started/stopped/restarted/reloaded, the
+script first checks the current state of the system, similar to what `systemctl
+list-units` shows. For each of the units, the script goes through the following
+checks:
+
+- Is the unit file still in the new system? If not, **stop** the service unless
+  it sets `X-StopOnRemoval` in the `[Unit]` section to `false`.
+
+- Is it a `.target` unit? If so, **start** it unless it sets
+  `RefuseManualStart` in the `[Unit]` section to `true` or `X-OnlyManualStart`
+  in the `[Unit]` section to `true`. Also **stop** the unit again unless it
+  sets `X-StopOnReconfiguration` to `false`.
+
+- Are the contents of the unit files different? They are compared by parsing
+  them and comparing their contents. If they are different but only
+  `X-Reload-Triggers` in the `[Unit]` section is changed, **reload** the unit.
+  The NixOS module system allows setting these triggers with the option
+  [systemd.services.\<name\>.reloadTriggers](#opt-systemd.services). If the
+  unit files differ in any way, the following actions are performed:
+
+  - `.path` and `.slice` units are ignored. There is no need to restart them
+    since changes in their values are applied by systemd when systemd is
+    reloaded.
+
+  - `.mount` units are **reload**ed. These mostly come from the `/etc/fstab`
+    parser.
+
+  - `.socket` units are currently ignored. This is to be fixed at a later
+    point.
+
+  - The rest of the units (mostly `.service` units) are then **reload**ed if
+    `X-ReloadIfChanged` in the `[Service]` section is set to `true` (exposed
+    via [systemd.services.\<name\>.reloadIfChanged](#opt-systemd.services)).
+
+  - If the reload flag is not set, some more flags decide if the unit is
+    skipped. These flags are `X-RestartIfChanged` in the `[Service]` section
+    (exposed via
+    [systemd.services.\<name\>.restartIfChanged](#opt-systemd.services)),
+    `RefuseManualStop` in the `[Unit]` section, and `X-OnlyManualStart` in the
+    `[Unit]` section.
+
+  - The rest of the behavior is decided whether the unit has `X-StopIfChanged`
+    in the `[Service]` section set (exposed via
+    [systemd.services.\<name\>.stopIfChanged](#opt-systemd.services)). This is
+    set to `true` by default and must be explicitly turned off if not wanted.
+    If the flag is enabled, the unit is **stop**ped and then **start**ed. If
+    not, the unit is **restart**ed. The goal of the flag is to make sure that
+    the new unit never runs in the old environment which is still in place
+    before the activation script is run.
+
+  - The last thing that is taken into account is whether the unit is a service
+    and socket-activated. Due to a bug, this is currently only done when
+    `X-StopIfChanged` is set. If the unit is socket-activated, the socket is
+    stopped and started, and the service is stopped and to be started by socket
+    activation.

nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md

@@ -0,0 +1,53 @@
+# What happens during a system switch? {#sec-switching-systems}
+
+Running `nixos-rebuild switch` is one of the more common tasks under NixOS.
+This chapter explains some of the internals of this command to make it simpler
+for new module developers to configure their units correctly and to make it
+easier to understand what is happening and why for curious administrators.
+
+`nixos-rebuild`, like many deployment solutions, calls `switch-to-configuration`
+which resides in a NixOS system at `$out/bin/switch-to-configuration`. The
+script is called with the action that is to be performed like `switch`, `test`,
+`boot`. There is also the `dry-activate` action which does not really perform
+the actions but rather prints what it would do if you called it with `test`.
+This feature can be used to check what service states would be changed if the
+configuration was switched to.
+
+If the action is `switch` or `boot`, the bootloader is updated first so the
+configuration will be the next one to boot. Unless `NIXOS_NO_SYNC` is set to
+`1`, `/nix/store` is synced to disk.
+
+If the action is `switch` or `test`, the currently running system is inspected
+and the actions to switch to the new system are calculated. This process takes
+two data sources into account: `/etc/fstab` and the current systemd status.
+Mounts and swaps are read from `/etc/fstab` and the corresponding actions are
+generated. If a new mount is added, for example, the proper `.mount` unit is
+marked to be started. The current systemd state is inspected, the difference
+between the current system and the desired configuration is calculated and
+actions are generated to get to this state. There are a lot of nuances that can
+be controlled by the units which are explained here.
+
+After calculating what should be done, the actions are carried out. The order
+of actions is always the same:
+- Stop units (`systemctl stop`)
+- Run activation script (`$out/activate`)
+- See if the activation script requested more units to restart
+- Restart systemd if needed (`systemd daemon-reexec`)
+- Forget about the failed state of units (`systemctl reset-failed`)
+- Reload systemd (`systemctl daemon-reload`)
+- Reload systemd user instances (`systemctl --user daemon-reload`)
+- Set up tmpfiles (`systemd-tmpfiles --create`)
+- Reload units (`systemctl reload`)
+- Restart units (`systemctl restart`)
+- Start units (`systemctl start`)
+- Inspect what changed during these actions and print units that failed and
+  that were newly started
+
+Most of these actions are either self-explaining but some of them have to do
+with our units or the activation script. For this reason, these topics are
+explained in the next sections.
+
+```{=docbook}
+<xi:include href="unit-handling.section.xml" />
+<xi:include href="activation-script.section.xml" />
+```

nixos/doc/manual/from_md/development/activation-script.section.xml

@@ -0,0 +1,150 @@
+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-activation-script">
+  <title>Activation script</title>
+  <para>
+    The activation script is a bash script called to activate the new
+    configuration which resides in a NixOS system in
+    <literal>$out/activate</literal>. Since its contents depend on your
+    system configuration, the contents may differ. This chapter explains
+    how the script works in general and some common NixOS snippets.
+    Please be aware that the script is executed on every boot and system
+    switch, so tasks that can be performed in other places should be
+    performed there (for example letting a directory of a service be
+    created by systemd using mechanisms like
+    <literal>StateDirectory</literal>,
+    <literal>CacheDirectory</literal>, … or if that’s not possible using
+    <literal>preStart</literal> of the service).
+  </para>
+  <para>
+    Activation scripts are defined as snippets using
+    <xref linkend="opt-system.activationScripts" />. They can either be
+    a simple multiline string or an attribute set that can depend on
+    other snippets. The builder for the activation script will take
+    these dependencies into account and order the snippets accordingly.
+    As a simple example:
+  </para>
+  <programlisting language="bash">
+system.activationScripts.my-activation-script = {
+  deps = [ &quot;etc&quot; ];
+  # supportsDryActivation = true;
+  text = ''
+    echo &quot;Hallo i bims&quot;
+  '';
+};
+</programlisting>
+  <para>
+    This example creates an activation script snippet that is run after
+    the <literal>etc</literal> snippet. The special variable
+    <literal>supportsDryActivation</literal> can be set so the snippet
+    is also run when <literal>nixos-rebuild dry-activate</literal> is
+    run. To differentiate between real and dry activation, the
+    <literal>$NIXOS_ACTION</literal> environment variable can be read
+    which is set to <literal>dry-activate</literal> when a dry
+    activation is done.
+  </para>
+  <para>
+    An activation script can write to special files instructing
+    <literal>switch-to-configuration</literal> to restart/reload units.
+    The script will take these requests into account and will
+    incorperate the unit configuration as described above. This means
+    that the activation script will <quote>fake</quote> a modified unit
+    file and <literal>switch-to-configuration</literal> will act
+    accordingly. By doing so, configuration like
+    <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.restartIfChanged</link>
+    is respected. Since the activation script is run
+    <emphasis role="strong">after</emphasis> services are already
+    stopped,
+    <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.stopIfChanged</link>
+    cannot be taken into account anymore and the unit is always
+    restarted instead of being stopped and started afterwards.
+  </para>
+  <para>
+    The files that can be written to are
+    <literal>/run/nixos/activation-restart-list</literal> and
+    <literal>/run/nixos/activation-reload-list</literal> with their
+    respective counterparts for dry activation being
+    <literal>/run/nixos/dry-activation-restart-list</literal> and
+    <literal>/run/nixos/dry-activation-reload-list</literal>. Those
+    files can contain newline-separated lists of unit names where
+    duplicates are being ignored. These files are not create
+    automatically and activation scripts must take the possiblility into
+    account that they have to create them first.
+  </para>
+  <section xml:id="sec-activation-script-nixos-snippets">
+    <title>NixOS snippets</title>
+    <para>
+      There are some snippets NixOS enables by default because disabling
+      them would most likely break you system. This section lists a few
+      of them and what they do:
+    </para>
+    <itemizedlist spacing="compact">
+      <listitem>
+        <para>
+          <literal>binsh</literal> creates <literal>/bin/sh</literal>
+          which points to the runtime shell
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>etc</literal> sets up the contents of
+          <literal>/etc</literal>, this includes systemd units and
+          excludes <literal>/etc/passwd</literal>,
+          <literal>/etc/group</literal>, and
+          <literal>/etc/shadow</literal> (which are managed by the
+          <literal>users</literal> snippet)
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>hostname</literal> sets the system’s hostname in the
+          kernel (not in <literal>/etc</literal>)
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>modprobe</literal> sets the path to the
+          <literal>modprobe</literal> binary for module auto-loading
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>nix</literal> prepares the nix store and adds a
+          default initial channel
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>specialfs</literal> is responsible for mounting
+          filesystems like <literal>/proc</literal> and
+          <literal>sys</literal>
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>users</literal> creates and removes users and groups
+          by managing <literal>/etc/passwd</literal>,
+          <literal>/etc/group</literal> and
+          <literal>/etc/shadow</literal>. This also creates home
+          directories
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>usrbinenv</literal> creates
+          <literal>/usr/bin/env</literal>
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>var</literal> creates some directories in
+          <literal>/var</literal> that are not service-specific
+        </para>
+      </listitem>
+      <listitem>
+        <para>
+          <literal>wrappers</literal> creates setuid wrappers like
+          <literal>ping</literal> and <literal>sudo</literal>
+        </para>
+      </listitem>
+    </itemizedlist>
+  </section>
+</section>

nixos/doc/manual/from_md/development/unit-handling.section.xml

@@ -0,0 +1,119 @@
+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-unit-handling">
+  <title>Unit handling</title>
+  <para>
+    To figure out what units need to be
+    started/stopped/restarted/reloaded, the script first checks the
+    current state of the system, similar to what
+    <literal>systemctl list-units</literal> shows. For each of the
+    units, the script goes through the following checks:
+  </para>
+  <itemizedlist>
+    <listitem>
+      <para>
+        Is the unit file still in the new system? If not,
+        <emphasis role="strong">stop</emphasis> the service unless it
+        sets <literal>X-StopOnRemoval</literal> in the
+        <literal>[Unit]</literal> section to <literal>false</literal>.
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Is it a <literal>.target</literal> unit? If so,
+        <emphasis role="strong">start</emphasis> it unless it sets
+        <literal>RefuseManualStart</literal> in the
+        <literal>[Unit]</literal> section to <literal>true</literal> or
+        <literal>X-OnlyManualStart</literal> in the
+        <literal>[Unit]</literal> section to <literal>true</literal>.
+        Also <emphasis role="strong">stop</emphasis> the unit again
+        unless it sets <literal>X-StopOnReconfiguration</literal> to
+        <literal>false</literal>.
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Are the contents of the unit files different? They are compared
+        by parsing them and comparing their contents. If they are
+        different but only <literal>X-Reload-Triggers</literal> in the
+        <literal>[Unit]</literal> section is changed,
+        <emphasis role="strong">reload</emphasis> the unit. The NixOS
+        module system allows setting these triggers with the option
+        <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.reloadTriggers</link>.
+        If the unit files differ in any way, the following actions are
+        performed:
+      </para>
+      <itemizedlist>
+        <listitem>
+          <para>
+            <literal>.path</literal> and <literal>.slice</literal> units
+            are ignored. There is no need to restart them since changes
+            in their values are applied by systemd when systemd is
+            reloaded.
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            <literal>.mount</literal> units are
+            <emphasis role="strong">reload</emphasis>ed. These mostly
+            come from the <literal>/etc/fstab</literal> parser.
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            <literal>.socket</literal> units are currently ignored. This
+            is to be fixed at a later point.
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            The rest of the units (mostly <literal>.service</literal>
+            units) are then <emphasis role="strong">reload</emphasis>ed
+            if <literal>X-ReloadIfChanged</literal> in the
+            <literal>[Service]</literal> section is set to
+            <literal>true</literal> (exposed via
+            <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.reloadIfChanged</link>).
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            If the reload flag is not set, some more flags decide if the
+            unit is skipped. These flags are
+            <literal>X-RestartIfChanged</literal> in the
+            <literal>[Service]</literal> section (exposed via
+            <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.restartIfChanged</link>),
+            <literal>RefuseManualStop</literal> in the
+            <literal>[Unit]</literal> section, and
+            <literal>X-OnlyManualStart</literal> in the
+            <literal>[Unit]</literal> section.
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            The rest of the behavior is decided whether the unit has
+            <literal>X-StopIfChanged</literal> in the
+            <literal>[Service]</literal> section set (exposed via
+            <link linkend="opt-systemd.services">systemd.services.&lt;name&gt;.stopIfChanged</link>).
+            This is set to <literal>true</literal> by default and must
+            be explicitly turned off if not wanted. If the flag is
+            enabled, the unit is
+            <emphasis role="strong">stop</emphasis>ped and then
+            <emphasis role="strong">start</emphasis>ed. If not, the unit
+            is <emphasis role="strong">restart</emphasis>ed. The goal of
+            the flag is to make sure that the new unit never runs in the
+            old environment which is still in place before the
+            activation script is run.
+          </para>
+        </listitem>
+        <listitem>
+          <para>
+            The last thing that is taken into account is whether the
+            unit is a service and socket-activated. Due to a bug, this
+            is currently only done when
+            <literal>X-StopIfChanged</literal> is set. If the unit is
+            socket-activated, the socket is stopped and started, and the
+            service is stopped and to be started by socket activation.
+          </para>
+        </listitem>
+      </itemizedlist>
+    </listitem>
+  </itemizedlist>
+</section>

nixos/doc/manual/from_md/development/what-happens-during-a-system-switch.chapter.xml

@@ -0,0 +1,122 @@
+<chapter xmlns="http://docbook.org/ns/docbook"  xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="sec-switching-systems">
+  <title>What happens during a system switch?</title>
+  <para>
+    Running <literal>nixos-rebuild switch</literal> is one of the more
+    common tasks under NixOS. This chapter explains some of the
+    internals of this command to make it simpler for new module
+    developers to configure their units correctly and to make it easier
+    to understand what is happening and why for curious administrators.
+  </para>
+  <para>
+    <literal>nixos-rebuild</literal>, like many deployment solutions,
+    calls <literal>switch-to-configuration</literal> which resides in a
+    NixOS system at <literal>$out/bin/switch-to-configuration</literal>.
+    The script is called with the action that is to be performed like
+    <literal>switch</literal>, <literal>test</literal>,
+    <literal>boot</literal>. There is also the
+    <literal>dry-activate</literal> action which does not really perform
+    the actions but rather prints what it would do if you called it with
+    <literal>test</literal>. This feature can be used to check what
+    service states would be changed if the configuration was switched
+    to.
+  </para>
+  <para>
+    If the action is <literal>switch</literal> or
+    <literal>boot</literal>, the bootloader is updated first so the
+    configuration will be the next one to boot. Unless
+    <literal>NIXOS_NO_SYNC</literal> is set to <literal>1</literal>,
+    <literal>/nix/store</literal> is synced to disk.
+  </para>
+  <para>
+    If the action is <literal>switch</literal> or
+    <literal>test</literal>, the currently running system is inspected
+    and the actions to switch to the new system are calculated. This
+    process takes two data sources into account:
+    <literal>/etc/fstab</literal> and the current systemd status. Mounts
+    and swaps are read from <literal>/etc/fstab</literal> and the
+    corresponding actions are generated. If a new mount is added, for
+    example, the proper <literal>.mount</literal> unit is marked to be
+    started. The current systemd state is inspected, the difference
+    between the current system and the desired configuration is
+    calculated and actions are generated to get to this state. There are
+    a lot of nuances that can be controlled by the units which are
+    explained here.
+  </para>
+  <para>
+    After calculating what should be done, the actions are carried out.
+    The order of actions is always the same:
+  </para>
+  <itemizedlist spacing="compact">
+    <listitem>
+      <para>
+        Stop units (<literal>systemctl stop</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Run activation script (<literal>$out/activate</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        See if the activation script requested more units to restart
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Restart systemd if needed
+        (<literal>systemd daemon-reexec</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Forget about the failed state of units
+        (<literal>systemctl reset-failed</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Reload systemd (<literal>systemctl daemon-reload</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Reload systemd user instances
+        (<literal>systemctl --user daemon-reload</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Set up tmpfiles (<literal>systemd-tmpfiles --create</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Reload units (<literal>systemctl reload</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Restart units (<literal>systemctl restart</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Start units (<literal>systemctl start</literal>)
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+        Inspect what changed during these actions and print units that
+        failed and that were newly started
+      </para>
+    </listitem>
+  </itemizedlist>
+  <para>
+    Most of these actions are either self-explaining but some of them
+    have to do with our units or the activation script. For this reason,
+    these topics are explained in the next sections.
+  </para>
+  <xi:include href="unit-handling.section.xml" />
+  <xi:include href="activation-script.section.xml" />
+</chapter>