nixosTest: Document stdout waiting behavior

nixos/doc/manual/development/writing-nixos-tests.section.md

@@ -159,6 +159,10 @@ The following methods are available on machine objects:
 `execute`
 
 :   Execute a shell command, returning a list `(status, stdout)`.
+    If the command detaches, it must close stdout, as `execute` will wait
+    for this to consume all output reliably. This can be achieved by
+    redirecting stdout to stderr `>&2`, to `/dev/console`, `/dev/null` or
+    a file.
     Takes an optional parameter `check_return` that defaults to `True`.
     Setting this parameter to `False` will not check for the return code
     and return -1 instead. This can be used for commands that shut down
@@ -179,6 +183,8 @@ The following methods are available on machine objects:
 
     -   Dereferencing unset variables fail the command.
 
+    -   It will wait for stdout to be closed. See `execute`.
+
 `fail`
 
 :   Like `succeed`, but raising an exception if the command returns a zero

nixos/doc/manual/from_md/development/writing-nixos-tests.section.xml

@@ -266,7 +266,12 @@ start_all()
       <listitem>
         <para>
           Execute a shell command, returning a list
-          <literal>(status, stdout)</literal>. Takes an optional
+          <literal>(status, stdout)</literal>. If the command detaches,
+          it must close stdout, as <literal>execute</literal> will wait
+          for this to consume all output reliably. This can be achieved
+          by redirecting stdout to stderr <literal>&gt;&amp;2</literal>,
+          to <literal>/dev/console</literal>,
+          <literal>/dev/null</literal> or a file. Takes an optional
           parameter <literal>check_return</literal> that defaults to
           <literal>True</literal>. Setting this parameter to
           <literal>False</literal> will not check for the return code
@@ -306,6 +311,12 @@ start_all()
               Dereferencing unset variables fail the command.
             </para>
           </listitem>
+          <listitem>
+            <para>
+              It will wait for stdout to be closed. See
+              <literal>execute</literal>.
+            </para>
+          </listitem>
         </itemizedlist>
       </listitem>
     </varlistentry>

nixos/doc/manual/from_md/release-notes/rl-2111.section.xml

@@ -423,6 +423,23 @@
   <section xml:id="sec-release-21.11-incompatibilities">
     <title>Backward Incompatibilities</title>
     <itemizedlist>
+      <listitem>
+        <para>
+          The NixOS VM test framework,
+          <literal>pkgs.nixosTest</literal>/<literal>make-test-python.nix</literal>,
+          now requires non-terminating commands such as
+          <literal>succeed(&quot;foo &amp;&quot;)</literal> to close
+          stdout. This can be done with a redirect such as
+          <literal>succeed(&quot;foo &gt;&amp;2 &amp;&quot;)</literal>.
+          This breaking change was necessitated by a race condition
+          causing tests to fail or hang. It applies to all methods that
+          invoke commands on the nodes, including
+          <literal>execute</literal>, <literal>succeed</literal>,
+          <literal>fail</literal>,
+          <literal>wait_until_succeeds</literal>,
+          <literal>wait_until_fails</literal>.
+        </para>
+      </listitem>
       <listitem>
         <para>
           The <literal>services.wakeonlan</literal> option was removed,

nixos/doc/manual/release-notes/rl-2111.section.md

@@ -128,6 +128,10 @@ In addition to numerous new and upgraded packages, this release has the followin
 
 ## Backward Incompatibilities {#sec-release-21.11-incompatibilities}
 
+- The NixOS VM test framework, `pkgs.nixosTest`/`make-test-python.nix`, now requires non-terminating commands such as `succeed("foo &")` to close stdout.
+  This can be done with a redirect such as `succeed("foo >&2 &")`. This breaking change was necessitated by a race condition causing tests to fail or hang.
+  It applies to all methods that invoke commands on the nodes, including `execute`, `succeed`, `fail`, `wait_until_succeeds`, `wait_until_fails`.
+
 - The `services.wakeonlan` option was removed, and replaced with `networking.interfaces.<name>.wakeOnLan`.
 
 - The `security.wrappers` option now requires to always specify an owner, group and whether the setuid/setgid bit should be set.