home-assistant: rerun tests failing with RuntimeError

The flakyness in home-assistant stems from premature event loop closing,
due to race conditions throughout the test suite.

> RuntimeError: Event loop is closed

They also happen during upstreams testsuite runs, but they silently
ignore them, so I don't expect a fix for them in the near future.

Alleviate our pains by rerunning tests that fail in this particular way.
wip/yesman
Martin Weinelt 3 years ago
parent 4e65797018
commit 598bc98ba1
No known key found for this signature in database
GPG Key ID: 87C1E9888F856759
  1. 6
      pkgs/servers/home-assistant/default.nix

@ -136,6 +136,7 @@ in with py.pkgs; buildPythonApplication rec {
# test infrastructure
asynctest
pytest-aiohttp
pytest-rerunfailures
pytest-xdist
pytestCheckHook
requests-mock
@ -292,7 +293,10 @@ in with py.pkgs; buildPythonApplication rec {
pytestFlagsArray = [
# limit amout of runners to reduce race conditions
"-n 2"
"-n auto"
# retry racy tests that end in "RuntimeError: Event loop is closed"
"--reruns 3"
"--only-rerun RuntimeError"
# assign tests grouped by file to workers
"--dist loadfile"
# tests are located in tests/

Loading…
Cancel
Save