Resolve "end-to-end tests: fixtures for starting redis and BEC services"
Description
"no-tmux" option for starting servers
Introduce a new way of starting bec servers with bec-server start --no-tmux
: just uses subprocess
to
start servers. Stopping is not implemented in this MR (since it would require to keep track of PIDs in
pid files or whatever).
Fixtures for end-2-end tests
Add new fixtures for tests, the code is in bec_client/test_fixtures.py
but it is made available as
a pytest plugin for use in other projects (thinking of ophyd devices or bec widgets).
The new fixtures make use of pytest-redis
to start Redis and rely on the new subprocess_start
method of ServiceHandler
(cf. above) to start services.
The more useful ones are:
-
bec_servers
: a fixture, that ensures Redis and all BEC servers are ready before the test starts- the scope of the fixture changes dynamically, from
session
tofunction
depending if--flush-redis
option is given or not - if
--bec-redis-host
is provided, indicates which host is running the Redis instance- this is mainly useful when Redis is running within Docker on a different network for example, for the CI
- if
--start-servers
command line argument is specified, servers are started automatically- if
--flush-redis
command line argument is provided, Redis DB is flushed after each test- as a consequence, BEC servers are also restarted at each test
-
--bec-redis-cmd
can be used to specify the command line for Redis execution- could bec
docker run ...
for example, otherwise it defaults toredis-server
- could bec
-
--files-path
can be used to specify the directory where to put output files (config.yaml, log files, file writer .h5 files)
- if
- the scope of the fixture changes dynamically, from
-
bec_client_with_demo_config
: a fixture returning aBECIPythonClient
object, which also initializes BEC with demo config- also
bec_client_lib_with_demo_config
, which returns a simpleBECClient
object instead of IPython client
- also
-
bec_client_fixture
: ultimate fixture providing a BEC IPython object with a cleaned scan queue -
bec_client_lib
: same asbec_client_fixture
, but with aBECClient
object
So basically, existing end-to-end tests can rely on bec_client_fixture
and for Qt tests we may want to
use bec_servers
or bec_client_lib
for example.
This MR also does the job of changing the end-to-end tests to use the fixtures, and is a first attempt to simplify gitlab-ci.yml
:
- auto-load config (no need to pre-load config via a script, it is done in fixtures)
- only 2
Dockerfile
- use
buildah
andpodman
instead ofdocker
(no need fordind
, works with Redis as a gitlab-ci service)
Finally, it also adds threads_check
fixture as a pytest plugin (so, available for other projects too), and enables it for auto-use in all parts of BEC -only- (except "device_server" see last comment) ; as a consequence a lot of tests had to be changed to ensure they didn't leave dangling threads in the wild. Special attention is made with loguru
threads, which are cleaned too.
Last but not least, isort
has been applied to the code base, also with @wakonig_k commit initially present in !532 (closed) .
Closes #183 (closed)