Code indexing in gitaly is broken and leads to code not being visible to the user. We work on the issue with highest priority.

Skip to content

Resolve "end-to-end tests: fixtures for starting redis and BEC services"

Description

"no-tmux" option for starting servers

Introduce a new way of starting bec servers with bec-server start --no-tmux: just uses subprocess to start servers. Stopping is not implemented in this MR (since it would require to keep track of PIDs in pid files or whatever).

Fixtures for end-2-end tests

Add new fixtures for tests, the code is in bec_client/test_fixtures.py but it is made available as a pytest plugin for use in other projects (thinking of ophyd devices or bec widgets). The new fixtures make use of pytest-redis to start Redis and rely on the new subprocess_start method of ServiceHandler (cf. above) to start services. The more useful ones are:

  • bec_servers: a fixture, that ensures Redis and all BEC servers are ready before the test starts
    • the scope of the fixture changes dynamically, from session to function depending if --flush-redis option is given or not
    • if --bec-redis-host is provided, indicates which host is running the Redis instance
      • this is mainly useful when Redis is running within Docker on a different network for example, for the CI
    • if --start-servers command line argument is specified, servers are started automatically
      • if --flush-redis command line argument is provided, Redis DB is flushed after each test
        • as a consequence, BEC servers are also restarted at each test
      • --bec-redis-cmd can be used to specify the command line for Redis execution
        • could bec docker run ... for example, otherwise it defaults to redis-server
      • --files-path can be used to specify the directory where to put output files (config.yaml, log files, file writer .h5 files)
  • bec_client_with_demo_config: a fixture returning a BECIPythonClient object, which also initializes BEC with demo config
    • also bec_client_lib_with_demo_config, which returns a simple BECClient object instead of IPython client
  • bec_client_fixture: ultimate fixture providing a BEC IPython object with a cleaned scan queue
  • bec_client_lib: same as bec_client_fixture, but with a BECClient object

So basically, existing end-to-end tests can rely on bec_client_fixture and for Qt tests we may want to use bec_servers or bec_client_lib for example.

This MR also does the job of changing the end-to-end tests to use the fixtures, and is a first attempt to simplify gitlab-ci.yml:

  • auto-load config (no need to pre-load config via a script, it is done in fixtures)
  • only 2 Dockerfile
  • use buildah and podman instead of docker (no need for dind, works with Redis as a gitlab-ci service)

Finally, it also adds threads_check fixture as a pytest plugin (so, available for other projects too), and enables it for auto-use in all parts of BEC -only- (except "device_server" see last comment) ; as a consequence a lot of tests had to be changed to ensure they didn't leave dangling threads in the wild. Special attention is made with loguru threads, which are cleaned too.

Last but not least, isort has been applied to the code base, also with @wakonig_k commit initially present in !532 (closed) .

Closes #183 (closed)

Edited by guijar_m

Merge request reports