Jungfraujoch

Application to receive data from the PSI JUNGFRAU and EIGER detectors.

Citation: F. Leonarski, M. Bruckner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler, Z. Matej, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang "Jungfraujoch: hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors" (2023), J. Synchrotron Rad., 30, 227-234 doi:10.1107/S1600577522010268.

The project is supported by :

• Innosuisse via Innovation Project "NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications" (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
• ETH Domain via Open Research Data Contribute project (Jan - Dec 2023)
• AMD University Program with donation of licenses of Ethernet IP cores and Vivado software

License

Operating Jungfraujoch requires license from the Paul Scherrer Institute.

Specifically, non-profit research facilities operating PSI JUNGFRAU and PSI EIGER detectors are granted the license to use the code and binary images within their facilities, as well as to modify the code according to their needs.

Sharing the code requires explicit permission from the Paul Scherrer Institute.

Compatible detectors

Currently, Jungfraujoch supports PSI JUNGFRAU and PSI EIGER detectors.

Hardware requirements

See hardware requirements documentation.

FPGA bitstream

Instructions see here

Detector

Jungfraujoch supports PSI JUNGFRAU and EIGER detectors. Jungfruajoch controls the detector via statically compiled slsDetectorPackage into its source code. It is important that detector firmware has to match slsDetectorPackage version used in Jungfraujoch (8.0.2 at the moment). See PSI Detector group website for details.

Operating system

Recommended operating system is Red Hat Enterprise Linux (RHEL) / Rocky Linux versions 8. For this operating system we provide RPMs with pre-built binaries to simplify deployment.

We do also operate one of the systems with Rocky 9 without issues. Running Jungfraujoch on Red Hat Enterprise Linux 7 is currently not tested and not recommended, but likely possible with providing some packages from external repositories. There are some limited tests with recent Ubuntu and Fedora distributions, though these are not systematic. Other linux platforms should work, but no tests were done so far.

Software dependencies

Required:

• C++20 compiler and C++20 standard library; recommended GCC 11+ or clang 14+ (Intel OneAPI, AMD AOCC)
• CMake version 3.21 or newer + GNU make tool
• zlib compression library

Optional:

• CUDA compiler version 11 or newer - required for MX fast feedback indexer
• NUMA library - to pin threads to nodes/CPUs
• Node.js - to make frontend

Automatically downloaded by CMake and statically linked:

• SLS Detector Package - see github.com/slsdetectorgroup/slsDetectorPackage
• Zstandard (Facebook) - see github.com/facebook/zstd
• Pistache webserver - see github.com/pistacheio/pistache
• Fast feedback indexer (Hans-Christian Stadler, PSI) - see github.com/paulscherrerinstitute/fast-feedback-indexer
• Catch2 testing library - see github.com/catchorg/Catch2
• HDF5 library - see github.com/HDFGroup/hdf5
• TIFF library - see gitlab.com/libtiff/libtiff

Please follow the link provided above to check for LICENSE file. Building code with dependencies above requires access from the build system to github.com.

Directly included in the repository:

• JSON parser/writer from N. Lohmann - see github.com/nlohmann/json
• Xilinx arbitrary precision arithmetic headers - see github.com/Xilinx/HLS_arbitrary_Precision_Types
• Bitshuffle filter from K. Masui - see github.com/kiyo-masui/bitshuffle
• Fast replacement for Bitshuffle pre-compression filter (Kal Cutter, DECTRIS) - see github.com/kalcutter/bitshuffle
• Tinycbor (Intel) - see github.com/intel/tinycbor
• LZ4 compression by Y.Collet - see github.com/lz4/lz4
• Spdlog logging library - see github.com/gabime/spdlog
• ZeroMQ library (through slsDetectorPackage) - see github.com/zeromq/libzmq

For license check LICENSE file in respective directory

Installation guide

Follow Installation guide.

Software components

• jfjoch_broker in broker - main service running on the Jungfraujoch server, responsible for control of the detector and data acquisition; Example configuration jfjoch_broker for the modules is given in configuration files present in etc directory. See details.
• jfjoch_writer in writer - HDF5 writer; HDF5 writer is designed to work on the same or separate server system. It has rather limited requirements in terms of performance and memory. The goal is to separate data acquisition node with custom FPGA hardware and file system node with stronger security/stability requirements. See details.

Web Frontend

Jungfraujoch is equipped with React-based web frontend for user-friendly experience. Frontend has the following options:

• Presenting current state of the detector
• Plotting results of online quality calculations
• Showing live view images from the detector
• JUNGFRAU calibration numbers
• Configuring the detector, as well as pedestal/initialization operations

Frontend is written in TypeScript. For details see frontend/ directory.

OpenAPI python client

Jungfraujoch is controlled with HTTP/REST interface defined with an OpenAPI specification. For convenience, we provide Python client in python_client directory.

Tests

Automated test routine is then accessible as tests/jfjoch_test. There are also benchmark routines:

• HDF5DatasetWriteTest to measure HDF5 dataset writing speed (single threaded)
• jfjoch_spot_finding_test to apply spot finding and indexing routines in Jungfraujoch to an example dataset - this is equivalent to FPGA spot finding algorithm, but NOT performance equivalent as it is particularly not-efficient
• jfjoch_action_test to test quality/performance of FPGA card(s) and software routines

In addition, tests are executed to verify that datasets written by Jungfraujoch are readable with XDS Durin plugin, XDS Neggia plygin and CrystFEL. Input files for these programs are placed in xds_durin, xds_neggia and crystfel folders. See .gitlab-ci.yml for details.