To create a new regression test the following files have to be created
in a directory called RegressionTestName
RegressionTestName.rt: Specify the variables tested against reference data
RegressionTestName.local: The local script executed to run the test when --run-local is specified
RegressionTestName.in: The input file of the simulation
Any deviation from this naming scheme will result in a failure. Additionally there has to be a directory called reference holding the reference data to compare the nightly runs against. This directory contains the following files:
RegressiontestName.lbal: The reference load balancing file
RegressionTestName.lbal.md5: The md5sum of the reference load balancing file
RegressionTestName.out: The simulation output file
RegressionTestName.out.md5: The md5sum of the simulation output file
RegressionTestName.stat: The stat file generated by the simulation
RegressionTestName.stat.md5: The md5sum of the stat file
Creating md5sums shows how md5sums can be generated for reference files.
All these files have to be placed in regression-tests/RegressionTests/ and be committed to the repository in order be run nightly.
Examples of working regression tests are located in regression-tests/RegressionTests/.
"External field test (without space charge)"
stat "rms_x" last 1E-15 #this is a comment
stat "rms_y" last 1E-15 #this is a comment
stat "rms_s" last 1E-15 #this is a comment
They start with a description of the regression test that will be
displayed on the web-page results. The rest of the file specifies what
will be compared against the reference data (one line per test). In
the above example we test the variable rms_x from the stat file by
using the last value found in the file with a precision of
1E-15. Everything after a # sign will be treated as comment and
therefore will be ignored.
Choices for files are: lbal, out, stat
Choices for variables are defined in the files
Choices for data points to compare are: last, avg
In case of data from the PROBE element, you can compare x,
y, z, px, py, pz, track_id, turn and time.
quantity: string that defines how the variable should be handled. Options are all (other options not implemented) all test fails if any particles in any plane in the loss file have variable variable_(ref) > tolerance
tolerance: floating point tolerance (absolute)
file_name: name of the loss file to be checked
Output in the loss file is assumed to be that of a PROBE element.
If a line of output is not compatible with PROBE output, test will ignore the line (not fail).
Test will always fail if no valid data was found in the loss file
or the loss file could not be opened.
Particles are grouped into plane according to a unique combination
of <Turn id> and <Element id>
A sample rt file looks like this:
"One hundred turn RK-4 tracking in ERIT FFAG using SBend3D; no space charge"
PROBE1.loss "x" all 1E-15
PROBE1.loss "y" all 1E-15