All successful regression tests wrongly report "failed with exit code 1"
Since November all tests report a "failed with exit code 1", even though the test ran successful
For example AWAGun-1 (http://amas.web.psi.ch/opal/regressionTests/master/results_2020-02-25.xml):
Description: AWAGun-1. Test failed with exit code 1
This was likely introduced in the major rewrite: !4 (merged)
Relevant lines, from https://gitlab.psi.ch/OPAL/NightlyBuild/blob/ec638a0fbe1cdc97ec30087dd20ff1688d4c5a37/scripts/OpalRegressionTests/regressiontest.py#L269:
# run test
if run_local:
exit_code = self.mpirun()
else:
self.submitToSGE()
self.waitUntilCompletion()
...
if exit_code != 0:
description += ". Test failed with exit code %d" % (exit_code)
simulation_report.addAttribute("description", description)
and
def mpirun(self):
...
exit_code = 0
with open(self.simname + "-RT.o", "wb") as f:
try:
...
except subprocess.TimeoutExpired:
...
return False
except subprocess.CalledProcessError as e:
...
return False
return True
I see two issues here:
- if
run_local
is False, thenexit_code
will not be defined -
mpirun
returns True / False, whileexit_code = self.mpirun()
suggests an integer is expected. This boolean is then checked against 0, which means the opposite of what was meant, (True == 1), so a successful test is reported as failed, and the reportedexit_code
is rather a boolean.