src issueshttps://gitlab.psi.ch/OPAL/src/-/issues2021-06-10T18:03:26+02:00https://gitlab.psi.ch/OPAL/src/-/issues/165opt-pilot: objectives based on design variables2021-06-10T18:03:26+02:00snuverink_jjochem.snuverink@psi.chopt-pilot: objectives based on design variablesas mentioned in https://gitlab.psi.ch/OPAL/src/issues/8#note_3112 writing objectives like (where d1 and d2 are design variables):
```
//obj1:OBJECTIVE,EXPR="fabs(d1+d2)";
```
seem not to work (not sure if it ever did). Since the optimi...as mentioned in https://gitlab.psi.ch/OPAL/src/issues/8#note_3112 writing objectives like (where d1 and d2 are design variables):
```
//obj1:OBJECTIVE,EXPR="fabs(d1+d2)";
```
seem not to work (not sure if it ever did). Since the optimiser still tries to look in the output file:
```
Exception while getting value from SDDS file: unkown column name!
```
Still such an objective could potentially be useful.
I might have a look at this eventually, or if there is a direct need for it. Right now putting a low weight on it.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/src/-/issues/263Avoid direct dependency of optimiser to OPAL?2021-06-10T17:57:56+02:00snuverink_jjochem.snuverink@psi.chAvoid direct dependency of optimiser to OPAL?See also discussion on commit 5d045b2, in order not to forget I thought it would be good to open an issue. Commit 5d045b2 introduced a direct dependency of the optimiser to OPAL.
Two solutions were proposed:
* [preprocessor variable](h...See also discussion on commit 5d045b2, in order not to forget I thought it would be good to open an issue. Commit 5d045b2 introduced a direct dependency of the optimiser to OPAL.
Two solutions were proposed:
* [preprocessor variable](https://gitlab.psi.ch/OPAL/src/commit/5d045b26ce4a51f1138cde80d13e59616c5e1bed#note_7617)
* [(over)write header file in CMake](https://gitlab.psi.ch/OPAL/src/commit/5d045b26ce4a51f1138cde80d13e59616c5e1bed#note_7621)
Question is if we want to keep the option of having the optimiser stand alone?
cc: @kraus, @frey_m, @adelmann.https://gitlab.psi.ch/OPAL/src/-/issues/337Sampler fails due to static variable in MueLuBottomSolver2020-07-14T10:29:45+02:00frey_mSampler fails due to static variable in MueLuBottomSolver### Summary
As in #336 the sampler crashes when dispatching new individuals after some finished.
### Steps to reproduce
Run the sampler with AMR Trilinos and the smoothed aggregation algebraic multigrid solver for the linear system o...### Summary
As in #336 the sampler crashes when dispatching new individuals after some finished.
### Steps to reproduce
Run the sampler with AMR Trilinos and the smoothed aggregation algebraic multigrid solver for the linear system of
equations.
### What is the current *bug* behavior?
The sampler crashes.
### What is the expected *correct* behavior?
The sampler runs smoothly.
### Possible fixes
Remove the static variable.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/239Remove requirement of data file for SAMPLER and OPTIMIZER2020-06-22T09:42:35+02:00adelmannRemove requirement of data file for SAMPLER and OPTIMIZERWe still use the runOPAL.py scheme with template / data file. However the design variables could be easily looked up in the dictionary or we invent new types: DvarReal etc.
[Edit: this has been moved to a separate issue, see #249! If we...We still use the runOPAL.py scheme with template / data file. However the design variables could be easily looked up in the dictionary or we invent new types: DvarReal etc.
[Edit: this has been moved to a separate issue, see #249! If we could define quantities of interest (a la optimizer) and write them to the json file lengthly post processing would be avoided.]
Edit:
- [x] remove the need for a data file (!288)
- [x] write a json file containing the values of the DVARs and objectives (#249)OPAL 2.4.0frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/504File header for Optimiser classes2020-04-08T16:57:44+02:00frey_mFile header for Optimiser classesThis issue adds the file header according to #482 to the optimiser classes.This issue adds the file header according to #482 to the optimiser classes.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/365Bug in Variator::variate()2019-10-16T15:26:19+02:00snuverink_jjochem.snuverink@psi.chBug in Variator::variate()Observed by @ext-kranjcevic_m:
> I've been looking at [..] commits c21cfcf1 and 29790810.
>In Optimizer/EA/Variate.h, in variate, the individual can be declared infeasible (lines 221, 224). However, I don't see that it is then also rem...Observed by @ext-kranjcevic_m:
> I've been looking at [..] commits c21cfcf1 and 29790810.
>In Optimizer/EA/Variate.h, in variate, the individual can be declared infeasible (lines 221, 224). However, I don't see that it is then also removed from individualsToEvaluate_m, which can result in a segmentation fault in dispatch_forward_solves, because ind (line 459 in Optimizer/EA/FixedPisaNsga2.tcc) can be a null pointer. Fortunately, when the number of tries in the variator is high enough (e.g., 100 tries), this is unlikely to happen.
>Executing lines 460-485 in Optimizer/EA/FixedPisaNsga2.tcc only if (ind != NULL) should fix the problem.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/src/-/issues/364Optimizer stops to write to disk after a couple of configurations2019-10-08T10:16:18+02:00bellotti_rOptimizer stops to write to disk after a couple of configurationsI'm having problems with the configurations in [this repo](https://gitlab.psi.ch/bellotti_r/awa_masters), in the directory ```optim-quads```.
Steps to reproduce
==================
- Run the batch job provided in the repo (change the pat...I'm having problems with the configurations in [this repo](https://gitlab.psi.ch/bellotti_r/awa_masters), in the directory ```optim-quads```.
Steps to reproduce
==================
- Run the batch job provided in the repo (change the path to OPAL to your path)
- The job runs normally, but after 2min or so, nothing is written to the disk anymore, even if you wait 20min
- Problems do not arise if ```--ntasks=8``` is set in the Slurm file
- The same job runs fine with OPAL 2.0, so it is not a bad configurationhttps://gitlab.psi.ch/OPAL/src/-/issues/257GA interface is appending new generation output to new json file, rather than...2019-07-04T17:15:48+02:00ext-edelen_aGA interface is appending new generation output to new json file, rather than having one generation output per json fileaccording to the input file:
```
INITIALPOPULATION=100,
MAXGENERATIONS=2,
NUM_MASTERS=1,
NUM_COWORKERS=8,
SIMTMPDIR="tmp",
TEMPLATEDIR="tmpl",
FIELDMAPDIR="fieldm...according to the input file:
```
INITIALPOPULATION=100,
MAXGENERATIONS=2,
NUM_MASTERS=1,
NUM_COWORKERS=8,
SIMTMPDIR="tmp",
TEMPLATEDIR="tmpl",
FIELDMAPDIR="fieldmaps",
NUM_IND_GEN=100,
GENE_MUTATION_PROBABILITY=0.8,
MUTATION_PROBABILITY=0.8,
RECOMBINATION_PROBABILITY=0.2;
```
It seems like this should be 200 samples (100 in population x 2 generations)
However, using mldb to convert to pk reports 300 samples:
```
OPAL ML Database Generator
Found 2 json files.
Write ML-Database 1nCGA-small-test.pk
xDim = 6 -> ['G0', 'G1', 'K0', 'K1', 'PH0', 'PH1']
yDim = 7 -> ['DE', 'EMITS', 'EMITX', 'EMITY', 'RMSS', 'RMSX', 'RMSY']
generations = 2
Data points = 300
```
Looking at the json files:
- 1_1nCGA_0.json has 100 individuals
- 2_1nCGA_0.json has 200 individuals
And if we compare the first individual in each case, they are both the same:
```
"G0": 63.7949,
"G1": 17.5757,
"K0": 474.876,
"K1": 185.284,
"PH0": -8.94274,
"PH1": -2.44182
```
So it looks like new data is just being appended to the new json file in subsequent generations rather than creating a new file for each generation with only that generation's data in it.snuverink_jjochem.snuverink@psi.chkraussnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/src/-/issues/318Optimizer: Generalizing objectives with sddsVariableAt2019-06-26T16:56:20+02:00frey_mOptimizer: Generalizing objectives with sddsVariableAt### Summary
Currently, the optimizer allows to evaluate an objective in the stat-file with ```sddsVariableAt``` according to ```spos```
only. I'd like to use other quantities as well.### Summary
Currently, the optimizer allows to evaluate an objective in the stat-file with ```sddsVariableAt``` according to ```spos```
only. I'd like to use other quantities as well.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/174optimiser run hasResultsAvailable()2019-04-06T10:08:12+02:00adelmannoptimiser run hasResultsAvailable()it seams that hasResultsAvailable() is sometimes true after I removed
the pid from the hash string. This was necessary when more than
one $CORE is used for a worker.
I probable need to add this back but not with the pid but with an
id ...it seams that hasResultsAvailable() is sometimes true after I removed
the pid from the hash string. This was necessary when more than
one $CORE is used for a worker.
I probable need to add this back but not with the pid but with an
id that represents the "worker with more than one core"
@ineichen can you point me to that structure.OPAL 2.0.0adelmannYves Ineichenadelmann2017-10-28https://gitlab.psi.ch/OPAL/src/-/issues/278add option to load initial population for GA2019-03-15T11:13:52+01:00ext-edelen_aadd option to load initial population for GAWe'd like to be able to specify an initial population to use in the OPAL GA.We'd like to be able to specify an initial population to use in the OPAL GA.frey_msnuverink_jjochem.snuverink@psi.chadelmannkrausfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/251Allow optimizer and sampler runs to restart from given h5 file2019-02-25T08:43:02+01:00krausAllow optimizer and sampler runs to restart from given h5 fileRestart currently doesn't work for runs when launched from the optimizer / sampler. This feature should be relatively easy to implement. Reported by M. Abo-Bakr.Restart currently doesn't work for runs when launched from the optimizer / sampler. This feature should be relatively easy to implement. Reported by M. Abo-Bakr.krausfrey_mkraushttps://gitlab.psi.ch/OPAL/src/-/issues/228Restart optimizer from generation file2019-02-23T10:03:25+01:00frey_mRestart optimizer from generation file@snuverink_j and I discussed that it would be a nice feature of restarting the optimizer from a generation file. If the user put the generation number too low, this feature would help to continue computation instead of starting from scra...@snuverink_j and I discussed that it would be a nice feature of restarting the optimizer from a generation file. If the user put the generation number too low, this feature would help to continue computation instead of starting from scratch.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/152More than 1 coworker2019-01-10T09:37:22+01:00adelmannMore than 1 coworker**--num-coworkers=2** does not work. Simulation of first generation is not terminating**--num-coworkers=2** does not work. Simulation of first generation is not terminatingYves IneichenYves Ineichenhttps://gitlab.psi.ch/OPAL/src/-/issues/269Halo evaluation as objective2018-12-11T09:14:40+01:00frey_mHalo evaluation as objectiveIt would be nice having a halo evaluation as an objective for the sampler and optimizer. I also think about multi-bunch simulations. We need either the halo written to the stat-file or evaluate it using the H5 file. In case of multi-bunc...It would be nice having a halo evaluation as an objective for the sampler and optimizer. I also think about multi-bunch simulations. We need either the halo written to the stat-file or evaluate it using the H5 file. In case of multi-bunch simulations where particles are assigned to energy bins, we'd need a stat file for every energy bin. Therefore, I think, it's easier to add new objective funtion.
I suggest something like this
```c++
halo(dir, step, bin)
```
where ```dir=x,y,z``` and ```bin``` the energy bin (default: 0).frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/233Optimiser: Parsing of expressions2018-12-10T14:34:25+01:00frey_mOptimiser: Parsing of expressionsI added the new function ```infNormRadialPeak```. However, it's not recognized. It crashes with ```Parsing failed!```. I do not see a difference to other expressions.
According to [line 180](https://gitlab.psi.ch/OPAL/src/blob/master/op...I added the new function ```infNormRadialPeak```. However, it's not recognized. It crashes with ```Parsing failed!```. I do not see a difference to other expressions.
According to [line 180](https://gitlab.psi.ch/OPAL/src/blob/master/optimizer/Expression/Expression.h#L180) it checks
```
if (success && iter != end) {
std::cout << "Parsing failed!" << std::endl;
throw new OptPilotException("Expression::parse()",
"Parsing failed!");
}
```
I think it should be ```!success``` and ```ìter == end``` instead. A first test confirms my proposal, i.e. it still works with
e.g. ```sumErrSqRadialPeak``` but now also with ```infNormRadialPeak```.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/235Optimiser occasional crashes with HDF5 enabled.2018-12-10T14:33:54+01:00snuverink_jjochem.snuverink@psi.chOptimiser occasional crashes with HDF5 enabled.@ext-neveu_n reported crashes with the optimizer. It turned out that the crashes were coming from the HDF5 output (with `ENABLEHDF5=FALSE` there were no more crashes).
Stack trace:
```
H5PartWrapper::storeCavityInformation()
H5PartWrapp...@ext-neveu_n reported crashes with the optimizer. It turned out that the crashes were coming from the HDF5 output (with `ENABLEHDF5=FALSE` there were no more crashes).
Stack trace:
```
H5PartWrapper::storeCavityInformation()
H5PartWrapper::open(int)
h5_open_file2
h5_error
h5_report_errorhandler
```
`H5PartWrapper::open(int)`:
```c++
void H5PartWrapper::open(h5_int32_t flags) {
close();
h5_prop_t props = H5CreateFileProp ();
MPI_Comm comm = Ippl::getComm();
h5_err_t h5err = H5SetPropFileMPIOCollective (props, &comm);
#if defined (NDEBUG)
(void)h5err;
#endif
assert (h5err != H5_ERR);
file_m = H5OpenFile (fileName_m.c_str(), flags, props);
assert (file_m != (h5_file_t)H5_ERR);
H5CloseProp (props);
}
```
So the opening of the file failed for some reason (Perhaps the optimiser has deleted the directory?).https://gitlab.psi.ch/OPAL/src/-/issues/259Structure of output JSON file2018-12-02T20:27:00+01:00huber_yStructure of output JSON fileRight now the output JSON file is structured in a way such that when read by python using the json module it is converted to an array of dictionaries where each directionary contains a key "ID" and the corresponding ID is the value, plus...Right now the output JSON file is structured in a way such that when read by python using the json module it is converted to an array of dictionaries where each directionary contains a key "ID" and the corresponding ID is the value, plus additonal key such as dvars containing the design variables.
For finding an ID it would in my eyes be easier if the file was structured such that the resulting array contains only one dictionary, where the keys are something like ID+"dvars" where ID is the actual ID, since this allows to easily access elements.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/src/-/issues/248Problems with flexible collimator when used with optimizer2018-11-20T20:30:57+01:00adelmannProblems with flexible collimator when used with optimizerI am getting messages like this:
```
░No reference point provided: using the origin
Individual not viable, I try again: iter= 0
Individual not viable, I try again: iter= 0
░No reference point provided: using the origin
Individua...I am getting messages like this:
```
░No reference point provided: using the origin
Individual not viable, I try again: iter= 0
Individual not viable, I try again: iter= 0
░No reference point provided: using the origin
Individual not viable, I try again: iter= 0
Individual not viable, I try again: iter= 1
Individual not viable, I try again: iter= 2
```
and then the optimisation is killed with
```
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 61951 RUNNING AT bdw-0026
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
Intel(R) MPI Library troubleshooting guide:
https://software.intel.com/node/561764
===================================================================================
```
No further information !
All simulation in progress, at the time when the optimisation is killed,
are terminating just fine.
There is the new flexible collimator involved and a constraint w.r.t. number of particles.
When removing the flexible collimator the optimisation works just fine.
Will add input files later. Need to catch my airplane now!krausadelmannkraushttps://gitlab.psi.ch/OPAL/src/-/issues/255OPAL GA interface does not raise error when design variable is not actually i...2018-11-06T23:01:39+01:00ext-edelen_aOPAL GA interface does not raise error when design variable is not actually in tmpl file, and it writes spurious data to jsonThe GA input file writes to the json file which values it sent to the simulation even if those variables cannot be found in the tmpl file.(Example: in our case, LRAD was set in the GA input file, but in the tmpl file the value that actua...The GA input file writes to the json file which values it sent to the simulation even if those variables cannot be found in the tmpl file.(Example: in our case, LRAD was set in the GA input file, but in the tmpl file the value that actually needs to be changed is Rc, so, Rc was constant, but our data from the json file shows a spurious change in LRAD)
The same issue likely applies to the random sample interface.
--> OPAL should raise an error when a variable from the GA input file cannot be found in the tmpl file.