pyOPALTools issueshttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues2018-03-15T18:52:31+01:00https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/2Json reader2018-03-15T18:52:31+01:00adelmannJson reader```
python json-reader-1.py 101
Found 999 json files.
('Read generation ', '101')
[u'rinit', u'tc01mb', u'prinit', u'tc02mb', u'tc03mb', u'phiinit', u'benergy']
[u'dpeak1', u'dpeak2', u'dpeak5_7', u'dpeak3_4', u'dpeak8_13']
adelm...```
python json-reader-1.py 101
Found 999 json files.
('Read generation ', '101')
[u'rinit', u'tc01mb', u'prinit', u'tc02mb', u'tc03mb', u'phiinit', u'benergy']
[u'dpeak1', u'dpeak2', u'dpeak5_7', u'dpeak3_4', u'dpeak8_13']
adelmann@fast-dude ~/git/pyOPALTools/tests/json-reader $ python json-reader-1.py 102
Found 999 json files.
('Read generation ', '102')
index 107 is out of bounds for axis 0 with size 107
```
Why give generation 107 an error?adelmannadelmannhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/3Pareto front viewer2017-06-29T13:27:00+02:00adelmannPareto front viewerUpdate to new format
[1_Ring.dat_0.json](/uploads/c428655026db65955a65d3cad90754a5/1_Ring.dat_0.json)Update to new format
[1_Ring.dat_0.json](/uploads/c428655026db65955a65d3cad90754a5/1_Ring.dat_0.json)frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/4testStat-1.py2017-11-11T10:41:57+01:00adelmanntestStat-1.pySeams to be confused with new SDDS reader
```python
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-90-5eae613b6252> ...Seams to be confused with new SDDS reader
```python
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-90-5eae613b6252> in <module>()
1 import statPlots as statpl
2
----> 3 statpl.opalStatOverviewPlot(fns)
~/git/pyOPALTools/statPlots.py in opalStatOverviewPlot(fns, title, pdfFn, addData, myLoc)
93 dataSrc.append(legendName)
94
---> 95 parser = SddsReader(fns[i])
96
97 if not x:
~/git/pyOPALTools/utils.py in __init__(self, filename)
171
172 self.memoryMapFile(filename)
--> 173 self.parseHeader()
174
175
~/git/pyOPALTools/utils.py in parseHeader(self)
277
278 else:
--> 279 raise Exception("Invalid syntax in SDDS header")
280
281 line = self.sdds_data_mem.readline()
Exception: Invalid syntax in SDDS header
```frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/5Format timing information2017-11-14T09:42:44+01:00adelmannFormat timing informationWe should use the timing.dat data and not the std out.
```
num Nodes CPU tot Wall tot
===============================================
mainTimer........... 512 1260 1269
num Nodes C...We should use the timing.dat data and not the std out.
```
num Nodes CPU tot Wall tot
===============================================
mainTimer........... 512 1260 1269
num Nodes CPU max Wall max CPU min Wall min CPU avg Wall avg
=======================================================================================
mainTimer........... 512 1260 1269 1246 1269 1253 1269
Binaryrepart........ 512 0.61 0.596 0.58 0.5907 0.5931 0.5933
Boundingbox......... 512 133.7 133.5 130.6 130.8 132 132
Compute Statistics.. 512 32.91 32.6 29.88 29.98 31.71 31.71
ComputePotential.... 512 511.9 512.4 482.4 483.1 493.8 495.1
Create Distr........ 512 8.89 8.901 8.79 8.835 8.858 8.862
External field eval. 512 36.51 36.68 6.84 7.06 19.13 19.33
Fast inside test.... 512 0 0 0 0 0 0
Histogram........... 512 0 0 0 0 0 0
Initialize geometry. 512 0 0 0 0 0 0
Inside test......... 512 0 0 0 0 0 0
Load Distr.......... 512 0 0 0 0 0 0
Particle Inside..... 512 0 0 0 0 0 0
Ray tracing......... 512 0 0 0 0 0 0
Secondary emission.. 512 0 0 0 0 0 0
SelfField total..... 512 942.6 943.1 905.2 909.5 925.9 928.1
SF: GreensFTotal.... 512 412 412.3 395.8 398.1 404.5 405.7
TIntegration1....... 512 12.14 11.98 10.78 11.05 11.54 11.54
TIntegration2....... 512 15.75 15.74 14.19 14.55 15 15
WakeField........... 512 0 0 0 0 0 0
Write H5-File....... 512 11.29 16.15 4.47 16.13 7.677 16.14
Write Stat.......... 512 4.89 4.755 4.42 4.678 4.687 4.684
```frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/6timing plots2017-11-22T08:41:30+01:00adelmanntiming plotsThe max/min (tplot.summary_plot(fname=fn4, grid=True, first=7)) of the timing plots is wrong.
![image](/uploads/6cec84199f7a659949f9163eb75ba0e4/image.png)
SelfField total..... 128 178.8 178.8 146 145.9 163.5 ...The max/min (tplot.summary_plot(fname=fn4, grid=True, first=7)) of the timing plots is wrong.
![image](/uploads/6cec84199f7a659949f9163eb75ba0e4/image.png)
SelfField total..... 128 178.8 178.8 146 145.9 163.5 163.6frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/7Timing: Parse problem size2018-01-25T12:20:11+01:00frey_mTiming: Parse problem sizeMentioned in https://gitlab.psi.ch/OPAL/src/issues/181#note_4222.Mentioned in https://gitlab.psi.ch/OPAL/src/issues/181#note_4222.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/8Statistics Plot - Single Bunch vs. Multi Bunches2019-03-26T16:07:11+01:00frey_mStatistics Plot - Single Bunch vs. Multi BunchesDevelop some statistics plotting methods for single and multi bunch simulations.Develop some statistics plotting methods for single and multi bunch simulations.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/9Implementation of envelope plot2018-03-27T13:58:58+02:00frey_mImplementation of envelope plotThe envelope plot is a nice view of the beamline for OPAL-T.
Please fill the gaps in
[LatticeParser.py](https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/utilities/LatticeParser.py)
and
[plot_envelope function](https://gitlab.psi.ch...The envelope plot is a nice view of the beamline for OPAL-T.
Please fill the gaps in
[LatticeParser.py](https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/utilities/LatticeParser.py)
and
[plot_envelope function](https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/plots.py#L155)ganz_pganz_phttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/10cleanup root directory2019-01-30T16:25:42+01:00adelmanncleanup root directoryI think we should delete all *.py und notebooks on the root level.
Do you need any of these files?I think we should delete all *.py und notebooks on the root level.
Do you need any of these files?frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/11plot_density slow for large number of particles2018-04-11T13:40:21+02:00snuverink_jjochem.snuverink@psi.chplot_density slow for large number of particlesIn the notebook RingCyclotron.ipynb the plot_density function is very slow for the example. I think it would be much faster and less memory-hungry if a histogram is made instead of scatter plot with colours.
e.g.
```python
plt.hist2d(xd...In the notebook RingCyclotron.ipynb the plot_density function is very slow for the example. I think it would be much faster and less memory-hungry if a histogram is made instead of scatter plot with colours.
e.g.
```python
plt.hist2d(xdata, ydata, (50, 50), cmap=plt.cm.jet)
```
gives within a second:
![image](/uploads/8c4ccb93cde6bbd5c4f25faff0a9772d/image.png)
@frey_m : I can add this if you'd agree.https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/12Removal of deprecated timing class.2018-04-14T20:08:27+02:00frey_mRemoval of deprecated timing class.The plots of ```pyOPALTools/timing/TimePlot.py``` got replaced by [opal/visualization/timing/plots.py](https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/timing/plots.py). Therefore, I remove the old class and all the ...The plots of ```pyOPALTools/timing/TimePlot.py``` got replaced by [opal/visualization/timing/plots.py](https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/timing/plots.py). Therefore, I remove the old class and all the tests.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/13Parsing and Plotting of probe and peak file data.2019-01-30T16:25:42+01:00frey_mParsing and Plotting of probe and peak file data.After a discussion with @snuverink_j we decided to add plotting and parsing tools of probe and peak file data.After a discussion with @snuverink_j we decided to add plotting and parsing tools of probe and peak file data.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/14load_dataset not working as expected2019-04-06T15:27:00+02:00ext-roussel_rload_dataset not working as expectedHey guys, I like the new pyOPALTools, Nicole gave you a nice shoutout.
Issue: load_dataset('directory_name') doesn't return .stat file in directory_name/. Returns empty list.
RyanHey guys, I like the new pyOPALTools, Nicole gave you a nice shoutout.
Issue: load_dataset('directory_name') doesn't return .stat file in directory_name/. Returns empty list.
Ryanext-roussel_rext-roussel_rhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/16Do not call plt.figure inside plotting functions2019-04-06T15:27:00+02:00frey_mDo not call plt.figure inside plotting functionsI think we shouldn't call
```python
plt.figure()
```
inside a plotting function. Otherwise we can't overlay multiple datasets.
Another solution could be of course to give a list of datasets to the plotting functions, thus, it would iter...I think we shouldn't call
```python
plt.figure()
```
inside a plotting function. Otherwise we can't overlay multiple datasets.
Another solution could be of course to give a list of datasets to the plotting functions, thus, it would iterate through them and call
```python
plt.figure()
```
only once.
It's really a design question. @snuverink_j What do you think?https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/1Selection of particle positions with detect_peaks not consistent2019-01-30T16:25:42+01:00snuverink_jjochem.snuverink@psi.chSelection of particle positions with detect_peaks not consistentThe turn separation calculation in cyclotron.py selects the x-axis crossings with a call to detect_peaks:
`pksx = detect_peaks(x, mph=0.04, mpd=100)`
This selects (see figure) the cyan points in the PSI Ring model. However, I would exp...The turn separation calculation in cyclotron.py selects the x-axis crossings with a call to detect_peaks:
`pksx = detect_peaks(x, mph=0.04, mpd=100)`
This selects (see figure) the cyan points in the PSI Ring model. However, I would expect the red points (indices: firstPoint + turnNr*stepsPerTurn). I think inherently the detect_peaks function can always go wrong. Therefore, I would like to change it to the stepsPerTurn selection.
![Orig_Position](/uploads/0053b7843f8bc4dbddb6bae69c7b5231/Orig_Position.png)https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/17JSON format change in Optimizer and Sampler2018-12-05T22:10:26+01:00frey_mJSON format change in Optimizer and SamplerThere's a JSON format change due to issue https://gitlab.psi.ch/OPAL/src/issues/259. We need therefore to adapt parsers. The new format contains a version tag such that we can still handle the deprecated data files.There's a JSON format change due to issue https://gitlab.psi.ch/OPAL/src/issues/259. We need therefore to adapt parsers. The new format contains a version tag such that we can still handle the deprecated data files.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/18Logger2019-07-04T11:30:59+02:00frey_mLoggerInstall a logging system for pyOPALTools. Started with b232ef78b82f07b6ab7b1db82b3868b544a68363. Before I did the commit, I created a [tag](https://gitlab.psi.ch/OPAL/pyOPALTools/tags/pyOPALTools-0.0.1). So, the users can still have the ...Install a logging system for pyOPALTools. Started with b232ef78b82f07b6ab7b1db82b3868b544a68363. Before I did the commit, I created a [tag](https://gitlab.psi.ch/OPAL/pyOPALTools/tags/pyOPALTools-0.0.1). So, the users can still have the old behaviour.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/19Dataset Information2019-04-06T15:27:00+02:00frey_mDataset InformationAs discussed with @snuverink_j we should add a function like ```summary``` to the dataset classes that prints the available variables etc.
- [x] DatasetBase.py
- [x] H5Dataset.py
- [x] MemoryDataset.py
- [x] PeakDataset.py
- [x] SolverD...As discussed with @snuverink_j we should add a function like ```summary``` to the dataset classes that prints the available variables etc.
- [x] DatasetBase.py
- [x] H5Dataset.py
- [x] MemoryDataset.py
- [x] PeakDataset.py
- [x] SolverDataset.py
- [x] TrackOrbitDataset.py
- [x] OptimizerDataset.py
- [x] ProbeHistDataset.py
- [x] StatDataset.py
- [x] GridDataset.py
- [x] LBalDataset.py
- [x] OutputDataset.py
- [x] SamplerDataset.py
- [x] TimeDataset.py
frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/20Remove duplicated code in datasets2019-04-06T15:27:00+02:00frey_mRemove duplicated code in datasetsSeveral datasets are based on the SDDS format. We should use a base class in order to get rid of duplicated code.Several datasets are based on the SDDS format. We should use a base class in order to get rid of duplicated code.https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/21Add AMR dataset to general interface2019-04-06T15:27:00+02:00frey_mAdd AMR dataset to general interfacefrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/22Plotting Classes2019-04-06T15:27:01+02:00frey_mPlotting ClassesAs discussed with @gsell we should implement base classes for plotting in order to avoid the if-statements
```Python
if not isinstance(ds, DatasetBase):
raise TypeError("Dataset '" + ds.filename +
"' not d...As discussed with @gsell we should implement base classes for plotting in order to avoid the if-statements
```Python
if not isinstance(ds, DatasetBase):
raise TypeError("Dataset '" + ds.filename +
"' not derived from 'DatasetBase'.")
if not ds.filetype == FileType.OPTIMIZER:
raise TypeError(ds.filename + ' is not an optimizer dataset.')
```
Each dataset would also be derived from the appropriate plotter class. This would also simplify adding new methods and avoiding errors.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/23Classes for dat a computation2019-04-06T15:27:01+02:00frey_mClasses for dat a computationAs done in #22 we should do the same for data computations.As done in #22 we should do the same for data computations.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/24Switching between plotting styles does not work properly2019-04-01T15:39:50+02:00frey_mSwitching between plotting styles does not work properlyWhen calling for example the ```jupyter``` style and then the ```poster``` style, the ```matplotlib.rcParams``` are not all set.When calling for example the ```jupyter``` style and then the ```poster``` style, the ```matplotlib.rcParams``` are not all set.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/25Removing deprecated pyH5root2019-04-06T15:27:01+02:00frey_mRemoving deprecated pyH5rootfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/26Removing of deprecated profiling directory2019-04-06T15:27:01+02:00frey_mRemoving of deprecated profiling directoryThe plots in *profiling/Memory.py* and *profiling/LoadBalancing.py* are already covered in the new opal package.The plots in *profiling/Memory.py* and *profiling/LoadBalancing.py* are already covered in the new opal package.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/27Remove deprecated amrPlots directory.2019-04-06T15:27:01+02:00frey_mRemove deprecated amrPlots directory.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/28Clean up timing directory2019-04-06T15:27:01+02:00frey_mClean up timing directoryfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/29ASCII Loss files2019-04-05T09:02:05+02:00snuverink_jjochem.snuverink@psi.chASCII Loss filesRequested by @ext\-calvo\_p : loss files (.loss) are missing.Requested by @ext\-calvo\_p : loss files (.loss) are missing.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/30Include mlmove branch into master2019-08-06T09:41:10+02:00frey_mInclude mlmove branch into master@ext\-neveu\_n Might it be possible to migrate the [mlmove](https://gitlab.psi.ch/OPAL/pyOPALTools/tree/mlmove) branch into the master?@ext\-neveu\_n Might it be possible to migrate the [mlmove](https://gitlab.psi.ch/OPAL/pyOPALTools/tree/mlmove) branch into the master?https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/32Documentation style Sphinx2020-04-20T11:08:56+02:00frey_mDocumentation style Sphinx@bellotti\_r changed the documentation in some classes using [Sphinx](http://www.sphinx-doc.org/en/master/). It's similar to Doxygen. We should use a consistent format. I'm fine with the new proposed documentation.@bellotti\_r changed the documentation in some classes using [Sphinx](http://www.sphinx-doc.org/en/master/). It's similar to Doxygen. We should use a consistent format. I'm fine with the new proposed documentation.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/33add_new_files_to_doc.sh presumes bash is located in /usr/bin/bash2019-06-17T15:35:33+02:00snuverink_jjochem.snuverink@psi.chadd_new_files_to_doc.sh presumes bash is located in /usr/bin/bashAs title says, which is not portable (at least not for SL6).
https://unix.stackexchange.com/questions/206350/what-is-the-difference-if-i-start-bash-with-bin-bash-or-usr-bin-env-bash proposes instead:
`#!/usr/bin/env bash`As title says, which is not portable (at least not for SL6).
https://unix.stackexchange.com/questions/206350/what-is-the-difference-if-i-start-bash-with-bin-bash-or-usr-bin-env-bash proposes instead:
`#!/usr/bin/env bash`snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/34Time attribute in h5 is not recognised2019-06-19T08:22:36+02:00adelmannTime attribute in h5 is not recognisedTime attribute in h5, created by a monitor is not recognisedTime attribute in h5, created by a monitor is not recognisedadelmannadelmannhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/35Optimizer: Pareto fron json file2019-06-24T09:55:38+02:00frey_mOptimizer: Pareto fron json fileThe optimizer data set is not able to handle the additional file.The optimizer data set is not able to handle the additional file.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/36Analyse Optimizer output2019-06-29T18:15:22+02:00frey_mAnalyse Optimizer outputSince https://gitlab.psi.ch/OPAL/src/merge_requests/102 OPAL writes a Pareto file. It would be nice having some tools to evaluate e.g. the *best* individual etc. A user might give function handles to be evaluated on the data set.Since https://gitlab.psi.ch/OPAL/src/merge_requests/102 OPAL writes a Pareto file. It would be nice having some tools to evaluate e.g. the *best* individual etc. A user might give function handles to be evaluated on the data set.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/38Built ML database from sampler objectives2019-07-03T09:18:40+02:00frey_mBuilt ML database from sampler objectivesCurrently, we can only build ML databases from sampler output with stat-files. However, the sampler can write objectives to the JSON file like the optimizer.Currently, we can only build ML databases from sampler output with stat-files. However, the sampler can write objectives to the JSON file like the optimizer.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/40Simplify interface2019-07-17T15:30:04+02:00frey_mSimplify interfacehttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/412d_halo_projection2019-07-20T15:49:58+02:00frey_m2d_halo_projectionfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/42turn number as argument to compute halo2019-07-20T15:49:58+02:00frey_mturn number as argument to compute haloThe turn number is part of the probe H5 file. It missing however the step number. Therefore, we should be able to call the function with the turn number as well.The turn number is part of the probe H5 file. It missing however the step number. Therefore, we should be able to call the function with the turn number as well.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/43Add merlin 6 template2019-07-18T15:33:50+02:00frey_mAdd merlin 6 templateAdd jobhandler template for Merlin 6Add jobhandler template for Merlin 6frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/45Use function to extract data2019-07-22T20:24:29+02:00frey_mUse function to extract dataInstead of
```Python
data = data[val == attrval]
```
we should use
```Python
data = self._select(data, turns, turn)
```
that checks if ```data``` is not empty.Instead of
```Python
data = data[val == attrval]
```
we should use
```Python
data = self._select(data, turns, turn)
```
that checks if ```data``` is not empty.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/46confidence interval using bootstrap2019-08-15T14:14:07+02:00frey_mconfidence interval using bootstrapWhen we have a surrogate model we might plot a confindence interval. In order to do the confidence interval of a surrogate model the bootstrap method can be applied.When we have a surrogate model we might plot a confindence interval. In order to do the confidence interval of a surrogate model the bootstrap method can be applied.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/47Select turn and bunch number from H52019-07-25T13:59:18+02:00frey_mSelect turn and bunch number from H5Currently we can only select a turn or a bunch from H5. It should be made a bit more general: First select a bunch and then a turn (if available).Currently we can only select a turn or a bunch from H5. It should be made a bit more general: First select a bunch and then a turn (if available).frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/48Additional radial statistics2019-08-15T14:14:07+02:00frey_mAdditional radial statisticsAdd the evaluation of radial moments and the radial projected emittanceAdd the evaluation of radial moments and the radial projected emittancefrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/49SDDS dataset dataframe access2019-08-15T14:14:07+02:00frey_mSDDS dataset dataframe access* use pandas dataframe also in master
* add function to get direct access to the pandas dataframe* use pandas dataframe also in master
* add function to get direct access to the pandas dataframefrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/50Performance improvement in H52019-08-15T14:14:07+02:00frey_mPerformance improvement in H5We currently do not use h5py properly leading to a performance bottleneck for large files.We currently do not use h5py properly leading to a performance bottleneck for large files.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/51Follow-up from "Resolve "Performance improvement in H5""2019-08-07T13:32:46+02:00frey_mFollow-up from "Resolve "Performance improvement in H5""The following discussion from !27 should be addressed:
- [ ] @snuverink_j started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/27#note_11978): (+2 comments)
> Should there be a check on the validity of `ste...The following discussion from !27 should be addressed:
- [ ] @snuverink_j started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/27#note_11978): (+2 comments)
> Should there be a check on the validity of `step` like in `getStepDataset()` ?frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/52add missing import2019-08-07T14:59:44+02:00frey_madd missing importfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/53JobHandler: Use of less used symbol2019-08-12T10:20:07+02:00frey_mJobHandler: Use of less used symbolUse `@` to replace keywords instead of `_`.Use `@` to replace keywords instead of `_`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/54sampler files not properly loaded2019-08-12T16:20:41+02:00frey_msampler files not properly loadedIn case of multiple masters we obtain multiple sampler output files. However, they are not properly parsed causing the program to crash if it tries to access an individual from another master.In case of multiple masters we obtain multiple sampler output files. However, they are not properly parsed causing the program to crash if it tries to access an individual from another master.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/55Plotting style gives error for matplotlib version < 2.12019-08-15T14:14:07+02:00snuverink_jjochem.snuverink@psi.chPlotting style gives error for matplotlib version < 2.1```python
from opal.visualization.styles import load_style
load_style('jupyter')
```
gives an error:
`KeyError: 'axes.formatter.min_exponent is not a valid rc parameter.See rcParams.keys() for a list of valid parameters.'````python
from opal.visualization.styles import load_style
load_style('jupyter')
```
gives an error:
`KeyError: 'axes.formatter.min_exponent is not a valid rc parameter.See rcParams.keys() for a list of valid parameters.'`snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/56Add H5 Probe Jupyter Notebook2019-08-12T17:35:52+02:00frey_mAdd H5 Probe Jupyter Notebookfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/57H5Parser::getNSteps() counts one too many2019-08-13T10:22:48+02:00snuverink_jjochem.snuverink@psi.chH5Parser::getNSteps() counts one too manyThe H5Parser counts one too many, this is because header `Attachment` is in the keys as well:
```python
self.__nsteps = len(self.__h5f.keys())
print(list(self.__h5f.keys()))
['Attachment', 'Step#0']
```
I propose to simply do:
```pyth...The H5Parser counts one too many, this is because header `Attachment` is in the keys as well:
```python
self.__nsteps = len(self.__h5f.keys())
print(list(self.__h5f.keys()))
['Attachment', 'Step#0']
```
I propose to simply do:
```python
self.__nsteps = len(self.__h5f.keys()) - 1 #don't count header
```snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/58update of probe plotting utilities2020-04-07T10:46:34+02:00frey_mupdate of probe plotting utilitiesDue to the changes made in https://gitlab.psi.ch/OPAL/src/merge_requests/161 the plotting functions of the probe dataset (H5) are not valid anymore if the probe is saved with `OPTION,PSDUMPEACHTURN=TRUE`. However, we need to guarantee bo...Due to the changes made in https://gitlab.psi.ch/OPAL/src/merge_requests/161 the plotting functions of the probe dataset (H5) are not valid anymore if the probe is saved with `OPTION,PSDUMPEACHTURN=TRUE`. However, we need to guarantee both versions.https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/59Matplotlib styles update2019-08-15T14:14:07+02:00frey_mMatplotlib styles update`mpl.rcParams['text.latex.unicode']` is deprecated in version > 2.1.`mpl.rcParams['text.latex.unicode']` is deprecated in version > 2.1.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/60Access monitor in file2019-09-05T10:32:14+02:00frey_mAccess monitor in fileThe monitor stat file writes a row per monitor to a SDDS. We should add a routine that allows to return a full row of a stat file.The monitor stat file writes a row per monitor to a SDDS. We should add a routine that allows to return a full row of a stat file.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/61Fix data selection in H5Statistics2019-10-14T15:48:35+02:00frey_mFix data selection in H5StatisticsWhen selecting a turn of a probe H5 file, the full data set is taken instead of just bunch specific if the argument `bunch > -1`.When selecting a turn of a probe H5 file, the full data set is taken instead of just bunch specific if the argument `bunch > -1`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/62SDDS data set: Return a dataframe rather than a raw array2019-09-01T16:08:18+02:00frey_mSDDS data set: Return a dataframe rather than a raw arrayRelated to #60Related to #60frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/63JSON parsing of design variable bounds2019-10-14T15:48:35+02:00frey_mJSON parsing of design variable boundsThe bounds of the design variables are returned as a string, e.g. `'[0, 1]'`, instead of an array.The bounds of the design variables are returned as a string, e.g. `'[0, 1]'`, instead of an array.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/64SDDS dataframe with mixed types2019-10-14T15:48:36+02:00frey_mSDDS dataframe with mixed typesThe Pandas data frame returns only strings if we have mixed datatype. We need to distinguish types.
edit: [minimal stat file without header](/uploads/4cba5b475cf51e80097e9b18fa9de72e/headerremoved.stat)The Pandas data frame returns only strings if we have mixed datatype. We need to distinguish types.
edit: [minimal stat file without header](/uploads/4cba5b475cf51e80097e9b18fa9de72e/headerremoved.stat)frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/65update phase space plotter2019-10-14T15:48:35+02:00frey_mupdate phase space plotterfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/66setup.py update?2019-10-14T15:48:36+02:00ext-neveu_nsetup.py update?While using pyOPALTools with a Steve Hudson at ANL, he got an import/path error related to yaml:
```
File "deap_libe_lcls_gun.py", line 37, in <module>
from fitness import fitness_opal
File "/lcrc/project/MCS-SLAC/shudson/emitta...While using pyOPALTools with a Steve Hudson at ANL, he got an import/path error related to yaml:
```
File "deap_libe_lcls_gun.py", line 37, in <module>
from fitness import fitness_opal
File "/lcrc/project/MCS-SLAC/shudson/emittance_minimization/code/fitness.py", line 17, in <module>
from libEopal import run_opal_nowatch
File "/lcrc/project/MCS-SLAC/shudson/emittance_minimization/code/libEopal.py", line 9, in <module>
from opal.datasets.StatDataset import *
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/__init__.py", line 1, in <module>
from .opal import load_dataset
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/opal.py", line 6, in <module>
from .datasets.filetype import FileType
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/datasets/__init__.py", line 1, in <module>
import opal.datasets.filetype
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/datasets/filetype.py", line 9, in <module>
from opal.utilities.logger import opal_logger
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/utilities/__init__.py", line 1, in <module>
import opal.utilities.logger
File "/lcrc/project/MCS-SLAC/software/pyOPALTools/opal/utilities/logger.py", line 28, in <module>
with open(path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'opal/utilities/log.yaml'
```
He suggested: "adding a package_data clause to the setup.py, that may make the import pick up yaml files.
For example in libensemble I use this:
```package_data={'libensemble.sim_funcs.branin': ['known_minima_and_func_values']},```
to pick up the data file known_minima_and_func_values.
In any case, the PYTHONPATH approach seems to work too."
Does this make sense to do here? Thanks!frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/67make YAML optional2019-10-14T15:44:24+02:00frey_mmake YAML optionalfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/68H5 analysis: select last step if negative step number2020-03-27T07:44:00+01:00frey_mH5 analysis: select last step if negative step numberfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/69Multiple steps in probe2020-04-07T10:41:36+02:00frey_mMultiple steps in probeWe need to update the `ProbePlotter` class since probe H5 files can now store multiple steps.We need to update the `ProbePlotter` class since probe H5 files can now store multiple steps.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/70Follow-up from "Resolve "Multiple steps in probe""2020-03-27T07:44:00+01:00frey_mFollow-up from "Resolve "Multiple steps in probe""The following discussions from !45 should be addressed:
- [x] @snuverink_j started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/45#note_14524): (+1 comment)
> sorry for being late with commenting. You could...The following discussions from !45 should be addressed:
- [x] @snuverink_j started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/45#note_14524): (+1 comment)
> sorry for being late with commenting. You could use the `hypot` function as well: https://docs.scipy.org/doc/numpy/reference/generated/numpy.hypot.html
- [x] @snuverink_j started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/45#note_14527): (+2 comments)
> Can the range become very large? In that case append might not be cpu efficient, and it is better to preallocate the array (e.g. https://stackoverflow.com/questions/2641691/building-up-an-array-in-numpy-scipy-by-iteration-in-python):
>
> ```python
> x = np.zeros(1,self.ds.size)
> for s in range(self.ds.size):
> x[i] = self.ds.getData('x', step=s)
> ```frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/71Use extend instead of append2020-03-27T07:44:00+01:00frey_mUse extend instead of appendThe list append of https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/46 causes wrong behaviour. We need to use list extend instead.The list append of https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/46 causes wrong behaviour. We need to use list extend instead.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/72README: point to wiki2020-03-27T07:44:00+01:00frey_mREADME: point to wikifrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/73SDDSParser has difficulties with OPAL 1.6 stat files2020-02-19T09:09:56+01:00snuverink_jjochem.snuverink@psi.chSDDSParser has difficulties with OPAL 1.6 stat filesThe SDDSParser seems to have problems reading version 1.6 stat files, like e.g. https://gitlab.psi.ch/OPAL/pyOPALTools/blob/da35eef62da7f618b6a51194a7710c8dc9c3a823/opal/test/Gantry2/70MeV_Gantry2_track16.stat
Minimal example (with pand...The SDDSParser seems to have problems reading version 1.6 stat files, like e.g. https://gitlab.psi.ch/OPAL/pyOPALTools/blob/da35eef62da7f618b6a51194a7710c8dc9c3a823/opal/test/Gantry2/70MeV_Gantry2_track16.stat
Minimal example (with pandas 0.25.1):
```python
import pandas as pd
keys = ['t', 's', 'numParticles', 'charge', 'energy', 'rms_x', 'rms_y', 'rms_s', 'rms_px', 'rms_py', 'rms_ps', 'emit_x', 'emit_y', 'emit_s', 'mean_x', 'mean_y', 'mean_s', 'max_x', 'max_y', 'max_s', 'xpx', 'ypy', 'zpz', 'notused1', 'notused2', 'Dx', 'DDx', 'Dy', 'DDy', 'Bx_head', 'By_head', 'Bz_head', 'Ex_head', 'Ey_head', 'Ez_head', 'Bx_ref', 'By_ref', 'Bz_ref', 'Ex_ref', 'Ey_ref', 'Ez_ref', 'Bx_tail', 'By_tail', 'Bz_tail', 'Ex_tail', 'Ey_tail', 'Ez_tail', 'dE', 'partsOutside']
pd.read_csv('Gantry2/70MeV_Gantry2_track16.stat', skiprows=56,
sep='\s+', names=keys, index_col=False)
```
```
t s numParticles charge energy rms_x rms_y rms_s rms_px rms_py ... Ey_ref Ez_ref Bx_tail By_tail Bz_tail Ex_tail Ey_tail Ez_tail dE partsOutside
0 rows × 49 columns
```
I think this really should have worked, but it seems the trailing space in the header string causes problems (even though skiprows specifies these should be skipped!).
Minimal file that has the same issue:
```ascii
description="26 Dispersion in x "
1.000000000000000e-12 3.159271238650963e+01 1000
```
```python
df = pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+', names=['t','s','numParticles'])
df.size
0
```
It can be fixed by forcing to use the "python" engine:
```python
pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+\t', names=['t','s','numParticles'])
df.size
3
```snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/74Selection of bunch in plot_probe_histogram2020-03-27T07:44:00+01:00frey_mSelection of bunch in plot_probe_histogramWe need to be able to select a single bunch in case of multi-bunch simulations.We need to be able to select a single bunch in case of multi-bunch simulations.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/75H5Parser method to close files2020-03-26T17:43:21+01:00snuverink_jjochem.snuverink@psi.chH5Parser method to close filesFrom Sebastian Thomas:
> I have a question concerning the H5Parser https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/parser/H5Parser.py
>
> I'm not an extremely experienced coder, but shouldn't the file opened in H5Parser.parse("f...From Sebastian Thomas:
> I have a question concerning the H5Parser https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/parser/H5Parser.py
>
> I'm not an extremely experienced coder, but shouldn't the file opened in H5Parser.parse("file") via
>
> `self.__h5f = h5py.File(fname, 'r')`
>
> be closed again? Wenn I use the parser on an H5 file in a juyter notebook, I run into some problems because for example OPAL tries to delete the monitor output, but is not allowed to, due to the file "beeing in use". I ask mainly out of curiosity.
Since h5py 2.4 opened file will automatically close:
http://docs.h5py.org/en/stable/whatsnew/2.4.html#files-will-now-auto-close
However, if the H5Parser still exists, this will not work, and it would be nice to have a manual option to close the file.
Proposal is to add a specific close method to H5Parser, as discussed with @frey\_m.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/76Additional arguments for the H5 version of plot_probe_histogram2020-04-20T08:09:39+02:00frey_mAdditional arguments for the H5 version of plot_probe_histogramBegin and end of steps to read by
```Python
begin = kwargs.pop('begin', 0)
end = kwargs.pop('end', self.ds.size)
```Begin and end of steps to read by
```Python
begin = kwargs.pop('begin', 0)
end = kwargs.pop('end', self.ds.size)
```frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/77Add licenses2020-04-08T17:05:46+02:00frey_mAdd licensesWe need to add the licenses of the different distributions to pyOPALTools.We need to add the licenses of the different distributions to pyOPALTools.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/78File header2020-04-09T08:32:13+02:00frey_mFile headerWe should add file headers like in OPAL, i.e., OPAL/src#482.We should add file headers like in OPAL, i.e., OPAL/src#482.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/79Add Jupyter Notebooks to Sphinx documentation2020-04-20T11:08:56+02:00snuverink_jjochem.snuverink@psi.chAdd Jupyter Notebooks to Sphinx documentationAs discussed in https://gitlab.psi.ch/OPAL/pyOPALTools/issues/32#note_19574, it would be good to include the jupyter notebooks in the sphinx documentation.As discussed in https://gitlab.psi.ch/OPAL/pyOPALTools/issues/32#note_19574, it would be good to include the jupyter notebooks in the sphinx documentation.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/80module uq : chaospy (3.2.9) chaospy.dot does not exist2020-04-14T10:44:11+02:00snuverink_jjochem.snuverink@psi.chmodule uq : chaospy (3.2.9) chaospy.dot does not existWhen I run the confidence_interval notebook I get the following error:
```
AttributeError Traceback (most recent call last)
<ipython-input-2-637cb2ef6afe> in <module>
16
17 y_train, y_pred, y_lo, y_...When I run the confidence_interval notebook I get the following error:
```
AttributeError Traceback (most recent call last)
<ipython-input-2-637cb2ef6afe> in <module>
16
17 y_train, y_pred, y_lo, y_up = uq.confidence_interval(x_train, y_train, alpha=alpha,
---> 18 n_boot=n_boot, sample=sample)
19
20 plt.figure(figsize=(16, 9))
~/Documents/OPAL/pyOPALTools/surrogate/uq.py in confidence_interval(self, x, y, alpha, **kwargs)
129
130 lo, up = bs.confidence_interval(alpha=alpha)
--> 131 lo_pce = cp.dot(poly, np.asarray(lo))
132 up_pce = cp.dot(poly, np.asarray(up))
133
AttributeError: module 'chaospy' has no attribute 'dot'
```
For this I had to add the following fix to bootstrap.py:
```diff
- return 2.0 * fitted - qlo, 2.0 * fitted - qhi
+ return 2.0 * np.asarray(fitted) - qlo, 2.0 * np.asarray(fitted) - qhi
```frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/81Auto deploy sphinx documentation2020-05-28T09:45:23+02:00snuverink_jjochem.snuverink@psi.chAuto deploy sphinx documentationSimilar to the OPAL manual, the sphinx documentation (#32) for pyOPALTools should be generated automatically on http://amas.web.psi.ch/docs/pyOPALTools/index.html
Note that the jupyter notebooks are now included (#79 !57) in the documen...Similar to the OPAL manual, the sphinx documentation (#32) for pyOPALTools should be generated automatically on http://amas.web.psi.ch/docs/pyOPALTools/index.html
Note that the jupyter notebooks are now included (#79 !57) in the documentation. The python package `nbsphinx` is needed, as well as `pandoc` and `latex`. There are some issues with the latex labels with matplotlib, therefore not all notebooks might build without problems, but this will then be fixed later.gsellgsellhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/82Sampler and Optimizer files of OPAL-2.2.0 cannot be loaded2020-04-22T16:05:30+02:00frey_mSampler and Optimizer files of OPAL-2.2.0 cannot be loadedSampler and optimizer files that are generated with OPAL-2.2.0 cannot be loaded due to the version number. Adding the line
```Python
self.__version_support = {
'2.1.0': self.__parse_version_2_1_0,
'2.2.0': self.__parse_version_2_...Sampler and optimizer files that are generated with OPAL-2.2.0 cannot be loaded due to the version number. Adding the line
```Python
self.__version_support = {
'2.1.0': self.__parse_version_2_1_0,
'2.2.0': self.__parse_version_2_1_0 # OPAL-2.2.0 support
}
```
would fix the problem. However, we should do a more general approach in order to avoid this error in future.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/83Documentation: import opal error for jupyter notebooks2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chDocumentation: import opal error for jupyter notebooksThe jupyter notebooks (e.g. http://amas.web.psi.ch/docs/pyOPALTools/Cyclotron.html) have import errors:
```python
from opal.opal import load_dataset, filetype
ds = load_dataset('./../../tests/testData/', ftype=filetype.TRACK_ORBIT)
---...The jupyter notebooks (e.g. http://amas.web.psi.ch/docs/pyOPALTools/Cyclotron.html) have import errors:
```python
from opal.opal import load_dataset, filetype
ds = load_dataset('./../../tests/testData/', ftype=filetype.TRACK_ORBIT)
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-2f139e8985fc> in <module>
----> 1 from opal.opal import load_dataset, filetype
ModuleNotFoundError: No module named 'opal'
```
The jupyter notebooks are generated by nbsphinx, but this doesn't copy the sys.path from sphinx, but only the environment variables, so the PYTHONPATH should be set in the main script `conf.py`. See also the documentation: https://nbsphinx.readthedocs.io/en/0.7.0/configuring-kernels.html#Environment-Variablessnuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/84Documentation: add last changed date2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chDocumentation: add last changed dateThe documentation is now automatically generated (#81), however there is no last changed date, so one is not sure if an update was successful.The documentation is now automatically generated (#81), however there is no last changed date, so one is not sure if an update was successful.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/85requirements.txt and setup.py out of date2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chrequirements.txt and setup.py out of dateThe requirements.txt and setup.py are out of date.The requirements.txt and setup.py are out of date.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/86Add new dataset for field data on grid2020-06-08T16:50:33+02:00frey_mAdd new dataset for field data on gridOPAL writes the field data on the grid to file if `DBG_SCALARFIELD` is enabled. We should provide a parser, dataset and plotting functions.OPAL writes the field data on the grid to file if `DBG_SCALARFIELD` is enabled. We should provide a parser, dataset and plotting functions.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/87Labels with # not correctly interpreted by tex2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chLabels with # not correctly interpreted by texplot labels like `plt.ylabel('#cores')` are not correctly interpreted in Tex mode (see, e.g. http://amas.web.psi.ch/docs/pyOPALTools/RingMultiBunch.html).
In https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/SamplerP...plot labels like `plt.ylabel('#cores')` are not correctly interpreted in Tex mode (see, e.g. http://amas.web.psi.ch/docs/pyOPALTools/RingMultiBunch.html).
In https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/SamplerPlotter.py#L201 we do
```python
isTex = mpl.rcParams['text.usetex']
xlabel = '#samples'
if isTex:
xlabel = '\\' + xlabel
```
Do we need to do this everywhere, or should we create an "escape_tex" method, or is there a better solution?
cc: @frey\_mfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/89Wrong implementation of functions in H5Dataset.2020-11-06T08:42:53+01:00frey_mWrong implementation of functions in H5Dataset.`H5Dataset.getMonitorDataFrame` and `H5Dataset.getH5DataFrame` are not properly implemented.`H5Dataset.getMonitorDataFrame` and `H5Dataset.getH5DataFrame` are not properly implemented.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/90Follow-up from "Resolve "H5Parser method to close files""2020-06-09T15:31:26+02:00frey_mFollow-up from "Resolve "H5Parser method to close files""The following discussion from !51 should be addressed:
- [ ] @frey_m started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/51#note_22727): (+12 comments)
> @snuverink\_j I get an error when `H5Parser.close` ...The following discussion from !51 should be addressed:
- [ ] @frey_m started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/51#note_22727): (+12 comments)
> @snuverink\_j I get an error when `H5Parser.close` is called. My script only loads a H5 file and calls the print function. I use the h5py version 2.10.0. Have you experienced any error like this?
>
> ```bash
> Exception ignored in: <bound method H5Dataset.__del__ of <opal.datasets.H5Dataset.H5Dataset object at 0x7f08da008f98>>
> Traceback (most recent call last):
> File "/home/matthias/Documents/projects/pyOPALTools/opal/datasets/H5Dataset.py", line 98, in __del__
> File "/home/matthias/Documents/projects/pyOPALTools/opal/parser/H5Parser.py", line 59, in close
> File "/home/matthias/Documents/projects/pyOPALTools/pyOPALTools.venv/lib/python3.5/site-packages/h5py/_hl/files.py", line 432, in close
> File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
> File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
> File "h5py/h5f.pyx", line 267, in h5py.h5f.get_obj_ids
> File "h5py/h5i.pyx", line 37, in h5py.h5i.wrap_identifier
> SystemError: Parent module 'h5py' not loaded, cannot perform relative import
> ```
**Possible fix:**
Replacing the line `self.__h5f.close()` in `H5Parser.close` by `del self.__h5f`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/91Installation with setup.py fails2020-11-06T08:42:53+01:00frey_mInstallation with setup.py failsThe installation with `setup.py` fails with the error message:
```bash
package init file 'jobhandler\__init__.py' not found (or not a regular file)
error: package directory 'opal\visualization\stylesoptPilot' does not exist
```
The reaso...The installation with `setup.py` fails with the error message:
```bash
package init file 'jobhandler\__init__.py' not found (or not a regular file)
error: package directory 'opal\visualization\stylesoptPilot' does not exist
```
The reason is a missing comma in the `PACKAGES` list.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/92Missing sphinx entries for FieldDataset2020-11-06T08:42:53+01:00frey_mMissing sphinx entries for FieldDatasetfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/93Update on FieldDataset2020-06-18T13:29:01+02:00frey_mUpdate on FieldDatasetThis issue should
- add the file FieldAnalysis.py
- provide the line plot functionality to FieldPlotter
- do some changes on FieldParser and FieldDatasetThis issue should
- add the file FieldAnalysis.py
- provide the line plot functionality to FieldPlotter
- do some changes on FieldParser and FieldDatasetfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/95index mask evaluated at wrong location in FieldPlotter2020-11-06T08:42:54+01:00frey_mindex mask evaluated at wrong location in FieldPlotterThe index mask is evaluated at the wrong location in the line plot funtion of `FieldPlotter`.The index mask is evaluated at the wrong location in the line plot funtion of `FieldPlotter`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/96Simplify file type checking2020-07-21T10:13:46+02:00frey_mSimplify file type checkingThe file type checking is quite error-prone. I suggest simplifying the interface with a function `check_file` for each parser.The file type checking is quite error-prone. I suggest simplifying the interface with a function `check_file` for each parser.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/97Missing colon2020-11-06T08:42:54+01:00frey_mMissing colonfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/98Integrate UQ interface2020-09-15T11:38:14+02:00frey_mIntegrate UQ interfaceUpgrade the surrogate package with the new UQ interface.Upgrade the surrogate package with the new UQ interface.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/99Update LossParser after update2021-06-01T09:40:31+02:00snuverink_jjochem.snuverink@psi.chUpdate LossParser after updatein src!494 (for OPAL 2.6) the ASCII loss output was updated from:
```
# Element: NAME , x (m), y (m), z (m), px ( ), py ( ), pz ( ), id
```
to
```
# x (m), y (m), z (m), px ( ), py ( ), pz ( ), id;
```
The LossParser should b...in src!494 (for OPAL 2.6) the ASCII loss output was updated from:
```
# Element: NAME , x (m), y (m), z (m), px ( ), py ( ), pz ( ), id
```
to
```
# x (m), y (m), z (m), px ( ), py ( ), pz ( ), id;
```
The LossParser should be adapted such that it reads in the new and the old format.snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/101tex font cannot be used in matplotlib after import pyOPAL2021-12-19T23:37:51+01:00ext-piot_ptex font cannot be used in matplotlib after import pyOPALThere seems to be an issue with adding tex symbols in matplotlib after import pyOPAL. The simple code snippet below
```
from matplotlib import rc
from opal.opal import load_dataset
import matplotlib.pyplot as plt
FontSize=22
FontSizeLab...There seems to be an issue with adding tex symbols in matplotlib after import pyOPAL. The simple code snippet below
```
from matplotlib import rc
from opal.opal import load_dataset
import matplotlib.pyplot as plt
FontSize=22
FontSizeLabelAxis=22
rc('xtick',labelsize=FontSize)
rc('ytick',labelsize=FontSize)
rc('text', usetex=True)
x=np.linspace(-10,10,100)
plt.plot (x,x**2)
plt.xlabel (r'$x$ ')
```
give the error
```
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-4-11cccf49f543> in <module>
14 x=np.linspace(-10,10,100)
15 plt.plot (x,x**2)
---> 16 plt.xlabel (r'$x$ ')
/usr/lib64/python3.9/site-packages/matplotlib/pyplot.py in xlabel(xlabel, fontdict, labelpad, loc, **kwargs)
3113 @_copy_docstring_and_deprecators(Axes.set_xlabel)
3114 def xlabel(xlabel, fontdict=None, labelpad=None, *, loc=None, **kwargs):
-> 3115 return gca().set_xlabel(
3116 xlabel, fontdict=fontdict, labelpad=labelpad, loc=loc,
3117 **kwargs)
/usr/lib64/python3.9/site-packages/matplotlib/axes/_axes.py in set_xlabel(self, xlabel, fontdict, labelpad, loc, **kwargs)
245 elif loc == 'right':
246 kwargs.update(x=1, horizontalalignment='right')
--> 247 return self.xaxis.set_label_text(xlabel, fontdict, **kwargs)
248
249 def get_ylabel(self):
/usr/local/lib/python3.9/site-packages/opal-1.0.0-py3.9.egg/opal/utilities/wrapper.py in run(*args, **kwargs)
35 @functools.wraps(fun)
36 def run(*args, **kwargs):
---> 37 return new_fun(fun, *args, **kwargs)
38 return run
39
/usr/local/lib/python3.9/site-packages/opal-1.0.0-py3.9.egg/opal/utilities/wrapper.py in new_label(fun, *args, **kwargs)
67 # we need to add the signs otherwise LaTex formulas
68 # are not properly compiled
---> 69 re._alphanum_str = re._alphanum_str.union(signs)
70 lst[idx] = re.escape(lst[idx])
71 # remove signs again
AttributeError: module 're' has no attribute '_alphanum_str'
```ext-piot_pext-piot_p