pyOPALTools issueshttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues2019-04-01T15:39:50+02:00https://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/24Switching between plotting styles does not work properly2019-04-01T15:39:50+02:00frey_mSwitching between plotting styles does not work properlyWhen calling for example the ```jupyter``` style and then the ```poster``` style, the ```matplotlib.rcParams``` are not all set.When calling for example the ```jupyter``` style and then the ```poster``` style, the ```matplotlib.rcParams``` are not all set.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/34Time attribute in h5 is not recognised2019-06-19T08:22:36+02:00adelmannTime attribute in h5 is not recognisedTime attribute in h5, created by a monitor is not recognisedTime attribute in h5, created by a monitor is not recognisedadelmannadelmannhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/35Optimizer: Pareto fron json file2019-06-24T09:55:38+02:00frey_mOptimizer: Pareto fron json fileThe optimizer data set is not able to handle the additional file.The optimizer data set is not able to handle the additional file.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/52add missing import2019-08-07T14:59:44+02:00frey_madd missing importfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/54sampler files not properly loaded2019-08-12T16:20:41+02:00frey_msampler files not properly loadedIn case of multiple masters we obtain multiple sampler output files. However, they are not properly parsed causing the program to crash if it tries to access an individual from another master.In case of multiple masters we obtain multiple sampler output files. However, they are not properly parsed causing the program to crash if it tries to access an individual from another master.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/55Plotting style gives error for matplotlib version < 2.12019-08-15T14:14:07+02:00snuverink_jjochem.snuverink@psi.chPlotting style gives error for matplotlib version < 2.1```python
from opal.visualization.styles import load_style
load_style('jupyter')
```
gives an error:
`KeyError: 'axes.formatter.min_exponent is not a valid rc parameter.See rcParams.keys() for a list of valid parameters.'````python
from opal.visualization.styles import load_style
load_style('jupyter')
```
gives an error:
`KeyError: 'axes.formatter.min_exponent is not a valid rc parameter.See rcParams.keys() for a list of valid parameters.'`snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/57H5Parser::getNSteps() counts one too many2019-08-13T10:22:48+02:00snuverink_jjochem.snuverink@psi.chH5Parser::getNSteps() counts one too manyThe H5Parser counts one too many, this is because header `Attachment` is in the keys as well:
```python
self.__nsteps = len(self.__h5f.keys())
print(list(self.__h5f.keys()))
['Attachment', 'Step#0']
```
I propose to simply do:
```pyth...The H5Parser counts one too many, this is because header `Attachment` is in the keys as well:
```python
self.__nsteps = len(self.__h5f.keys())
print(list(self.__h5f.keys()))
['Attachment', 'Step#0']
```
I propose to simply do:
```python
self.__nsteps = len(self.__h5f.keys()) - 1 #don't count header
```snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/61Fix data selection in H5Statistics2019-10-14T15:48:35+02:00frey_mFix data selection in H5StatisticsWhen selecting a turn of a probe H5 file, the full data set is taken instead of just bunch specific if the argument `bunch > -1`.When selecting a turn of a probe H5 file, the full data set is taken instead of just bunch specific if the argument `bunch > -1`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/63JSON parsing of design variable bounds2019-10-14T15:48:35+02:00frey_mJSON parsing of design variable boundsThe bounds of the design variables are returned as a string, e.g. `'[0, 1]'`, instead of an array.The bounds of the design variables are returned as a string, e.g. `'[0, 1]'`, instead of an array.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/64SDDS dataframe with mixed types2019-10-14T15:48:36+02:00frey_mSDDS dataframe with mixed typesThe Pandas data frame returns only strings if we have mixed datatype. We need to distinguish types.
edit: [minimal stat file without header](/uploads/4cba5b475cf51e80097e9b18fa9de72e/headerremoved.stat)The Pandas data frame returns only strings if we have mixed datatype. We need to distinguish types.
edit: [minimal stat file without header](/uploads/4cba5b475cf51e80097e9b18fa9de72e/headerremoved.stat)frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/71Use extend instead of append2020-03-27T07:44:00+01:00frey_mUse extend instead of appendThe list append of https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/46 causes wrong behaviour. We need to use list extend instead.The list append of https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/46 causes wrong behaviour. We need to use list extend instead.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/73SDDSParser has difficulties with OPAL 1.6 stat files2020-02-19T09:09:56+01:00snuverink_jjochem.snuverink@psi.chSDDSParser has difficulties with OPAL 1.6 stat filesThe SDDSParser seems to have problems reading version 1.6 stat files, like e.g. https://gitlab.psi.ch/OPAL/pyOPALTools/blob/da35eef62da7f618b6a51194a7710c8dc9c3a823/opal/test/Gantry2/70MeV_Gantry2_track16.stat
Minimal example (with pand...The SDDSParser seems to have problems reading version 1.6 stat files, like e.g. https://gitlab.psi.ch/OPAL/pyOPALTools/blob/da35eef62da7f618b6a51194a7710c8dc9c3a823/opal/test/Gantry2/70MeV_Gantry2_track16.stat
Minimal example (with pandas 0.25.1):
```python
import pandas as pd
keys = ['t', 's', 'numParticles', 'charge', 'energy', 'rms_x', 'rms_y', 'rms_s', 'rms_px', 'rms_py', 'rms_ps', 'emit_x', 'emit_y', 'emit_s', 'mean_x', 'mean_y', 'mean_s', 'max_x', 'max_y', 'max_s', 'xpx', 'ypy', 'zpz', 'notused1', 'notused2', 'Dx', 'DDx', 'Dy', 'DDy', 'Bx_head', 'By_head', 'Bz_head', 'Ex_head', 'Ey_head', 'Ez_head', 'Bx_ref', 'By_ref', 'Bz_ref', 'Ex_ref', 'Ey_ref', 'Ez_ref', 'Bx_tail', 'By_tail', 'Bz_tail', 'Ex_tail', 'Ey_tail', 'Ez_tail', 'dE', 'partsOutside']
pd.read_csv('Gantry2/70MeV_Gantry2_track16.stat', skiprows=56,
sep='\s+', names=keys, index_col=False)
```
```
t s numParticles charge energy rms_x rms_y rms_s rms_px rms_py ... Ey_ref Ez_ref Bx_tail By_tail Bz_tail Ex_tail Ey_tail Ez_tail dE partsOutside
0 rows × 49 columns
```
I think this really should have worked, but it seems the trailing space in the header string causes problems (even though skiprows specifies these should be skipped!).
Minimal file that has the same issue:
```ascii
description="26 Dispersion in x "
1.000000000000000e-12 3.159271238650963e+01 1000
```
```python
df = pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+', names=['t','s','numParticles'])
df.size
0
```
It can be fixed by forcing to use the "python" engine:
```python
pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+\t', names=['t','s','numParticles'])
df.size
3
```snuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/82Sampler and Optimizer files of OPAL-2.2.0 cannot be loaded2020-04-22T16:05:30+02:00frey_mSampler and Optimizer files of OPAL-2.2.0 cannot be loadedSampler and optimizer files that are generated with OPAL-2.2.0 cannot be loaded due to the version number. Adding the line
```Python
self.__version_support = {
'2.1.0': self.__parse_version_2_1_0,
'2.2.0': self.__parse_version_2_...Sampler and optimizer files that are generated with OPAL-2.2.0 cannot be loaded due to the version number. Adding the line
```Python
self.__version_support = {
'2.1.0': self.__parse_version_2_1_0,
'2.2.0': self.__parse_version_2_1_0 # OPAL-2.2.0 support
}
```
would fix the problem. However, we should do a more general approach in order to avoid this error in future.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/83Documentation: import opal error for jupyter notebooks2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chDocumentation: import opal error for jupyter notebooksThe jupyter notebooks (e.g. http://amas.web.psi.ch/docs/pyOPALTools/Cyclotron.html) have import errors:
```python
from opal.opal import load_dataset, filetype
ds = load_dataset('./../../tests/testData/', ftype=filetype.TRACK_ORBIT)
---...The jupyter notebooks (e.g. http://amas.web.psi.ch/docs/pyOPALTools/Cyclotron.html) have import errors:
```python
from opal.opal import load_dataset, filetype
ds = load_dataset('./../../tests/testData/', ftype=filetype.TRACK_ORBIT)
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-2f139e8985fc> in <module>
----> 1 from opal.opal import load_dataset, filetype
ModuleNotFoundError: No module named 'opal'
```
The jupyter notebooks are generated by nbsphinx, but this doesn't copy the sys.path from sphinx, but only the environment variables, so the PYTHONPATH should be set in the main script `conf.py`. See also the documentation: https://nbsphinx.readthedocs.io/en/0.7.0/configuring-kernels.html#Environment-Variablessnuverink_jjochem.snuverink@psi.chsnuverink_jjochem.snuverink@psi.chhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/87Labels with # not correctly interpreted by tex2020-11-06T08:42:53+01:00snuverink_jjochem.snuverink@psi.chLabels with # not correctly interpreted by texplot labels like `plt.ylabel('#cores')` are not correctly interpreted in Tex mode (see, e.g. http://amas.web.psi.ch/docs/pyOPALTools/RingMultiBunch.html).
In https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/SamplerP...plot labels like `plt.ylabel('#cores')` are not correctly interpreted in Tex mode (see, e.g. http://amas.web.psi.ch/docs/pyOPALTools/RingMultiBunch.html).
In https://gitlab.psi.ch/OPAL/pyOPALTools/blob/master/opal/visualization/SamplerPlotter.py#L201 we do
```python
isTex = mpl.rcParams['text.usetex']
xlabel = '#samples'
if isTex:
xlabel = '\\' + xlabel
```
Do we need to do this everywhere, or should we create an "escape_tex" method, or is there a better solution?
cc: @frey\_mfrey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/89Wrong implementation of functions in H5Dataset.2020-11-06T08:42:53+01:00frey_mWrong implementation of functions in H5Dataset.`H5Dataset.getMonitorDataFrame` and `H5Dataset.getH5DataFrame` are not properly implemented.`H5Dataset.getMonitorDataFrame` and `H5Dataset.getH5DataFrame` are not properly implemented.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/90Follow-up from "Resolve "H5Parser method to close files""2020-06-09T15:31:26+02:00frey_mFollow-up from "Resolve "H5Parser method to close files""The following discussion from !51 should be addressed:
- [ ] @frey_m started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/51#note_22727): (+12 comments)
> @snuverink\_j I get an error when `H5Parser.close` ...The following discussion from !51 should be addressed:
- [ ] @frey_m started a [discussion](https://gitlab.psi.ch/OPAL/pyOPALTools/merge_requests/51#note_22727): (+12 comments)
> @snuverink\_j I get an error when `H5Parser.close` is called. My script only loads a H5 file and calls the print function. I use the h5py version 2.10.0. Have you experienced any error like this?
>
> ```bash
> Exception ignored in: <bound method H5Dataset.__del__ of <opal.datasets.H5Dataset.H5Dataset object at 0x7f08da008f98>>
> Traceback (most recent call last):
> File "/home/matthias/Documents/projects/pyOPALTools/opal/datasets/H5Dataset.py", line 98, in __del__
> File "/home/matthias/Documents/projects/pyOPALTools/opal/parser/H5Parser.py", line 59, in close
> File "/home/matthias/Documents/projects/pyOPALTools/pyOPALTools.venv/lib/python3.5/site-packages/h5py/_hl/files.py", line 432, in close
> File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
> File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
> File "h5py/h5f.pyx", line 267, in h5py.h5f.get_obj_ids
> File "h5py/h5i.pyx", line 37, in h5py.h5i.wrap_identifier
> SystemError: Parent module 'h5py' not loaded, cannot perform relative import
> ```
**Possible fix:**
Replacing the line `self.__h5f.close()` in `H5Parser.close` by `del self.__h5f`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/91Installation with setup.py fails2020-11-06T08:42:53+01:00frey_mInstallation with setup.py failsThe installation with `setup.py` fails with the error message:
```bash
package init file 'jobhandler\__init__.py' not found (or not a regular file)
error: package directory 'opal\visualization\stylesoptPilot' does not exist
```
The reaso...The installation with `setup.py` fails with the error message:
```bash
package init file 'jobhandler\__init__.py' not found (or not a regular file)
error: package directory 'opal\visualization\stylesoptPilot' does not exist
```
The reason is a missing comma in the `PACKAGES` list.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/95index mask evaluated at wrong location in FieldPlotter2020-11-06T08:42:54+01:00frey_mindex mask evaluated at wrong location in FieldPlotterThe index mask is evaluated at the wrong location in the line plot funtion of `FieldPlotter`.The index mask is evaluated at the wrong location in the line plot funtion of `FieldPlotter`.frey_mfrey_mhttps://gitlab.psi.ch/OPAL/pyOPALTools/-/issues/97Missing colon2020-11-06T08:42:54+01:00frey_mMissing colonfrey_mfrey_m