SDDSParser has difficulties with OPAL 1.6 stat files
The SDDSParser seems to have problems reading version 1.6 stat files, like e.g. https://gitlab.psi.ch/OPAL/pyOPALTools/blob/da35eef62da7f618b6a51194a7710c8dc9c3a823/opal/test/Gantry2/70MeV_Gantry2_track16.stat
Minimal example (with pandas 0.25.1):
import pandas as pd
keys = ['t', 's', 'numParticles', 'charge', 'energy', 'rms_x', 'rms_y', 'rms_s', 'rms_px', 'rms_py', 'rms_ps', 'emit_x', 'emit_y', 'emit_s', 'mean_x', 'mean_y', 'mean_s', 'max_x', 'max_y', 'max_s', 'xpx', 'ypy', 'zpz', 'notused1', 'notused2', 'Dx', 'DDx', 'Dy', 'DDy', 'Bx_head', 'By_head', 'Bz_head', 'Ex_head', 'Ey_head', 'Ez_head', 'Bx_ref', 'By_ref', 'Bz_ref', 'Ex_ref', 'Ey_ref', 'Ez_ref', 'Bx_tail', 'By_tail', 'Bz_tail', 'Ex_tail', 'Ey_tail', 'Ez_tail', 'dE', 'partsOutside']
pd.read_csv('Gantry2/70MeV_Gantry2_track16.stat', skiprows=56,
sep='\s+', names=keys, index_col=False)
t s numParticles charge energy rms_x rms_y rms_s rms_px rms_py ... Ey_ref Ez_ref Bx_tail By_tail Bz_tail Ex_tail Ey_tail Ez_tail dE partsOutside
0 rows × 49 columns
I think this really should have worked, but it seems the trailing space in the header string causes problems (even though skiprows specifies these should be skipped!).
Minimal file that has the same issue:
description="26 Dispersion in x "
1.000000000000000e-12 3.159271238650963e+01 1000
df = pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+', names=['t','s','numParticles'])
df.size
0
It can be fixed by forcing to use the "python" engine:
pd.read_csv("Gantry2/example.txt", skiprows=1, sep='\s+\t', names=['t','s','numParticles'])
df.size
3
Edited by snuverink_j