Add code for 1D detector

Created by: ivan-usov

Created by: ivan-usov

@JakHolzer , I created a pull request will all changes grouped on the same branch. This way it will be easier to introduce changes, that might affect all files at the same time. You should be able to push to that branch (det1d) too.

Created by: JakHolzer

I think it should be fine. The ccl files does not have the headers and there is only one data column, rest is numbers or is calculated, not read. So I think its alright, important think is that you know what is what :)

út 15. 9. 2020 v 16:46 odesílatel Ivan Usov notifications@github.com napsal:

@ivan-usov commented on this pull request.

In pyzebra/load_1D.py https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r488727539 :

•            variable = variable.strip()

•            if variable in META_VARS_FLOAT:

•                det_variables["meta"][variable] = float(value)

•            elif variable in META_VARS_STR:

•                det_variables["meta"][variable] = str(value)[:-1].strip()

•            elif variable in META_UB_MATRIX:

•                det_variables["meta"][variable] = re.findall(r"[-+]?\d*\.\d+|\d+", str(value))

•        if "#data" in line:

•            # this is the end of metadata and the start of data section

•            break

•    # read data

•    if det_variables["file_type"] == "ccl":

•        decimal = list()

•        data = infile.readlines()

Btw, that commit also renames "omega" into "om", and "counts" into "Counts", that is consistent with the column names in dat files. I hope it doesn't break you other code, otherwise let me know what names you prefer, it will be easy to fix, e.g. with

row_names = next(infile).split()

row_names = ["NP", "omega", "counts", ...]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r488727539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM2BBRVPO2FLEFKK74RXULLSF545RANCNFSM4RNBW7BQ .

Created by: ivan-usov

These paddings look artificial to me. Do you think it was just tabs between numbers, and the different padding sizes are because of different lengths of those written numbers?

Created by: JakHolzer

I'm sorry, I think that the answer I send from email got stuck somewhere. The files look very organized, so I think its really read line by line in this and first 6 chars correspond to number and so on. Oksana would like to keep it this way.

Created by: JakHolzer

Well, the structure of the files seems very uniform and aligned, so I wanted to keep that. I have never worked with the software that uses it, so I'm not sure how it reads the data. My suggestion is to leave it like this and we can ask Oksana to try load one file with padding and second with tabs to see if it works.

st 16. 9. 2020 v 11:27 odesílatel Ivan Usov notifications@github.com napsal:

@ivan-usov commented on this pull request.

In pyzebra/comm_export.py https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r489296559 :

•        corr_value = np.abs(value * np.sin(ang1) * np.cos(ang2))

•        return corr_value

+def export_comm(data, path, lorentz=False):

• """exports data in the *.comm format
• :param lorentz: perform Lorentz correction
• :param path: path to file + name
• :arg data - data to export, is dict after peak fitting
• """
• align = ">"
• if data["meta"]["indices"] == "hkl":

•    extension = ".comm"

•    padding = [6, 4, 10, 8]

These paddings look artificial to me. Do you think it was just tabs between numbers, and the different padding sizes are because of different lengths of those written numbers?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/paulscherrerinstitute/pyzebra/pull/11#pullrequestreview-489441678, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM2BBRU7NNDFJCK25MZNFQTSGCAJNANCNFSM4RNBW7BQ .

Created by: JakHolzer

the detectorDistance is a number in ccl files, but in dat files its date, which breaks the code. I asked Oksana if it can be fixed in the software that produces the *.dat files. If it for whatever reason cant be fixed, I suggest to move it to META_VARS_STR.

Created by: JakHolzer

Review: Approved

Created by: ivan-usov

@JakHolzer , I would suggest not to extend this PR into parametric study, and merge it rather sooner than later. Do you have changes that are not related to parametric study still to be added here?

Created by: JakHolzer

None that I can think of at the moment. I think that parametric study is now complete (hopefully), maybe some minor additions such as saving the data if we agree how to do it. Nonetheless Romain would like to fit more peaks, which was not what the ccl code was written for, so there might come some changes into fit2 again if I find a way how to make it generic.

st 14. 10. 2020 v 15:52 odesílatel Ivan Usov notifications@github.com napsal:

@JakHolzer https://github.com/JakHolzer , I would suggest not to extend this PR into parametric study, and merge it rather sooner than later. Do you have changes that are not related to parametric study still to be added here?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/paulscherrerinstitute/pyzebra/pull/11#issuecomment-708417862, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM2BBRQJ3Y5KD7PXEU2R4JTSKWUIRANCNFSM4RNBW7BQ .

Created by: ivan-usov

@JakHolzer , could you confirm that this is correct, especially when the number of measurements is odd

Created by: JakHolzer

I believe so, I've tested that now on some model data and it seems everything is in order. The omega angle in the first line (not the array) is centre of the the scan and doesent have to be in the omega array. Ive tried it on odd and even numbers and mean of omega array is equal to omega value. for pints in 20, 21: omega = 31.4 step = 0.1

om = np.linspace(float(omega) - (int(pints) / 2) * float(step),
    float(omega) + (int(pints) / 2) * float(step),
    int(pints))

print(om)
print(np.mean(om))

gives: [30.4 30.50526316 30.61052632 30.71578947 30.82105263 30.92631579 31.03157895 31.13684211 31.24210526 31.34736842 31.45263158 31.55789474 31.66315789 31.76842105 31.87368421 31.97894737 32.08421053 32.18947368 32.29473684 32.4 ] 31.4 [30.35 30.455 30.56 30.665 30.77 30.875 30.98 31.085 31.19 31.295 31.4 31.505 31.61 31.715 31.82 31.925 32.03 32.135 32.24 32.345 32.45 ] 31.4

In odd, the omega is middle value, in even case the omega itself is not there. Or am I missing something?

Created by: ivan-usov

OK, I didn't know how this "om" array should be constructed at all. This is clearly dependent on how daq works, so if you know that this array is calculated this way, then it's all good.

Created by: JakHolzer

Oh I see, I apologise. Thank you for so throughout correction :)

po 19. 10. 2020 v 15:44 odesílatel Ivan Usov notifications@github.com napsal:

@ivan-usov commented on this pull request.

In pyzebra/load_1D.py https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r507738757 :

@@ -165,7 +165,15 @@ def parse_1D(fileobj, data_type): for name, val in zip(col_names, line.split()): data_cols[name].append(float(val))

•    measurements = dict(data_cols)

•    data_cols['h_index'] = float(metadata['title'].split()[-3])

•    data_cols['k_index'] = float(metadata['title'].split()[-2])

•    data_cols['l_index'] = float(metadata['title'].split()[-1])

•    data_cols['temperature'] = metadata['temp']

•    data_cols['mag_field'] = metadata['mf']

•    data_cols['omega_angle'] = metadata['omega']

•    data_cols['number_of_measurements'] = len(data_cols['om'])

•    data_cols['monitor'] = data_cols['Monitor1'][0]

•    measurements["1"] = dict(data_cols)

The key should be just an integer 1, e.g. measurements[1] = dict(data_cols)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/paulscherrerinstitute/pyzebra/pull/11#pullrequestreview-511751197, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM2BBRUQGCFAYYI3DQJYJ2DSLQ7CDANCNFSM4RNBW7BQ .

Created by: JakHolzer

Well, its not necessary for dat files which actually carry the om array, in ccl its simplified to omega centre, step and number of points, so this is only way of getting it.

po 19. 10. 2020 v 16:12 odesílatel Ivan Usov notifications@github.com napsal:

@ivan-usov commented on this pull request.

In pyzebra/load_1D.py https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r507781946 :

•        # first line

•        measurement_number, *params = line.split()

•        for param, (param_name, param_type) in zip(params, ccl_first_line):

•            d[param_name] = param_type(param)

•        decimal.append(bool(Decimal(d["h_index"]) % 1 == 0))

•        decimal.append(bool(Decimal(d["k_index"]) % 1 == 0))

•        decimal.append(bool(Decimal(d["l_index"]) % 1 == 0))

•        # second line

•        next_line = next(fileobj)

•        params = next_line.split()

•        for param, (param_name, param_type) in zip(params, ccl_second_line):

•            d[param_name] = param_type(param)

•        d["om"] = np.linspace(

OK, I didn't know how this "om" array should be constructed at all. This is clearly dependent on how daq works, so if you know that this array is calculated this way, then it's all good.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/paulscherrerinstitute/pyzebra/pull/11#discussion_r507781946, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM2BBRTHNOPYSCPDKIF76KDSLRCMZANCNFSM4RNBW7BQ .

Merged by: ivan-usov at 2020-10-22 10:07:50 UTC