# <h1>Table of Contents<span class="tocSkip"></span></h1>
# <div class="toc"><ul class="toc-item"><li><span><a href="#Configuration" data-toc-modified-id="Configuration-1"><span class="toc-item-num">1 </span>Configuration</a></span></li><li><span><a href="#Support-Routines" data-toc-modified-id="Support-Routines-2"><span class="toc-item-num">2 </span>Support Routines</a></span><ul class="toc-item"><li><span><a href="#Visualization" data-toc-modified-id="Visualization-2.1"><span class="toc-item-num">2.1 </span>Visualization</a></span></li></ul></li><li><span><a href="#Dataset-creation" data-toc-modified-id="Dataset-creation-3"><span class="toc-item-num">3 </span>Dataset creation</a></span><ul class="toc-item"><li><span><a href="#Dataset-reading-and-preprocessing-definition" data-toc-modified-id="Dataset-reading-and-preprocessing-definition-3.1"><span class="toc-item-num">3.1 </span>Dataset reading and preprocessing definition</a></span></li><li><span><a href="#Make-dataset" data-toc-modified-id="Make-dataset-3.2"><span class="toc-item-num">3.2 </span>Make dataset</a></span></li><li><span><a href="#Training/Test-Split" data-toc-modified-id="Training/Test-Split-3.3"><span class="toc-item-num">3.3 </span>Training/Test Split</a></span></li><li><span><a href="#Data-scaling-for-DNN-training" data-toc-modified-id="Data-scaling-for-DNN-training-3.4"><span class="toc-item-num">3.4 </span>Data scaling for DNN training</a></span></li></ul></li><li><span><a href="#DNN-Model-definitions" data-toc-modified-id="DNN-Model-definitions-4"><span class="toc-item-num">4 </span>DNN Model definitions</a></span><ul class="toc-item"><li><span><a href="#L2reg-and-gaussian-noise" data-toc-modified-id="L2reg-and-gaussian-noise-4.1"><span class="toc-item-num">4.1 </span>L2reg and gaussian noise</a></span></li><li><span><a href="#Model-with-Dropout" data-toc-modified-id="Model-with-Dropout-4.2"><span class="toc-item-num">4.2 </span>Model with Dropout</a></span></li></ul></li><li><span><a href="#DNN-Training-runs-(data-not-corrected-for-zeroes)" data-toc-modified-id="DNN-Training-runs-(data-not-corrected-for-zeroes)-5"><span class="toc-item-num">5 </span>DNN Training runs (data not corrected for zeroes)</a></span><ul class="toc-item"><li><span><a href="#Andi's-initial-DNN-using-gn" data-toc-modified-id="Andi's-initial-DNN-using-gn-5.1"><span class="toc-item-num">5.1 </span>Andi's initial DNN using gn</a></span></li><li><span><a href="#without-any-regularization" data-toc-modified-id="without-any-regularization-5.2"><span class="toc-item-num">5.2 </span>without any regularization</a></span><ul class="toc-item"><li><span><a href="#Investigation-of-model-performance-errors" data-toc-modified-id="Investigation-of-model-performance-errors-5.2.1"><span class="toc-item-num">5.2.1 </span>Investigation of model performance errors</a></span></li><li><span><a href="#outlier-investigation-after-answer-by-Jochem/Pavle" data-toc-modified-id="outlier-investigation-after-answer-by-Jochem/Pavle-5.2.2"><span class="toc-item-num">5.2.2 </span>outlier investigation after answer by Jochem/Pavle</a></span></li></ul></li><li><span><a href="#Trying-to-reproduce-best-hyperscan-run" data-toc-modified-id="Trying-to-reproduce-best-hyperscan-run-5.3"><span class="toc-item-num">5.3 </span>Trying to reproduce best hyperscan run</a></span></li><li><span><a href="#Try-out-DNN-with-dropout" data-toc-modified-id="Try-out-DNN-with-dropout-5.4"><span class="toc-item-num">5.4 </span>Try out DNN with dropout</a></span></li></ul></li><li><span><a href="#Hyperparameter-scans-(data-not-corrected-for-zeroes)" data-toc-modified-id="Hyperparameter-scans-(data-not-corrected-for-zeroes)-6"><span class="toc-item-num">6 </span>Hyperparameter scans (data not corrected for zeroes)</a></span><ul class="toc-item"><li><span><a href="#Test:-Make-a-hyperparameter-scan-A" data-toc-modified-id="Test:-Make-a-hyperparameter-scan-A-6.1"><span class="toc-item-num">6.1 </span>Test: Make a hyperparameter scan A</a></span></li><li><span><a href="#Offline-batch-parameter-scan" data-toc-modified-id="Offline-batch-parameter-scan-6.2"><span class="toc-item-num">6.2 </span>Offline batch parameter scan</a></span></li><li><span><a href="#TODO-ModelB" data-toc-modified-id="TODO-ModelB-6.3"><span class="toc-item-num">6.3 </span>TODO ModelB</a></span></li><li><span><a href="#TODO:-Model-C:-scan-regulatisation-and-noise" data-toc-modified-id="TODO:-Model-C:-scan-regulatisation-and-noise-6.4"><span class="toc-item-num">6.4 </span>TODO: Model C: scan regulatisation and noise</a></span></li></ul></li><li><span><a href="#DNN-runs-with-data-set-cleaned-for-zero-energy-measurements" data-toc-modified-id="DNN-runs-with-data-set-cleaned-for-zero-energy-measurements-7"><span class="toc-item-num">7 </span>DNN runs with data set cleaned for zero energy measurements</a></span><ul class="toc-item"><li><span><a href="#without-any-regularization" data-toc-modified-id="without-any-regularization-7.1"><span class="toc-item-num">7.1 </span>without any regularization</a></span></li></ul></li><li><span><a href="#SVM-to-see-what-a-linear-model-can-do" data-toc-modified-id="SVM-to-see-what-a-linear-model-can-do-8"><span class="toc-item-num">8 </span>SVM to see what a linear model can do</a></span></li></ul></div>
# <div class="toc"><ul class="toc-item"><li><span><a href="#Configuration" data-toc-modified-id="Configuration-1"><span class="toc-item-num">1 </span>Configuration</a></span></li><li><span><a href="#Support-Routines" data-toc-modified-id="Support-Routines-2"><span class="toc-item-num">2 </span>Support Routines</a></span><ul class="toc-item"><li><span><a href="#Visualization" data-toc-modified-id="Visualization-2.1"><span class="toc-item-num">2.1 </span>Visualization</a></span></li></ul></li><li><span><a href="#Dataset-creation" data-toc-modified-id="Dataset-creation-3"><span class="toc-item-num">3 </span>Dataset creation</a></span><ul class="toc-item"><li><span><a href="#Dataset-reading-and-preprocessing-definition" data-toc-modified-id="Dataset-reading-and-preprocessing-definition-3.1"><span class="toc-item-num">3.1 </span>Dataset reading and preprocessing definition</a></span></li><li><span><a href="#Make-dataset" data-toc-modified-id="Make-dataset-3.2"><span class="toc-item-num">3.2 </span>Make dataset</a></span></li><li><span><a href="#Examining-data" data-toc-modified-id="Examining-data-3.3"><span class="toc-item-num">3.3 </span>Examining data</a></span></li><li><span><a href="#Training/Test-Split" data-toc-modified-id="Training/Test-Split-3.4"><span class="toc-item-num">3.4 </span>Training/Test Split</a></span></li><li><span><a href="#Data-scaling-for-DNN-training" data-toc-modified-id="Data-scaling-for-DNN-training-3.5"><span class="toc-item-num">3.5 </span>Data scaling for DNN training</a></span></li></ul></li><li><span><a href="#DNN-Model-definitions" data-toc-modified-id="DNN-Model-definitions-4"><span class="toc-item-num">4 </span>DNN Model definitions</a></span><ul class="toc-item"><li><span><a href="#L2reg-and-gaussian-noise" data-toc-modified-id="L2reg-and-gaussian-noise-4.1"><span class="toc-item-num">4.1 </span>L2reg and gaussian noise</a></span></li><li><span><a href="#Model-with-Dropout" data-toc-modified-id="Model-with-Dropout-4.2"><span class="toc-item-num">4.2 </span>Model with Dropout</a></span></li></ul></li><li><span><a href="#DNN-Training-runs-(data-not-corrected-for-zeroes)" data-toc-modified-id="DNN-Training-runs-(data-not-corrected-for-zeroes)-5"><span class="toc-item-num">5 </span>DNN Training runs (data not corrected for zeroes)</a></span><ul class="toc-item"><li><span><a href="#Andi's-initial-DNN-using-gn" data-toc-modified-id="Andi's-initial-DNN-using-gn-5.1"><span class="toc-item-num">5.1 </span>Andi's initial DNN using gn</a></span></li><li><span><a href="#without-any-regularization" data-toc-modified-id="without-any-regularization-5.2"><span class="toc-item-num">5.2 </span>without any regularization</a></span><ul class="toc-item"><li><span><a href="#Investigation-of-model-performance-errors" data-toc-modified-id="Investigation-of-model-performance-errors-5.2.1"><span class="toc-item-num">5.2.1 </span>Investigation of model performance errors</a></span></li><li><span><a href="#outlier-investigation-after-answer-by-Jochem/Pavle" data-toc-modified-id="outlier-investigation-after-answer-by-Jochem/Pavle-5.2.2"><span class="toc-item-num">5.2.2 </span>outlier investigation after answer by Jochem/Pavle</a></span></li></ul></li><li><span><a href="#Trying-to-reproduce-best-hyperscan-run" data-toc-modified-id="Trying-to-reproduce-best-hyperscan-run-5.3"><span class="toc-item-num">5.3 </span>Trying to reproduce best hyperscan run</a></span></li><li><span><a href="#Try-out-DNN-with-dropout" data-toc-modified-id="Try-out-DNN-with-dropout-5.4"><span class="toc-item-num">5.4 </span>Try out DNN with dropout</a></span></li></ul></li><li><span><a href="#Hyperparameter-scans-(data-not-corrected-for-zeroes)" data-toc-modified-id="Hyperparameter-scans-(data-not-corrected-for-zeroes)-6"><span class="toc-item-num">6 </span>Hyperparameter scans (data not corrected for zeroes)</a></span><ul class="toc-item"><li><span><a href="#Test:-Make-a-hyperparameter-scan-A" data-toc-modified-id="Test:-Make-a-hyperparameter-scan-A-6.1"><span class="toc-item-num">6.1 </span>Test: Make a hyperparameter scan A</a></span></li><li><span><a href="#Offline-batch-parameter-scan" data-toc-modified-id="Offline-batch-parameter-scan-6.2"><span class="toc-item-num">6.2 </span>Offline batch parameter scan</a></span></li><li><span><a href="#TODO-ModelB" data-toc-modified-id="TODO-ModelB-6.3"><span class="toc-item-num">6.3 </span>TODO ModelB</a></span></li><li><span><a href="#TODO:-Model-C:-scan-regulatisation-and-noise" data-toc-modified-id="TODO:-Model-C:-scan-regulatisation-and-noise-6.4"><span class="toc-item-num">6.4 </span>TODO: Model C: scan regulatisation and noise</a></span></li></ul></li><li><span><a href="#DNN-runs-with-data-set-cleaned-for-zero-energy-measurements" data-toc-modified-id="DNN-runs-with-data-set-cleaned-for-zero-energy-measurements-7"><span class="toc-item-num">7 </span>DNN runs with data set cleaned for zero energy measurements</a></span><ul class="toc-item"><li><span><a href="#without-any-regularization" data-toc-modified-id="without-any-regularization-7.1"><span class="toc-item-num">7.1 </span>without any regularization</a></span></li></ul></li><li><span><a href="#SVM-to-see-what-a-linear-model-can-do" data-toc-modified-id="SVM-to-see-what-a-linear-model-can-do-8"><span class="toc-item-num">8 </span>SVM to see what a linear model can do</a></span></li></ul></div>