Code indexing in gitaly is broken and leads to code not being visible to the user. We work on the issue with highest priority.

Skip to content

Performance increase for Waveform plotting

appel_c requested to merge fix/improve_performance_and_downsampling_for_waveform into pre_release_v2

Description

This MR improves performance of the WaveformWidget for large async data.

Summary of features for release notes

Certain devices are capable of sending their data on their own frequency to BEC. One supported example for this is the NIDAQ at Debye. Data rates can be around 20-30MB/s, with around 8 signals each having 1e5-1e7 points. BECWidget can plot these updates, however, we would like to ensure a good user experience. We revisited the plotting behavior of such signals, and significantly improved the performance and responsiveness of the Waveform. (TODO, add sentence on benchmark)

Detailed updates:

  • Async Data can either be plotted vs 'index' or another 'device_signal'. For any plotting, there is a fallback mechanism to 'index' plotting if the length of the requested x_data ('device_signal') differs from the y_data.
  • For large datasets (N>1000), we now automatically deactivate scatterers, and activate downsampling.
  • Downsampling remains interactive. Zooming into a dataset will continuously adjust the sampling factor, allowing users to see single values of their large datasets.
  • Performance benchmarks:
    • simulation:
    • Deybe:

Type of Change

  • add 'add_slice' functionality to Waveform Widget
  • use getOriginalDataset() to retrieve the data, instead of getData (only viewed data)
  • Above N=1000, scatterers are deactivated, downsampling activated with auto=True, method='peak' and clipToView enabled. ClipToView is now enabled per default for all curves
  • Add custom hook to ViewAll button to temporarily deactivate clipToView. Otherwise the autoRange will not work properly. Note, it should be carefully checked if this affects other widgets too, i.e. MultiWaveformWidget or ScatterWaveform. A brief test did not indicate this.
  • Multiple async curves are supported now.
  • Tests (e2e) are added, to ensure updates of the three different kinds ('add' , 'add_slice' , 'replace') are functional
  • Async live updates would either plot 'timestamp' or 'index', but not a device signal. This was adjusted, and plotting async data vs timestamp was deprecated. We can not ensure that timestamp data is there. Most likely, we will have a single value for timestamp in the DeviceMessage, with a full Waveform. The future logic will be:
    • 'Index' plotting is fallback for ['index', 'auto', 'timestamp'].
    • 'Device' & 'entry'. Plotting will be attempted. If x vs y are datasets have a missmatch, we fallback to index (TBD)

Additional Comments

Definition of Done

  • Documentation is up-to-date. (TODO)
Edited by appel_c

Merge request reports