Uldis LocansDKSOPAL module
DKS OPAL module provides CUDA and OpenMP implementations for algorithms that allow OPAL to offload FFT Poisson solver and particle matter interaction simulations. Using DKSBase module OPAL can schedule memory management and data transfers to the devices and using the OPAL module schedule the kernel execution on the device. For the FFT Poisson solver gather and scatter data transfers are used with the GPU to allow multiple CPU cores to share one GPU.
For the FFT Poisson solver DKS contains a FFT module, that allows to perform the Fast Fourier transformations on the accelerator. For GPU devices DKS uses cuFFT library provides by Nvidia, while for Intel MIC DKS uses Intels MKL library. Additionally FFT Poisson solver requires calculation of Greens function and element by element multiplication of arrays. DKS contains optimized CUDA and OpenMP implementations of these algorithms.
In order to perform Monte Carlo simulations for particle matter interaction, DKS contains CUDA and OpenMP codes for energy loss and Coulomb scattering calculations. For random numbers DKS uses Nvidias cuRand library and Intels MKL VSL library. In addition DKS provides function to move dead particles to the end of the array, this is done by a loop on the Intel MIC and using Thrust library to sort the array on the GPU.
Full list of DKSOPAL module functions are available in in DKS documentation. The full description of DKSMusrfit module and its interaction with musrfit can be found in Thesis Chapter 4.