Dimensionality Independence

Master's thesis project: making IPPL work for problems in any number of dimensions.

In order to achieve dimensionality independence in the code, the arguments need to be variadic, but we still need to be able to manipulate specific elements of the argument list in order to implement stencils, boundary conditions, and other components. We can categorize the implementation issues:

• Cleanly creating data types for arbitrary rank problems
  • Recursive templating for arbitrary depth pointers (vinciguerra_a/ippl@15a35d3a)
  • Recursive templating for arbitrary length tuples (required for parameter pack expansion for functor objects, see below) (vinciguerra_a/ippl@dc8b5ee8)
• Generalized range policy generator (vinciguerra_a/ippl@ce38cba9)
• Dimensionality independent kernel signatures
  • ... achieved via templated lambdas (requires C++20)
    • Templated lambdas for parallel_for (vinciguerra_a/ippl@d10d4805)
    • Functor wrapper for parallel_reduce (vinciguerra_a/ippl@d1af0ddc)
  • ... achieved via array arguments
    • Generalized wrapper for functors (vinciguerra_a/ippl@d54f9b45)
• Generalized argument access by dimension
  • Reimplementing std::apply for CUDA-compatible data structures (STL doesn't work on CUDA) (vinciguerra_a/ippl@33ff2590)
  • Manipulating arguments as an array (vinciguerra_a/ippl@33ff2590)
  • Manipulating arguments using STL constexpr tuple access functions (vinciguerra_a/ippl@ed841d7c)
• Generalized field operations
  • Cross product? Needs a definition for ranks other than 3
  • Gradient (vinciguerra_a/ippl@c9809ae7)
  • Divergence (vinciguerra_a/ippl@36ca90f9)
  • Laplace (vinciguerra_a/ippl@3d0c78ec)
  • Curl? Needs a definition for ranks other than 3
  • Hessian (vinciguerra_a/ippl@e017898b, vinciguerra_a/ippl@080718f1)
• Generalized boundary conditions
• Generalized halo communication
• Generalized particle operations
  • Scatter/gather (vinciguerra_a/ippl@c2227dba, vinciguerra_a/ippl@c0612729)
  • Locating (vinciguerra_a/ippl@a51a46fe)
• Rewriting unit tests to test all ranks

Then we need to verify the implementation.

• Reproducing results
• Benchmarking

Once the implementation is verified, we can add new simulations.

• Landau damping in fewer dimensions
  • 2D (vinciguerra_a/ippl@d62d7d60)
• Advection problems in 4D

Supplemental additions:

• Expanded unit testing
  • FFT
    • Determine if heFFTe supports 1D or 4D+
    • 2D and 3D (vinciguerra_a/ippl@17b42b3b)
  • Halo communication and neighbor finding (vinciguerra_a/ippl@ac213820)

Toolchain requirements:

• Upgrading to CUDA 12 for C++20 support
  • Gwendolen
  • JUWEL
  • Perlmutter