|
|
= Overlays
|
|
|
:sectnums:
|
|
|
:TOC:
|
|
|
|
|
|
This is work in progress!!!
|
|
|
|
|
|
== Overview
|
|
|
|
|
|
Idea: seamless stacking of multiple module hierarchies, use cases:
|
|
|
|
|
|
* develop/build new modules without exposing them to other users
|
|
|
* transparent/seamless support for modules optimised for certain CPUs
|
|
|
* modules for HPC systems like slurm and required software to support HPC hardware.
|
|
|
* modules on another filesystem or location than the default (@PSI: `/opt/psi`)
|
|
|
* ...
|
|
|
|
|
|
|
|
|
== Overlay stacks
|
|
|
Overlays can be stacked. The first overlay - the overlay at the bottom of the stack - is called "base overlay". Multiple loaded overlays are organised as stack. Loading an overlay pushes it on top of the stack. It is only possible to unload the overlay on top of the stack.
|
|
|
|
|
|
The 'overlay' at the bottom of the stack is called the base overlay and cannot be unloaded.
|
|
|
|
|
|
== Limitations
|
|
|
|
|
|
The relative directory structure of a hierarchy group must be identical to the base overlay. It is also not possible to define a new hierarchical group.
|
|
|
|
|
|
Overlays must be loaded before the first module.
|
|
|
|
|
|
Unloading an overlay is only possible if no modules are loaded.
|
|
|
|
|
|
Note: an overlay can add new non-hierarchical groups.
|
|
|
|
|
|
== Overlay configuration
|
|
|
|
|
|
=== Definition
|
|
|
An overlay is defined by
|
|
|
* a root directory for the software installation
|
|
|
* a root directory for the modulefiles
|
|
|
* a name
|
|
|
|
|
|
In most cases both root directories are identical.
|
|
|
|
|
|
An overlay is defined by the two root directories. Different names can be used for the same overlay.
|
|
|
|
|
|
If only the root directory for the software installation is specified, this is also used as root for the modulefiles.
|
|
|
|
|
|
=== Overlay definition files
|
|
|
|
|
|
Configuration files are in YAML format.
|
|
|
|
|
|
The system wide configuration file is `$PMODULES_ROOT/config/overlays.yaml`.
|
|
|
|
|
|
User can define their own overlays in `$HOME/.Pmodules/overlays.yaml`.
|
|
|
|
|
|
All overlays must be defined in the system or user configuration file.
|
|
|
|
|
|
The user configuration file is queried first and can be used to overwrite the definition of an overlay in the system configuration file.
|
|
|
|
|
|
Format:
|
|
|
----
|
|
|
---
|
|
|
name_of_overlay:
|
|
|
root_dir: <DIR>
|
|
|
modulefiles_root_dir: <DIR>
|
|
|
type: normal|hidding|replacing
|
|
|
...
|
|
|
----
|
|
|
|
|
|
`name_of_overlay`:: Is the name of the overlay.
|
|
|
|
|
|
`root_dir`:: Is the root directory for software installation.
|
|
|
The specification of `root_dir` is mandatory.
|
|
|
|
|
|
`modulefiles_root_dir`:: Is the root directory for the installation of modulefiles. If it is not specified, `root_dir` is used. This is the most usual case.
|
|
|
|
|
|
`type`:: Specifies the type of the overlay. The default is `normal`. See below for the type specifications.
|
|
|
|
|
|
== Overlay types
|
|
|
|
|
|
`normal`:: In an overlay of this types the modules in the overlay are added to the already available modules provided by the overlays on the stack. If modules with identical name/version exist in multiple overlays, the module from the overlay which is higher on the stack is used. In other words modules with identical name/version are replaced by overlays loaded later.
|
|
|
+
|
|
|
Example: Assume we have the overlays `base` and `private`. In the base overlay are the modules `git/2.33.1`, `gnuplot/5.4.0` and `gnuplot/5.4.2` in the group `Tools`. In private overlay are the modules `gnuplot/5.4.2` and `gnuplot/5.4.3` (maybe compiled with a different configuration).
|
|
|
+
|
|
|
After loading the private overlay the following Gnuplot modules are available:
|
|
|
+
|
|
|
* `git/2.33.1`and `gnuplot/5.4.0` from the base overlay
|
|
|
* `gnuplot/5.4.2` and `gnuplot/5.4.3`from the private overlay
|
|
|
+
|
|
|
Note: the module `gnuplot/5.4.2` from the base overlay has been replaced by the module with the same name from the private overlay.
|
|
|
|
|
|
`hiding`:: An overlay of this types hides all modules with the same name in previous loaded overlays.
|
|
|
+
|
|
|
Example: Assume we have the same overlays and Gnuplot modules as in the example above.
|
|
|
+
|
|
|
After loading the private overlay with type `hiding` the following Gnuplot modules are available:
|
|
|
+
|
|
|
* `git/2.33.1` from the base overlay
|
|
|
* `gnuplot/5.4.2` and `gnuplot/5.4.3`from the private overlay
|
|
|
+
|
|
|
Note: the Gnuplot modules from the base overlay are hidden!
|
|
|
|
|
|
`replacing`:: A overlay of this types replaces hole groups.
|
|
|
+
|
|
|
Example: If you load the private overlay from the examples above as replacing, the following modules are available in the group `Tools`:
|
|
|
+
|
|
|
* `gnuplot/5.4.2` and `gnuplot/5.4.3`from the private overlay
|
|
|
+
|
|
|
Note: the Git module from the base overlay is not available any more.
|
|
|
|
|
|
== Variant files
|
|
|
With the implementation of overlays a new format for the so called variant files is necessary. In the old format extensions are almost impossible. Nowadays a common format used for configuration files is YAML.
|
|
|
|
|
|
==== YAML format of variant files
|
|
|
----
|
|
|
---
|
|
|
overlay: <overlay-name>
|
|
|
module_name/module_version:
|
|
|
variants:
|
|
|
- relstage: <release-stage>
|
|
|
modulefiles_root_dir: <modulefile-root-dir>
|
|
|
dependencies:
|
|
|
- dependency_1
|
|
|
- dependency_2
|
|
|
- ...
|
|
|
...
|
|
|
----
|
|
|
`overlay`:: Name of the overlay the module will be installed in. The specified overlay must be defined in one of the overlay definition files. If no overlay is specified the base overlay is used.
|
|
|
|
|
|
`module_name/module_version`:: Specification of the module with this name and version. For each version a specification should be added.
|
|
|
|
|
|
`variants`:: Array of variants.
|
|
|
|
|
|
`relstage`:: Release stage, either `unstable`, `stable` or `deprecated`. If not specified the release stage defaults to `unstable`.
|
|
|
|
|
|
`modulefiles_root_dir`:: This key can be used to overwrite the `modulefiles_root_dir` specified in the overlay definition file.
|
|
|
|
|
|
`dependencies`:: Array of dependencies. Dependencies can have one of the following prefixes:
|
|
|
+
|
|
|
* `b:` build-dependency: A build-dependency is required at build time and must be loaded to compile the module. But it is not required at runtime.
|
|
|
* `i:` install-dependency: A install-dependency is required at runtime, but must not be loaded at runtime of the module. An example is the Intel compiler. It requires GCC but GCC must not be loaded. A install-dependency is loaded at build-time.
|
|
|
* no prefix: dependencies without prefix must be loaded at build- and runtime. |
|
|
\ No newline at end of file |