|
|
= Compiling OPAL
|
|
|
Achim Gsell
|
|
|
:doctype: article
|
|
|
:numbered:
|
|
|
:toc:
|
|
|
:toc-placement: preamble
|
|
|
:toclevels: 2
|
|
|
|
|
|
// Need some preamble to get TOC:
|
|
|
{empty}
|
|
|
|
|
|
'''
|
|
|
|
|
|
== Introduction
|
|
|
|
|
|
Compiling OPAL can be quite challenging due to all the required
|
|
|
libraries. Even if some libraries are available on the system, this
|
|
|
doesn't mean that they can be used for compiling OPAL due to missing
|
|
|
features. In this document we explain how to compile OPAL from scratch
|
|
|
- including the recommended compiler and all libraries.
|
|
|
|
|
|
== Requirements
|
|
|
|
|
|
=== System requirements
|
|
|
|
|
|
* Operating system
|
|
|
** Linux (tested with RHEL 6,7)
|
|
|
** Windows with Windows Services for Linux (WSL)
|
|
|
** Mac with macos > 10.12
|
|
|
* Required development tools/packages
|
|
|
** macOS
|
|
|
*** Xcode
|
|
|
** Linux (including WSL)
|
|
|
*** Python 2.7
|
|
|
*** zlib >= 1.2.11 including development package
|
|
|
*** bzip2
|
|
|
*** gzip
|
|
|
*** tar
|
|
|
*** m4
|
|
|
*** bison or compatible
|
|
|
*** flex or compatible
|
|
|
*** C and C++ compiler
|
|
|
|
|
|
{nbsp}
|
|
|
|
|
|
=== Required software
|
|
|
|
|
|
Before compiling everything yourself, you might want to try the versions available on your system. Check the requirements below before trying. The following software usually works without problems: cmake >= 3.0.0, gcc >= 5.3.0, gsl >= 2.4, boost >= 1.59, MPI.
|
|
|
The list below shows the required software, the minimal and
|
|
|
the recommended version.
|
|
|
|
|
|
[width="100%",options="header"]
|
|
|
|=======
|
|
|
| **Name** | **minimum Version** | **recommended Version** | **Notes**
|
|
|
|
|
|
| C++ Compiler | C++ 11 | C++ 17 | version is minimum/recommended C++ standard. For further instructions see section below.
|
|
|
|
|
|
| cmake | 3.0.0 | 3.11.4 |
|
|
|
1. CMake shipped with RHEL 7 and older cannot be used! +
|
|
|
2. GCC 4.8.x shipped with RHEL7 cannot be used! +
|
|
|
3. Please note that version 3.6.3 has some known issues with hdf5. +
|
|
|
4. CMake version 3.11.4 and newer requires a C++ compiler supporting C++11.
|
|
|
|
|
|
| Open MPI | 1.10.2 | 3.1.2 | as alternative MPICH can be used
|
|
|
|
|
|
| hdf5 | 1.8.18 | 1.10.3 | A parallel version is required.
|
|
|
|
|
|
| GNU Scientific Library | 2.4 | 2.5 |
|
|
|
|
|
|
| H5hut | 2.0.0rc6 | 2.0.0rc6 | A parallel version is required.
|
|
|
|
|
|
| boost | 1.59 | 1.68.0
|
|
|
| The following boost libraries are required: `chrono`, `filesystem`,
|
|
|
`iostreams`, `regex`, `serialization`, `system`
|
|
|
|
|
|
| google-test | 1.7.0 | 1.7.0 | optional: Required to run the unit-tests
|
|
|
|
|
|
| ParMETIS | 4.0.0 | 4.0.3
|
|
|
| optional: required for OPAL compiled with SAAMG solver.
|
|
|
|
|
|
| OpenBLAS | 0.2.9 | 0.2.20
|
|
|
| optional: required for OPAL compiled with SAAMG solver.
|
|
|
|
|
|
| trilinos | 12.10.1 | 12.10.1
|
|
|
| optional: required for OPAL compiled with SAAMG solver.
|
|
|
|
|
|
| VTK | 7.1 | 8.0.0
|
|
|
| optional: required of you want to compile H5hut with the VTK to H5hut mesh converter.
|
|
|
|=======
|
|
|
|
|
|
=== C++ compiler
|
|
|
|
|
|
==== GCC on Linux, macOS, WSL
|
|
|
|
|
|
GCC can be used on Linux, macOS and Windows Services for Linux.
|
|
|
|
|
|
[width="100%",options="header"]
|
|
|
|=======
|
|
|
| **Name** | **minimum Version** | **recommended Version** | **Notes**
|
|
|
|
|
|
| GNU Multiple Precision Arithmetic Library (GMP) | 6.0.0 | 6.1.2
|
|
|
| required to compile GCC
|
|
|
|
|
|
| GNU MPFR Library | 3.1.0 | 4.0.1 | required to compile GCC
|
|
|
|
|
|
| GNU MPC Library | 1.0.0 | 1.1.0 | required to compile GCC
|
|
|
|
|
|
| GNU Compiler Collection | 5.3.0 | 7.3.0 |
|
|
|
1. On macOS use GCC >= 7.1.0 or Clang shipped with Xcode. +
|
|
|
2. We observed some issues compiling GCC 7.3.0 on macOS 10.13 with Xcode 10. First issue was that `bits/basic_ios.h` was not found. This might have been an issue due too many parallel `make` processes and could be solved by re-running `make` with a single process. Second issue was, that `g++` was called instead of the newly build staging binary `xg++`. This issue could be solved by re-running `make` in the `gcc` sub-directory.
|
|
|
|=======
|
|
|
|
|
|
==== Clang for macOS
|
|
|
|
|
|
For the time being Clang is only tested and recommended for macOS.
|
|
|
|
|
|
[width="100%",options="header"]
|
|
|
|=======
|
|
|
| **Name** | **minimum Version** | **recommended Version** | **Notes**
|
|
|
|
|
|
| Clang (LLVM) | 9.0.0 | |
|
|
|
1. Apple LLVM version 9.0.0 (clang-900.0.38) shipped with Xcode 9.1 (macOS 10.12) +
|
|
|
2. Apple LLVM version 10.0.1 (clang-1000.11.45.5) shipped with Xcode 10.2 (macOS 10.14)
|
|
|
|=======
|
|
|
|
|
|
'''
|
|
|
|
|
|
== Setup required software
|
|
|
|
|
|
=== Linux @PSI
|
|
|
|
|
|
The easiest and recommended solution is to use the **Pmodules** environment available at PSI. Please link:Pmodules[read this wiki-page] if you are not using a PSI Linux distribution or you do not know whether the Pmodules are installed on your system.
|
|
|
|
|
|
'''
|
|
|
|
|
|
==== Load the OPAL tool-chain
|
|
|
|
|
|
**For OPAL 1.6.x**
|
|
|
```
|
|
|
module use /afs/psi.ch/project/amas/modulefiles
|
|
|
module load opal-toolchain/1.6
|
|
|
```
|
|
|
|
|
|
**For OPAL 2.0**
|
|
|
```
|
|
|
module use /afs/psi.ch/project/amas/modulefiles
|
|
|
module load opal-toolchain/2.0
|
|
|
```
|
|
|
|
|
|
**For OPAL master**
|
|
|
```
|
|
|
module use /afs/psi.ch/project/amas/modulefiles
|
|
|
module load opal-toolchain/master
|
|
|
```
|
|
|
|
|
|
> **Note:** For the time being there is no CUDA support for OPAL 2.x
|
|
|
|
|
|
> **Note:** For your convenience we recommend to add the line +
|
|
|
`module use /afs/psi.ch/project/amas/modulefiles` +
|
|
|
to your `.bashrc` or `.bash_profile`.
|
|
|
|
|
|
=== Build everything from scratch
|
|
|
|
|
|
We provide recipes to build all required software. These recipes can be used on Linux, macOS and Windows Services for Linux. They are tested with:
|
|
|
|
|
|
* Redhat Enterprise Linux 7
|
|
|
* macOS 10.12 with Xcode 9
|
|
|
* macOS 10.13 with Xcode 10.1
|
|
|
* Windows Services for Linux (ongoing)
|
|
|
|
|
|
>
|
|
|
With the last update of Xcode to version 10.2 it is *not* possible to compile GCC without patches. For macOS the recommended solution to compile OPAL is to use Clang.
|
|
|
|
|
|
==== Setting up the environment
|
|
|
|
|
|
Run the following command to download the recipes:
|
|
|
|
|
|
[listing]
|
|
|
--
|
|
|
git clone git@gitlab.psi.ch:OPAL/build-recipes.git
|
|
|
--
|
|
|
|
|
|
|
|
|
To keep thinks simple, we set and use a couple of environment variables.
|
|
|
|
|
|
`PREFIX`::
|
|
|
Installation prefix for all software we have to compile and install. In the following instructions we use `${HOME}/OPAL`.
|
|
|
|
|
|
`DOWNLOADS_DIR`::
|
|
|
This is the directory where we store downloaded files. In this instruction we use `${PREFIX}/Downloads`.
|
|
|
|
|
|
`SRC_DIR`::
|
|
|
This is the directory where we unpack and compile software. Here we use `${PREFIX}/src`.
|
|
|
|
|
|
`njobs`::
|
|
|
number of parallel `make` jobs. The number should be less or equal the number of core on your system plus one.
|
|
|
|
|
|
To setup the environment variables and create the required directory hierarchy, _source_ the following script:
|
|
|
|
|
|
link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/setup.sh[setup.sh]
|
|
|
|
|
|
'''
|
|
|
|
|
|
==== Compile
|
|
|
|
|
|
Build the software in this order:
|
|
|
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/010-build-gmp[GMP]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/020-build-mpfr[MPFR]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/030-build-mpc[MPC]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/040-build-gcc[GCC]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/050-build-cmake[CMake]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/060-build-open-mpi[open-mpi]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/070-build-hdf5[HDF5]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/080-build-gsl[GSL]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/090-build-h5hut[H5hut]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/110-build-boost[Boost]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/200-build-parmetis[ParMETIS]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/210-build-openblas[OpenBLAS]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/220-build-trilinos[Trilinos]
|
|
|
|
|
|
'''
|
|
|
|
|
|
=== macOS: Using Macports
|
|
|
|
|
|
If you have a Mac and Macports installed, you can install and use the following port to compile _OPAL_:
|
|
|
|
|
|
----
|
|
|
#select on of
|
|
|
#GCC=gcc5
|
|
|
#GCC=gcc6
|
|
|
GCC=gcc7
|
|
|
|
|
|
sudo port install cmake
|
|
|
sudo port install ${GCC}
|
|
|
sudo port install gsl +${GCC}
|
|
|
sudo port install OpenBLAS +${GCC}
|
|
|
sudo port install openmpi-${GCC}
|
|
|
sudo port install hdf5 +${GCC} +cxx +openmpi +threadsafe
|
|
|
sudo port install parmetis +${GCC} +openmpi
|
|
|
----
|
|
|
|
|
|
The following software is not provided by Macports and must be installed by executing the recipes:
|
|
|
|
|
|
. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/090-build-h5hut[H5hut]
|
|
|
. Set toolset for Boost:
|
|
|
+
|
|
|
[source,sh]
|
|
|
----
|
|
|
export TOOLSET='gcc'
|
|
|
----
|
|
|
. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/build-boost[Boost]
|
|
|
. optional: link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/build-trilinos[Trilinos]
|
|
|
|
|
|
> There are some strange issues with the `boost` boost from Macports if you want to compile with `variants +gcc7 +openmp`.
|
|
|
|
|
|
'''
|
|
|
|
|
|
=== macOS: Xcode only
|
|
|
|
|
|
> This needs to be tested!
|
|
|
|
|
|
The C/C++ compiler Clang shipped with Xcode can be used to compile _OPAL_. A current CMake binary package can be downloaded from http://www.cmake.org. All other packages must be compiled:
|
|
|
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/060-build-open-mpi[open-mpi]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/070-build-hdf5[HDF5]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/080-build-gsl[GSL]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/090-build-h5hut[H5hut]
|
|
|
1. link:https://gitlab.psi.ch/OPAL/build-recipes/raw/master/110-build-boost[Boost]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/200-build-recipes/raw/master/build-parmetis[ParMETIS]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/210-build-recipes/raw/master/build-openblas[OpenBLAS]
|
|
|
1. optional: link:https://gitlab.psi.ch/OPAL/220-build-recipes/raw/master/build-trilinos[Trilinos]
|
|
|
|
|
|
== Building OPAL
|
|
|
|
|
|
=== Get the OPAL sources
|
|
|
|
|
|
Clone the OPAL Git repository
|
|
|
|
|
|
```
|
|
|
mkdir -p "${SRC_DIR}/OPAL"
|
|
|
cd "${SRC_DIR}/OPAL"
|
|
|
git clone git@gitlab.psi.ch:OPAL/src.git
|
|
|
```
|
|
|
|
|
|
Select the OPAL branch you want to compile. The current stable branch is `OPAL-2.0`:
|
|
|
|
|
|
```
|
|
|
cd "${SRC_DIR}/OPAL/src"
|
|
|
git checkout OPAL-2.0
|
|
|
```
|
|
|
|
|
|
To checkout the (unstable) development branch run:
|
|
|
|
|
|
```
|
|
|
cd "${SRC_DIR}/OPAL/src"
|
|
|
git checkout master
|
|
|
```
|
|
|
|
|
|
To get a list of all available branches, you can run the command
|
|
|
|
|
|
```
|
|
|
git branch -a
|
|
|
```
|
|
|
|
|
|
from inside your clone.
|
|
|
|
|
|
|
|
|
=== Setup OPAL build environment
|
|
|
|
|
|
The environment must be set up as described above for building the dependencies.
|
|
|
|
|
|
=== Configure OPAL
|
|
|
|
|
|
OPAL uses CMake to configure the build process. You can either pass options to `cmake` or you can run the command `ccmake` in the build directory **after** calling `cmake`.
|
|
|
|
|
|
--------------------------------------
|
|
|
mkdir -p "${SRC_DIR}/OPAL/build" && cd "$_"
|
|
|
CC=mpicc CXX=mpicxx cmake \
|
|
|
--prefix="${PREFIX}" \
|
|
|
"${SRC_DIR}/OPAL/src"
|
|
|
--------------------------------------
|
|
|
|
|
|
The following table shows the most important options:
|
|
|
[width="80%"]
|
|
|
|=======
|
|
|
| `CMAKE_BUILD_TYPE` | build type can be either `Debug`, `Release`, `RelWithDebInfo` or `MinSizeRel` |
|
|
|
| `CMAKE_INSTALL_PREFIX` | prefix for installation |
|
|
|
| `ENABLE_AMR_SOLVER` | enable AMR solver | requires BoxLib
|
|
|
| `ENABLE_DKS` | enable DKS | requires Cuda >= 8.0.x and DKS >= 1.1.1
|
|
|
| `ENABLE_SAAMG_SOLVER` | enable SAAMG solver | requires trilinos >= 12.x and parmetis >= 4.0.3
|
|
|
|=======
|
|
|
|
|
|
|
|
|
=== Compile and install OPAL
|
|
|
|
|
|
```
|
|
|
cd "${SRC_DIR}/OPAL/build"
|
|
|
make -j ${NJOBS}
|
|
|
make install
|
|
|
```
|
|
|
|
|
|
=== Compile OPAL with the SAAMG solver
|
|
|
|
|
|
To compile OPAL with the SAAMG follow the steps in the section "Compile OPAL", but replace the configuration step with:
|
|
|
--------------------------------------
|
|
|
mkdir -p "${SRC_DIR}/OPAL/build"
|
|
|
cd "${SRC_DIR}/OPAL/build"
|
|
|
CC=mpicc CXX=mpicxx cmake \
|
|
|
--prefix="${PREFIX}" \
|
|
|
-D ENABLE_SAAMG_SOLVER=TRUE \
|
|
|
"${SRC_DIR}/OPAL/src"
|
|
|
--------------------------------------
|
|
|
''' |
|
|
\ No newline at end of file |