Build/install HiPACE++
Developers
If you are new to CMake, this short tutorial from the HEP Software foundation is the perfect place to get started with it. If you just want to use CMake to build the project, jump into sections 1. Introduction, 2. Building with CMake and 9. Finding Packages.
Dependencies
HiPACE++ depends on the following popular third party software. Please see installation instructions below in the Developers section.
a mature C++17 compiler: e.g. GCC 7, Clang 7, NVCC 11.0, MSVC 19.15 or newer
AMReX development: we automatically download and compile a copy of AMReX
openPMD-api 0.15.1+: we automatically download and compile a copy of openPMD-api
Platform-dependent, at least one of the following:
CUDA Toolkit 11.0+: for NVIDIA GPU support (see matching host-compilers)
ROCm 5.2+: for AMD GPU support
FFTW3: for CPUs (only used serially, but multi-threading supported; not needed for GPUs)
Optional dependencies include:
MPI 3.0+: for multi-node and/or multi-GPU execution
OpenMP 3.1+: for threaded CPU execution
CCache: to speed up rebuilds (needs 3.7.9+ for CUDA)
Please choose one of the installation methods below to get started:
HPC platforms
If you want to use HiPACE++ on a specific high-performance computing (HPC) systems, jump directly to our HPC system-specific documentation.
Using the Spack package manager
The dependencies can be installed via the package manager Spack (macOS/Linux):
spack env create hipace-dev
spack env activate hipace-dev
spack add adios2 # for .bp file support
spack add ccache
spack add cmake
spack add fftw
spack add hdf5 # for .h5 file support
spack add mpi
spack add pkgconfig # for fftw
# optional:
# spack add cuda
spack install
(in new terminals, re-activate the environment with spack env activate hipace-dev
again)
Note
On Linux distributions, the InstallError "OpenMPI requires both C and Fortran compilers"
can occur because the Fortran compilers are sometimes not set automatically in Spack.
To fix this, the Fortran compilers must be set manually using spack config edit compilers
(more information can be found here).
For GCC, the flags f77 : null
and fc : null
must be set to f77 : gfortran
and fc : gfortran
.
On macOS, a Fortran compiler like gfortran might be missing and must be installed by hand to fix this issue.
Using the Brew package manager
The dependencies can be installed via the package manager Homebrew (macOS/Linux):
brew update
brew install adios2 # for .bp file support
brew install ccache
brew install cmake
brew install fftw
brew install hdf5-mpi # for .h5 file support
brew install libomp
brew install pkg-config # for fftw
brew install open-mpi
Now, cmake --version
should be at version 3.15.0 or newer.
Configure your compiler
For example, using a GCC on macOS:
export CC=$(which gcc)
export CXX=$(which g++)
If you also want to select a CUDA compiler:
export CUDACXX=$(which nvcc)
export CUDAHOSTCXX=$(which g++)
Build & Test
If you have not downloaded HiPACE++ yet, please clone it from GitHub via
git clone https://github.com/Hi-PACE/hipace.git $HOME/src/hipace # or choose your preferred path
From the base of the HiPACE++ source directory, execute:
# find dependencies & configure
cmake -S . -B build
# build using up to four threads
cmake --build build -j 4
# run tests
(cd build; ctest --output-on-failure)
Note: the from_file tests require the openPMD-api with python bindings. See
documentation of the openPMD-api for more information.
An executable HiPACE++ binary with the current compile-time options encoded in its file name will be created in bin/
.
Additionally, a symbolic link named hipace
can be found in that directory, which points to the last built HiPACE++ executable. You can inspect and modify build options after running cmake .. with either
ccmake build
or by providing arguments to the CMake call
cmake -S . -B build -D<OPTION_A>=<VALUE_A> -D<OPTION_B>=<VALUE_B>
CMake Option |
Default & Values |
Description |
|
RelWithDebInfo/Release/Debug |
Type of build, symbols & optimizations |
|
NOACC/CUDA/SYCL/HIP/OMP |
On-node, accelerated computing backend |
|
ON/OFF |
Multi-node support (message-passing) |
|
SINGLE/DOUBLE |
Floating point precision (single/double) |
|
ON/OFF |
openPMD I/O (HDF5, ADIOS2) |
|
LEAPFROG/AB5 |
Use leapfrog or fifth-order Adams-Bashforth plasma pusher |
HiPACE++ can be configured in further detail with options from AMReX, which are documented in the AMReX manual.
Developers might be interested in additional options that control dependencies of HiPACE++. By default, the most important dependencies of HiPACE++ are automatically downloaded for convenience:
CMake Option |
Default & Values |
Description |
|
None |
Path to AMReX source directory (preferred if set) |
|
|
Repository URI to pull and build AMReX from |
|
|
Repository branch for |
|
ON/OFF |
Needs a pre-installed AMReX library if set to |
|
ON/OFF (default is set to value of |
Build openPMD with MPI support, although I/O is always serial |
|
None |
Path to openPMD-api source directory (preferred if set) |
|
|
Repository URI to pull and build openPMD-api from |
|
|
Repository branch for |
|
ON/OFF |
Needs a pre-installed openPMD-api library if set to |
|
ON/OFF |
Compile AMReX multigrid solver. |
For example, one can also build against a local AMReX copy.
Assuming AMReX’ source is located in $HOME/src/amrex
, add the cmake
argument -DHiPACE_amrex_src=$HOME/src/amrex
.
Relative paths are also supported, e.g. -DHiPACE_amrex_src=../amrex
.
Or build against an AMReX feature branch of a colleague.
Assuming your colleague pushed AMReX to https://github.com/WeiqunZhang/amrex/
in a branch new-feature
then pass to cmake
the arguments: -DHiPACE_amrex_repo=https://github.com/WeiqunZhang/amrex.git -DHiPACE_amrex_branch=new-feature
.
You can speed up the install further if you pre-install these dependencies, e.g. with a package manager.
Set -DHiPACE_<dependency-name>_internal=OFF
and add installation prefix of the dependency to the environment variable CMAKE_PREFIX_PATH.
Please see the short CMake tutorial that we linked in the Developers section if this sounds new to you.
Documentation
The documentation is written at the RST format, to compile the documentation locally use
cd docs
# optional: --user
python3 -m pip install -r requirements.txt # only the first time
make html
open build/html/index.html
The last line would work on MacOS. On another platform, open the html file with your favorite browser.