Table of Content
[[TOC]]
The installation instructions will use the following environment variables
WM_PROJECT_DIR
: directory, where OpenFOAM source code provided by The OpenFOAM Foundation is installedWM_PROJECT_USER_DIR
: OpenFOAM user directory, where user's code or addons can be installedWM_PROJECT_SITE
: OpenFOAM site directory, where addons can be installedWM_PROJECT_VERSION
: OpenFOAM version, e.g., dev, 6, 9, 9-s.1To compile the HZDR Multiphase Addon for OpenFOAM an installation of OpenFOAM provided by The OpenFOAM Foundation is required. The source code is available at the OpenFOAM-dev repository at GitHub.
Note, if you have downloaded the HZDR Multiphase Addon for OpenFOAM from Rossendorf Data Repository (RODARE) an archive with the required commit of the OpenFOAM release source code is available for download. Obtaining the source code from GitHub or the provided archive is equal, instead of cloning the repository unpack the archive to the required folder and follow the instructions.
On an Ubuntu Linux system install the following general packages for OpenFOAM
sudo apt-get install build-essential flex bison git-core cmake zlib1g-dev \
libboost-system-dev libboost-thread-dev libopenmpi-dev openmpi-bin \
gnuplot libreadline-dev libncurses-dev libxt-dev ca-certificates libfl-dev
Create a new directory for OpenFOAM
mkdir -p $HOME/OpenFOAM
cd $HOME/OpenFOAM
and enter the newly created directory. Now, clone the ThirdParty sources from GitHub and reset the sources to the required commit sha
git clone https://github.com/OpenFOAM/ThirdParty-dev.git ThirdParty-9-s.1
(cd ThirdParty-9-s.1 && git reset --hard 588fb48a605b80819d371735fba4de2aa3d69ae6)
The same has to be done for the OpenFOAM sources, clone from GitHub repository and reset to the required commit sha
git clone https://github.com/OpenFOAM/OpenFOAM-dev.git OpenFOAM-9-s.1
(cd OpenFOAM-9-s.1 && git reset --hard 01f0753ac7bafff1c39df37b5259adf4df2f380e)
Once all repositories are successfully cloned and reset, the compilation of OpenFOAM can be started as normal
sed -i -e "s/\(WM_PROJECT_VERSION=\).*$/\19-s.1/" OpenFOAM-9-s.1/etc/bashrc
sed -i -e "s/\(WM_PROJECT_VERSION=\).*$/\19-s.1/" OpenFOAM-9-s.1/etc/cshrc
source $HOME/OpenFOAM/OpenFOAM-9-s.1/etc/bashrc
(cd $WM_PROJECT_DIR && ./Allwmake -j)
Once the compilation process is finished successfully, load the OpenFOAM environment and test the installation
source $HOME/OpenFOAM/OpenFOAM-9-s.1/etc/bashrc
foamVersion
For more details concerning other Linux operating systems, additional software packages or more read the installation instructions for OpenFOAM-dev from Source Repository.
There are two possible ways to obtain the source code for the HZDR Multiphase Addon for OpenFOAM:
For usage of the GPU (graphic processor unit) accelerated population balance method a NVIDIA GPU with CUDA support is required as well as an installation of the NVIDIA Cuda Toolkit with the NVCC compiler.
The HZDR Multiphase Addon repository includes content that is versioned using git-lfs
sudo apt-get install git-lfs
to store large binary files in the repository.
Then create a directory for OpenFOAM user developments, clone and compile the addon
source $HOME/OpenFOAM/OpenFOAM-9-s.1/etc/bashrc
mkdir -p $WM_PROJECT_USER_DIR
git clone --single-branch git@gitlab.hzdr.de:openfoam/fwdc/Developments.git $WM_PROJECT_USER_DIR
(cd $WM_PROJECT_USER_DIR && ./Allwmake -j)
Download tar archive for HZDR Multiphase Addon for OpenFOAM from RODARE, unpack into OpenFOAM user directory and compile the addon
source $HOME/OpenFOAM/OpenFOAM-9-s.1/etc/bashrc
mkdir -p $WM_PROJECT_USER_DIR
tar -xzf HZDR-Multiphase-Addon-<version>.tgz -C $WM_PROJECT_USER_DIR
(cd $WM_PROJECT_USER_DIR && ./Allwmake -j)
Docker is a popular software for Operating-System-Level virtualisation and can be used to deliver software in packages and a pre-defined environment. Those packages are called Docker images and a running instance of such an image is called a Docker container. Those containers are isolated from each other and from the host system. For Helmholtz and Friends the HZDR Multiphase Addon for OpenFOAM together with a pre-installed working environment for OpenFOAM can be provided as Docker image. The Docker images in the corresponding Docker registry feature support for graphical desktop applications, such as Paraview. Supported are Intel and NVIDIA graphic cards, but currently no AMD graphic cards.
Optional: A useful tool for defining, deploying and managing Docker
applications is docker compose
. The HZDR Multiphase Addon for OpenFOAM
contains a set of docker compose files to build and launch the desired Docker
images.
Follow the installation instructions for Docker and your operating system on the Docker homepage. For security reasons, it is highly recommended to install Docker in rootless mode.
Optional: If the CUDA functionality of the HZDR Multiphase Addon for
OpenFOAM is of interest, the Nvidia Container Toolkit
has to be installed. Also, if docker compose
is of interest, follow the
installation instructions for docker compose V2+.
The following table contains a list of available Docker images in the project registry
including a pre-compiled OpenFOAM (openfoam*
, recommended for developers) or
also a pre-compiled HZDR Multiphase Addon (hzdr*
, recommended for users).
Docker image tag | OpenFOAM | HZDR Multiphase Addon | X11 | Cuda | Compiler | Compile Option |
---|---|---|---|---|---|---|
hzdrLinuxGccOpt | :white_check_mark: | :white_check_mark: | :white_check_mark: Intel | :x: | Gcc | Optimised |
hzdrNvidiaGccOpt | :white_check_mark: | :white_check_mark: | :white_check_mark: Nvidia | :x: | Gcc | Optimised |
hzdrCudaGccOpt | :white_check_mark: | :white_check_mark: | :x: | :white_check_mark: | Gcc, Nvcc | Optimised |
openfoamLinuxGccOpt | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Gcc | Optimised |
openfoamLinuxClangOpt | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Clang | Optimised |
openfoamLinuxGccDebug | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Gcc | Debugging |
openfoamNvidiaGccOpt | :white_check_mark: | :x: | :white_check_mark: Nvidia | :x: | Gcc | Optimised |
openfoamNvidiaGccDebug | :white_check_mark: | :x: | :white_check_mark: Nvidia | :x: | Gcc | Debugging |
openfoamCudaGccOpt | :white_check_mark: | :x: | :x: | :white_check_mark: | Gcc, Nvcc | Optimised |
The source code of the OpenFOAM Foundation release is installed inside the
docker image in $HOME/OpenFOAM/
and the HZDR Multiphase Addon for OpenFOAM is
located in $WM_PROJECT_SITE
.
docker login registry.hzdr.de
# No X11 support
docker run -it registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> bash
# X11 support for Intel
xhost +
docker run -it --device=/dev/dri:/dev/dri --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> bash
# X11 support for nvidia
xhost +
docker run -it --gpus all --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> bash
# No X11, Cuda support
xhost +
docker run -it --gpus all registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> bash
Note, you have to replace <image>
with an image tag.
For a more comfortable usage a bash script foamDockerRun.sh
is provided to run an OpenFOAM Docker image.
docker login registry.hzdr.de
# X11 support for Intel
foamDockerRun.sh registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image>
# X11 support for nvidia
foamDockerRun.sh --gpus all registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image>
# No X11 but Cuda support
foamDockerRun.sh --gpus all registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image>
For more information and options run foamDockerRun.sh --help
or docker run --help
.
Note, you have to replace <image>
with an image tag.
docker compose
To use docker compose
, first download the pre-configured compose
and environment variable
files. Alternatively, you can clone the repository
and run docker-compose
from inside the directory.
docker compose
uses services to run a pre-defined Docker environment. The
following table contains a list of the available services and the Docker images
that are used. From within the directory containing the compose and environment
variable files, run
docker compose --profile '*' config --services
to get a full list of all available services.
Service | OpenFOAM | HZDR Multiphase Addon | X11 | Cuda | Compiler | Compile Option |
---|---|---|---|---|---|---|
hzdrLinux | :white_check_mark: | :white_check_mark: | :white_check_mark: Intel | :x: | Gcc | Optimised |
hzdrNvidia | :white_check_mark: | :white_check_mark: | :white_check_mark: Nvidia | :x: | Gcc | Optimised |
hzdrCuda | :white_check_mark: | :white_check_mark: | :x: | :white_check_mark: | Gcc, Nvcc | Optimised |
openfoamLinux | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Gcc | Optimised |
openfoamLinuxDebug | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Gcc | Debugging |
openfoamLinuxProf | :white_check_mark: | :x: | :white_check_mark: Intel | :x: | Gcc | Profiling |
openfoamNvidia | :white_check_mark: | :x: | :white_check_mark: Nvidia | :x: | Gcc | Optimised |
openfoamNvidiaDebug | :white_check_mark: | :x: | :white_check_mark: Nvidia | :x: | Gcc | Debugging |
openfoamNvidiaProf | :white_check_mark: | :x: | :white_check_mark: Nvidia | :x: | Gcc | Profiling |
openfoamCuda | :white_check_mark: | :x: | :x: | :white_check_mark: | Gcc, Nvcc | Optimised |
To launch a service from the table above do
docker login registry.hzdr.de
# Launch service in detached mode (background)
docker compose up -d <service>
# execute command in running container
docker compose exec <service> bash
# stop service
docker compose down <service>
Note, you have to replace <service>
with a service name.
The source code of the OpenFOAM Foundation release is installed inside the
docker image in $HOME/OpenFOAM/
and the HZDR Multiphase Addon for OpenFOAM is
located in $WM_PROJECT_SITE
.
This is currently not supported.
Most software packages on Linux systems store there configurations in
~/.config
. To persistently store and share configurations for software
packages, such as Paraview, this directory should be defined and created as a
Docker volume, and shared between Docker containers.
# create docker volume
docker volume create config
# mount volume when launching container
docker run -it --volume config:/root/.config <image> bash
When using foamDockerRun.sh
this volume will be created and shared
automatically. When using docker compose
the volume has to be created
manually, but will be shared automatically.
The Git version control system typically stores its configuration in
~/.gitconfig
. It is recommended to copy the git configuration into the Docker
containers instead of binding the file as a volume to it. When using pure Docker
the following commands will copy the git configuration file into the container
CONTAINERID=$(docker create -it <image>)
docker cp $HOME/.gitconfig $CONTAINERID:/root/.gitconfig
docker start -i $CONTAINERID
For docker compose
at least docker compose V2+
is required, then use the following commands
docker compose up -d <service>
docker compose cp $HOME/.gitconfig <service>:/root/.gitconfig
docker compose exec <service> bash
The script foamDockerRun.sh
will copy the git configuration into the container
automatically. Note, you have to replace <image>
or <service>
with a
image tag or a
service name, respectively.
The required Dockerfile
for building the Docker images with the required OpenFOAM
Foundation version and the HZDR Multiphase Addon for OpenFOAM is available in
etc/docker
. The Dockerfile
is a multi-stage Dockerfile
with the following
two stages:
openfoam
- build Openfoam image without HZDR Multiphase Addonhzdraddon
- build Openfoam image and HZDR Multiphase AddonFor building the Docker images use the following command
docker build --target <openfoam|hzdraddon> --tag <image> --build-arg ... -f etc/docker/Dockerfile .
The available variables for build-arg
are defined by the Dockerfile
and the
default options are listed in the following table
Name | Default | Options | Description |
---|---|---|---|
FROMIMAGE | ubuntu:20.04 | nvidia/opengl:base-ubuntu20.04, nvidia/cuda:11.2.0-devel-ubuntu20.04 | base image with operating system |
WM_PROJECT_VERSION | dev | - | OpenFOAM version, e.g. dev, 9, 9-s.1, ... |
WM_COMPILE_OPTION | Opt | Debug, Prof | Compile Option for OpenFOAM |
WM_COMPILER | Gcc | Clang | Compiler for OpenFOAM |
OFREPOSITORY | https://github.com/OpenFOAM/OpenFOAM-dev.git | - | Git repository for OpenFOAM-dev |
TPREPOSITORY | https://github.com/OpenFOAM/ThirdParty-dev.git | - | Git repository for ThirdParty-dev |
WM_PROJECT_COMMIT | Head | - | Valid Git commit sha for OpenFOAM-dev |
THIRDPARTY_COMMIT | Head | - | Valid Git commit sha for ThirdParty-dev |
PARAVIEW | No | Yes | Build Paraview |
Note, that in particular WM_PROJECT_COMMIT
and THIRDPARTY_COMMIT
have to be
set to the correct commit sha, otherwise the HZDR Multiphase Addon will not
build correctly. The most recent values can be found in README.md.
Building the images with docker compose
is the most easy way and requires
almost no configuration. Use the following commands to build the Docker images
by yourself
# current sprint cycle release
docker compose -f docker-compose.yml -f etc/docker-compose/deployment.yml build <service>
# current OpenFOAM-dev release
docker compose --env-file etc/docker-compose/.env-dev build -f docker-compose.yml -f etc/docker-compose/deployment.yml <service>
The Docker images provide an OpenFOAM environment and the HZDR Multiphase Addon for OpenFOAM. However, additional software like editors or graphical diff tools are not installed as default to keep the size of the Docker images small. If additional software packages are required, the Docker images can be customised.
docker commit
The simplest way is to run the desired Docker image
docker login registry.hzdr.de
# No X11 support
docker run -it registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> bash
and install all additional software packages inside the container
apt-get update
apt-get install ADD_UBUNTU_PACKAGE_HERE
Once all software packages are installed successfully, leave the container
running and open a new terminal. Use docker ps
to extract the container id of
the running container and create a new image from it
docker commit <container id> registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<custom image>
Finally, stop the container and run the new custom image
# No X11 support
docker run -it registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<custom image> bash
Note, you have to replace <image>
with an image tag.
The recommended way is to create your own Dockerfile. An example is shown below
# Build variables required for docker image
ARG FROMIMAGE
FROM $FROMIMAGE
# Install custom ubuntu packages
RUN apt-get update \
&& apt-get install -qy --fix-missing --no-install-recommends \
ADD_UBUNTU_PACKAGE_HERE \
ADD_UBUNTU_PACKAGE_HERE \
ADD_UBUNTU_PACKAGE_HERE
# Slim down packages
RUN apt-get clean autoclean && apt-get autoremove -y
To build your custom image with the selected software packages run
docker build --build-arg FROMIMAGE=registry.hzdr.de/openfoam/fwdc/developments/9-s.1:<image> .
Note, you have to replace <image>
with an image tag.
It is also possible to build a custom image with docker compose
. Therefore, a
Dockerfile
as shown above is required as well as a docker-compose.override.yml
in the project repository folder. Each service refers to a specific Docker image
than can be customised according to
version: "3.9"
services:
<service>:
build:
dockerfile: Dockerfile
args:
FROMIMAGE: "${REGISTRY}/9-s.1:<image>"
Note, you have to replace <image>
and <service>
with an
image tag and with a
service name, respectively. Finally, run the service
# Build customised docker image
docker compose build <service>
# Launch service in deattached mode (background)
docker compose up -d <service>
# execute command in running container
docker compose exec <service> bash
For further information about the structure of the Compose file and its syntax, refer to the general information about Docker Compose as well as the Compose Specification.