# README for HZDR Multiphase Case Collection for OpenFOAM

[[_TOC_]]

## Description

HZDR Multiphase Case Collection for OpenFOAM contains simulation setups for the
open-source CFD software OpenFOAM with the HZDR Multiphase Addon for
OpenFOAM[^19]. The simulation setups are separated into mono- and polydisperse
bubbly flows utilising the HZDR Baseline model[^20] set, setups for a
morphology-adaptive multifield two-fluid model[^21] (disperse and resolved
interfaces) and miscellaneous cases.

### Cases using the HZDR Baseline model set

#### baseline/1998_Liu

* Reference for experiment: Liu (1998)[^11]
* Reference for case setup: Rzehak et al. (2021)[^15], Kriebitzsch and Rzehak
  (2016)[^8]

#### baseline/2005_Lucas_et_al

* Reference for experiment: Lucas et al. (2005)[^12]
* Reference for case setup: Lehnigk et al. (2021)[^9]

#### baseline/2008_Shawkat

* Reference for experiment: Shawkat et al. (2008)[^16]
* Reference for case setup: Kriebitzsch and Rzehak (2016)[^8]

#### baseline/2009_Hosokawa

* Reference for experiment: Hosokawa and Tomiyama (2009)[^4]
* Reference for case setup: Rzehak et al. (2021)[^15]

#### baseline/2013_Hosokawa_and_Tomiyama

* Reference for experiment: Hosokawa and Tomiyama (2013)[^5]
* Reference for case setup: Kriebitzsch and Rzehak (2016)[^8], Liao et al.
  (2020)[^10]

#### baseline/2016_Kim_et_al

* Reference for experiment: Kim et al. (2016)[^7]
* Reference for case setup: Liao et al. (2020)[^10]

### Cases using the morphology-adaptive modelling approach

#### multimorph/channelCuberoEtAl2014

* Reference for case setup: Cubero et al. (2014)[^33]

#### multimorph/columnTray/MP-06

#### multimorph/elbowCuberoEtAl2014

* Reference for case setup: Cubero et al. (2014)[^33]

#### multimorph/entrainment/hydraulicJump2D

#### multimorph/entrainment/plungingJet2D_Coarse

* Reference for experiment: Chanson et al. (2004)[^34]

#### multimorph/entrainment/plungingJetChansonEtAl2004

* Reference for experiment: Chanson et al. (2004)[^34]

#### multimorph/risingBubbleAdelsbergerEtAl2014

* Reference for case setup: Adelsberger et al. (2014) [^30]

#### multimorph/risingBubbleBalcazarEtAl2015

* Reference for experiment: Bhaga and Weber (1981)[^3]
* Reference for direct numerical simulation: Balcazar et al. (2015)[^2]
* Reference for case setup: Meller et al. (2021)[^13]

#### multimorph/risingBubbleFrederixEtAl2021/regimeII

* Reference for direct numerical simulation: Tripathi et al. (2015)[^31]
* Reference for case setup: Frederix et al. (2021)[^32]

#### multimorph/risingBubbleHysingEtAl2009

* References for case setup: Hysing et al. (2009)[^6], Meller et al.
  (2021)[^13], Meller et al. (2022)[^14]

#### multimorph/risingBubbleMellerEtAl2022

* Reference for case setup: Meller et al. (2022)[^14]

#### multimorph/taylorGreenVortex

#### multimorph/verticalChannel

#### multimorph/wenka/2D-MP3-23

* Reference for experiment: Stäbler (2007)[^17]
* Reference for case setup: Tekavcic et al. (2021)[^18]

### Miscellaneous cases

#### misc/multiphase/addonMultiphaseEulerFoam/1991_Akhtar_et_al

* Reference for experiment: Akhtar et al. (1991)[^1]
* Reference for case setup: Lehnigk et al. (2021)[^9]

## Installation

### General remarks

The installation instructions will use the following environment variables

* `FOAM_RUN`: directory where OpenFOAM simulation setups are stored

### HZDR Multiphase Addon for OpenFOAM

Depending on what you have access to:

* **Helmholtz Code Base**[^22]: For Helmholtz and Friends via Helmholtz AAI[^23]
  or using a HZDR guest account
* **Rossendorf Data Repository (RODARE)**[^19]: For everybody

you can install the HZDR Multiphase Addon for OpenFOAM in several ways:

* as Debian packages
* by compiling from sources
* by downloading and running the provided Docker Images

Follow the installation instructions in your preferred download source and
make sure your OpenFOAM environment is setup correctly, e.g. by running
`foamVersion`

### HZDR Multiphase Case Collection from Helmholtz Code Base

Note that the repository includes content that is versioned using
git-lfs[^29] to store large binary files in the repository:

```shell
sudo apt update
sudo apt install git-lfs
```

After successful installation, simply clone the HZDR Multiphase Case Collection
into OpenFOAM run directory

```shell
mkdir -p $FOAM_RUN
git clone --single-branch git@codebase.helmholtz.cloud:openfoam/fwdc/Cases.git $FOAM_RUN
```

### HZDR Multiphase Case Collection from Rossendorf Data Repository (RODARE)

Download tar archive for the HZDR Multiphase Addon for OpenFOAM[^24]
from RODARE and unpack it into  OpenFOAM run directory

```shell
mkdir -p $FOAM_RUN
tar -xzf HZDR-Multiphase-Case-Collection-<version>.tgz -C $FOAM_RUN
```

## Workflow Management with Snakemake

The repository is configured as a Snakemake[^25] workflow, which allows the
scalable execution of a set of cases as well as the creation of self-contained
HTML reports containing all generated plots and runtime statistics.

Note: The Snakemake workflow feature is not yet part of the Rossendorf Data
Repository (RODARE)[^24] and requires access to the Helmholtz Code Base[^22].

### Installation of Snakemake

Snakemake can be directly installed from `pip`:

```shell
sudo apt install python3 python3-pip
pip3 install snakemake==7.18.2 pygments --upgrade
```

The package `pygments` is required for creating Snakemake reports. For
alternative ways of installing Snakemake, refer to the Snakemake
documentation[^25]. Currently, the minimum supported version is 7.18.2.

### Executing a Snakemake Workflow

#### Recommended case structure

The workflow implementation assumes that cases archived in the repository
feature scripts for case

1. **configuration**, `Allconfigure`, e.g. setting parameters with
`foamDictionary`
2. **pre-processing**, `Allpre`, e.g. defining cell sets with `topoSet` and
meshing with `blockMesh`
3. **solution**, `Allsolve`, i.e. mesh decomposition, running the corresponding
application and mesh reconstruction
4. **post-processing**, `Allpost`, e.g. line sampling and generation of plots
5. **execution**, `Allrun`, which executes all of the former scripts and
constitutes the entry point for a user who wants to run a case
6. **resetting**, `Allclean`, i.e. to return to the original state of the case
7. **validation**, `Allvalidate`, e.g. comparing results against reference
solutions
8. **updating reference solutions**, `Allupdate`

While this is the recommended structure for efficiently using the workflow to
maintain a validation data base, the only **mandatory scripts** are `Allrun`
and `Allclean`. If any of the other scripts are not present, they are simply
not executed during the corresponding step in the workflow. If the `Allsolve`
script is missing, `Allrun` is executed as a fallback solution. This way,
cases that only feature the standard scripts `Allrun` and `Allclean` can be
added to the workflow as well.

#### Local execution

The setup and result directories as well as the cases to be included in the
workflow need to be specified in `workflow.yml`. To configure the workflow, go
to the root directory of the repository, e.g.

```shell
cd $FOAM_RUN
```

assuming it has been cloned into the OpenFOAM run directory. Then run

```shell
snakemake -q --cores <numberOfCores> Allconfigure
```

with `<numberOfCores>` >= 1, which copies the preconfigured setups into
the result directory and generates a case-specific `Snakefile`. For more verbose
output, remove the `-q` option from the Snakemake command. Pre-processing of the
cases is a separate step achieved by

```shell
snakemake -q -k --cores <numberOfCores> Allpre
```

Note the `-k` option which keeps the workflow execution going even if an
individual case fails, which otherwise stops by default. In any case, a partial
failure will cause the workflow to return a non-zero exit code. Finally, the
cases can then be run by

```shell
snakemake -q -k --cores <numberOfCores> -k Allsolve
```

with `<numberOfCores>` >= 2. If there are cases that are set up with more
subdomains than the number of threads passed to the snakemake command, they will
be automatically scaled down. Substituting `<numberOfCores>` by `all` makes
Snakemake utilize all available cores. If hyperthreading is activated on the
machine, it will use all threads. After the solution step, the results
can be post-processed via

```shell
snakemake -q -k --cores <numberOfCores> Allpost
```

All generated graphs can then be gathered in an HTML validation report by

```shell
snakemake --report report.html Allpost
```

which can be viewed in a webbrowser. In order to include runtime information for
the solution step in the report, include the Allsolve rule in the command

```shell
snakemake --report report.html Allsolve Allpost
```

The workflow can be reset to the status after the configuration step by

```shell
snakemake -q --cores <numberOfCores> Allclean
```

In order to quickly test the Snakemake workflow, it can be useful to set all
setups to run a single time step only, as done in a merge request pipeline for
example. This can be achieved through the `tutorials/Alltest`[^22] script, by
copying it into the repositories root directory and running

```shell
./Alltest -from . -to . -configure
```

before the workflow configuration step.

#### Cluster execution

For the configuration, post-processing and reporting steps of the workflow, the
commands are the same as described in the section on [local execution](#local-execution).
For running all cases on a cluster running the Slurm Workload Manager, a
top-level job needs to be submitted which executes the solve rule by

```shell
snakemake --cluster "sbatch -p <partition> -n {threads}" --jobs <number> -k Allsolve
```

where `<jobs>` should be substituted by maximum number of (sub)jobs that
Snakemake is supposed to submit on the cluster. The resources required for each
job are themselves extracted from the `threads` keyword in the individual
Snakefiles. An example submission script is provided by `etc/hemera`. The same
command can also be applied to execute other rules, if their ressource demand
justifies it.

## Repository Structure

The HZDR Multiphase Case Collection for OpenFOAM includes the following main
directories and files:

* **baseline:** directory containing mono- and poly-disperse bubbly flow cases
* **etc:** header templates for dictionaries, scripts and Snakefiles
* **flotation:** directory for three-phase flotation cases
* **multimorph:** directory for cases using the morphology-adaptive modelling
  approach
* **misc:** directory for various `interFoam` and `addonMultiphaseEulerFoam`
  setups with experimental data
* **workflow:** stores workflow-relevant files for `baseline_testing`[^26]
  workflows and reStructuredText[^27] headers for Snakemake [^25] reports
* **baseline.rules:** top-level simulation workflow specification
* **baseline.yaml:** top-level workflow configuration file
* **baseline-test.yaml:** top-level workflow configuration file for workflow
  tests and dry runs
* **configure.rules:** top-level configuration workflow specification
* **codemeta.json:** software metadata according to The CodeMeta Project[^28]
* **CONTRIBUTING.md**: how to contribute to the project
* **LICENSE**: licensing information

## How to cite us?

When using the HZDR Multiphase Case Collection for OpenFOAM cite us as

>>>
Haensch, S., Draw, M., Evdokimov, I., Khan, H., Kamble, V., Krull, B., Lehnigk,
R., Liao, Y., Lyu, H., Meller, R., Schlegel, F., Tekavcic, M. (2023).
HZDR Multiphase Case Collection for OpenFOAM. Rodare.
<http://doi.org/10.14278/rodare.811>
>>>

## Links and References

[^1]: Akhtar, M. K., Xiong, Y., & Pratsinis, S. E. (1991). Vapor synthesis of titania powder by titanium tetrachloride oxidation. AIChE Journal, 37(10), 1561-1570.
[^2]: Balcazar, N., Lehmkuhl, O., Jofre, L., & Oliva, A. (2015). Level-set simulations of buoyancy-driven motion of single and multiple bubbles. International Journal of Heat and Fluid Flow, 56, 91-107.
[^3]: Bhaga, D., & Weber, M. E. (1981). Bubbles in viscous liquids: shapes, wakes and velocities. Journal of fluid Mechanics, 105, 61-85.
[^4]: Hosokawa, S., & Tomiyama, A. (2009). Multi-fluid simulation of turbulent bubbly pipe flows. Chemical Engineering Science, 64(24), 5308-5318.
[^5]: Hosokawa, S., & Tomiyama, A. (2013). Bubble-induced pseudo turbulence in laminar pipe flows. International journal of heat and fluid flow, 40, 97-105.
[^6]: Hysing, S. R., Turek, S., Kuzmin, D., Parolini, N., Burman, E., Ganesan, S., & Tobiska, L. (2009). Quantitative benchmark computations of two-dimensional bubble dynamics. International Journal for Numerical Methods in Fluids, 60(11), 1259-1288.
[^7]: Kim, M., Lee, J. H., & Park, H. (2016). Study of bubble-induced turbulence in upward laminar bubbly pipe flows measured with a two-phase particle image velocimetry. Experiments in Fluids, 57(4), 1-21.
[^8]: Kriebitzsch, S., & Rzehak, R. (2016). Baseline model for bubbly flows: simulation of monodisperse flow in pipes of different diameters. Fluids, 1(3), 29.
[^9]: Lehnigk, R., Bainbridge, W., Liao, Y., Lucas, D., Niemi, T., Peltola, J., & Schlegel, F. (2022). An open-source population balance modeling framework for the simulation of polydisperse multiphase flows. AIChE Journal, 68(3), e17539.
[^10]: Liao, Y., Upadhyay, K., & Schlegel, F. (2020). Eulerian-Eulerian two-fluid model for laminar bubbly pipe flows: Validation of the baseline model. Computers & Fluids, 202, 104496.
[^11]: Liu, T. J. (1998, June). The role of bubble size on liquid phase turbulent structure in two-phase bubbly flow. In Proc. Third International Congress on Multiphase Flow ICMF (Vol. 98, pp. 8-12).
[^12]: Lucas, D., Krepper, E., & Prasser, H. M. (2005). Development of co-current air-water flow in a vertical pipe. International Journal of Multiphase Flow, 31(12), 1304-1328.
[^13]: Meller, R., Schlegel, F., & Lucas, D. (2021). Basic verification of a numerical framework applied to a morphology adaptive multifield two-fluid model considering bubble motions. International Journal for Numerical Methods in Fluids, 93(3), 748-773.
[^14]: Meller, R., Schlegel, F., & Klein, M. (2022). Sub-grid Scale Modelling and a-Posteriori Tests with a Morphology Adaptive Multifield Two-Fluid Model Considering Rising Gas Bubbles. Flow, Turbulence and Combustion 108, 895-922.
[^15]: Rzehak, R., Liao, Y., Meller, R., Schlegel, F., Lehnigk, R., & Lucas, D. (2021). Radial pressure forces in Euler-Euler simulations of turbulent bubbly pipe flows. Nuclear Engineering and Design, 374, 111079.
[^16]: Shawkat, M. E., Ching, C. Y., & Shoukri, M. (2008). Bubble and liquid turbulence characteristics of bubbly flow in a large diameter vertical pipe. International Journal of Multiphase Flow, 34(8), 767-785.
[^17]: Stäbler, T. D. (2007). Experimentelle Untersuchung und physikalische Beschreibung der Schichtenströmung in horizontalen Kanälen. PhD Thesis, Universitaet Stuttgart.
[^18]: Tekavcic, M., Meller, R., & Schlegel, F. (2021). Validation of a morphology adaptive multi-field two-fluid model considering counter-current stratified flow with interfacial turbulence damping. Nuclear Engineering and Design, 379, 111223.
[^19]: <https://doi.org/10.14278/rodare.767>
[^20]: <https://www.hzdr.de/db/Cms?pNid=121&pOid=50671>
[^21]: <https://hzdr.de/multimorph>
[^22]: <https://codebase.helmholtz.cloud/openfoam/fwdc/Developments>
[^23]: <https://aai.helmholtz.de>
[^24]: <https://doi.org/10.14278/rodare.811>
[^25]: <https://snakemake.readthedocs.io/en/stable/>
[^26]: <https://codebase.helmholtz.cloud/openfoam/fwdc/Documentation/-/wikis/Baseline%20Workflow/Baseline-Workflow:-Testing-with-Snakemake>
[^27]: <https://wiki.python.org/moin/reStructuredText>
[^28]: <https://codemeta.github.io>
[^29]: <https://git-lfs.github.com>
[^30]: Adelsberger, J., Esser, P., Griebel, M., Groß, S., Klitz, M., & Rüttgers, A. (2014). 3D incompressible two-phase flow benchmark computations for rising droplets. In Proceedings of the 11th world congress on computational mechanics (WCCM XI), Barcelona, Spain (Vol. 179).
[^31]: Tripathi, M. K., Sahu, K. C., & Govindarajan, R. (2015). Dynamics of an initially spherical bubble rising in quiescent liquid. Nature communications, 6(1), 1-9.
[^32]: Frederix, E. M. A., Dovizio, D., Mathur, A., & Komen, E. M. J. (2021). All-regime two-phase flow modeling using a novel four-field large interface simulation approach. International Journal of Multiphase Flow, 145, 103822.
[^33]: Cubero, A., Sánchez-Insa, A., & Fueyo, N. (2014). A consistent momentum interpolation method for steady and unsteady multiphase flows. Computers & chemical engineering, 62, 96-107.
[^34]: Chanson, H., Aoki, S. I., & Hoque, A. (2004). Physical modelling and similitude of air bubble entrainment at vertical circular plunging jets. Chemical engineering science, 59(4), 747-758.
