README for Multiphase Cases Repository by HZDR

[[TOC]]

Multiphase Cases Repository by HZDR

This repository contains simulation setups of the Multiphase Cases Repository by HZDR for OpenFOAM Foundation Software 1. The simulation setups are divided into mono- and polydisperse bubbly flows utilising the set of Baseline models of HZDR2, setups using the morphology-adaptive multifield two-fluid model3 (unresolved and resolved interfaces) and miscellaneous cases.

Acknowledgement: OpenFOAM(R) is a registered trade mark of OpenCFD Limited, producer and distributor of the OpenFOAM(R) software via www.openfoam.com. The Multiphase Cases Repository by HZDR for OpenFOAM Foundation Software is not compatible with the software released by OpenCFD Limited, but is based on the software released by the OpenFOAM Foundation via www.openfoam.org.

Highlights of the Multiphase Cases Repository by HZDR

Cases using the Baseline model set by HZDR

Folder Reference for Experiment Reference for Case Setup
cases/baseline/1998_Liu Liu (1998)4 Rzehak et al. (2021)5, Kriebitzsch and Rzehak (2016)6
cases/baseline/1999_Pfleger_et_al Pfleger et al. (1999)7 :x:
cases/baseline/2000_Deen_et_al Deen et al. (2000)8 :x:
cases/baseline/2005_Lucas_et_al Lucas et al. (2005)9 Lehnigk et al. (2022)10
cases/baseline/2008_Shawkat Shawkat et al. (2008)11 Kriebitzsch and Rzehak (2016)[^8]
cases/baseline/2009_Hosokawa Hosokawa and Tomiyama (2009)12 Rzehak et al. (2021)[^15]
cases/baseline/2009_Mudde_et_al Mudde et al. (2009)13 :x:
cases/baseline/2012_Akbar_et_al Akbar et al. (2012)14 :x:
cases/baseline/2013_Hosokawa_and_Tomiyama Hosokawa and Tomiyama (2013)15 Kriebitzsch and Rzehak (2016)[^8], Liao et al. (2020)16
cases/baseline/2016_Kim_et_al Kim et al. (2016)17 Liao et al. (2020)[^10]
cases/baseline/2019_Ziegenhein_and_Lucas Ziegenhein and Lucas (2019)18 :x:
Folder Reference for Experiment/Direct Numerical Simulation Reference for Case Setup
cases/multimorph/2009_Hysing_et_al Hysing et al. (2009)19 Hysing et al. (2009)[^6], Meller et al. (2021)20
cases/multimorph/2014_Adelsberger_et_al :x: Adelsberger et al. (2014) 21
cases/multimorph/2014_Cubero_et_al :x: Cubero et al. (2014)22
cases/multimorph/2015_Balcazar_et_al Bhaga and Weber (1981)23, Balcazar et al. (2015)24 Meller et al. (2021)[^13]
cases/multimorph/columnTray/MP-06 Wiedemann et al. (2023)25 Wiedemann et al. (2023)[^33]
cases/multimorph/hydraulicJump2D :x: :x:
cases/multimorph/plungingJetChansonEtAl2004 Chanson et al. (2004)26 :x:
cases/multimorph/risingBubbleFrederixEtAl2021/regimeII Tripathi et al. (2015)27 Frederix et al. (2021)28
cases/multimorph/risingBubbleHysingEtAl2009 :x: Meller et al. (2021, 2022)[^13],29
cases/multimorph/risingBubbleMellerEtAl2022 :x: Meller et al. (2022)[^14]
cases/multimorph/shipHullAirLubrication Elbing et al. (2008)30 :x:
cases/multimorph/taylorGreenVortex :x: :x:
cases/multimorph/verticalChannel :x: :x:
cases/multimorph/wenka/2D-MP3-23 Staebler (2007)31 Tekavcic et al. (2021)32

Miscellaneous cases

Folder Reference for Experiment Reference for Case Setup
cases/misc/multiphase/addonMultiphaseEuler/1991_Akhtar_et_al Akhtar et al. (1991)33 Lehnigk et al. (2022)[^9]

Installation

Prerequisites

For running the cases in this repository, you need to install the software provided through the Multiphase Code Repository by HZDR for OpenFOAM Foundation Software. Depending on what you have access to

  • Helmholtz Code Base34: For Helmholtz and Friends via Helmholtz AAI35 or using a HZDR guest account
  • Rossendorf Data Repository (RODARE)[^19]: For everybody

you can install the software in several ways:

  • as Debian packages
  • by compiling from sources
  • by pulling the provided Docker or Apptainer Images

Follow the installation instructions for your preferred approach and make sure your environment is setup correctly, e.g. by running foamVersion.

The repository is prepared for batch processing the contained setups and creating validation reports containing all generated plots as well as runtime statistics. Batch processing is realised using the Snakemake workflow management system36. If you wish to use this functionality, install the multiphasepy package37 by following the corresponding installation instructions. Installing the package with recommended dependencies automatically provides Snakemake and the required plugin for cluster execution in the correct version. Note that batch processing with a standalone installation of Snakemake will not work, since the workflow implementation relies on functionality provided by multiphasepy.

General remarks

The installation instructions will use the following environment variable:

  • FOAM_RUN: directory where simulation setups are stored

Multiphase Cases Repository by HZDR from Helmholtz Code Base

Note that this repository includes content that is versioned using git-lfs38 to store large binary files in the repository

sudo apt update
sudo apt install git-lfs

After successful installation, simply clone the repository

mkdir -p $FOAM_RUN
git clone --single-branch git@codebase.helmholtz.cloud:fwdc/multiphase/cases.git $FOAM_RUN

Multiphase Cases Repository by HZDR from Rossendorf Data Repository (RODARE)

Download tar archive 39 from RODARE and unpack it

mkdir -p $FOAM_RUN
tar -xzf Multiphase-Cases-Repository-<version>.tgz -C $FOAM_RUN

Running cases

Standalone cases

Cases can be run by executing the Allrun script from within the case folder.

Templated cases

Note that for templated cases (containing a template directory) the specific case name/number, as listed in constant/caseTable under #Case, needs to be set in constant/caseSettings.orig. Without modification a default case (typically the first listed in caseTable) will be executed. The tabulated case settings are set automatically inside a separate Allconfigure script. Note that the Allconfigure script needs to be executed prior to Allrun.

Batch processing cases

Batch runs are defined in the file workflow.yml. Users must specify the source directory containing the cases in their original form, the target directory where the cases are copied to for execution and the cases to be included in the run. With the single_timestep option a batch run can be quickly tested before its submission.

Profiles are used to configure the execution in different environments, i.e. locally on a PC (profiles/local/config.yaml) or remotely on a Slurm-based HPC cluster (profiles/slurm/config.yaml).

Snakemake also supports the use of Apptainer images, i.e. the simulation runs can be processed on any system that has Snakemake and Apptainer installed. Consult profiles/local/config.yaml or profiles/slurm/config.yaml for further information.

Local execution

The maximum number of cores to be utilized in a batch run and other options are set in the file profiles/local/config.yaml which needs to be edited accordingly. Snakemake will scale down cases that request more than the provided number of cores.

To initialize the target directory, change to the repository's root directory and run

snakemake --profile profiles/local init

which copies the setups into the target_dir directory and automatically generates a .snakefile for each case. The location of the relevant configuration file config.yaml is given through the --profile option.

Then start the workflow by

snakemake --profile profiles/local

which configures and runs all cases listed in workflow.yml by executing the case-level Allconfigure and Allrun scripts. The former is only executed if present, the latter is mandatory. Configuring and running cases can also be split into separate steps by

snakemake --profile profiles/local Allconfigure
snakemake --profile profiles/local Allrun

An HTML report can be generated by

snakemake --report

which gathers PNG files from case-level postProcessing/report directories of all cases included in the workflow.

This only works if the workflow has finished without any errors. For a running workflow or a workflow that contains failed cases, this command throws an IncompleteFilesException. To generate a premature report that contains only the plots of simulations that are already finished use the --rerun-incomplete, --ri option

snakemake --report --ri

Any of the above steps can also be executed for a subset of cases only, by configuring the workflow.yml accordingly. Re-running any step of the workflow can be done with the --forceall, -F option.

Slurm cluster execution

The maximum number of jobs to be submitted in parallel as well as the partition to submit the jobs to are provided in the file profiles/slurm/config.yaml which needs to be edited accordingly. Note that the current template assumes that the cluster runs the Slurm workload manager. The command sequence is identical to a local execution, but another profile has to be utilized:

snakemake --profile profiles/slurm init

The set of commands to run the workflow can simply be issued on the submit node.

snakemake --profile profiles/slurm

Cancelling Ctrl+c the execution sends an scancel to the individual jobs submitted by Snakemake. To allow logging out use the terminal multiplexer tmux to create a separate session for every workflow you start

tmux new-session -s <sessionName>
snakemake --profile profiles/slurm init
snakemake --profile profiles/slurm

Detach from the session by pressing Ctrl+b, then d. You can then log out. In order to kill the workflow, reattach to the session

tmux attach -t <sessionName>

and cancel the script execution by pressing Ctrl+c. An existing tmux session is killed by Ctrl+d or entering exit. Note that existing sessions can be listed by

tmux ls

Debugging a batch run

The default settings in the configuration files (profiles/local and profiles/slurm)

keep-going: True      # Keeps workflow alive if a single job fails.
keep-incomplete: True # Keeps incomplete log files.
quiet: rules          # Reports failed jobs.
                      # Other options: progress (report everything)
                      #                all (report nothing)

cause the workflow to continue running even if individual jobs fail. Snakemake will report about errors, e.g.

Error in rule Allrun_run_case:
    jobid: 1
    input: run/case/log.Allconfigure
    output: run/case/log.Allrun, run/case/report
    shell:

            workflow/scripts/Allrun.sh "$(dirname run/case/log.Allrun)" "3" "True"

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Exiting because a job execution failed. Look above for error message

The content of this second last error message is irrelevant, as it just points to the general shell code for executing the All<script> of the corresponding rule. To debug a case, consult the corresponding log.All<script>, which is kept due to the keep-incomplete: true setting. The case-level All<script> exits at the first error that occurs and reports the error code returned by the last command. The actual error message is also shown, unless the problematic command writes to its own log file, e.g. if started with the OpenFOAM run function runApplication. In the latter case, consult the individual log file.

If multiple jobs failed, triggering the problematic workflow step again, e.g.

snakemake --profile profiles/local Allrun

provides a compact list of failed jobs

IncompleteFilesException:
The files below seem to be incomplete. If you are sure that certain files are \
not incomplete, mark them as complete with

    snakemake --cleanup-metadata <filenames>

To re-generate the files rerun your command with the --rerun-incomplete flag.
Incomplete files:
run/case/log.Allrun
run/case/report

A problematic case can be fixed in-place, i.e. in the target_dir directory.

  • Rerunning the problematic rule requires the --rerun-incomplete, --ri option, e.g.

    snakemake --profile profiles/local Allrun --ri

  • Note that there is also the Allclean rule

    snakemake --profile profiles/local Allclean

    which resets all cases listed in the workflow.yml within the target directory. Caution: for this the workflow.yml may only contain the problematic cases. Otherwise all cases are cleaned and results are lost. After fixing cases, rerun the workflow as usual.

A problematic case can also be fixed directly in the source cases directory.

  • One way to do so is to first remove the case from the target directory and then to simply restart the workflow.

  • An alternative to deletion is to force the execution of the init rule

    snakemake --profile profiles/local init -F

    which deletes the cases from the target directory and copies them again from the source directory. Caution: for this the workflow.yml may only contain the problematic cases. Otherwise all cases are re-initalized and results are lost.

Model Testing

For efficiently testing the influence of a certain model or parameter selection for a range of simulation setups, e.g. in the scope of a Snakemake workflow, the corresponding dictionary entry can be placed at a central location and included in individual cases using the #include directive provided by OpenFOAM.

In the following example, the lift model selection in constant/phaseProperties for a case using the addonMultiphaseEuler solver module is centralized.

lift
{
    air_dispersedIn_water
    {
        #include "~/OpenFOAM/cases/models/lift.cfg"
    }
}

The content of ~/OpenFOAM/cases/models/lift.cfg could be

type  Tomiyama;

aspectRatio
{
    type  Wellek;
}

Note: To allow parallel testing of different model selections, the directory containing the files to be included should be relative to the directory to which the Cases Repository was cloned. Globally overwriting a model via the #includeEtc is discouraged for this reason.

Quantification of the prediction quality

A systematic analysis of results can be supported by a quantification of the agreement of simulation results and experimental data. For that purpose a fuzzy logic controller 40 is introduced that evaluates the agreement of predicted plots with validation data. It generates a single goodness value between 0 and 1 for the overall prediction quality. This goodness evaluation is performed for all available validation plots individually.

For details on the fuzzy controller please checkout the mpyfuzzy utility of the multiphasepy package [^34] and use mpyfuzzy --help for further options.

The fuzzy logic evaluation can be executed via an additional Snakemake rule after a successful workflow run by

snakemake --profile profiles/local Allfuzzy

This will execute a fuzzy script for all cases that contain a case.yml file, and (currently) an Allfuzzy script that executes the mpyfuzzy command for the given case. The case.yml specifies the name and location of experimental data, reference and current solution of all available validation fields.

Alternatively, the fuzzy script can be executed for individual cases without using Snakemake by simply executing mpyfuzzy --case CASE.

The results of the fuzzy evaluation, including error metrics and goodness values for both reference and current solution, are then written into a case level postProcessing/mpyfuzzy directory. Separate .csv files are created for each validation field.

For plotting purposes a jupyter notebook to be found under workflow/scripts/plotGoodness.ipynb can be used. A working environment for the notebook is provided with the multiphasepy package [^34].

Note that for the notebook script to work it has to be copied into the top-level workflow directory. Cases to be plotted can then be controlled via the workflow.yml file.

The script plots the error metrics and goodness results for all validation fields for the selected cases. The script also averages goodness results over all the individual validation fields in order to produce one global goodness value for each case, and plots this overall goodness for all selected cases.

Analysing the Feature Hierarchy of Cases

Results of Model testing can be visualized using a decision tree analysis 41. Keywords describing each case and listed in the case.yml files serve as labels, while the values computed as a Quantification of the prediction quality can be used as the target feature for building decision tree models.

An automated script for plotting a decision tree from the change in goodness after model testing can be found under workflow/scripts/decisionTreeAnalysis.ipynb. In order to use the script the sklearn42 package needs to be installed, everything else is provided with the multiphasepy package [^34].

Repository Structure

The repository includes the following main directories and files:

Directory Description
etc header templates for dictionaries, scripts and Snakefiles, template for case setup, keyword list
profiles contains configuration files for executing the Snakemake[^25] workflow locally or on a cluster
cases/baseline directory containing mono- and poly-disperse bubbly flow cases
cases/flotation directory for three-phase flotation cases
cases/misc directory for various incompressibleVoF and addonMultiphaseEuler setups with experimental data
cases/multimorph directory for cases using the morphology-adaptive modelling approach
workflow stores files relevant for batch processing with Snakemake workflow execution and report generation
codemeta.json software metadata according to The CodeMeta Project43
CONTRIBUTING.md how to contribute to the project
LICENSE licensing information
workflow.yml top-level configuration file for batch runs

How to cite us?

When using the Multiphase Cases Repository by HZDR cite as

> Haensch, S., Draw, M., Evdokimov, I., Khan, H., Kamble, V., Krull, B., Lehnigk, R., Liao, Y., Lyu, H., Meller, R., Schlegel, F., Li, S., Tekavcic, M. (2024). Multiphase Cases Repository by HZDR for OpenFOAM Foundation Software. Rodare. http://doi.org/10.14278/rodare.811 >

Links and References

  1. https://doi.org/10.14278/rodare.767↩

  2. https://www.hzdr.de/db/Cms?pNid=121&pOid=50671↩

  3. https://hzdr.de/multimorph↩

  4. Liu, T. J. (1998, June). The role of bubble size on liquid phase turbulent structure in two-phase bubbly flow. In Proc. Third International Congress on Multiphase Flow ICMF (Vol. 98, pp. 8-12).↩

  5. Rzehak, R., Liao, Y., Meller, R., Schlegel, F., Lehnigk, R., & Lucas, D. (2021). Radial pressure forces in Euler-Euler simulations of turbulent bubbly pipe flows. Nuclear Engineering and Design, 374, 111079.↩

  6. Kriebitzsch, S., & Rzehak, R. (2016). Baseline model for bubbly flows: simulation of monodisperse flow in pipes of different diameters. Fluids, 1(3), 29.↩

  7. Pfleger, D., Gomes, S., Gilbert, N., & Wagner, H. G. (1999). Hydrodynamic simulations of laboratory scale bubble columns fundamental studies of the Eulerian-Eulerian modelling approach. Chemical Engineering Science, 54(21), 5091-5099.↩

  8. Deen, N.G., Hjertager, B.H., & Solberg, T. (2000a) Comparison of PIV and LDA Measurement Methods Applied to the Gas-Liquid Flow in a Bubble-Column. In: 10th Int. Symp. on Appl. of Laser Techniques to Fluid Mech., Lisbon, Portugal.↩

  9. Lucas, D., Krepper, E., & Prasser, H. M. (2005). Development of co-current air-water flow in a vertical pipe. International Journal of Multiphase Flow, 31(12), 1304-1328.↩

  10. Lehnigk, R., Bainbridge, W., Liao, Y., Lucas, D., Niemi, T., Peltola, J., & Schlegel, F. (2022). An open-source population balance modeling framework for the simulation of polydisperse multiphase flows. AIChE Journal, 68(3), e17539.↩

  11. Shawkat, M. E., Ching, C. Y., & Shoukri, M. (2008). Bubble and liquid turbulence characteristics of bubbly flow in a large diameter vertical pipe. International Journal of Multiphase Flow, 34(8), 767-785.↩

  12. Hosokawa, S., & Tomiyama, A. (2009). Multi-fluid simulation of turbulent bubbly pipe flows. Chemical Engineering Science, 64(24), 5308-5318.↩

  13. Mudde, R. F., Harteveld, W. K., and van den Akker, H. E. A., (2009). Uniform flow in bubble-columns. Industrial & Engineering Chemistry Research 48, 148-158.↩

  14. Akbar, M. H. M., Hayashi, K., Hosokawa, S., & Tomiyama, A. (2012). Bubble tracking simulation of bubble-induced pseudoturbulence. Multiphase Science and Technology, 24, 197-222.↩

  15. Hosokawa, S., & Tomiyama, A. (2013). Bubble-induced pseudo turbulence in laminar pipe flows. International journal of heat and fluid flow, 40, 97-105.↩

  16. Liao, Y., Upadhyay, K., & Schlegel, F. (2020). Eulerian-Eulerian two-fluid model for laminar bubbly pipe flows: Validation of the baseline model. Computers & Fluids, 202, 104496.↩

  17. Kim, M., Lee, J. H., & Park, H. (2016). Study of bubble-induced turbulence in upward laminar bubbly pipe flows measured with a two-phase particle image velocimetry. Experiments in Fluids, 57(4), 1-21.↩

  18. Ziegenhein, T. and Lucas, D. 2019. The critical bubble diameter of the lift force in technical and environmental, buoyancy-driven bubbly flows. International Journal of Multiphase Flow, 116, pp. 26-38.↩

  19. Hysing, S. R., Turek, S., Kuzmin, D., Parolini, N., Burman, E., Ganesan, S., & Tobiska, L. (2009). Quantitative benchmark computations of two-dimensional bubble dynamics. International Journal for Numerical Methods in Fluids, 60(11), 1259-1288.↩

  20. Meller, R., Schlegel, F., & Lucas, D. (2021). Basic verification of a numerical framework applied to a morphology adaptive multifield two-fluid model considering bubble motions. International Journal for Numerical Methods in Fluids, 93(3), 748-773.↩

  21. Adelsberger, J., Esser, P., Griebel, M., Gross, S., Klitz, M., & Ruettgers, A. (2014). 3D incompressible two-phase flow benchmark computations for rising droplets. In Proceedings of the 11th world congress on computational mechanics (WCCM XI), Barcelona, Spain (Vol. 179).↩

  22. Cubero, A., Sanchez-Insa, A., & Fueyo, N. (2014). A consistent momentum interpolation method for steady and unsteady multiphase flows. Computers & chemical engineering, 62, 96-107.↩

  23. Bhaga, D., & Weber, M. E. (1981). Bubbles in viscous liquids: shapes, wakes and velocities. Journal of fluid Mechanics, 105, 61-85.↩

  24. Balcazar, N., Lehmkuhl, O., Jofre, L., & Oliva, A. (2015). Level-set simulations of buoyancy-driven motion of single and multiple bubbles. International Journal of Heat and Fluid Flow, 56, 91-107.↩

  25. Wiedemann, P., Meller, R., Schubert, M., & Hampel, U. (2023). Application of a hybrid multiphase CFD approach to the simulation of gas-liquid flow at a trapezoid fixed valve for distillation trays. Chemical Engineering Research and Design, 193, 777-786.↩

  26. Chanson, H., Aoki, S. I., & Hoque, A. (2004). Physical modelling and similitude of air bubble entrainment at vertical circular plunging jets. Chemical engineering science, 59(4), 747-758.↩

  27. Tripathi, M. K., Sahu, K. C., & Govindarajan, R. (2015). Dynamics of an initially spherical bubble rising in quiescent liquid. Nature communications, 6(1), 1-9.↩

  28. Frederix, E. M. A., Dovizio, D., Mathur, A., & Komen, E. M. J. (2021). All-regime two-phase flow modeling using a novel four-field large interface simulation approach. International Journal of Multiphase Flow, 145, 103822.↩

  29. Meller, R., Schlegel, F., & Klein, M. (2022). Sub-grid Scale Modelling and a-Posteriori Tests with a Morphology Adaptive Multifield Two-Fluid Model Considering Rising Gas Bubbles. Flow, Turbulence and Combustion 108, 895-922.↩

  30. Elbing, B. R., Winkel, E. S., Lay, K. A., Ceccio, S. L., Dowling, D. R., & Perlin, M. (2008). Bubble-induced skin-friction drag reduction and the abrupt transition to air-layer drag reduction. Journal of Fluid Mechanics, 612, 201-236. https://doi.org/10.1017/s0022112008003029↩

  31. Staebler, T. D. (2007). Experimentelle Untersuchung und physikalische Beschreibung der Schichtenstroemung in horizontalen Kanaelen. PhD Thesis, Universitaet Stuttgart.↩

  32. Tekavcic, M., Meller, R., & Schlegel, F. (2021). Validation of a morphology adaptive multi-field two-fluid model considering counter-current stratified flow with interfacial turbulence damping. Nuclear Engineering and Design, 379, 111223.↩

  33. Akhtar, M. K., Xiong, Y., & Pratsinis, S. E. (1991). Vapor synthesis of titania powder by titanium tetrachloride oxidation. AIChE Journal, 37(10), 1561-1570.↩

  34. https://codebase.helmholtz.cloud/fwdc/multiphase/code↩

  35. https://aai.helmholtz.de↩

  36. https://snakemake.readthedocs.io/en/stable/↩

  37. https://codebase.helmholtz.cloud/fwdc/python↩

  38. https://git-lfs.github.com↩

  39. https://doi.org/10.14278/rodare.811↩

  40. Haensch, S., Evdokimov, I. & Schlegel, F. (2021). A workflow for the sustainable development of closure models for bubbly flows. Chemical Engineering Science, 244, 116807.↩

  41. Evdokimov, I. & Haensch, S. (2021). Quality Assessment of CFD Software Using Workflows and Decision Trees. In: Ivannikov ISPRAS Open Conference, Moscow, Russian Federation.↩

  42. https://scikit-learn.org/stable/↩

  43. https://codemeta.github.io↩