Machine Learning for CFD Turbulence Closures

I wrote a couple previous posts on some interesting work using deep learning to accelerate topology optimization, and a couple neural network methods for accelerating computational fluid dynamics (with source). This post is about a use of machine learning in computational fluid dynamics (CFD) with a slightly different goal: to improve the quality of solutions. Rather than a focus on getting to solutions more quickly, this post covers work focused on getting better solutions. A better solution is one that has more predictive capability. There is usually a trade-off between predictive capability, and how long it takes to get a solution. The most well-known area for improvement in predictive capability of state-of-the-practice, industrial CFD is in our turbulence and transition modeling. There are a proliferation of approaches to tackling that problem, but the overall strategy that seems to be paying off is for CFD'ers to follow the enormous investment being made by the large tech companies in techniques, open source libraries, and services for machine learning. How can those free / low-cost tools and techniques be applied to our problems?

The authors of Machine Learning Models of Errors in Large Eddy Simulation Predictions of Surface Pressure Fluctuations used machine learning techniques to model the error in their LES solutions. See an illustration of the instantaneous density gradient magnitude of the developing boundary layer from that paper shown to the right. Here's the abstract,

We investigate a novel application of deep neural networks to modeling of errors in prediction of surface pressure fluctuations beneath a compressible, turbulent flow. In this context, the truth solution is given by Direct Numerical Simulation (DNS) data, while the predictive model is a wall-modeled Large Eddy Simulation (LES
). The neural network provides a means to map relevant statistical flow-features within the LES solution to errors in prediction of wall pressure spectra. We simulate a number of flat plate turbulent boundary layers using both DNS and wall-modeled LES to build up a database with which to train the neural network. We then apply machine learning techniques to develop an optimized neural network model for the error in terms of relevant flow features

They implemented three types of neural networks

• Multi-layer perceptron (MLP) with 159 input nodes, 3 fully-connected hidden layers and an output layer of 159 nodes
• Convolution neural network (CNN) with 159 input nodes, one-dimensional convolutional layer, then a max pool layer and then output layers
• Convolution-Deconvolution neural network (CDNN) with convolution, max-pool, fully-connected and then finally output layers

They also tried different types of training and hold-out approaches for their small database of supersonic boundary layers. The data set for training is small since direct numerical simulation (DNS) is so compute intensive.

A paradigm for data-driven predictive modeling using field inversion and machine learning illustrates a related approach to learning turbulence model-form error as opposed to learning optimal model parameter values. Often in practice the nominal values suggested in the literature for a specific turbulence model are used with no learning (optimization, calibration) for a specific flow. Here's the abstract

We propose a modeling paradigm, termed field inversion and machine learning (FIML), that seeks to comprehensively harness data from sources such as high-fidelity simulations and experiments to aid the creation of improved closure models for computational physics applications. In contrast to inferring model parameters, this work uses inverse modeling to obtain corrective, spatially distributed functional terms, offering a route to directly address model-form errors. Once the inference has been performed over a number of problems that are representative of the deficient physics in the closure model, machine learning techniques are used to reconstruct the model corrections in terms of variables that appear in the closure model. These reconstructed functional forms are then used to augment the closure model in a predictive computational setting. As a first demonstrative example, a scalar ordinary differential equation is considered, wherein the model equation has missing and deficient terms. Following this, the methodology is extended to the prediction of turbulent channel flow. In both of these applications, the approach is demonstrated to be able to successfully reconstruct functional corrections and yield accurate predictive solutions while providing a measure of model form uncertainties.

In much the same spirit as that paper, here's a set of slides on Physics-Informed Machine Learning for Predictive Turbulence Modeling. The author suggests an important difference in approach for applying machine learning to computational physics, "Assist but respect models: Machine learning should be used to correct/improve existing models, not to replace them. Thus, we learn the model discrepancy, not the model output directly." [emphasis in original, slide 12]

I really like this approach because it allows us to benefit from all of the sunk cost in developing the Reynolds Averaged Navier Stokes (RANS) codes engineers use in anger today. We can add these machine learning approaches as "wrappers" around the existing tool-set to improve our predictive capability. A couple critical questions the author presents for applying these approaches,

• Where does the training data come from?
• What are the quantities to learn (responses, targets, dependent variables)? Are they universal, at least to some extent?
• What are the features (predictors, independent variables)?
• What learning algorithm should be used?

The author presents answers addressing these questions for a RANS example.

Another set of slides based on a Phase I NASA LEARN Project illustrates an approach to learning turbulence models. They give a proof of concept using a supervised learning approach where the features are the terms in a Spalart-­‐Allmaras turbulence model.

The slides cover some tweaks the authors made to the loss function to make it more applicable to CFD. The slides also summarize a paper on transition modeling where they fit an indeterminacy field with a couple machine learning techniques: Gausian processes and neural networks. Here's a paper that applies this data-driven, or data augmented, approach to a two-equation RANS model.

This data-driven machine learning approach to improving CFD has plenty of good ideas left to pursue. You could learn parameters of existing models, learn entirely new models, learn how to blend different models, build specific data sets to learn solutions in peculiar application areas, and probably a ton more I haven't even thought of yet. All of these results and different approaches to improving our CFD predictions are exciting, but there's always a catch. So you want to use machine learning to solve your problem? Now you have another problem: how are you going to get enough data? I like the vision for the future at the end of those LEARN Project slides: "A continuously augmented curated database/website of high-­‐fidelity CFD solutions (and experimental data!) that are input to the machine learning process." Something like that would benefit the good ideas we have now, and the ones we haven't got to yet.

This is a pretty neat area of research. Please drop a comment with links to more!