Recently, I dedicated 10 days to the Stanford FLAME AI Workshop: Future Learning Approaches for Modeling and Engineering. This event aimed to provide insights into the confluence of machine learning with fluid dynamics, turbulence, and combustion.

The workshop was a careful blend of expert talks and an engaging ML challenge. We had thought leaders like George Karniadakis from Brown University giving a lecture about “Physics-informed Neural Networks (PINNs) and Neural Operators” [Rai19P] [Kar21P] [DeepONet], Anima Anandkumar from Caltech and Nvidia who discussed “Neural Operators” [Li20F] [Pat22F], Stephan Hoyer from Google Research talking about “Deep Learning with Differentiable Physical for Fluid Dynamics and Weather Forecasting” [Lam23G], and Steve Brunton from the University of Washington sharing thoughts on “Machine Learning for Scientific Discovery” [Bru16D, Bru22D]. These sessions, among many others, have broadened my understanding of how intricate the relationship between fluid dynamics and AI can be.

**Figure 1.** High-resolution image of turbulent flow simulation. Every sample is a slice of a three-dimensional domain and consists of four channels: density and velocties in x-, y-, and z-direction.

Diving Deeper: The ML Challenge

While the lectures were highly inspiring, the core of FLAME was its Kaggle ML challenge. It was structured around the idea of super-resolution in turbulent flows: image super-resolution (SR) addresses the inverse problem of reconstructing high-resolution (HR) images from their low-resolution (LR) counterparts, cf. [Lim17E]. With a data set consisting just over 1000 training samples of snapshots of turbulent flow simulations - taken from a structured data set [Chu22B] - participants had the task of scaling up images of only 16x16 pixels to high-resolution images of 128x128 pixels, which eventually helps to accelerate the simulation of turbulent flows.

The comparatively small dataset and the tight time constraints were a real challenge. I tried to apply physics-informed DeepONets to the problem, but unfortunately couldn’t get it to work within the deadline. In the end, my approach using a simple upscaling supplemented by some convolutional layers (see GitHub) scored the 10th position on the Kaggle leaderboard, and I’m happy to be invited towards a joint publication. Many other leading approches used modifcations of EDSR, a deep residual network architecture for image super-resolution, and the winning approach was based on a recently proposed super-resolution neural operator [Wei23S].

Personal Reflections on the Workshop

Being based in Munich and juggling a 9-hour time difference wasn’t easy, but my interest in the intersection of simulation and AI along with the exciting speakers and engaging challenge of this workshop made it worthwhile.

In all, FLAME 2023 was an enlightening experience: It deepened my understanding and fueled my passion in a field that already fascinates me, and it was a joy to connect with the vibrant community centered around simulation and AI. I look forward to future events of this kind!

References

[Kar21P]

Physics-informed machine learning, George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, Liu Yang.

May 2021

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often …

[Rai19P]

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, M. Raissi, P. Perdikaris, G. E. Karniadakis.

Feb 2019

We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential …

[Li20F]

Fourier Neural Operator for Parametric Partial Differential Equations, Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar.

Oct 2020

The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an …

[Pat22F]

FourCastNet: A global data-driven high-resolution weather model using adaptive Fourier neural operators, Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, Animashree Anandkumar.

Feb 2022

FourCastNet, short for Fourier Forecasting Neural Network, is a global data-driven weather forecasting model that provides accurate short to medium-range global predictions at $0.25^{\circ}$ resolution. FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning …

[Lam23G]

Learning skillful medium-range global weather forecasting., Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, Peter Battaglia.

Nov 2023

Global medium-range weather forecasting is critical to decision-making across many social and economic domains. Traditional numerical weather prediction uses increased compute resources to improve forecast accuracy, but cannot directly use historical weather data to improve the underlying model. We introduce a machine learning-based method called "GraphCast", which can be trained directly from …

[Bru16D]

Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Steven L. Brunton, Joshua L. Proctor, J. Nathan Kutz.

Apr 2016

Extracting governing equations from data is a central challenge in many diverse areas of science and engineering. Data are abundant whereas models often remain elusive, as in climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover …

[Bru22D]

Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Steven L. Brunton, J. Nathan Kutz.

May 2022

Data-driven discovery is revolutionizing how we model, predict, and control complex systems. Now with Python and MATLAB®, this textbook trains mathematical scientists and engineers for the next generation of scientific discovery by offering a broad overview of the growing intersection of data-driven methods, machine learning, applied optimization, and classical fields of engineering mathematics …