Physics-informed DeepONets

Physics-informed DeepONets can learn the solution operator of arbitrary partial differential equations, even without any paired input-output training data. This methodology is reported to have the potential to accelerate simulations by up to three orders of magnitude compared to conventional methods.

The dynamic behavior of physical systems is often described by conservation and constitutive laws, expressed as systems of partial differential equations (PDEs). A classical task involves the use of computational tools to solve such equations across a range of scenarios, e.g., different domain geometries, input parameters, and initial and boundary conditions. Solving these so-called parametric PDEs using traditional tools (e.g., finite element methods) comes with an enormous computational cost, as independent simulations need to be performed for every different domain geometries or input parameter.

In the noteworthy paper [Wan21L], the authors propose the framework of physics-informed DeepONets for solving parameterized PDEs. It is a simple yet remarkably effective extension of the DeepONet framework [Lu21L] (cf. Learning nonlinear operators: the DeepONet architecture). By constraining the outputs of a DeepONet to approximately satisfy an underlying governing law, substantial improvements in predictive accuracy, enhanced generalization performance even for out-of-distribution prediction, as well as enhanced data efficiency can be observed.

Figure 1 [Wan21L]: Making DeepONets physics informed. The DeepONet architecture consists of two subnetworks, the branch net for extracting latent representations of input functions and the trunk net for extracting latent representations of input coordinates at which the output functions are evaluated. A continuous and differentiable representation of the output functions is then obtained by merging the latent representations extracted by each subnetwork via a dot product. Automatic differentiation can then be used to formulate appropriate regularization mechanisms for biasing the DeepONet outputs to satisfy a given system of PDEs. BC, boundary conditions; IC, initial conditions. Operator learning techniques have demonstrated early promise across a range of applications, but their application to solving parametric PDEs faces two fundamental challenges: First, they require a large corpus of paired input-output observations and, second, their predicted output functions are not guaranteed to satisfy the underlying PDE. Motivated by the fact that the outputs of a DeepONet model are differentiable with respect to their input coordinates, one can use automatic differentiation to formulate an appropriate regularization mechanism in the spirit of physics-informed neural networks [Rai19P]. The target output functions of the DeepONet are biased to satisfy the underlying PDE constraints by incorporating these into the loss function of the network.

When the collection of all trainable weights of a DeepONet is denoted by $\theta$, the network is optimized with respect to the loss function $$ \mathcal{L}(\theta) = \mathcal{L}{\small\text{operator}}(\theta) + \mathcal{L}{\small\text{physics}}(\theta) $$ where $\mathcal{L}{\small\text{operator}}$ fits the available solution measurements and $\mathcal{L}{\small\text{physics}}$ enforces the underlying PDE constraints.

Figure 4 [Wan21L]: Solving a parametric Burgers’ equation. (Top) Exact solution versus the prediction of the best-trained physics-informed DeepONet. The resulting relative L2 error of the predicted solution is 3%. (Bottom) Computational cost (s) for performing inference with a trained physics-informed DeepONet model [conventional or modified multilayer perceptron (MLP) architecture], as well as corresponding timing for solving a PDE with a conventional spectral solver (58). Notably, a trained physics-informed DeepONet model can predict the solution of $O(10^3)$ time-dependent PDEs in a fraction of a second, up to three orders of magnitude faster compared to a conventional PDE solver. Reported timings are obtained on a single NVIDIA V100 graphics processing unit (GPU).

The authors demonstrate the effectiveness of the physics-informed DeepONets in solving parametric ordinary differential equations, diffusion-reaction and Burgers’ transport dynamics, as well as advection and Eikonal equations. In the diffusion-reaction example, the physics-informed DeepONet yields ∼80% improvement in prediction accuracy with 100% reduction in the dataset size required for training. For Burgers’ equation, notably, a trained physics-informed DeepONet model can predict the solution up to three orders of magnitude (1000x) faster compared to a conventional solver.

Figure 6 [Wan21L]: Solving a parametric eikonal equation (airfoils). (Top) Exact airfoil geometry versus the zero-level set obtained from the predicted signed distance function for three different input examples in the test dataset. (Bottom) Predicted signed distance function of a trained physics-informed DeepONet for three different airfoil geometries in the test dataset.

Despite the promise of the framework demonstrated in the paper, the authors acknowledge that many technical questions remain. For instance, for a given parametric governing law: What is the optimal feature embedding or network architecture of a physics-informed DeepONet? Addressing these questions might not only enhance the performance of physics-informed DeepONets, but also introduce a paradigm shift in how we model and simulate complex physical systems.

If you are interested in using physics-informed DeepONets (and other physics-informed neural operators) in your application, check out continuiti, our Python package for learning function operators with neural networks that includes utilities for implementing PDEs in the loss function.


In this series