physics-informed neural networks

Most of the physical laws that govern the dynamics of a system can be described by partial differential equations. For example, the Navier–Stokes equations{{cite book |last1=Batchelor |first1=G. K. |title=An introduction to fluid dynamics |date=2000 |publisher=Cambridge University Press |location=Cambridge, U.K. |isbn=978-0-521-66396-0 |edition=2nd pbk.}} are a set of partial differential equations derived from the conservation laws (i.e., conservation of mass, momentum, and energy) that govern fluid mechanics. The solution of the Navier–Stokes equations with appropriate initial and boundary conditions allows the quantification of flow dynamics in a precisely defined geometry. However, these equations cannot be solved exactly and therefore numerical methods must be used (such as finite differences, finite elements and finite volumes). In this setting, these governing equations must be solved while accounting for prior assumptions, linearization, and adequate time and space discretization.

Recently, solving the governing partial differential equations of physical phenomena using deep learning has emerged as a new field of scientific machine learning (SciML), leveraging the universal approximation theorem{{cite journal |last1=Hornik |first1=Kurt |last2=Tinchcombe |first2=Maxwell |last3=White |first3=Halbert |title=Multilayer feedforward networks are universal approximators |journal=Neural Networks |date=1989-01-01 |volume=2 |issue=5 |pages=359–366 |doi=10.1016/0893-6080(89)90020-8 |s2cid=2757547 |url=https://www.sciencedirect.com/science/article/abs/pii/0893608089900208 |language=en |issn=0893-6080|url-access=subscription }} and high expressivity of neural networks. In general, deep neural networks could approximate any high-dimensional function given that sufficient training data are supplied.{{cite journal |last1=Arzani |first1=Amirhossein |last2=Dawson |first2=Scott T. M. |title=Data-driven cardiovascular flow modelling: examples and opportunities |journal=Journal of the Royal Society Interface |year=2021 |volume=18 |issue=175 |pages=20200802|doi=10.1098/rsif.2020.0802 |arxiv=2010.00131 |pmid=33561376 |pmc=8086862 }} However, such networks do not consider the physical characteristics underlying the problem, and the level of approximation accuracy provided by them is still heavily dependent on careful specifications of the problem geometry as well as the initial and boundary conditions. Without this preliminary information, the solution is not unique and may lose physical correctness. On the other hand, physics-informed neural networks (PINNs) leverage governing physical equations in neural network training. Namely, PINNs are designed to be trained to satisfy the given training data as well as the imposed governing equations. In this fashion, a neural network can be guided with training data that do not necessarily need to be large and complete. Potentially, an accurate solution of partial differential equations can be found without knowing the boundary conditions.{{cite journal |last1=Arzani |first1=Amirhossein |last2=Wang |first2=Jian-Xun |last3=D'Souza |first3=Roshan M. |title=Uncovering near-wall blood flow from sparse data with physics-informed neural networks |journal=Physics of Fluids |date=2021-06-07 |volume=33 |issue=7 |page=071905 |doi=10.1063/5.0055600 |arxiv=2104.08249|bibcode=2021PhFl...33g1905A |s2cid=233289904 }} Therefore, with some knowledge about the physical characteristics of the problem and some form of training data (even sparse and incomplete), PINN may be used for finding an optimal solution with high fidelity.

PINNs allow for addressing a wide range of problems in computational science and represent a pioneering technology leading to the development of new classes of numerical solvers for PDEs. PINNs can be thought of as a meshfree alternative to traditional approaches (e.g., CFD for fluid dynamics), and new data-driven approaches for model inversion and system identification.{{cite journal |last1=Raissi |first1=Maziar |last2=Perdikaris |first2=Paris |last3=Karniadakis |first3=George Em |title=Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations |journal=Journal of Computational Physics |date=2019-02-01 |volume=378 |pages=686–707 |doi=10.1016/j.jcp.2018.10.045 |bibcode=2019JCoPh.378..686R |osti=1595805 |s2cid=57379996 |language=en |issn=0021-9991|doi-access=free }} Notably, the trained PINN network can be used for predicting the values on simulation grids of different resolutions without the need to be retrained.{{cite arXiv |last1=Markidis |first1=Stefano |title=Physics-Informed Deep-Learning for Scientific Computing |date=2021-03-11 |class=math.NA |eprint=2103.09655}} In addition, they allow for exploiting automatic differentiation (AD){{cite arXiv |last1=Baydin |first1=Atilim Gunes |last2=Pearlmutter |first2=Barak A. |last3=Radul |first3=Alexey Andreyevich |last4=Siskind |first4=Jeffrey Mark |title=Automatic differentiation in machine learning: a survey |date=2018-02-05 |class=cs.SC |eprint=1502.05767}} to compute the required derivatives in the partial differential equations, a new class of differentiation techniques widely used to derive neural networks assessed to be superior to numerical or symbolic differentiation.

A general nonlinear partial differential equation can be:

$u\_t\; +\; N[u;\; \backslash lambda]=0,\; \backslash quad\; x\; \backslash in\; \backslash Omega,\; \backslash quad\; t\; \backslash in[0,\; T]$

where $u(t,x)$ denotes the solution, $N[\backslash cdot;\; \backslash lambda]$ is a nonlinear operator parameterized by $\backslash lambda$, and $\backslash Omega$ is a subset of $\backslash mathbb\{R\}^\{D\}$. This general form of governing equations summarizes a wide range of problems in mathematical physics, such as conservative laws, diffusion process, advection-diffusion systems, and kinetic equations. Given noisy measurements of a generic dynamic system described by the equation above, PINNs can be designed to solve two classes of problems:

• data-driven solution
• data-driven discovery of partial differential equations.

The data-driven solution of PDE computes the hidden state $u(t,x)$ of the system given boundary data and/or measurements $z$, and fixed model parameters $\backslash lambda$. We solve:

$u\_t\; +\; N[u]=0,\; \backslash quad\; x\; \backslash in\; \backslash Omega,\; \backslash quad\; t\; \backslash in[0,\; T]$.

By defining the residual $f(t,x)$ as

$f\; :=\; u\_t\; +\; N[u]=0$,

and approximating $u(t,x)$ by a deep neural network. This network can be differentiated using automatic differentiation. The parameters of $u(t,x)$ and $f(t,x)$ can be then learned by minimizing the following loss function $L\_\{tot\}$:

$L\_\{tot\}\; =\; L\_\{u\}\; +\; L\_\{f\}$.

Where $L\_\{u\}\; =\; \backslash Vert\; u-z\backslash Vert\_\{\backslash Gamma\}$ is the error between the PINN $u(t,\; x)$ and the set of boundary conditions and measured data on the set of points $\backslash Gamma$ where the boundary conditions and data are defined, and $L\_\{f\}\; =\; \backslash Vert\; f\backslash Vert\_\{\backslash Gamma\}$ is the mean-squared error of the residual function. This second term encourages the PINN to learn the structural information expressed by the partial differential equation during the training process.

This approach has been used to yield computationally efficient physics-informed surrogate models with applications in the forecasting of physical processes, model predictive control, multi-physics and multi-scale modeling, and simulation.{{cite arXiv |last1=Raissi |first1=Maziar |last2=Yazdani |first2=Alireza |last3=Karniadakis |first3=George Em |title=Hidden Fluid Mechanics: A Navier–Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data |date=2018-08-13 |class=cs.CE |eprint=1808.04327}} It has been shown to converge to the solution of the PDE.{{Cite journal |last=Doumèche |first=Nathan |last2=Biau |first2=Gérard |last3=Boyer |first3=Claire |date=2025-08-01 |title=On the convergence of PINNs |url=https://projecteuclid.org/journals/bernoulli/volume-31/issue-3/On-the-convergence-of-PINNs/10.3150/24-BEJ1799.full |journal=Bernoulli |volume=31 |issue=3 |doi=10.3150/24-bej1799 |issn=1350-7265|arxiv=2305.01240 }}

Given noisy and incomplete measurements $z$ of the state of the system, the data-driven discovery of PDE results in computing the unknown state $u(t,x)$ and learning model parameters $\backslash lambda$ that best describe the observed data and it reads as follows:

$u\_t\; +\; N[u;\; \backslash lambda]=0,\; \backslash quad\; x\; \backslash in\; \backslash Omega,\; \backslash quad\; t\; \backslash in[0,\; T]$.

By defining $f(t,x)$ as

$f\; :=\; u\_t\; +\; N[u;\; \backslash lambda]=0$,

and approximating $u(t,x)$ by a deep neural network, $f(t,x)$ results in a PINN. This network can be derived using automatic differentiation. The parameters of $u(t,x)$ and $f(t,x)$, together with the parameter $\backslash lambda$ of the differential operator can be then learned by minimizing the following loss function $L\_\{tot\}$:

$L\_\{tot\}\; =\; L\_\{u\}\; +\; L\_\{f\}$.

Where $L\_\{u\}\; =\; \backslash Vert\; u-z\backslash Vert\_\{\backslash Gamma\}$, with $u$ and $z$ state solutions and measurements at sparse location $\backslash Gamma$, respectively and $L\_\{f\}\; =\; \backslash Vert\; f\backslash Vert\_\{\backslash Gamma\}$ residual function. This second term requires the structured information represented by the partial differential equations to be satisfied in the training process.

This strategy allows for discovering dynamic models described by nonlinear PDEs assembling computationally efficient and fully differentiable surrogate models that may find application in predictive forecasting, control, and data assimilation.{{cite journal |last1=Fu |first1=Jinlong |last2=Xiao |first2=Dunhui |last3=Fu |first3=Rui |last4=Li |first4=Chenfeng |last5=Zhu |first5=Chuanhua |last6=Arcucci |first6=Rossella |last7=Navon |first7=Ionel M. |title=Physics-data combined machine learning for parametric reduced-order modelling of nonlinear dynamical systems in small-data regimes |journal=Computer Methods in Applied Mechanics and Engineering |date=February 2023 |volume=404 |pages=115771 |doi=10.1016/j.cma.2022.115771|bibcode=2023CMAME.404k5771F |s2cid=254397985 |url=https://cronfa.swan.ac.uk/Record/cronfa62140 }}{{cite journal |last1=Raissi |first1=Maziar |last2=Yazdani |first2=Alireza |last3=Karniadakis |first3=George Em |title=Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations |journal=Science |date=2020-02-28 |volume=367 |issue=6481 |pages=1026–1030 |doi=10.1126/science.aaw4741 |pmid=32001523 |language=en |issn=0036-8075|pmc=7219083 |bibcode=2020Sci...367.1026R }}{{cite arXiv |last1=Mishra |first1=Siddhartha |last2=Molinaro |first2=Roberto |title=Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs |date=2021-04-01 |class=math.NA |eprint=2007.01138}}{{cite arXiv |last1=Ryck|first1=Tim De |last2=Jagtap|first2=Ameya D.|last3=Mishra |first3=Siddhartha |title=Error estimates for physics informed neural networks approximating the Navier–Stokes equations |date=2022 |class=math.NA |eprint=2203.09346}}{{cite conference |last1=Welin Odeback |first1=Oliver |last2=Bragone |first2=Federica |last3=Laneryd |first3=Tor |last4=Luvisotto |first4=Michele |last5=Morozovska |first5=Kateryna |title=Physics-Informed Neural Networks for prediction of transformer's temperature distribution |conference= 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA) |location=Nassau, Bahamas |date=2022-12-12 |pages=1579–1586 |publisher=IEEE |url=https://ieeexplore.ieee.org/document/10068906}}

PINN is unable to approximate PDEs that have strong non-linearity or sharp gradients that commonly occur in practical fluid flow problems. Piece-wise approximation has been an old practice in the field of numerical approximation. With the capability of approximating strong non-linearity extremely light weight PINNs are used to solve PDEs in much larger discrete subdomains that increases accuracy substantially and decreases computational load as well.{{Cite journal |last1=Dwivedi |first1=Vikas |last2=Parashar |first2=Nishant |last3=Srinivasan |first3=Balaji |date=2021-01-08 |title=Distributed learning machines for solving forward and inverse problems in partial differential equations |url=https://www.sciencedirect.com/science/article/pii/S0925231220314090 |journal=Neurocomputing |language=en |volume=420 |pages=299–316 |doi=10.1016/j.neucom.2020.09.006 |s2cid=225014697 |issn=0925-2312|arxiv=1907.08967 }} DPINN (Distributed physics-informed neural networks) and DPIELM (Distributed physics-informed extreme learning machines) are generalizable space-time domain discretization for better approximation. DPIELM is an extremely fast and lightweight approximator with competitive accuracy. Domain scaling on the top has a special effect. Another school of thought is discretization for parallel computation to leverage usage of available computational resources.

XPINNs{{cite journal |last1=Jagtap |first1=Ameya D. |last2=Karniadakis |first2=George Em |title=Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations |journal=Communications in Computational Physics|date=2020 |volume=28 |issue=5 |pages=2002–2041 |doi=10.4208/cicp.OA-2020-0164 |bibcode=2020CCoPh..28.2002K |osti=2282003 |s2cid=229083388 }} is a generalized space-time domain decomposition approach for the physics-informed neural networks (PINNs) to solve nonlinear partial differential equations on arbitrary complex-geometry domains. The XPINNs further pushes the boundaries of both PINNs as well as Conservative PINNs (cPINNs),{{cite journal |last1=Jagtap |first1=Ameya D. |last2=Kharazmi |first2=Ehsan |last3=Karniadakis |first3=George Em |title=Conservative Physics-Informed Neural Networks on Discrete Domains for Conservation Laws: Applications to forward and inverse problems |journal=Computer Methods in Applied Mechanics and Engineering|date=2020 |volume=365 |pages=113028|doi=10.1016/j.cma.2020.113028|bibcode=2020CMAME.365k3028J |s2cid=216333271 |doi-access=free }} which is a spatial domain decomposition approach in the PINN framework tailored to conservation laws. Compared to PINN, the XPINN method has large representation and parallelization capacity due to the inherent property of deployment of multiple neural networks in the smaller subdomains. Unlike cPINN, XPINN can be extended to any type of PDEs. Moreover, the domain can be decomposed in any arbitrary way (in space and time), which is not possible in cPINN. Thus, XPINN offers both space and time parallelization, thereby reducing the training cost more effectively. The XPINN is particularly effective for the large-scale problems (involving large data set) as well as for the high-dimensional problems where single network based PINN is not adequate. The rigorous bounds on the errors resulting from the approximation of the nonlinear PDEs (incompressible Navier–Stokes equations) with PINNs and XPINNs are proved. However, DPINN debunks the use of residual (flux) matching at the domain interfaces as they hardly seem to improve the optimization.

File:Xtfc scheme.png

In the PINN framework, initial and boundary conditions are not analytically satisfied, thus they need to be included in the loss function of the network to be simultaneously learned with the differential equation (DE) unknown functions. Having competing objectives during the network's training can lead to unbalanced gradients while using gradient-based techniques, which causes PINNs to often struggle to accurately learn the underlying DE solution. This drawback is overcome by using functional interpolation techniques such as the Theory of functional connections (TFC)'s constrained expression, in the Deep-TFC{{cite journal |last1=Leake |first1=Carl |last2=Mortari |first2=Daniele |title=Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations |journal=Machine Learning and Knowledge Extraction |date=March 2020 |volume=2 |issue=1 |pages=37–55 |doi=10.3390/make2010004 |doi-access=free |pmid=32478283 |pmc=7259480 |language=en |issn=2504-4990}} framework, which reduces the solution search space of constrained problems to the subspace of neural network that analytically satisfies the constraints.{{cite journal |last1=Leake |first1=Carl |last2=Mortari |first2=Daniele |title=Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations |journal=Machine Learning and Knowledge Extraction |date=12 March 2020 |volume=2 |issue=1 |pages=37–55 |doi=10.3390/make2010004|pmid=32478283 |pmc=7259480 |doi-access=free }} A further improvement of PINN and functional interpolation approach is given by the Extreme Theory of Functional Connections (X-TFC) framework, where a single-layer Neural Network and the extreme learning machine training algorithm are employed.{{cite journal |last1=Schiassi |first1=Enrico |last2=Furfaro |first2=Roberto |last3=Leake |first3=Carl |last4=De Florio |first4=Mario |last5=Johnston |first5=Hunter |last6=Mortari |first6=Daniele |title=Extreme theory of functional connections: A fast physics-informed neural network method for solving ordinary and partial differential equations |journal=Neurocomputing |date=October 2021 |volume=457 |pages=334–356 |doi=10.1016/j.neucom.2021.06.015|s2cid=236290147 |doi-access=free |bibcode=2021Neurc.457..334S }} X-TFC allows to improve the accuracy and performance of regular PINNs, and its robustness and reliability are proved for stiff problems, optimal control, aerospace, and rarefied gas dynamics applications.{{cite journal |last1=Schiassi |first1=Enrico |last2=De Florio |first2=Mario |last3=Ganapol |first3=Barry D. |last4=Picca |first4=Paolo |last5=Furfaro |first5=Roberto |title=Physics-informed neural networks for the point kinetics equations for nuclear reactor dynamics |journal=Annals of Nuclear Energy |date=March 2022 |volume=167 |pages=108833 |doi=10.1016/j.anucene.2021.108833|bibcode=2022AnNuE.16708833S |s2cid=244913655 }}{{cite journal |last1=Schiassi |first1=Enrico |last2=D’Ambrosio |first2=Andrea |last3=Drozd |first3=Kristofer |last4=Curti |first4=Fabio |last5=Furfaro |first5=Roberto |title=Physics-Informed Neural Networks for Optimal Planar Orbit Transfers |journal=Journal of Spacecraft and Rockets |date=4 January 2022 |volume=59 |issue=3 |pages=834–849 |doi=10.2514/1.A35138|bibcode=2022JSpRo..59..834S |s2cid=245725265 }}{{cite journal |last1=De Florio |first1=Mario |last2=Schiassi |first2=Enrico |last3=Ganapol |first3=Barry D. |last4=Furfaro |first4=Roberto |title=Physics-informed neural networks for rarefied-gas dynamics: Thermal creep flow in the Bhatnagar–Gross–Krook approximation |journal=Physics of Fluids |date=April 2021 |volume=33 |issue=4 |pages=047110 |doi=10.1063/5.0046181|bibcode=2021PhFl...33d7110D |s2cid=234852360 }}

Regular PINNs are only able to obtain the solution of a forward or inverse problem on a single geometry. It means that for any new geometry (computational domain), one must retrain a PINN. This limitation of regular PINNs imposes high computational costs, specifically for a comprehensive investigation of geometric parameters in industrial designs. Physics-informed PointNet (PIPN) {{cite journal |last1=Kashefi |first1=Ali |last2=Mukerji |first2=Tapan |title= Physics-informed PointNet: A deep learning solver for steady-state incompressible flows and thermal fields on multiple sets of irregular geometries |journal= Journal of Computational Physics |date=2022 |volume=468 |page=111510 |doi=10.1016/j.jcp.2022.111510|arxiv=2202.05476 |bibcode=2022JCoPh.46811510K |s2cid=246823515 }} is fundamentally the result of a combination of PINN's loss function with PointNet.{{cite journal |last1=Qi |first1=Charles |last2= Su |first2=Hao |last3=Mo |first3= Kaichun |last4= Guibas |first4= Leonidas |title= Pointnet: Deep learning on point sets for 3d classification and segmentation |journal= Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition |date=2017 |pages=652–660 |arxiv=1612.00593 |url=https://openaccess.thecvf.com/content_cvpr_2017/papers/Qi_PointNet_Deep_Learning_CVPR_2017_paper.pdf}} In fact, instead of using a simple fully connected neural network, PIPN uses PointNet as the core of its neural network. PointNet has been primarily designed for deep learning of 3D object classification and segmentation by the research group of Leonidas J. Guibas. PointNet extracts geometric features of input computational domains in PIPN. Thus, PIPN is able to solve governing equations on multiple computational domains (rather than only a single domain) with irregular geometries, simultaneously. The effectiveness of PIPN has been shown for incompressible flow, heat transfer and linear elasticity.{{cite arXiv |last1=Kashefi |first1=Ali |last2=Mukerji |first2=Tapan |title= Physics-informed PointNet: On how many irregular geometries can it solve an inverse problem simultaneously? Application to linear elasticity |date=2023 |class=cs.LG |eprint=2303.13634}}

Physics-informed neural networks (PINNs) have proven particularly effective in solving inverse problems within differential equations,{{Cite journal |last1=Thanasutives |first1=Pongpisit |last2=Morita |first2=Takashi |last3=Numao |first3=Masayuki |last4=Fukui |first4=Ken-ichi |date=2023-02-01 |title=Noise-aware physics-informed machine learning for robust PDE discovery |url=https://iopscience.iop.org/article/10.1088/2632-2153/acb1f0 |journal=Machine Learning: Science and Technology |volume=4 |issue=1 |pages=015009 |doi=10.1088/2632-2153/acb1f0 |issn=2632-2153|arxiv=2206.12901 |bibcode=2023MLS&T...4a5009T }} demonstrating their applicability across science, engineering, and economics. They have shown to be useful for solving inverse problems in a variety of fields, including nano-optics,{{Cite web |title=Optica Publishing Group |url=https://opg.optica.org/oe/viewmedia.cfm?uri=oe-28-8-11618&html=true |access-date=2024-03-18 |website=opg.optica.org}} topology optimization/characterization,{{Cite journal |last1=Zhang |first1=Enrui |last2=Dao |first2=Ming |last3=Karniadakis |first3=George Em |last4=Suresh |first4=Subra |date=2022-02-18 |title=Analyses of internal structures and defects in materials using physics-informed neural networks |journal=Science Advances |language=en |volume=8 |issue=7 |pages=eabk0644 |doi=10.1126/sciadv.abk0644 |issn=2375-2548 |pmc=8849303 |pmid=35171670|bibcode=2022SciA....8..644Z }} multiphase flow in porous media,{{Cite journal |last1=Serebrennikova |first1=Alexandra |last2=Teubler |first2=Raimund |last3=Hoffellner |first3=Lisa |last4=Leitner |first4=Erich |last5=Hirn |first5=Ulrich |last6=Zojer |first6=Karin |date=2022-12-01 |title=Transport of Organic Volatiles through Paper: Physics-Informed Neural Networks for Solving Inverse and Forward Problems |journal=Transport in Porous Media |language=en |volume=145 |issue=3 |pages=589–612 |doi=10.1007/s11242-022-01864-7 |bibcode=2022TPMed.145..589S |issn=1573-1634|doi-access=free }}{{Cite journal |last1=Abbasi |first1=Jassem |last2=Andersen |first2=Pål Østebø |date=2024-01-01 |title=Application of Physics-Informed Neural Networks for Estimation of Saturation Functions from Countercurrent Spontaneous Imbibition Tests |url=https://onepetro.org/SJ/article/doi/10.2118/218402-PA/540447/Application-of-Physics-Informed-Neural-Networks |journal=SPE Journal |volume=29 |issue=4 |language=en |pages=1710–1729 |doi=10.2118/218402-PA |issn=1086-055X|url-access=subscription }} and high-speed fluid flow.{{Cite journal |last1=Jagtap |first1=Ameya D. |last2=Mao |first2=Zhiping |last3=Adams |first3=Nikolaus |last4=Karniadakis |first4=George Em |date=October 2022 |title=Physics-informed neural networks for inverse problems in supersonic flows |url=https://doi.org/10.1016/j.jcp.2022.111402 |journal=Journal of Computational Physics |volume=466 |pages=111402 |doi=10.1016/j.jcp.2022.111402 |arxiv=2202.11821 |bibcode=2022JCoPh.46611402J |issn=0021-9991}} PINNs have demonstrated flexibility when dealing with noisy and uncertain observation datasets. They also demonstrated clear advantages in the inverse calculation of parameters for multi-fidelity datasets, meaning datasets with different quality, quantity, and types of observations. Uncertainties in calculations can be evaluated using ensemble-based or Bayesian-based calculations.{{Cite journal |last1=Yang |first1=Liu |last2=Meng |first2=Xuhui |last3=Karniadakis |first3=George Em |date=January 2021 |title=B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data |url=https://doi.org/10.1016/j.jcp.2020.109913 |journal=Journal of Computational Physics |volume=425 |pages=109913 |doi=10.1016/j.jcp.2020.109913 |arxiv=2003.06097 |bibcode=2021JCoPh.42509913Y |issn=0021-9991}}{{cite arXiv |last1=Vicens Figueres |first1=Júlia |last2=Vanderhaeghen |first2=Juliette |last3=Bragone |first3=Federica |last4=Morozovska |first4=Kateryna |last5=Shukla |first5=Khemraj |title=$PINN - a Domain Decomposition Method for Bayesian Physics-Informed Neural Networks |date=2025 |class=cs.LG |eprint=2504.19013v1}}

PINNs can also be used in connection with symbolic regression for discovering the mathematical expression in connection with discovery of parameters and functions. One example of such application is the study on chemical ageing of cellulose insulation material,{{cite arXiv |last1=Bragone |first1=Federica |last2=Morozovska |first2=Kateryna |last3=Laneryd |first3=Tor |last4=Shukla |first4=Khemraj |last5=Markidis |first5=Stefano |title=Discovering Partially Known Ordinary Differential Equations: a Case Study on the Chemical Kinetics of Cellulose Degradation |date=2025 |class=cs.LG |eprint=2504.03484}} in this example PINNs are used to first discover a parameter for a set of ordinary differential equations (ODEs) and later a function solution, which is later used to find a more fitting expression using a symbolic regression with a combination of operators.

Ensemble of physics-informed neural networks is applied for solving plane elasticity problems. Surrogate networks are intended for the unknown functions, namely, the components of the strain and the stress tensors as well as the unknown displacement field, respectively. The residual network provides the residuals of the partial differential equations (PDEs) and of the boundary conditions. The computational approach is based on principles of artificial intelligence. Aliki D. Mouratidou, Georgios A. Drosopoulos & Georgios E. Stavroulakis,(2024) Ensemble of physics-informed neural networks for solving plane elasticity problems with examples, Acta Mechanica, 235, pp. 6703–6722. https://doi.org/10.1007/s00707-024-04053-3 This approach can be extended to nonlinear elasticity problems, where the constitutive equations are nonlinear. PINNs can also be used for Kirchhoff plate bending problems with transverse distributed loads and to contact models with elastic Winkler’s foundations.{{Cite journal |last=Muradova |first=Aliki D. |last2=Stavroulakis |first2=Georgios E. |date=2021-12-30 |title=PHYSICS-INFORMED NEURAL NETWORKS FOR ELASTIC PLATE PROBLEMS WITH BENDING AND WINKLER-TYPE CONTACT EFFECTS |url=http://www.sscm.kg.ac.rs/jsscm/index.php/volume-15-number-2-2021/283-paper-05-2021-2 |journal=Journal of the Serbian Society for Computational Mechanics |volume=15 |issue=2 |pages=46–55 |doi=10.24874/jsscm.2021.15.02.05|doi-access=free }}

Deep backward stochastic differential equation method is a numerical method that combines deep learning with Backward stochastic differential equation (BSDE) to solve high-dimensional problems in financial mathematics. By leveraging the powerful function approximation capabilities of deep neural networks, deep BSDE addresses the computational challenges faced by traditional numerical methods like finite difference methods or Monte Carlo simulations, which struggle with the curse of dimensionality. Deep BSDE methods use neural networks to approximate solutions of high-dimensional partial differential equations (PDEs), effectively reducing the computational burden. Additionally, integrating Physics-informed neural networks (PINNs) into the deep BSDE framework enhances its capability by embedding the underlying physical laws into the neural network architecture, ensuring solutions adhere to governing stochastic differential equations, resulting in more accurate and reliable solutions.{{cite journal | last1=Han | first1=J. | last2=Jentzen | first2=A. | last3=E | first3=W. | title=Solving high-dimensional partial differential equations using deep learning | journal=Proceedings of the National Academy of Sciences | volume=115 | issue=34 | pages=8505–8510 | year=2018 | doi=10.1073/pnas.1718942115 | doi-access=free | pmid=30082389 | pmc=6112690 | arxiv=1707.02568 | bibcode=2018PNAS..115.8505H }}

An extension or adaptation of PINNs are Biologically-informed neural networks (BINNs). BINNs introduce two key adaptations to the typical PINN framework: (i) the mechanistic terms of the governing PDE are replaced by neural networks, and (ii) the loss function $L\_\{tot\}$ is modified to include $L\_\{constr\}$, a term used to incorporate domain-specific knowledge that helps enforce biological applicability. For (i), this adaptation has the advantage of relaxing the need to specify the governing differential equation a priori, either explicitly or by using a library of candidate terms. Additionally, this approach circumvents the potential issue of misspecifying regularization terms in stricter theory-informed cases. Lagergren JH, Nardini JT, Baker RE, Simpson MJ, Flores KB (2020). "Biologically-informed neural networks guide mechanistic modeling from sparse experimental data." PLoS Comput Biol 16(12): e1008462. https://doi.org/10.1371/journal.pcbi.1008462Nardini JT (2024). "Forecasting and Predicting Stochastic Agent-Based Model Data with Biologically-Informed Neural Networks." Bull Math Biol 86, 130. https://doi.org/10.1007/s11538-024-01357-2

A natural example of BINNs can be found in cell dynamics, where the cell density $u(x,t)$ is governed by a reaction-diffusion equation with diffusion and growth functions $D(u)$ and $G(u)$, respectively:

$u\_t\; =\; \backslash nabla\; \backslash cdot\; (\; D(u)\; \backslash nabla\; u\; )\; +\; G(u)\; u\; ,\; \backslash quad\; x\; \backslash in\; \backslash Omega,\; \backslash quad\; t\; \backslash in[0,\; T]$

In this case, a component of $L\_\{constr\}$ could be $||D||\_\backslash Gamma$ for , which penalizes values of $D$ that fall outside a biologically relevant diffusion range defined by $D\_\{min\}\; \backslash leq\; D\; \backslash leq\; D\_\{max\}$. Furthermore, the BINN architecture, when utilizing multilayer-perceptrons (MLPs), would function as follows: an MLP is used to construct $u\_\{MLP\}(x,t)$ from model inputs $(x,t)$, serving as a surrogate model for the cell density $u(x,t)$. This surrogate is then fed into the two additional MLPs, $D\_\{MLP\}(u\_\{MLP\})$ and $G\_\{MLP\}(u\_\{MLP\})$, which model the diffusion and growth functions. Automatic differentiation can then be applied to compute the necessary derivatives of $u\_\{MLP\}$,$D\_\{MLP\}$ and $G\_\{MLP\}$ to form the governing reaction-diffusion equation.

Note that since $u\_\{MLP\}$ is a surrogate for the cell density, it may contain errors, particularly in regions where the PDE is not fully satisfied. Therefore, the reaction-diffusion equation may be solved numerically, for instance using a method-of-lines approach.

Translation and discontinuous behavior are hard to approximate using PINNs. They fail when solving differential equations with slight advective dominance and hence asymptotic behaviour causes the method to fail. Such PDEs could be solved by scaling variables.[https://arxiv.org/abs/2111.02987]{{cite arXiv |eprint=2111.02987 |class=cs.LG |first1=Siddharth |last1=Rout |title=Numerical Approximation in CFD Problems Using Physics Informed Machine Learning |year=2019}} Master's Thesis, Indian Institute of Technology Madras, July 2019.

This difficulty in training of PINNs in advection-dominated PDEs can be explained by the Kolmogorov n–width of the solution. {{cite journal |last1=Mojgani |first1=Rambod |last2=Balajewicz |first2=Maciej |last3=Hassanzadeh |first3=Pedram |title=Kolmogorov n–width and Lagrangian physics-informed neural networks: A causality-conforming manifold for convection-dominated PDEs |journal=Computer Methods in Applied Mechanics and Engineering |date=February 2023 |volume=404 |pages=115810 |doi=10.1016/j.cma.2022.115810|arxiv=2205.02902 }}

They also fail to solve a system of dynamical systems and hence have not been a success in solving chaotic equations.{{cite journal | arxiv=2104.02556 | author1=Eric Aislan Antonelo | last2=Camponogara | first2=Eduardo | author3=Laio Oriel Seman | author4=Eduardo Rehbein de Souza | last5=Jordanou | first5=Jean P. | last6=Hubner | first6=Jomi F. | title=Physics-informed neural nets for control of dynamical systems | journal=Neurocomputing | year=2024 | volume=579 | doi=10.1016/j.neucom.2024.127419 }} One of the reasons behind the failure of regular PINNs is soft-constraining of Dirichlet and Neumann boundary conditions which pose a multi-objective optimization problem which requires manually weighing the loss terms to be able to optimize.

More generally, posing the solution of a PDE as an optimization problem brings with it all the problems that are faced in the world of optimization, the major one being getting stuck in local optima. {{Cite arXiv |last1=Wang |first1=Sifan |last2=Teng |first2=Yujun |last3=Perdikaris |first3=Paris |date=2020-01-13 |title=Understanding and mitigating gradient pathologies in physics-informed neural networks |class=cs.LG |eprint=2001.04536}}

• [https://arxiv.org/pdf/2111.02987 Physics Informed Neural Network]
• [https://github.com/maziarraissi/PINNs/ PINN] – repository to implement physics-informed neural network in Python
• [https://github.com/AmeyaJagtap/XPINNs/ XPINN] – repository to implement extended physics-informed neural network (XPINN) in Python
• PIPN [https://github.com/Ali-Stanford/PhysicsInformedPointNetElasticity]– repository to implement physics-informed PointNet (PIPN) in Python

Category:Differential equations

Category:Deep learning