TY - JOUR

T1 - Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression

AU - Morimoto, Masaki

AU - Fukami, Kai

AU - Maulik, Romit

AU - Vinuesa, Ricardo

AU - Fukagata, Koji

N1 - Funding Information:
M.M., Ka.F, and Ko.F thank the support by JSPS KAKENHI Grant Number 18H03758 and 21H05007 . R.V. acknowledges the financial support by the Göran Gustafsson Foundation . This material is based upon work supported by the U.S. Department of Energy (DOE), Office of Science, Office of Advanced Scientific Computing Research , under Contract DE-AC02-06CH11357 . This research was funded in part and used resources of the Argonne Leadership Computing Facility , which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357 . This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. DOE or the United States Government.
Publisher Copyright:
© 2022

PY - 2022/11/15

Y1 - 2022/11/15

N2 - We use Gaussian stochastic weight averaging (SWAG) to assess the epistemic uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sampled weights, which can be used to obtain ensemble predictions. The average of such an ensemble can be regarded as the ‘mean estimation’, whereas its standard deviation can be used to construct ‘confidence intervals’, which enable us to perform uncertainty quantification (UQ) with regard to the training process of neural networks. We utilize representative neural-network-based function approximation tasks for the following cases: (i) a two-dimensional circular-cylinder wake; (ii) the DayMET dataset (maximum daily temperature in North America); (iii) a three-dimensional square-cylinder wake; and (iv) urban flow, to assess the generalizability of the present idea for a wide range of complex datasets. SWAG-based UQ can be applied regardless of the network architecture, and therefore, we demonstrate the applicability of the method for two types of neural networks: (i) global field reconstruction from sparse sensors by combining convolutional neural network (CNN) and multi-layer perceptron (MLP); and (ii) far-field state estimation from sectional data with two-dimensional CNN. We find that SWAG can obtain physically-interpretable confidence-interval estimates from the perspective of epistemic uncertainty. This capability supports its use for a wide range of problems in science and engineering.

AB - We use Gaussian stochastic weight averaging (SWAG) to assess the epistemic uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sampled weights, which can be used to obtain ensemble predictions. The average of such an ensemble can be regarded as the ‘mean estimation’, whereas its standard deviation can be used to construct ‘confidence intervals’, which enable us to perform uncertainty quantification (UQ) with regard to the training process of neural networks. We utilize representative neural-network-based function approximation tasks for the following cases: (i) a two-dimensional circular-cylinder wake; (ii) the DayMET dataset (maximum daily temperature in North America); (iii) a three-dimensional square-cylinder wake; and (iv) urban flow, to assess the generalizability of the present idea for a wide range of complex datasets. SWAG-based UQ can be applied regardless of the network architecture, and therefore, we demonstrate the applicability of the method for two types of neural networks: (i) global field reconstruction from sparse sensors by combining convolutional neural network (CNN) and multi-layer perceptron (MLP); and (ii) far-field state estimation from sectional data with two-dimensional CNN. We find that SWAG can obtain physically-interpretable confidence-interval estimates from the perspective of epistemic uncertainty. This capability supports its use for a wide range of problems in science and engineering.

KW - Fluid flows

KW - Machine learning

KW - Neural network

KW - Uncertainty quantification

UR - http://www.scopus.com/inward/record.url?scp=85135344596&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85135344596&partnerID=8YFLogxK

U2 - 10.1016/j.physd.2022.133454

DO - 10.1016/j.physd.2022.133454

M3 - Article

AN - SCOPUS:85135344596

VL - 440

JO - Physica D: Nonlinear Phenomena

JF - Physica D: Nonlinear Phenomena

SN - 0167-2789

M1 - 133454

ER -