Investigation of Physics-Informed Neural Networks to Reconstruct a Flow Field with High Resolution

Abstract

Particle image velocimetry (PIV) is a widely used experimental technique in ocean engineering, for instance, to study the vortex fields near marine risers and the wake fields behind wind turbines or ship propellers. However, the flow fields measured using PIV in water tanks or wind tunnels always have low resolution; hence, it is difficult to accurately reveal the mechanics behind the complex phenomena sometimes observed. In this paper, physics-informed neural networks (PINNs), which introduce the Navier–Stokes equations or the continuity equation into the loss function during training to reconstruct a flow field with high resolution, are investigated. The accuracy is compared with the cubic spline interpolation method and a classic neural network in a case study of reconstructing a two-dimensional flow field around a cylinder, which is obtained through direct numerical simulation. Finally, the validated PINN method is applied to reconstruct a flow field measured using PIV and shows good performance.

1. Introduction

Particle image velocimetry (PIV) is a transient and noncontact fluid mechanics velocity measurement method that uses tracers and a high-speed camera. It has a wide range of applications within the field of offshore engineering. For instance, it is used to observe the wakes around a cylinder in a uniform flow to reveal the mechanics of vortex-induced vibration and many new vortex-shedding modes that are very important in the flow-induced load prediction of marine rises [1,2,3]. PIV is also applied to the study of the wakes behind wind turbines or ship propellers and can be used for the design of the airfoil profiles of such structures [4,5,6]. However, due to the restrictions of the experimental apparatus, the flow field measured using PIV does not always have very high resolution.

Over the years, to further study the mechanism behind the complex phenomena of the flow field, many scholars have devoted considerable effort to finding methods to enhance the resolution of the data obtained through PIV. The most convenient approach would be to interpolate the low-resolution flow field with methods such as Lagrange interpolation, Newton interpolation, and Hermite interpolation. However, the performance is not always satisfactory for cases in which the turbulent flow field has a complex vortex morphology.

In recent years, machine learning has developed rapidly and has been widely used in a variety of fields. Some scholars have applied it to fluid dynamics. Rao et al. [7] applied deep learning to low-Reynolds-number steady-state and transient laminar flow simulations. Fukami et al. [8] used a convolutional neural network (CNN) model and a hybrid downsampled skip-connection multiscale (DSC/MS) machine learning model to reconstruct laminar flow from low-resolution flow field data. Meng et al. [9] predicted the drag and lift coefficients of blunt body flow and cylindrical flow based on a CNN model, which significantly improved the prediction accuracy. However, the traditional neural network is normally considered a “black box” that cannot reflect the physical properties of the flow field. To address this problem, Karpatne et al. [10] developed physics-guided neural networks that can consider the physical relationships between temperature, density, and water depth. Raissi et al. [11,12,13,14,15] proposed the concept of physics-informed neural networks (PINNs) for nonlinear partial differential equations. They also conducted research on deep learning of vortex-induced vibration (VIV) and hidden fluid mechanics. The framework of PINNs imposes physical information constraints on the loss function of the neural network so that a more generalized model can be trained with fewer data samples. The primary advantage of using PINNs lies in their superior extrapolation capabilities compared to other interpolation methods.

Many researchers have applied PINNs to various fields where the physical information involved is different depending on the specific research objectives. Cai et al. [16,17] trained a neural network by minimizing a loss function composed of a data mismatch term and residual terms associated with the coupled Navier–Stokes and heat transfer equations. They inferred a complete continuous three-dimensional velocity field and pressure field from the three-dimensional temperature field snapshot obtained from Tomo-BOS imaging. Mao et al. [18] used a PINN to approximate Euler’s equations for simulating high-speed aerodynamic flows. Pang et al. [19] extended PINNs to fractional PINNs to solve space–time fractional advection–diffusion equations and analyzed the convergence systematically. Cuomo et al. [20] made a comprehensive review of scientific machine learning through PINNs and compared its advantages and disadvantages with traditional neural networks.

In this study, the performance of a PINN to reconstruct a flow field with high resolution was investigated. The ability of the PINN to deal with the difficulties induced by the sparseness and noise of the data was mainly evaluated. The flow around a cylinder in a uniform from direct numerical simulation (DNS) and experimental data measured by PIV were used in this case study. The predicted accuracy was compared with that of the cubic spline interpolation method and the classic neural network. It was found that the PINN yielded a better reconstructed flow field in all the cases studied in this paper.

2. Methodology

The classic case of a uniform flow past a cylinder in a two-dimensional space was studied. In this case study, the flow field is represented by six physical quantities: the time t, the coordinates x and y, the velocity of the flow particle u and v, and the pressure p.

2.1. Classic Neural Networks

The topology of the classic multilayer perceptron (MLP) neural network [21] applied in this case study is shown in Figure 1, consisting of input, hidden, and output layers. The time t and the coordinates x and y are selected as the three features for the input layer, while the velocity of the flow particle u and v and the pressure p are the features for the output layer. Therefore, both the input and output layers are composed of three nodes. Different numbers of hidden layers and different numbers of nodes in the layers are tested to determine the best topology. Each node is connected to all the nodes in front and behind it, thus forming a neural network. The connection of the node can be expressed as:

$$a_{i,j}=f\left({\displaystyle \sum _{n=1}^j{w}_{i,j,n}\cdot a_{i-1,n}+b_{i,j}}\right)$$

(1)

where a_i,j refers to the jth node in the ith layer in the neural network. W_i,j,n and b_i,j are the weight and bias at the corresponding node, respectively. F(x) refers to the activation function.

The mean square error (MSE) is defined as the loss function, which can be expressed as:

$$L_1=MSE\left(u,v,p\right)=\frac{1}{3n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^3{\left(Y_{i,j}^{out}-Y_{i,j}^{true}\right)}^2}}$$

(2)

Here, n refers to the number of samples. Y_i,1, Y_i,2, and Y_i,3 represent the ith sampling data of u, v, and p, respectively. The superscripts true and out refer to the true and predicted values, respectively.

2.2. Physics-Informed Neural Networks

The neural network introduced above is considered a “black box” since it only considers the features in the input and output layers to be independent numbers. However, there sometimes exist some physical relations among them. In this case study, the continuity equation and Navier–Stokes equations should be satisfied for viscous incompressible Newtonian fluids and can be written as [22]:

$$\begin{array}{l}{u}_x+v_y=0\\ u_t+u\cdot u_x+v\cdot v_y=-\frac{1}{\rho }{p}_x+\nu \left(u_{xx}+u_{yy}\right)\\ v_t+u\cdot v_x+v\cdot v_y=-\frac{1}{\rho }{p}_y+\nu \left(v_{xx}+v_{yy}\right)\end{array}$$

(3)

Here, the subscripts t, x, and y refer to the partial derivative with respect to the time and the coordinate, while the subscripts xx and yy represent the second-order partial derivative to the coordinate. These physical relations can be considered in the neural network by defining a new loss function. In the MLP, the loss function is defined as the difference between the true value and the predicted value of the velocity and pressure. Here, in the PINN, another three components are introduced into the loss function that can be expressed as:

$$\begin{array}{l}{L}_2=e_0+{\beta }_1{e}_1+{\beta }_2{e}_2+{\beta }_3{e}_3\\ e_0=MSE\left(u,v,p\right)\\ e_1=u_x+v_y\\ e_2=u_t+u\cdot u_x+v\cdot v_y+\frac{1}{\rho }{p}_x-\nu \left(u_{xx}+u_{yy}\right)\\ e_3=v_t+u\cdot v_x+v\cdot v_y+\frac{1}{\rho }{p}_y-\nu \left(v_{xx}+v_{yy}\right)\end{array}$$

(4)

where e₀ represents the mean square error between the output values and true values of u, v, and p, as also described in Equation (2). E₁, e₂, and e₃ refer to the difference between the left and right sides of Equation (3). Β_i corresponds to a weight for each term. The topology of the neural network is shown in Figure 2. It is termed PINN #1 in this paper, where β₁, β₂, and β₃ are set to be equal to one. This means that the weight of satisfaction of the continuity equation and Navier–Stokes equations are considered to be as equally important as the mean square error of the velocity and pressure field.

For the DNS training data, the pressure can be obtained directly from the numerically simulated results. However, for the PIV data, only the velocity field can be measured, and the reconstruction of the pressure field based on the velocity field with noise can introduce uncertain error [23]. Therefore, a PINN that only considers the continuity equation, as shown in Figure 3, denoted by PINN #2 in this paper, was also studied. In PINN #2, β₂ and β₃ are set to zero. The loss function L₃ is written as:

$$\begin{array}{l}{L}_3=e_0^{\prime }+e_1\\ e_0^{\prime }=MSE\left(u,v\right)\\ e_1=u_x+v_y\end{array}$$

(5)

Here, $e_0^{\prime }$ represents the mean square error between the output values and true values of u and v.

3. Training Data

Two sets of data of the flow field around a two-dimensional cylinder [24] were studied, which were numerically simulated based on the DNS method and experimentally measured using PIV. Since the numerical results already had a very high resolution and no noise, the data were first diluted and then added with some white Gaussian noise. The reconstruction of the processed field data could be used to evaluate the performance of different neural networks.

3.1. Original and Diluted DNS Data

The computational efforts for the DNS increased rapidly with an increasing Reynolds number. In this case, the Reynolds number was 200, and the data were obtained based on the finite volume method described in [25,26]. The center of the cylinder was at the origin of the coordinates, as shown in Figure 4. The sampling area was −3D to 8D in the x direction and −3D to 3D in the Y direction, where D refers to the diameter of the cylinder. The length of the selected data was 50 s, with a sampling frequency of 4 Hz. Meanwhile, at each time step, the distribution of the sampling spatial points was inhomogeneous, and there were more sampling points near the cylinder and in the wake field, as shown in Figure 4, with a total number of 9957. The flow field of the vortex shedding was periodic. The contour and time series of the velocity and pressure at positions A, B, and C are depicted in Figure 5.

As mentioned above, the DNS data already had a high resolution. Therefore, the data were first diluted, which means that a large amount of the data was removed. The remaining data were used to train the neural network for reconstructing the missing flow field. The reconstructed result was compared with the original removed data to validate the accuracy of the proposed methods. As 50%, 20%, 10%, and 5% of the sampling points were left, their spatial distribution is shown in Figure 6.

3.2. PIV Data

The quality of the PIV data depends heavily on the density of the tracer particles and the performance of the high-speed camera. Normally, the data have a lower resolution, which may not meet research requirements. In this case, PIV data were gathered from a recirculation water channel with a test section measuring 300 mm × 250 mm × 2000 mm. The flow was sufficiently seeded by hollow glass spheres, with an average diameter of 20 μm and a density of 1.03 g/mm², which were well mixed by recirculating in the water channel. Then, during the experiment, the images were recorded using a charge-coupled device (CCD) camera. Details of the experimental apparatus can be seen in [27]. The cylinder diameter was 15 mm, which was far less than the cylinder length. The incoming flow velocity was 0.111 m/s. With the kinematic viscosity of water $\nu$ = 1.111 × 10⁻⁶ m²/s, the Reynolds number was 1500. The range of the measurement area by PIV was 96 mm × 80 mm. The sampling points tended to be evenly distributed, with a total of 152 × 127 velocity vectors. The pressure could not be measured using PIV directly. In addition, only a rectangular area of 60 mm × 40 mm near the cylinder was selected for the training. The center of the cylinder was at the origin of the coordinates, which was out of the view of the PIV camera. The sampling area was 2.7D to 6.6D in the x direction and −1.3D to 1.3D in the y direction, where D refers to the diameter of the cylinder, as shown in Figure 7. The length of the measured data was 600 s, with a sampling frequency of 5 Hz. The time slot when vortex shedding occurred was selected.

4. Results and Discussion

The performance of the neural network was strongly related to the hyperparameters selected, including, for instance, the number of hidden layers, the number of nodes in each layer, and the activation function. Therefore, a hyperparameter analysis was first performed. Then, the determined topology of the neural network was used for the reconstruction of the diluted DNS data and measured PIV data.

4.1. Hyperparameter Analysis

Different combinations of the hyperparameters listed in

Table 1. List of the hyperparameters analyzed.

No.	Hyperparameter	Value
1	Number of hidden layers	5
2		10
3		15
4	Number of nodes in each hidden layer	72
5		96
6		120
7	Activation function	Sin
8		Tanh
9		ReLU
10		Leaky ReLU

were tested for the DNS data with a dilution rate of 50%. The performance of the trained network was evaluated based on the mean square error between the initial and predicted values, as well as the training efficiency.

During the training process, the Adam optimizer was adopted, which divided the overall epochs into three parts, and each part reduced the learning rate to 10% of the previous part. The number of epochs remained the same for different combinations of hyperparameters. It was trained until the loss function converged. An example of the loss function during the training is given in Figure 8.

Considering the accuracy and training time needed, the most appropriate hyperparameters were selected as follows: the number of neural network layers was 10, the number of nodes in each layer was 96, and the activation function was the Sin function. The predicted results of the 9957 sampling spatial points described in Figure 4 are all plotted in Figure 9. It showed very good accuracy. The hyperparameter combination was also used later for the PINN. The time series of the DNS data and the predicted values of u, v, and p at three selected positions are also plotted in Figure 10. They coincided with each other very well in this case.

4.2. Reconstruction of the DNS Data with Different Dilution Rates

The flow field was diluted with only 50%, 30%, 20%, 10%, 5%, and 2% of the data left, which were used to train the neural network. The training of the neural network was conducted on a laptop computer. It had a twelve-core 2.6GHz CPU, an RTX1660Ti display card, and a 16G RAM. Taking the case of the reconstruction of 5% diluted DNS data as an example, to guarantee the convergence of the training, the MLP, PINN #1, and PINN #2 took 18 min, 5.1 h, and 54 min, respectively. The process of the dilution of the flow field simulated using DNS to low resolution and the reconstruction of the diluted flow field to high resolution is shown in Figure 11.

The reconstructed velocity and pressure fields using different neural networks in different cases are compared in Figure 12. In this figure, the initial and predicted flow fields are smoothed. When the flow field was diluted to approximately 10%, the classic MLP neural network was still able to predict the missing data with quite a high accuracy. It was difficult to see the difference from the contour plot. With a further decrease in the dilution rate of 5% and 2%, the MLP neural network showed poor performance, where the predicted result was quite different from the initial values, especially for the velocity along the flow direction. Moreover, when the velocity gradient changed greatly, the prediction accuracy became worse, and the reconstructed field was prone to be inconsistent to some extent. However, both the PINN #1 and PINN #2 performed very well, even for the case where only 5% and 2% of the data were left for training. This can also be seen from the comparison of the MSE values based on the MLP and the PINNs, as shown in Figure 13. For the case of a dilution rate of 2%, the MSE for the classic neural network was more than ten times that of the MSEs for the PINNs. In addition, PINN #2, which only used the continuity equation as the loss function, showed a better performance than PINN #1, which was unexpected. It could have been attributed to the fact that the pressure could actually be determined through the pressure Poisson equation for a two-dimensional flow.

4.3. Reconstruction of the DNS Data with White Noise

The flow field measured using PIV in water channels always has noise due to the experimental environment or the test apparatus. To better simulate reality and evaluate the anti-interference ability of the neural network, a certain amount of noise was added to the flow velocity and pressure from the DNS. The noise selected in this case study followed a Gaussian distribution, with a mean value of zero and a standard deviation of 0.3, which can be written as:

$$\begin{array}{l}{Y}_j^{\prime }=Y_j\cdot \left(1+\epsilon \right),j=1,2,3\\ \epsilon -N\left(\mu ,{\sigma }^2\right), \mathrm{where} \mu =0;\sigma =0.3\end{array}$$

(6)

Here, Y₁, Y₂, and Y₃ represent the original data of u, v, and p, respectively. $Y_j^{\prime }$ refers to the data with added white noise ε. The contour plots of u, v, and p with noise are shown in Figure 14.

The performance of the MLP, PINN #1, and PINN #2 on the reconstruction of the flow field with white noise was compared. It can be seen very clearly from the contour plot of the velocity along the flow direction that the MLP yielded a bad prediction. In contrast, it is difficult to see the difference between the predicted data and the DNS data of the velocity and pressure from the contour plot. Therefore, the distribution of the error was also plotted. The MSE of the velocity and pressure based on the PINNs was much smaller than the MSE based on the MLP. In addition, the spatial distribution of the error based on the MLP was more stochastic in the whole flow field, while the error was normally very large in the areas where the velocity gradient of the wake varied greatly. This could be attributed to the continuity equation or the Navier–Stokes equations embedded in the loss function.

To see the accuracy of the reconstruction of the flow field more clearly, the reconstructed velocity and pressure along the x-axis at y = 0.6D and y = 0 are plotted in Figure 15. The noise could be filtered by both the MLP and PINN methods. A better curve fitting was achieved by using the PINNs than the MLP from the comparison of the correlation coefficients.

4.4. Reconstruction Based on PIV Data

The flow field around a two-dimensional cylinder measured with PIV is shown in the first row in Figure 16. The resolution was not as high as that of the DNS numerical simulation data, and noise could be seen very clearly in the original data. The reconstruction of the flow field by using the MLP and PINN #2 is also depicted in this figure. The localized structural similarity index (SSIM) [28] between the original and reconstructed flow fields measured via PIV as well as the MSE value were computed. PINN #2 yielded a larger SSIM and a smaller MSE value, which means it yielded a better reconstructed flow field. In addition, the reconstructed velocities u and v along the x-axis at y = 0 and y = 0.6D were plotted and compared with the measured PIV data, as shown in Figure 17. It was very clear that PINN #2 gave a better curve fit of the PIV data than the classic neural network. Hence, PINN #2 was applied to reconstruct the velocity field measured by PIV to a high-resolution field at different time steps, as shown in Figure 18.

5. Conclusions

In this paper, the performances of a classic neural network and two physically informed neural networks in reconstructing a high-resolution flow field were evaluated and compared. The PINNs incorporated the continuity equation or Navier–Stokes equations in the loss function. The flow around a circular cylinder was used in this case study. The methodology considered can also be used in other cases, such as the wake fields behind wind turbines or ship propellers.

A hyperparameter analysis was first carried out to determine an appropriate topology of the neural network, i.e., the number of hidden layers, the number of nodes in each layer, and the activation function.

The effects of the sparseness and noise of the training data on the reconstruction quality were mainly studied. Both the numerical simulation data based on DNS and the experimental data measured by PIV were analyzed.

The effect of sparseness was discussed by processing the DNS data with different dilution rates. It was found that all three neural networks worked very well when the dilution rate was larger than 10%. As the dilution rate decreased to 5% and 2%, the flow field reconstructed by the MLP neural network was very different from the true value, while PINN #1 and PINN #2 still worked very well. In the case with a dilution rate of 2%, the MSE for the classic neural network was more than ten times that of the PINNs.

The influence of noise was studied for the DNS data with added white noise and the PIV data. Both the classic neural network and the PINNs showed their ability to filter out noise, and overfitting did not occur in this case study. In the case of the DNS data, the MSE of the flow velocity u based on the classic neural network was more than five times the MSE using the PINN. For the reconstruction of the PIV data, the curve fit of the velocity in the whole field using the PINN was also much closer to the true value. Finally, the PINN was applied to reconstruct the flow field measured using PIV to a high-resolution field without noise.

It should also be noted that the cases in this paper were characterized by a low Reynolds number. The reconstruction of fine vortex structures was also not within the scope of this paper. However, some published research has proved the great potential of PINNs in analyzing cases with a series of high Reynolds numbers up to 10⁴ [29,30]. In these studies, the numerical data simulated by the Reynolds average Navier–Stokes (RANS) method were used for training a neural network. In future studies, efforts are still required to be devoted to dealing with the reconstruction of much more turbulent flow fields and much smaller-scale vortex structures.