Welcome to the new IOA website! Please reset your password to access your account.

Volume : 44

Part : 2

Proceedings of the Institute of Acoustics

 

 

Physics-aware learning of nonlinear limit cycles and adjoint limit cycles

 

Defne Ege Ozan1, Imperial College London, London, UK

Luca Magri2, Imperial College London & The Alan Turing Institute, London, UK

 

ABSTRACT

 

Thermoacoustic oscillations occur when the heat released by a flame is sufficiently in phase with the acoustic pressure. Under this condition, the linear instability can saturate to a nonlinear self-excited oscillation with a large amplitude. A typical nonlinear regime is a limit cycle, which is characterised by a periodic orbit in the thermoacoustic phase space. In this paper, we develop a physics-aware data-driven method to predict periodic solutions using forward neural networks. The physics is constrained in two ways. First, the training is informed by a physical residual, which penalises solutions that violate the conservation of mass, momentum, and energy. Second, periodicity is imposed by introducing periodic activation functions in the neural network. We test the algorithm on synthetic data generated from a nonlinear time-delayed model of a Rijke tube. We extend our study to learning the adjoint variables of the Rijke system. Adjoint methods offer a cheap and easy way to calculate the gradients with respect to design parameters, We find that (i) periodic solutions of thermoacoustic systems can be accurately learned with this method, (ii) for periodic data, periodic activations outperform conventional activations in terms of extrapolation capability beyond the training range, and (iii) exploiting the physical constraints, fewer data is sufficient to achieve a good performance. This work opens up possibilities for the prediction of nonlinear thermoacoustics by combining physical knowledge and data.

 

1. INTRODUCTION

 

During combustion processes, the unsteady heat release can couple with sound waves to cause self sustained oscillations to arise. These oscillations, known as thermoacoustic oscillations, are undesired because they can be damaging to the system’s structure [1–3]. Therefore, much research is aimed at suppressing these oscillations [4]. In this context, it is paramount to obtain accurate models of the nonlinear dynamics of thermoacoustic systems. These dynamics are described by partial differential equations, which are time-delayed, nonlinear, and hence time-consuming to solve. Data-driven methods have emerged as computationally cheaper alternatives to the traditional direct numerical simulation methods [5]. In machine learning, data-driven models are often considered to be black box models. However, when modelling applications in thermoacoustics are concerned, there is a vast knowledge of domain-specific physics that can still be leveraged. Physics-aware data-driven methods bridge the gap between the two approaches. By incorporating physical information, data driven models are designed to provide solutions that are compatible with conservation laws [6].

 

A typical nonlinear regime of thermoacoustic dynamics is a limit cycle [7], which is characterised by a periodic orbit in the phase space [8]. In this work, physics-aware forward neural networks are developed to learn periodic solutions of thermoacoustics. To augment the data-driven learning, we impose the thermoacoustic physics in two ways. First, during training, the networks are informed by a physics residual, which is computed by evaluating the system equations. Solutions that violate the conservation of mass, momentum, and energy are penalised. In other fluids communities, this approach is also known as physics-informed neural networks [9]. Second, we constrain the periodicity of the dynamics. Conventional forward neural networks are in general employed for interpolating data points, but fail to extrapolate periodic functions beyond the training range [10]. We circumvent this problem by implementing periodic functions as nonlinear activations in the network structure [10].

 

The paper is structured as follows: Section 2 provides the physical model of the Rijke tube. Its adjoint system is derived in Section 3. Section 4 presents the proposed physics-aware data-driven method to learn periodic solutions of thermoacoustics. Section 5 applies the method on synthetic data from the Rijke tube and its adjoint, in which extrapolation capability and robustness are discussed. Conclusions and future directions end the paper.

 

2. PHYSICAL MODEL

 

We consider a prototypical thermoacoustic system known as the Rijke tube, which consists of a duct with a heat source located inside. The Rijke tube is a typical setup used to study fundamental principles of thermoacoustic instabilities because it is a simple system which captures the qualitative nonlinear dynamics and bifurcations observed in real-life applications [11–13]. The governing equations are derived from mass, momentum and energy conservation. The dynamics are governed by

 

 

where u and p are the non-dimensionalised velocity and pressure deviations from the mean flow, ζ is the modal damping,  is the heat release rate, and δ(x−xf) is the Dirac delta at the heat source location xf [12]. The spatial coordinate x is non-dimensionalised by the tube length such that x ∈ [0, 1]. For the purpose of this study, we consider a heat law given by a sigmoid function as

 

 

where β is the heat strength, and 𝜏 denotes the time delay between the heat release and the velocity. A similar heat-release law was employed by [14] for the reduced-order modelling of thermoacoustic instabilities.

 

The boundary conditions are fully reflective, p(x = 0) = p(x = 1) = 0. The solution is expressed by a Galerkin decomposition [15] on Ng acoustic eigenfunctions [11–13]:

 

 

where jπ are the non-dimensional angular frequencies of the purely acoustic modes. By substituting pressure and velocity variables in Equation 1 with their Galerkin decompositions in Equation 3 and projecting the dynamics onto the Galerkin modes, the dynamics of the Galerkin variables ηj and µj are described by a 2Ng dimensional system of ordinary differential equations

 

 

For brevity, the damping is assumed to be constant for all modes, ζj = c. The time-delayed problem is transformed to an initial value problem via an advection function with the dummy variable v [16]

 

 

The time-delayed velocity u(xf, t − 𝜏) is, thus, provided by v(X = 1, t). Equation 5 is discretised with Nc + 1 points, which utilises a Chebyshev spectral method [17]. The final system of ordinary differential equations includes, in addition, the Nc degrees of freedom from v.

 

The dynamical behaviour of the system is dictated by the flame location, xf, the heat release strength, β, and the time delay, 𝜏. We use the following system’s parameters, which result in a system in the limit cycle regime; Ng = 10, Nc = 10, xf = 0.2, β = 5, 𝜏 = 0.2, c = 0.1.

 

3. ADJOINT SYSTEM

 

Adjoint methods are an accurate and cheap method to calculate the gradient of a quantity of interest with respect to system’s parameters. For example, for the thermoacoustic system, this quantity can be the time-averaged acoustic energy, which relates to the energy of the undesired oscillations, given by

 

 

The gradient of this quantity can be essential for the optimisation of the system’s parameters, e.g. heat release strength, to minimise the acoustic energy.

 

We now formally introduce the adjoint of the Rijke tube system. Let q ∈ ℝNq, Nq = 2Ng + Nc, denote the state vector that contains the variables ηj, µj, and v, and let p ∈ ℝNp represent the system’s parameters that we are interested in optimising. The dynamics are then written in a compact form as

 

 

The initial conditions are independent of the system’s parameters

 

 

The variation of a cost functional (q, p) with respect to the system’s parameters is expressed analytically as

 

 

Notice that the direct problem requires , which grows in size with the number of parameters and whose computation is, thus, preferably avoided. Instead, the adjoint problem is derived from the Lagrangian functional of Equation 9, where Equation 7 and Equation 8 are the constraints [18]. For the special case where  is a time-averaged quantity of over [0, T], the adjoint system is finally described by [18]

 

 

where q+ ∈ ℝNq are called the adjoint variables ((.)T denotes the transpose operation). The operator  is the linearisation of the Rijke tube system around a base solution. In this case, the only non-linearity is in the heat release term. The derivative terms are not explicitly shown here, however the calculation is straightforward.

 

This concludes the models used to generate data for the Rijke tube system and its adjoint.

 

4. LEARNING PERIODIC SOLUTIONS

 

4.1. Feedforward Neural Networks

 

A feedforward neural network (FNN) defines a nonlinear mapping between an input and an output vector [19]. Let y ∈ ℝn0 be an input vector, a multi-layer feedforward neural network, f : ℝn0→ ℝnL, is defined as

 

 

where W(l) ∈ ℝnl × nl-1, b(l) ∈ ℝnl, and ϕ(l): ℝnl → ℝnl  are called weights, biases, and activation functions at each layer l, respectively. We will refer to the trainable weights and biases as θ. The FNN that models solutions from the Rijke tube takes the spatial and time coordinates, x and t, as inputs to provide the pressure and velocity, p(x, t) and u(x, t), i.e. f : (x, t) →  . This architecture is shown in Figure 1. The network’s weights and biases are optimized with respect to a loss function

 

 

The loss functions are defined in Section 4.2.

 

4.2. Loss functions

 

The data-driven loss is quantified by the mean-squared error (MSE)

 

 

where  is the provided training dataset and  are the predictions obtained from the neural network. Physics-information can be incorporated in the loss function with additional losses M  and E  as

 

 

 

Figure 1: Feedforward neural network architecture to model solutions from thermoacoustic systems.

 

where M  and E  are the residuals from momentum balance in Equation 1a and energy balance in Equation 1b, respectively. The regularization hyperparameters λD, λM, and λE help the scaling of the different losses. Obtaining M  and E  requires the evaluation of the partial differential equation in Equation 1 with the network. We employ automatic differentiation of the output variables of the network with respect to the input variables in order to obtain the gradients found in the governing equations. The gradients are necessary for computing the physical residuals which are in the form of partial differential equations.

 

4.3. Activation functions

 

Commonly used activation functions are the rectified linear unit (ReLU), ϕ(z) = max{0,z}, and hyperbolic tangent, ϕ(z) = tanh(z). Whilst weights and biases rotate and shift the data, activation functions apply nonlinear transformations. The multi-layered composition of these operations enables neural networks to approximate any nonlinear function in a bounded region [20]. However, the extrapolation capability of neural networks is limited for periodic data. Networks with ReLU or tanh activations are inherently incapable of extrapolating periodic functions. This is because the extrapolation behaviour is determined by the analytical form of the activation function [10], i.e. ReLU acts linearly, and tanh saturates to a constant. A physically motivated alternative is to use periodic activation functions such as ϕ(z) = sin(z) [21] or ϕ(z) = z + sin2(z) [10]. Thus, the physics of the system, specifically the periodic nature of the solutions, is hard-constrained in the network’s architecture itself via the choice of the activation function.

 

5. RESULTS

 

The Rijke tube system is simulated to generate synthetic data for 508 time-units. The training data is chosen as the first 4 time-units long time series after the 500 time-units long transient is removed; the validation data is the next 4 time-units long time series after the training. (Validation data refers to a dataset that is kept separate during training and is, thus, used to tune the hyperparameters.) The spatio temporal spacing is 0.05 for both the spatial and time coordinates. For the ReLU and tanh networks, the input data (x, t) is standardised. The training is performed with 2-layer neural networks with 200 neurons at each layer for 500 epochs using ReLU, tanh, and sin activations. The sin activation is set as  following a hyperparameter tuning. Adam optimiser with a decaying learning rate schedule that starts from 0.001 executes the training.

 

Figure 2 shows the evolution of training and validation losses during training when using different activations in the network. Although all networks can fit the training data, only the network with the sin activation performs well on the validation set. This means the sin activation is also the physically suitable activation, which improves generalisability. The resulting predictions are shown in Figure 3 (a) for the location x = 0.25 where the extrapolations of the different activation functions are demonstrated. We investigate robustness to noise by adding a zero mean Gaussian noise with a standard deviation of 0.5 to the training data. For this case, a physics-informed network with sin activation is also trained. The weights λD, λM, and λE are set as 10, 1, and 1, respectively. The predictions in this case are shown in Figure 3 (b) for the same velocity time series. In the presence of noise, overfitting can occur if the model is overparametrised, e.g. the number of neurons is too high. This is observed for the sin network. In this case, the physics-information improves prediction accuracy.

 

 

Figure 2: Training (a) and validation (b) losses of the data-driven forward neural network during training when using ReLU, tanh, and sin activation functions. Validation data is taken from a time window after the training range.

 

 

Figure 3: Predictions with the data-driven and physics-informed (PI) forward neural networks when the training dataset is on a fine spatio-temporal grid. In (a) the training data is noise-free, in (b) zero mean Gaussian noise with standard deviation 0.5 is added to it. The time evolution of the velocity at x = 0.25 is shown. Training and validation datasets are separated with the dashed vertical line.

 

In real life applications, the available data may be scarce. Next, in order to emulate such a scenario, we investigate a case where the spatio-temporal spacing is increased to 0.1. The same network architecture as before is used and the training is performed for 2000 epochs. The velocity predictions along the tube at time step t = 1.2 are shown in Figure 4, when the training dataset is (a) on the fine spatio-temporal grid, and (b) on the coarse spatio-temporal grid. When the available data is scarce, the physics-information promotes correct interpolation between the given training data points.

 

 
Figure 4: Predictions of data-driven and physics-informed (PI) forward neural networks when the training dataset is (a) on a fine spatio-temporal grid and (b) on a coarse spatio-temporal grid.

 

Last, we consider the problem of predicting the adjoint variables. The training is performed with a single layer network with 200 neurons that maps the time to the adjoint variables associated with the Galerkin variables ηj  and µj from Equation 3, i.e.  Figure 5 shows the prediction of the adjoint variables associated with the modes η1 and µ4 in (a) and (b) respectively as examples.

 

 

Figure 5: Prediction of the adjoint limit cycles of the variables associated with η1 and µ4 modes.

 

6. CONCLUSION

 

A physics-aware data-driven method has been developed to predict the limit cycle solutions of thermoacoustic systems. This model consists of a forward neural network that maps time and space to pressure and velocity. The network is constrained via periodic activation functions which introduce an inductive bias towards periodic solutions. The physics of the thermoacoustic system are further imposed with a physical residual in the loss term during training. It is shown that the proposed neural network can extrapolate periodic solutions in contrast to conventional activations. When data is scarce, the physics-information acts as a regularizer and directs the training towards physically accurate models. The adjoint variables of the thermoacoustic system are also predicted within this framework. This work opens up possibilities of combining popular data-driven methods, such as deep learning, with physical knowledge in order to obtain nonlinear models of thermoacoustic systems that can be used for prediction and optimization purposes. Future research directions include investigation of long-term predictability and application of the algorithm to more realistic scenarios with flames and experimental data.

 

ACKNOWLEDGEMENTS

 

D. E. Ozan and L. Magri acknowledge financial support from the ERC Starting Grant No. PhyCo 949388. L. Magri acknowledges financial support TUM Institute for Advanced Study (German Excellence Initiative and the EU 7th Framework Programme No. 291763). The authors report no conflict of interest.

 

REFERENCES

 

  1. Timothy C. Lieuwen and Vigor Yang. Combustion instabilities in gas turbine engines operational experience, fundamental mechanisms and modeling. Progress in astronautics and aeronautics ; v. 210. American Institute of Aeronautics and Astronautics, Reston, VA, 2005.

  2. F.E.C. Culick and Paul Kuentzmann. Unsteady motions in combustion chambers for propulsion systems. NATO RTO-AG-AVT-039, AGARDograph, 2006.

  3. Ann P. Dowling and Yasser Mahmoudi. Combustion noise. Proceedings of the Combustion Institute, 35(1):65–100, 2015.

  4. Anuradha M. Annaswamy and Ahmed F. Ghoniem. Active control in combustion systems. IEEE Control Systems Magazine, 15(6):49–63, 1995.

  5. Steven L. Brunton, Bernd R. Noack, and Petros Koumoutsakos. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 52(1):477–508, 2020.

  6. Anuj Karpatne, Gowtham Atluri, James H. Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar. Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering, 29(10):2318–2331, 2017.

  7. Lipika Kabiraj, R. I. Sujith, and Pankaj Wahi. Bifurcations of Self-Excited Ducted Laminar Premixed Flames. Journal of Engineering for Gas Turbines and Power, 134(3), 2011.

  8. Steven Henry Strogatz. Nonlinear dynamics and chaos with applications to physics, biology, chemistry and engineering. Addison-Wesley, Reading, Massachusetts, 1994.

  9. Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.

  10. Liu Ziyin, Tilman Hartwig, and Masahito Ueda. Neural networks fail to learn periodic functions and how to fix it. In Advances in Neural Information Processing Systems, volume 33, pages 1583–1594, 2020.

  11. Koushik Balasubramanian and R. I. Sujith. Thermoacoustic instability in a rijke tube: Non normality and nonlinearity. Physics of Fluids, 20(4):044103, 2008.

  12. Matthew P. Juniper. Triggering in the horizontal rijke tube: non-normality, transient growth and bypass transition. Journal of Fluid Mechanics, 667:272–308, 2011.

  13. Priya Subramanian, R. I. Sujith, and P. Wahi. Subcritical bifurcation and bistability in thermoacoustic systems. Journal of Fluid Mechanics, 715:210–238, 2013.

  14. Nicolas Noiray. Linear Growth Rate Estimation From Dynamics and Statistics of Acoustic Signal Envelope in Turbulent Combustors. Journal of Engineering for Gas Turbines and Power, 139(4), 2016.

  15. Philip Holmes, John L. Lumley, Gahl Berkooz, and Clarence W. Rowley. Galerkin projection, page 106–129. Cambridge Monographs on Mechanics. Cambridge University Press, 2012.

  16. Francisco Huhn and Luca Magri. Gradient-free optimization of chaotic acoustics with reservoir computing. Phys. Rev. Fluids, 7:014402, 2022.

  17. Lloyd N. Trefethen. Spectral methods in MATLAB. Software, environments, tools. Society for Industrial and Applied Mathematics„ Philadelphia, PA, 2000.

  18. Luca Magri. Adjoint methods as design tools in thermoacoustics. Applied Mechanics Reviews, 71(2), 2019.

  19. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.

  20. Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989.

  21. Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems, volume 33, pages 7462–7473, 2020.

 

1 d.ozan@imperial.ac.uk

2 l.magri@imperial.ac.uk