Transient Growth in Non-Normal Linear Systems
Analysis of a Non-normal System
Quantifying Instantaneous Growth
The stability analysis of linear dynamical systems, represented by \dot{\mathbf{x}} = \mathbf{A} \mathbf{x}, is fundamentally tied to the eigenvalues of the matrix \mathbf{A}. The established criterion states that if all eigenvalues possess strictly negative real parts, the system is asymptotically stable, and all trajectories converge to the origin as t \to \infty.
While this conclusion about the long-term behavior is correct, it can obscure a class of phenomena of significant practical importance: the possibility of substantial transient energy growth. Systems that are asymptotically stable may first exhibit a period of considerable amplification before the eventual decay takes hold.
This behavior is characteristic of non-normal systems and is of particular relevance in fields such as fluid dynamics, where transient growth can trigger nonlinear instabilities in an otherwise linearly stable flow.
Normality of linear operators
A matrix \mathbf{A} is defined as normal if it commutes with its conjugate transpose, \mathbf{A}^H. For real matrices, this condition simplifies to commuting with its transpose:
\mathbf{A}\mathbf{A}^T = \mathbf{A}^T\mathbf{A}
The spectral theorem guarantees that a matrix is normal if and only if it possesses a complete set of orthonormal eigenvectors.
Symmetric (\mathbf{A} = \mathbf{A}^T), skew-symmetric (\mathbf{A} = -\mathbf{A}^T), and orthogonal (\mathbf{A}^{-1} = \mathbf{A}^T) matrices are all subclasses of normal matrices.
For normal systems, the intuition derived from eigenvalue analysis is complete. The orthogonal eigenvectors form a basis in which the system dynamics are entirely decoupled and purely described by exponential growth or decay along these orthogonal directions. Any initial state’s energy \|\mathbf{x}(0)\|^2 will decay monotonically if all eigenvalues have negative real parts.
A system is termed non-normal if its governing matrix does not commute with its transpose.
\mathbf{A}\mathbf{A}^T \ne \mathbf{A}^T\mathbf{A}
The eigenvectors of a non-normal matrix are not orthogonal. This departure from orthogonality is the underlying mechanism that permits transient growth.
Analysis of a non-normal system
Consider the following two-dimensional linear system, which, despite its apparent simplicity, encapsulates the essential features of non-normal dynamics:
\frac{\mathrm{d}}{\mathrm{d}t} \begin{bmatrix} v \\ x \end{bmatrix} = \begin{bmatrix} -0.009 & 1 \\ 0 & -0.01 \end{bmatrix} \begin{bmatrix} v \\ x \end{bmatrix}
Since the matrix \mathbf{A} is upper triangular, its eigenvalues the diagonal entries:
\begin{aligned} & \lambda_1 = -0.009 \\ & \lambda_2 = -0.01 \end{aligned}
Both eigenvalues are negative, indicating that the system is asymptotically stable and any initial condition will ultimately decay to the origin.
However, a numerical solution for the initial condition \mathbf{x}(0) =^T reveals a different short-term story.
The state vector’s norm initially grows by a significant factor before the predicted asymptotic decay commences. To understand this, we must examine the eigenvectors.
The eigenvector \boldsymbol{\xi}_1 corresponding to \lambda_1 = -0.009 is found by solving (\mathbf{A} - \lambda_1\mathbf{I})\boldsymbol{\xi}_1 = \mathbf{0}:
\begin{aligned} & \begin{bmatrix} 0 & 1 \\ 0 & -0.001 \end{bmatrix} \begin{bmatrix} v_1 \\ x_1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \\ & \boldsymbol{\xi}_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \end{aligned}
The eigenvector \boldsymbol{\xi}_2 corresponding to \lambda_2 = -0.01 is:
\begin{aligned} & \begin{bmatrix} 0.001 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} v_2 \\ x_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \\ & \quad 0.001v_2 + x_2 = 0 \end{aligned}
A possible choice for this eigenvector is \boldsymbol{\xi}_2 = [1, -0.001]^T. A more numerically stable choice would be [-1000, 1]^T.
The two eigenvectors are nearly parallel, which the result in strong non-normality. A formal check confirms the non-normal nature of \mathbf{A}:
\begin{aligned} \mathbf{A}\mathbf{A}^T &= \begin{bmatrix} -0.009 & 1 \\ 0 & -0.01 \end{bmatrix} \begin{bmatrix} -0.009 & 0 \\ 1 & -0.01 \end{bmatrix} = \begin{bmatrix} 1.000081 & -0.01 \\ -0.01 & 0.0001 \end{bmatrix} \\ \mathbf{A}^T\mathbf{A} &= \begin{bmatrix} -0.009 & 0 \\ 1 & -0.01 \end{bmatrix} \begin{bmatrix} -0.009 & 1 \\ 0 & -0.01 \end{bmatrix} = \begin{bmatrix} 0.000081 & -0.009 \\ -0.009 & 1.0001 \end{bmatrix} \end{aligned}
Clearly, \mathbf{A}\mathbf{A}^T \ne \mathbf{A}^T\mathbf{A}.
Mechanism of transient growth
The solution to the system is given by \mathbf{x}(t) = e^{\mathbf{A}t}\mathbf{x}(0).
For a diagonalizable matrix, this is \mathbf{x}(t) = \mathbf{T}e^{\mathbf{D}t}\mathbf{T}^{-1}\mathbf{x}(0), where \mathbf{T} is the matrix of eigenvectors. The norm of the solution is bounded by:
\|\mathbf{x}(t)\| \le \|\mathbf{T}\| \|e^{\mathbf{D}t}\| \|\mathbf{T}^{-1}\| \|\mathbf{x}(0)\|
The term \|e^{\mathbf{D}t}\| is guaranteed to decay as t \to \infty since its entries are e^{\lambda_i t} with \Re(\lambda_i)<0. However, when eigenvectors are nearly parallel, the eigenvector matrix \mathbf{T} becomes ill-conditioned.
This means its condition number, \kappa(\mathbf{T}) = \|\mathbf{T}\|\|\mathbf{T}^{-1}\|, is large. For a short time, the large, constant pre-factor \kappa(\mathbf{T}) can overwhelm the slow decay of the exponential term, resulting in \|\mathbf{x}(t)\| > \|\mathbf{x}(0)\|.
A more physical interpretation can be given. Any initial condition \mathbf{x}(0) can be expressed as a linear combination of the eigenvectors:
\mathbf{x}(0) = c_1\boldsymbol{\xi}_1 + c_2\boldsymbol{\xi}_2
Because the eigenvectors are nearly parallel, achieving a specific initial condition may require the coefficients c_1 and c_2 to be large and of opposite sign, leading to a near-cancellation.
The time evolution is then:
\mathbf{x}(t) = c_1 e^{\lambda_1 t} \boldsymbol{\xi}_1 + c_2 e^{\lambda_2 t} \boldsymbol{\xi}_2
Although both exponential terms decay, they do so at slightly different rates. This breaks the initial cancellation, and the large underlying magnitudes of the components c_1\boldsymbol{\xi}_1 and c_2\boldsymbol{\xi}_2 are temporarily revealed, causing the norm of their sum to grow before both eventually decay to zero.
Quantifying instantaneous growth
A more direct way to probe the potential for growth is to examine the instantaneous rate of change of the system’s “energy,” defined as E(t) = \|\mathbf{x}(t)\|^2 = \mathbf{x}^H \mathbf{x}.
\begin{aligned} \frac{\mathrm{d}E}{\mathrm{d}t} & = \frac{\mathrm{d}}{\mathrm{d}t}(\mathbf{x}^H\mathbf{x}) = \dot{\mathbf{x}}^H\mathbf{x} + \mathbf{x}^H\dot{\mathbf{x}} = (\mathbf{A}\mathbf{x})^H\mathbf{x} + \mathbf{x}^H(\mathbf{A}\mathbf{x}) \\ & = \mathbf{x}^H\mathbf{A}^H\mathbf{x} + \mathbf{x}^H\mathbf{A}\mathbf{x} = \mathbf{x}^H(\mathbf{A}^H + \mathbf{A})\mathbf{x} \end{aligned}
Instantaneous energy growth is possible if \frac{\mathrm{d}E}{\mathrm{d}t} > 0 for some state \mathbf{x}.
This requires the Hermitian part of the matrix, \mathbf{A}_H = \frac{1}{2}(\mathbf{A}^H + \mathbf{A}), to not be negative definite.
The eigenvalues of \mathbf{A}_H determine the maximum and minimum possible rates of energy change.
This relates to the concept of the numerical range (or field of values) of \mathbf{A}, denoted W(\mathbf{A}), which is the set of all Rayleigh quotients for the matrix:
W(\mathbf{A}) = \{ \mathbf{z}^H\mathbf{A}\mathbf{z} \in \mathbb{C} \mid \|\mathbf{z}\|=1 \}
For any normal matrix, the numerical range is the convex hull of its eigenvalues. If all eigenvalues are in the left-half complex plane, so is the numerical range, and no growth is possible.
For a non-normal matrix, the numerical range can be substantially larger than the convex hull of its eigenvalues and may extend into the right-half plane, even when all eigenvalues lie in the left-half plane.
The maximum real part of any element in W(\mathbf{A}) gives the maximum possible instantaneous growth rate.
This provides a quantitative tool for assessing transient behavior that is independent of the long-term asymptotic stability predicted by the eigenvalues.