In multivariate statistics, if ε is a vector of n random variables, and Λ is an n -dimensional symmetric matrix, then the scalar quantity ε T Λ ε is known as a quadratic form in ε .
It can be shown that
E [ ε T Λ ε ] = tr [ Λ Σ ] + μ T Λ μ where μ and Σ are the expected value and variance-covariance matrix of ε , respectively, and tr denotes the trace of a matrix. This result only depends on the existence of μ and Σ ; in particular, normality of ε is not required.
A book treatment of the topic of quadratic forms in random variables is
Since the quadratic form is a scalar quantity E [ ε T Λ ε ] = tr ( E [ ε T Λ ε ] ) . Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that
tr ( E [ ε T Λ ε ] ) = E [ tr ( ε T Λ ε ) ] . Next, by the cyclic property of the trace operator,
E [ tr ( ε T Λ ε ) ] = E [ tr ( Λ ε ε T ) ] . Another application of linearity of expectation tells us that
E [ tr ( Λ ε ε T ) ] = tr ( Λ E ( ε ε T ) ) . A standard property of variances then tells us that this is
tr ( Λ ( Σ + μ μ T ) ) . Applying the cyclic property of the trace operator again, we get
tr ( Λ Σ ) + tr ( Λ μ μ T ) = tr ( Λ Σ ) + tr ( μ T Λ μ ) = tr ( Λ Σ ) + μ T Λ μ . In general, the variance of a quadratic form depends greatly on the distribution of ε . However, if ε does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that Λ is a symmetric matrix. Then,
var [ ε T Λ ε ] = 2 tr [ Λ Σ Λ Σ ] + 4 μ T Λ Σ Λ μ In fact, this can be generalized to find the covariance between two quadratic forms on the same ε (once again, Λ 1 and Λ 2 must both be symmetric):
cov [ ε T Λ 1 ε , ε T Λ 2 ε ] = 2 tr [ Λ 1 Σ Λ 2 Σ ] + 4 μ T Λ 1 Σ Λ 2 μ Some texts incorrectly state that the above variance or covariance results hold without requiring Λ to be symmetric. The case for general Λ can be derived by noting that
ε T Λ T ε = ε T Λ ε so
ε T Λ ~ ε = ε T ( Λ + Λ T ) ε / 2 But this is a quadratic form in the symmetric matrix Λ ~ = ( Λ + Λ T ) / 2 , so the mean and variance expressions are the same, provided Λ is replaced by Λ ~ therein.
In the setting where one has a set of observations y and an operator matrix H , then the residual sum of squares can be written as a quadratic form in y :
RSS = y T ( I − H ) T ( I − H ) y . For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix σ 2 I , RSS / σ 2 has a chi-squared distribution with k degrees of freedom and noncentrality parameter λ , where
k = tr [ ( I − H ) T ( I − H ) ] λ = μ T ( I − H ) T ( I − H ) μ / 2 may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections. If H y estimates μ with no bias, then the noncentrality λ is zero and RSS / σ 2 follows a central chi-squared distribution.
