Inverse Wishart distribution - Alchetron, the free social encyclopedia


Notation W − 1 ( Ψ , ν ) {displaystyle {mathcal {W}}^{-1}({mathbf {Psi } },u )} Parameters ν > p − 1 {displaystyle u >p-1} degrees of freedom (real) Ψ > 0 {displaystyle mathbf {Psi } >0} scale matrix (pos. def) Support X {displaystyle mathbf {X} } is positive definite PDF \| Ψ \| ν 2 2 ν p 2 Γ p ( ν 2 ) \| X \| − ν + p + 1 2 e − 1 2 tr ⁡ ( Ψ X − 1 ) {displaystyle {rac {left\|{mathbf {Psi } }ight\|^{rac {u }{2}}}{2^{rac {u p}{2}}Gamma _{p}({rac {u }{2}})}}left\|mathbf {X} ight\|^{-{rac {u +p+1}{2}}}e^{-{rac {1}{2}}operatorname {tr} ({mathbf {Psi } }mathbf {X} ^{-1})}} Γ p {displaystyle Gamma _{p}} is the multivariate gamma function t r {displaystyle mathrm {tr} } is the trace function Mean Ψ ν − p − 1 {displaystyle {rac {mathbf {Psi } }{u -p-1}}} For ν > p + 1 {displaystyle u >p+1} Mode Ψ ν + p + 1 {displaystyle {rac {mathbf {Psi } }{u +p+1}}}

In statistics, the inverse Wishart distribution, also called the inverted Wishart distribution, is a probability distribution defined on real-valued positive-definite matrices. In Bayesian statistics it is used as the conjugate prior for the covariance matrix of a multivariate normal distribution.

Density

The probability density function of the inverse Wishart is:

| Ψ | ν 2 2 ν p 2 Γ p ( ν 2 ) | X | − ν + p + 1 2 e − 1 2 tr ⁡ ( Ψ X − 1 )

where X and Ψ are p × p positive definite matrices, and Γ_p(·) is the multivariate gamma function.

Distribution of the inverse of a Wishart-distributed matrix

If A ∼ W ( Σ , ν ) and Σ is of size p × p , then X = A − 1 has an inverse Wishart distribution X ∼ W − 1 ( Σ − 1 , ν ) .

Marginal and conditional distributions from an inverse Wishart-distributed matrix

Suppose A ∼ W − 1 ( Ψ , ν ) has an inverse Wishart distribution. Partition the matrices A and Ψ conformably with each other

A = [ A 11 A 12 A 21 A 22 ] , Ψ = [ Ψ 11 Ψ 12 Ψ 21 Ψ 22 ]

where A i j and Ψ i j are p i × p j matrices, then we have

i) A 11 is independent of A 11 − 1 A 12 and A 22 ⋅ 1 , where A 22 ⋅ 1 = A 22 − A 21 A 11 − 1 A 12 is the Schur complement of A 11 in A ;

ii) A 11 ∼ W − 1 ( Ψ 11 , ν − p 2 ) ;

iii) A 11 − 1 A 12 | A 22 ⋅ 1 ∼ M N p 1 × p 2 ( Ψ 11 − 1 Ψ 12 , A 22 ⋅ 1 ⊗ Ψ 11 − 1 ) , where M N p × q ( ⋅ , ⋅ ) is a matrix normal distribution;

iv) A 22 ⋅ 1 ∼ W − 1 ( Ψ 22 ⋅ 1 , ν ) , where Ψ 22 ⋅ 1 = Ψ 22 − Ψ 21 Ψ 11 − 1 Ψ 12 ;

Conjugate distribution

Suppose we wish to make inference about a covariance matrix Σ whose prior p ( Σ ) has a W − 1 ( Ψ , ν ) distribution. If the observations X = [ x 1 , … , x n ] are independent p-variate Gaussian variables drawn from a N ( 0 , Σ ) distribution, then the conditional distribution p ( Σ | X ) has a W − 1 ( A + Ψ , n + ν ) distribution, where A = X X T .

Because the prior and posterior distributions are the same family, we say the inverse Wishart distribution is conjugate to the multivariate Gaussian.

Due to its conjugacy to the multivariate Gaussian, it is possible to marginalize out (integrate out) the Gaussian's parameter Σ .

P ( X | Ψ , ν ) = ∫ P ( X | Σ ) P ( Σ | Ψ , ν ) d Σ = | Ψ | ν 2 Γ p ( ν + n 2 ) π n p 2 | Ψ + A | ν + n 2 Γ p ( ν 2 )

(this is useful because the variance matrix Σ is not known in practice, but because Ψ is known a priori, and A can be obtained from the data, the right hand side can be evaluated directly). The inverse-Wishart distribution as a prior can be constructed via existing transferred prior knowledge.

Moments

The following is based on Press, S. J. (1982) "Applied Multivariate Analysis", 2nd ed. (Dover Publications, New York), after reparameterizing the degree of freedom to be consistent with the p.d.f. definition above.

The mean:

E ( X ) = Ψ ν − p − 1 .

The variance of each element of X :

Var ⁡ ( x i j ) = ( ν − p + 1 ) ψ i j 2 + ( ν − p − 1 ) ψ i i ψ j j ( ν − p ) ( ν − p − 1 ) 2 ( ν − p − 3 )

The variance of the diagonal uses the same formula as above with i = j , which simplifies to:

Var ⁡ ( x i i ) = 2 ψ i i 2 ( ν − p − 1 ) 2 ( ν − p − 3 ) .

The covariance of elements of X are given by:

Cov ⁡ ( x i j , x k l ) = 2 ψ i j ψ k l + ( ν − p − 1 ) ( ψ i k ψ j l + ψ i l ψ k j ) ( ν − p ) ( ν − p − 1 ) 2 ( ν − p − 3 )

A univariate specialization of the inverse-Wishart distribution is the inverse-gamma distribution. With p = 1 (i.e. univariate) and α = ν / 2 , β = Ψ / 2 and x = X the probability density function of the inverse-Wishart distribution becomes

p ( x | α , β ) = β α x − α − 1 exp ⁡ ( − β / x ) Γ 1 ( α ) .

i.e., the inverse-gamma distribution, where Γ 1 ( ⋅ ) is the ordinary Gamma function.

A generalization is the inverse multivariate gamma distribution.

Another generalization has been termed the generalized inverse Wishart distribution, G W − 1 . A p × p positive definite matrix X is said to be distributed as G W − 1 ( Ψ , ν , S ) if Y = X 1 / 2 S − 1 X 1 / 2 is distributed as W − 1 ( Ψ , ν ) . Here X 1 / 2 denotes the symmetric matrix square root of X , the parameters Ψ , S are p × p positive definite matrices, and the parameter ν is a positive scalar larger than 2 p . Note that when S is equal to an identity matrix, G W − 1 ( Ψ , ν , S ) = W − 1 ( Ψ , ν ) . This generalized inverse Wishart distribution has been applied to estimating the distributions of multivariate autoregressive processes.

A different type of generalization is the normal-inverse-Wishart distribution, essentially the product of a multivariate normal distribution with an inverse Wishart distribution.

References

Inverse-Wishart distribution Wikipedia

(Text) CC BY-SA

Contents