Normal inverse Wishart distribution - Alchetron, the free social encyclopedia


Notation ( μ , Σ ) ∼ N I W ( μ 0 , λ , Ψ , ν ) {\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})\sim \mathrm {NIW} ({\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )} Parameters μ 0 ∈ R D {\displaystyle {\boldsymbol {\mu }}_{0}\in \mathbb {R} ^{D}\,} location (vector of real) λ > 0 {\displaystyle \lambda >0\,} (real) Ψ ∈ R D × D {\displaystyle {\boldsymbol {\Psi }}\in \mathbb {R} ^{D\times D}} inverse scale matrix (pos. def.) ν > D − 1 {\displaystyle \nu >D-1\,} (real) Support μ ∈ R D ; Σ ∈ R D × D {\displaystyle {\boldsymbol {\mu }}\in \mathbb {R} ^{D};{\boldsymbol {\Sigma }}\in \mathbb {R} ^{D\times D}} covariance matrix (pos. def.) PDF f ( μ , Σ \| μ 0 , λ , Ψ , ν ) = N ( μ \| μ 0 , 1 λ Σ ) W − 1 ( Σ \| Ψ , ν ) {\displaystyle f({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )={\mathcal {N}}({\boldsymbol {\mu }}\|{\boldsymbol {\mu }}_{0},{\tfrac {1}{\lambda }}{\boldsymbol {\Sigma }})\ {\mathcal {W}}^{-1}({\boldsymbol {\Sigma }}\|{\boldsymbol {\Psi }},\nu )}

In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix (the inverse of the precision matrix).

Definition

Suppose

μ | μ 0 , λ , Σ ∼ N ( μ | μ 0 , 1 λ Σ )

has a multivariate normal distribution with mean μ 0 and covariance matrix 1 λ Σ , where

Σ | Ψ , ν ∼ W − 1 ( Σ | Ψ , ν )

has an inverse Wishart distribution. Then ( μ , Σ ) has a normal-inverse-Wishart distribution, denoted as

( μ , Σ ) ∼ N I W ( μ 0 , λ , Ψ , ν ) .

Probability density function

f ( μ , Σ | μ 0 , λ , Ψ , ν ) = N ( μ | μ 0 , 1 λ Σ ) W − 1 ( Σ | Ψ , ν )

Marginal distributions

By construction, the marginal distribution over Σ is an inverse Wishart distribution, and the conditional distribution over μ given Σ is a multivariate normal distribution. The marginal distribution over μ is a multivariate t-distribution.

Posterior distribution of the parameters

Suppose the sampling density is a multivariate normal distribution

y i | μ , Σ ∼ N p ( μ , Σ )

where y is an n × p matrix and y i (of length p ) is row i of the matrix .

With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly

( μ , Σ ) ∼ N I W ( μ 0 , λ , Ψ , ν ) .

The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart

( μ , Σ | y ) ∼ N I W ( μ n , λ n , Ψ n , ν n ) ,

where

μ n = λ μ 0 + n y ¯ λ + n λ n = λ + n ν n = ν + n Ψ n = Ψ + S + λ n λ + n ( y ¯ − μ 0 ) T ( y ¯ − μ 0 ) w i t h , S = ∑ i = 1 n ( y i − y ¯ ) T ( y i − y ¯ ) .

To sample from the joint posterior of ( μ , Σ ) , one simply draws samples from Σ | y ∼ W − 1 ( Ψ n , ν n ) , then draw μ | Σ , y ∼ N p ( μ n , Σ / ν n ) . To draw from the posterior predictive of a new observation, draw y ~ | μ , Σ , y ∼ N p ( μ , Σ ) , given the already drawn values of μ and Σ .

Generating normal-inverse-Wishart random variates

Generation of random variates is straightforward:

Sample Σ from an inverse Wishart distribution with parameters Ψ and ν
Sample μ from a multivariate normal distribution with mean μ 0 and variance 1 λ Σ

The normal-Wishart distribution is essentially the same distribution parameterized by precision rather than variance. If ( μ , Σ ) ∼ N I W ( μ 0 , λ , Ψ , ν ) then ( μ , Σ − 1 ) ∼ N W ( μ 0 , λ , Ψ − 1 , ν ) .

The normal-inverse-gamma distribution is the one-dimensional equivalent.

The multivariate normal distribution and inverse Wishart distribution are the component distributions out of which this distribution is made.

References

Normal-inverse-Wishart distribution Wikipedia

(Text) CC BY-SA

Contents