In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF). Every distribution possessing a moment-generating function is a member of a natural exponential family, and the use of such distributions simplifies the theory and computation of generalized linear models.
Contents
- Probability distribution function PDF of the univariate case scalar domain scalar parameter
- Probability distribution function PDF of the general case multivariate domain andor parameter
- Moment and cumulant generating function
- Examples
- Properties
- Univariant case
- Multivariate case
- Natural exponential families with quadratic variance functions NEF QVF
- The six NEF QVFs
- Properties of NEF QVF
- References
Probability distribution function (PDF) of the univariate case (scalar domain, scalar parameter)
The natural exponential family (NEF) is a subset of the exponential family. NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in the exponential family with parameter θ can be written with probability density function (PDF)
where
[Note that slightly different notation is used by the originator of the NEF, Carl Morris. Morris uses ω instead of η and ψ instead of A.]
Probability distribution function (PDF) of the general case (multivariate domain and/or parameter)
Suppose that
where in this case the parameter
Moment and cumulant generating function
A member of a natural exponential family has moment generating function (MGF) of the form
The cumulant generating function is by definition the logarithm of the MGF, so it is
Examples
The five most important univariate cases are:
These five examples – Poisson, binomial, negative binomial, normal, and gamma – are a special subset of NEF, called NEF with quadratic variance function (NEF-QVF) because the variance can be written as a quadratic function of the mean. NEF-QVF are discussed below.
Distributions such as the exponential, chi-squared, Rayleigh, Weibull, Bernoulli, and geometric distributions are special cases of the above five distributions. Many common distributions are either NEF or can be related to the NEF. For example: the chi-squared distribution is a special case of the gamma distribution. The Bernoulli distribution is a binomial distribution with n = 1 trial. The exponential distribution is a gamma distribution with shape parameter α = 1 (or k = 1 ). The Rayleigh and Weibull distributions can each be written in terms of an exponential distribution.
Some exponential family distributions are not NEF. The lognormal and Beta distribution are in the exponential family, but not the natural exponential family.
The parameterization of most of the above distributions has been written differently from the parameterization commonly used in textbooks and the above linked pages. For example, the above parameterization differs from the parameterization in the linked article in the Poisson case. The two parameterizations are related by
for
This alternative parameterization can greatly simplify calculations in mathematical statistics. For example, in Bayesian inference, a posterior probability distribution is calculated as the product of two distributions. Normally this calculation requires writing out the probability distribution functions (PDF) and integrating; with the above parameterization, however, that calculation can be avoided. Instead, relationships between distributions can be abstracted due to the properties of the NEF described below.
An example of the multivariate case is the multinomial distribution with known number of trials.
Properties
The properties of the natural exponential family can be used to simplify calculations involving these distributions.
Univariant case
1. The cumulants of an NEF can be calculated as derivatives of the NEF's cumulant generating function. The nth cumulant is the nth derivative with respect to θ of the cumulant generating function.
The cumulant generating function is
The first cumulant is
The mean is the first moment and always equal to the first cumulant, so
The variance is always the second moment, and it is always related to the first and second cumulants by
so that
The nth cumulant is
2. Natural exponential families (NEF) are closed under convolution.
Given independent identically distributed (iid)
3. The variance function for random variables with an NEF distribution can be written in terms of the mean.
4. The first two moments of a NEF distribution uniquely specify the distribution within that family of distributions.
Multivariate case
In the multivariate case, the mean vector and covariance matrix are
where
Natural exponential families with quadratic variance functions (NEF-QVF)
A special case of the natural exponential families are those with quadratic variance functions. Six NEFs have quadratic variance functions (QVF) in which the variance of the distribution can be written as a quadratic function of the mean. These are called NEF-QVF. The properties of these distributions were first described by Carl Morris.
The six NEF-QVFs
The six NEF-QVF are written here in increasing complexity of the relationship between variance and mean.
1. The normal distribution with fixed variance
2. The Poisson distribution
3. The Gamma distribution
4. The binomial distribution
5. The negative binomial distribution
6. The (not very famous) distribution generated by the generalized hyperbolic secant distribution (NEF-GHS) has
Properties of NEF-QVF
The properties of NEF-QVF can simplify calculations that use these distributions.
1. Natural exponential families with quadratic variance functions (NEF-QVF) are closed under convolutions of a linear transformation. That is, a convolution of a linear transformation of an NEF-QVF is also an NEF-QVF, although not necessarily the original one.
Given independent identically distributed (iid)
Let
then the new NEF-QVF has variance function
where
2. Let
3. NEF-QVF have conjugate prior distributions on μ in the Pearson system of distributions (also called the Pearson distribution although the Pearson system of distributions is actually a family of distributions rather than a single distribution.) Examples of conjugate prior distributions of NEF-QVF distributions are the normal, gamma, reciprocal gamma, beta, F-, and t- distributions. Again, these conjugate priors are not all NEF-QVF.
4. If
These properties together with the above notation can simplify calculations in mathematical statistics that would normally be done using complicated calculations and calculus.