Exact statistics, such as that described in exact test, is a branch of statistics that was developed to provide more accurate results pertaining to statistical testing and interval estimation by eliminating procedures based on asymptotic and approximate statistical methods. The main characteristic of exact methods is that statistical tests and confidence intervals are based on exact probability statements that are valid for any sample size. Exact statistical methods help avoid some of the unreasonable assumptions of traditional statistical methods, such as the assumption of equal variances in classical ANOVA. They also allow exact inference on variance components of mixed models.
Contents
When exact p-values and confidence intervals are computed under a certain distribution, such as the normal distribution, then the underlying methods are referred to as exact parametric methods. The exact methods that do not make any distributional assumptions are referred to as exact nonparametric methods. The latter has the advantage of making fewer assumptions whereas, the former tend to yield more powerful tests when the distributional assumption is reasonable. For advanced methods such as higher-way ANOVA regression analysis, and mixed models, only exact parametric methods are available.
When the sample size is small, asymptotic results given by some traditional methods may not be valid. In such situations, the asymptotic p-values may differ substantially from the exact p-values. Hence asymptotic and other approximate results may lead to unreliable and misleading conclusions.
The approach
All classical statistical procedures are constructed using statistics which depend only on observable random vectors, whereas generalized estimators, tests, and confidence intervals used in exact statistics take advantage of the observable random vectors and the observed values both, as in the Bayesian approach but without having to treat constant parameters as random variables. For example, in sampling from a normal population with mean
and that
Now suppose the parameter of interest is the coefficient of variation,
where
Generalized p-values
Classical statistical methods do not provide exact tests to many statistical problems such as testing Variance Components and ANOVA under unequal variances. To rectify this situation, the generalized p-values are defined as an extension of the classical p-values so that one can perform tests based on exact probability statements valid for any sample size.