![]() | ||
In statistics and in probability theory, distance correlation is a measure of statistical dependence between two random variables or two random vectors of arbitrary, not necessarily equal dimension. It is zero if and only if the random variables are statistically independent, contrary to Pearson's correlation, which can be zero for dependent random variables.
Contents
- Background
- Distance covariance
- Distance variance and standard deviation
- Distance correlation
- Distance variance
- Generalization
- Alternative definition of distance covariance
- Alternative formulation Brownian covariance
- References
The distance correlation is derived from a number of other quantities that are used in its specification, specifically: distance variance, distance standard deviation and distance covariance. These quantities take the same roles as the ordinary moments with corresponding names in the specification of the Pearson product-moment correlation coefficient.
These distance-based measures can be put into an indirect relationship to the ordinary moments by an alternative formulation (described below) using ideas related to Brownian motion, and this has led to the use of names such as Brownian covariance and Brownian distance covariance.
Background
The classical measure of dependence, the Pearson correlation coefficient, is mainly sensitive to a linear relationship between two variables. Distance correlation was introduced in 2005 by Gabor J Szekely in several lectures to address this deficiency of Pearson’s correlation, namely that it can easily be zero for dependent variables. Correlation = 0 (uncorrelatedness) does not imply independence while distance correlation = 0 does imply independence. The first results on distance correlation were published in 2007 and 2009. It was proved that distance covariance is the same as the Brownian covariance. These measures are examples of energy distances.
Distance covariance
Let us start with the definition of the sample distance covariance. Let (Xk, Yk), k= 1, 2, ..., n be a statistical sample from a pair of real valued or vector valued random variables (X, Y). First, compute the n by n distance matrices (aj, k) and (bj, k) containing all pairwise distances
where || ⋅ || denotes Euclidean norm. Then take all doubly centered distances
where
The statistic Tn = n dCov2n(X, Y) determines a consistent multivariate test of independence of random vectors in arbitrary dimensions. For an implementation see dcov.test function in the energy package for R.
The population value of distance covariance can be defined along the same lines. Let X be a random variable that takes values in a p-dimensional Euclidean space with probability distribution μ and let Y be a random variable that takes values in a q-dimensional Euclidean space with probability distribution ν, and suppose that X and Y have finite expectations. Write
Finally, define the population value of squared distance covariance of X and Y as
One can show that this is equivalent to the following definition:
where E denotes expected value, and
This identity shows that the distance covariance is not the same as the covariance of distances, cov(||X-X' ||, ||Y-Y' ||). This can be zero even if X and Y are not independent.
Alternatively, the squared distance covariance can be defined as the weighted L2 norm of the distance between the joint characteristic function of the random variables and the product of their marginal characteristic functions:
where ϕX, Y(s, t), ϕX(s), and ϕY(t) are the characteristic functions of (X, Y), X, and Y, respectively, p, q denote the Euclidean dimension of X and Y, and thus of s and t, and cp, cq are constants. The weight function
Distance variance and standard deviation
The distance variance is a special case of distance covariance when the two variables are identical. The population value of distance variance is the square root of
where
The sample distance variance is the square root of
which is a relative of Corrado Gini’s mean difference introduced in 1912 (but Gini did not work with centered distances).
The distance standard deviation is the square root of the distance variance.
Distance correlation
The distance correlation of two random variables is obtained by dividing their distance covariance by the product of their distance standard deviations. The distance correlation is
and the sample distance correlation is defined by substituting the sample distance covariance and distance variances for the population coefficients above.
For easy computation of sample distance correlation see the dcor function in the energy package for R.
Distance correlation
(i)
(ii)
(iii)
Distance covariance
(i)
(ii)
(iii) If the random vectors
Equality holds if and only if
(iv)
This last property is the most important effect of working with centered distances.
The statistic
An unbiased estimator of
Distance variance
(i)
(ii)
(iii)
(iv) If
Equality holds in (iv) if and only if one of the random variables
Generalization
Distance covariance can be generalized to include powers of Euclidean distance. Define
Then for every
One can extend
This is non-negative for all such
Alternative definition of distance covariance
The original distance covariance has been defined as the square root of
Alternately, one could define distance covariance to be the square of the energy distance:
Under these alternate definitions, the distance correlation is also defined as the square
Alternative formulation: Brownian covariance
Brownian covariance is motivated by generalization of the notion of covariance to stochastic processes. The square of the covariance of random variables X and Y can be written in the following form:
where E denotes the expected value and the prime denotes independent and identically distributed copies. We need the following generalization of this formula. If U(s), V(t) are arbitrary random processes defined for all real s and t then define the U-centered version of X by
whenever the subtracted conditional expected value exists and denote by YV the V-centered version of Y. The (U,V) covariance of (X,Y) is defined as the nonnegative number whose square is
whenever the right-hand side is nonnegative and finite. The most important example is when U and V are two-sided independent Brownian motions /Wiener processes with expectation zero and covariance |s| + |t| - |s-t| = 2 min(s,t) (for nonnegative s, t only). (This is twice the covariance of the standard Wiener process; here the factor 2 simplifies the computations.) In this case the (U,V) covariance is called Brownian covariance and is denoted by
There is a surprising coincidence: The Brownian covariance is the same as the distance covariance:
and thus Brownian correlation is the same as distance correlation.
On the other hand, if we replace the Brownian motion with the deterministic identity function id then Covid(X,Y) is simply the absolute value of the classical Pearson covariance,