![]() | ||
The Theil index is a statistic primarily used to measure economic inequality and other economic phenomena, though it has also been used to measure racial segregation. The basic Theil index TT is the same as redundancy in information theory which is the maximum possible entropy of the data minus the observed entropy. It is a special case of the generalized entropy index. It can be viewed as a measure of redundancy, lack of diversity, isolation, segregation, inequality, non-randomness, and compressibility. It was proposed by econometrician Henri Theil at the Erasmus University Rotterdam.
Contents
Formula
For a population of N "agents" each with characteristic x, the situation may be represented by the list xi (i=1,...,N) where xi is the characteristic of agent i. For example, if the characteristic is income, then xi is the income of agent i. The Theil index is defined as:
where
Equivalently, if the situation is characterized by a discrete distribution function fk (k=0,...,W) where fk is the fraction of the population with income k and W = Nμ is the total income, then
where
Note that in this case income k is an integer and k=1 represents the smallest increment of income possible (e.g., cents).
if the situation is characterized by a continuous distribution function f(k) (supported from 0 to infinity) where f(k)dk is the fraction of the population with income k to k+dk, then the Theil index is:
where the mean is:
Theil indices for some common continuous probability distributions are given in the table below:
If everyone has the same income, then TT equals 0. If one person has all the income, then TT gives the result
The Theil index measures an entropic "distance" the population is away from the egalitarian state of everyone having the same income. The numerical result is in terms of negative entropy so that a higher number indicates more order that is further away from the complete equality. Formulating the index to represent negative entropy instead of entropy allows it to be a measure of inequality rather than equality.
Derivation from entropy
The Theil index is derived from Shannon's measure of information entropy
where
The Theil index is
When
The Theil index measures what is called redundancy in information theory. It is the left over "information space" that was not utilized to convey information, which reduces the effectiveness of the price signal. The Theil index is a measure of the redundancy of income (or other measure of wealth) in some individuals. Redundancy in some individuals implies scarcity in others. A high Theil index indicates the total income is not distributed evenly among individuals in the same way an uncompressed text file does not have a similar number of byte locations assigned to the available unique byte characters.
Decomposability
One of the advantages of the Theil index is that it is a weighted average of inequality within subgroups, plus inequality between those subgroups. For example, inequality within the United States is the average inequality within each state, weighted by state income, plus the inequality between states.
If for the Theil index the population is divided into
The decomposition of the overall Theil index which identifies the share attributable to the between-region component becomes a helpful tool for the positive analysis of regional inequality as it suggests the relative importance of spatial dimension of inequality.
The decomposability is a property of the Theil index which the more popular Gini coefficient does not offer. The Gini coefficient is more intuitive to many people since it is based on the Lorenz curve. However, it is not easily decomposable like the Theil.
Applications
In addition to multitude of economic applications, the Theil index has been applied to assess performance of irrigation systems and distribution of software metrics.