Puneet Varma (Editor)

De Moivre–Laplace theorem

Updated on
Edit
Like
Comment
Share on FacebookTweet on TwitterShare on LinkedInShare on Reddit
De Moivre–Laplace theorem

In probability theory, the de Moivre–Laplace theorem, which is a special case of the central limit theorem, states that the normal distribution may be used as an approximation to the binomial distribution under certain conditions. In particular, the theorem shows that the probability mass function of the random number of "successes" observed in a series of n independent Bernoulli trials, each having probability p of success (a binomial distribution with n trials), converges to the probability density function of the normal distribution with mean np and standard deviation √np(1-p), as n grows large, assuming p is not 0 or 1.

Contents

The theorem appeared in the second edition of The Doctrine of Chances by Abraham de Moivre, published in 1738. Although de Moivre did not use the term "Bernoulli trials", he wrote about the probability distribution of the number of times "heads" appears when a coin is tossed 3600 times.

This is one derivation of the particular Gaussian function used in the normal distribution.

Theorem

As n grows large, for k in the neighborhood of np we can approximate

( n k ) p k q n − k ≃ 1 2 π n p q e − ( k − n p ) 2 2 n p q , p + q = 1 ,   p , q > 0

in the sense that the ratio of the left-hand side to the right-hand side converges to 1 as n → ∞.

Proof

Note that k cannot be fixed or it would quickly fall outside the range of interest as n → ∞. What is needed is to let k vary but always be a fixed number of standard deviations from the mean, so that it is always associated with the same point on the standard normal distribution. We can do this by defining

k = n p + x n p q

for some fixed x. Then, for example, when x = 1, k will always be 1 standard deviation from the mean. From this definition we have the approximations k→np and k n → p as n → ∞.

However, the left-hand side requires that k be an integer. Keeping the notation but assuming that k is the nearest integer given by the definition, this is seen to be inconsequential in the limit by noting that as n → ∞ the change in x required to make k an integer becomes small and successive integer values of k produce converging values on the right-hand side:

1 2 π n p q e − ( k + 1 − n p ) 2 2 n p q 1 2 π n p q e − ( k − n p ) 2 2 n p q = e 2 n p − 2 k − 1 2 n p q ≃ e − x n p q ≃ e 0 = 1 as  n → ∞ .

The proof consists of transforming the left-hand side to the right-hand side by three approximations.

First, according to Stirling's formula, we can replace the factorial of a large number n with the approximation:

n ! ≃ n n e − n 2 π n as  n → ∞ .

Thus

( n k ) p k q n − k = n ! k ! ( n − k ) ! p k q n − k ≃ n n e − n 2 π n k k e − k 2 π k ( n − k ) n − k e − ( n − k ) 2 π ( n − k ) p k q n − k = n 2 π k ( n − k ) n n k k ( n − k ) n − k p k q n − k = n 2 π k ( n − k ) ( n p k ) k ( n q n − k ) n − k

Next, use the approximation k n → p to match the root above to the desired root on the right-hand side.

( n k ) p k q n − k ≃ 1 2 π n k n ( 1 − k n ) ( n p k ) k ( n q n − k ) n − k ≃ 1 2 π n p q ( n p k ) k ( n q n − k ) n − k p + q = 1

Finally, rewrite the expression as an exponential and use the Taylor Series approximation for ln(1+x):

ln ⁡ ( 1 + x ) ≃ x − x 2 2 + x 3 3 − . . .

Then

( n k ) p k q n − k ≃ 1 2 π n p q exp ⁡ { ln ⁡ ( ( n p k ) k ) + ln ⁡ ( ( n q n − k ) n − k ) } = 1 2 π n p q exp ⁡ { − k ln ⁡ ( k n p ) + ( k − n ) ln ⁡ ( n − k n q ) } = 1 2 π n p q exp ⁡ { − k ln ⁡ ( n p + x n p q n p ) + ( k − n ) ln ⁡ ( n − n p − x n p q n q ) } = 1 2 π n p q exp ⁡ { − k ln ⁡ ( 1 + x q n p ) + ( k − n ) ln ⁡ ( 1 − x p n q ) } p + q = 1 = 1 2 π n p q exp ⁡ { − k ( x q n p − x 2 q 2 n p + . . . ) + ( k − n ) ( − x p n q − x 2 p 2 n q − . . . ) } = 1 2 π n p q exp ⁡ { ( − n p − x n p q ) ( x q n p − x 2 q 2 n p + . . . ) + ( n p + x n p q − n ) ( − x p n q − x 2 p 2 n q − . . . ) } = 1 2 π n p q exp ⁡ { ( − n p − x n p q ) ( x q n p − x 2 q 2 n p + . . . ) − ( n q − x n p q ) ( − x p n q − x 2 p 2 n q − . . . ) } = 1 2 π n p q exp ⁡ { ( − x n p q + 1 2 x 2 q − x 2 q + . . . ) + ( x n p q + 1 2 x 2 p − x 2 p − . . . ) } = 1 2 π n p q exp ⁡ { − 1 2 x 2 q − 1 2 x 2 p − . . . } = 1 2 π n p q exp ⁡ { − 1 2 x 2 ( p + q ) − . . . } ≃ 1 2 π n p q exp ⁡ { − 1 2 x 2 } = 1 2 π n p q e − ( k − n p ) 2 2 n p q

References

De Moivre–Laplace theorem Wikipedia
(Text) CC BY-SA