Puneet Varma

Scoring algorithm

Updated on
Share on FacebookTweet on TwitterShare on LinkedIn

Scoring algorithm, also known as Fisher's scoring, is a form of Newton's method used in statistics to solve maximum likelihood equations numerically, named after Ronald Fisher.

Contents

Sketch of Derivation

Let Y 1 , , Y n be random variables, independent and identically distributed with twice differentiable p.d.f. f ( y ; θ ) , and we wish to calculate the maximum likelihood estimator (M.L.E.) θ of θ . First, suppose we have a starting point for our algorithm θ 0 , and consider a Taylor expansion of the score function, V ( θ ) , about θ 0 :

V ( θ ) V ( θ 0 ) J ( θ 0 ) ( θ θ 0 ) ,

where

J ( θ 0 ) = i = 1 n | θ = θ 0 log f ( Y i ; θ )

is the observed information matrix at θ 0 . Now, setting θ = θ , using that V ( θ ) = 0 and rearranging gives us:

θ θ 0 + J 1 ( θ 0 ) V ( θ 0 ) .

We therefore use the algorithm

θ m + 1 = θ m + J 1 ( θ m ) V ( θ m ) ,

and under certain regularity conditions, it can be shown that θ m θ .

Fisher scoring

In practice, J ( θ ) is usually replaced by I ( θ ) = E [ J ( θ ) ] , the Fisher information, thus giving us the Fisher Scoring Algorithm:

θ m + 1 = θ m + I 1 ( θ m ) V ( θ m ) .

References

Scoring algorithm Wikipedia


Similar Topics
First to Fight (film)
Cookies (film)
Michael Shelley (mathematician)
Topics