Neha Patil (Editor)

Jackknife resampling

Updated on
Edit
Like
Comment
Share on FacebookTweet on TwitterShare on LinkedInShare on Reddit

In statistics, the jackknife is a resampling technique especially useful for variance and bias estimation. The jackknife predates other common resampling methods such as the bootstrap. The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the estimate and then finding the average of these calculations. Given a sample of size N , the jackknife estimate is found by aggregating the estimates of each N − 1 -sized sub-sample.

Contents

The jackknife technique was developed by Maurice Quenouille (1949, 1956). John Tukey (1958) expanded on the technique and proposed the name "jackknife" since, like a physical jack-knife (a compact folding knife), it is a rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool.

The jackknife is a linear approximation of the bootstrap.

Estimation

The jackknife estimate of a parameter can be found by estimating the parameter for each subsample omitting the ith observation to estimate the previously unknown value of a parameter (say x ¯ i ).

x ¯ i = 1 n − 1 ∑ j ≠ i n x j

Variance estimation

An estimate of the variance of an estimator can be calculated using the jackknife technique.

Var ( j a c k k n i f e ) = n − 1 n ∑ i = 1 n ( x ¯ i − x ¯ ( . ) ) 2

where x ¯ i is the parameter estimate based on leaving out the ith observation, and x ¯ ( . ) = 1 n ∑ i n x ¯ i is the estimator based on all of the subsamples.

Bias estimation and correction

The jackknife technique can be used to estimate the bias of an estimator calculated over the entire sample. Say θ ^ is the calculated estimator of the parameter of interest based on all n observations. Let

θ ^ ( . ) = 1 n ∑ i = 1 n θ ^ ( i )

where θ ^ ( i ) is the estimate of interest based on the sample with the ith observation removed, and θ ^ ( . ) is the average of these "leave-one-out" estimates. The jackknife estimate of the bias of θ ^ is given by:

Bias ^ ( θ ) = ( n − 1 ) ( θ ^ ( . ) − θ ^ )

and the resulting bias-corrected jackknife estimate of θ is given by:

θ ^ Jack = n θ ^ − ( n − 1 ) θ ^ ( . )

This removes the bias in the special case that the bias is O ( N − 1 ) and to O ( N − 2 ) in other cases.

This provides an estimated correction of bias due to the estimation method. The jackknife does not correct for a biased sample.

References

Jackknife resampling Wikipedia
(Text) CC BY-SA


Similar Topics