Partial leverage

In regression analysis, partial leverage is a measure of the contribution of the individual independent variables to the leverage of each observation. That is, if h_i is the i^th element of the diagonal of the hat matrix, the partial leverage is a measure of how h_i changes as a variable is added to the regression model.

The partial leverage is computed as:

( P L j ) i = ( X j ∙ [ j ] ) i 2 ∑ k = 1 n ( X j ∙ [ j ] ) k 2

where

j = index of independent variablei = index of observationX_j·[j] = residuals from regressing X_j against the remaining independent variables

Note that the partial leverage is the leverage of the i^th point in the partial regression plot for the j^th variable. Data points with large partial leverage for an independent variable can exert undue influence on the selection of that variable in automatic regression model building procedures.

In statistics, high-leverage points are those that are outliers with respect to the independent variables. In other words, high-leverage points have no neighbouring points in R p space, where p is the number of independent variables in a regression model. This makes the fitted model likely to pass close to a high leverage observation. Hence high-leverage points have the potential to cause large changes in the parameter estimates when they are deleted—i.e., to be influential points. Although an influential point will typically have high leverage, a high leverage point is not necessarily an influential point. The leverage is typically defined as the diagonal of the hat matrix, which is

H = X ( X ′ X ) − 1 X ′ .