Leverage

In Linear Regression (Simple or Multiple), any prediction yi* can be represented as a linear combination of the observations yj :

yi* = Σj hij yj

The coefficient of observation yi, that is hii , is called the leverage of that observation. The larger hii, the larger the contribution of yi to yi*.

It can be shown that :

• The value of any leverage is between 0 and 1 :

0 hii 1

• The sum of the leverages is p, the number of parameters of the regression model (that is, in general, the number of predictors + 1).

Leverages depend only on the {xj}, and are fixed, not random quantities.

It can be shown that an observation with a large leverage ("large" meaning substantially larger than the average value p/n, where n is the number of observations) lies at the periphery of the predictors' domain. An observation with a large leverage is called a "leverage observation".

By itself, the leverage is of limited usefulness because it does not take into account the residual of the observation. But the classical measures of the influence of an observation on the predictions of the model (e.g. DFFITS, Cook's distance) combine both leverages and residuals.
These quantites and their properties are described in one of the Tutorials on Linear Regression.

____________________________________________________________

 Simple Linear Regression Multiple Linear Regression DFFITS Cook's distance