Leverage

In Linear Regression (Simple or Multiple), any prediction y_i^* can be represented as a linear combination of the observations y_j :

 y_i^* = Σ_j h_ij y_j

The coefficient of observation y_i, that is h_ii , is called the leverage of that observation. The larger h_ii, the larger the contribution of y_i to y_i^*.

It can be shown that :

• The value of any leverage is between 0 and 1 :

0 h_ii 1

• The sum of the leverages is p, the number of parameters of the regression model (that is, in general, the number of predictors + 1).

Leverages depend only on the {x_j}, and are fixed, not random quantities.

It can be shown that an observation with a large leverage ("large" meaning substantially larger than the average value p/n, where n is the number of observations) lies at the periphery of the predictors' domain. An observation with a large leverage is called a "leverage observation".

By itself, the leverage is of limited usefulness because it does not take into account the residual of the observation. But the classical measures of the influence of an observation on the predictions of the model (e.g. DFFITS, Cook's distance) combine both leverages and residuals.
These quantites and their properties are described in one of the Tutorials on Linear Regression.

____________________________________________________________

Related readings :