Gauss-Markov theorem

In Simple or Multiple Linear Regression, the model parameters are most often calculated by the Least Squares method. The main advantage of this method is its mathematical simplicity, which allows an easy identification of the statistical properties of the calculated estimators, particularly their lack of bias and their variances. But there is no a priori reason to believe that these estimators are particularly good (low Mean Square Error).

The Gauss-Markov theorem is here to somewhat soften our worries about the quality of the Least Squares estimator of the vector of the model parameters. This theorem states that among all possible estimators that are both :

    * Linear in the observations,

    * And unbiased,

the Least Squares estimator is the one with the smallest variance.

-----

We give here a simple demonstration of the Gauss-Markov theorem .

 ________________________________________________________

Related readings :