Moments  (Estimation by the method of)

# Estimation of the moments of a distribution

Let X be a r.v. (either discrete or continuous) with probability distribution p(x). Let also x = {x1, x2, ..., xn} be a n-sample drawn form this distribution. Then the moments of the sample are consistent estimators of the moments of the distribution.

In other words :

* Let µi be the order i moment of the distribution :

µi = E[X i]

* Let mi(n) be the order i moment of a n-sample :

The mi(n) are realizations of the r.v. Mi(n) :

Then for any i, the sequence {Mi(n)} of r.v. converges in probability to µi when  grows without limit.

We outline here a proof of this result as an application of the generalized Weak Law of Large Numbers.

-----

In practical terms, this means that the larger the sample, the most trustworthy are the measure values of the sample moments as estimates of the true values of the corresponding moments of the distribution.

# Estimation of the parametersof a distribution

But the moments of the sample can be used for more than just estimating the moments of the distribution : they can also be used  for estimating one or several parameters of the distribution.

Consider a distribution f(xθ1, ..., θk), either continuous or discrete, that depends on k parameters.

• On the one hand, we have the fist k moments m1, m2 , ..., mk of the sample :

• On the other hand, we have the first k moments µ1, µ2 , ..., µk  of the distribution :

µi = E[xi]

Note that these moments are functions of the parameters (θ1, ..., θk) of the distribution.

Estimating the parameters (θ1, ..., θk) by the methods of moments consists in equating the (known) moments of the sample with the corresponding (unknown) moments of the distribution :

 mi = µi         for i = 1, 2, ..., k

This is a set of k equations. If this system  can be solved :

θi* = gi(m1, m2 , ..., mk)               i = 1, 2, ..., k

then every θi* is a convergent estimator of the corresponding parameter θi.

We illustrate here the method of moments by estimating simultaneously the two parameters (n, p) of the binomial distribution B(n, p).

-----

Although the method of moments is conceptually simpler than the method of maximum likelihood, estimators calculated by the method of moments do not have the good asymptotic properties of maximum likelihood estimators. Consequently, the method of moments is not widely used.

____________________________________________

 Estimation Maximum Likelihood Weak Law of Large Numbers