Moments  (Estimation by the method of)

Estimation of the moments of a distribution

Let X be a r.v. (either discrete or continuous) with probability distribution p(x). Let also x = {x₁, x₂, ..., x_n} be a n-sample drawn form this distribution. Then the moments of the sample are consistent estimators of the moments of the distribution.

In other words :

    * Let µ_i be the order i moment of the distribution :

µ_i = E[X ⁱ]

    * Let m_i⁽ⁿ⁾ be the order i moment of a n-sample :

The m_i⁽ⁿ⁾ are realizations of the r.v. M_i⁽ⁿ⁾ :

Then for any i, the sequence {M_i⁽ⁿ⁾} of r.v. converges in probability to µ_i when  grows without limit.

We outline here a proof of this result as an application of the generalized Weak Law of Large Numbers.

-----

In practical terms, this means that the larger the sample, the most trustworthy are the measure values of the sample moments as estimates of the true values of the corresponding moments of the distribution.

Estimation of the parametersof a distribution

But the moments of the sample can be used for more than just estimating the moments of the distribution : they can also be used  for estimating one or several parameters of the distribution.

Consider a distribution f(x | θ₁, ..., θ_k), either continuous or discrete, that depends on k parameters.

• On the one hand, we have the fist k moments m₁, m₂ , ..., m_k of the sample :

• On the other hand, we have the first k moments µ₁, µ₂ , ..., µ_k  of the distribution :

µ_i = E[xⁱ]

Note that these moments are functions of the parameters (θ₁, ..., θ_k) of the distribution.

Estimating the parameters (θ₁, ..., θ_k) by the methods of moments consists in equating the (known) moments of the sample with the corresponding (unknown) moments of the distribution :

This is a set of k equations. If this system  can be solved :

θ_i* = g_i(m₁, m₂ , ..., m_k)               i = 1, 2, ..., k

then every θ_i* is a convergent estimator of the corresponding parameter θ_i.

We illustrate here the method of moments by estimating simultaneously the two parameters (n, p) of the binomial distribution B(n, p).

-----

Although the method of moments is conceptually simpler than the method of maximum likelihood, estimators calculated by the method of moments do not have the good asymptotic properties of maximum likelihood estimators. Consequently, the method of moments is not widely used.

 ____________________________________________

Related readings :