Mean (of a probability distribution)

1) Discrete case

    If x₀, x₁ , ..., x_i , ... is a (possibly infinite) series of numbers, and if probability p_i is assigned to number x_i , then by definition, the mean m of this distribution is :

assuming that this quantity exists, that is, is less that .

In other words, the mean is the ponderated sum of the numbers, each number being weighted by its probability.

Of course, _i p_i = 1.

If there is only a finite number of numbers, and if they all carry the same probability, then the mean is the ordinary average.

2) Continuous distribution

    The distribution is now defined by a probability density function p(x). The definition of the mean is the same as above, with the summation symbol replaced by the integral symbol.

assuming that this quantity exists, that is, is less that 4.

 
Of course, the integral of p(x) is 1.

__________________________________________

It is sometimes heard that the mean defines the region where it is most likely to see new points pop-up. This is not the case : new points are likely to appear in regions where the probability density is high, which may be quite different from where the mean is. In this illustration, the mean is in a region where the probability density is "0", and no observation will ever show-up there.

____________________________________________

The most common estimator of the distribution mean  is the sample average. Please see here an animated illustration that will explain why.

 ____________________________________________

The vast majority of classical distributions have a mean. But some perfectly respectable distributions don't. The reason is always the same:

    * p(x) falls off fast enough at infinity to guarantee that its integral is finite (equal to 1).

    * But not fast enough to prevent the formal expression of the mean:

to be meaningless because the integral yields a quantity that is infinite.

If such is the case, then higher order moments do not exist either.

The most classical example of a distribution without a mean is the Cauchy distribution. Two other examples are Fisher's F_{n, 2} and F_{n, 1}.

 ____________________________________________

Related readings