Convolution

Let X and Y be two independent random variables, either discrete of continuous. What is the probability distribution of :

Z = X + Y  ?

Denote :

    * f_X (.) the distribution of X, and

    * f_Y (.) the distribution of Y.

Clearly, the probability distribution of Z (denote it f_Z (.)) must be some combination of f_X (.) and f_Y (.).

This combination is called the convolution of f_X (.) and f_Y (.), and is denoted "_*" :

f_Z (.) = f_{X + Y} (.) = f_X (.)_* f_Y (.)

Continuous variables

We already showed here that the convolution of two continuous distributions takes the form :

that we'll obtain again in the Tutorial below by another method.

This result receives the following useful graphic interpretation :

    * To calculate the value of f_{X + Y} (z), keep f_X (.) as is. Then :

        1) Flip f_Y (.) around the vertical axis (because of the "-" sign).

        2) Shift the flipped curve by "z".

        3) Multiply this curve by the f_X (.) curve. The result is the red curve in the illustration below.

        4) Integrate this product from -∞ to ∞.

The value of f_{X + Y} (z) is the pink area.

-----

We'll illustrate the concept of convolution of two continuous distributions by calculating the convolution of two normal distributions, two Cauchy distributions and two Gamma distributions.

Independent, non-negative integer-valued random variables

In the discrete case, the concept of convolution is particularly convenient in the case of independent non-negative integer valued random variables, that is, variables that represent a count.

We'll show that then :

which is formally similar to the expression for continuous variables.

The interpretation of this expression is now :

    * To calculate the value of f_{X + Y} (n) :

        1) Multiply each of the first (n + 1) terms of f_X (.) by the corresponding terms of  f_Y (.) in a crossed fashion as in the illustration below.

        2) Add these (n + 1) products.

The result is f_{X + Y} (n).

Although calculating the convolution of the two distributions of two independent, non-negative, integer-valued rv is the most direct way of obtaining the distribution of their sum, it is sometimes not the most convenient way. We'll see that calling on the proprties of the generating function usually leads to shorter and simpler calculations.

Algebraic properties of the convolution

Convolution is an operator to be found in many areas of mathematics and physics. In Probability Theory, its most important properties are :

The set of distributions is closed by convolution

As the sum of two r.v. is a r.v., the set of probability distributions is closed by convolution : the convolution of two probability distributions is a probability distribution.

Commutativity

If f_X (.) and f_T (.) are two probabiity distributions, then :

f_X (.)_* f_Y (.) = f_Y (.)_* f_X (.)

Going over the demonstrations of the above two expressions for the convolution of two probability distributions, you'll easily convince yourself that the roles of f_X (.) and f_Y (.) can be exchanged without altering the result.

This is another way of saying that the distribution of (X + Y) is the same as that of (Y + X) (in fact, these two r.v. are identical).

Associativity

Given three r.v. X, Y, and Z, the distribution of X + (Y + Z) is the same as that of (X + Y) + Z. Consequently :

f_X (.)_*[ f_Y (.)_*f_Z (.)] = [f_X (.)_* f_Y (.)]_*f_Z (.)

a result which can be proven directly without making reference to random variables.

___________________________________________________________________

In this Tutorial, we first demonstrate the convolution formulas both for the discrete (non-negative, integer valued rv) and the continuous case.

We then use these results for demonstrating the additivity property of the binomial, negative binomial and Poisson distributions. The geometric distribution may be considered a special case of the negative binomial distribution, but we also show directly that the sum of two iid geometric rv is negative binomial.

-----

Examples of convolution of continuous distributions are given in the next Tutorial.

CONVOLUTION

____________________________________________________

We now illustrate the continuous case by calculating the convolution of :

    * Two normal distributions. We'll conclude that the sum of two independent normally distributed rvs is also normal, with its variance being the sum of the variances of the variables (we already obtained this result by calling on the properties of the moment generating function).

    * Two Cauchy distributions. We'll conclude that the distribution of the sample mean is identical to the original Cauchy mother distribution irrespective of the sample size, a most striking result.

    * Two Gamma distributions (with the same spread parameter). We'll see that the Gamma distribution enjoys the additivity property. In the special case where the shape parameter is an integer, the result will show that the sum of iid exponential rvs is Gamma distributed.

We also establish this result here by calling on the properties of the moment generating function.

CONVOLUTION OF CONTINUOUS DISTRIBUTIONS

___________________________________________________

Related readings :