Convolution

Let X and Y be two independent random variables, either discrete of continuous. What is the probability distribution of :

Z = X + Y  ?

Denote :

* fX (.) the distribution of X, and

* fY (.) the distribution of Y.

Clearly, the probability distribution of Z (denote it fZ (.)) must be some combination of fX (.) and fY (.).

This combination is called the convolution of fX (.) and fY (.), and is denoted "*" :

fZ (.) = fX + Y (.) = fX (.)* fY (.)

# Continuous variables

We already showed here that the convolution of two continuous distributions takes the form :

that we'll obtain again in the Tutorial below by another method.

This result receives the following useful graphic interpretation :

* To calculate the value of fX + Y (z), keep fX (.) as is. Then :

1) Flip fY (.) around the vertical axis (because of the "-" sign).

2) Shift the flipped curve by "z".

3) Multiply this curve by the fX (.) curve. The result is the red curve in the illustration below.

4) Integrate this product from -∞ to ∞.

The value of fX + Y (z) is the pink area.

-----

We'll illustrate the concept of convolution of two continuous distributions by calculating the convolution of two normal distributions, two Cauchy distributions and two Gamma distributions.

# Independent, non-negative integer-valued random variables

In the discrete case, the concept of convolution is particularly convenient in the case of independent non-negative integer valued random variables, that is, variables that represent a count.

We'll show that then :

 fX+Y (n) = Σi fX (i).fY (n - i)              i = 0, 1, ..., n

which is formally similar to the expression for continuous variables.

The interpretation of this expression is now :

* To calculate the value of fX + Y (n) :

1) Multiply each of the first (n + 1) terms of fX (.) by the corresponding terms of  fY (.) in a crossed fashion as in the illustration below.

2) Add these (n + 1) products.

The result is fX + Y (n).

Although calculating the convolution of the two distributions of two independent, non-negative, integer-valued rv is the most direct way of obtaining the distribution of their sum, it is sometimes not the most convenient way. We'll see that calling on the proprties of the generating function usually leads to shorter and simpler calculations.

# Algebraic properties of the convolution

Convolution is an operator to be found in many areas of mathematics and physics. In Probability Theory, its most important properties are :

## The set of distributions is closed by convolution

As the sum of two r.v. is a r.v., the set of probability distributions is closed by convolution : the convolution of two probability distributions is a probability distribution.

## Commutativity

If fX (.) and fT (.) are two probabiity distributions, then :

fX (.)* fY (.) = fY (.)* fX (.)

Going over the demonstrations of the above two expressions for the convolution of two probability distributions, you'll easily convince yourself that the roles of fX (.) and fY (.) can be exchanged without altering the result.

This is another way of saying that the distribution of (X + Y) is the same as that of (Y + X) (in fact, these two r.v. are identical).

## Associativity

Given three r.v. X, Y, and Z, the distribution of X + (Y + Z) is the same as that of (X + Y) + Z. Consequently :

fX (.)*[ fY (.)*fZ (.)] = [fX (.)* fY (.)]*fZ (.)

a result which can be proven directly without making reference to random variables.

___________________________________________________________________

 Tutorial 1

In this Tutorial, we first demonstrate the convolution formulas both for the discrete (non-negative, integer valued rv) and the continuous case.

We then use these results for demonstrating the additivity property of the binomial, negative binomial and Poisson distributions. The geometric distribution may be considered a special case of the negative binomial distribution, but we also show directly that the sum of two iid geometric rv is negative binomial.

-----

Examples of convolution of continuous distributions are given in the next Tutorial.

CONVOLUTION

 Convolution of discrete distributions Convolution of continuous distributions Distribution function Probability density function Additivity of the binomial distribution Sum of two independent geometric distributions Additivity of the negative binomial distribution Additivity of the Poisson distribution TUTORIAL

____________________________________________________

 Tutorial 2

We now illustrate the continuous case by calculating the convolution of :

* Two normal distributions. We'll conclude that the sum of two independent normally distributed rvs is also normal, with its variance being the sum of the variances of the variables (we already obtained this result by calling on the properties of the moment generating function).

* Two Cauchy distributions. We'll conclude that the distribution of the sample mean is identical to the original Cauchy mother distribution irrespective of the sample size, a most striking result.

* Two Gamma distributions (with the same spread parameter). We'll see that the Gamma distribution enjoys the additivity property. In the special case where the shape parameter is an integer, the result will show that the sum of iid exponential rvs is Gamma distributed.

We also establish this result here by calling on the properties of the moment generating function.

CONVOLUTION OF CONTINUOUS DISTRIBUTIONS

 Convolution of two normal distributions Influence of a translation on the convolution Distribution of the sum of independent normal variables Convolution of two Cauchy distributions Partial fraction decomposition The integration Final result Distribution of the sample mean Convolution of two Gamma distributions Convolution of two Gamma distributions Additivity property of the Gamma distribution The sum of iid exponential rvs is Gamma distributed TUTORIAL

___________________________________________________

 Generating function