Let X and Y be two independent random variables, either discrete of continuous. What is the probability distribution of :
Z = X + Y ?
* fX (.) the distribution of X, and
* fY (.) the distribution of Y.
Clearly, the probability distribution of Z (denote it fZ (.)) must be some combination of fX (.) and fY (.).
This combination is called the convolution of fX (.) and fY (.), and is denoted "*" :
fZ (.) = fX + Y (.) = fX (.)* fY (.)
We already showed here that the convolution of two continuous distributions takes the form :
that we'll obtain again in the Tutorial below by another method.
This result receives the following useful graphic interpretation :
* To calculate the value of fX + Y (z), keep fX (.) as is. Then :
1) Flip fY (.) around the vertical axis (because of the "-" sign).
2) Shift the flipped curve by "z".
3) Multiply this curve by the fX (.) curve. The result is the red curve in the illustration below.
4) Integrate this product from -∞ to ∞.
The value of fX + Y (z) is the pink area.
We'll illustrate the concept of convolution of two continuous distributions by calculating the convolution of two normal distributions, two Cauchy distributions and two Gamma distributions.
In the discrete case, the concept of convolution is particularly convenient in the case of independent non-negative integer valued random variables, that is, variables that represent a count.
We'll show that then :
fX+Y (n) = Σi fX (i).fY (n - i) i = 0, 1, ..., n
which is formally similar to the expression for continuous variables.
The interpretation of this expression is now :
* To calculate the value of fX + Y (n) :
1) Multiply each of the first (n + 1) terms of fX (.) by the corresponding terms of fY (.) in a crossed fashion as in the illustration below.
2) Add these (n + 1) products.
The result is fX + Y (n).
Although calculating the convolution of the two distributions of two independent, non-negative, integer-valued rv is the most direct way of obtaining the distribution of their sum, it is sometimes not the most convenient way. We'll see that calling on the proprties of the generating function usually leads to shorter and simpler calculations.
Convolution is an operator to be found in many areas of mathematics and physics. In Probability Theory, its most important properties are :
As the sum of two r.v. is a r.v., the set of probability distributions is closed by convolution : the convolution of two probability distributions is a probability distribution.
If fX (.) and fT (.) are two probabiity distributions, then :
fX (.)* fY (.) = fY (.)* fX (.)
Going over the demonstrations of the above two expressions for the convolution of two probability distributions, you'll easily convince yourself that the roles of fX (.) and fY (.) can be exchanged without altering the result.
This is another way of saying that the distribution of (X + Y) is the same as that of (Y + X) (in fact, these two r.v. are identical).
Given three r.v. X, Y, and Z, the distribution of X + (Y + Z) is the same as that of (X + Y) + Z. Consequently :
fX (.)*[ fY (.)*fZ (.)] = [fX (.)* fY (.)]*fZ (.)
a result which can be proven directly without making reference to random variables.
In this Tutorial, we first demonstrate the convolution formulas both for the discrete (non-negative, integer valued rv) and the continuous case.
We then use these results for demonstrating the additivity property of the binomial, negative binomial and Poisson distributions. The geometric distribution may be considered a special case of the negative binomial distribution, but we also show directly that the sum of two iid geometric rv is negative binomial.
Examples of convolution of continuous distributions are given in the next Tutorial.
Convolution of discrete distributions
Convolution of continuous distributions
Probability density function
Additivity of the binomial distribution
Sum of two independent geometric distributions
Additivity of the negative binomial distribution
Additivity of the Poisson distribution
We now illustrate the continuous case by calculating the convolution of :
* Two normal distributions. We'll conclude that the sum of two independent normally distributed rvs is also normal, with its variance being the sum of the variances of the variables (we already obtained this result by calling on the properties of the moment generating function).
* Two Cauchy distributions. We'll conclude that the distribution of the sample mean is identical to the original Cauchy mother distribution irrespective of the sample size, a most striking result.
* Two Gamma distributions (with the same spread parameter). We'll see that the Gamma distribution enjoys the additivity property. In the special case where the shape parameter is an integer, the result will show that the sum of iid exponential rvs is Gamma distributed.
We also establish this result here by calling on the properties of the moment generating function.
CONVOLUTION OF CONTINUOUS DISTRIBUTIONS
Convolution of two normal distributions
Influence of a translation on the convolution
Distribution of the sum of independent normal variables
Convolution of two Cauchy distributions
Partial fraction decomposition
Distribution of the sample mean
Convolution of two Gamma distributions
Convolution of two Gamma distributions
Additivity property of the Gamma distribution
The sum of iid exponential rvs is Gamma distributed
Related readings :