Poisson distribution

Binomial and normal distribution

The binomial distribution B(n, p) is a truly fundamental distribution, but it is unfortunately inconvenient to use because it involves calculating factorials that are rapidly conducive to handling gigantic numbers. Quite early in the history of Probability Theory, the need was felt for a simpler distribution that could be used as an (approximate) substitute to the binomial distribution for large values of n.

As early as the end of the18^th century, de Moivre, after some long and rather tedious calculations, showed that the (properly scaled) normal distribution was indeed a good and simple approximation to the binomial distribution for large values of n. Today, all we have to do is invoke the Central Limit Theorem (CLT) to be convinced that it is indeed the case since the binomial distribution is the sum of iid random variables (Bernoulli r.v.).

Yet, as the name suggests, the CLT is an asymptotic result that says nothing about how good (or poor) the approximation of a given sum of n iid random variables by the normal distribution is for finite value of n. As it turns out, for small values of p (that is when studying rare events, a problem of practical interest), the B(n, p) distribution is strongly positively skewed, and this asymetry makes for a poor approximation the binomial distribution by the normal distribution even for large values of n.

Fortunately, there is another distribution, also quite simple, that does a much better job at approximating the binomial distribution for small values of p. This distribution, whose properties are detailed below, is the Poisson distribution (lower image of the above illustration).

----

You'll find here an animation allowing a finer comparison between the binomial and the Poisson distributions.

Definition of the Poisson distribution

The Poisson distribution is therefore defined as the limit of the B(n, p) binomial distribution under the following conditions :

    * n tends to infinity,

    * p tends to 0,

while the product np keeps the same constant value λ.

This definition can be generalized to the case where np only converges to a positive limit λ as n grows without limit.

The Poisson distribution is defined over the set of integers (including 0), and depends only on the single positive parameter λ (whereas the binomial distribution depended on the two parameters n and p). So all binomial distributions with a given value λ of the product np can be approximated by the same Poisson(λ) distribution for large values of n, the approximation being all the better that n is larger (and therefore p smaller).

Properties of the Poisson distribution

We'll establish the following results.

Probability mass function

The probability mass function of the Poisson distribution is :

Mean

The mean of the Poisson distribution is :

Variance

The variance of the Poisson distribution is :

Note that the variance is equal to the mean.

Moment generating function

The mgf of the Poisson distribution is :

We'll establish this result both :

    * By direct calculation,

    * And by calculating the limit of the moment generating function of the binomial distribution when n tends to infinity with np tending to a limit λ. The convergence property of the mgf then tells us that this limit is the mgf of the Poisson distribution.

Generating function

We show here that the generating function of the Poisson distribution is

We'll also establish this result by calculating the limit of the generating function of the binomial distribution.

Large values of the parameter

We'll show that the Poisson distribution can be efficiently approximated by a normal distribution for large values of λ (see animation below).

Additivity

Let

    * X₁ ~ Poisson(λ₁),

    * X₂ ~ Poisson(λ₂ ),

    * X₁ and X₂ independent.

Then

Animation

This animation simulates the Poisson distribution as follows :

    1) Observations are drawn repetitively from the exponential distribution Exp(λ) (yellow upper frame of the animation).

    2) The values of these observations are added until the sum exceeds 1.

    3) Suppose that the sum of the first k observations is less than 1, but that the (k + 1)^th observation makes the sum exceed 1. The integer k is then considered as an observation drawn from the Poisson(λ) distribution.

So this simulation seems at first to be resorting to a contorted ploy to simulate the Poisson distribution, but it is in reality quite respectful of the definition given above, as we explain below.

Estimation of the parameter of the Poisson distribution

Sufficient statistic

We show :

    * Here that the sample mean  is a sufficient statistic for the parameter λ of the Poisson distribution,

    * Here that this statistic is in fact not just sufficient, but also minimal sufficient,

    * And here that it is complete.

UMVUE

We calculate here the UMVUE of any analytic function of the parameter λ, with integer powers of λ as a special case. As an even more special case, we'll see that the sample mean  is the UMVUE of the parameter λ.

Another important special case is that of the UMVUE of e^-λ , which is also obtained :

    * Here as an application of the Lehmann-Scheffé theorem,

    * And here as an application of the Corollary of the Lehmann-Scheffé theorem.

Efficient estimation

We show here that the sample mean is an efficient estimator of the parameter λ of the Poisson distribution.

Exponential family

We show here that the Poisson distribution defines an exponential family. This will lead us to another demonstration of the completeness of the sample mean , and of its efficiency as an estimator of λ.

Poisson distribution and exponential distribution

The Poisson distribution is most commonly used to describe the dates of occurrence of rare events that take place randomly but "uniformly" over time. Typical examples are :

* Customers entering a shop.

* Telephone callers requesting a connection.

* Computing tasks requesting CPU or I/O service.

* etc...

The Poisson distribution will then often turn out to accurately describe the distribution of the number of events occuring during a time interval [t, t + Δt].

In such a setting, the time line is partitionned into contiguous slices of duration δt. One then assumes that each slice has a probability p = λδt to witness a unique event.. The number k of events that will occur during the time interval Δt = nδt is therefore binomial B(n, p).

As we reduce δt (and therefore p) and keep Δt constant, n = Δt/δt  increases. As δt tends to 0, the number k of events that will occur during the time period Δt may therefore be perceived as being drawn from the binomial distribution B(n, p) with an infinitely small probability p and an infinitely large number n of "tosses", in such a way that np keeps the same finite value λ. This limit distribution is, by definition, the Poisson distribution with parameter

np = (Δt/δt).(λδt) = λ.Δt

-----

The amount of time between two events is a random variable that, by definition, follows the geometric distribution Geometric(p). As p = λδt tends to 0, we know that the limit distribution of the geometric distribution is the exponential distribution Exp(λ).

So the number of events taking place in [t, t + Δt] may be regarded as being determined :

    * Either by the B(n, p) distribution in the limit of n tending to infinity and p tending to 0 with np remaining equal to a constant λ, and therefore by the Poisson(λΔt) distribution,

    * Or, alternatively, as the (random) number of Exp(λ) distributed time intervals that will fit in [t, t + Δt].

The equivalence of these two points of view justifies the simulation of the Poisson(λ) distribution used by the animation.

_____________

This equivalence is more rigorously developped in the entry on Poisson processes.

Poisson distribution and Gamma distribution

Consider the time interval [0, t], and do as the animation does : keep chaining independent observations drawn from Exp(λ) until the end of the chain finally falls to the right of t.

We (informally) justified the fact that the number of "links" of the chain in ]0, t] (here, 4) is Poisson(λt) distributed.

For a given integer k, consider these two events :

    1) T_k  is no larger than t.

    2) There are at least k events in [0, t].

These two events are clearly identical, and their probabilities are therefore equal.

-----

The distribution of T_k  is the Gamma distribution Γ(k, 1/λ).

So we established that

P{T_k ≤ t} = P{Poisson(λt) ≥ k}

If we reverse back to the more traditional notation for the Gamma distribution (with β = 1/λ and n instead of k), we (informally) established that given the two r.v. :

    * X ~ Γ(n, β),

    * Y ~ Poisson(x/β)

then

a result that is demonstrated here.

Note that the probability for X  to be no larger than x is, by definition, the value of the distribution function F_{n ,β} (x) of Γ(n, β) that is calculated here.

_________________________________________________

We first calculate directly the probability mass function of the Poisson distribution as the limit of the pmf of the binomial distribution for large n and small p with the product np converging to a positive value λ. We then calculate this pmf again by using a method based on the properties of the generating function.

We further guess its mean and variance by and intuitive argument, then calculate these quantities directly, using nothing but the ordinary definitions of the mean and of the variance.

The mean and variance of the Poisson distribution are also established here by calling on the properties of the generating function.

We then calculate the moment generating function of the Poisson distribution. We'll do it two ways :

    * By direct calculation,

    * And by calculating the limit of the moment generating function of the binomial distribution when n tends to infinity with np tending to a limit λ. The convergence property of the mgf then tells us that this limit is the mgf of the Poisson distribution.

We'll then derive again the mean and the variance of the Poisson distribution. This approach will turn out to be easier that the direct calculation.

-----

We finally show that the Poisson distribution tends to a normal distribution as λ becomes large (see animation). We'll do it by showing that the moment generating function of the standardized Poisson distribution converges to the mgf of the standard normal distribution as λ tends to infinity.

BASIC PROPERTIES OF THE POISSON DISTRIBUTION

_____________________________________________________

    1) We here show that the sum of two independent Poisson r.v. is also Poisson, and we calculate its parameter. This important property is called "Additivity", and is also encountered with other classical r.v. (e.g. binomial, Gamma).

The additivity property of the Poisson distribution is also established here by calling on the properties of the generating function.

    2) We also address the reverse problem : how to decompose a Poisson r.v. X into two independent Poisson r.v. X₁ and X₂ in such a way that X₁ + X₂ = X. We show how this can be done by introducing a series of auxillary binomial r.v. Z_n~B(n, p), and calculate the parameters of X₁ and X₂ as functions of p.

    3) We finally show that the joint distribution of a set of Poisson variables conditionally to their sum is a multinomial distribution.

ADDITIVITY OF POISSON R.V.s, SPLITTING A POISSON R.V.

DISTRIBUTION OF POISSON VARIABLES CONDITIONALLY TO THEIR SUM IS MULTINOMIAL

__________________________________________________

Related readings :