Poisson process

The time intervals between two consecutive ticks of a regular clock are, by definition, equal.

Consider now a random clock : after each tick, the clock draws a realization x of a random variable X, and waits x seconds before emitting the next tick. 

If X is chosen to be an exponential r.v. distributed as Exp(λ), the clock is called a Poisson process for reasons that will become clear in a moment.

Animation

This interactive animation illustrates the concept of "Poisson process".

Formal definition of a Poisson process

Counting process

From now on, we'll use the term event instead of "tick".

Following the standard presentation of stochastic processes, we'll consider an infinite sequence {X_n } of independent and identically distributed random variables taking only nonnegative values. These r.v. are called interarrival times.

    * We denote T_n the time of occurrence of the n^th event. It is a random variable.

    We have :

        T₀ = 0

        T_n = X₁ + X₂ + ... + X_n  

    * For a given date t, we denote N(t) the number of events taking place during the time interval ]0, t]. It is a random variable whose definition can be formalized as :

N(t) = max (n ≥ 0 : T_n ≤ t}

that is :

"The index of the last event occuring before date t"

    * We finally denote

{N(t), t ≥ 0}

 the (continuously) infinite collection of random variables N(t) indexed by t, and call the collection a counting process.

Poisson process

If all the X_is are iid Exp(λ), the counting process is called a Poisson process with rate λ, and will be denoted PP(λ).

Choosing the exponential distribution for the X_is may seem rather arbitrary, but we'll see that Poisson processes have unique properties that make them the most widely used model for describing events occuring randomly and one at a time (see below).

Interpretation of the rate λ

The mean µ of the Exp(λ) distribution is equal to 1/λ. Therefore, λ is the average number of events occuring in one time unit. Alternatively, µ = 1/λ is the average time between two consecutive events (see animation).

Distribution of T_n

 As the {X_n }are iid Exp(λ), T_n is distributed as Γ(n, 1/λ) (see here).

Sample path

A convenient graphic representation of a realization of a Poisson (or of any counting process) is its sample path :

All time intervals (T_{i + 1} - T_i ) between consecutive events are realizations of independent Exp(λ) random variables.

-----

A sample path is therefore the representation of a realization of a Poisson process, just as an observation is a realization of an ordinary random variable.

First properties of a Poisson process

Poisson, at last !

The number k of events taking place in the opening time interval ]0, t] is a random variable N(t).

We'll show that :

N(t) is therefore a Poisson distributed r.v., hence justifying the name "Poisson process".

Translation of the origin of time

            * Invariance of a PP(λ) process under a change of the origin of time

                Let s > 0 be a fixed date, and consider events taking place after s.

Denote N_s(t) the number of events taking place in the time interval ]s, s + t].

A new counting process {N_s(t), t ≥ 0} can be defined starting from s (instead of 0 for {N(t), t ≥ 0}).

We'll show that :

In other words, changing the origin of time does not change the PP(λ) nature of a Poisson process.

-----

This seemingly innocuous result is in fact very strong, and can be obtained only because of the unique memoryless property of the exponential distribution.

            * Independence with opening time intervals

                For a given t > s, N_s(t) is a random variable. Consider now a date u < s. N(u) is a random variable.

We'll show that :

So the numbers of events :

    * In an opening time interval,

    * and in another subsequent non overlapping time interval

are independent random variables.

Stationary and independent increments

Given any fixed time interval ]t, t +  Δt], we call the number of events occuring in this time interval an increment of the process. A process increments are said to be stationary if their distributions depend only on the duration Δt and not on the origin t of the time interval.

We'll show that the above two results lead to the following two fundamental properties of Poisson processes :

These two properties are illustrated by the upper and lower images of the following figure :

Covariance of two time intervals sharing the same origin

Increments oveer two non overlapping time intervals are independent, but increments over two overlapping time intervals are definitely not. Unfortunately, calculating the covariance of these rwo r.v. is intractable in the general case.

In the special case of two opening time intervals ]0, t] and ]0, t + s], we'll show that :

Notice that the result does not depend on s. The covariance of increments over two opening time intervals depends therefore only on the length of the shorter interval (and of course on λ).

_________

By virtue of the time translation invariance of a Poisson process, the result extends to any pair of time intervals sharing the same origin. 

Second definition of a Poisson process

Stationarity and independence of increments will lead us to a characteristic property of a Poisson process (that can therefore be used as a second definition of a PP).

We'll show that

It should therefore be emphasized that the single condition N(t) ~ Poisson(λt) is not sufficient for a {N(t), t ≥ 0} counting process to be a Poisson process.

Anti-bunching property and third definition of a Poisson process

The first definition of a Poisson process mentioned only that interarrival times were iid exponential random variables.

The second definition used both the stationarity and independence of increments, and the Poisson distribution of the number of events in an opening time interval.

We now describe another characteristic property of Poisson processes, that can therefore also be used as a (third) definition. Its nature is quite different, and less intuitive than that of that of the first two definitions.

Anti-bunching

The Poisson nature of the distribution of the number of events in an opening time interval is a direct consequence of our choice of the exponential distribution for defining interarrival times : had we chosen another distribution, the number of events in an opening time interval ]0, t] would have had another distribution f(k, t).

If we assume that f(k, t) has a Taylor expansion up to order 1 about 0 (a rather weak assumption), then

f(k, t) = f(k, 0) + f '(k, 0)t + o(t)

where  o(t)/t → 0 with t.

We now consider the probabilities to have either 0, or 1, or more than 1 observation in ]0, t].

        * k = 0

We have

P{N(t) = 0} = f(0, t) = f(0, 0) + f '(0, 0)t + o(t)

We assume that we are dealing with a so-called "pure renewal process", that is, with a 0 probability to observe an event at t = 0. Consequently :

f(0, 0) = 1

        * k = 1

We have

P{N(t) = 1} = f(1, t) = f(1, 0) + f '(1, 0)t + o(t)

The probability to have exactly 1 event in the vanishingly small time interval ]0, t] must tend to 0 with t, and therefore :

f(1, 0) = 0

        * k > 1

We now consider the probability to have more than 1 event in ]0, t].

We have

which is of the order of t : for small values of t, this probability is approximately proportional to t.

    * Special case

Consider the special case where the distribution f(k, t) of the number of events in the opening interval ]0, t] is such that :

f '(0, 0) = - f '(1, 0)

Then

P{N(t) > 1} =  o(t)

which means that it is now exceedingly improbable to observe more than 1 event in the vanishingly small time interval ]0, δt]. We have an "anti-bunching" effect : events are extremely reluctant to come in tightly packed groups (eventhough the interarrival times are independent).

Third definition of a Poisson process

Of course, the first question is : do distributions f(k, t) exist such that  f '(0, 0) = - f '(1, 0) ?

We'll easily verify that the Poisson(λt) distribution does indeed satisfy this condition. So a PP has the anti-bunching property.

But we'll also show that PP(λ) processes are the only pure renewal processes with this property, provided that we add the requirements for stationarity and independence of increments.

-----

Denote λ =  f '(0, 0) = - f '(1, 0). Then :

    * P{N(t) = 0} = 1 - λt + o(t),

    * P{N(t) = 1} = λt + o(t).

We'll show that :

where o(t) represents a quantity such that o(t)/t tends to 0 with t.

The Poisson process as the limit of a Bernoulli process

Bernoulli process

Intuitive understanding of a Poisson process can be helped by the following thought experiment. The time line from 0 to infinity is partitioned into contiguous slices, each one δt long. You walk along the line from t = 0 and at each step (of length δt), you toss a coin that has probability λδt to land on "Heads", which is considered as an "event".

Interarrival times

The number of tosses between two events follows, by definition, the geometric distribution with parameter λδt. By virtue of the memoryless property of the geometric distribution, the numbers of tosses between consecutive events are independent random variables.

As δt is made vanishingly small, we know that the limit distribution of the geometric distribution with parameter λδt is the distribution Exp(λ).

So, in the limit δt → 0, the interarrival times between two consecutive events are iid Exp(λ).

Increments

Let N be an integer. The distribution of the number of events in any N-step segment is the binomial distribution B(N, λδt). As δt is made vanishingly small and the length of the segment kept constant (N.δt = t = constant), we know that the limit distribution of B(N, λδ t) is Poisson(λt). So, in the limit, increments over a time interval t are Poisson(λt) distributed and are therefore stationary.

By the memoryless property of the geometric distribution, increments in non overlapping time intervals are independent.

"Anti-bunching"

By construction, each time slice will either contain 0 or 1 event, but never 2 or more events. This is where the inspiration for the second set of conditions of the third definition of a Poisson process is to be found.

Event times of a Poisson process and the order statistic property

Consider a fixed date t and, out of a multitude of realizations of PP(λ), retain only those realizations with exactly n events in the opening time interval ]0, t]. The dates of occurrence of these events are random variables T₁, T₂, ..., T_n. These variables have a joint probability distribution f(t₁, t₂, ..., t_n ).

We'll show that, conditionally to N(t) = n :

 
Notice that λ does not appear in this result.

But this is just the joint probability density of the order statistics of the uniform distribution in [0, t] for samples of size n.

So, conditionally to N(t) = n, the events in a ]0, t] time interval are uniformly distributed. Of course, because of the time translation invariance of a PP, this is also true for any time interval [s, s + t].

-----

This result may be interpreted in several ways, among which :

    1) Consider a list of dates of events over a certain time period. Assuming that is known for a fact that these events were generated by a PP, then analyzing this data set may dispense with the concept of PP altogether : it is just as correct to think of the data set as having been generated by n independent draws from a uniform distribution over this time period.

    2) In order to simulate the behavior of a PP(λ) over a given time interval ]0, t], repeat the following 2-step procedure :

        * First draw a random integer n from the Poisson(λt) distribution.

        * Then draw n observations form the Uniform(0, t) distribution.

Superposition of Poisson processes

Now, instead of considering a single Poisson process, we'll consider set-ups that will :

    1) Merge several independent Poisson processes into a single counting process, that will turn out to be also a Poisson process.

    2) and in the next section, split a Poisson process into several counting processes according to a procedure that will guarantee that the resulting processes are also a Poisson processes.

For simplicity, we consider only two Poisson processes that are merged into a single counting process. For example, think of :

    *  The traffic on a freeway as being a Poisson process {N₁(t), t ≥ 0} with intensity λ₁.

    *  The traffic on another freeway as being a Poisson process {N₂(t), t ≥ 0} with intensity λ₂.

After the two freeways merge, the (increased) traffic is a new counting process.

We'll show that under the assumption of independence of the two incoming traffics, the resulting traffic is again a Poisson process {N(t), t ≥ 0}, whose intensity λ is the sum of the intensities of the merging processes :

The result generalizes straightforwardly to any number of Poisson processes.

-----

Suppose now that n processes are superposed.

We'll show that in the stream of merged traffics, event #i originates from process k (1 ≤ k ≤ n) with probability p_k given by :

Note that :

    1) The probability for the next event to originate from a given process does not depend at all on which processes the preceding events belonged to.

    2) On the average, the number of events originating from a given process over a given time period is proportional to the intensity of this process, an intuitive result.

Splitting (or "thinning") of a Poisson process

We now consider the reverse set-up : an incoming stream of events described by a Poisson process {N(t), t ≥ 0} is going to be split into two (or more) streams of events. Think of a telephone switchboard that is handling a Poisson stream of incoming calls  : some callers will want to be put through to Sales, while others will want to be put through to Shipping (or other departments).

When we considered the superposition of Poisson processes, there was no need to specify how the processes were merged, are there is really only a single way this can be done. But now, we need to describe how the decision is made to dispatch a new event to either P₁ or P₂.

The basic Bernoulli splitting

We'll consider the simplest "splitting mechanism" : a (biased) coin is tossed, and the event is dispatched to either P₁ or P₂ depending on the outcome of the toss (Bernoulli trial) :

    * If "Heads" turns up (with probability p), the event is dispatched to process P₁,

    * but if "Tails" turns up (with probability q = 1 - p), the event is dispatched to process P₂.

Of course, you don't expect the switchboard operator to dispatch incoming calls randomly (although sometimes one may wonder) : in this example, the Bernoulli randomness is in the nature of the incoming call before the split takes place.

We'll show  that both P₁ and P₂ are Poisson processes, a natural but not totally obvious result. If the respective intensities of P₁ and P₂ are denoted λ₁ and λ₂, then :

which is exactly what one might have anticipated.

The result generalizes straightforwardly to any number of processes obtained by splitting a single process.

-----

We'll show that the two processes obtained by splitting the incoming stream are independent. Recall that this means that in any time interval Δt after splitting, the numbers on events in P₁ and the number of events in P₂ are independent rvs.

Although natural, this result is definitely not as obvious as the previous one. It makes the Bernoulli splitting mechanism a very attractive candidate model for describing splitting counting processes.

This result generalizes to any number of processes after a multiple-way split.

Inhomogenous Bernoulli splitting

In the preceding section, the Bernoulli splitting probability p was assumed to be constant throughout time. What if it varies, and we should therefore write p(t) instead of just p ? It seems that then everything is lost, and that nothing is left of what makes a counting process a Poisson process.

In fact, it is true that the counting processes resulting from the split are not Poisson processes anymore, but they still retain two of the three characteristic properties of a Poisson process :

    * The distribution of the number of events in any opening time interval (0, t] is still a Poisson random variable.

    * The increments are still independent.

What is lost is the stationarity of the increments.

-----

More precisely, the number of events in (0, t] in P₁ was Poisson(λpt) distributed in the case of the homogenous Bernoulli splitting. The simple quantity pt can be written in the complicated vay :

which just emphasizes the fact that all the infinitesimal time slices dt are "ponderated" by the constant "weight" p.

We'll show that the event count of the homogenous splitting case :

has now to be replaced by

with

So the number of events in an opening interval (0, t] is still Poisson distributed, with a pseudo-intensity equal to αλ, with α being the average value of p(t) over the considered time interval.

________________________________________________________________________________

In this Tutorial, we establish that a counting process is a PP(λ) process if and only if :

    * The number of events in an opening time interval ]0, t] is Poisson(λt) distributed.

    * And increments are both stationary and independent.

We'll need to establish first some preliminary results :

    * The number of events taking place in an opening time interval ]0, t] of a PP(λ) is Poisson(λt) distributed.

    * Changing the time origin of a PP(λ) does not change its PP(λ) nature, a fundamental result.

    * Increments are stationary.

    * Increments over an opening time interval and a subsequent non overlapping time interval are independent, a result that we'll then extend to any pair of non overlapping time intervals.

We'll then be in a position to establish the equivalence between the original definition of a PP(λ), and the second definition as given above.

SECOND DEFINITION OF A POISSON PROCESS

________________________________________________

We now show that a counting process is a Poisson process with rate λ if and only if :

    * Increments are stationary and independent.

    * N(0) = 0 and

         - P{N(t) = 0} = 1 - λt + o(t)

         - P{N(t) = 1} = λt + o(t)

         - P{N(t) > 1} = o(t)

This set of conditions will appear then as a third definition of a Poisson process.

-----

Showing that a Poisson process verifies this set of conditions is easy.

Proving the converse is a bit more delicate. We'll show that these conditions lead to a system of recursive differential equations linking P{N(t) = k} to

P{N(t) = k + 1}. We'll integrate this system, and thus discover that N(t) is Poisson(λt) distributed. It will then be an easy matter to conclude that the counting process defined by the above conditions is a PP(λ) process.

ANTI-BUNCHING PROPERTY

THIRD DEFINITION OF A POISSON PROCESS

 _______________________________________

In this Tutorial :

    1) We first calculate the covariance of the numbers of events in two time intervals sharing the same origin.

    2) We then address the important issue of the joint distribution of the arrival times in ]0, t] conditionally to the number n of events in this interval. We'll show that this joint distribution is identical to the joint distribution of the order statistics of the uniform distribution in [0, t] for samples of size n.

-----

Although the following table of contents is short, the Tutorial is not. Both topics, although not really difficult, require some care in explaining the lines of reasoning and calculation.

ADDITIONAL PROPERTIES OF POISSON PROCESSES

 ____________________________________________________________________

In this Tutorial, we first establish that when two independent Poisson processes are superposed (or "merged"), the resulting process is also Poisson, and establish its properties.

We then examine the reverse set-up : the incoming flux of events of a Poisson process is split on the fly towards two branches according to the outcomes of Bernoulli trials. We'll show that the resulting processes are also Poisson processes, whose properties we'll establish.

Finally, we show that even if the parameter of the Bernoulli splitting mechanism varies over time, the outgoing processes, although not Poisson anymore, still have independent Poisson distributed numbers of events in an opening time interval. Only the stationarity of increments is lost by the variations of the Bernoulli parameter.

SUPERPOSITION AND SPLITTING OF POISSON PROCESSES

 ______________________________________________________

Related readings :