Gamma  (distribution)

An important and flexible probability distribution.

The Gamma function

Definition of the Gamma function

It can be shown that the following integral

exists for all positive values of α. It defines a function of α known as the Gamma function which is denoted Γ(α). The Gamma function plays an important role in several areas of physics and mathematics.

Gamma function and factorials

We'll show that the Gamma function verifies the following recursion relation :

If α is an integer n, this relation becomes :

(n - 1)Γ(n - 1) = Γ(n)

Repeated use of this recursion relation then leads to

The Gamma function may thus be regarded as the generalization of the factorial function to all positive real numbers.

Special case : Γ(1/2)

We show here that

Calculations involving the Gamma function are often conducive to recursion chains ending with Γ(1/2) and so, humble as it looks, this result is indispensable in practice.

The Gamma distribution

The "basic" Gamma distribution

From the above definition of the Gamma function, we immediately deduce that

which shows that the integrand can be regarded as a probability density, with α playing the role of a "shape" parameter.

The "full" Gamma distribution

We may add some flexibility to this distribution by introducing a second (positive) parameter β as follows. Suppose that the r.v. T is distributed as the basic Gamma distribution. We then create the new r.v. X defined by :

X = βT      or     T = X / β

What is the distribution of X ?

By reporting to the result about the distribution of a scaled r.v., we see that the probability density function of X is :

which is, by definition, the "full" Gamma distribution. The Gamma distribution will be denoted Γ(α, β).

Note that β may be regarded as a "spread" parameter.

-----

When α is an integer, the Gamma distribution is sometimes called "Erlang's distribution".

Special cases

The definition of the Gamma distribution may look rather artificial, but in fact this distribution appears naturally in several circumstances.

Exponential distribution

Make α = 1 in the Gamma pdf to obtain :

which is the density function of the exponential distribution with parameter β (or λ = 1/β depending on the definition used for the exponential distribution)

So the exponential distribution is just a special case of the Gamma distribution.

Sum of i.i.d. exponential r.v.

We'll show that the sum of n independent r.v. all distributed as Exp(β) is distributed as Γ(n, β).

The pdf of the Gamma distribution then takes the particular form :

It is, by construction, the distribution of the date of the n^th event of a Poisson process.

-----

This result is illustrated here by an interactive animation.    

Chi-square distribution

Let n be any integer, and consider the Gamma distribution with :

    * α = n / 2

    * β = 2

The Gamma distribution now reads :

that we recognize as the Chi-square distribution with n degrees of freedom.

So the Chi-square distribution is just another special case of the Gamma distribution.

Animation

This interactive animation illustrates the Gamma distribution.

Basic properties of the Gamma distribution

We'll establish the following properties of the Gamma distribution.

Mean

Variance

 -----

This result can be written as :

The significance of this expression will appear when we consider the Gamma distribution as belonging to the natural exponential family.

Moments of all orders

In fact, the Γ(α, β) distribution has moments of all orders. We'll show that the moment of order n is :

Mode

We leave it as an exercise to show that :

            1) When α > 1 the mode of the Gamma distribution is given by :

Calculate its height.

            2) When α = 1, the Gamma distribution is a decreasing exponential whose intersect is 1/β. The mode is therefore 0.

Show that for α > 1 and a given β, the height of the mode tends to 1/β  as α tends to 1 from above. In other words, the height of the mode is (right) continuous at α = 1.

            3) When α < 1, the vertical axis is an asymptote of the monotonously decreasing Gamma distribution, which then has no mode.

Moment generating function

 We'll show that the moment generating function of the Gamma distribution is :

Additivity

Let {X_i }be n independent random variables with respective distributions Γ(α_i, β). Then their sum :

X = Σ_i X_i

is distributed as Gamma(α, β) with :

α = Σ_i α_i

Cumulative distribution function

The cdf F(x) of the Gamma distribution has no simple closed form except when α is an integer n. We'll show that then :

This result will be needed :

    * For establishing the "Poisson" nature of the distribution of the number of events in a given time interval of a Poisson process.

    * For establishing the general expression of the distribution of the record values of a distribution with a pdf.

Relationship with the Poisson distribution

From this result, we'll deduce the following relationship between the Gamma and the Poisson distributions :

a somewhat unanticipated result that receives here an intuitive interpretation.

Relationship with the Beta distribution

 Let :

    * X ~ Γ(α, θ)

    * Y ~ Γ(β, θ)

be two independent Gamma distributed r.v. with the same value θ for the second parameter.

It is shown here that :

Sufficient statistics

We identify here :

    * A sufficient statistic for the parameter α of the Gamma distribution Γ(α, β) when the value of β is known,

    * A sufficient statistic for the parameter β when the value of α is known,

    * And a vector sufficient statistic for the pair (α, β) when the values of both α or β are unknown.

-----

We show here that these statistics are in fact not just sufficient, but also minimal sufficient.

Record values of the exponential distribution

We show here that the record values of the exponential distribution are Gamma distributed.

____________________________________________________________________

In this Tutorial, we establish some basic properties of the Gamma distribution.

As just about any calculation about the Gamma distribution involves the recursion relation of the Gamma function, we first demonstrate this relation.

We then show that a general expression can be given for the moments of all orders of the Gamma distribution. From this relation, we'll derive :

    * The mean,

    * The second order moment,

    * And the variance

of the Gamma distribution.

We then calculate the moment generating function of the Gamma distribution. From this mgf, we'll derive again the first and second order moments.

But we'll notice that a general expression can be found by induction for the derivatives of all orders of the mgf of the Gamma distribution. We'll calculate this expression and then use the general properties of the mgf to derive again the moments of all orders of the Gamma distribution.

We then demonstrate the important additivity property of Gamma distributed random variables. As an immediate consequence, will show that the sum of iid exponential rvs is Gamma distributed. In this Tutorial, the result is established by resorting to the properties of the moment generating function, but we establish it again here by calculating the convolution of two Gamma distributions.

We finally calculate the distribution function of the Gamma distribution in the special case where α is an integer. We'll deduce an interesting and somewhat unanticipated relationship between the Gamma distribution and the Poisson distribution.

BASIC PROPERTIES OF THE GAMMA DISTRIBUTION

________________________________________________________

 _________________________________________________

Related readings :