Stirling's formula

The factorial function

n! = : 1.2.3......(n - 2).(n - 1).n

 is ubiquitous in mathematics, and consequently in probability theory (see for example the binomial, Poisson and Gamma distributions, or the distribution of order statistics).

Unfortunately, n! becomes astronomically large even for small values of n, and finding a convenient means of calculating approximate values of n! even for large values of n has been considered a priority as early as the XVIII^th century. It was then shown  that

This expression is known as Stirling's formula, or "Stirling's approximation".

We demonstrate Stirling's formula in the Tutorial below.

-----

The "~" symbol has a strong meaning. It tells us that the right term of the above expression is not just an approximate value of n!, but also that the approximation gets better and better as one considers larger and larger values of n. More technically, the ratio of the two terms of the expression tends to 1 as n tends to infinity. This asymptotic result makes Stirling's formula not only a practical tool for calculating approximate values of n!, but a powerful theoretical tool as well. For example, this site uses Stirling's formula :

    1) For demonstrating the convergence of Student's t distribution to a standard normal distribution when the sample size grows without limit,

    2) For demonstrating De Moivre theorem,

    3) For establishing a sufficient condition for the convergence of the hypergeometric distribution to a normal distribution.

_____________________

A more advanced demonstration than the one we give would show that

n! = n ⁿe^{- n}(2πn)^1/2 exp(Σ_i a_in^{- i} )          i = 1, 3, 5 ....

where the a_is are themselves known as series of inverse powers of n (so this expression is an exact, if inconvenient, expression for n!).

If we retain only the 1/n term, we have :

n! ~ n ⁿe^{- n}(2πn)^1/2 (1 + 1/12n)

which makes the numerical error on n! negligeable even for small values of n.

___________________________________________________________________________

In this Tutorial, we demonstrate Stirling's formula.

    1) As a little warm-up, we first identify a very simple, but also rather crude approximation for n!.

    2) We then attempt to demonstrate Stirling's formula by resorting to the properties of the Gamma function (which is the generalization of the factorial function to all positive real numbers). We'll obtain Stirling's formula in a few simple and elegant lines, but we'll fail to demonstrate that the formula is asymptotically exact. The reason is that the demonstration is borrowed from a general and powerful but difficult theory named the "saddle point approximation" that succeeds at demonstrating the full result, but that we unfortunately cannot develop in this Glossary.

    3) As the foregoing attempt was too ambitious, we decide to go back to known territory and turn to the familiar Central Limit Theorem (CLT). Indeed, applying the CLT to the exponential distribution will easily take us to Stirling's formula.

But on second thought, we'll realize that we interpreted the CLT as meaning that the probability density function (pdf) of a standardized sum of rvs with a density (as well as a mean and a variance) converges to the standard normal pdf, which the CLT does not say : we emphasize again that the CLT bears on distribution functions, not on pdfs.

So although our proof looks convincing enough, it is in fact severely flawed. It could be straightened out by showing that our extended interpretation of the CLT is correct in this particular case, a difficult task which lies beyond the bounds of this Glossary.

    4) So we'll finally get back to basics, and demonstrate Stirling's formula the old fashioned way, that is by using Wallis formula :

that we first fully demonstrate. This formula is the culminating point of the theory of Wallis integrals, whose basic properties we established when we calculated the variance of Student's t distribution.

Although the demonstration is a bit pedestrian, it is complete and allows us to reach the fundamental conclusion that Stirling's formula is asymptotically exact.

STIRLING'S FORMULA

 ______________________________________________________

Related readings :