Binomial distribution

You're playing Heads and Tails with a coin whose probability to produce a "Head" is p (and therefore the probability to produce a "Tail" is q = 1 - p).

You toss the coin n times, thus producing k "Heads" (and therefore n - k "Tails"). Of course, should you decide to proceed with a second series of n tosses, you would expect to generate a different number of "Heads". Therefore, k is the realization of a random variable that we denote X.

By definition, X has a binomial distribution with parameters n and p, which is denoted B(n, p). This distribution is completely determined by the list of the probabilities to produce k "Heads" in n tosses, for all values of k from 0 to n. We will denote theses probabilities P_n,p{X = k}.

_____________________________________

Basic properties of the binomial distribution

Probability mass function

The probability to obtain k Heads in a series of n tosses is :

where C(n, k) is the number of subsets of k objects in a set of n objects.

C(n, k) = n!/[k!.(n - k)!]

Mean

This is the number of Heads you'll get, on the average, in a series of n tosses.

Variance

Note that the variance is almost 0 if p is close to 0 or else close to 1. In the first case, you'll almost always get a very small number of Heads, and a number of Heads close to N is the other case. In both cases, this number is very stable and the variance is low.

Show that the variance is largest for p = .5.

Moment generating function

Additivity

Let   X₁ and X₂   be two independent binomial variables, respectively with distributions B(n₁ , p) and B(n₂ , p). Then the r.v. X = X₁ + X₂ is also binomial, with X~ B(n₁ + n₂, p).

Binomial distribution and Poisson distribution

For large n and small p, the binomial distribution can be approximated by the simpler Poisson distribution.

Multinomial distribution

The draws from which the binomial distribution is derived have only two possible outcomes ("Heads" or "Tails"). If these draws have more than just two possible outcomes (e.g. if you roll a die instead of tossing a coin), the distribution of the numbers of the various outcomes in a series of n draws is called the multinomial distribution, which is therefore a generalisation of the binomial distribution.

Estimation of the parameters

Estimation by the Method of Moments

We use the binomial distribution for illustrating the fact that the Method of Moments can be used for estimating simultaneously several parameters of a distribution when these parameters are not moments (as is the case, for instance, for the normal distribution).

In the Tutorial below, we use the method of moments for estimating both the parameters n and p of the binomial distribution B(n, p).

Estimation by Maximum Likelihood

The binomial distribution usually leads to cumbersome or even intractable calculations. Yet, we'll show how to estimate n (assuming that p is known) by the method of Maximum Likelihood. The result does not come in a closed form, but as an equation that can be solved by numerical methods.

Some additional results

        * We calculate here the distribution of the number of Heads in the first m tosses in a series of n tosses conditionally to the total number of Heads in the series of n tosses (we do it in the course of identifying a sufficient statistic for the parameter p of the binomial distribution).

        * In the same spirit, we calculate here the expectation of this same number of Heads.

-----

        * We calculate here the distributions of two independent binomial variables with the same parameter p but different sizes conditionally to the value of their sum. The result will be the key to the Fisher-Irwin test, that tests the identity of the values of the parameters of two independent Bernoulli populations.

Calculette

Please Use our Binomial Calculator to calculate the individual and cumulated probabilities you might need.

_____________________________________________________

We establish here the basic properties of the binomial distribution.

We conclude the Tutorial by estimating n in two ways :

    * By the method of moments.

    * By the methods of maximum likelihood.

BINOMIAL DISTRIBUTION : BASIC PROPERTIES

_______________________________________________________________

Related readings: