Conditional expectation

Click on a title to access the corresponding entry of the Glossary

1

Click on a number to access the detailed Table of Contents of the Tutorial

 

 

 

 

PROBABILITIES and RANDOM VARIABLES

 

 

 

 

 

 

Expectation

 

Tutorial

1

            The Law of the Unconscious Statistician
   Discrete case, continuous case.
   Expectation of a linear tranform.
   Expectation of a function of two r.v..
   Expectation of a linear combination.

 

 

 

Conditional expectation

 

Tutorial

1

Examples (discrete explicit, binomial and continuous).
   Expectation of the product of two random variables :
       * General case.
       * Independent variables.
   Necessary and sufficient condition for two r.v.
   to be independent.

 

 

 

Iterated expectation

 

Tutorials

1

Demonstrations of the Theorem of Iterated Expectation.

2

More examples of applications :
       * Mean of the geometric distribution
       * Covariance of two modalities of a multinomial
          distribution

3

Expectations of the sum and of the product of a random    number of iid random variable.

Tutorials

 4

Calculating probabilities by iterated expectation :
      * P{Y < X} and P{X + Y < a}

5

Moment generating function of the sum of a random  number    of iid random variables.
   Mean and variance of this sum.

6

The "Miner's problem".
   M.g.f. of the geometric distribution revisited.

 

 

 

Variance

 

Tutorial

1

Alternative expression of the variance
   Variance of a linear transform of a r.v.
   Variance of a linear combination of r.v.s.
   Estimating a variance by the empirical variance.
   The law of conditional variance :
         * Demonstration.
         * Example : the "broken stick" problem.

 

 

  

Covariance

 

Tutorial

1

Basic properties of covariance : its two fundamental    expression, linearity, relation with independence.

2

Example :Covariance of two modalities of a multinomial    distribution.

Animation

Covariance and correlation coefficient of a bidimensional and     adjustable sample.

 

  

 

Random vectors

 

Tutorial

1

Second form of the covariance matrix of a random vector.
   Expectation of a twice linearly transformed random vector.
   Covariance matrix of a linearly transformed vector.

 

 

 

Covariance matrix

 

Tutorial

1

Covariance matrices and positive semidefinite matrices.
   Singular covariance matrix, degenerate distribution.
   Diagonalization of a covariance matrix.
   Mahalanobis transformation and Mahalanobis distance.

 

 

 

Marginal distribution

 

Tutorial

1

Calculation of the distributions of :
       * The product,
       * and the ratio
   of two independent uniform random variables as the
   marginal distributions of their joint distribution.

Animation

Distributions of :
       * The product,
       * and the ratio
   of two independent uniform random variables.

 

 

 

Independent random variables

 

Tutorial

1

Independence of random variables can also be expressed in terms of :
       * Factored joint probability distribution.
       * Factored joint cumulative distribution function.
       * Factored probabilities.
       * Factored joint moment generating function.
   Independence and expectation.
   Pairwise independence is not independence : a counterexample.

 

 

 

 

Sampling without replacement

 

Tutorial

1

Population mean :
       * The sample mean is an unbiased estimator of the population mean.
       * Variance of the distribution of the sample mean.
   Unbiased estimator of the population variance.
   Unbiased estimation of a proportion. Variance of the estimator.

 

 

 

 

Moment generating function

 

Tutorial

1

Basic properties of the Moment Generating Function.

 

 

 

Convolution

 

Tutorial

1

Convolution and sum of independent rvs :
       * Continuous case.
       * Discrete case (non-negative integer-valued rv).
   Examples : binomial, geometric, negative binomial, Poisson.

Tutorial

2

Convolution of two normal distributions.
   Convolution of two Cauchy distributions.
   Convolution of two Gamma distributions.

 

 

 

 

Generating function

 

Tutorials

1

Examples : Bernoulli, binomial, geometric, Poisson.
   Differentiation of generating functions :
       * Generation of probabilities, uniqueness.
       * Factorial moments, moments. Examples : binomial,
           geometric, Poisson.

2

Generating function of the sum of independent rvs.
   Examples : Bernoulli, binomial, Poisson, geometric (negative    binomial distribution).

Tutorials

3

Generating function of a random sum of rvs.
    Expectation of a random sum.
    Thinning of a Poisson process.

4

Direct : convergence of pmfs implies convergence of gfs.
   Converse : convergence of gfs implies convergence of pmfs.

 

 

 

Central Limit Theorem

 

Tutorial

1

Demonstration assumes that the distribution has a moment generating function.

 

 

 

Transformations and functions of random variables

 

Tutorials

1

Simple examples of transformations :
       * Linear, square, square root, inverse.

2

Univariate transformations (monotone and general).
   Examples.
   The Probability Integral Transformation.

3

Multivariate transformations. Jacobian determinant.
    Multiple integration by change of variable.

Tutorials

 4

Distribution of the sum of random variables :
      * Theory.
      * Examples : exponential, uniform, Cauchy.

5

Ratio of two random variables :
      * Theory.
      * Examples : Student's T, Fisher's F, Cauchy.

 

 

 

Cauchy-Schwarz inequality

 

Tutorial

1

Cauchy-Schwarz inequality for random variables.
   Cauchy-Schwarz inequality for integrable functions.

 

 

  

Jacobian determinant

 

Tutorial

1

Bivariate transformations form first principles.

 

 

  

Jensen's inequality

 

Tutorial

1

Demonstration of Jensen's inequality (finite and continuous).

 

 

 

Kullback-Leibler distance

 

Tutorial

1

Justification of the definition.
   Nonnegativeness. Asymmetry.
   Kullback-Leibler distance and Maximum Likelihood.

2

Special case : normal distributions.

Animation

Two operating modes :
       * Two normal distributions.
       * Two samples.

 

 

 

Quantiles

 

Tutorial

1

QQ-plots :
    * Samples against a reference distribution.
    * Comparing two samples.

2

The mid-distribution function.
   The continuous quantile function.
   The Quantile Inter-Quartile (QIQ) transformation.

Case study

3

The problem.
   The data.
   The solution :
       * Identification of the distributions with QQ-plots.
       * Estimate the parameters of the distributions.
       * Simulating the distributions.

 

 

 

Slutsky's theorem

  

Tutorial

1

If :
       * {Xn} converges in distribution to X,
       * {Yn} converges in probability to a constant c,
    then
       * {Xn + Yn} converges in distribution to X + c,
       * {Xn.Yn} converges in distribution to cX.

Tutorial

2

Importance of the conditions :
       * On the continuity of the transformation function,
       * On the convergence to a constant.
    Sample variance is a consistant estimator.
    Sample mean is asymptotically normal.
    Student's t is asymptotically normal.

 

 

 

Weak Law of Large Numbers

 

Tutorial

1

Markov inequality.
   Chebyshev inequality.
   Weak Law of Large Numbers.
   Counter-example : a failure of the WLLN.
   Generalized Weak Law of Large Numbers.
   Fundamental Theorem of Statistics.
   Estimation of the moments of a distribution.

 

 

 

 

Strong Law of Large Numbers

 

Tutorials

1

Geometric interpretation of dyadic expansions.
    Dyadic intervals. Dyadic functions.
    Probabilitic interpretation of dyadic intervals.

2

The WLLN expressed in terms of dyadic functions.
   Rademacher functions.
   A property of step functions.
   Proof of the WLLN.

Tutorial

3

Normal numbers and non normal numbers.
   Strengthening Chebychev upper bound.
   Borel's normal numbers theorem.
   The Strong Law of Large Numbers for the
   "Heads and Tails" game.

 

 

 

 

 _____________________________________________________________________

 

Downloading the Tutorials

On-line access to the Tutorials

Other Tutorials