Standard error

Let p(x, θ) be a probability distribution whose parameter θ is to be estimated, and let θ^* be an estimator of θ. The most popular measure of the quality of this estimator is its Mean Squared Error (MSE) :

MSE = Var(θ^* ) + Bias(θ^* )²

In the frequent case where θ^* is unbiased, the quality of the estimator is then just measured by its variance Var(θ^*), or by the square-root thereof, its standard deviation.

The problem is that, more often than not, Var(θ^*) depends on θ and/or on other unknown parameters of the distribution, and the mathematical expression for

Var(θ^*) then contains unknow quantities and cannot be calculated. When there exist estimators for these parameters, it is common practice to plug the estimated values for these unknown parameters into the expression of Var(θ^*). The result is then an estimator of Var(θ ), the true variance of θ. The square root of the value of this estimator on the sample is called the Standard Error of the estimate of θ for this sample. It is an estimation of how heavily the measured value of θ ^* (the estimate of θ) depends on the particular sample at hand.

-----

The most commonly encountered Standard Error is that of the estimated mean.

Let p(x) be a probability distribution with mean µ and variance σ², and let {x_i , i = 1, 2, ..., n} be a sample drawn from this distribution. The sample mean

is an unbiased estimator of the distribution mean µ.

For any p(x), the variance of   is equal to σ²/n. Along the lines of the general notation used above, we therefore have

Var(θ ^*) = σ²/n

But the right hand expression contains σ², which is unknown. An (unbiased) estimator of σ² is

Substituting s ²* for σ² in Var(θ ^*) and taking the square root of both sides yields the Standard Error of the Mean (SEM) :

which can be calculated from the sample only, the result being an estimate of the spread of the distribution of the sample mean  around its mean value µ.

For large samples, (n - 1) is about equal to n, and the above expression then reduces to :

SEM = Sample Standard Deviation / Square Root of Sample size

____________________________________________

Related readings :