Random vector
Let {X1, X2 , ..., Xn} be a set of n random variables.
It is often convenient to consider this set as a single object x = {X1, X2 , ..., Xn} that will be called a random vector. This is particularly true when this set is to be submitted to linear transformations, thus making Linear Algebra the natural mathematical tool.
Just as a random variable is described by its probability distribution, a random vector is is described by the joint probability distribution of the n variables that make up the vector.

The above illustration represents the probability distribution of a random vector of dimension 2.
-----
The moments of a random vector are defined just as for a random variable (see below). But because of the "internal structure" of a random vector that a simple random variable is lacking, the definitions are both a bit more complex and richer that those pertaining to random variables. In particular :
* Multiplicative constants must be replaced by constant (not random) matrices, or constant vectors.
* Variances must be replaced by covariance matrices.
By definition, the expectation of a random vector is the vector whose components are the expectations of the r.v. making up the vector :

µ = E[x] is the center of gravity of the joint probability distribution of {X1, X2 , ..., Xn}.
It is immediately verified that the expectation is linear.
* If x and y are two random vectors, and if A and B are two constant matrices :
E[Ax + By] = AE[x] + BE[y]
* Also :
E[AxB] = AE[x]B
which is for vectors what E[aX] = aE[X] is for random variables.
We demonstrate this result in the course of calculating the covariance matrix of a multivariate normal distribution.
* If b is a constant vector :
E[Ax + b] = AE[x] + b
Covariance matrix
The definition of the variance of a random vector is the same as that of a random variable. Denote µ the vector E[X]. Then, by definition :
Var(x) = E[(x - µ)(x - µ)']
Var(x) is a symmetric, positive
(semi-) definite square matrix that is called the covariance
matrix of x, or the covariance matrix of the set of variables
{X1, X2 ,
..., Xn}. It is usually denoted
.
This expression may also be written :
Var(x) = E[xx'] - µµ'
which is for random vectors what :
Var(X) = E[X ²] - µ²
is for random variables.
It is easily shown that if b is a constant vector :
Var(A'X + b) = A'Var(X)A |
which is for random vectors what :
Var(aX + b) = a²Var(X)
is for random variables.
We demonstrate this result in the course of calculating the covariance matrix of a multivariate normal distribution.
If the matrix A' is a unique row vector a', and if the vector b is reduced to a unique component b, then a'X + b is just a random variable, and the "Var" of the left hand term is just the ordinary variance of a r.v.., and not a covariance matrix.
If, in addition, b = 0, the above expression gives the variance of the inner product of a fixed vector and a random vector.
In particular, if a is a unit vector, Var(a'X) is the variance of the projection P of the random vector X on the straight line D defined by the fixed unit vector a.

We therefore have :
|
Var(Projection of X
on the direction defined by a) = a' |
where
is
the covariance matrix of X.
This result is very useful, in particular when studying the multivariate normal distribution and in Discriminant Analysis.
Let :
* x be a random vector mx1:
x = {X1, X2 , ..., Xm }
* y be a random vector nx1:
y = {Y1, Y2 , ..., Yn}
The covariance Cov(x, y) of these two random vectors is defined exactly as is defined the covariance of two random variables :
Cov(x, y) = E[(x1 - µx )(x2 - µy )']
where µx and µy are the mean vectors of x and of y.
Developping this expression shows that Cov(x, y) is a matrix with m rows and n columns whose generic term is :
[Cov(x, y)]ij = Cov(Xi, Yj )
where "Cov" in the right hand side term is the ordinary covariance between two random variables.
-----
The covariance of two random vectors will appear naturally when we partition the covariance matrix of a multivariate normal distribution (see here).
It is easily verified that :
- Transpose
Cov(y, x) = [Cov(x, y)]'
- Variance of a sum or random vectors
If x and y have the same number of components :
Var(x
y) = Var(x) + Var(y)
2.Cov(x, y)
for Cov(x, y) is then a symmetric matrix.
- Orthogonality
If x and y have the same number of components, they are said to be orthogonal if :
x'y = 0
It is then immediately verified that :
Cov(x, y) = 0
and that therefore :
Var(x
y) = Var(x) + Var(y)
____________________________________________________________
Related readings :
|
Want to contribute to this site ? |