Independent and matched samples

When a test takes several samples into account, its description always mentions the fact that these samples are:

    * Either "independent",

    * or else "matched".

These two terms are described here in some detail.

_________________________
 

1) Independent samples

    Several samples are said to be "independent" if they were created by independent draws from one or several populations.

There are many ways to obtain independent samples. Here are a few:

    1) If p(x) is a probability distribution:

        * First draw n₁ observations from  p(x) and put them in Sample 1.

        * Then draw n₂ observations from  p(x) and put them in Sample 2.

Sample 1 and Sample 2 are independent samples.

    2) Consider a table with observations as rows and variables as columns, and let Y be one categorical variable of the table. Place in sample S_i all the observations in the table with modality M_i of the variable Y. All the S_i make up a collection of independent samples. The test will then bear on the values taken by some other variable (for example numerical) X.

The number of observations in S_i is the number of observations in the table with value M_i of the variable Y. The samples may therefore have different numbers of observations.

3) Consider several tables crossing observations and variables. For example, each of these tables could describe a group of patients in a large medical survey. The tables do not need to have the same number of observations or of variables. Yet, suppose that all tables have the "Blood pressure" variable.

The question "Do all these groups have the same average blood pressure ?" will be addressed by an "equality" test (like one-way ANOVA), and the samples will be considered independent.

Tests that need independent samples are used for answering questions related to global properties of the samples. For example:

    * Were these samples drawn from the same distribution ? (Kolmogorov-Smirnov test, Kruskal-Wallis test).

     * Assuming that the samples were drawn from normal populations with equal variances, do these populations have identical means ? (Student's t test for two independent samples, one-way ANOVA for more than two samples).

    *  Assuming that two samples were drawn from two normal populations with unequal (and unknown) variances, do these populations have identical means ? (Aspin-Welch test).

    * Assuming that two samples were drawn from two normal populations, do these populations have identical variances ? (Fisher's F test).

and many more.

_____________

But all questions relative to groups of observations cannot be addressed that way. We now describe the concept of "matched samples".

2) Matched samples

    Suppose that a new drug against high blood pressure is to be tested. The blood pressure of each patient in a group is measured before any treatment, and the numbers constitute Sample S₁.

After the treatment, the blood pressure of each patient is measured again, and these new values will consitute Sample S₂.

The question is, of course, "Did the patients' blood pressure drop significantly as a consequence of the treatment ?". Note that the question:

    * is not: "Are the numbers in S₂ globally lower than the numbers in S₁ ?",

    * but  is: "For each patient, subtract the pressure "After" from the pressure "Before". You get a series a numbers, the individual blood pressure variations. These numbers are expected to be positive. The question is: is the average of these numbers significantly above 0 ?".

This question is very different from the following question:
   " Is the average of S2 significantly lower than the average of S1?".
It could be envisioned that some patients would see their blood pressure drop as a result of the treatment, while some other patients would see their blood pressure increase, with, as a net result, a drop in the average blood pressure of the group. This situation would be clearly unsatisfactory from a clinical point of view, so detecting a global drop in blood pressure is not an interesting objective.

In this kind of situation, the samples are not considered globally, as was the case when asking "Were the samples drawn from the same population ?". Now to each observation (patient) in one sample corresponds one observation in each of the other samples. The test bears on the variations across the samples of a certain quantity for each observation.

The samples are then said to be matched. Tests on matched samples attempt to detect differences in the effects of the various conditions to which individuals were submitted. The active part of the test works not on the raw values of the variable under consideration, but on the results of some operations on the values of this variable on matched samples.

Note that matched samples always have the same number of observations.

 Matched samples are usually obtained by one of the two following procedures:

    1) A group of N individuals is submitted to k different "treatments". The test the tries to detect differences in the "effectiveness" of these treatments.

    2) One first create N groups of k individuals each. Each group is homogenous, that is, contains individuals whose attributes other than the attribute submitted to the test are as similar as possible. For example, in a clinical test, each group would contain individuals of the same sex, age, symptomes and similar medical records.

    * An individual is chosen randomly in each of the N groups, and submitted to treatment 1.

    * In each of the N groups, another individual is selected at random (among the remaining k - 1 individuals), and submitted to treatment 2.

    * etc...

Sample S_i is made of all the individuals that were submitted to treatment i.

Note that in this second method, one does not compare the values of a quantity measured on the same individual across treatments, but instead on different individuals (that were selected to be as similar to each other as possible).
It is often impossible to submit a single individual to several treatments (for example, metal parts that are submitted to a rupture test).
Even when this is possible, it may be meaningless (for example, testing on a student the effectiveness of several learning techniques meant to learn a new concept).

Many tests deal with matched samples. For example:

    * Given several matched samples, were they drawn from the identical populations ? (Friedman test).

    * Given two matched samples drawn from normal populations with equal variances, do these populations also have identical means ? (Student's t test for matched samples). 

 ___________________________

It should be clear that, given several groups of numbers, the question "Do these groups constitute independent samples or matched samples ?" is meaningless. The distinction between the two stems from:

    * The way the data was collected,

    * and what it is that the test is trying to detect (global differences or individual differences).