Newman-Keuls test

Reminder on ANOVA

Let {E₁, E₂, …, E_k } be k samples generated by independent normal distributions with identical variances, but whose means may be different. ANOVA tests the H₀ hypothesis according to which the means {µ₁, µ₂ , ... , µ_k} of these distributions are in fact identical.

The null hypothesis is therefore H₀ : µ₁ = µ₂ = µ₃ = … = µ_k, against
The alternative hypothesis H₁ : at least one mean is different from the other means.

If H₀ is rejected, the only possible conclusion is : "There is at least one group among the k groups whose mean is significantly different from the other group means at the chosen significance level". No clue is given as to which group(s) might have its (their) mean(s) significantly different from the general mean.

Many "multiple comparisons" tests have been developed for the purpose of analyzing the reasons that made ANOVA reject the null hypothesis. These tests are globally known as « a posteriori » or « post-hoc » tests.

The Newman-Keuls test is one of these tests.

The Newman-Keuls test

The Newman-Keuls test makes pairwise comparisons of group means after ANOVA has rejected the null hypothesis.

Any hastily designed multiple pairwise comparison procedure of group means is prone to reaching "paradoxical" conclusions. Suppose that we have three groups with m₁ < m₂ < m₃ . Then it is quite possible that the procedure declares :

m₁ and m₃ "not significantly different", but
m₁ and m₂ "significantly different".

This does not necessarily mean that the procedure is flawed, as group sizes have to be taken into account is any sensible mean comparison procedure. But the unexperienced reader might raise endless objections to such results.

The Newman-Keuls test is specifically designed to avoid this kind of embarassing situation. More specifically :

If the test has declared means m_i and m_j (m_i < m_j) not significantly different, then
Any pair of means m_l and m_n with m_i m_l m_n m_j will also be found "not significantly different" by the Newman-Keuls test.

The result of the test is as series of pairs of groups, the means in each pair being considered "significantly different" by the test at a chosen a significance level.

The Newman-Keuls statistics, critical values

For any pair of groups, the Newman-Keuls test produces a value « q_observed » of the test statistic. This value is compared to a theoretical critical value found in Newman-Keuls tables.

For each pair of groups, this critical value depends on :

The sizes of the groups,
The difference of the ranks of the means of the compared groups. For example, if the means are found to be in the order m₁ < m₂ < m₄ < m₃, then comparing m₁ to m₃ will use the value K = 4 in the Newman-Keuls table, and if m₂ is compared to m₄, then K = 3 will be selected.
The significance level a of the global test.

So, a new critical value has to be used for each and every new comparison.

If, for a comparison, q_observed is larger than the q_critical as read in the table, then the hypothesis that the means of the corresponding two populations are equal is rejected.

----

If the test fails to reject the hypothesis of equality of a certain pair of means, then it is garanteed that it will also fail to reject this hypothesis for any pair of means in-between the two original means. The test is then usually not conducted for these pairs.

An example

Corn fields are treated with 4 different fertilizers. Fields are randomly selected and dispatched in groups. The fields in each group have been treated with the same fertilizer. After the crop, an ANOVA is conducted for the purpose of figuring out if the average yields of each group are significantly different.

If ANOVA rejects the hypothesis that the yields are equal, it is the natural to ask which group(s) have their average yield(sd) significantly different from all the others.

The total number of possible comparisons is 6 ([4 x 3] / 2), but, possibly, not all of these comparisons will have to be carried out. Suppose that the group means are order ranked as m₂ < m₁ < m₄ < m₃ :

One first compares the extreme group means, i.e. m₂ and m₃ (K = 4).
Suppose that q_observed > q_critical . Then m₂ and m₃ are declared significantly different (at the chosen significance level).
One then compares the means of Group 2 and Group 4 (H₀ : µ₂ = µ₄ / H₁ : µ₂ µ₄). We now have K = 3.
Suppose that m₂ is found to be significantly different from m₄, and then also significantly different from m₁. We now start a new series of comparisons with m₁ as the reference group mean of the series : we project to compare m₁ with m₃ , and then m₁ with m₄.
Suppose that m₁ and m₃ are found to be not significantly different. Then it is known in advance that the pairs (m₁, m₄) and (m₄, m₃) will not exhibit any significant difference either (at the chosen significance level).