Closed testing procedure

In statistics, the closed testing procedure is a general method for performing more than one hypothesis test simultaneously.

The closed testing principle

Suppose there are k hypotheses H₁,..., H_k to be tested and the overall type I error rate is α. The closed testing principle allows the rejection of any one of these elementary hypotheses, say H_i, if all possible intersection hypotheses involving H_i can be rejected by using valid local level α tests; the adjusted p-value is the largest among those hypotheses. It controls the family-wise error rate for all the k hypotheses at level α in the strong sense.

Example

Suppose there are three hypotheses H₁,H₂, and H₃ to be tested and the overall type I error rate is 0.05. Then H₁ can be rejected at level α if H₁ ∩ H₂ ∩ H₃, H₁ ∩ H₂, H₁ ∩ H₃ and H₁ can all be rejected using valid tests with level α.

Special cases

The Holm–Bonferroni method is a special case of a closed test procedure for which each intersection null hypothesis is tested using the simple Bonferroni test. As such, it controls the family-wise error rate for all the k hypotheses at level α in the strong sense.

Multiple test procedures developed using the graphical approach for constructing and illustrating multiple test procedures are a subclass of closed testing procedures.

See also

• Multiple comparisons
• Holm–Bonferroni method
• Bonferroni correction