An Alpha-Exhaustive Multiple Testing Procedure

Abstract

A multiple testing procedure can be a single-step procedure such as Bonferroni's method or a stepwise procedure such as Hochberg's stepup method and Hommel's method. It can be an α-exhaustive or α-conservative approach. We develop a single α-exhaustive procedure that can improve power 2-5% over Hochberg's and Hommel's methods in common situations when the test statistics are mutually independent. The method can also be generalized to dependent test statistics. The idea behind our method is to construct the rejection rules using the product of marginal p-values and by controlling the upper bounds of the kth order terms so that α is controlled for any configuration of k null hypotheses. Such upper bounds or critical values are determined progressively from k = 1 towards k = K, the number of null hypotheses in the problem.