TY - JOUR
T1 - Enhancing power while controlling family-wise error
T2 - An illustration of the issues using electrocortical studies
AU - Yoder, Paul J.
AU - Blackford, Jennifer Urbano
AU - Waller, Niels G.
AU - Kim, Geunyoung
N1 - Funding Information:
The work for this article was conducted while the authors were partially supported by the NICHD core grant HD15052 (to John F. Kennedy Center, Vanderbilt University).
PY - 2004/5
Y1 - 2004/5
N2 - This study examined the relative family-wise error (FWE) rate and statistical power of multivariate permutation tests (MPTs), Bonferroni-adjusted alpha, and uncorrected-alpha tests of significance for bivariate associations. Although there are many previous applications of MPTs, this is the first to apply it to testing bivariate associations. Electrocortical studies were selected as an example class because the sample sizes that are typical of electrocortical studies published in 2001 and 2002 are small and their multiple significance tests are typically nonindependent. Because Bonferroni adjustments assume independent predictors, we expected that MPTs would be more powerful than the Bonferroni adjustment. Results support the following conclusions: (a) failure to control for multiple significance testing results in unacceptable FWE rates, (b) the FWE rate for the MPTs approximated the alpha set for the analyses, and (c) the statistical power advantage that MPTs provide over Bonferroni adjustments is important when using small sample sizes such as those that are typical of recent electrocortical studies.
AB - This study examined the relative family-wise error (FWE) rate and statistical power of multivariate permutation tests (MPTs), Bonferroni-adjusted alpha, and uncorrected-alpha tests of significance for bivariate associations. Although there are many previous applications of MPTs, this is the first to apply it to testing bivariate associations. Electrocortical studies were selected as an example class because the sample sizes that are typical of electrocortical studies published in 2001 and 2002 are small and their multiple significance tests are typically nonindependent. Because Bonferroni adjustments assume independent predictors, we expected that MPTs would be more powerful than the Bonferroni adjustment. Results support the following conclusions: (a) failure to control for multiple significance testing results in unacceptable FWE rates, (b) the FWE rate for the MPTs approximated the alpha set for the analyses, and (c) the statistical power advantage that MPTs provide over Bonferroni adjustments is important when using small sample sizes such as those that are typical of recent electrocortical studies.
UR - http://www.scopus.com/inward/record.url?scp=4644241165&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=4644241165&partnerID=8YFLogxK
U2 - 10.1080/13803390490510040
DO - 10.1080/13803390490510040
M3 - Article
C2 - 15512923
AN - SCOPUS:4644241165
SN - 1380-3395
VL - 26
SP - 320
EP - 331
JO - Journal of Clinical and Experimental Neuropsychology
JF - Journal of Clinical and Experimental Neuropsychology
IS - 3
ER -