Robust nonparametric tests of general linear model coefficients: A comparison of permutation methods and test statistics

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Statistical inference in neuroimaging research often involves testing the significance of regression coefficients in a general linear model. In many applications, the researcher assumes a model of the form Y=α+Xβ+Zγ+ε, where Y is the observed brain signal, and X and Z contain explanatory variables that are thought to be related to the brain signal. The goal is to test the null hypothesis H0:β=0 with the nuisance parameters γ included in the model. Several nonparametric (permutation) methods have been proposed for this problem, and each method uses some variant of the F ratio as the test statistic. However, recent research suggests that the F ratio can produce invalid permutation tests of H0:β=0 when the ε terms are heteroscedastic (i.e., have non-constant variance), which can occur for a variety of reasons. This study compares the classic F test statistic to the robust W (Wald) test statistic using eight different permutation methods. The results reveal that permutation tests using the F ratio can produce accurate results when the errors are homoscedastic, but high false positive rates when the errors are heteroscedastic. In contrast, permutation tests using the W test statistic produced valid results when the errors were homoscedastic, and asymptotically valid results when the errors were heteroscedastic. In the situation with homoscedastic errors, permutation tests using the W statistic showed slightly reduced power compared to the F statistic, but the difference disappeared as the sample size n increased. Consequently, the W test statistic is recommended for robust nonparametric hypothesis tests of regression coefficients in neuroimaging research.

Original languageEnglish (US)
Article number116030
StatePublished - Nov 1 2019

Bibliographical note

Funding Information:
Funded by a Single Semester Leave award from the University of Minnesota and NIH grants 1U01MH108150-01A1 and 1R01MH112583-01A1 .


  • General linear model
  • Neuroimaging
  • Permutation
  • Randomization
  • Robust statistics

PubMed: MeSH publication types

  • Comparative Study
  • Journal Article
  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

Fingerprint Dive into the research topics of 'Robust nonparametric tests of general linear model coefficients: A comparison of permutation methods and test statistics'. Together they form a unique fingerprint.

Cite this