Statistical nonparametric mapping: Multivariate permutation tests for location, correlation, and regression problems in neuroimaging

Nonparametric statistical inference via permutation testing is on the rise in neuroimaging research. This rise in popularity is likely in response to recent studies that have demonstrated limitations of parametric inference in certain situations. Nonparametric tests have the appeal of requiring fewer assumptions than their parametric counterparts, and are often touted as being more flexible and more useful for small samples. Furthermore, recent studies have demonstrated the robustness of nonparametric methods in situations when parametric inference fails. As a result, many nonstatistical neuroimaging researchers are likely to believe that nonparametric permutation tests are always a “safe choice” because the results do not depend on distributional assumptions and/or large sample approximations. Alas, this commonly held belief is not entirely accurate, given that nonparametric tests still do rely on assumptions and/or approximations for valid statistical inference. When these assumptions are met, nonparametric permutation tests have the potential to produce valid inferential results for the intended hypotheses. However, as I demonstrate, when these assumptions are violated, nonparametric permutation tests can produce invalid and/or misleading results, which have important implications for the use of such methods in neuroimaging research. All hope is not lost though, as recent theoretical developments in nonparametric statistics can improve current implementations of permutation tests in neuroimaging research. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling Data: Types and Structure > Image and Spatial Data.