2011-12-22
The overwhelming majority of research in the neurosciences employs p-values stemming from tests of statistical significance to decide on the presence or absence of an effect of some treatment variable. Although a continuous variable, the p-value is commonly used to reach a dichotomous decision about the presence of an effect around an arbitrary criterion of 0.05. This analysis strategy is widely used, but has been heavily criticized in the past decades. To counter frequent misinterpretations of p-values, methodologists advocate complementing or replacing p-values with measures of effect size (MES). Many psychological, biological, and medical journals now recommend reporting appropriate MES. One hindrance to the more frequent use of MES may be their scarcity in standard statistical software packages. Also, the arguably most widespread data analysis software in neuroscience, MATLAB, does not provide MES beyond correlation and receiver-operating characteristic analysis. In our article, we review the most common criticisms of significance testing and provide several neuroscience examples where usage of MES conveys insights not amenable through the use of p-values alone. We introduce an open-access MATLAB toolbox providing a wide range of MES to complement the frequently used types of hypothesis tests, such as t-tests and analysis of variance. The accompanying documentation provides calculation formulae, intuitions for interpretation, and example calculations for each measure. The toolbox described in this article is usable without sophisticated statistical knowledge and should be useful to neuroscientists wishing to enhance their repertoire of statistical reporting.
The overwhelming majority of research in the neurosciences employs p-values stemming from tests of statistical significance to decide on the presence or absence of an effect of some treatment variable. Although a continuous variable, the p-value is commonly used to reach a dichotomous decision about the presence of an effect around an arbitrary criterion of 0.05. This analysis strategy is widely used, but has been heavily criticized in the past decades. To counter frequent misinterpretations of p-values, methodologists advocate complementing or replacing p-values with measures of effect size (MES). Many psychological, biological, and medical journals now recommend reporting appropriate MES. One hindrance to the more frequent use of MES may be their scarcity in standard statistical software packages. Also, the arguably most widespread data analysis software in neuroscience, MATLAB, does not provide MES beyond correlation and receiver-operating characteristic analysis. In our article, we review the most common criticisms of significance testing and provide several neuroscience examples where usage of MES conveys insights not amenable through the use of p-values alone. We introduce an open-access MATLAB toolbox providing a wide range of MES to complement the frequently used types of hypothesis tests, such as t-tests and analysis of variance. The accompanying documentation provides calculation formulae, intuitions for interpretation, and example calculations for each measure. The toolbox described in this article is usable without sophisticated statistical knowledge and should be useful to neuroscientists wishing to enhance their repertoire of statistical reporting.
Copyright © BioPsy 2023
Last update: Oct 06, 2023
