COMMENTS ON PUBLISHED ARTICLE
Year : 2019 | Volume
: 41 | Issue : 2 | Page : 198-
Author's response to ‘Multiple Testing and Protection against Type I Error Using P Value Correction: Application in Cross-Sectional Study Designs’
Department of Psychopharmacology, National Institute of Mental Health and Neurosciences, Bangalore, Karnataka, India
Professor and Head, Department of Psychopharmacology, National Institute of Mental Health and Neurosciences, Bangalore - 560 029, Karnataka
|How to cite this article:|
Andrade C. Author's response to ‘Multiple Testing and Protection against Type I Error Using P Value Correction: Application in Cross-Sectional Study Designs’.Indian J Psychol Med 2019;41:198-198
|How to cite this URL:|
Andrade C. Author's response to ‘Multiple Testing and Protection against Type I Error Using P Value Correction: Application in Cross-Sectional Study Designs’. Indian J Psychol Med [serial online] 2019 [cited 2019 Oct 22 ];41:198-198
Available from: http://www.ijpm.info/text.asp?2019/41/2/198/253380
Menon correctly points out that, in univariate analyses, in order to protect against a Type I (false positive) statistical error, the P value for statistical significance may need to be adjusted when many hypotheses are tested; however, no adjustment of the P value is applied when the same variables are studied in a multivariable regression analysis. The situation is, therefore, paradoxical.
Several scenarios need to be considered here. First, in stepwise regressions, the false positive error rate is very high, and in one simulation, 30%–70% of the variables selected in different models were actually “pure noise”; therefore, stepwise regression procedures that use a statistical criterion for variable selection should emphatically be regarded as exploratory.
Second, multivariable regressions commonly seek to examine the effect of one independent variable on the outcome of interest; the other variables in the equation are present to adjust for confounding. In effect, therefore, there is only one hypothesis being tested, and so, there is no inflation of the false positive error rate, and no P value adjustment required.
Third, in multivariable regressions in which several variables are selected based on a priori hypotheses, and entered in the regression equation to determine the extent to which each variable influences the outcome of interest, many hypotheses are being tested. Therefore, the P value for statistical significance does need to be adjusted; that is, that this is not usually done does not alter the fact that it needs to be done.
Finally, in multivariable regressions in which many variables are entered without a priori hypotheses and with a view to explore and identify potential relationships with the outcome of interest, the analysis should emphatically be considered exploratory because it is, in effect, a fishing expedition and one that is most vulnerable to false positive errors.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
|1||Menon V. Multiple testing and protection against Type I error using P value correction: Application in cross-sectional study designs. Indian J Psychol Med 2019;41:197.|
|2||Andrade C. Multiple testing and protection against a type 1 (false positive) error using the Bonferroni and Hochberg corrections. Indian J Psychol Med 2019;41:99-100.|
|3||Norman GR, Streiner DL. Biostatistics: The Bare Essentials, 4th ed. Shelton, Connecticut, USA: People's Medical Publishing House; 2014.|