• Per Lytsy, Mikael Hartman, and Ronnie Pingel.
• Department of Public Health and Caring Sciences, University of Uppsala, Uppsala, Sweden.
• Ups. J. Med. Sci. 2022 Jan 1; 127.

BackgroundThe aim was to investigate inferences of statistically significant test results among persons with more or less statistical education and research experience.MethodsA total of 75 doctoral students and 64 statisticians/epidemiologist responded to a web questionnaire about inferences of statistically significant findings. Participants were asked about their education and research experience, and also whether a 'statistically significant' test result (P = 0.024, α-level 0.05) could be inferred as proof or probability statements about the truth or falsehood of the null hypothesis (H0) and the alternative hypothesis (H1).ResultsAlmost all participants reported having a university degree, and among statisticians/epidemiologist, most reported having a university degree in statistics and were working professionally with statistics. Overall, 9.4% of statisticians/epidemiologist and 24.0% of doctoral students responded that the statistically significant finding proved that H0 is not true, and 73.4% of statisticians/epidemiologists and 53.3% of doctoral students responded that the statistically significant finding indicated that H0 is improbable. Corresponding numbers about inferences about the alternative hypothesis (H1) were 12.0% and 6.2% about proving H1 being true and 62.7 and 62.5% for the conclusion that H1 is probable. Correct inferences to both questions, which is that a statistically significant finding cannot be inferred as either proof or a measure of a hypothesis' probability, were given by 10.7% of doctoral students and 12.5% of statisticians/epidemiologists.ConclusionsMisinterpretation of P-values and statistically significant test results persists also among persons who have substantial statistical education and who work professionally with statistics.© 2022 The Author(s). Published by Upsala Medical Society.

8 keywords...

hide…