post hoc analysis

{{Short description|Statistical analyses that were not specified before the data were seen}}

{{Distinguish|Post hoc theorizing}}

In a scientific study, post hoc analysis (from Latin post hoc, "after this") consists of statistical analyses that were specified after the data were seen.{{Cite web |title=What is the significance and use of post-hoc analysis studies? |url=https://www.cwauthors.com/article/significance-and-use-of-post-hoc-analysis-studies |access-date=2022-12-09 |website=www.cwauthors.com}}{{Cite web |date=2019-11-12 |title=11.8: Post Hoc Tests |url=https://stats.libretexts.org/Bookshelves/Applied_Statistics/Book%3A_An_Introduction_to_Psychological_Statistics_(Foster_et_al.)/11%3A_Analysis_of_Variance/11.08%3A_Post_Hoc_Tests |access-date=2022-12-09 |website=Statistics LibreTexts |language=en}} They are usually used to uncover specific differences between three or more group means when an analysis of variance (ANOVA) test is significant.{{Cite web |title=SAGE Research Methods - The SAGE Encyclopedia of Communication Research Methods |url=https://methods.sagepub.com/reference/the-sage-encyclopedia-of-communication-research-methods/i11008.xml |access-date=2022-12-09 |website=methods.sagepub.com |language=en}} This typically creates a multiple testing problem because each potential analysis is effectively a statistical test. Multiple testing procedures are sometimes used to compensate, but that is often difficult or impossible to do precisely. Post hoc analysis that is conducted and interpreted without adequate consideration of this problem is sometimes called data dredging (p-hacking) by critics because the statistical associations that it finds are often spurious.{{Cite journal |last=Zhang |first=Yiran |last2=Hedo |first2=Rita |last3=Rivera |first3=Anna |last4=Rull |first4=Rudolph |last5=Richardson |first5=Sabrina |last6=Tu |first6=Xin M. |date=2019-08-01 |title=Post hoc power analysis: is it an informative and meaningful analysis? |url=https://gpsych.bmj.com/content/32/4/e100069 |journal=General Psychiatry |language=en |volume=32 |issue=4 |pages=e100069 |doi=10.1136/gpsych-2019-100069 |issn=2517-729X|pmc=6738696 }}

Post hoc analyses are not inherently bad or good;{{cite book |year=2020 |author=American Psychological Association |title=Publication Manual of the American Psychological Association: the Official Guide to APA Style |publisher=American Psychological Association |location=Washington, DC |isbn=978-1433832178 |edition=7th}}{{rp|12–13}} rather, the main requirement for their ethical use is simply that their results not be mispresented as the original hypothesis.{{rp|12–13}} Modern editions of scientific manuals have clarified this point; for example, APA style now specifies that "hypotheses should now be stated in three groupings: preplanned–primary, preplanned–secondary, and exploratory (post hoc). Exploratory hypotheses are allowable, and there should be no pressure to disguise them as if they were preplanned."{{rp|12–13}}