null result
In science, a null result is a result without the expected content: that is, the proposed result is absent.{{cite journal |first=C. |last=Giunti |title=New ordering principle for the classical statistical analysis of Poisson processes with background |journal=Phys. Rev. D |volume=59 |issue=5 |pages=053001 |year=1999 |doi=10.1103/PhysRevD.59.053001 |arxiv=hep-ph/9808240 |bibcode=1999PhRvD..59e3001G |s2cid=14948954 |display-authors=etal}} It is an experimental outcome which does not show an otherwise expected effect. This does not imply a result of zero or nothing, simply a result that does not support the hypothesis.
In statistical hypothesis testing, a null result occurs when an experimental result is not significantly different from what is to be expected under the null hypothesis; its probability (under the null hypothesis) does not exceed the significance level, i.e., the threshold set prior to testing for rejection of the null hypothesis. The significance level varies, but common choices include 0.10, 0.05, and 0.01.{{Cite book|last1=Casella|first1=George|title=Statistical Inference|last2=Berger|first2=Roger|publisher=Duxbury|year=2002|isbn=0-534-24312-6|edition=2nd|pages=385}} However, a non-significant result does not necessarily mean that an effect is absent.{{Cite journal |last=Lakens |first=Daniël |date=2017 |title=Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses |journal=Social Psychological and Personality Science |language=en |volume=8 |issue=4 |pages=355–362 |doi=10.1177/1948550617697177 |issn=1948-5506 |pmc=5502906 |pmid=28736600}}{{Cite journal |last=Gross |first=Justin H. |date=2015 |title=Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance |url=https://onlinelibrary.wiley.com/doi/10.1111/ajps.12149 |journal=American Journal of Political Science |language=en |volume=59 |issue=3 |pages=775–788 |doi=10.1111/ajps.12149 |issn=0092-5853|url-access=subscription }}{{Cite journal |last1=Hartman |first1=Erin |last2=Hidalgo |first2=F. Daniel |date=2018 |title=An Equivalence Approach to Balance and Placebo Tests |url=https://onlinelibrary.wiley.com/doi/10.1111/ajps.12387 |journal=American Journal of Political Science |language=en |volume=62 |issue=4 |pages=1000–1013 |doi=10.1111/ajps.12387 |issn=0092-5853|hdl=1721.1/126115 |hdl-access=free }}{{Cite journal |last=Rainey |first=Carlisle |date=2014 |title=Arguing for a Negligible Effect |url=https://onlinelibrary.wiley.com/doi/10.1111/ajps.12102 |journal=American Journal of Political Science |language=en |volume=58 |issue=4 |pages=1083–1091 |doi=10.1111/ajps.12102 |issn=0092-5853}}
As an example in physics, the results of the Michelson–Morley experiment were of this type, as it did not detect the expected velocity relative to the postulated luminiferous aether. This experiment's famous failed detection, commonly referred to as the null result, contributed to the development of special relativity. The experiment did appear to measure a non-zero "drift", but the value was far too small to account for the theoretically expected results; it is generally thought to be inside the noise level of the experiment.{{Cite web |url=http://hsc.csu.edu.au/physics/core/space/9_2_4/924net.html#net4 |title=Role of the Michelson-Morley experiments in making determinations about competing theories |access-date=2003-07-17 |archive-url=https://web.archive.org/web/20121107000031/http://hsc.csu.edu.au/physics/core/space/9_2_4/924net.html#net4 |archive-date=2012-11-07 |url-status=dead }}
Publishing bias
Despite similar quality of execution and design,{{cite journal |last1=Easterbrook |first1=P. J. |last2=Berlin |first2=J. A. |last3=Gopalan |first3=R. |last4=Matthews |first4=D. R. |year=1991 |title=Publication bias in clinical research |journal=Lancet |volume=337 |issue=8746 |pages=867–872 |doi=10.1016/0140-6736(91)90201-Y |pmid=1672966 |s2cid=36570135 |doi-access=free}} papers with statistically significant results are three times more likely to be published than those with null results.{{cite journal |last1=Dickersin |first1=K. |author-link=Kay Dickersin |last2=Chan |first2=S. |last3=Chalmers |first3=T. C. |display-authors=etal |year=1987 |title=Publication bias and clinical trials |journal=Controlled Clinical Trials |volume=8 |issue=4 |pages=343–353 |doi=10.1016/0197-2456(87)90155-3 |pmid=3442991}} This unduly motivates researchers to manipulate their practices to ensure statistically significant results, such as by data dredging.{{cite journal |last1=Pearce |first1=J |last2=Derrick |first2=B |date=2019 |title=Preliminary testing: The devil of statistics? |journal=Reinvention: An International Journal of Undergraduate Research |volume=12 |issue=2 |doi=10.31273/reinvention.v12i2.339 |doi-access=free}}
Many factors contribute to publication bias.H. Rothstein, A. J. Sutton and M. Borenstein. (2005). Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley. Chichester, England; Hoboken, NJ.{{Cite journal |last1=Chopra |first1=Felix |last2=Haaland |first2=Ingar |last3=Roth |first3=Christopher |last4=Stegmann |first4=Andreas |date=2023 |title=The Null Result Penalty |url=https://doi.org/10.1093/ej/uead060 |journal=The Economic Journal |volume=134 |issue=657 |pages=193–219 |doi=10.1093/ej/uead060 |issn=0013-0133|url-access=subscription }} For instance, once a scientific finding is well established, it may become newsworthy to publish reliable papers that fail to reject the null hypothesis.{{cite journal |last1=Luijendijk |first1=HJ |last2=Koolman |first2=X |date=May 2012 |title=The incentive to publish negative studies: how beta-blockers and depression got stuck in the publication cycle. |journal=J Clin Epidemiol |volume=65 |issue=5 |pages=488–92 |doi=10.1016/j.jclinepi.2011.06.022 |pmid=22342262}} Most commonly, investigators simply decline to submit results, leading to non-response bias. Investigators may also assume they made a mistake, find that the null result fails to support a known finding, lose interest in the topic, or anticipate that others will be uninterested in the null results.
There are several scientific journals dedicated to the publication of negative or null results, including the following:
- Journal of Negative Results in Biomedicine (defunct)
- Journal of Pharmaceutical Negative Results
- Journal of Unsolved Questions
While it is not exclusively dedicated to publishing negative results, BMC Research Notes also publishes negative results in the form of research or data notes.