Open peer review

{{glossary}}

{{term|Open identities}}

{{defn|Open peer review may be defined as "any scholarly review mechanism providing disclosure of author and referee identities to one another at any point during the peer review or publication process".{{cite journal | vauthors = Ford E | title = Open peer review at four STEM journals: an observational overview | journal = F1000Research | volume = 4 | pages = 6 | date = 2015-07-20 | pmid = 25767695 | pmc = 4350441 | doi = 10.12688/f1000research.6005.2 | doi-access = free }} Then reviewer's identities may or may not be disclosed to the public. This is in contrast to the traditional peer review process where reviewers remain anonymous to anyone but the journal's editors. Authors' names are disclosed during the process in a single-blind organisation of reviews. In the double-blind process, authors' names and reviewers' names all remain anonymous except to the editor.}}

{{term|Open reports}}

{{defn|Open peer review may be defined as making the reviewers' reports public, instead of disclosing them to the article's authors only. This may include publishing the rest of the peer review history, i.e. the authors' replies and editors' recommendations. Most often, this concerns only articles that are accepted for publication, and not those that are rejected.}}

{{term|Open participation}}

{{defn|Open peer review may be defined as allowing self-selected reviewers to comment on an article, rather than (or in addition to) having reviewers who are selected by the editors. This assumes that the text of the article is openly accessible. The self-selected reviewers may or may not be screened for their basic credentials, and they may contribute either short comments or full reviews.}}

{{glossary end}}

In 1999, the open access journal Journal of Medical Internet Research{{cite web|url=http://www.jmir.org/ |title=JMIR Home |publisher=JMIR.org |access-date=4 January 2012}} was launched, which from its inception decided to publish the names of the reviewers at the bottom of each published article. Also in 1999, the British Medical Journal moved to an open peer review system, revealing reviewers' identities to the authors but not the readers, and in 2000, the medical journals in the open access BMC series{{cite web|url=http://www.biomedcentral.com/info/authors/bmcseries |title=BMC series |publisher=Biomedcentral.com |access-date=4 January 2012}} published by BioMed Central, launched using open peer review. As with the BMJ, the reviewers' names are included on the peer review reports. In addition, if the article is published the reports are made available online as part of the "pre-publication history"'.{{citation needed|date=January 2016}}

Several other journals published by the BMJ Group allow optional open peer review,{{cite journal |last=Smith |first=R. |title=Opening up BMJ peer review | journal = BMJ | volume = 318 | issue = 7175 | pages = 4–5 | date = January 1999 | pmid = 9872861 | pmc = 1114535 | doi = 10.1136/bmj.318.7175.4 }} as does PLoS Medicine, published by the Public Library of Science.{{Cite journal|url=http://journals.plos.org/plosmedicine/reviewer_guidelines.php#anonymity |title=PLoS Medicine: A Peer-Reviewed, Open-Access Journal |journal=PLOS Medicine |volume=3 |issue=11 |pages=e442 |doi=10.1371/journal.pmed.0030442 |pmid=17132052 |pmc=1664601 |date=27 March 2009 |access-date=4 January 2012|last1=Mathers |first1=Colin D |last2=Loncar |first2=Dejan |doi-access=free }} The BMJ's Rapid Responses allows ongoing debate and criticism following publication.{{cite journal |last1=Delamothe |first1=T. |last2=Smith |first2=R. | title = Twenty thousand conversations | journal = BMJ | volume = 324 | issue = 7347 | pages = 1171–2 | date = May 2002 | pmid = 12016170 | pmc = 1123149 | doi = 10.1136/bmj.324.7347.1171 }}

In June 2006, Nature launched an experiment in parallel open peer review: some articles that had been submitted to the regular anonymous process were also available online for open, identified public comment. The results were less than encouraging – only 5% of authors agreed to participate in the experiment, and only 54% of those articles received comments.{{cite journal |title=Overview: Nature's peer review trial |date=December 2006 |journal=Nature |doi=10.1038/nature05535 }}{{cite journal |doi=10.1038/444971b |pmid=17183274 |title=Peer review and fraud |journal=Nature |volume=444 |issue=7122 |pages=971–972 |year=2006 |bibcode=2006Natur.444R.971. |s2cid=27163842 |doi-access=free }} The editors have suggested that researchers may have been too busy to take part and were reluctant to make their names public. The knowledge that articles were simultaneously being subjected to anonymous peer review may also have affected the uptake.

In February 2006, the journal Biology Direct was launched by BioMed Central, adding another alternative to the traditional model of peer review. If authors can find three members of the Editorial Board who will each return a report or will themselves solicit an external review, the article will be published. As with Philica, reviewers cannot suppress publication, but in contrast to Philica, no reviews are anonymous and no article is published without being reviewed. Authors have the opportunity to withdraw their article, to revise it in response to the reviews, or to publish it without revision. If the authors proceed with publication of their article despite critical comments, readers can clearly see any negative comments along with the names of the reviewers.{{cite web | url=http://www.biology-direct.com/info/about/ | title=Aims and scope |work=Biology Direct}}{{third-party-inline|date=January 2016}}

In the social sciences, there have been experiments with wiki-style, signed peer reviews, for example in an issue of the Shakespeare Quarterly.{{cite news |url=https://www.nytimes.com/2010/08/24/arts/24peer.html?_r=1&ref=arts |work=The New York Times |title=For Scholars, Web Changes Sacred Rite of Peer Review |first=Patricia |last=Cohen |date=August 23, 2010}}

In 2010, the BMJ began publishing signed reviewer's reports alongside accepted papers, after determining that telling reviewers that their signed reviews might be posted publicly did not significantly affect the quality of the reviews.{{cite journal |last1=van Rooyen |first1=S. |last2=Delamothe |first2=T. |last3=Evans |first3=S. J. |title=Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial | journal = BMJ | volume = 341 | pages = c5729 | date = November 2010 | pmid = 21081600 | pmc = 2982798 | doi = 10.1136/bmj.c5729 }}

In 2011, Peerage of Science, an independent peer review service, was launched with several non-traditional approaches to academic peer review. Most prominently, these include the judging and scoring of the accuracy and justifiability of peer reviews, and concurrent usage of a single peer review round by several participating journals.{{citation needed|date=January 2016}} Peerage of Science went out of business only a few year after it was founded, because it could attract neither enough publishers nor enough reviewers.

Starting in 2013 with the launch of F1000Research, some publishers have combined open peer review with post-publication peer review by using a versioned article system. At F1000Research, articles are published before review, and invited peer review reports (and reviewer names) are published with the article as they come in.{{Cite magazine |author=Jeffrey Marlow |url=https://www.wired.com/2013/07/publish-first-ask-questions-later/ |title=Publish First, Ask Questions Later |magazine=Wired |date=July 23, 2013|access-date=2015-01-13}} Author-revised versions of the article are then linked to the original. A similar post-publication review system with versioned articles is used by Science Open launched in 2014.{{cite web |url=http://blog.scienceopen.com/2014/12/not-so-secret-pppr-recipe/ |title=The recipe for our (not so) secret Post-Publication Peer Review sauce! |author=Elizabeth Allen |date=September 29, 2017 |orig-year=December 8, 2014 |website=ScienceOpen.com |access-date=2015-01-13}}

Also in 2013, researchers from College of Information and Computer Sciences at University of Massachusetts Amherst founded OpenReview website{{cite web | url=https://openreview.net/about | title=OpenReview }} to host anonymized review reports together with articles, which is as of 2023 popular among computer scientists.

In 2014, Life implanted an open peer review system,{{Cite journal |title= Editorial|journal= Life|volume= 4|issue= 2|pages= 225–226|doi= 10.3390/life4020225|pmid= 25370195|pmc= 4187159|year= 2014|last1= Rampelotto|first1= Pabulo|bibcode= 2014Life....4..225R|doi-access= free}} under which the peer-review reports and authors' responses are published as an integral part of the final version of each article.

Since 2016, Synlett is experimenting with closed crowd peer review. The article under review is sent to a pool of 80+ expert reviewers who then collaboratively comment on the manuscript.{{cite web | title=The case for crowd peer review | website=Chemical & Engineering News | date=2018-11-26 | url=https://cen.acs.org/policy/publishing/case-crowd-peer-review/96/i47 | access-date=2020-03-10}}

In an effort to address issues with the reproducibility of research results, some scholars are asking that authors agree to share their raw data as part of the peer review process.{{cite web |title=The PRO Initiative for Open Science |url=https://opennessinitiative.org/ |website=Peer Reviewers' Openness Initiative |access-date=15 September 2018|date=2014-09-13 }} As far back as 1962, for example, a number of psychologists have attempted to obtain raw data sets from other researchers, with mixed results, in order to reanalyze them. A recent attempt resulted in only seven data sets out of fifty requests. The notion of obtaining, let alone requiring, open data as a condition of peer review remains controversial.{{cite journal |last1=Witkowski |first1=Tomasz |author-link1=Tomasz Witkowski |title=A Scientist Pushes Psychology Journals toward Open Data |journal=Skeptical Inquirer |date=2017 |volume=41 |issue=4 |pages=6–7 |url=https://forbiddenpsychology.wordpress.com/wp-content/uploads/2017/07/artykuc582.pdf}} [https://skepticalinquirer.org/2017/07/a-scientist-pushes-psychology-journals-toward-open-data/ Publisher's version]. In 2020 peer review lack of access to raw data led to article retractions in prestigious The New England Journal of Medicine and The Lancet. Many journals now require access to raw data to be included in peer review.[https://www.theguardian.com/world/2020/jun/12/covid-19-studies-based-on-flawed-surgisphere-data-force-medical-journals-to-review-processes Covid-19 studies based on flawed Surgisphere data force medical journals to review processes] The Guardian, 2020

These publishers and journals operate various types of open peer review:

• Publishers:
• BioMed Central
• BMJ Group
• Copernicus Publications
• European Molecular Biology Organization (EMBO)
• Frontiers
• MDPI (authors have the option to publish peer review reports, etc.){{Cite web |title=MDPI {{!}} The Editorial Process |url=https://www.mdpi.com/editorial_process |access-date=2022-03-16 |website=www.mdpi.com}}
• Nature Research
• [https://open-research-europe.ec.europa.eu/ Open Research Europe] (the European Union publishing platform for Horizon Europe and Horizon 2020 projects)
• Journals:
• eLife
• GigaScience{{Cite web|date=2021-09-21|title=Increasing the diversity and depth of the peer review pool through embracing identity|url=https://blog.oup.com/2021/09/increasing-the-diversity-and-depth-of-the-peer-review-pool-through-embracing-identity/|access-date=2022-01-27|website=OUPblog|language=en}}
• PeerJ
• PLOS
• ReScience C
• Semantic Web journal by IOS Press
• WikiJournal
• SciPost

Peer review at The BMJ,{{cite journal | vauthors = Groves T, Loder E | title = Prepublication histories and open peer review at the BMJ | journal = BMJ | volume = 349 | issue = sep03 13 | pages = g5394 | date = September 2014 | pmid = 25186622 | doi = 10.1136/bmj.g5394 | doi-access = free }} BioMed Central,{{cite web | url=http://www.biomedcentral.com/authors/authorfaq/medical | title=What is 'open peer review', as operated by the medical journals in the BMC series? | publisher=BioMed Central | access-date=31 July 2015}} EMBO,{{cite journal | vauthors = Pulverer B | title = Transparency showcases strength of peer review | journal = Nature | volume = 468 | pages = 29–31 | date = November 2010 | issue = 7320 | pmid = 21048742 | doi = 10.1038/468029a | bibcode = 2010Natur.468...29P | doi-access = free }} eLife,{{cite journal | url=https://elifesciences.org/about/peer-review | title=Peer review | journal=eLife | access-date=30 December 2019}} ReScience C, and the Semantic Web journal{{cite journal |last1=Janowicz |first1=Krzysztof |last2=Hitzler |first2=Pascal |title=Open and transparent: the review process of the Semantic Web journal |journal=Learned Publishing |date=January 2012 |volume=25 |issue=1 |pages=48–55 |doi=10.1087/20120107|doi-access=free }} involves posting the entire pre-publication history of the article online, including not only signed reviews of the article, but also its previous versions and in some cases names of handling editors and author responses to the reviewers. Furthermore, the Semantic Web journal publishes reviews of all submissions, including rejected ones, on its website, while eLife plans to publish the reviews not only for published articles, but also for rejected articles.

The European Geosciences Union operates public discussions where open peer review is conducted before suitable articles are accepted for publication in the journal.{{cite web | url=https://www.egu.eu/publications/statement/online-open-access-publishing/ | title=Online + Open Access Publishing | publisher=European Geosciences Union | access-date=30 December 2019}}

Sci, an open access journal which covers all research fields, adapted a post publication public peer-review (P4R) in which it promised authors immediate visibility of their manuscripts on the journal's online platform after a brief and limited check of scientific soundness and proper reporting and against plagiarism and offensive material; the manuscript is rendered open for public review by the entire community.{{Cite journal|last1=Rittman|first1=Martyn|last2=Vazquez|first2=Franck|date=June 2019|title=Sci—An Open Access Journal with Post-Publication Peer Review|journal=Sci|language=en|volume=1|issue=1|pages=1|doi=10.3390/sci1010001|doi-access=free}}{{Cite journal|last1=Jacob|first1=Claus|last2=Rittman|first2=Martyn|last3=Vazquez|first3=Franck|last4=Abdin|first4=Ahmad Yaman|date=June 2019|title=Evolution of Sci's Community-Driven Post-Publication Peer-Review|journal=Sci|language=en|volume=1|issue=1|pages=16|doi=10.3390/sci1010016.v1|doi-access=free}}{{Cite journal|last1=Vazquez|first1=Franck|last2=Lin|first2=Shu-Kun|last3=Jacob|first3=Claus|date=December 2020|title=Changing Sci from Post-Publication Peer-Review to Single-Blind Peer-Review|journal=Sci|language=en|volume=2|issue=4|pages=82|doi=10.3390/sci2040082|doi-access=free}}{{Cite journal|last1=Abdin|first1=Ahmad Yaman|last2=Nasim|first2=Muhammad Jawad|last3=Ney|first3=Yannick|last4=Jacob|first4=Claus|date=March 2021|title=The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)|journal=Publications|language=en|volume=9|issue=1|pages=13|doi=10.3390/publications9010013|doi-access=free}}

In 2021, the authors of nearly half of the articles published by Nature chose to publish the reviewer reports as well. The journal considers this as an encouraging trial of transparent peer review.

Some platforms, including some preprint servers, facilitate open peer review of

preprints.

• Beginning in 2007, the platform SciRatehttps://scirate.com allowed registered users to recommend articles posted on the arXiv preprint server, displaying the number of recommendations or "scites" each current preprint had received.
• Since 2013, the platform OpenReviewhttps://openreview.net/ provides a flexible system for performing open peer review, with various choices about "who has access to what information, and when". This platform is commonly used by computer science conferences.
• In 2017, the platform PREreviewhttps://prereview.org/ was launched to empower diverse and historically excluded communities of researchers (particularly those at the early stages of their careers) to find a voice, train, and engage in open peer review of preprints. Reviewers can review preprints from over 20 preprint servers on the platform.
• In 2019, the preprint server BioRxiv started allowing posting reviews alongside preprints, in addition to allowing comments on preprints. The reviews can come from journals or from platforms such as Review Commons.
• In 2019, Qeios launched a multidisciplinary, open-access scientific publishing platform that allows the open peer review of both preprints and final articles.{{Cite news |last=Coy |first=Peter |date=2022-01-28 |title=Opinion {{!}} How to Disseminate Science Quickly |language=en-US |work=The New York Times |url=https://www.nytimes.com/2022/01/28/opinion/peer-review-research-studies.html |access-date=2022-10-04 |issn=0362-4331}}
• In 2020, in the context of the COVID-19 pandemic, the platform Outbreak Science Rapid PREreview was launched in order to perform rapid open peer review of preprints related to emerging outbreaks. The platform initially worked with preprints from medRxiv, bioRxiv and arXiv.

===Argued===

Open identities have been argued to incite reviewers to be "more tactful and constructive" than they would be if they could remain anonymous, while however allowing authors to accumulate enemies who try to keep their papers from being published or their grant applications from being successful.{{cite journal | title = Pros and cons of open peer review | journal = Nature Neuroscience | volume = 2 | issue = 3 | pages = 197–8 | date = March 1999 | pmid = 10195206 | doi = 10.1038/nature04991 | last1 = Decoursey | first1 = Thomas }}

Open peer review in all its forms has been argued to favour more honest reviewing, and to prevent reviewers from following their individual agendas.{{cite web | url=http://www.elsevier.com/reviewers/what-is-peer-review | title=What is peer review? | publisher=Elsevier | access-date=31 July 2015}}

An article by Lonni Besançon et al. has also argued that open peer review helps evaluate the legitimacy of manuscripts that contain editorial conflict of interests; the authors argue that the COVID-19 pandemic has spurred many publishers to open up their review process, increasing transparency in the process.{{cite journal|last1=Besançon|first1=Lonni|last2=Peiffer-Smadja|first2=Nathan|last3=Segalas|first3=Corentin|last4=Jiang|first4=Haiting|last5=Masuzzo|first5=Paola|last6=Smout|first6=Cooper|last7=Billy|first7=Eric|last8=Deforet|first8=Maxime|last9=Leyrat|first9=Clémence|title=Open Science Saves Lives: Lessons from the COVID-19 Pandemic|journal=BMC Medical Research Methodology|year=2020|volume=21|issue=1|page=117|doi=10.1186/s12874-021-01304-y|pmid=34090351|pmc=8179078 |doi-access=free }}

In an experiment with 56 research articles accepted by the Medical Journal of Australia in 1996–1997, the articles were published online together with the peer reviewers' comments; readers could email their comments and the authors could amend their articles further before print publication.{{cite journal |doi=10.1016/S0140-6736(97)11510-0 |pmid=9708752 |title=The Medical Journal of Australia internet peer-review study |journal=The Lancet |volume=352 |issue=9126 |pages=441–445 |year=1998 |last1=Bingham |first1=Craig M. |last2=Higgins |first2=Gail |last3=Coleman |first3=Ross |last4=Van Der Weyden |first4=Martin B. |s2cid=34493476 }} The investigators concluded that the process had modest benefits for authors, editors and readers.

Some studies have found that open identities lead to an increase in the quality of reviews, while other studies find no significant effect.{{cite journal | last1 = Lee | first1 = Carole J. | last2 = Sugimoto | first2 = Cassidy R. | last3 =Zhang | first3 = Guo | last4 = Cronin | first4 = Blaise | name-list-style = vanc | title = Bias in peer review | journal = Journal of the American Society for Information Science and Technology | date = January 2013 | volume = 64 | issue = 1 | pages = 2–17 | doi = 10.1002/asi.22784 }}

Open peer review at BMJ journals has lent itself to randomized trials to study open identity and open report reviews. These studies did not find that open identities and open reports significantly affected the quality of review or the rate of acceptance of articles for publication, and there was only one reported instance of a conflict between authors and reviewers ("adverse event"). The only significant negative effect of open peer review was "increasing the likelihood of reviewers declining to review".{{cite journal | vauthors = van Rooyen S, Delamothe T, Evans SJ | title = Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial | language = en | journal = BMJ | volume = 341 | pages = c5729 | date = November 2010 | pmid = 21081600 | pmc = 2982798 | doi = 10.1136/bmj.c5729 | url = }}

In some cases, open identities have helped detect reviewers' conflicts of interests.{{cite journal | vauthors = Benos DJ, Bashari E, Chaves JM, Gaggar A, Kapoor N, LaFrance M, Mans R, Mayhew D, McGowan S, Polter A, Qadri Y, Sarfare S, Schultz K, Splittgerber R, Stephenson J, Tower C, Walton RG, Zotov A | title = The ups and downs of peer review | journal = Advances in Physiology Education | volume = 31 | issue = 2 | pages = 145–52 | date = June 2007 | pmid = 17562902 | doi = 10.1152/advan.00104.2006 }}

Open participation has been criticised as being a form of popularity contest in which well known authors are more likely to get their manuscripts reviewed than others.{{cite journal|last1=Besançon|first1=Lonni|last2=Rönnberg|first2=Niklas|last3=Löwgren|first3=Jonas|last4=Tennant|first4=Jonathan P.|last5=Cooper|first5=Matthew|title=Open up: a survey on open and non-anonymized peer reviewing|journal=Research Integrity and Peer Review|volume=5|issue=1|year=2020|page=8|issn=2058-8615|doi=10.1186/s41073-020-00094-z|pmid=32607252|pmc=7318523|doi-access=free}} However, even with this implementation of Open Peer Reviews, both authors and reviewers acknowledged that Open Reviews could lead to a higher quality of reviews, foster collaborations and reduce the "cite-me" effect.

According to a 2020 Nature editorial, experience from Nature Communications negates the concerns that open reports would be less critical, or would require an excessive amount of work from reviewers.

Thanks to published reviewer comments, it is possible to conduct quantitative studies of the peer review process. For example, a 2021 study has found that scrutiny by more reviewers mostly does not correlate with more impactful papers.

• Open peer commentary
• Open research
• Open science
• Open science data
• PubPeer

{{Reflist|refs=

{{cite journal | last1=Soergel | first1=David | last2=Saunders | first2=Adam | last3=McCallum | first3=Andrew | title=Open Scholarship and Peer Review: a Time for Experimentation | website=OpenReview | date=2013-05-14 | url=https://openreview.net/forum?id=xf0zSBd2iufMg | access-date=2023-12-05}}

{{cite journal | last1=Wolfram | first1=Dietmar | last2=Wang | first2=Peiling | last3=Abuzahra | first3=Fuad | title=An exploration of referees' comments published in open peer review journals: The characteristics of review language and the association between review scrutiny and citations | journal=Research Evaluation | publisher=Oxford University Press (OUP) | volume=30 | issue=3 | date=2021-03-13 | issn=0958-2029 | doi=10.1093/reseval/rvab005 | pages=314–322}}

{{cite journal | title=Nature is trialling transparent peer review — the early results are encouraging | journal=Nature | volume=603 | issue=7899 | date=2022-03-01 | issn=0028-0836 | doi=10.1038/d41586-022-00493-w | pages=8| pmid=35233099 | bibcode=2022Natur.603....8. | s2cid=247189806 | doi-access=free }}

{{cite journal | last=Kwon | first=Diana | title=Open-access journal eLife announces 'preprint first' publishing model | journal=Nature | publisher=Springer Science and Business Media LLC | date=2020-12-15 | issn=0028-0836 | doi=10.1038/d41586-020-03541-5 | pmid=33319829 | s2cid=229172479 }}

{{cite journal | last=Ross-Hellauer | first=Tony | title=What is open peer review? A systematic review | journal=F1000Research | publisher=F1000 Research Ltd | volume=6 | date=2017-08-31 | issn=2046-1402 | doi=10.12688/f1000research.11369.2 | page=588| pmid=28580134 | pmc=5437951 | doi-access=free }}

{{cite journal| title=Nature will publish peer review reports as a trial. | journal=Nature | year= 2020 | volume= 578 | issue= 7793 | pages= 8 | pmid=32025024 | doi=10.1038/d41586-020-00309-9 | bibcode=2020Natur.578....8. | doi-access=free }}

{{cite journal | last=Brainard | first=Jeffrey | title=In bid to boost transparency, bioRxiv begins posting peer reviews next to preprints | journal=Science | publisher=American Association for the Advancement of Science (AAAS) | date=2019-10-10 | issn=0036-8075 | doi=10.1126/science.aaz8160 | s2cid=211766434 }}

{{cite journal | last1=Johansson | first1=Michael A. | last2=Saderi | first2=Daniela | title=Open peer-review platform for COVID-19 preprints | journal=Nature | publisher=Springer Science and Business Media LLC | volume=579 | issue=7797 | year=2020 | issn=0028-0836 | doi=10.1038/d41586-020-00613-4 | pages=29| pmid=32127711 | bibcode=2020Natur.579...29J | doi-access=free }}

{{cite journal | last1 = Perkel | first1 = Jeffrey M. | title = Challenge to scientists: does your ten-year-old code still run? | journal = Nature | volume = 584 | pages = 656–658 | date = 2020-08-24 | issue = 7822 | doi = 10.1038/d41586-020-02462-7 | pmid = 32839567 | bibcode = 2020Natur.584..656P | doi-access = free }}

{{cite web | url=https://scipost.org/submissions/refereeing_procedure | title=Refereeing Procedure | publisher=SciPost | access-date=22 August 2021}}

{{cite web | title= Open Peer Review | website= PLOS |year = 2020 | url = https://plos.org/resource/open-peer-review | access-date = 2021-09-02 |archive-url= https://web.archive.org/web/20210902004347/https://plos.org/resource/open-peer-review |archive-date= 2021-09-02 |url-status=live }}

}}

Category:Peer review

Category:Open science

Category:Metascience