CovidSim

CovidSim is an agent-based model and was based on an earlier influenza model.{{cite journal | vauthors = Panovska-Griffiths J, Kerr CC, Waites W, Stuart RM | date = January 2021 |title=Mathematical modeling as a tool for policy decision making: Applications to the COVID-19 pandemic |journal=Handbook of Statistics|language=en|volume=44|pages=291–326 | pmc = 7857083 |doi=10.1016/bs.host.2020.12.001| isbn = 9780323852005 | doi-access=free}}

The codebase for the model was initially constructed {{circa}} 2005.{{citation| vauthors = Bostock B |title=How 'Professor Lockdown' helped save tens of thousands of lives worldwide — and carried COVID-19 into Downing Street|date=April 25, 2020|url=https://www.businessinsider.com/neil-ferguson-transformed-uk-covid-response-oxford-challenge-imperial-model-2020-4|work=Business Insider}}

For UK Prime Minister Boris Johnson, it was, according to David Adam writing in The Atlantic, "a critical factor in jolting the UK government into changing its policy on the pandemic" and order a nationwide lockdown to limit the spread of the Coronavirus.{{Cite web|title=Report 9 - Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand |url=http://www.imperial.ac.uk/medicine/departments/school-public-health/infectious-disease-epidemiology/mrc-global-infectious-disease-analysis/covid-19/report-9-impact-of-npis-on-covid-19/|access-date=2021-03-10|website=Imperial College London|language=en-GB}}{{cite web | vauthors = Kelland K |title=Sobering coronavirus study prompted Britain to toughen its approach|date=March 17, 2020|url=https://www.reuters.com/article/us-health-coronavirus-britain-research/sobering-coronavirus-study-prompted-britain-to-toughen-its-approach-idUSKBN2141EP|publisher=Reuters}}{{cite web | vauthors = Tufekci Z |title=Don't Believe the COVID-19 Models.That's not what they're for.|date=April 2, 2020|url=https://www.theatlantic.com/technology/archive/2020/04/coronavirus-models-arent-supposed-be-right/609271/|work=The Atlantic}}

In May 2020, a C++ derivative of the code was released to GitHub.{{Citation|title=mrc-ide/covid-sim|date=2021-03-08|url=https://github.com/mrc-ide/covid-sim|publisher=MRC Centre for Global Infectious Disease Analysis|access-date=2021-03-09}}

{{as of|2023|05}}, the current release tag is v0.15.0.{{Cite web|title=Release v0.15.0 · mrc-ide/covid-sim|url=https://github.com/mrc-ide/covid-sim/releases/tag/v0.15.0|access-date=2021-04-05|website=GitHub|language=en}} Additionally, an Anaconda package exists with release v0.8.0{{Cite web|title=Covid Sim :: Anaconda.org|url=https://anaconda.org/covid19/covid-sim|access-date=2021-04-05|website=anaconda.org}}

The software should be distinguished from the ICL's [https://covidsim.org/ COVID-19 Scenario Analysis Tool] (currently Version 4{{Cite web|title=Covid-19 Scenario Analysis|url=https://covidsim.org/v4.20210322/versions.html|access-date=2021-04-05|website=covidsim.org}}), which is hosted under the domain name https://www.covidsim.org, but according to the research documentation is relying on the model combined with a squire model, which is the underlying transmission model in the absence of vaccination.{{Cite web|title=SEIR transmission model of COVID-19|url=https://mrc-ide.github.io/squire/|access-date=2021-04-05|website=mrc-ide.github.io|language=en}}{{Cite web|last=Consulting|first=Bio Nano|title=Covid-19 Scenario Analysis|url=https://covidsim.org/v4.20210322/research.html|access-date=2021-04-05|website=covidsim.org|language=en}} Further details are available under ICL's Report 33.{{Cite web|title=Report 33 - Modelling the allocation and impact of a COVID-19 vaccine|url=http://www.imperial.ac.uk/medicine/departments/school-public-health/infectious-disease-epidemiology/mrc-global-infectious-disease-analysis/covid-19/report-33-vaccine/|access-date=2021-04-05|website=Imperial College London|language=en-GB}}

Note that the mentioned model ships with and is marked with a list of warnings and user information, e.g. no support, stochastic nature/kernel, criticality of input parameters etc.{{Cite web|title=mrc-ide/covid-sim|url=https://github.com/mrc-ide/covid-sim|access-date=2021-04-05|website=GitHub|language=en}}{{Cite web|title=COVID-19 planning tools|url=http://www.imperial.ac.uk/medicine/departments/school-public-health/infectious-disease-epidemiology/mrc-global-infectious-disease-analysis/covid-19/covid-19-planning-tools/|access-date=2021-04-05|website=Imperial College London|language=en-GB}}

American programmer John Carmack said in April 2020 that he worked on the code before it was released to the public, when it was a single 15,000-line C programming language file and "some of the functions looked like they were machine translated from Fortran", but that "it fared a lot better going through the gauntlet of code analysis tools I hit it with than a lot of more modern code".{{cite web | vauthors = Scheuber A, van Elsland SL |title=Codecheck confirms reproducibility of COVID-19 model results | work = Imperial News |date=June 2020 | publisher = Imperial College London |url= https://www.imperial.ac.uk/news/197875/codecheck-confirms-reproducibility-covid-19-model-results/|access-date=2021-03-10 }}{{cite twitter|title=Before the GitHub team started working on the code it was a single 15k line C file that had been worked on for a decade, and some of the functions looked like they were machine translated from Fortran.|user=ID_AA_Carmack|number=1254872369556074496 | vauthors = Carmack J |author-link = John Carmack |date=April 27, 2020}}

New Scientist reported in March 2020 that one group from the New England Complex Systems Institute reviewing the model suggested that it contained "systematic errors".{{Cite web | vauthors = Shen C, Taleb NN, Bar-Yam Y | date = 17 March 2020 |title=Review of Ferguson et al "Impact of non-pharmaceutical interventions..."|url=https://necsi.edu/review-of-ferguson-et-al-impact-of-non-pharmaceutical-interventions|access-date=2021-03-13|website=New England Complex Systems Institute|language=en-US}}{{citation | vauthors = Hamzelou J |title=UK's scientific advice on coronavirus is a cause for concern|date=March 23, 2020|url=https://www.newscientist.com/article/2238186-uks-scientific-advice-on-coronavirus-is-a-cause-for-concern/|work=New Scientist}} British newspaper The Telegraph reported that some software engineers who reviewed the new code called it "totally unreliable" and a "buggy mess".{{cite news| vauthors = Boland H |date=May 16, 2020|title=Coding that led to lockdown was 'totally unreliable' and a 'buggy mess', say experts|url=https://www.telegraph.co.uk/technology/2020/05/16/coding-led-lockdown-totally-unreliable-buggy-mess-say-experts/|access-date=May 22, 2020|website=The Telegraph}}

In the opinion of University of Oxford computer scientist Michael Wooldridge, the code was "developed without the ceremony and rigor" of professional products, which is not untypical for research software and often intended to be not understood by third parties, or to be reused; and "while the extensive criticism about relaxed software engineering practices is perhaps justified, it was not fundamentally flawed".{{Cite web| vauthors = Wooldridge M |title=The Software that Led to the Lockdown|url=https://cacm.acm.org/blogs/blog-cacm/246511-the-software-that-led-to-the-lockdown/fulltext|access-date=2021-03-09|website=cacm.acm.org|language=en}}

An independent review by Codecheck led by Dr Stephen Eglen of the University of Cambridge confirmed that they were able to reproduce the key findings from the response team's report by using the software.{{cite web | url = https://www.miragenews.com/codecheck-confirms-reproducibility-of-covid-19-model-results/ | title = Codecheck confirms reproducibility of COVID-19 model results | date = 2 June 2020 | work = Mirage News | access-date = 6 June 2020 }}

{{cite journal| vauthors = Eglen SJ |date=29 May 2020|title=CODECHECK certificate 2020-010|url=https://zenodo.org/record/3865491|access-date=2020-10-14|location=Geneva, Switzerland|doi=10.5281/zenodo.3865491}} PDF report available.

A June 2020 editorial in Nature declared the original CovidSim codebase met the requirements of scientific reproducibility.

{{cite journal | vauthors = Singh Chawla D | title = Critiqued coronavirus simulation gets thumbs up from code-checking efforts | journal = Nature | volume = 582 | issue = 7812 | pages = 323–324 | date = June 2020 | pmid = 32546864 | doi = 10.1038/d41586-020-01685-y | url = https://media.nature.com/original/magazine-assets/d41586-020-01685-y/d41586-020-01685-y.pdf | access-date = 2020-10-14 | bibcode = 2020Natur.582..323S | s2cid = 219700526 | doi-access = free }}

Further research exists to identify the following three sources of uncertainty in the simulation:{{Citation| vauthors = Winsberg E |title=Computer Simulations in Science |date=2019|url=https://plato.stanford.edu/archives/win2019/entries/simulations-science/|encyclopedia=The Stanford Encyclopedia of Philosophy| veditors = Zalta EN |edition=Winter 2019|publisher=Metaphysics Research Lab, Stanford University|access-date=2021-03-09}}{{Cite journal| vauthors = Leung K, Wu JT |date=February 2021|title=Quantifying the uncertainty of CovidSim |journal=Nature Computational Science|language=en|volume=1|issue=2|pages=98–99|doi=10.1038/s43588-021-00031-0|issn=2662-8457|doi-access=free}} parametric uncertainty, model structure uncertainty and scenario uncertainty:{{Cite journal| vauthors = Edeling W, Arabnejad H, Sinclair R, Suleimenova D, Gopalakrishnan K, Bosak B, Groen D, Mahmood I, Crommelin D, Coveney PV | display-authors = 6 |date=February 2021|title=The impact of uncertainty on predictions of the CovidSim epidemiological code |journal=Nature Computational Science|language=en|volume=1|issue=2|pages=128–135|doi=10.1038/s43588-021-00028-9 |doi-access=free|hdl=11245.1/4bf2b164-6e12-40aa-93f0-54e41abf1d42|hdl-access=free}} The simulation output depends critically on the inputs and can change up to 300% based on 940 parameters, of which 19 are considered most sensitive. Model structure and scenario uncertainty must therefore be understood.

The results obtained by Imperial using the model are consistent with other models that make similar assumptions.

Calibration of the model has been hampered by the lack of testing, especially the poor understanding of the prevalence of asymptomatic infection, however the Imperial College team makes reasonable assumptions{{Citation needed|reason=This is a statement about the validity of assumptions, but there is no indication about who concurs with the assumptions, or whether there is any contention or consensus on the matter. This is somewhat suspect, as the article on Neil Ferguson quotes Hendrik Streeck, "the authors assume, for example, that 50 percent of households in which there is a case do not comply with the voluntary quarantine. Where does such an assumption come from? I think we should establish more facts.". That suggests there is not consensus on the matter, and therefore it is unsafe to assert that the assumptions are reasonable.|date=May 2024}}. The model's reliance on a simplified picture of social interactions limits its extensibility to counterfactuals. The general nature of conclusions based on such a model can be expected to be similar to those of a simple compartmental model.

• Additional research is based on the model, e.g. for simulation of effect of school closures on mortality.{{cite journal | vauthors = Rice K, Wynne B, Martin V, Ackland GJ | title = Effect of school closures on mortality from coronavirus disease 2019: old and new predictions | journal = BMJ | volume = 371 | pages = m3588 | date = October 2020 | pmid = 33028597 | pmc = 7536648 | doi = 10.1136/bmj.m3588 | url = }}
• Wouter Edeling et al. contributed a FabSim3 plug-in called FabCovidSim,{{Cite web|title=An automation toolkit for complex simulation tasks — FabSim3 Initial Release documentation|url=https://fabsim3.readthedocs.io/en/latest/|access-date=2021-04-02|website=fabsim3.readthedocs.io|language=en}}{{Citation|last1=Wouter Edeling|title=FabCovidSim|date=2021-01-16|url=https://zenodo.org/record/4445290|access-date=2021-04-02|last2=Hamid Arabnejad|last3=Robert C Sinclair|last4=Diana Suleimenova|last5=Krishnakumar Gopalakrishnan|last6=Bartosz Bosak|last7=Derek Groen|last8=Imran Mahmood Qureshi Hashmi|last9=Daan Crommelin|doi=10.5281/zenodo.4445290}} which is based on EasyVVUQ, a Python 3 library to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations.{{Cite journal|last1=Richardson|first1=Robin A.|last2=Wright|first2=David W.|last3=Edeling|first3=Wouter|last4=Jancauskas|first4=Vytautas|last5=Lakhlili|first5=Jalal|last6=Coveney|first6=Peter V.|date=2020-04-29|title=EasyVVUQ: A Library for Verification, Validation and Uncertainty Quantification in High Performance Computing|journal=Journal of Open Research Software|language=en|volume=8|pages=11|doi=10.5334/jors.303|issn=2049-9647|doi-access=free}}{{Cite web|title=wedeling/EasyVVUQ|url=https://github.com/wedeling/EasyVVUQ|access-date=2021-04-02|website=GitHub|language=en}}
• In a recent publication on MedRxiv, which was now accepted by BMJ Open by Laydon et al., the authors utilize the model to "Measure the effects of the Tier system on the COVID-19 pandemic in the UK between the first and second national lockdowns, before the emergence of the B.1.1.7 variant of concern" and conclude that "...interventions at least as stringent as Tier 3 are required to suppress transmission, especially considering more transmissible variants, at least until effective vaccination is widespread or much greater population immunity has amassed."{{Cite medRxiv |last1=Laydon |first1=Daniel J. |last2=Mishra |first2=Swapnil |last3=Hinsley |first3=Wes R. |last4=Samartsidis |first4=Pantelis |last5=Flaxman |first5=Seth |last6=Gandy |first6=Axel |last7=Ferguson |first7=Neil M. |last8=Bhatt |first8=Samir |date=2021-02-24 |title=Impact of the Tier system on SARS-CoV-2 transmission in the UK between the first and second national lockdowns |language=en |medrxiv=10.1101/2021.02.23.21252277v1}}

• List of COVID-19 simulation models
• MRC Centre for Global Infectious Disease Analysis

{{Reflist}}

{{refbegin}}

• {{cite journal | vauthors = Schneider KA, Ngwa GA, Schwehm M, Eichner L, Eichner M | title = The COVID-19 pandemic preparedness simulation tool: CovidSIM | journal = BMC Infectious Diseases | volume = 20 | issue = 1 | pages = 859 | date = November 2020 | pmid = 33213360 | pmc = 7675392 | doi = 10.1186/s12879-020-05566-7 | doi-access = free }}

{{refend}}

• {{Github|mrc-ide/covid-sim}}
• [https://github.com/codecheckers/covid-report9 CodeCheckers covid-report9] on GitHub

Category:COVID-19 models

Category:Agent-based model