Note: This is an automated translation (using DeepL) of the original German article.
Attempt of an overview of the international modeling approaches
As part of the work on GECKO, the federal government also asked whether medium-term modeling of the pandemic process exists in other countries, what parameters are used, and whether the results are used. Niki Popper as a member of GECKO and his group tried to contribute to the answer. An overall view is difficult, the ECDC Modelling Hub is an important approach for international comparison and exchange.
In almost all European countries, models have been created since the beginning of the pandemic, which have meanwhile been extended to scenario modeling (note: for medium-term calculations, one can always only speak of scenarios). In the USA, well-known examples of modeling include Johns Hopkins University or the Center for Communicable Disease Dynamics (CCDD) at the Harvard T.H. Chan School of Public Health. Other countries (Israel, Brazil et al) are also working on models. A holistic overview is not reasonably deliverable ad hoc, as this would require a suitable systematic model review in order not to forget any models and, conversely, to be able to make at least basic statements about method, quality and limitations. This means that based on scientific publications and other information in a structured way, current approaches and their methodology would be evaluated.
In the USA, the Covid-19 Forecast Hub has existed since the beginning of the pandemic (March 2020) to improve the overview and comparability. For medium-term models and scenarios, the Covid-19 Scenario Hub was founded based on this, with more than 50 institutions participating. In the meantime, 15 scenario comparisons have been calculated here, which deal, among other things, with different vaccination behavior, different virus properties, but also different NPI strategies.
Based on this work, an ECDC modeling hub was also started and operated by ECDC together with the Centre for Mathematical Modelling of Infectious Diseases (CMMID) at the London School of Hygiene & Tropical Medicine (LSHTM). The European Scenario Hub is currently not only collecting, comparing and publishing corresponding, medium- and long-term scenarios from many European countries, but also scientifically analyzing the different methodologies.
Self-description ECDC Scenario Hub (for opening): “The Scenario Hub will be updated by collating long-term modeling scenarios for thirty EU/EEA Member States, the UK and Switzerland. Each potential scenario is composed of one or more plausible and policy-relevant variables (which could be, for example, biological parameters such as the rate of waning immunity, or assumed future changes in contact behaviours). Scenarios cover a period of nine to twelve months, including the coming winter period. Instead of just using single projections, the hub combines and compares different modeling projections from different modeling teams across Europe. The hub can thus contribute to increased confidence and robustness of drawn conclusions and insights.”
This step towards international comparability is important, among other things, to prevent possible cherry-picking, i.e. the possibility for decision-makers to select certain scenarios or model approaches according to desired results. For internationally networked quality assurance, standardization and comparability of methodology, there were therefore already before the Covid-19 pandemic corresponding discussions and work, for example by the Society for Medical Decision Making, see especially [1], which should help establish modeling good research practices in health systems research or also for the selection of methodologies e.g. [2]. However, the quality and intensity of research, as well as its use in policy making, continues to vary widely internationally.
What epidemiological parameters are used for this purpose?
Very different parameters and sources are used in the implementation of the Covid-19 models. Again, the quality varies widely between countries. Which epidemiological (and other parameters) are used depends on the policy maker’s question and the capabilities of the particular modeling approach. In order to be able to cover the scenarios desired by policy makers, the models submitted to the European Scenario Hub deal specifically with the following influencing factors or model details:
- Modeling NPI
- Compliance NPI
- Contact Tracing
- Testing
- Vaccine Efficacy Transmission
- Vaccine Efficacy Delay
- Vaccine Hesitancy
- Vaccine Immunity Duration
- Natural Immunity Duration
- Variants
- Hospitalization
- ICUs
- Case Fatality Rate
- Infection Fatality Rate
- Symptomatics/Asymptomatics
- Age Groups
- Importations
- …
- Confidence Interval Method
- Calibration
- Spatial Structure
- …
In order to be able to represent these influencing factors, (minimally) generally accessible data are used. These are available for Cases, Hospitalisations and Deaths from Euro Forecast Hub / JHU (Johns Hopkins University) and ECDC, Vaccinations and Tests from Our World in Data, Vaccination Effectiveness from VIEW-Hub, Variants from ECDC and Response Measures from ECDC, Contact Matrices from CoMix.
In addition, the various countries use current studies and national sources, which usually provide better data quality, to map all the influencing factors mentioned. This cannot be determined in a standardized way (as is also the case in Austria) and varies greatly in quality. As an example, the methodological approaches for the Scenario Hub details can be found in the model descriptions of the international models. Some European countries have a strong advantage over Austria, especially in the area of surveillance and in the collection of data on the status of the health care system itself. Beyond the aspects mentioned above, more extensive data sources, such as sequencing data, wastewater analyses, mobility data, etc., are also used in other countries; in Austria, for example, data from the Austrian Corona Panel Project (ACPP) are also used.
Are concrete measures already being taken to combat the pandemic on the basis of the results of this medium-term modeling?
It is difficult to answer to what extent the results are being used concretely. Assessments such as the risk assessment on Covid-19 of the Robert Koch Institute (RKI), similar to recommendations of the NIG or GECKO in Austria, are based on a variety of information, including sometimes and when needed scenarios from models. In any case, the ECDC has set their use as a goal for the launch of the Scenario Hub. Quote “It will also play a key role in supporting ECDC’s risk analysis, assessment of public health advice and strategic planning.”
Regarding the status quo: so far, 3 scenarios (comparisons) have been implemented at the European Scenario Hub. Currently, however, only a total of 8 groups have participated. The Report on Scenario 1 was published on 9/12. In the first comparison, for example, 2x2 scenarios were calculated, which deal with vaccination effect and immune waning: “Slow summer booster campaign (Summer)” or “Fast autumn booster campaign (Autumn)” as well as “Optimistic slow immune waning (Strong)” or “Pessimistic fast immune waning (Weak)”. Scenarios are then represented as follows for the countries or scenarios:
Example for Comparisons between different countries from https://covid19scenariohub.eu/report1.html
A corresponding standardization of scenario selection is desirable for policy support. (With the Austrian Variant Management Plan (VMP), a first step was taken by defining jointly agreed scenarios). Appropriate comparisons on the various parameters are also being implemented, although due to the long duration of the pandemic, consistent data comparisons are also becoming increasingly difficult in modeling. E.g., in terms of vaccination progress, a second booster dose for the 60+ age group has already been implemented in the Netherlands and Greece:
Example for different system situation in different countries from https://covid19scenariohub.eu/report1.html
The current, third scenario comparison is based on the EU document “COVID-19 - Sustaining EU Preparedness and Response: Looking ahead” and is intended to cover a range of scenarios for the future, long-term (10-year) COVID-19 burden (from a “reduced threat” to “unmanageable winters/new pandemic”), as well as variations in vaccination campaigns and the characteristics of new variants.
In most countries, standardized use of such models is still unlikely to have been implemented, if the ambition is to do so even beyond ECDC. As an example for Germany, for example, the OptimAgent project aims to develop optimized strategies for epidemic control in highly heterogeneous populations in 10 subprojects. “The goal of OptimAgent is to develop a standardized (!) model-based framework to support public health decision-making processes that can be used to evaluate a wide range of infection control interventions.” The project is scheduled to run until 2025. The current situation shows that the use of medium-term scenarios is in its infancy in terms of implementation, quality assurance (e.g., data, international comparison), and documented, replicable use for policy makers (e.g., to develop recommendations and strategies such as those of the Lancet Commission.
Literature
- Pitman R, Fisman D, Zaric GS, Postma M, Kretzschmar M, Edmunds J, Brisson M; ISPOR-SMDM Modeling Good Research Practices Task Force. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Med Decis Making. 2012 Sep-Oct;32(5):712-21. PMID: 22990086. https://doi.org/10.1177/0272989X12454578
- Miksch F, Jahn B, Espinosa KJ, Chhatwal J, Siebert U, Popper N (2019) Why should we apply ABM for decision analysis for infectious diseases?-An example for dengue interventions. PLoS ONE 14(8): e0221564. https://doi.org/10.1371/journal.pone.0221564