Effectiveness of BNT162b2, mRNA-1273, and ChAdOx1-S vaccines against severe covid-19 outcomes in a nationwide mass vaccination setting: cohort study

Data source

This cohort study used data from the French National Health Data System (Système National des Données de Santé (SNDS), formerly known as SNIIRAM)).59 SNDS covers the entire population of France (67 million residents). Each person is identified anonymously by a unique, lifelong number. Since 2006, SNDS has recorded all reimbursement data for outpatient care including drugs, imaging, and laboratory tests; inpatient care (including diagnoses and procedures performed) from the national hospital discharge database (Programme de Médicalisation des Systèmes d’Information (PMSI)); and health expenditure for patients with long term diseases, such as cancer and diabetes, which is fully reimbursed in France. The SNDS has been extensively used to conduct real life pharmacoepidemiological studies, including those on the covid-19 pandemic.60–72 The SNDS also contains sociodemographic data and, when applicable, the date of death.

Information on hospital stays is routinely collected monthly in the PMSI. In April 2020, the French government encouraged hospitals to report all hospital stays related to covid-19 once a week or once a fortnight until July 2020 and then monthly through a fast track procedure (fast track PMSI). Our study was based on the fast track PMSI database available as of 12 October 2021. A cut-off discharge date of 20 August 2021 was chosen to ensure completeness of data over the study period.

Definitions of all variables used in this study were based on the International Classification of Diseases, 10th revision (ICD-10) codes for primary and secondary diagnosis; Common Classification of Medical Procedures codes for procedures; and Anatomical Therapeutic Chemical, French medication ID (code identifiant de presentation, or common unit of dispensation unité commune de dispensation) codes for drugs. We used algorithms developed by national health insurance employees in the Diseases and Healthcare Expenditure Mapping.71 73 These algorithms are detailed in online supplemental table 1.

Information on covid-19 vaccination status (ie, vaccine products and dates of first and second injections) was derived from the national information system on covid-19 vaccines VAC-SI (Système d’Information Vaccin covid-19) database. Information on SARS-CoV-2 test results (by reverse transcription polymerase chain reaction, antigen, or antibody) was derived from the national information system on SARS-CoV-2 tests (the Système d’Informations de DEPistage database). We linked all the extracted data required for this study to the SNDS using individuals’ unique and anonymous identifiers.

Study design and population

We conducted a matched cohort study. Participants were eligible if they received at least one healthcare reimbursement between 2018 and 2020 (reflecting recent healthcare use before the vaccine rollout) and were aged ≥50 years. All newly vaccinated people were matched on a 1:1 ratio to randomly selected unvaccinated controls. Matching criteria included age, sex, and region of residence, and residency in a nursing home for those individuals aged ≥75 years (three categories: no, yes without inpatient pharmacy, and yes with inpatient pharmacy). Newly vaccinated people were eligible for inclusion, even if they had previously been selected as a control. All included individuals were followed up from the date of their inclusion (as defined by the vaccination date of the vaccinated individual, both for the vaccinated and their matched counterpart in each pair) until the end of follow-up. Follow-up of the pair ended at the earliest instance of the following events: occurrence of an outcome of interest, death, vaccination of the unvaccinated control, or the end of the study period (20 August 2021). From this cohort, we separately examined the three subcohorts, BNT162b2 mRNA, mRNA-1273, and ChAdOx1-S vaccine groups, and their respective control group. Individuals who received two doses of vaccine within an interval of <15 days were excluded along with their matched controls.

Outcomes of interest

The primary outcome was covid-19 related hospital admission, defined on the basis of one of the following principal or secondary diagnosis discharge codes derived from the ICD-10 codes: U07.10 (covid-19, respiratory form, virus identified), U07.11 (covid-19, respiratory form, virus not identified), U07.14 (covid-19, other clinical forms, virus identified), U07.15 (covid-19, other clinical forms, virus not identified), U04.9 (severe acute respiratory syndrome, unspecified). The secondary outcome was death in hospital from covid-19.

Covariables

In addition to variables used for matching (ie, age, sex, region of residence, and nursing home status), information was made available on the following characteristics:

• Social deprivation index categorised by quintiles as a marker of socioeconomic status based on the residence area’s median household income, percentage of high school graduates in the population aged ≥15 years, percentage of manual workers in the labour force, and unemployment in the individual’s city of residence

• History of positive covid-19 status based on history of covid-19 related hospitalisation or positive SARS-CoV-2 infection test before inclusion date

• History of influenza vaccination in 2018 or 2019

• Presence of health conditions that were previously shown to be associated with severe covid-19 outcomes.71

These considered health conditions were: frailty, alcohol related conditions, cardiovascular risk factors (that is, smoking related conditions, hypertension, diabetes mellitus, dyslipidaemia, and obesity related conditions), cardiovascular diseases (that is, coronary heart disease, heart failure, cardiac rhythm disorder, valvular heart disease, occlusive peripheral arterial disease, stroke, and pulmonary embolism), chronic respiratory conditions, dialysis, having a transplanted kidney, liver failure, active cancer, depression, psychosis, dementia, epilepsy, Parkinson’s disease, inflammatory bowel disease, rheumatoid arthritis, and ankylosing spondylitis.

Statistical analysis

The categorical variables are reported as frequencies with percentages and the continuous variables reported as means with standard deviations. To report the balance in each individual covariable between vaccine and control groups, the difference in proportions for categorical variables and means for continuous variables is standardised (standardised means difference).74 The imbalance between the groups is defined as an absolute value greater than 0.10.74

We conducted Cox proportional hazards models that were systematically adjusted for matching variables to compare the incidence of outcomes of interest between the vaccinated and control groups. We ran two types of models: the first model adjusted for matching variables, and the second was model 1 but with further adjustment for all the baseline characteristics described in the Covariables section above. For all models, vaccine effectiveness was calculated as 1−hazard ratio (HR)×100; with the lower and upper limits of the confidence interval calculated as (1−HR_Upper)×100 and(1−HR_Lower)×100, respectively.75

We considered seven follow-up time intervals to estimate the risk reduction over time: four after the receipt of the first dose (≥0, 0-6, 7-13, and 14-28 days for the BNT162b2 and mRNA-1273 vaccines or 14-84 days for the ChAdOx1-S vaccine) and three after the receipt of the second dose (≥0, ≥7, and ≥14 days). The primary outcome measure was vaccine effectiveness estimated ≥14 days after the second dose.

We conducted two sets of complementary analyses. Firstly, to study the duration of vaccine protection against the studied outcomes, we estimated vaccine effectiveness by monthly intervals starting 14 days after the second dose. Secondly, to examine vaccine protection at the time of the circulation of the delta variant (B.1.617.2) of SARS-CoV-2, we estimated vaccine effectiveness between 1 July and 20 August 2021. In France, the proportion of the delta variant increased from 55.4% on 1 July 2021 to 80.5% 10 days later; on 20 August, delta represented 95.2% of all monitored variants (online supplemental figure 1).

Subgroup analyses were undertaken by sex and age groups (age 50-64, 65-74, 75-84, ≥85 years) among vaccinated individuals (≥14 days after the second dose) and their unvaccinated controls. We used SAS Enterprise Guide version 4.3 software (SAS Institute, Cary, NC) for all our analyses.

Patient and public involvement

Patients and the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Principal findings

We did a cohort study in a nationwide mass vaccination setting of more than 11 million vaccinated people aged ≥50 years, matched with their unvaccinated controls. We found that all three vaccines (BNT162b2, mRNA-1273, and ChAdOx1 nCoV-19) decreased the risk of covid-19 related hospital admission by ≥90% after individuals received the two doses (estimated vaccine effectiveness 91% for BNT162b2, 95% for mRNA-1273, 91% for ChAdOx1 nCoV-19). We observed similar results when effectiveness was examined against covid-19 related death in hospital. Our results also suggest that the level of vaccine effectiveness remained higher than 90% at 5-6 months following at least 14 days after the receipt of the second dose for the mRNA vaccines (BNT162b2, mRNA-1273) and at 3-4 months for the ChAdOx1-S vaccine. Additionally, the effectiveness of the three vaccines appeared to be maintained against the delta variant. Furthermore, while mRNA vaccines appeared consistently effective across all age subgroups, ChAdOx1-S vaccine effectiveness tended to decrease in people aged ≥65 years compared with those aged 50-64 years.

Comparison with other studies

Our estimates of the two dose vaccine effectiveness of the mRNA vaccines against covid-19 related hospital admission or covid-19 related death in hospital are consistent with the 81.0% to 97.4% estimates reported in previously published studies based on different designs (test negative, case-control studies and cohort studies).7 9 22 25–27 29 31 34 35 42 44–54 Although estimates of the two dose vaccine effectiveness of ChAdOx1-S are less abundant, they were also similar to our results, ranging from 81.5% to 97.5%.42 43 46 48 50 52 55–58 Two reasons might explain the lack or delayed publication of results on the two dose vaccine effectiveness of the ChAdOx1-S vaccine. It had less worldwide use than the other vaccines and its interdose interval was long (≥12 weeks in the UK, where ChAdOx1-S was mainly used) compared with that of the mRNA vaccines (four weeks). These reasons might be due to restrictions in the ChAdOx1-S vaccine supply and the temporary loss of trust in this vaccine following reports of blood clots and one death after its use in March 2021.76 We found that the effectiveness of two doses of the ChAdOx1-S vaccine was similar to that of the mRNA vaccines. This finding, based on more than 3 million people vaccinated with the ChAdOx1 nCoV-19 vaccine, is reassuring.

Several studies have shown that the effectiveness of mRNA vaccines to protect against SARS-CoV-2 infection waned with time77 (eg, effectiveness fell from 88% (within the first month after full vaccination) to 47% (after about five months),25 78% to 17%,28 82% to 33%,45 84% to 69%,46 92% to 47%,48 88% to 68%,49 90% to 70%,51 and 94% to 80%53). However, these vaccines maintained better effectiveness against severe covid-19 than against SARS-CoV-2 infection. Two dose vaccine effectiveness varied from 87% (within the first month after full vaccination) to 88% (after five months),25 96.0% to 88.9%,28 96% to 80%,45 98.6% to 93.8%,46 89% to 64%,48 94% to 92%,49 92% to 81%,51 and 98% to 80%.53 We also found that the effectiveness of two doses of mRNA vaccines against severe covid-19 remained at a high level, from 90% for BNT162b2 and 92% for mRNA-1273 within the first month after the receipt of two doses to 94% for BNT162b2 and 98% for mRNA-1273 at 5-6 months after the receipt of two doses.

Owing to a long interdose interval for the ChAdOx1-S vaccine and its delayed use from February 2021 in France, we could not estimate its effectiveness at 5-6 months after the receipt of two doses. However, its effectiveness was estimated at 90% after 3-4 months versus 94% during the first month after full vaccination. In a study from England, researchers found a similar result to ours: from 96% effectiveness within the first month after the receipt of two doses to 90% after four months46; in two studies conducted in Sweden and Scotland, researchers found a larger decrease in vaccine effectiveness: 89% to 64%,48 and 84% to 64%,57 respectively. Effectiveness of all vaccine products with a longer follow-up still needs to be studied.

For some of the results, the confidence intervals for the ChAdOx1-S vaccine are much wider than for the mRNA vaccines it is being compared to. These wider confidence intervals could be explained by a systematically shorter follow-up time after the second dose, leaving less time to develop outcomes of interest. This shorter follow-up time was caused by a longer interdose interval for the ChAdOx1-S vaccine than for the mRNA vaccines.

Two dose vaccine effectiveness during the time of the circulation of the delta variant of SARS-CoV-2 was 89% for the BNT162b2 vaccine, 95% for the mRNA-1273 vaccine, and 91% for the ChAdOx1-S vaccine. A similar result was reported in published studies on these vaccines from different countries.25 45 46 50 51 53

We also observed that mRNA vaccines were highly effective (≥90%) across all studied age categories. However, the ChAdOx1-S vaccine’s effectiveness tended to decrease with age. Lower effectiveness (80%) among adults aged ≥75 years than among those aged ≤74 years might be due partly to a selection bias, because this vaccine was not formally recommended for this older portion of the population. Decreased vaccine effectiveness of ChAdOx1 nCoV-1958 and mRNA vaccines44 45 52 among older participants has also been observed elsewhere.

Strengths and limitations

This study had some limitations. Firstly, given the observational nature of the data, our study could have been affected by residual confounding owing to differences between vaccinated individuals and unvaccinated controls, especially in terms of healthcare seeking behaviour or risk of infection, which could have been affected by changes in public health policy or transmission dynamics during the study.78 To take into account these differences, each newly vaccinated person was matched on the day of the vaccination to an unvaccinated control of the same age, sex, region of residence, and nursing home status for individuals aged ≥75 years. Subsequently, we adjusted all analyses on history of influenza vaccine received during the previous two years before the covid-19 outbreak and for a varied range of comorbidities that were shown to be associated with covid-19 related hospital admission.71

Secondly, during the first week after the receipt of the first dose, a risk reduction was observed among vaccinated individuals, although such an early reduction could not be due to the effect of the vaccine. This effect might be explained by healthier or more health conscious people being more likely to be included in the vaccinated group, and by the temporary exclusion of the vaccination of individuals having symptoms suggestive of covid-19 in the unvaccinated group. Owing to the matching procedure, vaccinated and unvaccinated groups had a similar proportion of comorbidities except that vaccinated individuals were less likely to be socially disadvantaged, less likely to be infected by SARS-CoV-2 before vaccination, and more likely to be vaccinated against influenza than unvaccinated controls (online supplemental tables 2–4). We took into account observed differences by adjusting for these covariables and using specific outcomes related to covid-19 vaccines.79 However, residual healthy vaccinee bias could have overestimated vaccine effectiveness against severe covid-19.

Thirdly, our results could have been overestimated because vaccinated individuals were more likely to live in an environment surrounded by vaccinated individuals. Two studies have shown that individuals without SARS-CoV-2 immunity had at least 39% lower risk of infection as the number of immune family members increased.80 81 Finally, our study did not look at covid-19 vaccine effectiveness against non-severe covid-19 outcomes, such as asymptomatic or mild SARS-CoV-19 infections that did not lead to hospital admission.

SNDS, a claims database comprising of the population of France, has allowed us to comprehensively examine the effectiveness of three covid-19 vaccines in adults aged ≥50 years. This study also provides estimates of the effectiveness of two doses of the ChAdOx1-S vaccine at a national level in France, among the 3 million individuals who have received this vaccine. Our study population, derived from the general population, should limit the risk of selection bias.

Conclusions

This nationwide cohort study of more than 11 million vaccinated people matched with unvaccinated controls showed that the BNT162b2, mRNA-1273, and ChAdOx1-S vaccines were highly and equally effective against severe covid-19 outcomes after completing a two dose vaccination schedule. The level of effectiveness persisted at 5-6 months after the receipt of the second dose for mRNA vaccines, at 3-4 months after the second dose of the ChAdOx1-S vaccine, and at the time of the delta variant of SARS-CoV-2 circulation. These results from the study conducted between 27 December 2020 and 20 August 2021 cannot be assumed to be entirely transferable to the current or future dominant variants. The effectiveness of covid-19 vaccines should be carefully examined over a longer period and with regard to possible surges of new SARS-CoV-2 variants of concern.