Efficacy of first dose of covid-19 vaccine versus no vaccination on symptoms of patients with long covid: target trial emulation based on ComPaRe e-cohort

Data sources

The ComPaRe long covid cohort is an ongoing nationwide e-cohort (ie, a cohort where recruitment and follow-up are performed online) of patients with long covid in France, nested in the ComPaRe research programme (www.compare.aphp.fr), an umbrella e-cohort of patients with chronic conditions.15 Participation opened in November 2020 and is ongoing. The cohort includes adult patients who have reported a SARS-CoV-2 infection (whether or not confirmed by a positive polymerase chain reaction (PCR) test result or serological assay, or both) and have symptoms persisting for more than three weeks after the original infection. Recruitment took place from calls on social and general media, by partner patient associations, on the official French contact tracing app TousAntiCOVID, and by a snowball sampling method where participants were encouraged to invite people who had covid-19 and persisting symptoms to enrol.16

Participants in the ComPaRe long covid cohort are contacted for follow-up every 60 days by email with links to an online questionnaire. At each observation (eg, T0=cohort enrolment, T1, T2), patients are asked if they still have symptoms related to covid-19. Those who report persisting symptoms complete the long covid symptom tool and impact tool, a pair of validated patient reported instruments assessing, respectively, 53 long covid symptoms and six dimensions of patients' lives that can be affected by the disease.17 Those reporting no symptoms are asked to specify the date when they first noticed the absence of symptoms.

Since 11 May 2021, every 45 days patients have been self-reporting their covid-19 vaccination status in a different online questionnaire. Those who have been vaccinated report the vaccine received, the date or dates of vaccination, and any adverse effects. In September 2021, all patients who had not previously reported being vaccinated were contacted by email to confirm their vaccination status.

Eligibility criteria

Our analyses used data from patients enrolled in the ComPaRe long covid cohort before 1 May 2021. We included adult patients (aged ≥18 years) with a confirmed or suspected SARS-CoV-2 infection, diagnosed by a health professional, whose symptoms persisted for >3 weeks after the original infection, and who reported at least one symptom attributable to long covid at baseline. We excluded patients whose date of first symptoms was <3 months before baseline because at the time of the study the recommendation in France was to delay vaccination for three months for patients who had recently been infected with the SARS-CoV-2 virus.18 We also excluded patients reporting a history of severe allergy in ComPaRe because of the likelihood of a history of anaphylaxis, a contraindication to vaccination at the time of the study.

Outcomes

The primary outcome was the score on the long covid symptom tool, a validated patient reported outcome developed from patients' lived experience of long covid, assessing the number of symptoms of long covid (online supplemental material 1). The symptom tool score ranges from 0 (ie, remission of disease) to 53, and has been shown to correlate with patients' quality of life and functional limitations. Reproducibility of the score was excellent (intraclass correlation coefficient 0.83, 95% confidence interval 0.80 to 0.86).17 Because the long covid symptom tool is a symptom count score, any change in the score relates to an objective change in patients' perception of their symptoms (ie, at least one symptom has disappeared or appeared between the two measurements). We also investigated the rate of remission of the disease (ie, complete disappearance of symptoms).online supplemental material 1

We used the long covid impact tool, a second validated patient reported outcome, to assess the effect of the disease on patients’ social, professional, and family lives. The impact tool score ranges from 0 (no impact) to 60 (maximal impact) and has been shown to be highly correlated with patients' quality of life and patients' perceived severity of their disease. The impact tool score showed excellent reproducibility (intraclass correlation coefficient 0.84, 95% confidence interval 0.80 to 0.87).17 We also analysed the score on the impact tool classified according to its Patient Acceptable Symptom State, which represents the level of a continuous outcome measure below which patients consider themselves well.19 In a previous study, we estimated that the Patient Acceptable Symptom State for the impact tool was 30/60.17 All outcomes were assessed at 120 days after baseline.

Adverse events after vaccination were analysed by one investigator (V-TT) from participants’ open text answers to the related questions in the online questionnaire. Adverse events were categorised as serious based on the definition of the US Food and Drug Administration: adverse events resulting in death, life threatening, requiring admission to hospital or prolonging an existing stay in hospital, resulting in persistent or substantial disability, or requiring a specific intervention to prevent permanent impairment or damage.20

Study groups and follow-up

To define a vaccinated group and a matched unvaccinated control group in a population where most patients were vaccinated against covid-19, we used the cohort data to emulate a sequence of three trials which were then pooled.21 In the first trial, we identified all patients who met the eligibility criteria when they were enrolled in the ComPaRe long covid cohort (ie, their first observation point, T0). Patients who received their first covid-19 vaccine injection with the ChAdOx1 (AstraZeneca), BNT162b2 mRNA (Pfizer-BioNTech), Ad26.COV2.S (Johnson & Johnson), or mRNA-1273 (Moderna) vaccines between baseline and 60 days (ie, their second observation point, T1), were classified as the vaccinated group and matched in a 1:1 ratio to patients who did not receive the vaccine in the same period (T0 to T1), classified as the control group. Patients were followed up for 120 days (ie, their third observation point, T2, and endpoint of the first trial). Unvaccinated controls who were vaccinated before T2 were censored at the date of vaccination.

We repeated this procedure by emulating two more trials, by considering baseline at 60 days (ie, T1) for the second trial, and T2 for the third; we applied a similar follow-up strategy (ie, follow-up until T3 and T4, respectively). At the baseline of each of the three trials, patients' eligibility criteria were reassessed and those who no longer met the eligibility criteria (eg, because they no longer reported symptoms) were excluded from that trial. Control patients who had since received a covid-19 vaccine were eligible for inclusion in the vaccinated group even though they were in the control group in the first (or second) trial. However, a patient could only be selected once for the control group and once for the vaccinated group. Online supplemental material 2 defines the sequence of trials in more detail.

Statistical analysis

Our causal contrast of interest was the per protocol effect. Within each of the three trials, each patient who was vaccinated was matched to an unvaccinated control according to their probability of getting vaccinated against covid-19 given their baseline covariates (ie, the propensity score). The propensity score was calculated with a non-parsimonious multivariable logistic regression model including variables planned and prespecified before the outcome analyses: sex; age; educational level (≥2 years post-secondary education—higher education v lower); number of comorbidities (self-reported with the International Classification of Primary Care, version 2)22; SARS-CoV-2 infection confirmed by laboratory analysis (yes for patients reporting a positive test result for SARS-CoV-2 infection by PCR test or serological assay, or both, and otherwise no); interval from the start of covid-19 symptoms; history of admission to hospital for covid-19 during the acute phase; score on long covid symptom tool at baseline; and score on long covid impact tool at baseline. Standardised differences were examined to assess balance, with a threshold of 10% indicating a clinically meaningful imbalance.23 Propensity score matching used a calliper width of 0.2 of the pooled standard deviation of the logit of the propensity score.24

For our analysis, we pooled the three trials and estimated the effect of treatment with one model, including a trial covariate, rather than fitting a separate model for each trial. Also, because some people participated in more than one trial, we used a robust variance estimator to estimate conservative 95% confidence intervals.

To correct for the induced time varying selection generated by the artificial censoring of patients in the unvaccinated group at the date of their first vaccine injection, we used inverse probability of censoring weighting with weights proportional to the inverse of the probability of remaining uncensored until each time point, given the baseline covariates. Stabilised weights were obtained by multiplying the weights by the overall probability of being uncensored at each time point.25 To assess the quality of the correction, we compared the number of patients at risk, over time, in the two groups, and the balance of the baseline covariates between the two groups, 120 days after inclusion in the trials (online supplemental material 3).

In the survival analyses, to account for immortal time bias, baseline was considered as the vaccination date for patients in the vaccinated group and the vaccination date of their matched patient for those in the control group. Outcomes were studied in the total population and in a subgroup restricted to participants with a confirmed SARS-CoV-2 infection. Two post hoc subgroup analyses were conducted: onset of symptoms ≤12 months versus >12 months, and vaccine type.

We used the E value to evaluate how the results could be affected by unmeasured confounding. The E value measures the minimum strength of association an unmeasured confounder would need to have with both the intervention and the outcome to fully explain away the treatment effect.26

We performed several sensitivity analyses. Firstly, we restricted the study population to patients who had been included in only one of the three trials (ie, excluding patients included twice in the study, once in the unvaccinated group and then in the vaccinated group) to examine the potential correlation induced by including the same patient in several trials. Secondly, we analysed how our design, based on a sequence of emulated trials, could affect our results by estimating separate treatment effect estimates for each trial and then conducting a meta-analysis with a fixed effect approach. Finally, we conducted a secondary analysis that used standardised mortality ratio weighting as an alternative to propensity score matching.27

Missing baseline and outcome variables were handled by multiple imputation with chained equations that used the other variables available. All statistical analyses were performed with the R statistical package version 4.0.3 (The R Foundation for Statistical Computing, www.R-project.org/).

Patient and public Involvement

The study is a reanalysis of existing data. Patients were involved in design of the questionnaires and measurement tools used in the cohort. In ComPaRe, lay summaries of research results are systematically shared with participants and partner patient associations on the project's homepage.

Participants

Among the 1296 patients included in the ComPaRe long covid cohort before 1 May 2021 who provided their vaccination status, 910 were included in the analyses (455 patients in the vaccinated group matched to 455 patients in the control group) (online supplemental material 4). Online supplemental materials 5–7 give details of the definition and matching procedures in the sequence of trials. In each trial and after pooling all trials, all variables included in the propensity score models had standardised differences of <10% after matching (online supplemental material 8).

Median age of the patients was 47 years (interquartile range 40-54); 733 (80.5%) were women, 545 (60.1%) had a confirmed SARS-CoV-2 infection, and 81 (8.9%) had been admitted to hospital during the acute phase of the covid-19 disease. The interval between the onset of symptoms and baseline was 10.7 months (interquartile range 6.4-12.4). Patients had been infected before 1 May 2021 and thus were not infected with the delta or omicron variants of the SARS-CoV-2 virus. Characteristics of patients were similar in the vaccinated and control groups (table 1).

In the vaccinated group, 359 patients (78.9%) received the BNT162b2 mRNA vaccine, 48 (10.5%) the ChAdOx1 vaccine, 47 (10.3%) the mRNA-1273 vaccine, and one (0.2%) the Ad26.COV2.S vaccine. The median interval between baseline and the first covid-19 vaccine injection in the vaccinated group was 38 days (interquartile range 24-53).

Sixty nine patients were lost to follow-up (32 in the vaccinated group and 37 in the control group), and 275 (60.4%) patients in the control group were censored at their vaccination date. The median interval between baseline and censoring was 90 days (interquartile range 72.5-105).

Outcomes

At 120 days after baseline, long covid was less severe in the vaccinated group than in the control group. The mean score on the long covid symptom tool was 13.0 (standard deviation 9.4) in the vaccinated group and 14.8 (9.8) in the control group (mean difference −1.8, 95% confidence interval −3.0 to −0.5). By 120 days, 16.6% patients in the vaccinated group (n=57) reported remission of all symptoms from long covid compared with 7.5% (n=27) in the control group (risk difference 9.1%, 95% confidence interval 5.0% to 13.2%; hazard ratio 1.93, 1.18 to 3.14; E value 3.26) (figure 1).

Request permission

Long covid affected patients' lives less in the vaccinated group than in the control group. The mean score on the long covid impact tool was 24.3 (standard deviation 16.7) in the vaccinated group and 27.6 (16.7) in the control group (mean difference −3.3, 95% confidence interval −5.7 to −1.0). The proportion of patients reporting an unacceptable symptom state (impact tool score more than the Patient Acceptable Symptom State) was 38.9% in the vaccinated group and 46.4% in the control group (risk difference −7.4%, 95% confidence interval −14.5% to −0.3%; E value 1.67) (table 2).

The effect of vaccination on the severity and impact of long covid was similar in the subgroup of patients with covid-19 confirmed by laboratory test results. The mean difference in scores for the symptom tool was −1.8 (95% confidence interval −3.4 to −0.2) and for the impact tool −3.8 (−6.7 to −0.8) (online supplemental material 9). Similarly, we found no evidence that the effect of vaccination on the severity of the disease varied by time since the original infection (online supplemental material 10).

The difference in outcomes for the 359 patients who received a first dose of BNT162b2 mRNA (Pfizer-BioNTech) and their 359 matched controls was −1.9 (95% confidence interval −3.4 to −0.5) for the symptom tool score and −3.6 (−6.1 to −1.0) for the impact tool score. For the 47 patients who received a first dose of mRNA-1273 (Moderna) and their 47 matched controls, the difference in outcomes was 1.5 (−2.0 to 5.1) and 2.6 (−4.8 to 10.1) for the symptom tool and impact tool scores, respectively. For the 48 patients who received a first dose of ChAdOx1 (Astra Zeneca) and their 48 matched controls, the difference in outcomes was −3.9 (−7.7 to −0.1) and −7.8 (−14.7 to −0.9) for the symptom tool and impact tool scores, respectively.

Our main results were consistent with a sensitivity analysis where patients who were vaccinated could not be used as controls in observation periods before they received their vaccine. The mean difference in scores was −1.6 (95% confidence interval −3.2 to −0.1) for the symptom tool and −3.1 (−5.9 to −0.2) for the impact tool (online supplemental material 11). We investigated whether the delay between baseline and vaccination affected patients' outcomes 120 days after baseline and found no evident interaction (online supplemental material 12). A sensitivity analysis where the treatment effect estimates from the three emulated trials were calculated separately and in a meta-analysis of the three trials showed results consistent with the primary analysis (online supplemental material 13). Our analysis with inverse probability of treatment weighting gave similar results to propensity score matching (online supplemental material 14).

Adverse effects reported in the vaccinated group

All patients who were vaccinated were asked to report any adverse events after vaccination. In total, 26/455 (5.7%) patients self-reported an adverse effect after vaccination (table 3). Four (0.9%) were considered serious adverse events: two (0.4%) requiring admission to hospital (one for deep vein thrombosis and one for meningitis), and two (0.4%) required a visit to the emergency department. Other adverse events included relapse of long covid symptoms (n=13, 2.8%) and known local and systemic reactions to vaccination (eg, shoulder pain, mild fever) (n=5, 1%).

Principal findings

In patients with long covid, we found that the first covid-19 vaccine injection was associated with a reduction in the severity of the disease and on the effect on patients' social, professional, and family lives at 120 days after baseline. In particular, our results showed that the remission rate of long covid symptoms was 16.6% in the vaccinated group (n=57) compared with 7.5% (n=27) in the control group (hazard ratio 1.93, 95% confidence interval 1.18 to 3.14). Receiving a vaccine was also associated with a significant increase in the proportion of patients reporting an acceptable symptom state; for every 13 patients with long covid who are vaccinated, one will have a notable decrease in the disease’s effect on their life. Side effects related to the vaccine seemed to be generally rare, with only two (0.4%) patients reporting an adverse effect that required admission to hospital. But for some patients, vaccination seemed to worsen the disease or cause a relapse. These results are in line with other reports investigating the efficacy of vaccination on symptoms of patients who already have long covid which showed that 20-40% of patients reported worsening of their symptoms or relapses after vaccination.8 28

Millions of patients have persistent symptoms after infection with the SARS CoV-2 virus and many more might be at risk in the future. To our knowledge, this is the first study of a potential intervention that could reduce the burden of long COVID on care systems. Showing the effectiveness of vaccination on symptoms of long covid might also help our understanding of the causes of the persistence of symptoms after SARS-CoV-2 infection. The finding that vaccines might reduce symptoms for patients who already have long covid implies that, for at least some patients, the disease could be related to the presence of a persistent viral reservoir, or of virus fragments capable of stimulating the immune system.11 This hypothesis is supported by a recent study which detected circulating SARS-CoV-2 spike in most patients with long covid up to 12 months after infection. Another potential mechanism is the development of an autoimmune response induced by the infection and the reset of these autoimmune processes by vaccination. Another recent study found that covid-19 survivors had increased levels of circulating antinuclear or extractable nuclear antibodies that correlated with the presence of long covid symptoms one year after onset.29 30 Future research should examine the symptoms and characteristics of patients who improved after vaccination to identify targets for specific treatments for long covid.

Strengths and limitations of this study

Our study showed how cohort data can be used to answer comparative effectiveness questions when randomised trials are no longer feasible.14 Recruitment for a randomised trial to evaluate vaccination in patients with long covid would be difficult, if not impossible, in western countries where most of the population has already been vaccinated.

Our study had some limitations. Firstly, despite the use of robust methods and statistical techniques to make causal inferences from observational data, the intervention was not randomly assigned, and potential unmeasured confounders could have biased our results. For example, patients' motivation to receive a covid-19 vaccine was not taken into account, although it might be related to their perception of their long covid symptoms.

Secondly, participants in ComPaRe were asked to complete the online questionnaires every 60 days and so we could not define follow-ups relative to the time of vaccination. In the time-to-event analysis, the study baseline was defined as the vaccination date for patients in the vaccinated group, and the vaccination date of their matched patient for those in the control group; the treatment effect was thus directly related to the efficacy of vaccination at 120 days after vaccination. For the analysis of the two patient scores (symptom and impact tools), however, baseline was defined as the patient's state 120 days before the outcome measurement and up to 60 days before vaccination; the treatment effect was thus related to the efficacy of vaccination from 60 to 120 days after vaccination. To account for variation in outcome measurement times, we assessed how time since vaccination affected the study outcome and found no evidence of an interaction between the delay from baseline to the time of vaccination and the severity of patients' symptoms or the effect on their lives (online supplemental material 10).

Thirdly, all patients in our study were infected before 1 May 2021, and so were not infected with the recent delta and omicron variants of the SARS-CoV-2 virus. The effectiveness of vaccination on persistent symptoms occurring after infection with these variants is unknown. Also, patients in this study had not been vaccinated before their infection and long covid, and whether our results are applicable to breakthrough infections in vaccinated individuals is unclear.

Fourthly, because the outcome was self-reported and the intervention was not blinded, the difference in outcomes between the two groups might be a result of specific and placebo (non-specific) effects. The direction and magnitude of these non-specific effects are difficult to estimate, however, because at the time of our study, reports of both improved and worsening of symptoms of long covid after vaccination had been reported.

Fifthly, adverse events after vaccination were collected with an online questionnaire that did not ask patients about the precise time of these events.

Sixthly, patients might not have remembered the date of remission of symptoms accurately, although this error should be limited because two months is a relatively short time frame and remission is an important date for patients with debilitating symptoms. Also, recall bias is unlikely because recalling the date of their remission should have been the same in the two arms of the study.

Finally, compared with patients described by the Office for National Statistics in their report on the prevalence of ongoing symptoms after covid-19 infection in the UK, our study had fewer young (<25 years old) and elderly (>70 years old) patients, more women, and more patients who had been admitted to hospital during their acute infection.1 These differences might affect the generalisability of our findings but do not affect the internal validity of our results.

Conclusions

In this study, we found that covid-19 vaccination reduced the severity of symptoms and the effect of long covid on patients' social, professional, and family lives in those who already have persistent symptoms of infection. These findings might be helpful in encouraging patients to be vaccinated after SARS-CoV-2 infection and could further our knowledge of the mechanisms underlying long covid.