Non-pharmaceutical interventions and covid-19 burden in the United States: retrospective, observational cohort study

Study design, setting, and population

We performed a retrospective, observational cohort study of the US population between 19 January 2020 (first covid-19 diagnosis in the US) and 7 March 2021. The primary analysis evaluated the dates of state specific adoption or discontinuation of non-pharmaceutical interventions on covid-19 case and mortality rates.

Data collection

State level case and mortality counts per day were obtained from the Covid-19 Tracking Project,31 which aggregate public reports of covid-19 diagnostic and death certification data for the US. The data reported are obtained from state public health authorities or official media accounts.24 32 The dominant covid-19 virus during the period of study was the original virus (SARS CoV-2), followed by the alpha variant (B.1.1.7) at the end of the observational period.33 34

We obtained dates for adoption and discontinuation of non-pharmaceutical interventions studied from publicly available reports (online supplemental table S1).24 35–40 Policies that were incorporated into the analysis included orders to stay at home, indoor public gathering bans (mild or severe), indoor restaurant dining bans, and public mask mandates. We recorded dates for each policy adoption or discontinuation from 21 February 2020 to 29 January 2021 for the time dependent modelling. When sources were not in agreement with the date of adoption or discontinuation, we reviewed executive orders and media outlet publications.

Definitions were set for each non-pharmaceutical intervention used. We defined indoor public gathering bans with a maximum of 10 or fewer people as severe, and indoor public gathering bans with more than 10 people as mild. Indoor restaurant dining bans were adopted when indoor dining was banned and were discontinued when indoor dining was reinstated, regardless of capacity specification or outdoor dining policies. Public mask mandates also applied to indoor mandates, regardless of outdoor policy. We defined stay-at-home orders as any statewide policy ordering discontinuation of all non-essential travel from home.

Statistical analysis

We used each state’s daily case and death counts to model the rate of change in the number of cases and deaths due to covid-19 per week (velocity). We obtained case and death velocity by taking the first derivative with respect to time of the log cumulative daily case (and death) counts reported at the state level.41 We defined establishment of community transmission (≥100 confirmed cases for each state) as time zero. Cubic splines were fit to the log cumulative counts to facilitate precise calculation of the first derivative with respect to time. We applied a log link to map the case (or death) velocities to the entire real line which allowed for modelling of velocities as a linear function.

We used a breakpoint analysis to evaluate changes in case velocity attributable to policy adoptions or discontinuations. This approach identifies time periods with deviations in the covid-19 case or death velocity rates. A change in covid-19 growth rate is determined if adding a new regression slope at a specific date decreases the residual sum of squares sufficiently to improve the bayesian information criterion. Breakpoints were empirically identified for each state using the strucchange R package for dating structural changes in regression models.42 A minimal segment length of 14 days was set by specifying a minimum number of observations per segment (14). As a result of the minimal segment size specification, each state had at most one breakpoint per 7 day week, which allowed for determination of whether a state experienced an increase, decrease, or no change in each week. We excluded breakpoints after 5 February 2021 for cases and 12 February 2021 for deaths to align with the data of adoption and discontinuation dates of non-pharmaceutical interventions, which extended up to 29 January 2021, plus anticipated lag time.

Assessments of the impact of policy adoption assumed prespecified lag times related to the natural course of transmission and disease. We estimated lags of 3-10 days from policy adoption to changes in transmission, 5-10 days from infection to covid-19 diagnosis, and 6-15 days from covid-19 diagnosis to death (online supplemental material). Based on these intervals, we considered policies adopted between 0 and 21 days (0-3 weeks; ie, lag time) before the start date in each week as potentially influential to case velocity. We considered policies adopted between seven and 35 days (1-5 weeks) before the start date of a given week as potentially significant to death velocity. We created generalised linear models from generalised estimating equations to evaluate policy changes in relation to shifts in covid-19 case and death velocities. We defined a three level outcome variable for case and death velocities (increase, no change, or decrease). Ordinal logistic regression with a cumulative logit link function modelled the probability of a state week corresponding to a decrease in case (or death) velocity versus no change, and no change versus an increase in case (or death) velocity.

We specified an independent working correlation structure to account for correlation within US states. We fit two sets of models that included a fixed effect for the given modelled policy (a separate unadjusted model for each policy), and a fixed effect for each policy (one adjusted model for all four policies). We subsequently describe this second set of models with the term mutually adjusted model in this article. Time variability shared across states was accounted for by including week number as a continuous fixed effect and using three change points to allow for piecewise trends during phases of the pandemic. We aligned four phases with the modelled calendar weeks (online supplemental material).

To account for uncertainty in our natural history estimations described, we performed a sensitivity analysis of shifting the lag time interval by 7 days before or 7 days after our base assumption of lag time (online supplemental material). The sensitivity analysis was a check to determine whether the lag times we chose up front (not biased by the data) were reasonable. We chose the lag times before analysis on the basis of what we thought was reasonable based on review of covid-19 transmission (online supplemental material). The analysis also explored, in a novel but limited fashion, how optimal lag times between adoption or discontinuation of non-pharmaceutical interventions and potential impacts in covid-19 case or death velocity might differ by intervention.

Policy models were fit to data consisting of one observation per state, per week, starting with the first week in which a breakpoint was observed. To summarise results from the cumulative logit model formulation, we use the phrase "decrease in case velocity" to imply either the probability of observing a decreasing breakpoint versus no change or observing no change versus an increasing breakpoint. Analyses were performed in R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).43

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Associations between non-pharmaceutical interventions and covid-19 case velocities

We identified 603 case breakpoints (table 1). A decrease in case velocity was observed in 433 (71.8%) breakpoints and an increase in case velocity was observed in 170 (28.2%). Across all 50 US states, the median number of case breakpoints was 12 (range 7-17). We graphed the number of breakpoints with an increase in case velocity and a decrease in case velocity chronologically (online supplemental figure S2), and plotted cumulative cases over time with their estimated breakpoints for the states with the three highest and lowest covid-19 burdens, according to cases per capita (figure 2). Plots of the derivative of the logarithm of cumulative cases (velocity) and the logarithm of the derivative of the logarithm (data implemented in modelling) for these states are available in online supplemental figures S4 and S5).

Request permission

Ordinal logistic regression models including a single policy adoption or discontinuation estimated increased odds of a decrease in case velocity after adoption of stay-at-home orders (odds ratio 2.02, 95% confidence interval 1.63 to 2.52), indoor restaurant dining bans (1.62, 1.25 to 2.10), public mask mandates (2.18, 1.47 to 3.23), and severe indoor public gathering bans (1.68, 1.31 to 2.16). Adoption of a mild indoor public gathering ban was associated with decreased odds of a reduction in case velocity (0.51, 0.39 to 0.68; table 2). In sensitivity analysis, discontinuation of indoor restaurant dining bans trended toward progressive association with decreasing odds of a reduction in case velocity as lag time was extended from baseline assumptions (ie, allowing additional time between policy discontinuation and measurement of effect; online supplemental eTable 1). The strongest association of decreased odds of a reduction in case velocity in sensitivity analysis was for a time lag time shifted 7 days more distal than our base assumption (odds ratio 0.39, 95% confidence interval 0.26 to 0.60; online supplemental eTable 1). Progressive strength of association with increased lag time was also noted for both adoption and discontinuation of stay-at-home orders, adoption of public mask mandates, and adoption of severe indoor public gathering bans (online supplemental eTable 1).

In mutually adjusted models, adoption of a stay-at-home order was associated with decreasing case velocity (adjusted odds ratio 1.47, 95% confidence interval 1.04 to 2.07), as was adoption of a public masking mandate (2.27, 1.51 to 3.41). Adoption of a mild indoor public gathering ban was associated with a subsequent increase in case velocities (0.46, 0.34 to 0.61), yet adoption of a severe indoor indoor public gathering ban trended toward the opposite association (1.38, 0.97 to 1.95; table 3). In sensitivity analysis, adoption of indoor restaurant dining ban was associated with decreasing case velocity for a lag time shifted 7 days more distal than our base assumption (2.05, 1.22 to 3.45; online supplemental eTable 2). Associations to case velocity were also strongest at a lag time 7 days greater lthan our base assumption for the adoption of indoor restaurant dining bans, discontinuation of indoor restaurant dining bans, and adoption of public mask mandates (online supplemental eTable 2).

Associations between non-pharmaceutical interventions and covid-19 death velocities

A total of 461 death breakpoints were identified (table 1). 330 (71.6%) of breakpoints corresponded to a decrease in death velocity and 131 (28.4%) corresponded to an increase in death velocity. Across all 50 US states, the median number of death breakpoints was 9 (range 4-17; online supplemental figure S3). We plotted cumulative deaths over time with their estimated breakpoints for the states with the three lowest or highest number of covid-19 cases per capita (online supplemental figure S6). Plots of the derivative of the logarithm of cumulative deaths (velocity) and the logarithm of the derivative of the logarithm (data implemented in modelling) for these states are available in the online supplemental figures S7 and S8.

Ordinal logistic regression models including a single policy estimated increased odds of a decrease in death velocity after adoption of stay-at-home orders (odds ratio 2.00, 95% confidence interval 1.53 to 2.62), indoor restaurant dining bans (1.50, 1.15 to 1.95), and severe indoor public gathering bans (1.45, 1.12 to 1.88; table 2). In sensitivity analysis, adoption of public mask mandate was associated with a decrease in death velocity for a lag time shifted 7 days more distal than our base assumption (1.84, 1.29 to 2.62; online supplemntal eTable 1). Associations to death velocity were also strongest at 7 days greater lag time than our base assumption for adoption of stay-at-home orders, adoption of indoor restaurant dining bans, and adoption of severe indoor public gathering bans (online supplemental eTable 1). In mutually adjusted models, institution of an order to stay at home was associated with decreasing death velocity (adjusted odds ratio 1.89, 95% confidence interval 1.25 to 2.87; table 3). Adoption of indoor restaurant dining bans (1.74, 1.15 to 2.63) and adoption of public mask mandates (1.95, 1.35 to 2.82) were associated with a decrease in death velocity for the more distally shifted lag time (online supplemental eTable 2).

Principal findings

In our study, we found associations between the adoption of multiple non-pharmaceutical interventions and decreasing covid-19 case and mortality burdens in the US. With respect to cases, our adjusted time dependent models found that stay-at-home orders and public mask mandates were associated with decreases in the rate of new diagnoses of covid-19. Even after adjusting for three other concurrently adopted interventions, public mask mandates were associated with over twice the likelihood of reduced covid-19 transmission. Public mask mandates could encourage behavioural modifications as well as directly reduce the odds of transmission by using a physical barrier.44 45 Covid-19 is now understood to be transmitted primarily through aerosol spread in close contact.46 The US state level observations provide support for mask mandates in reducing the case burden of respiratory epidemics or pandemics.

US states that adopted mild indoor gathering bans had increases in covid-19 case burden relative to states that did not adopt a mild indoor public gathering ban. Results from the mutually adjusted policy model suggested indoor restaurant dining bans and severe indoor public gathering bans could be associated with decreased case velocity (both P<0.10). We found a stronger direction of association with a lag time 7 days more distal than our base assumption, suggesting that indoor dining bans and severe indoor public gathering bans might take longer to confer reductions in case burden.

Overall, we found that gathering bans with limits of more than 10 people were insufficient and were associated with exacerbation of covid-19 spread, possibly because US states often selected these bans as an alternative to the more effective severe ban. Furthermore, different non-pharmaceutical interventions could have been associated with increases or decreases in covid-19 burden owing to behavioural changes linked to the intervention but not specifically resolved by it. For example, indoor public gathering bans with maximums greater than 10 people might not inherently be ineffective in decreasing the burden of covid-19, but they could elicit a different generalised public response, especially relative to those severe bans limiting gatherings to 10 people or fewer. Brauner et al28 made a related finding of progressively stricter gathering bans inferring decreased covid-19 burden, including a median reduction greater than 35% in instantaneous reproduction number associated with the policy adoption of gatherings limited to 10 people or fewer. Overall, these observations are consistent with concerns regarding the indoor transmission of covid-19 among large groups of individuals in public settings. Policies of non-pharmaceutical interventions that discourage large gatherings are effective at reducing respiratory transmission.

With respect to mortality, stay-at-home orders were associated with decreasing covid-19 mortality in unadjusted and adjusted models, and in all sensitivity analyses considered. Additionally, adoption of indoor restaurant dining bans and adoption of public mask mandates were associated with decreasing covid-19 mortality in unadjusted and adjusted models incorporating a lag time 7 days more distal than our base assumption. One explanation of this finding is that the peak onset of effect of these four non-pharmaceutical interventions could differ. Adoption of indoor restaurant dining ban and public mask mandate might take longer to confer benefit, while stay-at-home orders could confer more immediate and sustained benefit. Despite some suggestion of benefit in some non-pharmaceutical interventions with respect to mortality, overall, the strengths of association were more profound for cases than for deaths. Sample size limitations, limited variation in timing of policies adopted, and temporal variation in the progression of covid-19 to death all limit our ability to attribute deviations in daily death counts to specific policy actions. Furthermore, interventions that are associated with case reductions but not mortality could reflect a shift towards infection among younger cohorts at lower risk for death. This observation might be particularly true for the public mask mandates, which appeared in later phases of the covid-19 pandemic.

Comparison with previous studies

Our modelling approach allowed us to evaluate the merits of various non-pharmaceutical interventions concomitantly in a time dependent fashion. Previous studies have generally focused on pandemic influenza and relied on expert opinion or modelling rather than real world data.7 9 13 26 47 In fact, the most recent pandemic influenza plan by the US Department of Health and Human Services described study of non-pharmaceutical interventions in the status of a data collection phase.26 Some retrospective data regarding these interventions and viral pandemics have been published. An analysis of US cities found an association between increased duration of non-pharmaceutical interventions and total mortality reduction.13 Auger et al24 found that school closures were associated with decreased covid-19 incidence and mortality but adjustment for other interventions was not included. Bendavid et al48 reported, in an international comparison of 10 countries including the US, no observable benefit of more restrictive non-pharmaceutical interventions (stay-at-home order, non-essential business closures) compared with less restrictive interventions (social distancing guidelines, discouraging travel, and ban on large gatherings). We find the limited sample size and lack of variation in this study makes an absence of evidence conclusion difficult. Brauner et al28 found an additional inferred decrease in instantaneous reproductive numbers with a stay-at-home order even when accounting for gathering bans, business closure mandates, and school closures.

Although our analysis used a different methodological approach and set of non-pharmaceutical interventions, we also found an additional association of benefit for the interventions—including when we accounted for other interventions adopted. Our analysis found that both the strength and the direction of benefit related to severity of a gathering ban is important when assessing transmission dynamics. We found associations between multiple non-pharmaceutical interventions and decreased case burden of covid-19 in adjusted models (stay-at-home order, public masking mandate, and severe gathering ban), which is supportive of previous expert opinions encouraging early, sustained, and layered application of these interventions to mitigate consequences of pandemic viral disease.

Limitations of the study

Our analyses included several limitations. US state government enforcement of recommendations and policies varied. The public adherence to stated policies might vary owing to regional differences in preferences and beliefs. We did not attempt to measure markers of behavioural change based on the adoption or discontinuation of policies, and focused primarily on outcomes such as known transmission and deaths. We used a breakpoint analysis of smoothed count data (7 day averages) to analyse time periods characterised by similar case velocities. This statistical approach allowed for aggregating time periods across the limited sample of states (n=50). Our model did not adjust for national recommendations and policies. The two most prominent of these announcements included the Centres for Disease Control and Prevention recommendation of wearing cloth face coverings in public from 3 April 20204 and "The President’s Coronavirus Guidelines for America" enacted 16 March 2020, which included avoiding non-essential travel and avoiding social gatherings in groups of more than 10 people.3

Additionally, this study did not account for county or municipal level variation in policies of non-pharmaceutical interventions. The early period of the covid-19 pandemic in the US probably had lower rates of case ascertainment and differences in testing capacity between states. Although we cannot explicitly control testing capacity and policy by state in the statistical models, the availability of diagnostic tests grew linearly during the period of analysis and therefore would be unlikely to explain shifts in either case or death velocities.

This analysis was unable to evaluate the impact of non-pharmaceutical interventions with substantial temporal overlap, such as school closures, and the impact of multiple interventions adopted during concomitant periods could contribute to instability of estimates. We believe that this limitation of temporal overlap should be noted in prior publications.19 20 24 To resolve this limitation, we selected a smaller subset of interventions for evaluation. However, collinearity of these interventions limits our ability to draw the strongest conclusions, but this limitation is inherent in the data and non-experimental design. Although one alternative would have been to present model estimates from univariate analysis only, we believe that the exploratory multiple regression analyses are informative. Evaluation of our models for high correlation (online supplemental material) was reassuring and within expectations for non-pharmaceutical interventions adopted and discontinued at similar dates.

We acknowledge that lower socioeconomic status could associate with a higher covid-19 burden49–51 and could even vary at the state level.19 20 We expect that socioeconomic factors were generally static during our study, especially in comparison with the exponential increases and decreases in covid-19 case or death velocities and the frequent adoption or discontinuation of the four non-pharmaceutical interventions observed. Although socioeconomic factors would influence the intercept of our models, we would not expect changes in case velocity on a week by week basis to be attributable. Importantly, the Coronavirus Aid, Relief, and Economic Security Act (CARES Act), passed by the US Congress and signed into US law on 27 March 2020, homogenised unemployment benefits in the US for most of the duration of our study in both length and amount of benefit.52

Our model also relied on several unverified assumptions, such as the length and placement of the policy adoption window relative to a given week, and the minimal segment specification of two weeks for breakpoint identification. Although these decisions were based on expert knowledge and review of the literature, the impact of these assumptions is unknown. Furthermore, any uncertainty in the establishment of the empirically estimated breakpoints was not reflected in the subsequent policy models, which suggests less precision in our final estimates. We believe that the breakpoint analysis is a more blinded approach to segmenting shifts in case and death counts, compared with arbitrary decisions to create time cutpoints for pandemic waves that have the potential to introduce bias. Finally, although all four of the non-pharmaceutical interventions studied were associated with a decrease in covid-19 cases over the period of study in univariate analysis, we acknowledge that the effect of these interventions on the total number of covid-19 cases and deaths during the ongoing covid-19 pandemic is not known.

Conclusion

Adoption of several non-pharmaceutical interventions used by US states during the covid-19 pandemic were associated with subsequent decreases in covid-19 case burden. When accounting for the adoption of all four interventions modelled, a stay-at-home order was the most strongly associated with decreases in covid-19 mortality. Both restaurant dining and severe indoor public gathering bans (limiting to <10 people) were more strongly associated with reductions in transmission compared with mild indoor public gathering bans (limiting to >10 people). These findings reinforce efforts to deploy non-pharmaceutical interventions early and encourage adherence to limit the spread of dangerous respiratory epidemics.