Quantifying the impact of unmeasured confounding in observational studies with the E value

Introduction

Randomised controlled trials are the gold standard of causal inference of the benefits and risks of drug treatments. Successful randomisation removes both measured and unmeasured baseline confounding, enabling causal interpretation of an observed association.1 Randomised controlled trials, however, are not practically or ethically suitable for many types of research questions, including rare risks of treatments. Notably, few randomised controlled trials include pregnant women. Instead, the primary source of information on the safety of treatments in pregnancy is observational studies, relying on routinely collected (secondary) data from registries and other electronic databases.2 3 Observational studies based on secondary data allow adjustment of the analysis for the confounders measured in a particular database, whereby residual (unmeasured) confounding from poorly measured, unmeasured, or unknown characteristics might remain.

Approaches to removal or quantification of unmeasured confounding include external adjustment (correction of findings by using external information); use of instrumental variables (emulating randomisation); self-controlled designs (removal of time independent confounding); or negative and positive controls (benchmarking findings against established associations).4–7 Another option to deal with unmeasured confounding in observational research is to use the E value, a readily calculated and easily interpreted statistical tool that assesses the minimum strength of potential unmeasured confounding needed to explain away an effect. The E value was introduced in 2017 by VanderWeele and Ding.8 In common with many new epidemiological tools, integrating the E value into applied health research has taken time, and many researchers might still be unfamiliar with the method. In this paper, we want to raise awareness about the E value and encourage researchers to include this tool among their standard methods in sensitivity analyses. We first describe the E value method and then we show its use in a worked example of a meta-analysis combining the evidence for the association between the use of antidepressants in the first trimester of pregnancy and the risk of miscarriage.

The E value method

In a study with a specific exposure and outcome, the E value is the minimum strength of association that the unmeasured confounders should have with both the exposure and the outcome to explain away the observed association.8 The exposure refers to any characteristic that may explain or predict the presence of a study outcome.9 For a particular observed exposure-outcome risk ratio (RR), the E value can be calculated based on formula (1), involving only the estimate itself.10

The same approach applies to odds ratios and hazard ratios if the outcome is rare. If the outcome is common, the odds ratio and hazard ratio must be modified (eg, by replacing RR in formula (1) with the square root of the odds ratios).8

A strong association has a large E value, which suggests that the unmeasured confounding must be strongly associated with both the exposure and outcome to fully explain the association. In contrast, a small E value suggests that weak unmeasured confounding would be enough to explain the association. A lower E value might indicate that confounding rather than causality is a more plausible explanation than a higher E value. The E value does not have a specific range, and whether its value is considered large or small depends on the particular exposure and outcome and the amount of controlled confounding.11 For example, when examining the association between the use of glucocorticoids and the risk of suicide among patients with cancer, the incidence rate ratio was high (7.2) and the dose-response pattern showed that the highest cumulative dose was associated with a 20-fold increase in risk compared with non-use. In this example, the calculated E value indicated that to explain away the association, a hypothetical confounder would need be associated with a 14-fold higher use of glucocorticoids and a threefold greater risk of suicide. The authors judged that such a confounder is not likely to exist given the confounding already adjusted for in the analysis.12 Thus labelling the E value large or small depends on knowledge of the subject matter, the strength of the observed association, and amount of confounding removed.

The E value can be calculated without any assumptions regarding unmeasured confounders, such as being binary or consisting of only one unmeasured confounding factor.10 The E value estimates the overall strength of potential unmeasured confounding rather than the effect of individual confounding factors. With this information, investigators can assess whether one or several specific unmeasured confounders could plausibly explain away the observed association in a particular study.

In addition to the E value, Ding and VanderWeele introduced the joint bounding factor, B (formula 2).10

RR_EU denotes the strength of association between the unmeasured confounder and the exposure. B does not require assumptions about the structure of the unmeasured confounding.10 RR_UD denotes the strength of association between the unmeasured confounder and the outcome, as illustrated in a directed acyclic graph representing a simplified form of the confounding structure (figure 1).

Request permission

The joint bounding factor, B, could take an infinite number of values depending on RR_UD and RR_EU. If the joint bounding factor B is set to equal the observed risk ratio, the joint bounding factor describes the different combinations of RR_UD and RR_EU that would have the joint minimum strength to explain away the association. The E value=B when RR_UD=RR_EU, which is one possible combination of RR_UD and RR_EU. A range of different possible combinations exist, which are often illustrative for the setting studied (figure 2).

Request permission

For example, if RR_EU is known, the joint bounding factor can be used to estimate the minimum strength of association between the unmeasured confounder and the outcome (RR_UD) needed to explain away the association.8 In other settings, information might be available on the magnitude of the association between the strongest unobserved confounder and the outcome (RR_UD). The joint bounding factor can similarly be used to estimate RR_EU.

Example: meta-analysis of the association between use of antidepressants in pregnancy and risk of miscarriage

One in five women have depression during pregnancy, and about 13% of pregnant women take antidepressants.13 14 Because both untreated depression and drug treatments might adversely affect pregnancy, treatment of depression in pregnant women involves balancing the benefits of treated depression against the potential treatment related risks to the mother and unborn child. Information about the safety of the use of antidepressants is therefore important for the healthcare professional when making these decisions.

We conducted a meta-analysis combining the evidence for the association between the use of antidepressants in the first trimester of pregnancy and the risk of miscarriage. Eligible studies were identified by a search in the PubMed and Embase databases from 2000 to February 2021, based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (online supplemental appendix).15 The combined effect was estimated with the random effects model and Episheet software (figure 3).16

Request permission

Figure 3

Forest plot showing risk ratio and 95% confidence interval (CI) for each study included in the meta-analysis. A random effects model was used. Computing measures of heterogeneity (τ²=0.039 and Q=223.28) showed low heterogeneity apart from the study by Johansen, but removing this study did not change the overall estimate of risk ratio substantially

The weighted estimate for the risk ratio of miscarriage in those who received treatment with antidepressants was 1.41 (95% confidence interval 1.22 to 1.63). To evaluate if this estimate is potentially biased by unmeasured confounding, we applied the E value, which we estimated at 2.17. Thus the unmeasured confounders must be associated with both use of antidepressants and risk of miscarriage by a risk ratio of at least 2.17 to fully explain away the observed risk ratio of 1.41. Tobacco and alcohol are prevalent and plausible confounders associated with both miscarriage and depression and hence with the use of antidepressants (figure 1).17–20 We used these confounders in our application of the E value method. Petersen et al estimated that the risk of substantial (>35 units/week) alcohol use and use of antidepressants was RR_EU=10.25.18 Sundermann et al found that alcohol use (>35 units/week) increased the risk of miscarriage (RR_UD=3.1).17 Because both estimates are stronger than the calculated E value, substantial alcohol use could explain the observed association between the use of antidepressants and risk of miscarriage.

Based on data from Johansen et al, we estimated that the association between smoking and use of antidepressants was RR_EU=2.06.21 Because the risk ratio was <2.17 (the E value), smoking could not fully explain the association when applying the E value method. By using the joint bounding factor, however, smoking could still explain the observed association if the association between smoking and miscarriage is sufficient. Figure 4 shows the different combinations of RR_EU and RR_UD that jointly would have the minimum strength required to explain the association.

Request permission

Figure 4

Different combinations of RR_EU and RR_UD that jointly would have the minimum strength required to explain the observed association in the meta-analysis. RR_EU=strength of association between the unmeasured confounder and exposure; RR_UD=strength of association between the unmeasured confounder and outcome

Smoking had an RR_EU value of 2.06, which requires RR_UD=2.3 to fully explain the association. Pineles et al found an increased risk of miscarriage associated with smoking of 1.32.19 Because this value is below the required RR_UD of 2.3, smoking is unlikely to explain the observed risk ratio of 1.41 on its own. The E value can similarly be applied to the results of any individual study.

Discussion

Confounded estimates could make safe treatments seem unsafe and might potentially limit treatment options for pregnant women. Equally problematic is that treatments might seem safe when they are not, hindering the use of safer alternatives. In our example, the weighted estimate from the meta-analysis suggested that the use of antidepressants in pregnancy increased the risk of miscarriage. Hence the treatment would be considered unsafe to use during pregnancy. The use of the E value method, however, suggested that the association could be explained away by unobserved confounding, plausibly attributable to alcohol consumption. In other words, an analysis incorporating adequately measured alcohol consumption could give a null finding. The E value method also showed that confounding by smoking could not explain the observed association. Tools helping to understand the potential role of confounding are therefore important when analysing observational studies.

The ultimate goal of causal inference is removal of confounding. In a particular dataset, total confounding might come from a few or many confounders. Unmeasured confounding is a combination of unknown and residual confounding.22 The E value is a relatively new addition to the strategies for assessing unmeasured confounding in observational studies. Although the E value does not allow partitioning of confounding sources, it provides an estimate of the joint unmeasured confounding, needed for causal inference. Another advantage of the E value is that no assumptions are needed about the structure of the unmeasured confounding, and it is simple to calculate from the risk ratio. The E value is therefore easy to implement as a sensitivity analysis in observational studies. Knowledge of the subject matter should help identify whether any specific unmeasured confounders can plausibly cause the potential confounding.

The E value has limitations. A low E value does not always mean that an unmeasured confounder (eg, the smoking example) could fully explain the association but possibly only to some extent. Similarly, a high E value does not always rule out unmeasured confounding. For example, confounding by indication might strongly confound an association even when the E value is high. Some unmeasured confounders (eg, those with a low prevalence) that fulfil the requirements of the E value might not explain the observed effect. Thus the prevalence of a particular confounder should be considered carefully. Moreover, the E value deals with confounding that inflates the magnitude of an association and cannot help evaluate confounding that masks a true association.22 Another important caveat, applicable to any setting, is that any estimate of RR_EU and RR_UD used in computing the E value is itself subject to an assumption of no bias. Finally, confounding is not the only source of bias in epidemiological studies. Therefore, the results of an E value analysis should be part of a series of sensitivity analyses addressing all threats to validity.