Changes in medication safety indicators in England throughout the covid-19 pandemic using OpenSAFELY: population based, retrospective cohort study of 57 million patients using federated analytics

Study design

We conducted a retrospective cohort study using general practice primary care electronic health record data from all GP practices in England, supplied by the electronic health record vendors TPP and EMIS.

Data source

Primary care records managed by the GP software providers TPP and EMIS are available in OpenSAFELY, a data analytics platform created by our team with the approval of NHS England to address urgent covid-19 research questions (https://opensafely.org). OpenSAFELY provides a secure software interface allowing the analysis of pseudonymied primary care patient records from England in near real-time within the electronic health record vendor’s highly secure data centre. This interface avoids the need for large volumes of potentially disclosive, pseudonymised patient data to be transferred off-site. Therefore, in addition to other technical and organisational controls, any risk of re-identification is minimised. Similarly pseudonymised datasets from other data providers are securely provided to the electronic health record vendor and linked to the primary care data. The TPP dataset analysed within OpenSAFELY (hereafter OpenSAFELY-TPP) is based on 24.2 million people registered with 2546 GP surgeries using the TPP SystmOne software. The EMIS dataset analysed within OpenSAFELY (hereafter OpenSAFELY-EMIS) is based on 32.6 million people registered with 3821 GP surgeries using EMIS. These datasets contain pseudonymised data such as coded diagnoses and physiological parameters, including blood test results requested by the practice. OpenSAFELY makes extensive use of electronic health record data regarding primary care prescriptions in the NHS in England. Briefly, health professionals who are able to write prescriptions in primary care in the NHS in England include GPs, suitably qualified nurses, pharmacists, and physiotherapists. With very few exceptions, every prescription written by these prescribers is recorded within the patient’s record in general practice clinical systems. These prescriptions are then dispensed by a dispensing service commissioned by the NHS, usually a community pharmacy or dispensing doctor in rural locations. No free text data are included. Further details can be found later in the information governance and ethical approval sections. More information about the platform is available on OpenSAFELY's website; in our review for the UK Government's Department of Health and Social Care16; and in our previous publications.13

Study population

We included patients who were: alive, aged 18-120 years, registered with an OpenSAFELY-TPP or OpenSAFELY-EMIS practice, and recorded as at risk of at least one potentially hazardous prescribing indicator. A patient was considered as being at risk if they were categorised into at least one of the PINCER indicator denominators as defined by the SNOMED-CT)⇓,¹⁷ ⇓NHS dictionary of medicines and devices (dm+d) codes,17 and associated logic developed by PRIMIS for the PINCER programme, as assessed on the first of each month between 1 September 2019 and 1 September 2021, inclusive. This time period was chosen to adequately cover the period of service disruption onset and subsequent service recovery due to the covid-19 pandemic in the UK.

Study measures

Definitions of the hazardous prescribing indicators are described in box 1. The percentage for each indicator is formed of a numerator, which captures patients deemed by the indicator to be at risk of a potentially hazardous prescribing event and a denominator, which captures all patients for which assessment of the indicator is clinically meaningful. Higher indicator percentages represent potentially poorer performance on medication safety (online supplemental file 1). Indicators belong to one of three groups: those associated with gastrointestinal bleeds; those associated with cautioned medications; and those associated with blood test monitoring.

We specified each indicator in analytical code using PRIMIS SNOMED-CT codelists17 and the OpenSAFELY framework. We generated the numerator and denominator for each indicator every month between 1 September 2019 and 1 September 2021, and then calculated monthly percentages for each practice. For indicators assessing numerical values, only unambiguous results were used in the calculation of indicator percentages (eg, an estimated glomerular filtration rate value of >30 was considered ambiguous for an indicator requiring the identification of patients with a rate of <45). Note that this functionality was not available in OpenSAFELY-EMIS at the time of the study, therefore, results for the chronic renal failure and NSAID indicator are reported for OpenSAFELY-TPP practices only.

The monthly indicator percentages were summarised as deciles and presented as decile charts across all practices each month. We also calculated the mean rate across practices in quarter 1 (Q1) 2020 and 2021 for each indicator as well as total counts of the numerator and denominator for each indicator across the two years. Note that in these cumulative data, repeated events will be counted for each month the event occurs (eg, if a patient with heart failure is prescribed an oral NSAID in two separate months, this action is represented as two separate events). Across this period, we also calculated the ratio of hazardous prescribing events to unique patients who had those events (to give an indication of the extent of repeated hazardous prescribing) and the number and percentage of practices with at least one instance of potentially hazardous prescribing at any point across the period.

Each blood test monitoring indicator has an associated monitoring window (eg, lithium concentrations are required to be checked within three months). Should no action have been taken to rectify covid-19 related delays, then 100% of patients will have had delayed blood test monitoring by the end of the relevant monitoring window. For each blood test monitoring indicator, we have calculated the projected month of maximum impact from the onset of covid-19 related disruption in March 2020 (ie, June 2020 for the methotrexate and lithium monitoring indicators, September 2020 for the amiodarone monitoring indicator, and May 2021 for the ACE inhibitor monitoring indicator). Each decile plot has been annotated to indicate when this projected date of maximum impact would have occurred.

Software and reproducibility

For data management and analysis, we used the OpenSAFELY software libraries and Python (Python 3.8). A federated analysis involves carrying out patient level analysis in multiple secure datasets, then later combining them. Codelists and code for data management and data analysis were specified once using the OpenSAFELY tools; then transmitted securely to the OpenSAFELY-TPP platform within TPP’s secure environment, and separately to the OpenSAFELY-EMIS platform within EMIS’s secure environment, where they were each executed separately against local patient data. Summary results were then reviewed to assess the potential for patient or practice re-identification before being released and combined for the final outputs. All code for the OpenSAFELY platform for data management, analysis, and secure code execution is shared for review and re-use under open licences at github.com/OpenSAFELY. Decile charts were drawn using Seaborn and matplotlib.

Information governance

NHS England is the data controller for OpenSAFELY-EMIS and OpenSAFELY-TPP; EMIS and TPP are the data processors; all study authors using OpenSAFELY have the approval of NHS England. This implementation of OpenSAFELY is hosted within the EMIS and TPP environments, which are accredited to the ISO 27001 information security standard and are are Data Security and Protection Toolkit compliant.18

Patient data have been pseudonymised for analysis and linkage using industry standard cryptographic hashing techniques and all pseudonymised datasets transmitted for linkage onto OpenSAFELY are encrypted. Access to the platform is via a virtual private network connection, restricted to a small group of researchers; the researchers hold contracts with NHS England and only access the platform to initiate database queries and statistical models. All database activity is logged and only aggregate statistical outputs leave the platform environment following best practice for anonymisation of results, such as statistical disclosure control for low cell counts.19

The OpenSAFELY research platform adheres to the obligations of the UK General Data Protection Regulation (known as GDPR) and the Data Protection Act 2018. In March 2020, the Secretary of State for Health and Social Care used powers under the UK Health Service (Control of Patient Information) Regulations 2002 (known as COPI) to require organisations to process confidential patientinformation for the purposes of protecting public health, providing healthcare services to the public, and monitoring and managing the covid-19 outbreak and incidents of exposure; thereby removings the requirement for patient consent.20 This regulation was extended in November 2022 for the NHS England OpenSAFELY covid-19 research platform.21 In some cases of data sharing, the common law duty of confidence is met using, for example, patient consent or support from the Health Research Authority Confidentiality Advisory Group.22

Taken together, these provide the legal bases to link patient datasets on the OpenSAFELY platform. GP practices, from which the primary care data are obtained, are required to share relevant health information to support the public health response to the pandemic, and have been informed of the OpenSAFELY analytics platform.

Patient and public involvement

We have involved patients and the public in various ways: we developed a public website that provides a detailed description of the platform in language suitable for a lay audience; we have participated in two citizen juries exploring public trust in OpenSAFELY23; we are co-developing an explainer video; we have patient representation who are experts by experience on our OpenSAFELY Oversight Board; we have partnered with Understanding Patient Data to produce lay explainers on the importance of large datasets for research; we have presented at various online public engagement events to key communities (eg, Healthcare Excellence Through Technology; Faculty of Clinical Informatics annual conference; NHS Assembly; HDRUK symposium); and more. To ensure the patient voice is represented, we are working closely to decide on language choices with appropriate medical research charities (eg, Association of Medical Research Charities). We will share information and interpretation of our findings through press releases, social media channels, and plain language summaries.

Indicators associated with gastrointestinal bleeding

The six indicators of potentially hazardous prescribing in relation to gastrointestinal bleeds decreased across the study period with no evidence of an increase in hazardous prescribing as a result of covid-19 induced service disruption (figure 1 (top two rows), OpenSAFELY-TPP only and OpenSAFELY-EMIS only decile charts are provided in online supplemental figures 1A and 2A, respectively). The mean percentage of warfarin/DOAC and NSAID was 1.39% in Q1 2020; the equivalent rate in 2021 was 1.18%. Similarly, the percentage of peptic ulceration and antiplatelet reduced from 4.24% in Q1 2020 to 3.85% in Q1 2021 (table 2). This improving trend was only marginally impacted during the months of the greatest service disruption due to the covid-19 pandemic. Gastrointestinal bleed indicators had lower incidence among some practices than others. For example, the percentage of practices that had at least one hazardous prescribing event for the aspirin and other antiplatelet indicator was 90.73% and for age ≥65 years and NSAID indicator was 99.01%.

Request permission

Indicators associated with cautioned medications

Time trends and variation for all three cautioned medication indicators are presented in figure 1 (bottom row) (OpenSAFELY-TPP only and OpenSAFELY-EMIS only decile charts are provided in online supplemental figures 1b and 2b, respectively). No evidence suggests that the covid-19 related disruption to service delivery had any substantial effect on compliance for all indicators associated with cautioned medications. Notably lower incidence of practices that had at least one hazardous prescribing event were reported for the chronic renal failure and NSAID indicator and for the heart failure and NSAID indicator with 87.08% and 88.30% of practices, respectively, reporting at least one hazardous prescribing event; in comparison, the asthma and beta blocker indicator had 99.06% of practices that had at least one hazardous prescribing event.

Indicators associated with blood test monitoring

All blood test monitoring indicators exhibited an increase in delayed monitoring immediately after the onset of covid-19 (May-July 2020; figure 2). These increased rates showed considerable recovery by August-September 2020, in the case of lithium and no level recording (31.47% Q1 2020 v 38.44% Q1 2021), methotrexate and no full blood count (18.64% v 22.73%), and methotrexate and no liver function test (19.62% v 23.27%). The amiodarone and no thyroid function test indicator had similar results (36.20% v 39.23%), although the initial recovery period after covid-19 extended into October-November 2020 (table 2). As with the other groups of indicators, the incidence among practices is high, although this value varies by indicator: incidence varies from 92.85% (lithium and no level recording) to 98.99% (angiotensin-converting enzyme inhibitors or loop diuretic, no blood tests).

Request permission

The indicator of angiotensin-converting enzyme inhibitors or loop diuretic and no blood tests had a noticeably different covid-19 response pattern than the other blood test monitoring indicators. Here,the monitoring worsened steadily over a longer period of time, increasing from a mean of 5.16% to 12.14% between Q1 2020 and Q1 2021, and began to recover in June 2021. The assessment window for this indicator is substantially wider than the windows for the other blood test monitoring indicators: within 15 months of prescription compared with three months for lithium and methotrexate or six months for amiodarone.

Summary

Despite substantial barriers to the delivery of primary care during the covid-19 pandemic, good performance was maintained across a diverse range of widely evaluated and nationally adopted indicators of safe prescribing. Delays were evident in delivering some medication related blood test monitoring within the time window specified in the safety measure; especially for those blood tests where the time window for compliance is already very long, and tests infrequent (specifically the angiotensin-converting enzyme inhibitor or loop diuretic indicator). However, all indicators exhibited considerable recovery by the end of the study period.

Strengths and limitations

This study has a range of strengths. Compared with other routes of access to primary care data, OpenSAFELY offers a more complete coverage of more patients with greater controls on security and complete transparency in terms of the method and reproducibility. Previous audits for compliance with PINCER or similar measures and indicators in primary care rely on manual audit within a practice, or analyses of data downloaded from a group of practices. By contrast, OpenSAFELY executes analyses in a secure environment inside the electronic health record provider data centre, across the full set of all structured data in the GP record including all tests, prescriptions, diagnostic codes, and referrals. Additionally, although the underlying GP data are stored in two very different settings (TPP and EMIS), PINCER indicators were described for almost all GP practices (about 99%) in England for the first time by use of a single analysis in OpenSAFELY. The necessary variables and analyses were defined once, then executed in each setting identically, with the outputs aggregated afterwards, in a process known as federated analytics. Overall, this national platform is uniquely able to capture the patient journey for 57 million people in England while prioritising patient privacy.

Another strength is the transparency and reproducibility of the analysis. OpenSAFELY openly shares on GitHub all code for the platform, data curation, and analysis, from raw data to completed output. These data are in standard formats for scientific review and efficient re-use under open licences by all.

Additionally, the indicators for each safety behaviour are robust. All eligible patients and targeted clinical safety behaviours were developed for the national PINCER medication safety programme. This programme has been extensively peer reviewed and evaluated throughout the NHS over many years, with strong support from clinicians and commissioners.

We also note some limitations. Our results are only descriptive in nature: we have not attempted to statistically assess the extent to which indicator rates changed during the period of service disruption after the onset of covid-19, or the extent to which prepandemic rates were recovered. Furthermore, our study period does not allow for consideration of a time before the pandemic in which variation in indicator rates over time could inform such statistical hypothesis testing. Finally, we acknowledge that our data will only include prescriptions and test results carried out in primary care, or those in secondary care that are returned to GPs as structured data. As such, these data might not include test results communicated by letter or phone (eg, tests requested while a person is in hospital or psychiatric outpatients). However, this method is aligns with others already used in the national PINCER programme to evaluate compliance with the targeted safety behaviours using primary care data alone.

Comparison of existing literature

A systematic review of healthcare usage during the pandemic, encompassing 81 studies across 20 countries, found that healthcare use (eg, visits, admissions, diagnostics, and therapeutics) reduced by 37% during the pandemic, highlighting a substantial reduction in April-May 2020.24 WHO also identified substantial disruption to countries' healthcare capacity for non-communicable diseases in a rapid assessment in May 2020.25 A population based cohort study conducted using the OpenSAFELY platform reported that clinical activity in relation to blood tests declined in the months after covid-19 onset, but also reported recovery of these same tests by September 2020.8 These findings concord with our observations of the blood test monitoring indicators, where delays were substantial in the same period of time. Interrupted service delivery leading to reduced NSAID prescriptions after acute presentations might also explain the temporary reduction of the first gastrointestinal bleed indicator (prescription of an oral NSAID, without co-prescription of an ulcer healing drug, to a patient of ≥65 years) in April-July 2020. This finding is supported by data for this period in OpenPrescribing26 and lower than predicted rates of prescribing for naproxen and ibuprofen in this period.27 Elsewhere, we have found evidence of prioritisation of anticoagulant services, with blood tests to manage high risk anticoagulants being prioritised during the initial stages of the covid-19 pandemic.8 Data from our study also suggest that prescribing in relation to anticoagulants is a priority, with all gastrointestinal bleed indicators being unaffected, and continuing to decline, after covid-19 onset.

In the early stages of the pandemic, in recognition of the increased risk of medication related harm during the covid-19 pandemic, NHS England and local Clinical Commissioning Groups (known as CCGs) revised guidance regarding blood test monitoring. They extended the recommended monitoring window for some patient populations (eg, in relation to lithium28 and methotrexate29) or advised to monitor blood for lower risk medications if possible (eg, ACE inhibitors30), if clinically safe to do so. Some evidence in our data suggests that practices did adopt this revised guidance, with postrecovery blood test monitoring often less than prepandemic levels (particularly in the case of methotrexate and lithium).

Implications for policy and research

The variation in service recovery observed in the blood test monitoring indicators might partly be due to the assessment window for each indicator, and clinicians prioritising urgent work during the pandemic. For example, the protracted recovery of the ACE inhibitor monitoring was possibly due to primary care services proactively prioritising monitoring of higher risk prescriptions, such as methotrexate, so as to minimise the impact of service disruption on patient care. The systems around the monitoring of high risk drugs (eg, clinical system alerts) also probably contributed towards expedited recovery of the other blood test monitoring indicators, particularly in the case of lithium and methotrexate. The decile chart for this indicator starts to plateau well before the ‘worst case scenario’ time point, suggesting that most primary care providers successfully implemented recovery programmes in this clinical domain. Further areas for research include using innovative change detection methods31 to ascertain practice level features that affect recovery and resilience in the context of service disruption to inform WHO and NHS England’s recommendations to build back better.

The potential impact of this analysis for data usage in the NHS is considerable. Historically, as a result of practical and privacy challenges around accessing GP data at scale. Each practice participating in the PINCER programme has been required to manually execute the necessary computerised searches before individually uploading their results for central oversight; in some centres, data for a group of practices and patients can be downloaded and analysed in larger volumes. This manual approach introduces delays and increases the resource cost of monitoring safety. Using the OpenSAFELY framework, we were able to execute a single analysis for almost the entire population of England in near real time, while leaving data in situ. This approach is efficient: analyses can be easily updated, and expanded, because they are executed in a single framework from re-executable code. Patient trust is also preserved: OpenSAFELY was the single most highly trusted covid-19 data project in a rigorous Citizens Jury sponsored by the NHS and the National Data Guardian.23 Furthermore, the additional data that are also securely accessible through the OpenSAFELY tools can be used to describe PINCER indicators in fine grained demographic or clinical subpopulations. These tools can facilitate near real time audit and feedback in the context of rapidly evolving pressures on the health service and are readily extendable to other clinical and challenges.

More broadly, this analysis shows the benefits of collaborative working with shared open source code in the NHS: it built on the work of PRIMIS in establishing and then publicly releasing the full code for a set of rigorously tested medication safety indicators; and then implemented the open code from PINCER in the open source framework of OpenSAFELY, to assess a critical public health question on a national scale. Open working. as shown here. is strongly supported by senior stakeholders in multiple sectors16 32 33 and can bring substantial benefits. Open working facilitates efficient re-use of previous technical work; ensures fidelity through the consistent implementation of data curation and analysis across all organisations; supports complete reproducibility; enables error checking by all interested parties; and facilitates public and professional trust.

Conclusion

NHS GP data can be analysed at national scale to generate insights on service delivery. Potentially hazardous prescribing was largely unaffected by covid-19 in a dataset of 57 million patients’ full primary care health records in England.