Article Text
PDF
Abstract
Objectives To assess whether recommendations of individually oriented lifestyle interventions (IOLIs) in guidelines from the National Institute for Health and Care Excellence (NICE) were underpinned by evidence of benefit, and whether harms and opportunity costs were considered.
Design Cross sectional survey.
Setting UK.
Data sources NICE guidelines and supporting evidence.
Eligibility criteria All NICE pathways for IOLI recommendations (ie, non-drug interventions that healthcare professionals administer to adults to achieve a healthier lifestyle and improve health) were searched systematically on 26 August 2020. One author screened all retrieved pathways for candidate guidelines, while a second author verified these judgments. Two authors independently and in duplicate screened all retrieved guidelines and recommendations for eligibility, extracted data, and evaluated the evidence cited and the outcomes considered. Disagreements were noted and resolved by consensus.
Results Within 57 guidelines, 379 NICE recommendations were found for IOLIs; almost all (n=374; 99%) recommended the lifestyle intervention and five (1%) recommended against the intervention. Of the 379 recommendations, 13 (3%) were supported by moderate or high certainty evidence of a beneficial effect on patient relevant outcomes (n=7; 2%) or surrogate outcomes (n=13; 3%). 19 (5%) interventions considered psychosocial harms, 32 (8%) considered physical harms, and one (<1%) considered the opportunity costs of implementation. No intervention considered the burden placed on individuals by these recommendations.
Conclusion Few NICE recommendations of lifestyle interventions are supported by reliable evidence. While this finding does not contest the beneficial effects of healthy habits, guidelines recommending clinicians to try to change people’s lifestyle need to be reconsidered given the substantial uncertainty about the effectiveness, harms, and opportunity costs of such interventions.
- health policy
Data availability statement
Data are available in a public, open access repository. All data is available in appendices, which are published in the Open Science Framework (osf.io/42juh).
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
The beneficial effects of healthy lifestyle habits are uncontested
Health professionals have targeted individuals with unhealthy habits through lifestyle interventions
It is uncertain whether these interventions are an effective and feasible way to achieve healthier lifestyle habits
WHAT THIS STUDY ADDS
The National Institute for Health and Care Excellence recommends that clinicians implement individually oriented lifestyle interventions for a large proportion of the UK population
These recommendations were rarely underpinned by reliable evidence of a beneficial effect, with minimal if any consideration of harms or burdens placed on the individual or of opportunity costs to the healthcare system
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE, OR POLICY
These findings do not contest the beneficial effects of healthy lifestyle habits, but they have implications for guideline panels when they make recommendations about lifestyle interventions, and how these should be sensibly implemented in a healthcare system with constrained resources
Introduction
Unhealthy lifestyles, such as smoking, physical inactivity, and an unhealthy diet, are associated with common, serious diseases that place a substantial burden on individuals and the healthcare system.1 Communities have responded to this through, for example, indoor smoking bans, taxes on sugary soft drinks and tobacco, construction of bicycle paths, and prohibitions of trans fatty acids.2 Over the past decades, however, a strategy focused on individuals rather than structural incentives has gained momentum.3 4 An individually oriented strategy often relies on healthcare professionals intervening to promote healthy lifestyles through, for example, dietary advice for people with obesity or motivational interviewing to promote physical activity. Policymakers prioritise such individually oriented lifestyle interventions (IOLIs) in guidelines for a wide range of conditions such as hypertension,5 type 2 diabetes,6 obesity,7 and early stages of chronic obstructive pulmonary disease.8 Uptake and adherence to such IOLI recommendations are sometimes promoted through financial incentives directed at healthcare personnel9 and individuals.10
The beneficial effects of healthy lifestyle habits are uncontested. However, for these interventions to work, the intervention must improve the likelihood of both the immediate goal (eg, behaviour change and weight loss) and the ultimate effect on health (eg, prevent ischaemic heart disease). In other words, just because physical exercise is beneficial it does not mean that advising individuals to exercise more will also be beneficial. Furthermore, to target individuals with unhealthy habits might cause harm,11 12 and given the large number of individuals with unhealthy lifestyle habits, an individualised approach also has the potential for opportunity costs within health systems owing to the considerable time required. Consequently, it is uncertain whether the healthcare system is the optimal place to achieve healthier lifestyle habits in general populations.
As the primary source of guidance for the NHS, the National Institute for Health and Care Excellence (NICE) publishes various guidelines for clinical practice. NICE recommendations for IOLIs are one method by which clinicians are urged to intervene to change a person’s unhealthy lifestyle. This study aimed to assess whether recommendations of IOLIs in guidelines from NICE were underpinned by evidence of benefit, and whether harms and opportunity costs were reported. We chose to assess guidelines from NICE as these are widely respected for their methodology and are likely to represent a best case scenario.
Methods
A study protocol was registered in the Open Science Framework (osf.io/42juh) on 29 November 2020, after our search was performed but before any data extraction or analysis began.
Study design, definitions, and eligibility criteria
In this cross sectional survey, we systematically reviewed all recommendations of IOLIs provided in NICE guidelines. An IOLI was defined as any non-drug intervention provided by healthcare personnel to any adult (over age 18 years) with the aim to improve health by changing one or more of the following lifestyle habits: physical activity or exercise, diet, tobacco use, and alcohol use. Examples of such interventions are motivational interviewing to promote physical activity or healthy diet; prescriptions for physical exercise; enrolment in weight loss or diet programmes; or brief advice from healthcare personnel on diet, exercise or physical activity, tobacco cessation, or alcohol use. Interventions targeting specific symptoms (eg, exercises to treat low back pain or urinary incontinence, dietary advice to reduce gallbladder attacks, treatment of substance abuse disorders) or those delivered outside clinical settings (eg, workplace) were not eligible.
Data sources and selection process
We systematically searched NICE Pathways, an online resource, on 26 August 2020 to identify all IOLI recommendations using the following search string: lifestyle OR physical activity OR exercise OR diet OR smoking OR alcohol OR tobacco. We searched NICE Pathways to optimise the sensitivity to detect IOLI recommendations in the guidelines. One author (MJ) screened all NICE Pathways for any candidate IOLI recommendations, while a second author (MR) verified these judgments. Two authors (MJ and MR) then independently, and in duplicate, assessed each guideline for eligibility. Any disagreements were resolved by discussion with a third author (LA). Search results and reasons for exclusions are available in online supplemental appendix 1,13 and a PRISMA flowchart is available in online supplemental appendix 3.
Supplemental material
Data extraction and analysis
Two authors (MJ and LA), working in duplicate and independently, extracted data from included guidelines into a standardised and prepiloted data extraction form. Disagreements were solved through discussion with a third author and consensus. We estimated interobserver agreement using raw agreement and chance adjusted agreement using the phi (φ) statistic.14 Because the agreement levels were very high (>92% for all judgments, mean 96%), we report only the raw agreement levels, which are easier to interpret.
The full data extraction form, including all extracted data, is available in online supplemental appendix 2.13 The information extracted included (but was not limited to) the targeted lifestyle habit; the targeted population; the nature of the cited evidence to support the recommendation (see definition below); the reported certainty of the evidence to support the recommendation; whether psychosocial harms, physical harms, treatment burden, and opportunity costs were considered in any way; and whether the specific aspects related to opportunity costs were considered or reported. These aspects included the time needed to perform the intervention per individual, the relevant category of healthcare personnel, the eligible fraction of the population, the human resources needed to fully implement the intervention, the financial resources needed to fully implement the intervention, and a potential for redistribution of resources from other interventions or from vulnerable to privileged groups.
We planned to rely on the judgments made by the guideline authors about the certainty of the evidence (ie, GRADE—grading of recommendations assessment, development, and evaluation).15 However, when piloting the data extraction form, we found that the certainty of the evidence was often either not assessed or not reported, unclear, or inconsistently reported; and that the relevance of the evidence for the specific recommendation was often unclear (eg, the certainty was assessed for studies investigating the effect on biochemical markers for participants who changed behaviour—while the recommendation was advice to change behaviour). We concluded that to only extract the certainty of the evidence as reported in the guidelines would provide insufficient information.
We considered performing a GRADE assessment for each recommendation but found that the necessary information was often not available in the guidelines. We therefore chose instead to include the following in our data extraction (referred to as the nature of the cited evidence to support the recommendation): whether no studies were cited to support the recommendation; whether the only cited studies were observational studies; whether the cited studies differed importantly from the recommendation in terms of the intervention, the population or the outcome; and whether the cited studies investigated the recommended intervention but suggested only a short term beneficial effect or no beneficial effect. The choice of these categories was based on the findings during piloting, as well as existing knowledge within the author team. We judged the cited evidence to be of questionable relevance if it referred only to observational studies of correlation between a lifestyle habit and worse outcomes; studies suggesting no beneficial effect of the recommended intervention; or evidence that was not directly relevant to either the recommendation’s target population, the intervention, or the intended outcome (see examples in the results section below).
We followed the guidance provided by GRADE16 to judge whether an outcome was a surrogate or a patient relevant outcome. Of note, we classified weight loss, smoking cessation rates, and increases in physical activity to be surrogate outcomes (see limitations section in the discussion section below).
Regarding psychosocial harms, physical harms, and treatment burden, we judged these harms to be considered if they were mentioned in any way in the guideline. For example, if guideline authors reported that no adverse effects were reported in the cited studies, we would judge that this guideline considered harms. Further, for all guidelines who considered harms beyond stating that the cited studies did not report any harms, we extracted how harms were considered and list the exact quotes in online supplemental appendix 3.
Descriptive analyses were used to summarise the results. Online supplemental appendix 2 presents all extracted data.13 The included NICE evidence syntheses supporting the recommendations are stored as pdf documents and are available on request.
Patient and public involvement
A patient advocate (HB) has been involved in the design, conduct, reporting, and dissemination plans of this research as an author.
Results
From 635 NICE Pathways available, we identified 57 NICE guidelines that made a total of 379 IOLI recommendations (median 3 recommendations per guideline, range 1-10). Of the 57 included guidelines, 13 (23%) were directly focused on lifestyle (eg, weight management and lifestyle services for overweight or obese adults; physical activity and brief advice in primary care). The remaining 44 guidelines (77%) were for specific conditions (eg, patients with breast cancer, acute coronary syndromes, age related macular degeneration, and chronic kidney disease) and contained one or more IOLI recommendations. In about a quarter of the included guidelines, the population eligible for the recommendations constituted at least a quarter of the general UK population (see online supplemental appendix 3).
Characteristics of IOLI recommendations
Of the 379 included IOLIs, 112 (30%) were on tobacco cessation, 74 (20%) on diet, 60 (16%) on physical activity or exercise, 21 (6%) on alcohol use, and 111 (29%) recommended multiple lifestyle interventions (eg, both diet and physical activity). Almost all (n=374; 99%) recommended the lifestyle intervention and five (1%) recommended against it. Eight (2%) recommended the intervention only when judged appropriate by healthcare personnel, while 366 (98%) recommended the intervention for all applicable patients (eg, dietary advice for all people with a body mass index over 30).
Evidence underpinning the recommendations
Table 1 and figure 1 describes the evidence cited in support of the IOLI recommendations. Inter-reviewer agreement was >92% for all judgments (mean 96%). Of the 379 recommendations, 123 (32%) did not cite any supportive evidence. For 181 (48%) recommendations, the evidence cited was judged to be of questionable relevance to support a beneficial effect. Of these recommendations, 71 (19%) referred only to evidence from observational studies (eg, people with obesity have worse outcomes than people without obesity); 61 (16%) referred to evidence that was not directly relevant to either the recommendation’s target population (eg, population of the research evidence was pregnant women only while the target population of the recommendation was all women of fertile age) or to the intervention (eg, research evidence was on intensive and complex interventions while the intervention in the recommendation was brief advice), or the cited evidence had limited relevance to support that the recommended intervention will lead to the intended outcome (eg, evidence suggested weight loss leads to beneficial outcomes while the recommendation assumes dietary advice would cause weight loss); and 49 (13%) cited evidence suggesting no beneficial effect of the recommended intervention but recommended it anyway (eg, studies of brief advice on physical activity did not show a clinically relevant effect on the measured outcomes).
Evidence cited to support 379 recommendations of individually oriented lifestyle interventions (IOLI) in 57 NICE guidelines included in study
Evidence cited to underpin a beneficial effect of the recommended interventions.
Of the 379 recommendations, 75 (20%) referred to research evidence suggesting a beneficial effect of the recommended lifestyle intervention. Of these 75 recommendations, 74 (20% of total) reported a beneficial effect on surrogate outcomes, and 14 (4% of total) reported a beneficial effect on patient relevant outcomes (table 1). Of the 74 recommendations who reported an effect on surrogate outcomes, 13 reported a beneficial effect based on high or moderate certainty evidence (3% of total), of which seven became non-significant with long term follow-up. Of the 14 recommendations who reported an effect on patient relevant outcomes, seven reported a beneficial effect based on high or moderate certainty evidence (2% of total), of which three became non-significant with long term follow-up.
Consideration of harms for the individual
Possible psychosocial harms from the intervention were considered in 19 (5%) recommendations, of which 13 did so only by including quality of life as an outcome (ie, harms were not specifically considered); of these, 10 did not find any studies reporting on quality of life. Possible physical harms were considered in 32 (8%) recommendations, of which 31 did so only by stating that no adverse events were reported in the cited evidence. None of the 379 recommendations considered the treatment burden placed on the patient because of the recommendation. Altogether, three recommendations considered any harm to the individual beyond what was reported in the cited studies. All of these recommendations are listed in online supplemental appendix 3.
Consideration of clinical opportunity costs
For one of the 379 recommendations, the opportunity costs for the healthcare system of implementing IOLIs was considered (online supplemental appendix 3). None of the recommendations considered redistribution of human and financial resources from other interventions with greater beneficial effect, or from vulnerable to privileged groups. Likewise, none of the recommendations presented estimates of the proportion of the population for which the recommendation would be applicable, or the human or financial resources that would be needed if the intervention were to be fully implemented in eligible individuals.
Discussion
We found that NICE recommended IOLIs to a large proportion of the UK population. Of 379 IOLI recommendations included in 57 guidelines, only a minority (1%) recommended against the intervention and very few (3%) were supported by high or moderate certainty evidence of a beneficial effect on patient relevant outcomes or surrogate outcomes. Furthermore, only a small proportion of the IOLI recommendations discussed any potential harms of the interventions (5% mentioned psychosocial and 8% physical harms) or considered the opportunity costs within the healthcare system (<1%), and none considered the burden these interventions place on individuals.
Meaning of the study
The findings of our study do not contest the beneficial effects of healthy lifestyle habits. On the contrary, high certainty evidence support the association between a healthy lifestyle and improved health. However, evidence for beneficial effects of healthy habits should not be confused with evidence for beneficial effects of lifestyle interventions. In other words, just because losing weight is beneficial, it does not mean that advising people to lose weight will also be beneficial because the advice may not help people change behaviour.
Further, our findings do not rule out that IOLIs have beneficial effects, merely that the evidence cited to support recommendations of lifestyle interventions made by one of the most respected guideline institutions in the world is very limited. It could be argued that the need for high certainty evidence to support recommendations of IOLIs is lower than for drug or surgical interventions, since the risk of physical harms is likely to be lower and since the evidence to support beneficial effects of healthy lifestyle habits is uncontested. Yet, other factors should be considered.
At the individual level, to target individuals with unhealthy habits might cause harm.11 12 People with unhealthy lifestyles should arguably be able to see their clinician for a specific health concern without receiving unrequested and unrelated lifestyle advice. The patient’s agenda might not be fully considered in encounters in which the healthcare professional must deliver IOLIs, detracting time from patient centred care. IOLIs can place a substantial burden on the individual and their caregivers, particularly among those who are eligible for multiple interventions and already carry a substantial burden of treatment related to chronic multimorbidity.12 17
At the healthcare system level, the time, energy, and attention needed for healthcare personnel to implement IOLI recommendations must be carefully considered given their existing workload, available resources, and all the other tasks assigned to them. Given the large fraction of general populations around the world that have unhealthy lifestyle habits, and that IOLIs often require considerable clinician time, IOLI recommendations have potential to cause substantial opportunity costs within health systems. At the societal level, the observed global increase in costs of healthcare services draws resources from other societal sectors capable of improving the health and wellbeing of the population,18 19 a situation currently augmented by the covid-19 pandemic. A redistribution of resources from societal interventions to individually oriented interventions within the healthcare system could counteract a rational use of community resources.19 20
Given these concerns, it seems reasonable to expect IOLI recommendations to be supported by evidence that the interventions at least succeed in achieving the change in behaviour on which the health goal is predicated. Yet, behaviour change has generally been shown to be very difficult to achieve for the individual, particularly when living under the influence of political, corporate, and socioeconomic determinants of health.
An additional ad hoc finding of our study was that the rigour by which the beneficial effects of lifestyle interventions was evaluated was inconsistent; in many cases, the cited studies did not investigate the effect of the recommended intervention. The same methodological rigour might currently not be applied to recommendations on lifestyle interventions as for recommendations on other medical interventions. While a limited evidence base is not the fault of the guideline panels, we believe that these findings highlight a need for a closer adherence of guideline panels to frameworks for how to interpret and evaluate evidence when issuing recommendations on IOLIs.21
Based on our findings, we have identified some issues that could help bring the presented aspects into sharper relief: (1) low certainty evidence of potential short term beneficial effects on surrogate outcomes were sometimes assumed to automatically translate into long term beneficial effects on patient relevant outcomes; (2) evidence of a correlation between a lifestyle habit and better outcomes, or evidence of a beneficial effect from behaviour change, were sometimes interpreted as direct evidence that advice on lifestyle would have a beneficial effect (ie, it was assumed that the intervention will make people change behaviour despite no evidence to support this); (3) healthcare personnel’s time was considered (at best) as a financial cost only and not as a finite resource to be carefully prioritised; (4) each recommendation was evaluated separately without considering the implications of implementing all recommendations combined; (5) the resource use required for the recommended intervention was (at best) considered in relation to interventions within healthcare only, and not in relation to other types of societal interventions aimed at improving health and wellbeing of the population.
Limitations and strengths of the study
Our results rely on subjective judgments about the quality and scope of the underlying evidence. Although the inter-reviewer agreement was high (between 92% and 100%, mean 96%; table 1), using duplicate assessors, discussion, and consensus might not have fully controlled for investigator biases. In general, we sought to err on making favourable judgments. For example, we classified a recommendation as considering psychosocial harm when its text simply mentioned that no studies reported on the IOLI’s effect on quality of life.
The variable degree of explicitness of reporting in the guidelines, however, further complicated their evaluation. For example, the relevance of the cited evidence (as well as the GRADE assessments, when available) to evaluate a beneficial effect of the recommended intervention was often questionable. We therefore developed post hoc categories to evaluate the relevance of the cited evidence more fully. Furthermore, because many of the guidelines did not use an explicit method for evidence rating, our judgments could have underestimated the proportion of recommendations that were based on moderate to high certainty evidence. Since only 4% of the recommendations were underpinned by any evidence of a beneficial effect from the recommended intervention on patient relevant outcomes, this prospect seems unlikely.
We classified weight loss, smoking cessation rates, and increases in physical activity as surrogate outcomes. These outcomes themselves can be argued as patient relevant outcomes (ie, outcomes that patients can experience and value). Yet, even if we were to reclassify them, our conclusions remain unchanged because high to moderate certainty evidence of a beneficial effect on any surrogate outcomes underpinned only 3% of the recommendations.
The central strength of our review was in the complete sample of recommendations from a well respected guideline authority, transparency and adherence to prespecified methods detailed in a registered protocol, and full reporting of all raw data extracted (online supplemental appendices 1–3). Future analyses of the recommendations from other guideline authorities can help determine how our findings can apply to other settings. The high quality of the process followed by NICE to formulate recommendations, however, suggests that our results might represent a best case scenario.
Conclusions
We found that recommendations of individually oriented lifestyle interventions in state-of-the-art guidelines were rarely supported by evidence of a beneficial effect, and that harms and burdens at the individual level, and opportunity costs at the healthcare system level, were not explicitly considered. The findings of our study do not contest the beneficial effects of healthy lifestyle habits. However, guidelines recommending clinicians to try to change people’s lifestyle need to be reconsidered given the substantial uncertainty about the effectiveness, harms, and opportunity costs of such interventions.
Data availability statement
Data are available in a public, open access repository. All data is available in appendices, which are published in the Open Science Framework (osf.io/42juh).
Ethics approval
Not applicable.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @LoaiAlbarqouni
Contributors MJ conceived the idea for this project and drafted the project plan. All authors contributed with important intellectual content to develop the project plan. MJ, MR, and LA performed the screening. MJ and LA performed data extraction, formal analysis, and drafted the manuscript. All authors contributed with important intellectual content through reviewing and editing the manuscript. MJ and LA are guarantors and accept full responsibility for the work and the conduct of the study, had access to the data, and controlled the decision to publish. All authors have accepted the final version of the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. Transparency: The lead author (the guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
Funding This project was partly funded by Swedish Research Council for Health, Working Life, and Welfare (project number 2019-00928). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.