235
British Journal of Health Psychology (2014), 19, 235–239 © 2013 The British Psychological Society www.wileyonlinelibrary.com
Editorial
Health Psychology interventions
There is now widespread acceptance that behaviour plays a key role in the development of many long-term conditions and that behaviour change is essential for both prevention and treatment (Khaw et al., 2008; Mokdad, Marks, Stroup, & Gerberding, 2004). Our discipline has taken important strides forward in theory development and in extending our understanding of the factors that guide intention formation and behavioural enaction. Significant advances are also being made in developing a reliable taxonomy of behaviour and behaviour change techniques (Michie et al., 2013). However, Health Psychology now needs to start to deliver robust evidence that our interventions can lead to lasting changes in behaviour, which in turn, lead to significant health benefits.
Measuring behaviour Psychology is the scientific study of behaviour, but we do not measure behaviour objectively as often as we should. We have tended to focus more on attitudes and intentions rather than on overt behaviour. As Johnston and Dixon (2008) highlight, Health Psychology theories often focus rather more on intrapsychic phenomena – thoughts and emotions – that may determine behaviour, rather than on the behaviour itself. When we do measure behaviour, it is commonly via self-report. Of course, some important outcomes of interventions, such as quality of life, can only be measured by self-report. However, social desirability bias is well recognized, for example alcohol intake is often under-reported. In order to persuade others that our interventions are effective and important, we need to place far greater emphasis on the objective measurement of overt behaviour as a key outcome. We also need to distinguish between interventions where behaviour change is the valued outcome (e.g., increasing physical activity) and where the targeted behaviour is a mediator of the valued outcome (e.g., increasing medication adherence to reduce the risk of stroke). In the latter case, the health outcome will clearly depend on both the effectiveness of the psychological intervention and the strength of the relationship between the targeted behaviour (medication adherence) and the health outcome (stroke).
DOI:10.1111/bjhp.12082
236
Editorial
Sample sizes are often too small Many Health Psychology intervention studies use relatively small samples. By way of example, we recently conducted a simple trial of a two-session intervention to increase adherence to medication in stroke patients, with 30 participants in each arm (O’Carroll, Chambers, Dennis, Sudlow, & Johnston, 2013). Around the same time, one of my physician co-authors on this paper led a simple trial of compression stockings to reduce the risk of DVTs (the CLOTS 3 trial) – he had 1,438 participants in each arm (CLOTS et al., 2013). In terms of scale, we are worlds apart. In Health Psychology, we often use power calculations to justify using relatively small sample sizes. For example, in grant applications, we may state that our sample size will enable us to detect a large effect size (e.g., d = .8). However, to non-psychologists reading such applications, testing for effect sizes like this may arouse suspicion, as there are few medical treatments for long-term conditions that have such large effects. Furthermore, if the study then does not produce a ‘significant’ effect, the psychological intervention may be judged to have ‘failed’. However, in relation to other treatments, the intervention may have produced a really important modest treatment effect, which is not statistically significant as a result of the low sample size. Button et al. (2013) recently reported in Nature Review Neuroscience that small sample sizes undermine the reliability of neuroscience. The same charge can be levelled at Health Psychology intervention studies. We need to engage in more collaborative, multicentre intervention trials which are powered to detect small to medium treatment effect sizes.
How effective are commonly used treatments? Some intensive, complex psychological interventions can produce impressive large treatment effect sizes. However, many less intensive interventions in Health Psychology may produce more modest effects, but we should not be apologetic about this. The general public (and many Health Psychologists?) may be surprised at the modest efficacy of many established medical treatments. This may be because of the way the treatment effects are often marketed, for example ‘Treatment X can reduce the risk of stroke by 20%’. Claims such as this are usually presenting a relative risk reduction (RRR), rather than an absolute risk/response difference (ARD). The ARD is always much lower. The following examples are taken from Leucht, Hierl, Kissling, Dold, and Davis (2012). Statin treatment for reduction in cholesterol levels reduced cardio-vascular events from 18% to 14% (primary and secondary prevention combined), a RRR of 21%, but an ARD of 4%. In the secondary prevention of cardiovascular disease, low-dose aspirin reduced serious cardiovascular events per year from 8.2% to 6.7%, an RRR of 19%, but an ARD of 1.5%. These ARDs are important but are quite small, perhaps lower than many participants taking these medications may believe? Trewby et al. (2002) conducted a seminal study on this topic. They aimed to find the threshold of benefit for a hypothetical cholesterol-lowering drug, below which patients would not be prepared to take the drug. They found that three quarters of participants would not take a drug offering 5% or less absolute risk reduction over 5 years. This is roughly the efficacy of best current medical treatment! The general public’s expectation of benefit from a preventive drug may thus be significantly higher than the actual benefit provided by current drug strategies. To be clear, the preceding section is not a criticism of the effectiveness of medical treatments, but rather it highlights the fact that many established medical treatments are
Editorial
237
modestly effective. It is also worth noting that a recent meta-analysis of the effectiveness of psychotherapy treatment studies for depression in adults concluded that in the highest quality studies, the standardized mean treatment effect was a modest d = .22 (Cuijpers, van Straten, Bohlmeijer, Hollon, & Andersson, 2009). If Health Psychology was clearer about the importance of small to medium treatment effects and more confident in itself as a discipline, then we should be unapologetic in seeking and reporting modest effect sizes in intervention studies. Prentice and Miller (1992) set out clear guidelines when small effect sizes should be considered impressive, for example when the outcome is difficult to influence. If we can manage to achieve small but sustained health behaviour change, this would represent a considerable achievement. Furthermore, many Health Psychology intervention studies simply report pre- and post-treatment effects. We need to provide more convincing evidence of behaviour change sustained over the longer term. More work needs to be conducted on the need and cost-effectiveness of post-intervention ‘booster’ sessions.
Health Psychology Homeopathy A further tension that can emerge is in relation to the cost of Health Psychology interventions. If a positive psychological treatment effect is reported, there may be pressure exerted to see whether it can be delivered more simply and at less expense. To take a hypothetical example, a two- or three-session intervention based on modifying illness and/or treatment perceptions may show a significant health benefit (e.g., Petrie, Cameron, Ellis, Buick, & Weinman, 2002). However, others may argue that this could never be delivered in a cash-strapped National Health Service as it would be far too expensive to deliver, and, they may ask ‘Could this not be delivered more simply, for example in a leaflet?’ This is a dangerous road to follow. Apart from the fact that this would represent a completely different treatment, it is highly likely that any treatment effect would be significantly diluted and potentially lost. This may lead others to conclude that psychological interventions do not work ‘in the real world’. Of course, Health Psychology interventions are often multicomponent, and we need to determine the active ingredients. However, would we ask whether a 10-day course of antibiotics could be reduced to 1 day or renal dialysis from three times a week to once a week to make it simpler and cheaper? We need to collaborate more with Health Economists to estimate the savings to health services that may be achieved via successful behaviour change interventions, properly delivered.
The hypocrisy regarding replication Scientific discovery is based on the refutation of hypotheses and the principle that effects must be robust and replicable. Before we can have confidence in a scientific finding, it should be confirmed independently by others. In reality, we rarely do this. Findings are valued because they are novel, replications are not. It is very rare to see an exact replication of a Health Psychology intervention using the same method and measures. Indeed, many Journals request a cover letter with new submissions, where the author is asked to state how the paper’s results are important and novel. Thus, our journals are largely populated with ‘one-off’, small-scale studies that are never replicated, and we therefore do not know whether most of the findings in Health Psychology are robust. We need to change the mindset in behavioural science where high-quality replications are
238
Editorial
valued. To take a local example, in the United Kingdom, the Research Excellence Framework (REF) dominates academia; it is how we are judged. All academics must submit their four ‘best’ papers for peer review. They are rated as 1*–4* on the criteria of ‘originality, significance and rigour’. Replications are unlikely to score well. Behavioural science, as a cumulative process, is thus being impeded by the failure to recognize the importance of replications, and REF acts a strong disincentive to conducting replications. Furthermore, an intervention is only likely to become widely accepted after being subject to systematic review, and this process requires replication. The Association for Psychological Science (APS) have made a recent welcome development in this area; their journal Perspectives on Psychological Science now invites Registered Replication Reports.
Conclusion As a discipline, we need to collaborate more and conduct larger, multicentre, collaborative, intervention studies, where behaviour is measured objectively. We need to test whether the behaviour change is maintained, if so then the study should be replicated. If we find robust small to medium treatment effects in relation to sustained behaviour change, we should celebrate and promote them.
Acknowledgement I would like to thank Julie Chambers, Eamonn Ferguson, Rona Moss-Morris, Daryl O’Connor, Rory O’Connor, and two reviewers for their helpful comments on a draft of this paper.
Conflict of interest I confirm that this work is my own and that I have no conflict of interests in relation to the content. Ronan E. O’Carroll (Division of Psychology, University of Stirling, UK)
References Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munaf o, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365–376. doi:10.1038/nrn3475 Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2009). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. Psychological Medicine, 40, 211. doi:10.1017/S0033291709006114 CLOTS (Clots in Legs Or sTockings after Stroke) Trials Collaboration, Dennis, M., Sandercock, P., Reid, J., Graham, C., Forbes, J., & Murray, G. (2013). Effectiveness of intermittent pneumatic compression in reduction of risk of deep vein thrombosis in patients who have had a stroke (CLOTS 3): A multicentre randomised controlled trial. Lancet, 382, 516–524. doi:10.1016/ S0140-6736(13)61050-8 Johnston, M., & Dixon, D. (2008). Current issues and new directions in psychology and health: What happened to behaviour in the decade of behaviour? Psychology and Health, 3, 509–513. doi:10. 1080/08870440701816728
Editorial
239
Khaw, K. T., Wareham, N., Bingham, S., Welch, A., Luben, R., & Day, N. (2008). Combined impact of health behaviours and mortality in men and women: The EPIC-Norfolk prospective population study. PLoS Medicine, 5 (1), e12. doi:10.1371/journal.pmed.0050012 Leucht, S., Hierl, S., Kissling, W., Dold, M., & Davis, J. M. (2012). Putting the efficacy of psychiatric and general medicine medication into perspective: Review of meta-analyses. The British Journal of Psychiatry, 200 (2), 97–106. doi:10.1192/bjp.bp.111.096594 Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., . . . Wood, C. E. (2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine. Advance online publication. doi:10.1007/ s12160-013-9486-6 Mokdad, A. H., Marks, J. S., Stroup, D. F., & Gerberding, J. L. (2004). Actual causes of death in the United States, 2000. JAMA: The Journal of the American Medical Association, 291, 1238–1245. doi:10.1001/jama.291.10.1238 O’Carroll, R. E., Chambers, J. A., Dennis, M., Sudlow, C., & Johnston, M. (2013). Improving adherence to medication in stroke survivors: A pilot randomised controlled trial. Annals of Behavioral Medicine, 46, 358–368. doi:10.1007/s12160-013-9515-5 Petrie, K. J., Cameron, L. D., Ellis, C. J., Buick, D., & Weinman, J. (2002). Changing illness perceptions after myocardial infarction: An early intervention randomized controlled trial. Psychosomatic Medicine, 64 (4), 580–586. Prentice, D. A., & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112 (1), 160. doi:10.1037/0033-2909.112.1.160 Trewby, P. N., Reddy, A. V., Trewby, C. S., Ashton, V. J., Brennan, G., & Inglis, J. (2002). Are preventive drugs preventive enough? A study of patients’ expectation of benefit from preventive drugs. Clinical Medicine, 2, 527–533. doi:10.7861/clinmedicine.2-6-527
British Journal of Health Psychology (2014), 19, 235–239 © 2013 The British Psychological Society www.wileyonlinelibrary.com
Editorial
Health Psychology interventions
There is now widespread acceptance that behaviour plays a key role in the development of many long-term conditions and that behaviour change is essential for both prevention and treatment (Khaw et al., 2008; Mokdad, Marks, Stroup, & Gerberding, 2004). Our discipline has taken important strides forward in theory development and in extending our understanding of the factors that guide intention formation and behavioural enaction. Significant advances are also being made in developing a reliable taxonomy of behaviour and behaviour change techniques (Michie et al., 2013). However, Health Psychology now needs to start to deliver robust evidence that our interventions can lead to lasting changes in behaviour, which in turn, lead to significant health benefits.
Measuring behaviour Psychology is the scientific study of behaviour, but we do not measure behaviour objectively as often as we should. We have tended to focus more on attitudes and intentions rather than on overt behaviour. As Johnston and Dixon (2008) highlight, Health Psychology theories often focus rather more on intrapsychic phenomena – thoughts and emotions – that may determine behaviour, rather than on the behaviour itself. When we do measure behaviour, it is commonly via self-report. Of course, some important outcomes of interventions, such as quality of life, can only be measured by self-report. However, social desirability bias is well recognized, for example alcohol intake is often under-reported. In order to persuade others that our interventions are effective and important, we need to place far greater emphasis on the objective measurement of overt behaviour as a key outcome. We also need to distinguish between interventions where behaviour change is the valued outcome (e.g., increasing physical activity) and where the targeted behaviour is a mediator of the valued outcome (e.g., increasing medication adherence to reduce the risk of stroke). In the latter case, the health outcome will clearly depend on both the effectiveness of the psychological intervention and the strength of the relationship between the targeted behaviour (medication adherence) and the health outcome (stroke).
DOI:10.1111/bjhp.12082
236
Editorial
Sample sizes are often too small Many Health Psychology intervention studies use relatively small samples. By way of example, we recently conducted a simple trial of a two-session intervention to increase adherence to medication in stroke patients, with 30 participants in each arm (O’Carroll, Chambers, Dennis, Sudlow, & Johnston, 2013). Around the same time, one of my physician co-authors on this paper led a simple trial of compression stockings to reduce the risk of DVTs (the CLOTS 3 trial) – he had 1,438 participants in each arm (CLOTS et al., 2013). In terms of scale, we are worlds apart. In Health Psychology, we often use power calculations to justify using relatively small sample sizes. For example, in grant applications, we may state that our sample size will enable us to detect a large effect size (e.g., d = .8). However, to non-psychologists reading such applications, testing for effect sizes like this may arouse suspicion, as there are few medical treatments for long-term conditions that have such large effects. Furthermore, if the study then does not produce a ‘significant’ effect, the psychological intervention may be judged to have ‘failed’. However, in relation to other treatments, the intervention may have produced a really important modest treatment effect, which is not statistically significant as a result of the low sample size. Button et al. (2013) recently reported in Nature Review Neuroscience that small sample sizes undermine the reliability of neuroscience. The same charge can be levelled at Health Psychology intervention studies. We need to engage in more collaborative, multicentre intervention trials which are powered to detect small to medium treatment effect sizes.
How effective are commonly used treatments? Some intensive, complex psychological interventions can produce impressive large treatment effect sizes. However, many less intensive interventions in Health Psychology may produce more modest effects, but we should not be apologetic about this. The general public (and many Health Psychologists?) may be surprised at the modest efficacy of many established medical treatments. This may be because of the way the treatment effects are often marketed, for example ‘Treatment X can reduce the risk of stroke by 20%’. Claims such as this are usually presenting a relative risk reduction (RRR), rather than an absolute risk/response difference (ARD). The ARD is always much lower. The following examples are taken from Leucht, Hierl, Kissling, Dold, and Davis (2012). Statin treatment for reduction in cholesterol levels reduced cardio-vascular events from 18% to 14% (primary and secondary prevention combined), a RRR of 21%, but an ARD of 4%. In the secondary prevention of cardiovascular disease, low-dose aspirin reduced serious cardiovascular events per year from 8.2% to 6.7%, an RRR of 19%, but an ARD of 1.5%. These ARDs are important but are quite small, perhaps lower than many participants taking these medications may believe? Trewby et al. (2002) conducted a seminal study on this topic. They aimed to find the threshold of benefit for a hypothetical cholesterol-lowering drug, below which patients would not be prepared to take the drug. They found that three quarters of participants would not take a drug offering 5% or less absolute risk reduction over 5 years. This is roughly the efficacy of best current medical treatment! The general public’s expectation of benefit from a preventive drug may thus be significantly higher than the actual benefit provided by current drug strategies. To be clear, the preceding section is not a criticism of the effectiveness of medical treatments, but rather it highlights the fact that many established medical treatments are
Editorial
237
modestly effective. It is also worth noting that a recent meta-analysis of the effectiveness of psychotherapy treatment studies for depression in adults concluded that in the highest quality studies, the standardized mean treatment effect was a modest d = .22 (Cuijpers, van Straten, Bohlmeijer, Hollon, & Andersson, 2009). If Health Psychology was clearer about the importance of small to medium treatment effects and more confident in itself as a discipline, then we should be unapologetic in seeking and reporting modest effect sizes in intervention studies. Prentice and Miller (1992) set out clear guidelines when small effect sizes should be considered impressive, for example when the outcome is difficult to influence. If we can manage to achieve small but sustained health behaviour change, this would represent a considerable achievement. Furthermore, many Health Psychology intervention studies simply report pre- and post-treatment effects. We need to provide more convincing evidence of behaviour change sustained over the longer term. More work needs to be conducted on the need and cost-effectiveness of post-intervention ‘booster’ sessions.
Health Psychology Homeopathy A further tension that can emerge is in relation to the cost of Health Psychology interventions. If a positive psychological treatment effect is reported, there may be pressure exerted to see whether it can be delivered more simply and at less expense. To take a hypothetical example, a two- or three-session intervention based on modifying illness and/or treatment perceptions may show a significant health benefit (e.g., Petrie, Cameron, Ellis, Buick, & Weinman, 2002). However, others may argue that this could never be delivered in a cash-strapped National Health Service as it would be far too expensive to deliver, and, they may ask ‘Could this not be delivered more simply, for example in a leaflet?’ This is a dangerous road to follow. Apart from the fact that this would represent a completely different treatment, it is highly likely that any treatment effect would be significantly diluted and potentially lost. This may lead others to conclude that psychological interventions do not work ‘in the real world’. Of course, Health Psychology interventions are often multicomponent, and we need to determine the active ingredients. However, would we ask whether a 10-day course of antibiotics could be reduced to 1 day or renal dialysis from three times a week to once a week to make it simpler and cheaper? We need to collaborate more with Health Economists to estimate the savings to health services that may be achieved via successful behaviour change interventions, properly delivered.
The hypocrisy regarding replication Scientific discovery is based on the refutation of hypotheses and the principle that effects must be robust and replicable. Before we can have confidence in a scientific finding, it should be confirmed independently by others. In reality, we rarely do this. Findings are valued because they are novel, replications are not. It is very rare to see an exact replication of a Health Psychology intervention using the same method and measures. Indeed, many Journals request a cover letter with new submissions, where the author is asked to state how the paper’s results are important and novel. Thus, our journals are largely populated with ‘one-off’, small-scale studies that are never replicated, and we therefore do not know whether most of the findings in Health Psychology are robust. We need to change the mindset in behavioural science where high-quality replications are
238
Editorial
valued. To take a local example, in the United Kingdom, the Research Excellence Framework (REF) dominates academia; it is how we are judged. All academics must submit their four ‘best’ papers for peer review. They are rated as 1*–4* on the criteria of ‘originality, significance and rigour’. Replications are unlikely to score well. Behavioural science, as a cumulative process, is thus being impeded by the failure to recognize the importance of replications, and REF acts a strong disincentive to conducting replications. Furthermore, an intervention is only likely to become widely accepted after being subject to systematic review, and this process requires replication. The Association for Psychological Science (APS) have made a recent welcome development in this area; their journal Perspectives on Psychological Science now invites Registered Replication Reports.
Conclusion As a discipline, we need to collaborate more and conduct larger, multicentre, collaborative, intervention studies, where behaviour is measured objectively. We need to test whether the behaviour change is maintained, if so then the study should be replicated. If we find robust small to medium treatment effects in relation to sustained behaviour change, we should celebrate and promote them.
Acknowledgement I would like to thank Julie Chambers, Eamonn Ferguson, Rona Moss-Morris, Daryl O’Connor, Rory O’Connor, and two reviewers for their helpful comments on a draft of this paper.
Conflict of interest I confirm that this work is my own and that I have no conflict of interests in relation to the content. Ronan E. O’Carroll (Division of Psychology, University of Stirling, UK)
References Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munaf o, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365–376. doi:10.1038/nrn3475 Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2009). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. Psychological Medicine, 40, 211. doi:10.1017/S0033291709006114 CLOTS (Clots in Legs Or sTockings after Stroke) Trials Collaboration, Dennis, M., Sandercock, P., Reid, J., Graham, C., Forbes, J., & Murray, G. (2013). Effectiveness of intermittent pneumatic compression in reduction of risk of deep vein thrombosis in patients who have had a stroke (CLOTS 3): A multicentre randomised controlled trial. Lancet, 382, 516–524. doi:10.1016/ S0140-6736(13)61050-8 Johnston, M., & Dixon, D. (2008). Current issues and new directions in psychology and health: What happened to behaviour in the decade of behaviour? Psychology and Health, 3, 509–513. doi:10. 1080/08870440701816728
Editorial
239
Khaw, K. T., Wareham, N., Bingham, S., Welch, A., Luben, R., & Day, N. (2008). Combined impact of health behaviours and mortality in men and women: The EPIC-Norfolk prospective population study. PLoS Medicine, 5 (1), e12. doi:10.1371/journal.pmed.0050012 Leucht, S., Hierl, S., Kissling, W., Dold, M., & Davis, J. M. (2012). Putting the efficacy of psychiatric and general medicine medication into perspective: Review of meta-analyses. The British Journal of Psychiatry, 200 (2), 97–106. doi:10.1192/bjp.bp.111.096594 Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., . . . Wood, C. E. (2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine. Advance online publication. doi:10.1007/ s12160-013-9486-6 Mokdad, A. H., Marks, J. S., Stroup, D. F., & Gerberding, J. L. (2004). Actual causes of death in the United States, 2000. JAMA: The Journal of the American Medical Association, 291, 1238–1245. doi:10.1001/jama.291.10.1238 O’Carroll, R. E., Chambers, J. A., Dennis, M., Sudlow, C., & Johnston, M. (2013). Improving adherence to medication in stroke survivors: A pilot randomised controlled trial. Annals of Behavioral Medicine, 46, 358–368. doi:10.1007/s12160-013-9515-5 Petrie, K. J., Cameron, L. D., Ellis, C. J., Buick, D., & Weinman, J. (2002). Changing illness perceptions after myocardial infarction: An early intervention randomized controlled trial. Psychosomatic Medicine, 64 (4), 580–586. Prentice, D. A., & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112 (1), 160. doi:10.1037/0033-2909.112.1.160 Trewby, P. N., Reddy, A. V., Trewby, C. S., Ashton, V. J., Brennan, G., & Inglis, J. (2002). Are preventive drugs preventive enough? A study of patients’ expectation of benefit from preventive drugs. Clinical Medicine, 2, 527–533. doi:10.7861/clinmedicine.2-6-527
