p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Available online at www.sciencedirect.com
Public Health journal homepage: www.elsevier.com/puhe
Original Research
Evaluation of a programme in population health management for GP trainees S. Ahluwalia a,*, D. de Silva b, N. Chana c a Health Education North Central and East London, Stewart House, 32 Russell Square, London WC1B5DN, United Kingdom b The Evidence Centre, London, United Kingdom c Health Education South London, Stewart House, 32 Russell Square, London WC1B5DN, United Kingdom
article info
abstract
Article history:
Objectives: There is an increasing focus on improving outcomes for populations. General
Received 14 November 2013
practice has a key role to play in achieving better patient outcomes. A programme in
Received in revised form
population health management was developed for GP trainees in their fourth year. The
2 August 2014
objective was to evaluate this educational intervention for impact upon learners, and
Accepted 6 August 2014
lessons learned in the delivery of this innovative programme.
Available online 23 October 2014
Methods: The evaluation approach combined qualitative and quantitative approaches involving GP trainees, GP trainers, and faculty.
Keywords:
Results: Involvement in the programme had a positive impact upon trainees' knowledge of
Population health management
population health management, developed transferable skills, and encouraged a shift in
GP training
mindset to considering the needs of a practice population. The impact upon patients and
Service improvement
practices is also described. Lessons learned for enhancing the programme are explained.
Curriculum development
Conclusions: Encouraging GP trainees to become involved with population health has significant benefits for their professional development. The Royal College of General Practitioners (RCGP) curriculum should consider incorporating population health competencies. Crown Copyright © 2014 Published by Elsevier Ltd on behalf of The Royal Society for Public Health. All rights reserved.
Introduction Despite an increase in quality of life and the overall health of the UK population, significant health inequalities remain.1e4 Government policies emphasise the need for health and social services to work together to address these challenges and to take a ‘population health’ approach that identifies risks and targets prevention and care based on epidemiological
evidence.5e7 There are a number of components of such an approach.8 General practice has a fundamental role to play in supporting preventive efforts, targeting care appropriately and addressing health inequalities, and as such is key to population health initiatives.9 A population health approach seeks to examine factors that determine poor health and outcomes in society or among specific populations and develop interventions to improve the health of these groups.10 It has
* Corresponding author. E-mail addresses: [email protected] (S. Ahluwalia), [email protected] (D. de Silva), [email protected] (N. Chana). http://dx.doi.org/10.1016/j.puhe.2014.08.009 0033-3506/Crown Copyright © 2014 Published by Elsevier Ltd on behalf of The Royal Society for Public Health. All rights reserved.
926
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
been recognised that population health education needs to be included within the training programmes for general practitioners.11,12 General practice training in the United Kingdom is a threeyear programme that combines a mixture of hospital and community based posts offering learning opportunities to cover the skills, knowledge and qualities defined by the Royal College of General Practitioners (RCGP) in its curriculum for independent practice (http://www.rcgp.org.uk/gp-trainingand-exams/gp-curriculum-overview.aspx). GP trainees will enter higher specialty training following medical school and a two year foundation programme. More recently, there have been calls to increase the duration of GP training from three to four years with the RCGP working with Health Education England to develop the educational and financial case for such an extension to GP training.13 In preparation for this extension to GP training, the London School of General Practice (LSGP) has developed a number of programmes with a fourth year of training (ST4). The increasing emphasis on population health outcomes and health inequalities motivated the development of a programme on population health management as part of the ST4 year. During the fourth year, which began for the first time in August 2012, trainees spent half of their time in general practice and the other half developing skills in another area such as leadership and management, medical education, research and specialisms such as end of life care or paediatrics.
The population health programme The aim of the population health programme was to help ST4s recognise their responsibility to support wider groups and communities and to begin gaining the skills needed to make improvements for population groups at practice level. Competencies were developed by reference to relevant frameworks from international sources.14e16 The programme involved: five two-hour seminars to introduce new concepts; implementation of a practical project to improve population health outcomes within a general practice; ad hoc support from a content expert in this field; and preparation of a report and presentation on findings and learning. Programme documents and copies of presentations were stored on this Facebook page. The approach to assessment of learning outcomes in population health was formative by means of a written report and presentation on the practice based project. By the end of the programme, it was anticipated that trainees would have an understanding of the epidemiology of a problem presenting in a primary care setting; how to modify risk factors for a range of conditions at a GP practice-population level; how to engage colleagues and practice staff in making change; how to apply preventative approaches to avoidable disease in different clinical circumstances; and how to assess health care needs at a practice population level and take appropriate action. Box 1 lists the desired competencies.
Box 1 London School of General Practice population health programme competencies. 1. Understand the concept of population health and why it is important for GPs Be familiar with literature about population health and health inequalities Be able to apply the concepts of population health in real practice Be able to debate the ethics of allocating limited health care resources, and the tensions of advocating for individuals vs populations 2. Be able to identify the needs of populations (health needs assessment) Be able to use websites/databases and practice IT systems to identify population groups and their needs, including potential health inequalities among population groups Recognise that the needs of populations are not just related to health, and consider the social, environmental, and political determinants that influence the local practice population's burden of disease and access to health services 3. Be able to work with a multidisciplinary team to plan and sustain change Recognise the importance of influencing others in order to achieve population health goals and demonstrate the ability to work as a part of a team, both within the practice and with professionals outside the practice in ways that actively seek to reduce inequalities in health outcomes or improve population health Demonstrate project management skills in planning and implementing a practical project that delivers results within a set time period Be able to list the costs and benefits of changes, and to weigh up the overall effects of population health interventions (cost effectiveness analysis) Demonstrate that strategies have been put in place to sustain change
In order to gain a programme certificate, trainees were required to submit a written report about their population health project and provide a presentation outlining their findings. The programme was not compulsory and success with the project did not influence the outcome of obtaining an overall certificate of completion of training (CCT) for the trainee.
Evaluation purpose and methods The population health programme was a pilot, so the GP School was interested in understanding whether such a curriculum could make a difference to the way GP trainees are able to demonstrate a population based health care approach in practice. The evaluation was undertaken by an independent external organisation, The Evidence Centre, (http://www.
927
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
evidencecentre.com) to test whether the curriculum developed skills in population health that can be carried forward to health service planning and delivery. Box 2 lists the types of questions considered by the evaluation. Box 2 Evaluation questions. Does the module make a difference to trainees? To what extent do GP specialist trainees achieve the prespecified learning outcomes? Do the trainees' projects suggest that a curriculum like this can make a difference to patient or population outcomes? What were the challenges and key successes with the curriculum? Were some learning methods favoured over others? Was the timing of the seminars and learning activities appropriate? Does the curriculum have any unanticipated outcomes?
The evaluation used a before-and-after quantitative design to examine changes in attitudes and perceived competences. This was coupled with a descriptive qualitative approach to delve into more depth regarding the value of the curriculum and potential challenges. ST4 and GP trainer response rates are provided in Table 1. The methods included an online survey with ST4 trainees before and after the end of the population health module; observation of seminars and trainee presentations; informal discussions and interviews with a selection of trainees; informal discussions with faculty members; telephone interviews with a selection of trainers; online survey of trainers; and a review of trainees' project reports to collate outcomes information. An online survey tool was specifically designed for this evaluation. The before and after survey used a four point scale (very confident, confident, unconfident, very unconfident). This was combined into a dichotomous variable (confident/ not confident) for presentational purposes. The interviews ranged between ten and 45 min each. They were conducted by the evaluation team commissioned for the purpose. The questions asked focused on processes and outcomes, including potential ways to improve, learning achievements and impacts. A semi structured approach was used, with an interview guide used for prompts and additional questions asked as relevant to the conversation. A short observation
Table 1 e Evaluation response rates. Response rates ST4 online survey ST4 telephone feedback GP trainer feedback (telephone and online)
sheet was used to collect data about seminars, with much of the observation being open ended and reactive. Observation did not quantify the number of times specific actions or behaviours occurred, it was used to see qualitatively the types of content and approach used during the learning sessions. The interview data were coded to allow theme analysis. The survey data were pre coded. The observational data were not coded because this was designed to provide contextual rather than quantitative information. As this was an evaluation rather than a piece of research, formal NHS research ethics committee approval was not required. Even so, standard ethical principles were followed in terms of respecting participants and safeguarding confidentiality. Trainees and trainers were invited to participate in an online survey, which was anonymous and non-compulsory. Self-selection indicated consent. Similarly, trainees, trainers and members of the module team were invited by email to take part in confidential telephone interviews. Self-selection by provision of their own contact details and a time for interview was used to indicate consent. The invitation emails provided information about the purpose of the evaluation and noted that participation was voluntary and would not have any effect on anything else. Signed consent forms were not used. All participants received a copy of the draft report and their (anonymised) quotes for approval.
Results of evaluation The results of this evaluation are categorised into three main areas: the impact on trainees; patients and practices; and lessons for improving the population health module. The level of engagement of ST4 trainees is highlighted in Table 2.
Impact upon trainees a) Impact upon knowledge The module helped trainees address many of the competencies initially planned. Not unsurprisingly, it was the first time that they had been exposed to information about population health, so this was a very new way of thinking and working for them. Through the process of carrying out this project, and attending the relevant seminars, I have gained insight and definitely had a change of mindset when considering populations rather than patients, epidemiology rather than diagnosis, and risk modification rather than treatment. (excerpt from project report)
Table 2 e Learner level of engagement.
At start of programme
At the end of programme
30/38 (79%)
28/38 (74%) 20/38 (53%) 30/38 (79%)
Level of engagement with programme Completed all components Submitted report only Attended seminars only Did not engage at all Other reasons (e.g. maternity leave)
Percentage 23/38 2/38 3/38 8/38 2/38
(61%) (5%) (8%) (21%) (5%)
928
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Fig. 1 e Before and after comparison of ST4 trainees' confidence in their abilities.
The survey suggested that trainees felt that their knowledge and confidence about various components of population health had improved (see Fig. 1) and trainers had reasonably similar views (see Fig. 2). Half of all GP trainers believed that the module had instilled new knowledge, skills and confidence in their trainee (50%). [This programme] moved my trainee out of the consulting room when thinking about her responsibilities. She was able to identify a particular feature (poor level of understanding) in a high risk population and hence able to challenge to conventional ways of working. (GP trainer) Other areas identified as key learning through the course of the module included a greater awareness of health inequalities and identifying issues facing populations, ethics
and resource allocation, and greater confidence working as part of a team. There was one area where the module did not appear to have had a sizeable impact on trainees' knowledge and confidence: using strategies to promote behaviour change within consultations (76% vs 83%). Trainees said they had gained further knowledge about what works and does not work in their practices and to support patients. For example trainees learned about deficiencies in record keeping, new ways to use electronic practice systems efficiently, different approaches to contact patients with reminders about screening or other appointments, and collecting feedback from patients. Overall, trainees reported that taking part in projects helped them learn about very practical issues and the potential for making change within practices.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
929
Fig. 2 e GP trainers' confidence in their assigned trainee's abilities at end of programme. It has given a good understanding of how to look at populations and the impact that can be gained by even small interventions. The programme has given insight into resources that are available and how to go about in the future designing and implementing a larger population project. (GP trainee) Clinical trainers also reinforced learning by trainees and themselves regarding the practicalities of implementing change at practice level. The challenges of causing a change in a practice became obvious during this project. (GP trainer) Trainees said they had learnt more about what it means to be a good commissioner and how it is important to balance
the needs of wider groups with those of individual circumstances when making decisions about prescribing or when commissioning services. The small numbers of trainees involved in this evaluation mean that it would be spurious to provide calculations of statistical significance, but the trends are positive. It is also important to note that not all learning may be a result of the course. b) Impact upon skills Having to undertake a practical project was seen by many as a useful way of developing important and transferable skills. Though the population health projects did not run smoothly for many trainees, they acknowledged that there was much to be learned from things going wrong. Some
930
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Fig. 3 e Before and after comparison of ST4 trainees' self-perceived skills.
suggested that these project management skills were transferable to GPs' ongoing careers and that they also helped to facilitate a sense of empowerment and ability to create and sustain change. Doing a practical project is really good for getting skills. You can only really get experience by doing things in practice so the project teaches you what the barriers are and how to overcome them. (GP trainee) Most trainees said that a key skill they learnt related to using data, or recognising potential gaps or issues with data. Another area of learning was deficiencies with current practice record systems and datasets. One frustration was in using codes for diagnosis and codes for screening. It really highlighted the need for proper coding during
consultation. This was something I must admit I thought that GP partners were a little obsessed about but I now see the value of doing well (GP trainee) Understanding how to identify population needs was a key module competency and one that trainees reported that they achieved. the most worthwhile thing was learning how to do a needs assessment (GP trainee) Another key area of learning was the need to work with others and in teams to make change happen. Through undertaking a needs assessment trainees made links with local public health teams. Similarly, trainees spoke about how they had to motivate others to move their projects forward and use their leadership skills.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
I learnt the importance of involving others in the practice, so it is not an individuals project, but a whole practice thing (GP trainee) Engaging with other members of the practice team as well as potential other organisations was key to success of improving services for populations and important for longevity. Others suggested that their projects were not as successful as they would have liked due to not being able to engage other team members fully. Before the module one quarter of trainees thought they had strong leadership skills (see Fig. 3). After the module this had doubled (from 27% to 50%). There was a similar trend with perceived management skills (from 19% to 50%). There were some areas where trainees reported no change including basic and more specialised clinical skills, prescribing, and patient safety. There were some areas where the module might have been expected to show a difference but did not-this included organisational skills. c) Changing attitudes A key finding of this evaluation is that trainees developed an understanding of their role looking at the needs of wider populations and the role of GPs in commissioning. Issues such as allocation of resources and funding decisions had become clearer as well. Others suggested that they had more of a sense of what they could achieve as a GP and their central role in facilitating change. It certainly has changed the way I consider my role as a GP and the obligation I have to the entire practice population rather than just the patients I see day to day. (GP trainee) Trainees felt empowered to make simple changes to services that could have a significant impact on health outcomes. They also began to see themselves as agents of change as representatives safeguarding public well-being. Some trainees also described how they had changed the way they talk to patients. All of these factors came together to promote a change in the overall mind-set of some trainees. It is estimated that at least half of those who completed the module benefitted in this way. I have changed the way in which I think about my role as a GP and now give more consideration to my responsibilities in improving the health of the local population, not just the patients I have individual contact with. (GP trainee)
∙ Improvements in the way contact details are kept at practices; ∙ Improved information for health promotion; ∙ Better review processes for services for patients with learning disabilities; Some trainees felt that their projects had broader benefits in terms of re-establishing trust and communication with patients, especially among groups that may feel disempowered or experience significant health inequalities. These (the project) have been vital in promoting the positive message that the doctors in the surgery are committed to their health; and that they are taking an active role in managing their disease risk outside of compulsory health checks or QOF reviews. (GP trainee) Trainees and trainers felt that the projects had positive outcomes for the practice as a whole. Learning how to understand the needs of the practice population, identifying unmet needs and take steps to make improvement were key. Indeed, the projects were well received and practices made specific changes to processes, systems and protocols as a result. I think this is much better than doing an audit. I felt that my project really made a difference to my practice. The outcomes from my project were well received. We had meetings to discuss the findings and things have already changed as a result of what I found. Getting that positive feedback and seeing how you make a change was good. (GP trainee) GP trainers benefitted from attending the seminar series, but also from the outcomes of projects. The practice and I personally benefitted as there were a few ideas generated that questioned how we work, how to value our staff, how to develop our patient representative group, how to get younger people involved and empower them to generate ideas. (GP trainer)
Important lessons for educational delivery Box 3 provides a list of the topics covered in the five programme seminars. Box 3 List of seminar topics.
Impact on patients and practices The projects were undertaken over a short timeframe, so longterm benefits for patients or patient groups cannot be demonstrated, but the project reports suggest that there have been some immediate benefit for individual patients, families and wider communities in some instances. These have included: ∙ Better screening for conditions such as diabetes, CVD, osteoporosis, cancer; ∙ Improvements in self management;
931
Introduction to population health Project management Needs assessment Examples of population health projects Health coaching Resource allocation and ethics Data analysis Writing skills Life after VTS Project presentations (given by trainees)
932
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
The feedback on the seminar series was positive, with comments for improvement relating to the timing of the seminars as well as areas for development. The most frequently commented on seminar was on needs assessment and the importance of this session early in the module. It was the assessment of what the population needs that was most important. Learning how to do this was crucial and could be emphasised more in future. (Faculty member) Topics identified for future modules included sessions on influencing skills, economic analysis, how to write a report and how to co-produce with patients. The role of content experts was seen as positive in supporting learners. However, the content experts described the need for greater clarity in terms of their role, and sharing this early with learners and GP trainers. In particular, the role of content experts in giving early feedback (e.g. on project outlines to learners) was seen as positive. We have not been briefed about answering specific questions about projects. There should be more consistency across what the content experts are saying but there has been little information given to us so we are all just winging it. (Faculty member) Feedback from trainers suggests that more could be done to engage and support them to be part of the population health module. Whilst nine out of ten trainers thought that it was important for GPs to be able to plan improvements for their practice population (95%) and agreed that doing an improvement project during GP training was a good idea (89%), views about the population health module itself were mixed (see Fig. 3). A number of trainers said that they did not fully understand the concept of population health or the value of including this in an ST4 year. Due to the part time nature of the ST4 year within general practice, I felt that my trainee had to keep the project quite simple and I don't really see a difference between this and the audit/ SIP/ QIP that the ST3s already have to do. (GP trainer) In telephone interviews and survey responses, trainers said they did not feel involved in the programme. They largely said they were not given information or reminders about seminar dates, that they were not clear about the submission requirements and that they were not contacted throughout the module for support or feedback. Material was placed on an online learning environment, which trainers had access to, but trainers said that this was difficult to access or inconvenient. Trainees and faculty felt that there was more work to be done to promote the population health projects as practice projects, rather than projects for an individual trainee. This includes getting trainers to mark individual project write-ups, and require trainees to present their projects to the whole practice. Trainees who decided not to engage with the programme offered a number of reasons including being advised by their GP trainers that their learning needs required they
concentrate on other learning aspects during the course of the fourth year, and being in a practice that would not accommodate the requirements of the programme to be able to attend the learning events or engage with the population health project. The project team used the online web presence to create a community of combined learning. However, this did not work so well partly because of accessing the site and partly due to the technical challenges of the site itself. It was suggested that an alternative might be to develop a facebook-style web page that was more easily accessible and user-friendly.
Discussion Summary of findings Feedback from trainees, trainers, faculty and review of project reports suggests that the population health curriculum has had an impact on the majority of eligible trainees. Only one fifth chose not to engage with the module. Two thirds completed the practical project. Practical projects and selfreport surveys suggested experience in identifying population health issues, collecting and using data for planning purposes, working with other professionals to implement improvement and other learning outcomes. Trainee projects were small-scale and completed over a short timeframe, but there were reportedly benefits for patient and population outcomes including better engagement with individuals and groups, an increase in the uptake of screening for various conditions and identification of conditions that may not otherwise have been picked up for some time (such as cancer, diabetes and hypertension). Key success factors comprised contextualising population health as an essential skill, facilitating learning through a practical project, running seminars regularly and having support and feedback available. Challenges included ad hoc planning, trainers and trainees not allocating enough time to prioritise the module, insufficiently engaging trainers and practices and lack of consistent support from content experts and trainers.
Limitations Any GP trainee undertaking the population health module did so on a voluntary basis. This self-selected group meant that the impact of such training on a broader range of GP trainees is unclear. The evaluation cannot tell us whether the positive trends seen in the questionnaires administered are statistically significant and sustainable in the longer term. The numbers involved in the evaluation were too small to allow this sort of analysis e instead the focus was on any broad trends. The study was not powered to test for differences. The authors sought to demonstrate the potential value of ongoing testing, but cannot state that the module made a definite change. As a result of the evaluation, the GP School developed the module further and tested the changes successfully during 2013e14.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Implications for practice This evaluation demonstrates that participating in the programme leads to a perceived increase in knowledge and awareness of population health, and may be a powerful approach to supporting the development of future GPs for their roles as commissioners. Though population health has mention in legal frameworks (e.g. the Health and Social Care Act), it has yet to be incorporated within the curriculum for GP training. The use of a service improvement project has benefits that extend beyond population health to include competencies covered by a number of other curriculum statements (including the GP in the Wider Professional Environment). The authors would suggest that such an approach provides greater opportunities for professional development than traditional assessments such as undertaking an audit. Whilst this programme has been piloted in the fourth year of GP training it is important to see how this might be incorporated within the standard three year training programme. The rising costs of providing health care mean that developing GPs skilled in improving population health outcomes will become an increasingly important part of their work. The authors anticipate that such approaches will be vital in ensuring that the NHS moves from a reactive to proactive health care service. Perhaps the time has come to consider incorporating population health outcomes within the Royal College of General Practitioners curriculum so that all GPs are equipped to make a difference to outcomes for populations as well as individuals.
Author statements Ethical approval NHS research ethics committee noted that no approval was required because this is an educational evaluation.
Funding None declared.
Competing interests None declared.
933
references
1. Douglas L, Szatkowski L. Socioeconomic variations in access to smoking cessation interventions in UK primary care: insights using the Mosaic classification in a large dataset of primary care records. BMC Public Health 2013;13(1):546. 2. Samaranayaka S, Gulliford MC. Trends in cardiovascular risk factors among people with diabetes in a population based study, Health Survey for England 1994e2009. Prim Care Diabetes (Published online May 2013). 3. Dalton AR, Bottle A, Soljak M, Majeed A, Millett C. Ethnic group differences in cardiovascular risk assessment scores: national cross-sectional study. Ethn Health (Published online May 2013). E, Gallagher JE. 4. Al-Haboubi M, Klass C, Jones K, Bernabe Inequalities in the use of dental services among adults in inner South East London. Eur J Oral Sci 2013;121(3 Pt 1):176e81. 5. www.gov.uk/government/uploads/system/uploads/ attachment_data/file/153485/dh_131897.pdf.pdf. 6. www.gov.uk/government/uploads/system/uploads/ attachment_data/file/127045/Public-health-role-of-localauthorities-factsheet.pdf.pdf. 7. www.scotland.gov.uk/Publications/2010/05/10102307/0. 8. Kindig D, Stoddart G. What is population health? Am J Public Health 2003;93(3):380e3. 9. Carson-Stevens A, Patel E, Nutt SL, Bhatt J, Panesar SS. The social movement drive: a role for junior doctors in healthcare reform. J R Soc Med (Published online June 2013). 10. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet 2006;367(9524):1747e57. 11. Wills J, Reynolds J, Swanwick T. “Just a lovely luxury?” What can public health attachments add to postgraduate general practice training? Educ Prim Care 2009;20:278e84. 12. Harris MF, Zwar NA. Care of patients with chronic disease: the challenge for general practice. Med J Aust 2007;187:104e7. 13. Riley B, Rughani A. The future of GP specialty training: enhanced and extended. BJGP June 1, 2012;62:288e9. 14. Royal Australian College of General Practitioners. The RACGP curriculum for Australian general practice: population health and public health (accessed 16 January 2012), http://www.racgp.org. au/Content/NavigationMenu/educationandtraining/ Trainingcurriculum/11Populationhealthandpublichealth.pdf; 2009. 15. Australian College of Rural and Remote Medicine. ACCRM: primary curriculum (accessed 16 January 201), http://www.acrrm. org.au/files/uploads/pdf/curriculum/primary/ACRRM-PrimaryCurriculum_3rd-edition_25-09-09_with-cover.pdf; 2009. 16. Lisa J, Chamberlain LJ, Wang NE, Ho ET, Banchoff AW, Braddock CH, Gesundheit N. Integrating collaborative population health projects into a medical student curriculum at Stanford. Acad Med 2008;83:338e44.
Available online at www.sciencedirect.com
Public Health journal homepage: www.elsevier.com/puhe
Original Research
Evaluation of a programme in population health management for GP trainees S. Ahluwalia a,*, D. de Silva b, N. Chana c a Health Education North Central and East London, Stewart House, 32 Russell Square, London WC1B5DN, United Kingdom b The Evidence Centre, London, United Kingdom c Health Education South London, Stewart House, 32 Russell Square, London WC1B5DN, United Kingdom
article info
abstract
Article history:
Objectives: There is an increasing focus on improving outcomes for populations. General
Received 14 November 2013
practice has a key role to play in achieving better patient outcomes. A programme in
Received in revised form
population health management was developed for GP trainees in their fourth year. The
2 August 2014
objective was to evaluate this educational intervention for impact upon learners, and
Accepted 6 August 2014
lessons learned in the delivery of this innovative programme.
Available online 23 October 2014
Methods: The evaluation approach combined qualitative and quantitative approaches involving GP trainees, GP trainers, and faculty.
Keywords:
Results: Involvement in the programme had a positive impact upon trainees' knowledge of
Population health management
population health management, developed transferable skills, and encouraged a shift in
GP training
mindset to considering the needs of a practice population. The impact upon patients and
Service improvement
practices is also described. Lessons learned for enhancing the programme are explained.
Curriculum development
Conclusions: Encouraging GP trainees to become involved with population health has significant benefits for their professional development. The Royal College of General Practitioners (RCGP) curriculum should consider incorporating population health competencies. Crown Copyright © 2014 Published by Elsevier Ltd on behalf of The Royal Society for Public Health. All rights reserved.
Introduction Despite an increase in quality of life and the overall health of the UK population, significant health inequalities remain.1e4 Government policies emphasise the need for health and social services to work together to address these challenges and to take a ‘population health’ approach that identifies risks and targets prevention and care based on epidemiological
evidence.5e7 There are a number of components of such an approach.8 General practice has a fundamental role to play in supporting preventive efforts, targeting care appropriately and addressing health inequalities, and as such is key to population health initiatives.9 A population health approach seeks to examine factors that determine poor health and outcomes in society or among specific populations and develop interventions to improve the health of these groups.10 It has
* Corresponding author. E-mail addresses: [email protected] (S. Ahluwalia), [email protected] (D. de Silva), [email protected] (N. Chana). http://dx.doi.org/10.1016/j.puhe.2014.08.009 0033-3506/Crown Copyright © 2014 Published by Elsevier Ltd on behalf of The Royal Society for Public Health. All rights reserved.
926
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
been recognised that population health education needs to be included within the training programmes for general practitioners.11,12 General practice training in the United Kingdom is a threeyear programme that combines a mixture of hospital and community based posts offering learning opportunities to cover the skills, knowledge and qualities defined by the Royal College of General Practitioners (RCGP) in its curriculum for independent practice (http://www.rcgp.org.uk/gp-trainingand-exams/gp-curriculum-overview.aspx). GP trainees will enter higher specialty training following medical school and a two year foundation programme. More recently, there have been calls to increase the duration of GP training from three to four years with the RCGP working with Health Education England to develop the educational and financial case for such an extension to GP training.13 In preparation for this extension to GP training, the London School of General Practice (LSGP) has developed a number of programmes with a fourth year of training (ST4). The increasing emphasis on population health outcomes and health inequalities motivated the development of a programme on population health management as part of the ST4 year. During the fourth year, which began for the first time in August 2012, trainees spent half of their time in general practice and the other half developing skills in another area such as leadership and management, medical education, research and specialisms such as end of life care or paediatrics.
The population health programme The aim of the population health programme was to help ST4s recognise their responsibility to support wider groups and communities and to begin gaining the skills needed to make improvements for population groups at practice level. Competencies were developed by reference to relevant frameworks from international sources.14e16 The programme involved: five two-hour seminars to introduce new concepts; implementation of a practical project to improve population health outcomes within a general practice; ad hoc support from a content expert in this field; and preparation of a report and presentation on findings and learning. Programme documents and copies of presentations were stored on this Facebook page. The approach to assessment of learning outcomes in population health was formative by means of a written report and presentation on the practice based project. By the end of the programme, it was anticipated that trainees would have an understanding of the epidemiology of a problem presenting in a primary care setting; how to modify risk factors for a range of conditions at a GP practice-population level; how to engage colleagues and practice staff in making change; how to apply preventative approaches to avoidable disease in different clinical circumstances; and how to assess health care needs at a practice population level and take appropriate action. Box 1 lists the desired competencies.
Box 1 London School of General Practice population health programme competencies. 1. Understand the concept of population health and why it is important for GPs Be familiar with literature about population health and health inequalities Be able to apply the concepts of population health in real practice Be able to debate the ethics of allocating limited health care resources, and the tensions of advocating for individuals vs populations 2. Be able to identify the needs of populations (health needs assessment) Be able to use websites/databases and practice IT systems to identify population groups and their needs, including potential health inequalities among population groups Recognise that the needs of populations are not just related to health, and consider the social, environmental, and political determinants that influence the local practice population's burden of disease and access to health services 3. Be able to work with a multidisciplinary team to plan and sustain change Recognise the importance of influencing others in order to achieve population health goals and demonstrate the ability to work as a part of a team, both within the practice and with professionals outside the practice in ways that actively seek to reduce inequalities in health outcomes or improve population health Demonstrate project management skills in planning and implementing a practical project that delivers results within a set time period Be able to list the costs and benefits of changes, and to weigh up the overall effects of population health interventions (cost effectiveness analysis) Demonstrate that strategies have been put in place to sustain change
In order to gain a programme certificate, trainees were required to submit a written report about their population health project and provide a presentation outlining their findings. The programme was not compulsory and success with the project did not influence the outcome of obtaining an overall certificate of completion of training (CCT) for the trainee.
Evaluation purpose and methods The population health programme was a pilot, so the GP School was interested in understanding whether such a curriculum could make a difference to the way GP trainees are able to demonstrate a population based health care approach in practice. The evaluation was undertaken by an independent external organisation, The Evidence Centre, (http://www.
927
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
evidencecentre.com) to test whether the curriculum developed skills in population health that can be carried forward to health service planning and delivery. Box 2 lists the types of questions considered by the evaluation. Box 2 Evaluation questions. Does the module make a difference to trainees? To what extent do GP specialist trainees achieve the prespecified learning outcomes? Do the trainees' projects suggest that a curriculum like this can make a difference to patient or population outcomes? What were the challenges and key successes with the curriculum? Were some learning methods favoured over others? Was the timing of the seminars and learning activities appropriate? Does the curriculum have any unanticipated outcomes?
The evaluation used a before-and-after quantitative design to examine changes in attitudes and perceived competences. This was coupled with a descriptive qualitative approach to delve into more depth regarding the value of the curriculum and potential challenges. ST4 and GP trainer response rates are provided in Table 1. The methods included an online survey with ST4 trainees before and after the end of the population health module; observation of seminars and trainee presentations; informal discussions and interviews with a selection of trainees; informal discussions with faculty members; telephone interviews with a selection of trainers; online survey of trainers; and a review of trainees' project reports to collate outcomes information. An online survey tool was specifically designed for this evaluation. The before and after survey used a four point scale (very confident, confident, unconfident, very unconfident). This was combined into a dichotomous variable (confident/ not confident) for presentational purposes. The interviews ranged between ten and 45 min each. They were conducted by the evaluation team commissioned for the purpose. The questions asked focused on processes and outcomes, including potential ways to improve, learning achievements and impacts. A semi structured approach was used, with an interview guide used for prompts and additional questions asked as relevant to the conversation. A short observation
Table 1 e Evaluation response rates. Response rates ST4 online survey ST4 telephone feedback GP trainer feedback (telephone and online)
sheet was used to collect data about seminars, with much of the observation being open ended and reactive. Observation did not quantify the number of times specific actions or behaviours occurred, it was used to see qualitatively the types of content and approach used during the learning sessions. The interview data were coded to allow theme analysis. The survey data were pre coded. The observational data were not coded because this was designed to provide contextual rather than quantitative information. As this was an evaluation rather than a piece of research, formal NHS research ethics committee approval was not required. Even so, standard ethical principles were followed in terms of respecting participants and safeguarding confidentiality. Trainees and trainers were invited to participate in an online survey, which was anonymous and non-compulsory. Self-selection indicated consent. Similarly, trainees, trainers and members of the module team were invited by email to take part in confidential telephone interviews. Self-selection by provision of their own contact details and a time for interview was used to indicate consent. The invitation emails provided information about the purpose of the evaluation and noted that participation was voluntary and would not have any effect on anything else. Signed consent forms were not used. All participants received a copy of the draft report and their (anonymised) quotes for approval.
Results of evaluation The results of this evaluation are categorised into three main areas: the impact on trainees; patients and practices; and lessons for improving the population health module. The level of engagement of ST4 trainees is highlighted in Table 2.
Impact upon trainees a) Impact upon knowledge The module helped trainees address many of the competencies initially planned. Not unsurprisingly, it was the first time that they had been exposed to information about population health, so this was a very new way of thinking and working for them. Through the process of carrying out this project, and attending the relevant seminars, I have gained insight and definitely had a change of mindset when considering populations rather than patients, epidemiology rather than diagnosis, and risk modification rather than treatment. (excerpt from project report)
Table 2 e Learner level of engagement.
At start of programme
At the end of programme
30/38 (79%)
28/38 (74%) 20/38 (53%) 30/38 (79%)
Level of engagement with programme Completed all components Submitted report only Attended seminars only Did not engage at all Other reasons (e.g. maternity leave)
Percentage 23/38 2/38 3/38 8/38 2/38
(61%) (5%) (8%) (21%) (5%)
928
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Fig. 1 e Before and after comparison of ST4 trainees' confidence in their abilities.
The survey suggested that trainees felt that their knowledge and confidence about various components of population health had improved (see Fig. 1) and trainers had reasonably similar views (see Fig. 2). Half of all GP trainers believed that the module had instilled new knowledge, skills and confidence in their trainee (50%). [This programme] moved my trainee out of the consulting room when thinking about her responsibilities. She was able to identify a particular feature (poor level of understanding) in a high risk population and hence able to challenge to conventional ways of working. (GP trainer) Other areas identified as key learning through the course of the module included a greater awareness of health inequalities and identifying issues facing populations, ethics
and resource allocation, and greater confidence working as part of a team. There was one area where the module did not appear to have had a sizeable impact on trainees' knowledge and confidence: using strategies to promote behaviour change within consultations (76% vs 83%). Trainees said they had gained further knowledge about what works and does not work in their practices and to support patients. For example trainees learned about deficiencies in record keeping, new ways to use electronic practice systems efficiently, different approaches to contact patients with reminders about screening or other appointments, and collecting feedback from patients. Overall, trainees reported that taking part in projects helped them learn about very practical issues and the potential for making change within practices.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
929
Fig. 2 e GP trainers' confidence in their assigned trainee's abilities at end of programme. It has given a good understanding of how to look at populations and the impact that can be gained by even small interventions. The programme has given insight into resources that are available and how to go about in the future designing and implementing a larger population project. (GP trainee) Clinical trainers also reinforced learning by trainees and themselves regarding the practicalities of implementing change at practice level. The challenges of causing a change in a practice became obvious during this project. (GP trainer) Trainees said they had learnt more about what it means to be a good commissioner and how it is important to balance
the needs of wider groups with those of individual circumstances when making decisions about prescribing or when commissioning services. The small numbers of trainees involved in this evaluation mean that it would be spurious to provide calculations of statistical significance, but the trends are positive. It is also important to note that not all learning may be a result of the course. b) Impact upon skills Having to undertake a practical project was seen by many as a useful way of developing important and transferable skills. Though the population health projects did not run smoothly for many trainees, they acknowledged that there was much to be learned from things going wrong. Some
930
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Fig. 3 e Before and after comparison of ST4 trainees' self-perceived skills.
suggested that these project management skills were transferable to GPs' ongoing careers and that they also helped to facilitate a sense of empowerment and ability to create and sustain change. Doing a practical project is really good for getting skills. You can only really get experience by doing things in practice so the project teaches you what the barriers are and how to overcome them. (GP trainee) Most trainees said that a key skill they learnt related to using data, or recognising potential gaps or issues with data. Another area of learning was deficiencies with current practice record systems and datasets. One frustration was in using codes for diagnosis and codes for screening. It really highlighted the need for proper coding during
consultation. This was something I must admit I thought that GP partners were a little obsessed about but I now see the value of doing well (GP trainee) Understanding how to identify population needs was a key module competency and one that trainees reported that they achieved. the most worthwhile thing was learning how to do a needs assessment (GP trainee) Another key area of learning was the need to work with others and in teams to make change happen. Through undertaking a needs assessment trainees made links with local public health teams. Similarly, trainees spoke about how they had to motivate others to move their projects forward and use their leadership skills.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
I learnt the importance of involving others in the practice, so it is not an individuals project, but a whole practice thing (GP trainee) Engaging with other members of the practice team as well as potential other organisations was key to success of improving services for populations and important for longevity. Others suggested that their projects were not as successful as they would have liked due to not being able to engage other team members fully. Before the module one quarter of trainees thought they had strong leadership skills (see Fig. 3). After the module this had doubled (from 27% to 50%). There was a similar trend with perceived management skills (from 19% to 50%). There were some areas where trainees reported no change including basic and more specialised clinical skills, prescribing, and patient safety. There were some areas where the module might have been expected to show a difference but did not-this included organisational skills. c) Changing attitudes A key finding of this evaluation is that trainees developed an understanding of their role looking at the needs of wider populations and the role of GPs in commissioning. Issues such as allocation of resources and funding decisions had become clearer as well. Others suggested that they had more of a sense of what they could achieve as a GP and their central role in facilitating change. It certainly has changed the way I consider my role as a GP and the obligation I have to the entire practice population rather than just the patients I see day to day. (GP trainee) Trainees felt empowered to make simple changes to services that could have a significant impact on health outcomes. They also began to see themselves as agents of change as representatives safeguarding public well-being. Some trainees also described how they had changed the way they talk to patients. All of these factors came together to promote a change in the overall mind-set of some trainees. It is estimated that at least half of those who completed the module benefitted in this way. I have changed the way in which I think about my role as a GP and now give more consideration to my responsibilities in improving the health of the local population, not just the patients I have individual contact with. (GP trainee)
∙ Improvements in the way contact details are kept at practices; ∙ Improved information for health promotion; ∙ Better review processes for services for patients with learning disabilities; Some trainees felt that their projects had broader benefits in terms of re-establishing trust and communication with patients, especially among groups that may feel disempowered or experience significant health inequalities. These (the project) have been vital in promoting the positive message that the doctors in the surgery are committed to their health; and that they are taking an active role in managing their disease risk outside of compulsory health checks or QOF reviews. (GP trainee) Trainees and trainers felt that the projects had positive outcomes for the practice as a whole. Learning how to understand the needs of the practice population, identifying unmet needs and take steps to make improvement were key. Indeed, the projects were well received and practices made specific changes to processes, systems and protocols as a result. I think this is much better than doing an audit. I felt that my project really made a difference to my practice. The outcomes from my project were well received. We had meetings to discuss the findings and things have already changed as a result of what I found. Getting that positive feedback and seeing how you make a change was good. (GP trainee) GP trainers benefitted from attending the seminar series, but also from the outcomes of projects. The practice and I personally benefitted as there were a few ideas generated that questioned how we work, how to value our staff, how to develop our patient representative group, how to get younger people involved and empower them to generate ideas. (GP trainer)
Important lessons for educational delivery Box 3 provides a list of the topics covered in the five programme seminars. Box 3 List of seminar topics.
Impact on patients and practices The projects were undertaken over a short timeframe, so longterm benefits for patients or patient groups cannot be demonstrated, but the project reports suggest that there have been some immediate benefit for individual patients, families and wider communities in some instances. These have included: ∙ Better screening for conditions such as diabetes, CVD, osteoporosis, cancer; ∙ Improvements in self management;
931
Introduction to population health Project management Needs assessment Examples of population health projects Health coaching Resource allocation and ethics Data analysis Writing skills Life after VTS Project presentations (given by trainees)
932
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
The feedback on the seminar series was positive, with comments for improvement relating to the timing of the seminars as well as areas for development. The most frequently commented on seminar was on needs assessment and the importance of this session early in the module. It was the assessment of what the population needs that was most important. Learning how to do this was crucial and could be emphasised more in future. (Faculty member) Topics identified for future modules included sessions on influencing skills, economic analysis, how to write a report and how to co-produce with patients. The role of content experts was seen as positive in supporting learners. However, the content experts described the need for greater clarity in terms of their role, and sharing this early with learners and GP trainers. In particular, the role of content experts in giving early feedback (e.g. on project outlines to learners) was seen as positive. We have not been briefed about answering specific questions about projects. There should be more consistency across what the content experts are saying but there has been little information given to us so we are all just winging it. (Faculty member) Feedback from trainers suggests that more could be done to engage and support them to be part of the population health module. Whilst nine out of ten trainers thought that it was important for GPs to be able to plan improvements for their practice population (95%) and agreed that doing an improvement project during GP training was a good idea (89%), views about the population health module itself were mixed (see Fig. 3). A number of trainers said that they did not fully understand the concept of population health or the value of including this in an ST4 year. Due to the part time nature of the ST4 year within general practice, I felt that my trainee had to keep the project quite simple and I don't really see a difference between this and the audit/ SIP/ QIP that the ST3s already have to do. (GP trainer) In telephone interviews and survey responses, trainers said they did not feel involved in the programme. They largely said they were not given information or reminders about seminar dates, that they were not clear about the submission requirements and that they were not contacted throughout the module for support or feedback. Material was placed on an online learning environment, which trainers had access to, but trainers said that this was difficult to access or inconvenient. Trainees and faculty felt that there was more work to be done to promote the population health projects as practice projects, rather than projects for an individual trainee. This includes getting trainers to mark individual project write-ups, and require trainees to present their projects to the whole practice. Trainees who decided not to engage with the programme offered a number of reasons including being advised by their GP trainers that their learning needs required they
concentrate on other learning aspects during the course of the fourth year, and being in a practice that would not accommodate the requirements of the programme to be able to attend the learning events or engage with the population health project. The project team used the online web presence to create a community of combined learning. However, this did not work so well partly because of accessing the site and partly due to the technical challenges of the site itself. It was suggested that an alternative might be to develop a facebook-style web page that was more easily accessible and user-friendly.
Discussion Summary of findings Feedback from trainees, trainers, faculty and review of project reports suggests that the population health curriculum has had an impact on the majority of eligible trainees. Only one fifth chose not to engage with the module. Two thirds completed the practical project. Practical projects and selfreport surveys suggested experience in identifying population health issues, collecting and using data for planning purposes, working with other professionals to implement improvement and other learning outcomes. Trainee projects were small-scale and completed over a short timeframe, but there were reportedly benefits for patient and population outcomes including better engagement with individuals and groups, an increase in the uptake of screening for various conditions and identification of conditions that may not otherwise have been picked up for some time (such as cancer, diabetes and hypertension). Key success factors comprised contextualising population health as an essential skill, facilitating learning through a practical project, running seminars regularly and having support and feedback available. Challenges included ad hoc planning, trainers and trainees not allocating enough time to prioritise the module, insufficiently engaging trainers and practices and lack of consistent support from content experts and trainers.
Limitations Any GP trainee undertaking the population health module did so on a voluntary basis. This self-selected group meant that the impact of such training on a broader range of GP trainees is unclear. The evaluation cannot tell us whether the positive trends seen in the questionnaires administered are statistically significant and sustainable in the longer term. The numbers involved in the evaluation were too small to allow this sort of analysis e instead the focus was on any broad trends. The study was not powered to test for differences. The authors sought to demonstrate the potential value of ongoing testing, but cannot state that the module made a definite change. As a result of the evaluation, the GP School developed the module further and tested the changes successfully during 2013e14.
p u b l i c h e a l t h 1 2 8 ( 2 0 1 4 ) 9 2 5 e9 3 3
Implications for practice This evaluation demonstrates that participating in the programme leads to a perceived increase in knowledge and awareness of population health, and may be a powerful approach to supporting the development of future GPs for their roles as commissioners. Though population health has mention in legal frameworks (e.g. the Health and Social Care Act), it has yet to be incorporated within the curriculum for GP training. The use of a service improvement project has benefits that extend beyond population health to include competencies covered by a number of other curriculum statements (including the GP in the Wider Professional Environment). The authors would suggest that such an approach provides greater opportunities for professional development than traditional assessments such as undertaking an audit. Whilst this programme has been piloted in the fourth year of GP training it is important to see how this might be incorporated within the standard three year training programme. The rising costs of providing health care mean that developing GPs skilled in improving population health outcomes will become an increasingly important part of their work. The authors anticipate that such approaches will be vital in ensuring that the NHS moves from a reactive to proactive health care service. Perhaps the time has come to consider incorporating population health outcomes within the Royal College of General Practitioners curriculum so that all GPs are equipped to make a difference to outcomes for populations as well as individuals.
Author statements Ethical approval NHS research ethics committee noted that no approval was required because this is an educational evaluation.
Funding None declared.
Competing interests None declared.
933
references
1. Douglas L, Szatkowski L. Socioeconomic variations in access to smoking cessation interventions in UK primary care: insights using the Mosaic classification in a large dataset of primary care records. BMC Public Health 2013;13(1):546. 2. Samaranayaka S, Gulliford MC. Trends in cardiovascular risk factors among people with diabetes in a population based study, Health Survey for England 1994e2009. Prim Care Diabetes (Published online May 2013). 3. Dalton AR, Bottle A, Soljak M, Majeed A, Millett C. Ethnic group differences in cardiovascular risk assessment scores: national cross-sectional study. Ethn Health (Published online May 2013). E, Gallagher JE. 4. Al-Haboubi M, Klass C, Jones K, Bernabe Inequalities in the use of dental services among adults in inner South East London. Eur J Oral Sci 2013;121(3 Pt 1):176e81. 5. www.gov.uk/government/uploads/system/uploads/ attachment_data/file/153485/dh_131897.pdf.pdf. 6. www.gov.uk/government/uploads/system/uploads/ attachment_data/file/127045/Public-health-role-of-localauthorities-factsheet.pdf.pdf. 7. www.scotland.gov.uk/Publications/2010/05/10102307/0. 8. Kindig D, Stoddart G. What is population health? Am J Public Health 2003;93(3):380e3. 9. Carson-Stevens A, Patel E, Nutt SL, Bhatt J, Panesar SS. The social movement drive: a role for junior doctors in healthcare reform. J R Soc Med (Published online June 2013). 10. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet 2006;367(9524):1747e57. 11. Wills J, Reynolds J, Swanwick T. “Just a lovely luxury?” What can public health attachments add to postgraduate general practice training? Educ Prim Care 2009;20:278e84. 12. Harris MF, Zwar NA. Care of patients with chronic disease: the challenge for general practice. Med J Aust 2007;187:104e7. 13. Riley B, Rughani A. The future of GP specialty training: enhanced and extended. BJGP June 1, 2012;62:288e9. 14. Royal Australian College of General Practitioners. The RACGP curriculum for Australian general practice: population health and public health (accessed 16 January 2012), http://www.racgp.org. au/Content/NavigationMenu/educationandtraining/ Trainingcurriculum/11Populationhealthandpublichealth.pdf; 2009. 15. Australian College of Rural and Remote Medicine. ACCRM: primary curriculum (accessed 16 January 201), http://www.acrrm. org.au/files/uploads/pdf/curriculum/primary/ACRRM-PrimaryCurriculum_3rd-edition_25-09-09_with-cover.pdf; 2009. 16. Lisa J, Chamberlain LJ, Wang NE, Ho ET, Banchoff AW, Braddock CH, Gesundheit N. Integrating collaborative population health projects into a medical student curriculum at Stanford. Acad Med 2008;83:338e44.
