Editorial
Surveys of postgraduate medical trainees in the UK
Annual General Medical Council trainee survey
In the UK, the best and most comprehensive trainee survey is run by the General Medical Council (2014). Each year, all trainees with a national training number are sent a link to complete the survey online. The response rate is very high (generally >90%). This is partly a result of the ability of the General Medical Council to lever participation as a professional responsibility and expectation and to mobilize the local education and training 666
boards to chase non-responders. The questions are carefully written to avoid leading answers, are relevant, and patterns of response can be tracked over time. One drawback of the General Medical Council survey is that it is relatively long; it can take over 30 minutes to complete. Although the delay from completion to reporting has improved, there is still a time lag which means trainees may have moved on before concerns raised in the survey are made known to an organization. Analysis of responses suggests that trainees who are disaffected or are generally unhappy about their training may score all the domains and questions poorly rather than considering each carefully. A recent addition to the survey is to allow trainees to fill out ‘white-space’ questions if they have concerns regarding patient safety. This has allowed the survey to extend its remit from purely a measure of educational satisfaction. However, the wording of these patient safety questions is important since the levels of reporting have varied sharply depending on the precise question being asked.
Other national surveys
Apart from the General Medical Council, several other nationwide surveys of medical trainees have been administered in the past. Some, e.g. the Foundation School Survey, are targeted at a specific subset of trainees while others, e.g. those administered through social media, are generally focussed on narrower issues, such as whistleblowing or views regarding end-of-life care. The Medical Protection Society is a large organization that provides professional indemnity to doctors in the UK. Recently they have started to carry out surveys of trainee doctors with the results published in 2013 and again in 2014 (Medical Protection Society, 2014). In the most recent survey of foundation doctors, 70% felt they did not have enough time to give the patients the care they needed, half had raised concerns about quality of care
in their workplace and 82% struggled with long hours. However, the survey included responses only from just over 1000 doctors, a fraction of the 15 000 foundation trainees in the UK. A British Medical Association survey also reported that half of all doctors cited a lack of capacity or time as a barrier to delivering patient care. Unlike the General Medical Council, surveys administered by the British Medical Association may be more likely to focus on the terms and conditions of service rather than broader questions of educational excellence. This particular study is actually a small cohort study of 431 doctors who qualified in 2006 (British Medical Association, 2014).
Regional and local surveys
Regionally, the local education and training boards have a variety of survey tools, but none have the rigour of the national General Medical Council survey. However, they do allow questions to be geared to issues identified as being of particular concern to that locality. At the individual hospital or general practice level, surveys are sometimes carried out. Although they have the benefit of local responsiveness, they raise issues of confidentiality since respondents are more easily identified in small local surveys. There has been a tendency to dissuade too many locally designed and administered surveys, partly because of concerns around ‘survey fatigue’. If trainees are asked to complete multiple surveys, often asking duplicate questions, sometimes poorly designed and time consuming, they may disengage from completing the national General Medical Council survey.
Value of surveys
What do these results tell us? First, they paint a reassuring picture that doctors continue to aspire to the highest clinical standards, even when faced with chronic time and capacity constraints. They also show that doctors feel confident to report
© 2014 MA Healthcare Ltd
Q
uality improvement initiatives generally collect data from a variety of sources, and increasing importance has been placed on listening to the views of key stakeholders. In medical education, the perceptions of postgraduate medical trainees are an important mechanism for identifying areas of good practice and driving improvements where deficiencies exist. Trainees can be asked to comment on a variety of domains including the standard of education and training, conditions of service, and the quality and safety of clinical services in the places they work. The views of trainees can be collected in a number of ways including focus groups, feedback forms and exit interviews, but the most common tool used is the structured survey. The advent of turn-key online survey solutions has led to a proliferation of surveys, as it is now relatively simple to produce and disseminate a list of questions to a large number of individuals. However, all surveys must be interpreted with caution and survey design is of paramount importance. Key pitfalls in survey design include selection bias, poor response rate, leading questions, and questions that identify broad problems but are not specific enough to help find solutions. Surveys should always be piloted before wider dissemination. It is important to realize that in nearly all surveys, it is perceptions that are being measured, rather than facts.
British Journal of Hospital Medicine, December 2014, Vol 75, No 12
h Journal of Hospital Medicine. Downloaded from magonlinelibrary.com by 139.080.123.050 on December 11, 2015. For personal use only. No other uses without permission. . All rights rese
Editorial concerns in the workplace. In the education and training domain, although outliers do exist, the overall trend has been towards improved satisfaction. The survey findings regarding long hours at work is a good example of how trainee perceptions and expectations have changed over the years. Twenty years ago, it was common for doctors to work 140 hours a week, with shifts lasting as long as 80 hours (Friday 9 am until Monday 5 pm). The European Working Time Directive limits the average working week to only 48 hours. There is ample evidence that many trainees work in excess of their rostered hours, but it is highly unlikely that any work 24 hours without a break, let alone the 80 continuous hours of the past. No doubt the intensity of the work has increased and the number of patients has risen, but there is incontrovertible evidence that working hours have fallen, and doctors no longer work much longer hours than many other employees, although those hours may be unsociable. Nevertheless, surveys continue to report that trainees perceive that their jobs require very long hours at work.
Use of surveys for other purposes
© 2014 MA Healthcare Ltd
There are concerns that the value and integrity of surveys may be undermined if they are used for reasons other than those for which they were intended. The General Medical Council trainee survey allows individual local education and training boards, hospitals and specialities to be compared nationally. Indeed, the General Medical Council survey reports ‘red triangle’ outliers when local results are significantly worse than the national average. While this may help individuals see where
their weaknesses are, the results have also been used to produce ‘league tables’ of trainee satisfaction. They have even been used as metrics to inform decisions about where training posts may be removed in the future. One needs to tread carefully here, since this may undermine the survey itself. For example, if I am surveyed about my small local post office, I may report that it gives me a very poor service, but if I thought that might lead to it being closed down, I might be more inclined to give it a better report, since a poor post office would be better than none at all. Other concerns include the fact that some questions are biased towards large teaching hospitals (access to educational resources) or super-specialized centres with low volumes of patients (workload).
Conclusions
The General Medical Council national survey is the best survey available that reports the perception of trainees in the UK, although care should be taken when using it as a major metric in deciding to decommission posts. Other surveys may be valuable but the response rate is often poor and selective, and it is important to consider the nature of the sponsoring organization. The results of surveys are best triangulated with other data (e.g. quality visits)
and seen in historical context. Individuals and organizations considering a survey should think carefully about the aims and design, and should consider whether excessive surveying will damage the response rate and accuracy of existing surveys. BJHM
John Alcolado
Director, North East Thames Foundation School/Consultant Physician Education Centre Queens Hospital Romford Essex RM7 OAG ([email protected])
Ishan Pankhania/Julian Siah/ Afsheen Khaku
Foundation Year 2 Doctor/Medical Education Fellow/Medical Education Fellow Academic Centre Queens Hospital Romford Essex British Medical Association (2014) BMA Cohort Doctor study. bma.org.uk/cohortstudy (accessed 1 October 2014) General Medical Council (2014) National training surveys. www.gmc-uk.org/education/surveys.asp (accessed 1 October 2014) Medical Protection Society (2014) Half of junior doctors have quality care concerns – supportive open culture needed. www.medicalprotection. org/uk/press-releases/half-of-junior-doctors-havequality-care-concerns (accessed 1 October 2014)
KEY POINTS n Surveys of postgraduate medical trainees provide valuable data regarding the quality of medical education. n The General Medical Council trainee survey is the most robust and comprehensive survey in the UK. n Surveys must be interpreted with caution, with the results triangulated with data from other sources. n Care should be exercised when using survey data to construct league tables.
Follow us on Twitter @bjhospmed and join the debate
British Journal of Hospital Medicine, December 2014, Vol 75, No 12
667
h Journal of Hospital Medicine. Downloaded from magonlinelibrary.com by 139.080.123.050 on December 11, 2015. For personal use only. No other uses without permission. . All rights rese
Surveys of postgraduate medical trainees in the UK
Annual General Medical Council trainee survey
In the UK, the best and most comprehensive trainee survey is run by the General Medical Council (2014). Each year, all trainees with a national training number are sent a link to complete the survey online. The response rate is very high (generally >90%). This is partly a result of the ability of the General Medical Council to lever participation as a professional responsibility and expectation and to mobilize the local education and training 666
boards to chase non-responders. The questions are carefully written to avoid leading answers, are relevant, and patterns of response can be tracked over time. One drawback of the General Medical Council survey is that it is relatively long; it can take over 30 minutes to complete. Although the delay from completion to reporting has improved, there is still a time lag which means trainees may have moved on before concerns raised in the survey are made known to an organization. Analysis of responses suggests that trainees who are disaffected or are generally unhappy about their training may score all the domains and questions poorly rather than considering each carefully. A recent addition to the survey is to allow trainees to fill out ‘white-space’ questions if they have concerns regarding patient safety. This has allowed the survey to extend its remit from purely a measure of educational satisfaction. However, the wording of these patient safety questions is important since the levels of reporting have varied sharply depending on the precise question being asked.
Other national surveys
Apart from the General Medical Council, several other nationwide surveys of medical trainees have been administered in the past. Some, e.g. the Foundation School Survey, are targeted at a specific subset of trainees while others, e.g. those administered through social media, are generally focussed on narrower issues, such as whistleblowing or views regarding end-of-life care. The Medical Protection Society is a large organization that provides professional indemnity to doctors in the UK. Recently they have started to carry out surveys of trainee doctors with the results published in 2013 and again in 2014 (Medical Protection Society, 2014). In the most recent survey of foundation doctors, 70% felt they did not have enough time to give the patients the care they needed, half had raised concerns about quality of care
in their workplace and 82% struggled with long hours. However, the survey included responses only from just over 1000 doctors, a fraction of the 15 000 foundation trainees in the UK. A British Medical Association survey also reported that half of all doctors cited a lack of capacity or time as a barrier to delivering patient care. Unlike the General Medical Council, surveys administered by the British Medical Association may be more likely to focus on the terms and conditions of service rather than broader questions of educational excellence. This particular study is actually a small cohort study of 431 doctors who qualified in 2006 (British Medical Association, 2014).
Regional and local surveys
Regionally, the local education and training boards have a variety of survey tools, but none have the rigour of the national General Medical Council survey. However, they do allow questions to be geared to issues identified as being of particular concern to that locality. At the individual hospital or general practice level, surveys are sometimes carried out. Although they have the benefit of local responsiveness, they raise issues of confidentiality since respondents are more easily identified in small local surveys. There has been a tendency to dissuade too many locally designed and administered surveys, partly because of concerns around ‘survey fatigue’. If trainees are asked to complete multiple surveys, often asking duplicate questions, sometimes poorly designed and time consuming, they may disengage from completing the national General Medical Council survey.
Value of surveys
What do these results tell us? First, they paint a reassuring picture that doctors continue to aspire to the highest clinical standards, even when faced with chronic time and capacity constraints. They also show that doctors feel confident to report
© 2014 MA Healthcare Ltd
Q
uality improvement initiatives generally collect data from a variety of sources, and increasing importance has been placed on listening to the views of key stakeholders. In medical education, the perceptions of postgraduate medical trainees are an important mechanism for identifying areas of good practice and driving improvements where deficiencies exist. Trainees can be asked to comment on a variety of domains including the standard of education and training, conditions of service, and the quality and safety of clinical services in the places they work. The views of trainees can be collected in a number of ways including focus groups, feedback forms and exit interviews, but the most common tool used is the structured survey. The advent of turn-key online survey solutions has led to a proliferation of surveys, as it is now relatively simple to produce and disseminate a list of questions to a large number of individuals. However, all surveys must be interpreted with caution and survey design is of paramount importance. Key pitfalls in survey design include selection bias, poor response rate, leading questions, and questions that identify broad problems but are not specific enough to help find solutions. Surveys should always be piloted before wider dissemination. It is important to realize that in nearly all surveys, it is perceptions that are being measured, rather than facts.
British Journal of Hospital Medicine, December 2014, Vol 75, No 12
h Journal of Hospital Medicine. Downloaded from magonlinelibrary.com by 139.080.123.050 on December 11, 2015. For personal use only. No other uses without permission. . All rights rese
Editorial concerns in the workplace. In the education and training domain, although outliers do exist, the overall trend has been towards improved satisfaction. The survey findings regarding long hours at work is a good example of how trainee perceptions and expectations have changed over the years. Twenty years ago, it was common for doctors to work 140 hours a week, with shifts lasting as long as 80 hours (Friday 9 am until Monday 5 pm). The European Working Time Directive limits the average working week to only 48 hours. There is ample evidence that many trainees work in excess of their rostered hours, but it is highly unlikely that any work 24 hours without a break, let alone the 80 continuous hours of the past. No doubt the intensity of the work has increased and the number of patients has risen, but there is incontrovertible evidence that working hours have fallen, and doctors no longer work much longer hours than many other employees, although those hours may be unsociable. Nevertheless, surveys continue to report that trainees perceive that their jobs require very long hours at work.
Use of surveys for other purposes
© 2014 MA Healthcare Ltd
There are concerns that the value and integrity of surveys may be undermined if they are used for reasons other than those for which they were intended. The General Medical Council trainee survey allows individual local education and training boards, hospitals and specialities to be compared nationally. Indeed, the General Medical Council survey reports ‘red triangle’ outliers when local results are significantly worse than the national average. While this may help individuals see where
their weaknesses are, the results have also been used to produce ‘league tables’ of trainee satisfaction. They have even been used as metrics to inform decisions about where training posts may be removed in the future. One needs to tread carefully here, since this may undermine the survey itself. For example, if I am surveyed about my small local post office, I may report that it gives me a very poor service, but if I thought that might lead to it being closed down, I might be more inclined to give it a better report, since a poor post office would be better than none at all. Other concerns include the fact that some questions are biased towards large teaching hospitals (access to educational resources) or super-specialized centres with low volumes of patients (workload).
Conclusions
The General Medical Council national survey is the best survey available that reports the perception of trainees in the UK, although care should be taken when using it as a major metric in deciding to decommission posts. Other surveys may be valuable but the response rate is often poor and selective, and it is important to consider the nature of the sponsoring organization. The results of surveys are best triangulated with other data (e.g. quality visits)
and seen in historical context. Individuals and organizations considering a survey should think carefully about the aims and design, and should consider whether excessive surveying will damage the response rate and accuracy of existing surveys. BJHM
John Alcolado
Director, North East Thames Foundation School/Consultant Physician Education Centre Queens Hospital Romford Essex RM7 OAG ([email protected])
Ishan Pankhania/Julian Siah/ Afsheen Khaku
Foundation Year 2 Doctor/Medical Education Fellow/Medical Education Fellow Academic Centre Queens Hospital Romford Essex British Medical Association (2014) BMA Cohort Doctor study. bma.org.uk/cohortstudy (accessed 1 October 2014) General Medical Council (2014) National training surveys. www.gmc-uk.org/education/surveys.asp (accessed 1 October 2014) Medical Protection Society (2014) Half of junior doctors have quality care concerns – supportive open culture needed. www.medicalprotection. org/uk/press-releases/half-of-junior-doctors-havequality-care-concerns (accessed 1 October 2014)
KEY POINTS n Surveys of postgraduate medical trainees provide valuable data regarding the quality of medical education. n The General Medical Council trainee survey is the most robust and comprehensive survey in the UK. n Surveys must be interpreted with caution, with the results triangulated with data from other sources. n Care should be exercised when using survey data to construct league tables.
Follow us on Twitter @bjhospmed and join the debate
British Journal of Hospital Medicine, December 2014, Vol 75, No 12
667
h Journal of Hospital Medicine. Downloaded from magonlinelibrary.com by 139.080.123.050 on December 11, 2015. For personal use only. No other uses without permission. . All rights rese
