Prescribing
A prescribing e–tutorial for medical students Finneas Catling and Jane Williams, Medical School, University of Bristol, UK Robert Baker, Medical School, University of Bristol, Bristol and Musgrove Park Hospital, Taunton, UK
SUMMARY Background: UK medical students’ confidence in their prescribing skills is low, and a significant proportion of prescriptions written by foundation year 1 (FY1) doctors contain errors. The Prescribing Safety Assessment (PSA) is a new national examination aimed at ensuring prescribing competence in undergraduates, but few PSAspecific preparatory resources are available to students. Methods: A needs analysis was performed and an online e–tutorial (Prepare for the PSA) was designed. The e–tutorial consists mainly of a practise exam that
closely mirrors the format of the PSA itself. After completing each question the user receives personalised feedback, and is given the opportunity to further their knowledge using interactive learning activities. A facility for users to contribute their own questions is included, and the e–tutorial also explains the rationale underlying the PSA and directs users to further learning resources. Numerous attempts were made to encourage the use of the e–tutorial amongst UK medical students and educators. A total of 248 final-year students at a UK medical school were asked to evaluate the e–tutorial
and rate their prescribing confidence before and after completing it. Results: The response rate in the evaluation was 72.7–73.9 per cent. The results show that completing the e–tutorial is associated with highly significant (p < 0.0001) increases in confidence across all prescribing skills. Discussion: Our e–tutorial is effective and meets the needs of medical students. Once the e–tutorial is disseminated more widely and is expanded by usercontributed questions, we hope that it will become established as a national platform for collaborative prescribing education.
Few PSA-specific preparatory resources are available to students
© 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37 33
tct_12100.indd 33
1/2/2014 11:04:44 AM
It will be necessary for candidates to pass the PSA in order to progress to graduation
INTRODUCTION
I
t is essential to patient safety and wellbeing that foundation doctors prescribe appropriately; however, the EQUIP study identified errors in 8.4 per cent of the 50 016 prescriptions written by foundation year 1 doctors reviewed in 2009. Qualitative data from the same study suggested that undergraduate medical education programmes have failed to provide adequate training in practical prescribing.1 Of the 2514 medical students surveyed during 2006–2008, only 27 per cent felt confident that they would possess the core skills outlined by the General Medical Council at the time of graduation.2,3
The Prescribing Safety Assessment (PSA) is a new national examination in the UK aimed at ensuring prescribing competence in final-year medical students, which is currently being piloted prior to its full implementation during the 2013/14 academic year. The proposed exam is 120 minutes long, is computer-based and contains eight question types, covering topics in medicine, surgery and the common specialities (see Table 1). It will be necessary for candidates to pass the PSA in order to progress to graduation and work as a foundation doctor, but, despite this, few specific preparatory resources are currently available whereby prospective candidates can test their knowledge and familiarise themselves with the exam format.4,5
DEVELOPMENT We conducted a needs analysis by surveying 45 penultimate-year students at a UK medical school: 82 per cent of the students felt that their teaching had prepared them slightly well or not at all well for prescribing duties as a foundation doctor. The vast majority of students felt not at
Table 1. The Prescribing Skills Assessment (PSA) question types PSA question type
Description in feedback survey
Prescribing
Writing a safe, effective and legal prescription in response to a clinical scenario.
Prescription review
Reviewing a prescription, and deciding which medications are inappropriate, ineffective or unsafe in the clinical circumstances.
Planning management
Choosing the most appropriate combination of therapies (medication, fluid, etc.) in a clinical situation.
Communicating information
Deciding which information is appropriate to communicate to a patient about their medication.
Calculation skills
Making an accurate drug dosage calculation based on numerical information.
Adverse drug reactions
Identifying and managing an adverse drug reaction, and spotting potentially dangerous drug interactions.
Drug monitoring
Deciding on how to monitor the beneficial and harmful effects of a medicine.
Data interpretation
Interpreting investigation results and appropriately altering a prescription based on this.
all confident, slightly confident or moderately confident in their ability to perform each PSA-assessed prescribing skill, with slightly confident being the most common response. When asked which features would be most helpful to include in an e–tutorial, practise examination questions, self-test quizzes and interactive clinical scenarios were the most popular responses. Recent research highlights the potential of e–learning innovations in prescribing education, including interactive learner systems that allow users to practise prescribing skills in the virtual environment and receive immediate feedback.5–7 Based on this potential and the needsanalysis results, we developed Prepare for the PSA (PftPSA), an e–tutorial for medical students
who are getting ready to take the PSA.8 Eleven penultimate-year medical students evaluated a draft version of the e–tutorial and provided free-text comments regarding features they liked and disliked, any technical problems they encountered and suggested improvements. Numerous refinements were made on the basis of this feedback. PftPSA is now available online.8 Access is free and unrestricted, and all original content is made available under a Creative Commons licence to allow other educators to use this resource. The e–tutorial begins by introducing the PSA, explaining its structure and the rationale underlying it, and directs users to further learning resources. The main section of PftPSA is a practise exam that closely mirrors
34 © 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37
tct_12100.indd 34
1/2/2014 11:04:45 AM
the format and appearance of the PSA itself: each question was written according to the guidance in the PSA Question Writing Manual, and was moderated by a senior clinician. An optional countdown timer simulates exam conditions. Figure 1 is an example of an e-tutorial question. After completing each question the user receives personalised feedback and is given the opportunity to further their knowledge via a related interactive learning activity (e.g. a question on the initial management of diabetic ketoacidosis is followed by a series of scenarios that cover the later treatment of the same patient). Figure 2 shows an example of e-tutorial feedback and interactive learning activity. When the user chooses to end the practise exam, their performance is summarised and they are invited to review their learning objectives. Every question and learning activity lists the research literature upon which it is based. As in the PSA, the content in PftPSA was primarily derived from the information contained in the British National Formulary (BNF). Where such information was insufficient, content was based on the major clinical guidelines available. Each practise exam in PftPSA is one-quarter the length of the PSA and is generated randomly from a question bank. When a user encounters a question that they have answered before, they are given the option to swap it for another of the same type. The exam is designed to be expandable: users can contribute their own questions via a user-friendly web form, and these are added to the bank after being moderated to ensure quality control. Considerable effort was expended to produce an attractive, web application-like interface, in keeping with the expectations of modern Internet users. Accessibility was maximised by providing advice on navigation
Accessibility was maximised by providing advice on navigation and troubleshooting
Figure 1. Example e-tutorial question
Figure 2. Example feedback and interactive learning activity
and troubleshooting, and by testing compatibility with different browsers and mobile devices. Several features, including a menu for skipping between questions, an integrated calculator, a progress bar and links to the BNF Online were included to enhance userfriendliness.
DISSEMINATION Numerous attempts were made to encourage the use of PftPSA amongst UK medical students and educators. PftPSA has been promoted on a national prescribing website and was presented at an international pharmacology conference, where it was demonstrated to the heads of Prescribing and Therapeutics at several UK medical schools.9,10 PftPSA has also been promoted via lectures to third-year medical students and at a general practitioner (GP) training day,
and via numerous e–mail and website announcements. The practise exam in PftPSA was used as a basis for practical prescribing workshops at a student conference, and these sessions received very positive feedback from the attendees. The e–tutorial is integrated with several popular social media websites, including a companion Facebook page that provides updates on recent changes to the e–tutorial and allows users to show their support by ‘liking’ it. Web analytics data for PftPSA show 572 visits from locations outside Bristol, suggesting that the e–tutorial is gaining national popularity.
EVALUATION The 2013 PSA pilot provided an ideal opportunity to evaluate PftPSA in our target population. We produced an anonymous
© 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37 35
tct_12100.indd 35
1/2/2014 11:04:45 AM
Completing the PftPSA is associated with highly significant increases in confidence across all prescribing skills
survey that asked respondents to provide free-text feedback and rate their prescribing confidence before and after completion of PftPSA (see Table 1). A year group of final-year medical students (n = 248) were contacted via multiple emails in the 2 weeks preceding the PSA pilot, and were offered the opportunity to complete PftPSA and an online version of the survey. Paper versions of the survey, including questions to identify duplicate responses and to measure the uptake of PftPSA, were offered to participating students (n = 247) on the day of the PSA pilot. A total of 207 non-duplicate survey responses were received; 137 respondents reported having completed PftPSA and 119–121 provided data on their pre- and post-PftPSA confidence in each prescribing skill. Extrapolating from the paper survey results, 66.0 per cent (95% CI 64.9– 67.1%) of the entire year group Proportions of students (117≤n≤121)
Pre Post
Pre Post
Pre Post
completed PftPSA, and the proportion of these students who also reported their prescribing confidence (i.e. the response rate) was 72.7–73.9 per cent (95% CIs 71.5–72.7% and 73.9–75.2%). This is hypothesised to represent a conservative estimate of the response rate, as students who completed PftPSA are likely to be overrepresented in the group who responded to the survey. Prescribing confidence data for students who completed the survey before and after the PSA pilot were not found to be significantly different on Mann– Whitney tests (1444 ≤ U ≤ 1716; –0.65 ≤ Z ≤ 0.97; 0.32 ≤ p(two-tail) ≤ 0.98), and these subgroups were subsequently combined. Prescribing confidence data were graphed (see Figure 3) and analysed using Wilcoxon signed-rank tests (see Table 2), showing that completing PftPSA is associated with highly significant increases in confidence across all prescribing skills.
Pre Post
Pre Post
Pre Post
Pre Post
Pre Post
100% 80% 60% 40% 20%
Several common themes were identified in the free-text feedback responses: the vast majority of students felt the e–tutorial helped them to meet the specified learning objectives, and that its design was clear and user-friendly. Many students who completed the feedback survey after the PSA pilot commented on the similarity of the PftPSA interface with that of the pilot. The most common suggestions for improvement of PftPSA were expansion of the question bank and refinement of the countdown timer.
LIMITATIONS Our evaluation used a subjective surrogate outcome measure (selfreported prescribing confidence), which has not been formally validated as corresponding to prescribing outcomes in the clinical environment. We felt this was appropriate for the following reasons. • Formally measuring these clinical outcomes is very difficult in practice, especially in a student cohort with no specified prescribing obligations to use as a basis for measurement. • Objective surrogate outcome measures (e.g. individual pilot PSA test scores) were not released for use by the researchers.
0%
Not at all confident
Slightly confident
Very confident
Extremely confident
• A recent study reported significant increases in short- and long-term prescribing confidence (measured
Moderately confident
Figure 3. A graphical representation of the prescribing confidence data
Table 2. Analysis of the prescribing confidence data using Wilcoxon signed-rank tests Prescribing
Prescription Planning CommuCalculation review management nicating skills information
Adverse drug Drug monireactions toring
Data interpretation
n
121
119
121
121
120
120
120
120
W
–817
–987
–946
–1114
–1971
–1280
–1439
–1224
Z
–4.76
–5.39
–5.71
–5.22
–6.44
–5.83
–5.87
–5.73
A prescribing e–tutorial for medical students Finneas Catling and Jane Williams, Medical School, University of Bristol, UK Robert Baker, Medical School, University of Bristol, Bristol and Musgrove Park Hospital, Taunton, UK
SUMMARY Background: UK medical students’ confidence in their prescribing skills is low, and a significant proportion of prescriptions written by foundation year 1 (FY1) doctors contain errors. The Prescribing Safety Assessment (PSA) is a new national examination aimed at ensuring prescribing competence in undergraduates, but few PSAspecific preparatory resources are available to students. Methods: A needs analysis was performed and an online e–tutorial (Prepare for the PSA) was designed. The e–tutorial consists mainly of a practise exam that
closely mirrors the format of the PSA itself. After completing each question the user receives personalised feedback, and is given the opportunity to further their knowledge using interactive learning activities. A facility for users to contribute their own questions is included, and the e–tutorial also explains the rationale underlying the PSA and directs users to further learning resources. Numerous attempts were made to encourage the use of the e–tutorial amongst UK medical students and educators. A total of 248 final-year students at a UK medical school were asked to evaluate the e–tutorial
and rate their prescribing confidence before and after completing it. Results: The response rate in the evaluation was 72.7–73.9 per cent. The results show that completing the e–tutorial is associated with highly significant (p < 0.0001) increases in confidence across all prescribing skills. Discussion: Our e–tutorial is effective and meets the needs of medical students. Once the e–tutorial is disseminated more widely and is expanded by usercontributed questions, we hope that it will become established as a national platform for collaborative prescribing education.
Few PSA-specific preparatory resources are available to students
© 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37 33
tct_12100.indd 33
1/2/2014 11:04:44 AM
It will be necessary for candidates to pass the PSA in order to progress to graduation
INTRODUCTION
I
t is essential to patient safety and wellbeing that foundation doctors prescribe appropriately; however, the EQUIP study identified errors in 8.4 per cent of the 50 016 prescriptions written by foundation year 1 doctors reviewed in 2009. Qualitative data from the same study suggested that undergraduate medical education programmes have failed to provide adequate training in practical prescribing.1 Of the 2514 medical students surveyed during 2006–2008, only 27 per cent felt confident that they would possess the core skills outlined by the General Medical Council at the time of graduation.2,3
The Prescribing Safety Assessment (PSA) is a new national examination in the UK aimed at ensuring prescribing competence in final-year medical students, which is currently being piloted prior to its full implementation during the 2013/14 academic year. The proposed exam is 120 minutes long, is computer-based and contains eight question types, covering topics in medicine, surgery and the common specialities (see Table 1). It will be necessary for candidates to pass the PSA in order to progress to graduation and work as a foundation doctor, but, despite this, few specific preparatory resources are currently available whereby prospective candidates can test their knowledge and familiarise themselves with the exam format.4,5
DEVELOPMENT We conducted a needs analysis by surveying 45 penultimate-year students at a UK medical school: 82 per cent of the students felt that their teaching had prepared them slightly well or not at all well for prescribing duties as a foundation doctor. The vast majority of students felt not at
Table 1. The Prescribing Skills Assessment (PSA) question types PSA question type
Description in feedback survey
Prescribing
Writing a safe, effective and legal prescription in response to a clinical scenario.
Prescription review
Reviewing a prescription, and deciding which medications are inappropriate, ineffective or unsafe in the clinical circumstances.
Planning management
Choosing the most appropriate combination of therapies (medication, fluid, etc.) in a clinical situation.
Communicating information
Deciding which information is appropriate to communicate to a patient about their medication.
Calculation skills
Making an accurate drug dosage calculation based on numerical information.
Adverse drug reactions
Identifying and managing an adverse drug reaction, and spotting potentially dangerous drug interactions.
Drug monitoring
Deciding on how to monitor the beneficial and harmful effects of a medicine.
Data interpretation
Interpreting investigation results and appropriately altering a prescription based on this.
all confident, slightly confident or moderately confident in their ability to perform each PSA-assessed prescribing skill, with slightly confident being the most common response. When asked which features would be most helpful to include in an e–tutorial, practise examination questions, self-test quizzes and interactive clinical scenarios were the most popular responses. Recent research highlights the potential of e–learning innovations in prescribing education, including interactive learner systems that allow users to practise prescribing skills in the virtual environment and receive immediate feedback.5–7 Based on this potential and the needsanalysis results, we developed Prepare for the PSA (PftPSA), an e–tutorial for medical students
who are getting ready to take the PSA.8 Eleven penultimate-year medical students evaluated a draft version of the e–tutorial and provided free-text comments regarding features they liked and disliked, any technical problems they encountered and suggested improvements. Numerous refinements were made on the basis of this feedback. PftPSA is now available online.8 Access is free and unrestricted, and all original content is made available under a Creative Commons licence to allow other educators to use this resource. The e–tutorial begins by introducing the PSA, explaining its structure and the rationale underlying it, and directs users to further learning resources. The main section of PftPSA is a practise exam that closely mirrors
34 © 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37
tct_12100.indd 34
1/2/2014 11:04:45 AM
the format and appearance of the PSA itself: each question was written according to the guidance in the PSA Question Writing Manual, and was moderated by a senior clinician. An optional countdown timer simulates exam conditions. Figure 1 is an example of an e-tutorial question. After completing each question the user receives personalised feedback and is given the opportunity to further their knowledge via a related interactive learning activity (e.g. a question on the initial management of diabetic ketoacidosis is followed by a series of scenarios that cover the later treatment of the same patient). Figure 2 shows an example of e-tutorial feedback and interactive learning activity. When the user chooses to end the practise exam, their performance is summarised and they are invited to review their learning objectives. Every question and learning activity lists the research literature upon which it is based. As in the PSA, the content in PftPSA was primarily derived from the information contained in the British National Formulary (BNF). Where such information was insufficient, content was based on the major clinical guidelines available. Each practise exam in PftPSA is one-quarter the length of the PSA and is generated randomly from a question bank. When a user encounters a question that they have answered before, they are given the option to swap it for another of the same type. The exam is designed to be expandable: users can contribute their own questions via a user-friendly web form, and these are added to the bank after being moderated to ensure quality control. Considerable effort was expended to produce an attractive, web application-like interface, in keeping with the expectations of modern Internet users. Accessibility was maximised by providing advice on navigation
Accessibility was maximised by providing advice on navigation and troubleshooting
Figure 1. Example e-tutorial question
Figure 2. Example feedback and interactive learning activity
and troubleshooting, and by testing compatibility with different browsers and mobile devices. Several features, including a menu for skipping between questions, an integrated calculator, a progress bar and links to the BNF Online were included to enhance userfriendliness.
DISSEMINATION Numerous attempts were made to encourage the use of PftPSA amongst UK medical students and educators. PftPSA has been promoted on a national prescribing website and was presented at an international pharmacology conference, where it was demonstrated to the heads of Prescribing and Therapeutics at several UK medical schools.9,10 PftPSA has also been promoted via lectures to third-year medical students and at a general practitioner (GP) training day,
and via numerous e–mail and website announcements. The practise exam in PftPSA was used as a basis for practical prescribing workshops at a student conference, and these sessions received very positive feedback from the attendees. The e–tutorial is integrated with several popular social media websites, including a companion Facebook page that provides updates on recent changes to the e–tutorial and allows users to show their support by ‘liking’ it. Web analytics data for PftPSA show 572 visits from locations outside Bristol, suggesting that the e–tutorial is gaining national popularity.
EVALUATION The 2013 PSA pilot provided an ideal opportunity to evaluate PftPSA in our target population. We produced an anonymous
© 2014 John Wiley & Sons Ltd. THE CLINICAL TEACHER 2014; 11: 33–37 35
tct_12100.indd 35
1/2/2014 11:04:45 AM
Completing the PftPSA is associated with highly significant increases in confidence across all prescribing skills
survey that asked respondents to provide free-text feedback and rate their prescribing confidence before and after completion of PftPSA (see Table 1). A year group of final-year medical students (n = 248) were contacted via multiple emails in the 2 weeks preceding the PSA pilot, and were offered the opportunity to complete PftPSA and an online version of the survey. Paper versions of the survey, including questions to identify duplicate responses and to measure the uptake of PftPSA, were offered to participating students (n = 247) on the day of the PSA pilot. A total of 207 non-duplicate survey responses were received; 137 respondents reported having completed PftPSA and 119–121 provided data on their pre- and post-PftPSA confidence in each prescribing skill. Extrapolating from the paper survey results, 66.0 per cent (95% CI 64.9– 67.1%) of the entire year group Proportions of students (117≤n≤121)
Pre Post
Pre Post
Pre Post
completed PftPSA, and the proportion of these students who also reported their prescribing confidence (i.e. the response rate) was 72.7–73.9 per cent (95% CIs 71.5–72.7% and 73.9–75.2%). This is hypothesised to represent a conservative estimate of the response rate, as students who completed PftPSA are likely to be overrepresented in the group who responded to the survey. Prescribing confidence data for students who completed the survey before and after the PSA pilot were not found to be significantly different on Mann– Whitney tests (1444 ≤ U ≤ 1716; –0.65 ≤ Z ≤ 0.97; 0.32 ≤ p(two-tail) ≤ 0.98), and these subgroups were subsequently combined. Prescribing confidence data were graphed (see Figure 3) and analysed using Wilcoxon signed-rank tests (see Table 2), showing that completing PftPSA is associated with highly significant increases in confidence across all prescribing skills.
Pre Post
Pre Post
Pre Post
Pre Post
Pre Post
100% 80% 60% 40% 20%
Several common themes were identified in the free-text feedback responses: the vast majority of students felt the e–tutorial helped them to meet the specified learning objectives, and that its design was clear and user-friendly. Many students who completed the feedback survey after the PSA pilot commented on the similarity of the PftPSA interface with that of the pilot. The most common suggestions for improvement of PftPSA were expansion of the question bank and refinement of the countdown timer.
LIMITATIONS Our evaluation used a subjective surrogate outcome measure (selfreported prescribing confidence), which has not been formally validated as corresponding to prescribing outcomes in the clinical environment. We felt this was appropriate for the following reasons. • Formally measuring these clinical outcomes is very difficult in practice, especially in a student cohort with no specified prescribing obligations to use as a basis for measurement. • Objective surrogate outcome measures (e.g. individual pilot PSA test scores) were not released for use by the researchers.
0%
Not at all confident
Slightly confident
Very confident
Extremely confident
• A recent study reported significant increases in short- and long-term prescribing confidence (measured
Moderately confident
Figure 3. A graphical representation of the prescribing confidence data
Table 2. Analysis of the prescribing confidence data using Wilcoxon signed-rank tests Prescribing
Prescription Planning CommuCalculation review management nicating skills information
Adverse drug Drug monireactions toring
Data interpretation
n
121
119
121
121
120
120
120
120
W
–817
–987
–946
–1114
–1971
–1280
–1439
–1224
Z
–4.76
–5.39
–5.71
–5.22
–6.44
–5.83
–5.87
–5.73
