Medical Teacher, Vol. 12, No. 314, 1990
259
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Twelve tips for organizing an Objective Structured Clinical Examination (OSCE)
R. M. HARDEN, Centre for Medical Education, The University, Ninewells Hospital and Medical School, Dundee, Scotland
The Objective Structured Clinical Examination or OSCE as it is more usually referred to, has been widely adopted as a tool to assess students’ or doctors’ competences in a range of subjects. The practical hints described in this article are aimed particularly at those who are undertaking for the first time the task of organizing the examination but they may be of interest also to a wider readership. When used correctly, the OSCE can be highly successful as an instrument to assess competence in medicine and the approach has many advantages over more traditional methods. Careful organization and planning is necessary, however, if the potential of the technique is to be realised.
What is to be Assessed Consider first what it is that you wish to assess. This should be related to the objectives of the course and may cover, for example, clinical methods, mastery of practical procedures, problem solving and clinical reasoning, and laboratory data interpretation. Is the emphasis on one, several, or all of these or indeed on other competencies not listed? Is the course concerned with the performance of the health professional in the hospital setting or with performance in a setting in the community? Is performance of the professional to be examined in isolation or as a member of a health care team? These are some of the issues that have to be addressed. It is important before planning proceeds with the OSCE to allow sufficient time to answer the question “What should be assessed?”
Tip 1 Produce a grid summarizing what is to be tested in your OSCE. Down the left hand side list the competencies such as history taking, physical examination, patient education etc. Along the top note the areas which should be represented
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
260
R. M. Harden in the assessment. In an examination for medical students this might be classified in terms of body systems or specialist areas such as cardiology, respiratory medicine, endocrinology etc. or it may be broader subject areas such as orthopaedics, geriatrics, paediatrics, clinical pharmacology etc. Note in each of the squares on the grid the numbers of the stations where competence is assessed in that area. For example, a ‘1’ in the square corresponding to history taking in endocrinology would reflect that at station one history taking ability is tested in a patient with hyperthyroidism. The number of entries in each of the horizontal rows will indicate the number of stations at which that competence, e.g. history taking, is assessed, while the number of entries in the vertical columns will indicate the number of stations at which the subject area, e.g. endocrinology, is assessed.
Duration of Station A decision has to be made about the standard length of time to be allocated for each station. This will depend to some extent on the competencies to be assessed in the examination. Times ranging from 4 to 15 minutes have been reported in different examinations and a five minute station probably most frequently chosen. The use of linked stations extends the time available to complete a task. At a linked station students may be asked, for example, to assimilate information about a patient at one station and are then expected to act on this at the following station or they may take a history or examine a patient at one station and this is followed by a station at which they are asked to interpret their findings or to take further action with regard to the findings. If necessary, you can include in the examination some stations which are allocated double the standard time. Such double stations will require to be duplicated in the examination.
Tip 2 Once the duration of stations has been fixed, make sure that the task expected of the student can be accomplished within the time. If in doubt reduce what is expected of the student. The exam should not be a race against time for the student unless this is one of the objectives being assessed. Where necessary use double or linked stations.
Number of Stations The number of stations in an examination together with the time allocated for each station determines the time required to complete the whole examination. Twenty stations each of five minutes can be completed in 1 hour 40 mins, while 20 stations each of 10 minutes require 3 hrs 20 mins to complete. While some teachers organise examinations of this length others, concerned that the longer examination is perhaps as much a test of the candidates’ endurance as it is of their competence in the areas assessed, would prefer to expose the students to two examinations each of 100 minutes. One of the most frequently used formats for an OSCE is 20 stations, each of five minutes duration. This allows each student to be assessed on a range of competencies and for two groups of students to be passed through one circuit of the examination in the course of a morning or afternoon.
Organizing an Objective Structured Clinical Examination
261
Tip 3 Within the time constraints include in the examination as many stations as possible as there is good evidence that the reliability of the examination is, in a large measure, dependent on the number of independent assessments of competence made during the examination.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Use of Examiners
In planning the examination it is important to consider the number of examiners available. Given that checklists for the examiners can be prepared and that the examiners can be rehearsed before the examination, it is possible to use junior as well as senior examiners in the OSCE. It is also possible-and many would say desirableto use examiners from a range of specialties and disciplines. For example health education officers take part at a station on patient education, and dietitians at a station on nutrition. Use examiners for what they are best suited, that is observing students undertaking a task. Observation of how students process information and come to a conclusion may also be a legitimate use of an examiner where one is interested in the students’ thought processes and the way in which they handle information. Tip 4 Make sure that examiners are fully briefed prior to the examination, both about the procedure for the OSCE in general, and in particular with regard to the station at which they are examining. They should have an opportunity to comment on the brief for the student and on any checklists or rating marking scales to be used. Provide them with a list of the resources that will be available at their stations. Range of Approaches The OSCE offers a flexible approach to the assessment of clinical competence. Examinations vary in the number of stations, the duration of stations and the format of the stations. In the assessment of history taking, for example, a number of different approaches have been adopted, using the OSCE format. Workers in the field have varied in their design of the station, the brief for the students and the use of checklists and rating scales. Some exams are arranged in the clinical setting of the wards, others in outpatient clinics or offices and others in halls or seminar room accommodation. The range of possibilities is limited only by the imagination of the examiner.
Tip 5 When planning an OSCE for the first time, you will find it helpful to talk with a number of individuals from different settings who have previously made use of the technique. If possible visit and watch their examinations. Failing this read some of the articles that have been written about the subject so that you can get a better idea of the range of options available. Having done this, you should then make up an examination to suit your own needs. New Stations
Pay particular attention when developing a station that the instructions are clear and unambiguous both for the student and the examiner, that the task can be undertaken in
262
R. M. Harden
the time available and that what is being tested is relevant to the objectives of the course.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 6 Ideally, you should test a new station with one or more students before it is used in an examination. If this is not possible show it to colleagues who were not involved with the development of the station and ask them to review it. If there is an external examiner let him or her see the station. Following this review process, make any changes necessary; small changes in wording can make a big difference in clarity. Organization of the Examination If the OSCE is well organized in advance there should be no problems on the day of the examination that cannot be dealt with satisfactorily. Advance organization includes appropriate briefing of examiners and students to gain their co-operation, the assembly of resources and patients required for the OSCE along with reserves and completion of appropriate arrangements with regard to the site where the examination is to be held.
Tip 7 There should be a co-ordinator appointed well in advance of the examination who has responsibility for taking overall charge of the advance planning of the examination and for its implementation on the day. Where the exam is being run simultaneously at multiple sites, in the same building or in different building, there should be an additional co-ordinator in charge at each site. The co-ordinators have a key role to play in ensuring the smooth running of the examination. Assigning Priority Those involved with an OSCE must give it a high priority. All the resources for the examination including examiners and patients must be assembled at the site of the examination at the correct time. Examiners who are clinicians must ensure that alternative arrangements are made to cover their clinical commitments. The smooth running of the OSCE depends on everyone playing their part; it cannot proceed until all examiners and patients are present and all the stations set up.
Tip 8 Those in a position of authority should make it clear to their staff that the OSCE has a high priority. If they are not to take charge of the examination themselves they should invest the necessary authority in the co-ordinators to whom they delegate responsibility for the exam. Resource Requirements The resources required for an OSCE will depend on the nature of the examination and on the design of the stations. All the resources necessary for the examination should be identified prior to the examination and the ncessary steps taken to procure them.
Organizing an Objective Structured Clinical Examination
263
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 9 It is helpful to produce a checklist of the resources required for each station in the examination. List what is required in terms of i) examiners who are observers at the stations and examiners required to mark any written answers; ii) patients, real and simulated; iii) equipment, for example a sphygnomanometer, and furniture, for example, chairs, tables or beds; iv) paperwork including checklists and instructions for the examiners, instructions or other sheets for students and a brief for simulated patients where required. Plan of the Examination and Directions Once the location of the examination has been decided and the stations fitted into the location, it is useful to prepare a plan of the layout on paper. On the plan note the position of each of the stations and indicate the path of the students from one station to the next. Give a copy of the plan to all examiners with their station marked, a copy to those setting up the examination and a copy to the students as part of the briefing on the morning of the examination, so that they get an overview of the circuit. Ensure that the position of each station is marked clearly at the site of the examination and provide direction arrows to guide the student as he exits from one station and proceeds to the next one. Students should not be stressed during an examination by any difficulty in finding their way round the circuit. A student who moves inadvertently to the wrong station in the examination may cause chaos with a major disruption not only for himself but for the other students.
Tip 10 After the examination is set up and the direction signs in place, ask someone who has not been involved with setting up the examination to walk round the circuit to ascertain that they can find their way easily from one station to the next. In some situations, it may help to use tape on the floor as a direction guide for the student. Change Signal
A range of methods are available to time the stations and provide the students and the examiners with an audible signal at five minute intervals or whatever time has been fixed for the station duration. This can range from a simple approach with a member of staff using a stop watch and a mechanical or electric bell, to devices available specifically for this purpose, or a computer appropriately programmed. Tip 11 Before the examination check that the audible signal can be heard clearly at all locations on the examination, if necessary with doors to rooms closed and screens drawn. Allow for background noise on the day of the examination, e.g. the patient may be talking. Records It is likely that the OSCE will become, if it is not already, part of your toolkit as an
264
R. M. Harden
examiner. It always takes longer to organize the examination on the first occasion than on subsequent occasions, providing information is kept about the OSCE and the materials used in the OSCE retained.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 12 Keep on OSCE file into which you put all the resources required for the OSCE such as station numbers, direction arrows, master sheets of instructions to students, checklists, rating scales, examples of correspondence notifying ward staff, patient, examiners etc., and a bank of questions used previously.
259
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Twelve tips for organizing an Objective Structured Clinical Examination (OSCE)
R. M. HARDEN, Centre for Medical Education, The University, Ninewells Hospital and Medical School, Dundee, Scotland
The Objective Structured Clinical Examination or OSCE as it is more usually referred to, has been widely adopted as a tool to assess students’ or doctors’ competences in a range of subjects. The practical hints described in this article are aimed particularly at those who are undertaking for the first time the task of organizing the examination but they may be of interest also to a wider readership. When used correctly, the OSCE can be highly successful as an instrument to assess competence in medicine and the approach has many advantages over more traditional methods. Careful organization and planning is necessary, however, if the potential of the technique is to be realised.
What is to be Assessed Consider first what it is that you wish to assess. This should be related to the objectives of the course and may cover, for example, clinical methods, mastery of practical procedures, problem solving and clinical reasoning, and laboratory data interpretation. Is the emphasis on one, several, or all of these or indeed on other competencies not listed? Is the course concerned with the performance of the health professional in the hospital setting or with performance in a setting in the community? Is performance of the professional to be examined in isolation or as a member of a health care team? These are some of the issues that have to be addressed. It is important before planning proceeds with the OSCE to allow sufficient time to answer the question “What should be assessed?”
Tip 1 Produce a grid summarizing what is to be tested in your OSCE. Down the left hand side list the competencies such as history taking, physical examination, patient education etc. Along the top note the areas which should be represented
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
260
R. M. Harden in the assessment. In an examination for medical students this might be classified in terms of body systems or specialist areas such as cardiology, respiratory medicine, endocrinology etc. or it may be broader subject areas such as orthopaedics, geriatrics, paediatrics, clinical pharmacology etc. Note in each of the squares on the grid the numbers of the stations where competence is assessed in that area. For example, a ‘1’ in the square corresponding to history taking in endocrinology would reflect that at station one history taking ability is tested in a patient with hyperthyroidism. The number of entries in each of the horizontal rows will indicate the number of stations at which that competence, e.g. history taking, is assessed, while the number of entries in the vertical columns will indicate the number of stations at which the subject area, e.g. endocrinology, is assessed.
Duration of Station A decision has to be made about the standard length of time to be allocated for each station. This will depend to some extent on the competencies to be assessed in the examination. Times ranging from 4 to 15 minutes have been reported in different examinations and a five minute station probably most frequently chosen. The use of linked stations extends the time available to complete a task. At a linked station students may be asked, for example, to assimilate information about a patient at one station and are then expected to act on this at the following station or they may take a history or examine a patient at one station and this is followed by a station at which they are asked to interpret their findings or to take further action with regard to the findings. If necessary, you can include in the examination some stations which are allocated double the standard time. Such double stations will require to be duplicated in the examination.
Tip 2 Once the duration of stations has been fixed, make sure that the task expected of the student can be accomplished within the time. If in doubt reduce what is expected of the student. The exam should not be a race against time for the student unless this is one of the objectives being assessed. Where necessary use double or linked stations.
Number of Stations The number of stations in an examination together with the time allocated for each station determines the time required to complete the whole examination. Twenty stations each of five minutes can be completed in 1 hour 40 mins, while 20 stations each of 10 minutes require 3 hrs 20 mins to complete. While some teachers organise examinations of this length others, concerned that the longer examination is perhaps as much a test of the candidates’ endurance as it is of their competence in the areas assessed, would prefer to expose the students to two examinations each of 100 minutes. One of the most frequently used formats for an OSCE is 20 stations, each of five minutes duration. This allows each student to be assessed on a range of competencies and for two groups of students to be passed through one circuit of the examination in the course of a morning or afternoon.
Organizing an Objective Structured Clinical Examination
261
Tip 3 Within the time constraints include in the examination as many stations as possible as there is good evidence that the reliability of the examination is, in a large measure, dependent on the number of independent assessments of competence made during the examination.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Use of Examiners
In planning the examination it is important to consider the number of examiners available. Given that checklists for the examiners can be prepared and that the examiners can be rehearsed before the examination, it is possible to use junior as well as senior examiners in the OSCE. It is also possible-and many would say desirableto use examiners from a range of specialties and disciplines. For example health education officers take part at a station on patient education, and dietitians at a station on nutrition. Use examiners for what they are best suited, that is observing students undertaking a task. Observation of how students process information and come to a conclusion may also be a legitimate use of an examiner where one is interested in the students’ thought processes and the way in which they handle information. Tip 4 Make sure that examiners are fully briefed prior to the examination, both about the procedure for the OSCE in general, and in particular with regard to the station at which they are examining. They should have an opportunity to comment on the brief for the student and on any checklists or rating marking scales to be used. Provide them with a list of the resources that will be available at their stations. Range of Approaches The OSCE offers a flexible approach to the assessment of clinical competence. Examinations vary in the number of stations, the duration of stations and the format of the stations. In the assessment of history taking, for example, a number of different approaches have been adopted, using the OSCE format. Workers in the field have varied in their design of the station, the brief for the students and the use of checklists and rating scales. Some exams are arranged in the clinical setting of the wards, others in outpatient clinics or offices and others in halls or seminar room accommodation. The range of possibilities is limited only by the imagination of the examiner.
Tip 5 When planning an OSCE for the first time, you will find it helpful to talk with a number of individuals from different settings who have previously made use of the technique. If possible visit and watch their examinations. Failing this read some of the articles that have been written about the subject so that you can get a better idea of the range of options available. Having done this, you should then make up an examination to suit your own needs. New Stations
Pay particular attention when developing a station that the instructions are clear and unambiguous both for the student and the examiner, that the task can be undertaken in
262
R. M. Harden
the time available and that what is being tested is relevant to the objectives of the course.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 6 Ideally, you should test a new station with one or more students before it is used in an examination. If this is not possible show it to colleagues who were not involved with the development of the station and ask them to review it. If there is an external examiner let him or her see the station. Following this review process, make any changes necessary; small changes in wording can make a big difference in clarity. Organization of the Examination If the OSCE is well organized in advance there should be no problems on the day of the examination that cannot be dealt with satisfactorily. Advance organization includes appropriate briefing of examiners and students to gain their co-operation, the assembly of resources and patients required for the OSCE along with reserves and completion of appropriate arrangements with regard to the site where the examination is to be held.
Tip 7 There should be a co-ordinator appointed well in advance of the examination who has responsibility for taking overall charge of the advance planning of the examination and for its implementation on the day. Where the exam is being run simultaneously at multiple sites, in the same building or in different building, there should be an additional co-ordinator in charge at each site. The co-ordinators have a key role to play in ensuring the smooth running of the examination. Assigning Priority Those involved with an OSCE must give it a high priority. All the resources for the examination including examiners and patients must be assembled at the site of the examination at the correct time. Examiners who are clinicians must ensure that alternative arrangements are made to cover their clinical commitments. The smooth running of the OSCE depends on everyone playing their part; it cannot proceed until all examiners and patients are present and all the stations set up.
Tip 8 Those in a position of authority should make it clear to their staff that the OSCE has a high priority. If they are not to take charge of the examination themselves they should invest the necessary authority in the co-ordinators to whom they delegate responsibility for the exam. Resource Requirements The resources required for an OSCE will depend on the nature of the examination and on the design of the stations. All the resources necessary for the examination should be identified prior to the examination and the ncessary steps taken to procure them.
Organizing an Objective Structured Clinical Examination
263
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 9 It is helpful to produce a checklist of the resources required for each station in the examination. List what is required in terms of i) examiners who are observers at the stations and examiners required to mark any written answers; ii) patients, real and simulated; iii) equipment, for example a sphygnomanometer, and furniture, for example, chairs, tables or beds; iv) paperwork including checklists and instructions for the examiners, instructions or other sheets for students and a brief for simulated patients where required. Plan of the Examination and Directions Once the location of the examination has been decided and the stations fitted into the location, it is useful to prepare a plan of the layout on paper. On the plan note the position of each of the stations and indicate the path of the students from one station to the next. Give a copy of the plan to all examiners with their station marked, a copy to those setting up the examination and a copy to the students as part of the briefing on the morning of the examination, so that they get an overview of the circuit. Ensure that the position of each station is marked clearly at the site of the examination and provide direction arrows to guide the student as he exits from one station and proceeds to the next one. Students should not be stressed during an examination by any difficulty in finding their way round the circuit. A student who moves inadvertently to the wrong station in the examination may cause chaos with a major disruption not only for himself but for the other students.
Tip 10 After the examination is set up and the direction signs in place, ask someone who has not been involved with setting up the examination to walk round the circuit to ascertain that they can find their way easily from one station to the next. In some situations, it may help to use tape on the floor as a direction guide for the student. Change Signal
A range of methods are available to time the stations and provide the students and the examiners with an audible signal at five minute intervals or whatever time has been fixed for the station duration. This can range from a simple approach with a member of staff using a stop watch and a mechanical or electric bell, to devices available specifically for this purpose, or a computer appropriately programmed. Tip 11 Before the examination check that the audible signal can be heard clearly at all locations on the examination, if necessary with doors to rooms closed and screens drawn. Allow for background noise on the day of the examination, e.g. the patient may be talking. Records It is likely that the OSCE will become, if it is not already, part of your toolkit as an
264
R. M. Harden
examiner. It always takes longer to organize the examination on the first occasion than on subsequent occasions, providing information is kept about the OSCE and the materials used in the OSCE retained.
Med Teach Downloaded from informahealthcare.com by UB der LMU Muenchen on 03/11/13 For personal use only.
Tip 12 Keep on OSCE file into which you put all the resources required for the OSCE such as station numbers, direction arrows, master sheets of instructions to students, checklists, rating scales, examples of correspondence notifying ward staff, patient, examiners etc., and a bank of questions used previously.
