HOW TO...

Construct Problem-Solving MCQs

Med Teach Downloaded from informahealthcare.com by Florida International University on 12/26/14 For personal use only.

B. JOORABCHI B . Joorabchi, MD, M.ED, is Chairman, Department of Medical Education, and Director, Regional Teacher Training Centre, Eastern Mediterranean Region, WHO, Shiraz University, Iran. He is presently Visiting Professor of Clinical Medicine at the University of Illinois, 190 Medical Sciences Building, Urbana, Illinois 61801, USA. This article gives ten guidelines for the construction of the one-from-five type problem-solving MCQs and provides some examples. Problem-solving multiple choice questions are perhaps the most difficult and challenging type of test items to

and worthy alternatives to the correct answer. Having found such rare alternatives, one must then nakedly arrange them in full view of the test-wise student, hoping that he does not spot all those thinly masked clues to the identity of the all too vulnerable correct answer. Here, I advocate the use of one-from-five type MCQs, because I believe that the usual true-false format, apart from encouraging guesswork, is unsuitable for measuring problem-solving abilities. Moreover, the multiple truefalse type is no improvement since it circumvents one of the few advantages offered by MCQs. An MCQmay be so written that the stems and choices taken together represent a central theme in which the distractors play subtle variatiqns; the task is to make a judgement among a number of finely tuned choices. This refinement is lost when the question calls for several independent ‘yes’and ‘no’decisions. Having avoided the pitfalls in the construction of MCQs, one then has to tackle the more difficult task of conforming to the problem-solving format. Several taxonomies for the problem-solving process have been offered. I prefer the following, which is a modification of traditional views (Barrows 1974) based on the results of field testing (Elstein et a1 1978): 1. Sensing the problem. 2. Initial hypothesis formulation. 3. Data gathering. 4. Analysis of data. 5 . Synthesis of data. 6 . Final hypothesis formulation. 7. Testing of hypothesis.

Medical Teacher Vol? No 1 1981

This is not to suggest an orderly progression-one usually flits back and forth depending on the complexity of the problem. Further, the hypothesis formulation and refinement is a continuous process with modifications, rejections and generation taking place during data gathering, analysis and synthesis.

A problem-solving MCQ is one which requires the student to go through at least some of the seven steps outlined above. Given the basic format of the MCQ, it is not possible to test for each of these steps. However, the following can be done: 1. Allow Data Gathering

Sensing the problem (Step 1) may be looked upon as passive data gathering, much as occurs on first looking at a patient or hearing the chief complaint. This takes place in most MCQs to a greater or lesser extent. The problem is in the active data gathering (Step 3): in the first place, data have to be provided to allow for data gathering. Recalling information stored in the memory is not data gathering. This means that each MCQneeds to start with a data base. In medicine, the most logical data base is, of course, a clinical case or any part of it; it may be a history, physical examination, laboratory data, or any combination of these or all of these. Pictures of lesions, expressions, specimens, x-rays, EKGs, patient charts, etc, may be included to broaden the data base and to impart a sense of realism. Alternatively, the data base may be an account of a research protocol, of actual results complete with graphs and charts, of raw data or discussions and conclusions - in short, any biomedical situation needing a resolution. 2. Provide a Sizeable Data Base

The data base-or the stem of the MCQ-has to be large enough to contain an adequate amount of information 9

with which to work. A couple of lines of compressed data will not do. Alternatively, the choices may contain all or part of the data base, for example, five descriptions of cases or experiments only one of which fits the question or data in the stem.

Med Teach Downloaded from informahealthcare.com by Florida International University on 12/26/14 For personal use only.

3 . Simulate Reality The writer has to create a sense of realism that is not only relevant to actual working conditions, but also challenges, attracts and motivates the student. The realism has to extend to the language, the setting, the timing, but most of all to the type of decisions expected of the student. As an example, in very few real-life situations is one called upon to diagnose a complex case at the very beginning and with only a few pieces of information given. 4. Include Appropriate ‘Noise’

One way of simulating reality is to include a little ‘noise’ in the data base. This may take the form of popular misconceptions, prejudices, fears and beliefs. More refined noise may be introduced into statements of cause and effect relationships, interpretation of data, and conclusions based on assumptions. Some of the choices should, of course, fit the noise to assure the complacent student.

5. Use Unprocessed Information As far as possible, the information given should be unprocessed and raw. The inclusion of an intermediary’s interpretation of data short-circuits the problem-solving chain. Statements like ‘cardiac examination was normal,’ ‘EKG showed RVH,’ or ‘blood gases indicated respiratory alkalosis’ are examples of such processing. Of course, incorrect interpretations and conclusions may be included as noise, but this is a special case. 6. Avoid Diagnostic Labels

Medical jargon commonly heard in presenting cases or in discussions with colleagues should never be used in examinations. Such statements as ‘butterfly rash,’ ‘migratory polyarthritis,’ ‘violaceous hue,’ ‘doughy abdomen,’ ‘weak, thready pulse,’ etc, represent diagnostic labels that are not available in real-life situations. Even worse, they provide very specific clues that reduce a potentially problem-solving exercise to recall of information or even to simple memory triggers.

7. Start with a Problem, not a Diagnosis If one starts with a specific diagnosis in mind it is very easy subconsciously to assume that all data provided point towards that diagnosis and that it is reasonable to expect a student to reach it. However, not being forewarned, the student may very easily and justifiably go astray. Conversely, the writer may give too many specific 10

clues to the diagnosis which may not realistically be available. Thus, do not begin with ‘Wilson’s disease’ but with liver dysfunction or neurological disorder.

8. Use Unfamiliar Cases If the same case or problem discussed in the classroom is used to examine students, the intended problem-solving exercise becomes a test of memory. This may be true even if only fragments of the problem are reproduced.

9. Test only Cognitive 0 bjectives Every examiner harbouring long-standing grievances based on slap-dash student habits, insists on including test questions to ventilate these grievances. Examples may be ordering lumbar punctures in head traumas, giving narcotics to patients with chronic lung disease, antibiotics for common colds, or mydriatics to glaucoma patients, and so on. However, this behaviour is governed more by attitudes than knowledge and one cannot measure attitudes with the usual paper and pencil tests, especially if the subject is aware of one’s intentions (Oppenheim 1968, Webb et al. 1966). The way a student answers such ‘motherhood’ questions has very little to do with his actual practice.

10. Select the Choices with Care There are many examples of well-written stems for problem-solving MCQs that are totally destroyed by obvious clues and defects in the choice of distractors. Some of these are so far-fetched as to be ridiculous. Care should be taken so that correct answers do not stand out by virtue of their length or excessive precision, by being the opposite pole of one of the distractors, or by containing a grammatical clue OT trigger word or phrase. If all this has the effect of dampening the enthusiasm for writing MCQs, at least for the easily written, simple recall types, it is intentional. All too often the seductive ease of scoring MCQs blinds one to the triviality of what is actually measured. The examples below (far from perfect models) are offered to kindle your imagination; I am sure you can improve upon them.

Problem-Solving MCQs: Some Examples

Data- Questions 1 and 2 You are a physiologist working in a cardiac laboratory and are very interested in the haemodynamics of various f o r m s of heart disease. T h e chart of the patient you are assisting with cardiac catheterization reads asfollows: “Thisfive-year-old boy was first noticed t o have cyanosis at three years of age. A t that time he only had some dyspnoea o n running, but later developed more sign@cant symptoms such as dyspnoea o n walking quickly, with frequent squatting and o n several occasions marked Medical Teacher V o l 3 No 1 1981

tachycardia, tachypnoea, and depressed sensorium. H e has never received any medications and had n o problems during infancy. O n examination h e has moderate and unzjCorm cyanosis with clubbing and seems in n o distress; his pulses are normal, there is n o lift, bulge, or thrill over theprecordium . . . ’’

0 0

A t this point, the cardiologist interrupts your reading and asks y o u to predict what y o u would f i n d in the pressure data in this case.

Data- Questions 5-10

Q1. You predict that:

Med Teach Downloaded from informahealthcare.com by Florida International University on 12/26/14 For personal use only.

0 A.

0 0 0

Right ventricular pressure would be higher than left. B. Left ventricular pressure would be higher than right. C. Pulmonary artery pressure would be normal. D. Pulmonary artery pressure would be about systemic. E. Right atrial pressure would be normal.

Q2. He further asks you to describe your findings on auscultation. You have not yet examined the patient, but you can say with some confidence:

0 A . First heart sound is loud. 0 B. Second heart sound is single. 17 C. Third heart sound is heard at the apex. 0 D. There is an apical diastolic murmur. 0 E. A pansystolic murmur is heard at the lower left sternal border.

Data- Questions 3 and 4 An outbreak of f o o d poisoning in a student cafeteria was investigated and the results are as shown in Table 1.

Food Chicken Salad Sandwich soup

Total 133 121 11 98

72.9 72.7 9.1 60.2

25 37 147 60

2 11 98 40

8 29.7 66.7 66.7

Q3. Which food was most probably contaminated?

0 A. Chicken 0 B. Salad 0 c. soup 0 D. Chicken and salad 0 E. Chicken, salad and soup 4 4 . Generally, which of the following statements is correct? A . The soup was probably partially contaminated. 12

(a)

120

100 50 -

ventricle

Right

0-

Figure 1. Five tracings obtained during cardiac catheterization. Q5. A 14-year-old boy with a three-month history of fever and weight loss followed by marked hepatomegaly oedema and neck vein congestion; there are no murmurs and the heart is not enlarged on x-ray. 0 ischaemic lung fields and no cardiomegaly.

Did not eat a particular food

Number PercentNumber Percentill age Total ill age 97 88 1 59

Consider the five tracings obtained during cardiac catheterization (Figure 1). For each of the brief clinical descriptions (Questions 5-10) select the one tracing that best f i t s the description and place its letter in the box.

Q6. A five-year-old boy with deep cyanosis and clubbing

Table 1. Results of food poisoning investigation. Ate a particular food

B. Cannot say whether the salad was contaminated. C. Something in the sandwich neutralized the contaminant. [7 D. There is not enough information to say any of the above.

0

Q7. A 52-year-old man is admitted with acute myocardial infarction and is stable. Five days later he develops severe pulmonary oedema and on examination has a grade 11-III/VI pansystolic murmur at the apex. 0 Q8. This patient has small pulses, increased left ventricular impulse, and a systolic thrill going into the neck. Q9. A four-month-old girl in heart failure with small femoral pulses which are felt distinctly later than the radial pulses. 0 QlO. A 23-year-old secretary with a ‘heart murmur’ since childhood noticed pains in her fingertips when she typed and complained of excessive fatigue. She was found to be anaemic and febrile and to have a long diastolic murmur at the third and fourth left intercostal spaces. 17 Data- Questions 11-13 Table 2 shows the results of cerebrospinalfZuid tests in f i v e cases. Match each of the following clinical histories with the appropriate set of tests and record its letter in the Medical Teacher V o l 3 No 1 1981

box. A choice may be used once, more than once, or not at all. Table 2. CSF test results. Protein Case

mg%

A B C D

600 250 80 2,000 300

E

Glucose mg%

RBC

WBC

mm3

mm3

300 10

500 150

10 300 30,000

60

40 10

4 40

50 60

10

70

70 10

70

% Poly 20

References Barrows, H. S., Problem-Based Learniq in Medicine, Education Monograph 4 , McMaster University, Hamilton, 1973. Elstein, A. S., Shulman, L. S. Sprafia, S. A., et al., Medical Problem Solving: An Analysis of Clinical Reasoning, Cambridge, Massachusetts, Hawarduniversity Press, 1978, p. 7 7 , 93, 168, 199, 273. Oppenheim, A. N., Questionnaire DeJign and Attitude Measurement, Basic Books Incorporated, New York, 1966, pp 73-78,151-154. Webb, E. J.. Campbell, D. T., Schwartz, R. D. and Sechnest, L., Unobtmue Measures: Nonreactive Research in S o h 1 Sciences, Rand McNally, Chicago, 1966, pp 12-34.

Further Reading

Med Teach Downloaded from informahealthcare.com by Florida International University on 12/26/14 For personal use only.

Q11. A young man with known hypertension developed

a sudden severe headache and became comatose six hours 0 before admission. Q12. A 10-year-old boy with a ‘common cold’ of four days’ duration manifested by cough, running nose, sore throat, and a temperature of 38°C. On physical examination he has a stiff neck, hyperaemia of conjunctiva, and moderate adenopathy. Has been on oral penicillin for the last four days. Ql3. A 15-year-old boy with ‘six months of cough and fever, and three weeks of headaches and diplopia. Has been taking no medications. 0

Harden, R. McG., Constmcting Multiple Choice Questions of the Multiple True/Falre Type, Medical Education Booklet No 10, Association for the Study of Medical Education, Dundee, 1979. Kennox, B . , Hints on the Setting and Evaluation of MCQs of the One from Five Type, Medical Education Booklet No 3, Association for the Study of Medical Education, Dundee, 1974.

Answers: l=E, 2=B, 3=A, 4=D, 5=d, 6=B, 7=E, 8=C, 9=C, 10=A, l l = E , 12=C, 13=A.

Mental Handicap Training Needs The British Government is inviting the General Nursing Council and the Central Council for Education and Training in Social Work to form a Working Group to look urgently .at introducing common elements into training courses for nurses and social workers dealing with theementally handicapped. However, the Government has not accepted the recommendation of the Committee of Enquiry into Mental Handicap Nursing and Care (the Jay Committee) that there should be immediate and fundamental changes to provide a common training for all staff-nurses and social workers-caring for the mentally handicapped. In response to this recommendation Mr Patrick Jenkin, Secretary of State for Social Services, stated that the Jay Committee’s philosophy and model of care for mental handicap services envisaged a radical change from the present pattern of services to one based on smaller and more local residential units in the community. While their approach was welcomed, it would have to be accepted that this shift would be gradual and take longer than the Committee had hoped, particularly in view of current necessary restraints on public expenditure. He also stated that the most severely handicapped and those with multiple handicaps would always need some form of NHS care and more experience is needed on whether this Medical Teacher V o l 3 No 1 1981

could be provided within the Committee’smodel of care. In these circumstawes, whilst the principles underlying the Committee’s ,recommendations were accepted, he believed it would not be right to urge immediate funda‘mental changes to the present training arrangements. This was not, in the Secretary of State’s view, the time to abandon a well tried form of training for nurses (who will continue to provide the majority of mental handicap care staff for some time to come) in favour of one which is comparatively new and vigorously opposed by nurses and major voluntary organizations. However, he concurred with the Committee that the training needs of NHS and social services staff have much in common and that progress should be made in the direction suggested by the Committee. Therefore, the General Nursing Councils and the Central Council for Education and Training in Social Work have been invited to set up a Working Group to look urgently at ways of introducing common elements within the separate forms of training and providing advice to authorities on the development of common in-service training courses and to come back with a plan for achieving this. This should also take account of the training needs of voluntary organizations. They have also been asked to consider the feasibility in the long term of a joint training leading to a joint qualification. 13

How To…:Construct Problem-Solving MCQs.

This article gives ten guidelines for the construction of the one-from-five type problem-solving MCQs and provides some examples...
384KB Sizes 2 Downloads 0 Views