Psychopharmacology Education Software The First of a Series Stuart Gitlow, M.D.; T. Bradley Tanner, M.D. University of Pittsburgh Medical Center Abstract A psychopharmacology testing package was generated by Multiple Choice7'M a Claris HyperCard 2.1 based software application. Multiple Choice was developed specifically to allow for the simple preparation of nonnetworked computerized testing. It permits tests of unlimited length to incorporate truelfalse, multiple choice (with 3-5 possible answers), and K-type questions. These examinations may incorporate explanations if desired. The tests maintain records to more easily permit the exanmnation's creator to quickly determine scores and question-specific results. The psychopharmacology testing package is a multi-level group of three examinations. It is to be the first in a series ofpsychiatry exams targeting medical students, psychiatry residents, and other interested professionals. Work is also in progress to convert both the psychopharmacology testing package and Multiple Choice itself to the PC-compatible platform.
Introduction
Rotations in medical schools and other allied health programs typically do not permit sufficient time for in-depth study of specialty fields of medicine. Practitioners and medical students who enter psychiatry rotations from another medical specialty or clerkship often do not possess the knowledge of subspecialty areas such as psychopharmacology necessary to be initially competent. This is most debilitating for those in their first year of a psychiatry residency. These young physicians may launch their first day of a psychiatry position from a standpoint where their last exposure to the field was during a clerkship in the third year of medical school. That often-short rotation would likely have been two to three years earlier. Now, with their medicine and neurology internship behind them, it is essential that they quickly relearn the fundamentals of psychiatric care. Our self-evaluation instrument in psychopharmacology provides an organized review of material presented within our educational courses to residents and provides a general review for clinical employees and students. It represents the first in a series of psychiatry subspecialty self-tests, each of which will be formatted similarly. The instrument is provided for use within the hospital library on Macintosh SE/30 and IIsi 0195-4210/91/$5.00 C 1992 AMIA, Inc.
945
computers. Those arriving at the workstation may choose from among three possible examinations, depending upon their likely skill level. The individual is then presented with a series of true/false, multiple choice, and K-type questions. After choosing a response to each question, the software indicates the correct answer and presents an explanation. The package then records whether the student selected correctly on the given question. This permits later determination of question-specific results. Finally, at the completion of the examination, the package presents and records the overall score. This constant availability and immediate feedback represent the major advantages of a computerized package over standard paper-based testing models.
Development
The questions within the psychopharmacology software package were drawn from paper-based examinations which had been used in previous years within the teaching environment at Western Psychiatric Institute and Clinic. Explanations were written and reviewed by the original committee that had developed the paper-based examinations. Developed under a grant from the National Health Libraries Consortium, the software was generated by Multiple ChoiceTm, a HyperCard 2.1 software application developed by Symposia for this express purpose. Although Multiple Choice permits complete examination generation, additional object-oriented programming permitted the addition of help screens, introductory remarks, and a variety of supplemental features.
Multiple Choice Multiple Choice enables the user to simply and easily generate a self-standing computerized examination software package containing any combination of true/false, multiple choice, and K-type questions. It permits the test creator to determine whether the test package should present explanations and offer immediate scoring of individual questions. No knowledge of scripting or programming on the part of the instructor is required at any time. Once a test package has been created and used by students, Multiple Choice can analyze the data stored within the examination itself. It can determine and list individual scores as well as question-specific scores. The latter results in an indication as to the percent choosing correctly
for each question within the examination. In this manner, instructors can quickly determine either inappropriate or incorrect questions, or areas of difficulty for their class as
Printing an Examination: The entire contents of the examination can be either saved to disk as a text file or
printed.
a whole.
Upon first running Multiple Choice, the user is presented with several choices: Examination Creation, Modification of Examination, Examination Results, and Printing an Examination. Examination Creation: At the start of the examination creation phase, examiners may choose one of three formats for their exam: No Feedback, Informed, or Explained. In the "No Feedback" mode, students choose answers within an exam and are not told whether their responses are correct. In the "Informed" mode, students are told if their responses are correct but they are not offered an explanation. In the "Explained" mode, students are informed as to whether their response is correct and also offered an explanation. Explanations are offered even if the student chooses the correct response. Instructors are encouraged to use the explanation field in this manner to further teach the student. Following the user's choice of test format, the question/response phase is entered. Within this section, the examiner can choose True/False, Multiple Choice (with 3, 4, or 5 responses possible), or K-type questions. Each exam may be designed with any combination of these question types, in any order desired. Questions and responses may be of any length. All fields are scrollable so as not to cause the screen resolution to be a limiting factor. After entering the question and each response, the examiner can enter an explanation if the exam is of the "Explained" type. finally, the correct response is noted and the examiner repeats the process until the exam is complete.
Modification of an Examination: Modifications to an examination previously created are Adding a question; Deleting a question; Rewording a question, response, or explanation; Altering which answer was indicated as being correct; Clearing statistics and examination scores from an exam in preparation for testing a new population. Examination Results: This section presents Examinee Name, Overall score for each examinee, and Percentage of examinees responding correctly for each question. This information can be printed. Although scores are stored within each exam, they are not available to those taking the tests without the Multiple Choice software. Safeguards have been incorporated within Multiple Choice and within generated examinations to prevent users from manipulating the stand-alone examinations in an effort to determine others' scores. Examinees must take the examination or quit from the program.
946
Additional Features: Several features remain to be added to the software. At the present time, there is no built-in capability of depicting illustrations or playing back sound or video with a question. These capabilities are in the process of being added. Additional question formats, such as Matching, are occasionally used within standardized examinations and should be added to the Multiple Choice software. Further, although Multiple Choice includes the capability of presenting an explanation to the examinee, this explanation is not dependent upon the choice of response. Additional explanations could be prompted from the examiner at the time of exam creation, thus allowing for this enhancement. finally, additional statistics might be collected, particularly one wherein the examiner could determine which incorrect response is most often chosen by examinees. The examinations are not completely fail-safe in terms of protection. A student knowledgeable in HyperTalk could override the built-in protection scheme by writing a small utility which would "dump" the contents of each field within the examination itself, thereby determining correct responses and prior examination results.
Presentation Our presentation will cover the logistics of self-examination, including choice of location and methods of achieving privacy. The psychopharmacology educational package, Multiple Choice itself, and pertinent HyperCard 2.1 programming techniques will each be demonstrated.
Introduction
Rotations in medical schools and other allied health programs typically do not permit sufficient time for in-depth study of specialty fields of medicine. Practitioners and medical students who enter psychiatry rotations from another medical specialty or clerkship often do not possess the knowledge of subspecialty areas such as psychopharmacology necessary to be initially competent. This is most debilitating for those in their first year of a psychiatry residency. These young physicians may launch their first day of a psychiatry position from a standpoint where their last exposure to the field was during a clerkship in the third year of medical school. That often-short rotation would likely have been two to three years earlier. Now, with their medicine and neurology internship behind them, it is essential that they quickly relearn the fundamentals of psychiatric care. Our self-evaluation instrument in psychopharmacology provides an organized review of material presented within our educational courses to residents and provides a general review for clinical employees and students. It represents the first in a series of psychiatry subspecialty self-tests, each of which will be formatted similarly. The instrument is provided for use within the hospital library on Macintosh SE/30 and IIsi 0195-4210/91/$5.00 C 1992 AMIA, Inc.
945
computers. Those arriving at the workstation may choose from among three possible examinations, depending upon their likely skill level. The individual is then presented with a series of true/false, multiple choice, and K-type questions. After choosing a response to each question, the software indicates the correct answer and presents an explanation. The package then records whether the student selected correctly on the given question. This permits later determination of question-specific results. Finally, at the completion of the examination, the package presents and records the overall score. This constant availability and immediate feedback represent the major advantages of a computerized package over standard paper-based testing models.
Development
The questions within the psychopharmacology software package were drawn from paper-based examinations which had been used in previous years within the teaching environment at Western Psychiatric Institute and Clinic. Explanations were written and reviewed by the original committee that had developed the paper-based examinations. Developed under a grant from the National Health Libraries Consortium, the software was generated by Multiple ChoiceTm, a HyperCard 2.1 software application developed by Symposia for this express purpose. Although Multiple Choice permits complete examination generation, additional object-oriented programming permitted the addition of help screens, introductory remarks, and a variety of supplemental features.
Multiple Choice Multiple Choice enables the user to simply and easily generate a self-standing computerized examination software package containing any combination of true/false, multiple choice, and K-type questions. It permits the test creator to determine whether the test package should present explanations and offer immediate scoring of individual questions. No knowledge of scripting or programming on the part of the instructor is required at any time. Once a test package has been created and used by students, Multiple Choice can analyze the data stored within the examination itself. It can determine and list individual scores as well as question-specific scores. The latter results in an indication as to the percent choosing correctly
for each question within the examination. In this manner, instructors can quickly determine either inappropriate or incorrect questions, or areas of difficulty for their class as
Printing an Examination: The entire contents of the examination can be either saved to disk as a text file or
printed.
a whole.
Upon first running Multiple Choice, the user is presented with several choices: Examination Creation, Modification of Examination, Examination Results, and Printing an Examination. Examination Creation: At the start of the examination creation phase, examiners may choose one of three formats for their exam: No Feedback, Informed, or Explained. In the "No Feedback" mode, students choose answers within an exam and are not told whether their responses are correct. In the "Informed" mode, students are told if their responses are correct but they are not offered an explanation. In the "Explained" mode, students are informed as to whether their response is correct and also offered an explanation. Explanations are offered even if the student chooses the correct response. Instructors are encouraged to use the explanation field in this manner to further teach the student. Following the user's choice of test format, the question/response phase is entered. Within this section, the examiner can choose True/False, Multiple Choice (with 3, 4, or 5 responses possible), or K-type questions. Each exam may be designed with any combination of these question types, in any order desired. Questions and responses may be of any length. All fields are scrollable so as not to cause the screen resolution to be a limiting factor. After entering the question and each response, the examiner can enter an explanation if the exam is of the "Explained" type. finally, the correct response is noted and the examiner repeats the process until the exam is complete.
Modification of an Examination: Modifications to an examination previously created are Adding a question; Deleting a question; Rewording a question, response, or explanation; Altering which answer was indicated as being correct; Clearing statistics and examination scores from an exam in preparation for testing a new population. Examination Results: This section presents Examinee Name, Overall score for each examinee, and Percentage of examinees responding correctly for each question. This information can be printed. Although scores are stored within each exam, they are not available to those taking the tests without the Multiple Choice software. Safeguards have been incorporated within Multiple Choice and within generated examinations to prevent users from manipulating the stand-alone examinations in an effort to determine others' scores. Examinees must take the examination or quit from the program.
946
Additional Features: Several features remain to be added to the software. At the present time, there is no built-in capability of depicting illustrations or playing back sound or video with a question. These capabilities are in the process of being added. Additional question formats, such as Matching, are occasionally used within standardized examinations and should be added to the Multiple Choice software. Further, although Multiple Choice includes the capability of presenting an explanation to the examinee, this explanation is not dependent upon the choice of response. Additional explanations could be prompted from the examiner at the time of exam creation, thus allowing for this enhancement. finally, additional statistics might be collected, particularly one wherein the examiner could determine which incorrect response is most often chosen by examinees. The examinations are not completely fail-safe in terms of protection. A student knowledgeable in HyperTalk could override the built-in protection scheme by writing a small utility which would "dump" the contents of each field within the examination itself, thereby determining correct responses and prior examination results.
Presentation Our presentation will cover the logistics of self-examination, including choice of location and methods of achieving privacy. The psychopharmacology educational package, Multiple Choice itself, and pertinent HyperCard 2.1 programming techniques will each be demonstrated.
