HyperText Across the Curriculum: Implications of Quizzes Linked to Full Text Course Materials John Jackson, Ma.Ed. Brian Duling, Ph.D. University of Virginia School of Medicine HSC Box 382 Charlottesville, VA 22908 (804) 924-1528 [email protected] ABSTRACT This demonstration shows a hypertext self assessment and teaching system that is linkced to an extensive set of referenced course handouts. The system allows easy courseware development with multiple authors working in a networked environment with either DOS or Macintosh computers. Searchable hypertext course handouts benefit not only the student trying to relate material between courses, but also assist the faculty in the development of appropriate instructional materials as well as course goals and objectives.
INTRODUCTION Medical students are avid test takers. In the first two years of a medical curriculum, students are asked to absorb tremendous numbers of facts. As a result, the students continually seek new ways to assess their understanding and to prepare for the next exam. In response to this need, faculty commonly put questions or old exams on reserve in the library, but these are rarely accompanied by feedback as to why any given answer is right or wrong. This demonstration will show a hypertext instructional system that exploits the students' inclination to use practice tests, provides feedback for each student decision, and links each question to related full text documents. We will also examine the utility of Boolean search functions across courses and course sections for course directors, faculty, and students.
DESCRIPTION Software Structure The software system has three underlying structures: a navigation hierarchy, question banks with corrective feedback, and extensive course notes. The user moves through the hierarchy to fmd the practice questions that are of interest, selects any answer to receive feedback, and has the option reviewing related course materials by clicldng a reference button. Boolean search functions can be accessed and applied to both questions and course materials at any time. (See figure 1.)
DEVELOPMENT This project began as a search for a computer based tool to deliver practice questions with corrective feedback. From the beginning we realized that acceptance of the system required that it be easy to use for both faculty and students. Our selection criteria included that the system must import existingformatted word processor documents, import existing illustrations, allow for painless updating of documents, function on both DOS and Macintosh platforms (since our library's lab is evenly split between the two), 0195-4210/91/$5.00 © 1992 AMIA, Inc.
operate smoothly in a Novel network environment, and provide the ability to link to other teaching programs. We hoped to meet these goals and at the same time make the process so simple that departmental secretaries can do the bulk of the development work if given clear instructions by a faculty member. An initial prototype was developed using HyperCard, but we chose to use Guide, from Owl International, for the second generation of the software because of its support of imported formatted text and cross-platform compatibility. Using Guide's programming language we developed an authoring shell for faculty use that automated the creation of course sections, feedback for each response, and references to the course handouts. With these specially developed tools, faculty can develop richly interactive applications utilizing both graphics and text without any knowledge of programming. Once we developed a working prototype using Guide, we demonstrated the program to directors of the courses (or their designees) for the first two years of the medical curriculum. The course directors immediately perceived that not only would this system give the students a powerful educational tool, but if widely implemented, it would give course directors the ability to use Boolean search functions to determine content of all the courses. For the first time the course directors unanimously endorsed the use of an instructional tool across the curriculum.
947
Boolean Search NFiu nomo___oo
l l o | ~Authoring
HANDOUTS
|,.
1.
0 AUT
OUESTIONS 4
~~~P
Boolean Search Figur-e 1. Program Features: Students' Perspective We have made the interface as intuitive and as easy to use as possible for the student. (See figure 2.) An on-line help or a tutorial is accessible at any time. All selections are made using a mouse so that the only time a student has to resort to the keyboard is to type in search terms. When working on practice questions the student merely has to click on a choice to get pop-up feedback as to why or why not the selection was correct. The "Previous" and "Next" buttons at the bottom of the screen allow the student to flip rapidly between questions within a section. The "Reference" Button
opens a window to the course handouts at the appropriate section. The student also has the option of searching the linked course handouts, or any other handouts in the growing database, for related terms using the Boolean "and" and "or" search functions. The search function can also be used to review questions across several courses related to a particular term. Once the student returns to the original question, the "Menu" button takes the student to a table of contents for that group of questions.
Fl =HELP
File Edit Search Navigate
1-
Cardiovascular Physiology- Duling
2. The total resistance through a parallel set of 3 tubes, of resistance 5. 1 0, and 20 units, respectively, is:
*
U,
U'
A. 0.35 units
B. 2.9 unitC Sorry! Your arithmetic must be incorrect or you may have misunderstood that the resitance of a
C. 3.5 unitc set of tubes in parallel will always be lest than the resitance of any of the tubes in the set
D. -.5 units
E. 29 units. F. 54 units
U
.3
9 Figure 2.
948
The "Control Panel" on the right of the screen provides the same functions for all documents. "Contents" immediately goes to the highest level of the hierarchy of documents; "Search" allows students to fmd questions on a particular topic across all courses; "Print" prints any or all of a document, including illustrations; and "Quit" closes all the documents and exits Guide. The student can always find where he or she is by glancing at the title bar at the top of the screen which lists the course, section, and faculty member. Authoring Features: The Faculty's Perspective Since this project's original inception it has blossomed from a set of practice questions for physiology to potentially an integrated question bank and set of course handouts for all the courses in the first two years of the medical curriculum. As a result more than a hundred faculty could eventually be involved in the creation and modification of hypertext applications within the integrated system. In order to make it as easy to use as possible we developed automated tools to create corrective feedback for each response, make references to other documents, start new documents, and expand existing documents. The authoring shell does not restrict the author to these tools alone however; he or she may make full use of all of Guide's other development tools at any time if needed. The system also lets the author import his or her existing word processor and graphics files to create the core of the fial hypertext documents. We developed these features based on the imperative that unless the system is both easy to use and flexible, the faculty will simply not adopt it, no matter what the course directors endorse. Faculty can update their course handouts each year, using their word processors, then use the authoring shell to automatically relink their questions to the revised handouts. This is accomplished by matching a unique text string embedded within each practice question to text in the handouts. If the authoring shell cannot automatically rebuild a link, it logs the error so that the link can be manually reestablished. The system encourages faculty to develop lectures and exercises appropriate to the students' knowledge level. By electronically searching the handouts of lectures preceding their own, faculty can find out at what level the students are currently functioning. The faculty member can know exactly the concepts and terms that the students have been exposed to before they walk into the lecture hall. An unexpected benefit of the system is that it forces the faculty member to test questions for content
949
validity. If there is not a section of the handouts that explains a given question, then that question should be eliminated from the practice quizzes, and any similar questions deleted from course examinations. Design and Delivery Environments Guide from Owl International runs on and ports between both the DOS and Macintosh environments. Guide 2.0 for the Macintosh will operate on any Mac Plus, SE, or II model with one megabyte of RAM. Our authoring shell has features that will require a Mac II system with a color monitor. Guide 3.0 for DOS runs under Microsoft Windows, so its minimum requirements include a 286, EGA or VGA monitor, and a mouse. We recommend a 386 with at least two megabytes of RAM for authoring. Guide applications can be ported from either DOS or Macintosh environments so long as the version numbers match, but version 2.0 on the Macintosh does not have the programming options necessary for our applications. As a result we are currently developing on DOS machines, but will extend our application to the Macintosh as soon as Owl releases version 3.0 for that platform. EVALUATION During the formative evaluation phase we have tested the system with both faculty and student users. In one-on-one and small group sessions faculty members have used and criticized the authoring and enduser systems. The evaluator also noted which and how often features were used, as well as any difficulties encountered. A small group of interested students was also tested on the system on a one-on-one basis to gain their perspective on what features were important and useful. All of the findings have been very positive. Having refined the system based upon the formative evaluations, we developed several modules of questions and handouts for both physiology and microbiology. Our summative evaluation tools will include written surveys of the students at several points during the year, data collection of student usage, written evaluations by faculty users, and interviews with both students and faculty. We will also survey faculty and course directors to see if they make use of the search capabilities to improve the contents and sequencing of instruction. The true measure of success, however, will be in the degree to which students and faculty voluntarily choose to use the hypertext system over printed materials.
INTRODUCTION Medical students are avid test takers. In the first two years of a medical curriculum, students are asked to absorb tremendous numbers of facts. As a result, the students continually seek new ways to assess their understanding and to prepare for the next exam. In response to this need, faculty commonly put questions or old exams on reserve in the library, but these are rarely accompanied by feedback as to why any given answer is right or wrong. This demonstration will show a hypertext instructional system that exploits the students' inclination to use practice tests, provides feedback for each student decision, and links each question to related full text documents. We will also examine the utility of Boolean search functions across courses and course sections for course directors, faculty, and students.
DESCRIPTION Software Structure The software system has three underlying structures: a navigation hierarchy, question banks with corrective feedback, and extensive course notes. The user moves through the hierarchy to fmd the practice questions that are of interest, selects any answer to receive feedback, and has the option reviewing related course materials by clicldng a reference button. Boolean search functions can be accessed and applied to both questions and course materials at any time. (See figure 1.)
DEVELOPMENT This project began as a search for a computer based tool to deliver practice questions with corrective feedback. From the beginning we realized that acceptance of the system required that it be easy to use for both faculty and students. Our selection criteria included that the system must import existingformatted word processor documents, import existing illustrations, allow for painless updating of documents, function on both DOS and Macintosh platforms (since our library's lab is evenly split between the two), 0195-4210/91/$5.00 © 1992 AMIA, Inc.
operate smoothly in a Novel network environment, and provide the ability to link to other teaching programs. We hoped to meet these goals and at the same time make the process so simple that departmental secretaries can do the bulk of the development work if given clear instructions by a faculty member. An initial prototype was developed using HyperCard, but we chose to use Guide, from Owl International, for the second generation of the software because of its support of imported formatted text and cross-platform compatibility. Using Guide's programming language we developed an authoring shell for faculty use that automated the creation of course sections, feedback for each response, and references to the course handouts. With these specially developed tools, faculty can develop richly interactive applications utilizing both graphics and text without any knowledge of programming. Once we developed a working prototype using Guide, we demonstrated the program to directors of the courses (or their designees) for the first two years of the medical curriculum. The course directors immediately perceived that not only would this system give the students a powerful educational tool, but if widely implemented, it would give course directors the ability to use Boolean search functions to determine content of all the courses. For the first time the course directors unanimously endorsed the use of an instructional tool across the curriculum.
947
Boolean Search NFiu nomo___oo
l l o | ~Authoring
HANDOUTS
|,.
1.
0 AUT
OUESTIONS 4
~~~P
Boolean Search Figur-e 1. Program Features: Students' Perspective We have made the interface as intuitive and as easy to use as possible for the student. (See figure 2.) An on-line help or a tutorial is accessible at any time. All selections are made using a mouse so that the only time a student has to resort to the keyboard is to type in search terms. When working on practice questions the student merely has to click on a choice to get pop-up feedback as to why or why not the selection was correct. The "Previous" and "Next" buttons at the bottom of the screen allow the student to flip rapidly between questions within a section. The "Reference" Button
opens a window to the course handouts at the appropriate section. The student also has the option of searching the linked course handouts, or any other handouts in the growing database, for related terms using the Boolean "and" and "or" search functions. The search function can also be used to review questions across several courses related to a particular term. Once the student returns to the original question, the "Menu" button takes the student to a table of contents for that group of questions.
Fl =HELP
File Edit Search Navigate
1-
Cardiovascular Physiology- Duling
2. The total resistance through a parallel set of 3 tubes, of resistance 5. 1 0, and 20 units, respectively, is:
*
U,
U'
A. 0.35 units
B. 2.9 unitC Sorry! Your arithmetic must be incorrect or you may have misunderstood that the resitance of a
C. 3.5 unitc set of tubes in parallel will always be lest than the resitance of any of the tubes in the set
D. -.5 units
E. 29 units. F. 54 units
U
.3
9 Figure 2.
948
The "Control Panel" on the right of the screen provides the same functions for all documents. "Contents" immediately goes to the highest level of the hierarchy of documents; "Search" allows students to fmd questions on a particular topic across all courses; "Print" prints any or all of a document, including illustrations; and "Quit" closes all the documents and exits Guide. The student can always find where he or she is by glancing at the title bar at the top of the screen which lists the course, section, and faculty member. Authoring Features: The Faculty's Perspective Since this project's original inception it has blossomed from a set of practice questions for physiology to potentially an integrated question bank and set of course handouts for all the courses in the first two years of the medical curriculum. As a result more than a hundred faculty could eventually be involved in the creation and modification of hypertext applications within the integrated system. In order to make it as easy to use as possible we developed automated tools to create corrective feedback for each response, make references to other documents, start new documents, and expand existing documents. The authoring shell does not restrict the author to these tools alone however; he or she may make full use of all of Guide's other development tools at any time if needed. The system also lets the author import his or her existing word processor and graphics files to create the core of the fial hypertext documents. We developed these features based on the imperative that unless the system is both easy to use and flexible, the faculty will simply not adopt it, no matter what the course directors endorse. Faculty can update their course handouts each year, using their word processors, then use the authoring shell to automatically relink their questions to the revised handouts. This is accomplished by matching a unique text string embedded within each practice question to text in the handouts. If the authoring shell cannot automatically rebuild a link, it logs the error so that the link can be manually reestablished. The system encourages faculty to develop lectures and exercises appropriate to the students' knowledge level. By electronically searching the handouts of lectures preceding their own, faculty can find out at what level the students are currently functioning. The faculty member can know exactly the concepts and terms that the students have been exposed to before they walk into the lecture hall. An unexpected benefit of the system is that it forces the faculty member to test questions for content
949
validity. If there is not a section of the handouts that explains a given question, then that question should be eliminated from the practice quizzes, and any similar questions deleted from course examinations. Design and Delivery Environments Guide from Owl International runs on and ports between both the DOS and Macintosh environments. Guide 2.0 for the Macintosh will operate on any Mac Plus, SE, or II model with one megabyte of RAM. Our authoring shell has features that will require a Mac II system with a color monitor. Guide 3.0 for DOS runs under Microsoft Windows, so its minimum requirements include a 286, EGA or VGA monitor, and a mouse. We recommend a 386 with at least two megabytes of RAM for authoring. Guide applications can be ported from either DOS or Macintosh environments so long as the version numbers match, but version 2.0 on the Macintosh does not have the programming options necessary for our applications. As a result we are currently developing on DOS machines, but will extend our application to the Macintosh as soon as Owl releases version 3.0 for that platform. EVALUATION During the formative evaluation phase we have tested the system with both faculty and student users. In one-on-one and small group sessions faculty members have used and criticized the authoring and enduser systems. The evaluator also noted which and how often features were used, as well as any difficulties encountered. A small group of interested students was also tested on the system on a one-on-one basis to gain their perspective on what features were important and useful. All of the findings have been very positive. Having refined the system based upon the formative evaluations, we developed several modules of questions and handouts for both physiology and microbiology. Our summative evaluation tools will include written surveys of the students at several points during the year, data collection of student usage, written evaluations by faculty users, and interviews with both students and faculty. We will also survey faculty and course directors to see if they make use of the search capabilities to improve the contents and sequencing of instruction. The true measure of success, however, will be in the degree to which students and faculty voluntarily choose to use the hypertext system over printed materials.
