BRIEF COMMUNICATIONS
QUESTIONS TO BE INVESTIGATED The library had certain questions it wished to have answered, while the chief residents had other By CECILIA STAUDT,* goals they wished to further. To answer the Assistant Librarian for Public Services overall question of the real worth of the program, BARBARA HALBROOK, the library set out to determine: Head, Reference Department 1. How much was the service used, and how ESTELLE BRODMAN, PH.D., much did it cost? Librarian and Professor of Medical History 2. For what purposes was the service used? 3. If the service was discontinued, would there Washington University School of Medicine be any carry-over in library use? St. Louis, Missouri 4. If the service were available in the library only, but not physically through residents' HISTORICAL BACKGROUND reports, would it be used as much as when A FEW reports of a new method of providing the librarian was there in person? medical information needed for patient care, 5. Would the users of the service be willing to namely by use of "clinical librarians," had appay something for it? peared in the literature before the summer of Finally, to satisfy both the chief residents and the 1974; but the experimental program at the Wash- library, the question of the quality of the service ington University School of Medicine Library did for education and patient care use had to be invesnot start before then and in some respects did not tigated. follow the generally accepted methods of other libraries. The program began when the chief resident of one service in the medical school hosMETHODOLOGY pital indicated he wished to enhance the quality of methods were employed to Four different the education of his interns and residents by making them aware of the potentialities of medical answer these questions: (1) A certain number of literature for patient care. The program differed in-depth interviews were held with residents as in methodology from most other similar programs they rotated off the services in which the clinical because the clinical librarians did not go on rounds librarian program was offered. (2) A questionwith the physician and his team, but instead sat in naire, to be returned by the user, was sent with on residents' reports, from which the librarians each literature search. (3) For a number of gleaned the problems for which a search of the months the service was offered at residents' reports on alternate months only; during the inliterature might be appropriate. Begun on July 18, 1974, for one group of tervening months the service was available by callresidents, the Washington University program ing or visiting the library. (4) Statistics on the use expanded on October 3, 1974, to include a second of the service by individual residents were maingroup of residents. When a third request for tained. similar service was received, the library felt it had to evaluate the entire program to see if the costbenefit ratios were favorable to enlarging the STATISTICAL FINDINGS clinical librarian program, maintaining it at its From January through June 1975 a total of current level, or dropping it. By January 1, 1975, thirty-nine residents on the two services used the outlines of the evaluation had been de- clinical librarians. These thirty-nine residents termined, and for the six-month period, generated 334 subject searches during the sixJanuary-June 1975, data gathering took place. month period, ranging from 33 searches one We wish to report on the methodology used and month to 80 searches another month (Table 1). the results obtained. As other libraries report Sixteen of the thirty-nine residents were questheir evaluations, thus providing data from a tioned in hour-long, in-depth interviews. The total larger number of cases than any one library can cost to the library for the service from January provide, a generalized evaluation of the entire through March 1975 averaged $661 per month (or clinical librarian concept may be possible. $17/month/resident) and included staff time, *Cecilia Staudt is now Reference Librarian at Linden- photocopy costs, and the cost of manual and comwood College, St. Charles, Missouri. puterized (mostly MEDLINE) searches (Table 2).
A Clinical Librarians' ProgramAn Attempt At Evaluation
236
Bull. Med. Libr. Assoc. 64(2) April 1976
BRIEF COMMUNICATIONS
TABLE 1 SIZE OF SAMPLE No. Residents in Study Number interviewed Average Searches/Month When librarian attended residents' reports When librarian did not attend residents' reports Difference Total Searches MEDLINE only Manual only MEDLINE and manual combined Search Questionnaires Sent
Questionnaires returned -MEDLINEonly -Manual only -Combined
39 16
Percent
100 41
45 47 334 279 37
4.44 100 84 11
18 334 193 160 19 13
5 100 58 83 10 7
FINDINGS 1. How much was the service used and how much did it cost? From January through June 1975 the service was used by all but one eligible resident. A total of 334 searches was undertaken for them; during the same period only 67 searches were made for residents of other services not participating in the experiment. Searches by the clinical librarian represented from 35% to 57% of all MEDLINE searches processed by the library. Total cost of maintaining the special service came to $17 per month per resident. 2. For what purpose was the service used? This was a four-part question, with the respondents able to check off more than one answer. Of the 193 returned questionnaires, 177 (92%) included answers to this question,
providing 324 responses. The results are shown in Table 3. The question was followed up in the interviews. Although fifteen of the searches were used for unspecified purposes, miscellaneous comments from the questionnaires and interviews indicated that searches were used to help prepare presentations at grand rounds, as teaching tools with students and interns, and for preparation of papers. Moreover, 76% of the searches were shared with an average of seven people in both individual and group situations. 3. Would there be any carry-over after the service was stopped? Although eleven of the sixteen residents (69%) interviewed said they would use the service after their residency, the proof of the pudding will only be found in the future. 4. Would residents use the service even if the librarian were not physically present at residents' reports, by visiting or calling the library? In actuality we found that slightly more use per month was made of the service when the librarian was not present at meetings than when she was, though the difference was too small to be significant. What was significant, however, was that this increase occurred only among those residents who had already been exposed to the service personally during their rotations. New residents who heard of the service during the months it was offered only through the library did not make use of it. 5. Would the users be willing to pay something for the service? Almost unanimously the residents interviewed said they would not use the service if they had to pay themselves. They cited reasons such as lack of money and their belief that the school should furnish such service as part of the educational
TABLE 2 COSTS OF SERVICE, JAN.-MAR. 1975 1975
January February March Total Aver./Mo.
Personnel ..........
242.58 ..........
242.58 *
MEDLINE
411.70 577.95 379.04
1,368.69 456.23
Manual search
Photocopy
Total cost
29.44 19.86 13.08 62.38 20.79
42.80 94.00 52.00 188.80 62.93
$ 483.94 934.39 444.12
$1,862.45 *
*Not meaningful because two out of three months were nonattendance months.
Bull. Med. Libr. Assoc. 64(2) April 1976
237
BRIEF COMMUNICATIONS TABLE 3 USE TO WHICH INFORMATION WAS PUT
Number of responses received Response on use In diagnosis or treatment of a particular patient To verify a belief or
proceeding To add to general knowledge
Other
No.
Percent
324
100
112
35
60 137 15
19 42 5
program for interns and residents. This finding is as puzzling as the many similar findings reported from other groups; people seem to want a service but are reluctant or unwilling to pay for it themselves or through increases in taxes, tuition, etc. In our case the Department of Medicine agreed with the residents on the school's responsibility to furnish the service; the department will pay half the costs for 1975/76. This does not solve the dilemma of the need to bridge the gap between what is technologically and manually feasible and what society is willing to purchase, even for such desirable ends as better medical care or education.
The library has thus answered the specific question it raised without finding any solution to the broader issue. Finally, how good was the service, as judged by the recipients? Of those who returned the questionnaires, 155 (85%) said the searches were adequate, while 28 (15%) said they were not. Those who said the searches were not adequate indicated the following reasons: (1) did not provide the needed information; (2) there was no literature on the subject; (3) poor quality of the article on the subject; and (4) important older articles were not caught by the search methods used. The average percent of "on-target" relevant articles retrieved was 25% of the total, with a range from 18% to 40%. All recipients reported that the searches aided them by making the first "cut" and thus giving them more time to read the literature rather than to search for it. No one said the searches were generally poor or useless. CONCLUSION The worth of clinical librarians' programs has not yet been proved quantitatively or unequiv-
238
ocally, but neither has the usefulness of librarianship in general. The study done at the Washington University School of Medicine Library adds some data to the growing amount of information on this topic. We hope that further studies will elucidate whether clinical librarianship is a fad or a useful product. Note: Copies of the questionnaire used in this study can be obtained on request from the Librarian at the Washington University School of Medicine Library, 4580 Scott Avenue, St. Louis, Missouri 63110.
Computer-Assisted Instruction in Library Orientation and Services By JOAN TOMAY HICKS, Librarian College of Veterinary Medicine University of Illinois at Urbana-Champaign
AT the University of Illinois a computer-assisted instruction (CAI) system has been developed [1-2] which uses a large Control Data Corporation computer connected to remote terminals with plasma display panels and typewriter-like keyboards. This system, PLATO (Programmed Logic for Automatic Teaching Operation), is very flexible. Through PLATO many types of programs have been created at all levels of instruction from kindergarten through university graduate study [3]. For example, some programs involve medical students' diagnoses of simulated disease syndromes [4] or identification of bacteriological unknowns [5]. At the College of Veterinary Medicine in Urbana there are about thirty PLATO computer consoles. Most of them are located in the Basic Sciences Building in a room across from the library; the rest are situated in the Small Animal Clinic some distance away. There are presently about seventy CAI programs of study for veterinary medical students. One of these programs is on the library and its resources. In the library program the title page which appears on the terminal display panel briefly describes the main parts of the program and indicates which keys to press to go on to find the different kinds of information. One choice on the title page for an overall introduction brings to the screen a schematic plan of the physical arrangement of the library, one part at a time, with short Bull. Med. Libr. Assoc. 64(2) April 1976
QUESTIONS TO BE INVESTIGATED The library had certain questions it wished to have answered, while the chief residents had other By CECILIA STAUDT,* goals they wished to further. To answer the Assistant Librarian for Public Services overall question of the real worth of the program, BARBARA HALBROOK, the library set out to determine: Head, Reference Department 1. How much was the service used, and how ESTELLE BRODMAN, PH.D., much did it cost? Librarian and Professor of Medical History 2. For what purposes was the service used? 3. If the service was discontinued, would there Washington University School of Medicine be any carry-over in library use? St. Louis, Missouri 4. If the service were available in the library only, but not physically through residents' HISTORICAL BACKGROUND reports, would it be used as much as when A FEW reports of a new method of providing the librarian was there in person? medical information needed for patient care, 5. Would the users of the service be willing to namely by use of "clinical librarians," had appay something for it? peared in the literature before the summer of Finally, to satisfy both the chief residents and the 1974; but the experimental program at the Wash- library, the question of the quality of the service ington University School of Medicine Library did for education and patient care use had to be invesnot start before then and in some respects did not tigated. follow the generally accepted methods of other libraries. The program began when the chief resident of one service in the medical school hosMETHODOLOGY pital indicated he wished to enhance the quality of methods were employed to Four different the education of his interns and residents by making them aware of the potentialities of medical answer these questions: (1) A certain number of literature for patient care. The program differed in-depth interviews were held with residents as in methodology from most other similar programs they rotated off the services in which the clinical because the clinical librarians did not go on rounds librarian program was offered. (2) A questionwith the physician and his team, but instead sat in naire, to be returned by the user, was sent with on residents' reports, from which the librarians each literature search. (3) For a number of gleaned the problems for which a search of the months the service was offered at residents' reports on alternate months only; during the inliterature might be appropriate. Begun on July 18, 1974, for one group of tervening months the service was available by callresidents, the Washington University program ing or visiting the library. (4) Statistics on the use expanded on October 3, 1974, to include a second of the service by individual residents were maingroup of residents. When a third request for tained. similar service was received, the library felt it had to evaluate the entire program to see if the costbenefit ratios were favorable to enlarging the STATISTICAL FINDINGS clinical librarian program, maintaining it at its From January through June 1975 a total of current level, or dropping it. By January 1, 1975, thirty-nine residents on the two services used the outlines of the evaluation had been de- clinical librarians. These thirty-nine residents termined, and for the six-month period, generated 334 subject searches during the sixJanuary-June 1975, data gathering took place. month period, ranging from 33 searches one We wish to report on the methodology used and month to 80 searches another month (Table 1). the results obtained. As other libraries report Sixteen of the thirty-nine residents were questheir evaluations, thus providing data from a tioned in hour-long, in-depth interviews. The total larger number of cases than any one library can cost to the library for the service from January provide, a generalized evaluation of the entire through March 1975 averaged $661 per month (or clinical librarian concept may be possible. $17/month/resident) and included staff time, *Cecilia Staudt is now Reference Librarian at Linden- photocopy costs, and the cost of manual and comwood College, St. Charles, Missouri. puterized (mostly MEDLINE) searches (Table 2).
A Clinical Librarians' ProgramAn Attempt At Evaluation
236
Bull. Med. Libr. Assoc. 64(2) April 1976
BRIEF COMMUNICATIONS
TABLE 1 SIZE OF SAMPLE No. Residents in Study Number interviewed Average Searches/Month When librarian attended residents' reports When librarian did not attend residents' reports Difference Total Searches MEDLINE only Manual only MEDLINE and manual combined Search Questionnaires Sent
Questionnaires returned -MEDLINEonly -Manual only -Combined
39 16
Percent
100 41
45 47 334 279 37
4.44 100 84 11
18 334 193 160 19 13
5 100 58 83 10 7
FINDINGS 1. How much was the service used and how much did it cost? From January through June 1975 the service was used by all but one eligible resident. A total of 334 searches was undertaken for them; during the same period only 67 searches were made for residents of other services not participating in the experiment. Searches by the clinical librarian represented from 35% to 57% of all MEDLINE searches processed by the library. Total cost of maintaining the special service came to $17 per month per resident. 2. For what purpose was the service used? This was a four-part question, with the respondents able to check off more than one answer. Of the 193 returned questionnaires, 177 (92%) included answers to this question,
providing 324 responses. The results are shown in Table 3. The question was followed up in the interviews. Although fifteen of the searches were used for unspecified purposes, miscellaneous comments from the questionnaires and interviews indicated that searches were used to help prepare presentations at grand rounds, as teaching tools with students and interns, and for preparation of papers. Moreover, 76% of the searches were shared with an average of seven people in both individual and group situations. 3. Would there be any carry-over after the service was stopped? Although eleven of the sixteen residents (69%) interviewed said they would use the service after their residency, the proof of the pudding will only be found in the future. 4. Would residents use the service even if the librarian were not physically present at residents' reports, by visiting or calling the library? In actuality we found that slightly more use per month was made of the service when the librarian was not present at meetings than when she was, though the difference was too small to be significant. What was significant, however, was that this increase occurred only among those residents who had already been exposed to the service personally during their rotations. New residents who heard of the service during the months it was offered only through the library did not make use of it. 5. Would the users be willing to pay something for the service? Almost unanimously the residents interviewed said they would not use the service if they had to pay themselves. They cited reasons such as lack of money and their belief that the school should furnish such service as part of the educational
TABLE 2 COSTS OF SERVICE, JAN.-MAR. 1975 1975
January February March Total Aver./Mo.
Personnel ..........
242.58 ..........
242.58 *
MEDLINE
411.70 577.95 379.04
1,368.69 456.23
Manual search
Photocopy
Total cost
29.44 19.86 13.08 62.38 20.79
42.80 94.00 52.00 188.80 62.93
$ 483.94 934.39 444.12
$1,862.45 *
*Not meaningful because two out of three months were nonattendance months.
Bull. Med. Libr. Assoc. 64(2) April 1976
237
BRIEF COMMUNICATIONS TABLE 3 USE TO WHICH INFORMATION WAS PUT
Number of responses received Response on use In diagnosis or treatment of a particular patient To verify a belief or
proceeding To add to general knowledge
Other
No.
Percent
324
100
112
35
60 137 15
19 42 5
program for interns and residents. This finding is as puzzling as the many similar findings reported from other groups; people seem to want a service but are reluctant or unwilling to pay for it themselves or through increases in taxes, tuition, etc. In our case the Department of Medicine agreed with the residents on the school's responsibility to furnish the service; the department will pay half the costs for 1975/76. This does not solve the dilemma of the need to bridge the gap between what is technologically and manually feasible and what society is willing to purchase, even for such desirable ends as better medical care or education.
The library has thus answered the specific question it raised without finding any solution to the broader issue. Finally, how good was the service, as judged by the recipients? Of those who returned the questionnaires, 155 (85%) said the searches were adequate, while 28 (15%) said they were not. Those who said the searches were not adequate indicated the following reasons: (1) did not provide the needed information; (2) there was no literature on the subject; (3) poor quality of the article on the subject; and (4) important older articles were not caught by the search methods used. The average percent of "on-target" relevant articles retrieved was 25% of the total, with a range from 18% to 40%. All recipients reported that the searches aided them by making the first "cut" and thus giving them more time to read the literature rather than to search for it. No one said the searches were generally poor or useless. CONCLUSION The worth of clinical librarians' programs has not yet been proved quantitatively or unequiv-
238
ocally, but neither has the usefulness of librarianship in general. The study done at the Washington University School of Medicine Library adds some data to the growing amount of information on this topic. We hope that further studies will elucidate whether clinical librarianship is a fad or a useful product. Note: Copies of the questionnaire used in this study can be obtained on request from the Librarian at the Washington University School of Medicine Library, 4580 Scott Avenue, St. Louis, Missouri 63110.
Computer-Assisted Instruction in Library Orientation and Services By JOAN TOMAY HICKS, Librarian College of Veterinary Medicine University of Illinois at Urbana-Champaign
AT the University of Illinois a computer-assisted instruction (CAI) system has been developed [1-2] which uses a large Control Data Corporation computer connected to remote terminals with plasma display panels and typewriter-like keyboards. This system, PLATO (Programmed Logic for Automatic Teaching Operation), is very flexible. Through PLATO many types of programs have been created at all levels of instruction from kindergarten through university graduate study [3]. For example, some programs involve medical students' diagnoses of simulated disease syndromes [4] or identification of bacteriological unknowns [5]. At the College of Veterinary Medicine in Urbana there are about thirty PLATO computer consoles. Most of them are located in the Basic Sciences Building in a room across from the library; the rest are situated in the Small Animal Clinic some distance away. There are presently about seventy CAI programs of study for veterinary medical students. One of these programs is on the library and its resources. In the library program the title page which appears on the terminal display panel briefly describes the main parts of the program and indicates which keys to press to go on to find the different kinds of information. One choice on the title page for an overall introduction brings to the screen a schematic plan of the physical arrangement of the library, one part at a time, with short Bull. Med. Libr. Assoc. 64(2) April 1976
