JOURNAL
OF SURGICAL
RESEARCH
20, 377-379 (1976)
A Critical
Appraisal Scientific
of the Effectiveness
of
Presentations’
A. R. KRAFT, M.D.2, J. D. SALETTA, M.D., G. S. MOSS, M.D. Departments of Surgery, Cook County Hospital and the University of Illinois, Abraham Lincoln School of Medicine, Chicago, Illinois
C. M. HERMAN, Naval Medical Research Institute,
M.D.
National Naval Medical Center, Bethesda, Maryland
AND R. K. TOMPKINS,
M.D.
University of California, Los Angeles, California Submitted for publication November 20.1975
The purpose of this investigation was to answer the question-how frequently do scientific presentations fail to transmit information effectively? To our knowledge there has never previously been a formal attempt to document this type of information quantitatively.
tempted to ascertain whether or not the presentor communicated information using the format of an introduction, materials and methods, results, and a summary or conclusion. The reviewers noted the use of slides and whether or not the slides were readable and self-explanatory.
MATERIALS AND METHODS The study was carried out at the 1974 Annual Meeting of the Association for Academic Surgery. Arbitrarily, 32 of the 108 papers presented at the meeting were reviewed and assessed (plenary session-10 papers; parallel scientific sessions-22 papers). The assessmentwas caried out using a standardized form (Table 1). Four members of the Association were assigned to review plenary session papers and two members were assigned to each of the parallel scientific sessions.The reviewers sat in various locations in the meeting room. The information that was derived was both objective and subjective. The reviewers at-
RESULTS
‘Presented at the 9th Annual Meeting, of the Association for Academic Surgery, November 13-15, 1975, Minneapolis, Minn. 2Mailing Address: Department of Surgery, Cook County Hospital, 1825 W. Harrison, Chicago, Illinois 60612.
A. Assessment of the Introductory
A question was posed or an hypothesis stated in five of the 32 papers reviewed. A slide was used in each of the five papers. However, in seven presentations the display of an introductory slide was not associated with a statement explaining the purpose of the paper. Approximately 50% of the slides that were shown were readable and selfexplanatory. This information is summarized in Table 2. B. Assessment of the Materials and Methods Section
In nearly two-thirds of the presentations that were reviewed, the design and strategy were clearly defined and explained to the audience. Slides were used in 60% of these papers but only one-third were readable and self-explanatory. This data is summarized in Table 3.
377 Copyright D 1976 by Academic Press, Inc. All rights of reproduction in any form reserved.
Section
378
JOURNAL OF SURGICAL
RESEARCH: VOL. 20, NO. 4, APRIL 1976
TABLE 1 Form Utilized for the Assessment of Each Presentation PaperII
Roam Location:
Exfc"lp.xaneo"s:
Yes
Introductory 51 ide(s) 51 idc
numbers
mLiqlltS
on/off
TI tlCd Easily
Legible
il of
I ines/sl
FrontWiddle>ack-
N" ___Materials/Methods 51 ides
Data 51 ides: Tabular
/I
4
n
/I
n
Y-N-
v -"a
Y-N-
V-N-
v -N-
Y2-w
V-N-
V-N-
Y-"wee
V-N-
Y-N--
Y-"-
Vw-N-
Y--"--
Data 51 ides: Graphic
Conclusion 5, ides
n
II
4
n
//
"mN--
Ve-N-
"N-
V-N-
V-.-N-
VL-v"--"-
VA-
V-N-
Vmm"--
Y-N-
Y-N-
V-"-
V-N-
y-N-
Y--N-
Y-N-
Y-"-
Y-N-
v _N-Y--N-
Y-N--
IY -N--
Y-"-
V--N--
Y-N-
V-N-
"L'-
V-N-
"2-m
"we!-
V-N--
V-N-
y-N-
ide
# words/line Sentence
usage
Pictoral li of
data
Labeled Graph
# of
bars
Linear
# of
_
V-N-
columns columns
Bar
V2
graph
Y-N-
Y-"-
Y-"-
Y-N-
V-"-Y-N.-
v -"A
V--NC--
"--N--u
V-"-
V-"-
V-N--
V-N-
Y-N-V>-
V-N--
V-N-
v -"-
"-N-
Y-N-.-
"Ne
Y-"w-
V-N-.-
VL-"-N--
V-N-
V-N-
Y_N-
V-N-.-
,v
-N-
mossages
Self-explanatorV Wandering
pointer
V
N
V
N
V
N
V
N
V
N
V
N
V
N
V
N
V
V
N
V
N
V
N
V
N
V
N
Y
N
V
N
V
N
V-N-
N
V
N
Y-N-
Niscellaneous
Signature
C. Assessment of the Results Section All papers included some form of data slide-pictorial, graphic or tabular. The data on these slides was clearly understood by the reviewers in less than 50% of the papers that were reviewed; 31% of the slides were readable and self-explanatory. This information is found in Table 4.
D. Assessment of the Conclusion Section Table 5 summarizes this data. A conclusion was stated in three-fourths of the papers that were reviewed and slides were shown in more than 50% of these papers. The slides pertaining to this section were readable and self-explanatory in 57% of the presentations.
TABLE 2 Data Obtained from the Assessment of the “Introductory” Section of the Papers that Were Reviewed
TABLE 3 Data Obtained from the Assessment of the “Material and Methods” Section of the Reviewed Papers
“Introductory”
Section
Question poseda Slide II tilized Slides readable and self-explanatory aOnly if slide shown.
Papers
Materials/methods section
5 of32 11 of 32
Design and strategy explained Slides utilized Slides readable and self-explanatory
6ofll
Papers 20 of 32 12 of 32 4 of 12
KRAFT ET AL.: CRITICAL
APPRAISAL
TABLE 4 Data Obtained from the “Results” Section Assessment of the Reviewed Papers Results section
Papers
Slides utilized Data “clearly” understood Slides readable and self-explanatory
32 of 32 14 of 32 12 of 32
OF SCIENTIFIC
379
PRESENTATIONS
TABLE 6 A Comparison of the Effectiveness of Transmitting Scientific Data with and without Visual Aids
Question posed Design explained Results “clear” Conclusion stated
With slide
Without slide
42% 37% 44% 57%
0% 40% N.A.a 22%
‘N.A. = Not applicable (all utilized slides).
E. Miscellaneous
Data
Sixty-six percent of the papers were read; the remainder were presented extemporaneously. There were mechanical problems relating to the presentation (slides shown out of order, pointer misuse, lights misused) in 45% of the presentations that were reviewed. DISCUSSION It is presumed that the use of visual aids in presenting scientific papers should enhance or at least augment communication between the presentor and his audience. The key feature of this study was to assesswhether or not an author was “getting his message across” and whether or not this was better accomplished with or without the use of slides. Table 6 synthesizes the study data. It clearly appeared that the chance of underTABLE 5 Data Obtained from the “Conclusion” Section Assessment of the Papers Under Review “Conclusion” Section
Papers
Conclusion stated Slides shown Readable and self-explanatory
25 of 32 14 of 32 8 of 14
standing the purpose for which a paper was being presented was decreased if a slide was not used. It appeared that the design and strategy of the experiment were equally explained with or without slides but regardless of this fact the majority of the papers poorly communicated this information. Only 44% of the data slides were self-explanatory, and two-thirds of these data slides were readable. If a summary slide was shown, there was a better likelihood of understanding the conclusion(s) derived from the investigation. On the basis of the foregoing information it appears that, of the presentations surveyed at the 1974 Association for Academic Surgery Meeting, the majority of presentations failed to transmit information effectively. SUMMARY A critical review was undertaken of onethird of the scientific presentations at the 1974 Annual Meeting of the Association for Academic Surgery. The purpose of this review was to determine the frequency with which scientific presentations failed to transmit information effectively. The data demonstrated that more than 50% of the surveyed presentations failed to transmit scientific data effectively.
OF SURGICAL
RESEARCH
20, 377-379 (1976)
A Critical
Appraisal Scientific
of the Effectiveness
of
Presentations’
A. R. KRAFT, M.D.2, J. D. SALETTA, M.D., G. S. MOSS, M.D. Departments of Surgery, Cook County Hospital and the University of Illinois, Abraham Lincoln School of Medicine, Chicago, Illinois
C. M. HERMAN, Naval Medical Research Institute,
M.D.
National Naval Medical Center, Bethesda, Maryland
AND R. K. TOMPKINS,
M.D.
University of California, Los Angeles, California Submitted for publication November 20.1975
The purpose of this investigation was to answer the question-how frequently do scientific presentations fail to transmit information effectively? To our knowledge there has never previously been a formal attempt to document this type of information quantitatively.
tempted to ascertain whether or not the presentor communicated information using the format of an introduction, materials and methods, results, and a summary or conclusion. The reviewers noted the use of slides and whether or not the slides were readable and self-explanatory.
MATERIALS AND METHODS The study was carried out at the 1974 Annual Meeting of the Association for Academic Surgery. Arbitrarily, 32 of the 108 papers presented at the meeting were reviewed and assessed (plenary session-10 papers; parallel scientific sessions-22 papers). The assessmentwas caried out using a standardized form (Table 1). Four members of the Association were assigned to review plenary session papers and two members were assigned to each of the parallel scientific sessions.The reviewers sat in various locations in the meeting room. The information that was derived was both objective and subjective. The reviewers at-
RESULTS
‘Presented at the 9th Annual Meeting, of the Association for Academic Surgery, November 13-15, 1975, Minneapolis, Minn. 2Mailing Address: Department of Surgery, Cook County Hospital, 1825 W. Harrison, Chicago, Illinois 60612.
A. Assessment of the Introductory
A question was posed or an hypothesis stated in five of the 32 papers reviewed. A slide was used in each of the five papers. However, in seven presentations the display of an introductory slide was not associated with a statement explaining the purpose of the paper. Approximately 50% of the slides that were shown were readable and selfexplanatory. This information is summarized in Table 2. B. Assessment of the Materials and Methods Section
In nearly two-thirds of the presentations that were reviewed, the design and strategy were clearly defined and explained to the audience. Slides were used in 60% of these papers but only one-third were readable and self-explanatory. This data is summarized in Table 3.
377 Copyright D 1976 by Academic Press, Inc. All rights of reproduction in any form reserved.
Section
378
JOURNAL OF SURGICAL
RESEARCH: VOL. 20, NO. 4, APRIL 1976
TABLE 1 Form Utilized for the Assessment of Each Presentation PaperII
Roam Location:
Exfc"lp.xaneo"s:
Yes
Introductory 51 ide(s) 51 idc
numbers
mLiqlltS
on/off
TI tlCd Easily
Legible
il of
I ines/sl
FrontWiddle>ack-
N" ___Materials/Methods 51 ides
Data 51 ides: Tabular
/I
4
n
/I
n
Y-N-
v -"a
Y-N-
V-N-
v -N-
Y2-w
V-N-
V-N-
Y-"wee
V-N-
Y-N--
Y-"-
Vw-N-
Y--"--
Data 51 ides: Graphic
Conclusion 5, ides
n
II
4
n
//
"mN--
Ve-N-
"N-
V-N-
V-.-N-
VL-v"--"-
VA-
V-N-
Vmm"--
Y-N-
Y-N-
V-"-
V-N-
y-N-
Y--N-
Y-N-
Y-"-
Y-N-
v _N-Y--N-
Y-N--
IY -N--
Y-"-
V--N--
Y-N-
V-N-
"L'-
V-N-
"2-m
"we!-
V-N--
V-N-
y-N-
ide
# words/line Sentence
usage
Pictoral li of
data
Labeled Graph
# of
bars
Linear
# of
_
V-N-
columns columns
Bar
V2
graph
Y-N-
Y-"-
Y-"-
Y-N-
V-"-Y-N.-
v -"A
V--NC--
"--N--u
V-"-
V-"-
V-N--
V-N-
Y-N-V>-
V-N--
V-N-
v -"-
"-N-
Y-N-.-
"Ne
Y-"w-
V-N-.-
VL-"-N--
V-N-
V-N-
Y_N-
V-N-.-
,v
-N-
mossages
Self-explanatorV Wandering
pointer
V
N
V
N
V
N
V
N
V
N
V
N
V
N
V
N
V
V
N
V
N
V
N
V
N
V
N
Y
N
V
N
V
N
V-N-
N
V
N
Y-N-
Niscellaneous
Signature
C. Assessment of the Results Section All papers included some form of data slide-pictorial, graphic or tabular. The data on these slides was clearly understood by the reviewers in less than 50% of the papers that were reviewed; 31% of the slides were readable and self-explanatory. This information is found in Table 4.
D. Assessment of the Conclusion Section Table 5 summarizes this data. A conclusion was stated in three-fourths of the papers that were reviewed and slides were shown in more than 50% of these papers. The slides pertaining to this section were readable and self-explanatory in 57% of the presentations.
TABLE 2 Data Obtained from the Assessment of the “Introductory” Section of the Papers that Were Reviewed
TABLE 3 Data Obtained from the Assessment of the “Material and Methods” Section of the Reviewed Papers
“Introductory”
Section
Question poseda Slide II tilized Slides readable and self-explanatory aOnly if slide shown.
Papers
Materials/methods section
5 of32 11 of 32
Design and strategy explained Slides utilized Slides readable and self-explanatory
6ofll
Papers 20 of 32 12 of 32 4 of 12
KRAFT ET AL.: CRITICAL
APPRAISAL
TABLE 4 Data Obtained from the “Results” Section Assessment of the Reviewed Papers Results section
Papers
Slides utilized Data “clearly” understood Slides readable and self-explanatory
32 of 32 14 of 32 12 of 32
OF SCIENTIFIC
379
PRESENTATIONS
TABLE 6 A Comparison of the Effectiveness of Transmitting Scientific Data with and without Visual Aids
Question posed Design explained Results “clear” Conclusion stated
With slide
Without slide
42% 37% 44% 57%
0% 40% N.A.a 22%
‘N.A. = Not applicable (all utilized slides).
E. Miscellaneous
Data
Sixty-six percent of the papers were read; the remainder were presented extemporaneously. There were mechanical problems relating to the presentation (slides shown out of order, pointer misuse, lights misused) in 45% of the presentations that were reviewed. DISCUSSION It is presumed that the use of visual aids in presenting scientific papers should enhance or at least augment communication between the presentor and his audience. The key feature of this study was to assesswhether or not an author was “getting his message across” and whether or not this was better accomplished with or without the use of slides. Table 6 synthesizes the study data. It clearly appeared that the chance of underTABLE 5 Data Obtained from the “Conclusion” Section Assessment of the Papers Under Review “Conclusion” Section
Papers
Conclusion stated Slides shown Readable and self-explanatory
25 of 32 14 of 32 8 of 14
standing the purpose for which a paper was being presented was decreased if a slide was not used. It appeared that the design and strategy of the experiment were equally explained with or without slides but regardless of this fact the majority of the papers poorly communicated this information. Only 44% of the data slides were self-explanatory, and two-thirds of these data slides were readable. If a summary slide was shown, there was a better likelihood of understanding the conclusion(s) derived from the investigation. On the basis of the foregoing information it appears that, of the presentations surveyed at the 1974 Association for Academic Surgery Meeting, the majority of presentations failed to transmit information effectively. SUMMARY A critical review was undertaken of onethird of the scientific presentations at the 1974 Annual Meeting of the Association for Academic Surgery. The purpose of this review was to determine the frequency with which scientific presentations failed to transmit information effectively. The data demonstrated that more than 50% of the surveyed presentations failed to transmit scientific data effectively.
