Recognition Accuracy of Current Operating Room Alarms Robert G. Loeb, MD, Brian R. Jones, MD, Rebecca A. Leonard, PhD, and Kendra Behrman, RN Departments of Anesthesiology and Otolaryngology, University of California-Davis, Sacramento, California
This prospective study was performed to determine whether anesthesia clinicians (i.e., both anesthesiologists and nurse anesthetists) can identify operating room alarms by their distinctive sounds and to identify factors related to alarm recognition accuracy. Nineteen alarms from 15 commonly used devices were recorded. These sounds were played, in a quiet room, to 44 anesthesia clinicians. The clinicians were asked to choose from a list the device that produced the alarm. After this recognition test, the clinicians rated the importance of each alarm and the frequency with which they heard it in the clinical
M
odern operating rooms are filled with devices that generate audible alarms intended to alert the clinician to potentially hazardous conditions. It is therefore important that the clinician be able to quickly identify any alarm, because the inability to do so can delay or prevent the appropriate corrective action (1). The three criteria for determining the effectiveness of audible alarms are audibility, attention demand, and recognition (2). Audibility refers to whether a sound can be heard in the presence of typical background noise. Attention demand is a measure of whether an unexpected alarm will be perceived. Recognition implies that the sound will be interpreted as an alarm and that its meaning will be understood. Noise-level readings taken during typical operations have established the fact that the operating room is indeed a noisy environment, comparable to that of a freeway (3,4). Even so, audible alarm signals that can be reliably detected above operating room noise levels have been identified (5). Stanford et al. (5) concluded that signals with spectral richness, Presented in part at the Annual Meeting of the Society of Anesthesiologists, Las Vegas, Nevada, October 1990. Accepted for publication May 19, 1992. Address correspondence to Dr. Loeb, Department of Anesthesiology, University of California-Davis, Medical Center, Sacramento, CA 95817. Reprints will not be available from the authors. 01992 by the International Anesthesia Research Society 0003-2999/92$5.00
Medical Center,
situation. Clinicians correctly identified the alarm source 34% of the time. The recognition rate was higher for alarms rated as heard more frequently; however, alarms that were rated as more important were less likely to be correctly identified. Complexity of the sound did not influence accuracy of recognition. Most errors were attributed to similarities in sound or function, or both, among alarms. We conclude that anesthetists cannot reliably identify current operating room alarms by their distinctive sounds. (Anesth Analg 1992;75:499-505)
frequency modulation, and temporal patterning could be reliably detected amid operating room equipment noises in a laboratory setting. There are anecdotal reports of difficulty in identifying the source of an alarm in the clinical environment (6,7). A presumed reason is that many alarms have similar sounds; yet, of 852 Canadian anesthetists responding to a survey, half (422) indicated that the distinctiveness of the alarm sound was more useful in determining its source than was either the direction of the sound source or a visual indicator (8). The aim of our study was to measure anesthesia clinicians’ ability to identify the source of familiar alarms by sound characteristics alone. We also investigated the factors that influence alarm recognition.
Methods Nineteen alarm sounds from 15 devices in our operating rooms were digitally recorded at 22 Hz with 8-bit resolution. The volume of each alarm was also measured with a precision sound-level meter (model 2209, Bruel and Kjaer Instrumentation Co., Marlborough, Mass.) (A scale). Each volume measurement was performed from the head of the operating room table and at a height of 4 ft, with the device in its typical location and set at its default volume level. The volume of each recorded alarm, played through headphones, was then adjusted to that of the original Anesth Analg 1992;75:499-505
499
500
LOEB ET AL. ALARM RECOGNITION ACCURACY
alarm by means of the sound-level meter and a commercial software program (SoundEdit 1.O, Farallon Computing, Berkeley, Calif.). The complexity of each alarm sound was rated by an audiologist who was unfamiliar with the operating room. She classified each as continuous tone, intermittent tones separated by periods of silence, or multitone. Our institutional review board granted approval, and all subjects consented to the study. Twenty-three anesthesia residents, 12 anesthesia attending physicians, and nine certified registered nurse anesthetists (CRNA) participated. Each of the subjects routinely used all of the devices from which the alarms were recorded. Each subject was tested while seated in a quiet room. The subject's hearing was screened at 1000, 2000, 3000, and 5000 Hz, and deficits >30 dB were noted. After the hearing test, the subject heard the recorded alarm sounds through calibrated headphones. The alarms were presented in random order, and each was presented twice during the session. The subject was asked to select from a list the device that generated the alarm sound. The list contained one entry for each of the 15 devices and an entry named "Don't know." Each entry listed the type of device, the manufacturer, and a short description of the device's appearance. Subjects received the following written instructions: Each alarm will sound for 17 seconds and will be followed by 10 seconds of silence. You should verbally identify the monitor which generates the alarm as soon as you can during the 27 seconds after the alarm begins to sound. If you have not responded after the 27 seconds have elapsed, you will be asked to either identify the monitor or indicate that you "don't know".
After the alarm recognition test, each subject was given a list of the 19 alarm sounds and was asked to indicate for each (a) how important the alarm is in clinical practice (critical, important, or advisory); and (b) how frequently, in his or her experience, the alarm occurred (frequently, infrequently, or rarely). Two demographic items were collected for each subject: (a) clinical training (resident, faculty, or CRNA); and (b) clinical experience (number of years since beginning anesthesia training). Data are expressed as frequencies of correct recognitions. Comparisons between groups of alarm importance and frequency of alarm occurrence were by 2 analysis. The correlation between each clinician's rating of alarm importance and frequency of alarm occurrence was analyzed with Spearman rank correlation. Least-squares regression was used to determine the effects of clinical experience and alarm volume on recognition rate. Tests were performed with Data Desk
ANESTH ANALG 1992:75499-505
version 3.0 (Odesta Corp., Northbrook, Ill.). A value of P < 0.05 was considered significant.
Results Forty-four anesthesia clinicians heard each of 19 alarms twice, providing a total of 1672 responses. Table 1 presents the frequencies of responses for each alarm sound. The correct responses are outlined on the main diagonal. The alarm sounds were correctly identified 571 times (34%) and incorrectly identified 843 times (50%);clinicians admitted to not knowing the correct answer 258 times (16%). The clinician with the best performance, a new resident, identified 26 alarms correctly (68%);the worst performance, by a clinician with 26 years' experience, was three correct (8%).There was no correlation between a subject's clinical experience and his or her ability to correctly idenhfy alarms (Figure 1). Hearing deficits were detected in six subjects. The scores of the clinicians with hearing deficits were not different from those with normal hearing. On the questionnaire, clinicians judged the tested alarms as critical 37% of the time, important 34% of the time, and advisory 29% of the time. When clinicians rated an alarm as more important, it was less likely that they would correctly identify it (Figure 2). Alarms were rated as being frequently, infrequently, and rarely heard 30%, 30%, and 41% of the time, respectively. The more frequently clinicians believed that they heard an alarm, the more likely it was that they would identify it (Figure 3). For each clinician, there was a negative correlation between how important he or she thought the alarm was and how frequently he or she thought that it was heard. The alarms believed to be less important were judged to be heard more frequently. The alarm with the highest recognition rate (Fisher & Paykel humidifier) was correctly identified 92% of the time, whereas that with the lowest recognition rate (Ohmeda 7000 Ventilator power failure) was correctly identified only 1% of the time. Six of the alarms were continuous tones, five were intermittent tones separated by periods of silence, and eight were multitone. Sound complexity did not affect recognition rate; however, there was a positive correlation between alarm volume and recognition rate (P < 0.01) (Figure 4). Table 1 presents the frequencies of responses for each alarm sound. The correct responses are outlined on the main diagonal. Entries outside the main diagonal and the "Don't know" column are mismatches; these may have occurred owing to confusion or chance. Mismatches due to random chance would result in a cell frequency of
LOEB ET AL. ALARM RECOGNITION ACCURACY
ANESTH ANALG 1992;75:4%505
501
Table 1. Clinicians' identification of alarm sounds, grouped by sound categories. Correct responses are outlined on the main diagonal. Relevant confusions are in bold type. Shading indicates a sound pattern similarity between the device on the horizontal axis and the one on the vertical axis.
Line Isolation Monitor Bard Alfenta Pump
3
2
1
1
NeUcor Pulse Oximeter
4
4
1
1
ValleyLab Electrmautery
1
2
2
Ohmeda 7000 Ventilator (disconnect)
6
3
1
Miniox Oxygen Analyzer Ohmeda Oxygen Analyzer SARA Mass Spectrometer
5 5
3
1
8
3
1
4
1
2
I 1
4
3
2
3
1
2
I
21
88
4
2
13
88
1
4
14
88
2
7
2 8 8
Datex Airway Gas Monitor Ohmeda Volumeter (threshold)
1
3
3
Ohmeda Volumeter (apnea)
1
1
2
Ohmeda Airway Pressure
5
9
1 1 0 1
2
Ohmeda Modulus I (low oxygen pressure) 10 Ohmeda Modulus II (power failure)
1
Dinamap (technical) Dinamap (threshold)
1
Puritan Bennett Multifunction
2
1
9
6
1
1
10
11
1
2
2
1
71
32
1 1
1
6
Fir & Paykel Humidfir
2
1
60
90 123 73
1
1
91
Tntal
A
146 107
Faculty and CRNAs
99
90 104
1
25
88
3
24
88
2
18
88
2
128
80
120 258 1672
(I)
Q
incorrectly identified
0
correc(ly identified
f
600
L
a
0
8 400
10
w-
0
z
J
0
2
3
5 10 15 20 25 30 35
Years of experience Figure 1. Number of alarm sounds correctly identified (of 38) versus the clinician's years of anesthesia experience.
200
5 =
o Critical
Important
Advisory
Clinical importance of alarm
Figure 2. Number of correctly and incorrectly identified alarm sounds versus the clinician's rating of the alarm's clinical importance. There is a significant difference between groups (P < 0.Oool).
[(No. of times that each alarm sound was presented) - (No. of "Don't know" responses)] + (No. of possible responses)
or (88 - (No. of "Don't know" responses)] + 15
for each row. Cell frequencies that exceed this threshold indicate a relevant confusion (9). Interpreted in
this way, 571 responses were correct (34%),573 were relevant confusions (34%), 258 were "Don't know" (16%),and 270 were random mismatches (16%).
502
LOEB ET AL. ALARM RECOGNITION ACCURACY
ANESTH ANALG 1992;75:499-505
800 u)
P)
0 C
600 L
3
0
i;
400
r
0 L
2
s z
200
0
sgbl 75%
52%
Frequently
Infrequently
Rarely
Frequency that alarm is heard in clinical practice
Figure 3. Number of correctly and incorrectly identified alarm sounds versus how frequently the clinician hears the alarm in clinical practice. There is a significant difference between groups (P < 0.00Ol).
/
a
I
.~
'"I t
n
f
z
a
-#
201
Y
060
t
65
1
70
75
80
85
I
90
Alarm volume (dB) Figure 4. The number of times a n alarm was correctly identified versus the alarm volume. Each alarm was presented 88 times, twice each to 44 subjects. The linear regression line is significant at P < 0.01.
Discussion The data indicate that anesthesia clinicians cannot determine the source of current commercial operating room alarms solely by sound characteristics. This contradicts the subjective impression (8). The study does not imply that alarms cannot be identified quickly in the clinical setting. During clinical use, identification may be facilitated by visual indicators on the monitor, the direction from which the sound is produced, and interpretation of the clinical scenario at the time of the event. Alarm recognition in the operating room is hampered when each device has a proprietary alarm sound. This is due to a number of factors. First, numerous sounds must be recognized. We identified 19 audible alarms in the operating rooms at our institution. Second, alarms may occur simultaneously. Third, the user may rarely use a particular device and thus infrequently hear a particular alarm. Experiments by Patterson and Milroy (10) indicate that subjects can readily learn to recognize four to six audible alarms, and 1 wk after learning 10 alarms,
subjects can reliably recognize only six. This is consistent with other studies that demonstrate the limitations of humans in differentiating among multiple stimuli (11). It suggests that the number of alarm sounds should be reduced to a maximum of six. The number of sounds can be decreased in two ways: standardization of sounds and limitation of alarms. Standardized alarms could reduce the total number of sounds and would ensure that unfamiliar equipment had familiar alarms. Limiting the use of audible alarms would reduce the total number of sounds and the chance of simultaneous alarms. Eventually, integrated monitoring systems with centralized alarms would be ideal, but in the interim, the situation could be improved by standardizing audible alarms and limiting their use. It is intuitive that the most important alarms should have sounds that are most easily identified by anesthesiologists; however, we found an inverse relationship between alarm importance and recognizability. For example, the Fisher & Paykel humidifier alarm had a low importance rating but was very well recognized. This alarm is loud, continuous, and has a complex sound pattern. These features may elicit a strong affective response in the anesthetist that may increase recognition. A related study by Finley and Cohen (12) found no correlation between the perceived urgency of commercial operating room alarm sounds and the urgency of the clinical situation. Thus, there is accumulating evidence that current commercial auditory alarms are poorly correlated to the situations of which they are warning. Although our study demonstrates a problem with the current unstructured approach to alarms, it provides little data to guide manufacturers in developing more recognizable alarms. There was a positive correlation between alarm volume and recognition rate; however, we do not recommend that alarm volumes be increased indiscriminately. An anesthesiologist's initial response might be to silence a louder alarm rather than address the clinical problem that caused it. Also, louder alarms would be more disruptive to the rest of the operating room team. We presumed that the more complex alarm sounds would be better recognized, but our data do not support this conclusion. Additional research is needed to develop alarm sounds that are recognizable and detectable but not disruptive. The more urgent and recognizable sounds should be reserved for the more important alarms, and before general use, the sounds should be evaluated in operating rooms to determine their effect on personnel performance. We looked for patterns of recognition errors in our data to understand the processes involved in recognizing alarm sounds. This method has been used previously to investigate factors that affect other
LOEB ET AL. ALARM RECOGNlTlON ACCURACY
ANESTH ANALG 1992;75:499-505
503
Table 2. Alarm Sound Categories Category description Continuous Broadband
Distinct tone Broadband, siren Intermittent tone Approximate cycle time of 1 s
Approximate cycle time of 0.3 s Burst of intermittent tone separated by long pause
Greater than two tones (these two not similar to each other)
Alarm category Line Isolation Monitor Nellcor Pulse Oximeter Bard Alfenta Pump Buzzing of Electrocautery" Ohmeda 7000 Ventilator (power failure) Miniox Oxygen Analyzer Ohmeda Modulus I (low oxygen pressure) Ohmeda Oxygen Analyzer Ohmeda Volumeter (threshold) Ohmeda Volumeter (apnea) Ohmeda Airway Pressure SARA Mass Spectrometer Datex Airway Gas Monitor Ohmeda 7000 Ventilator (disconnect) ValleyLab electrocautery Dinamap (threshold) Dinamap (technical) Ohmeda Modulus I1 (power failure) Puritan Bennett Multifunction Fisher & Paykel Humidifier
UEventhough it is not an alarm, the "buzzing" sound of electrocautery was included in this category. See text for explanation.
recognition tasks (9). We were most interested in understanding the relevant confusions, because these signify systematic errors in the interpretation of the alarm sounds. Retrospective review of the data suggested that many mismatches resulted from two underlying causes: (a) sound confusion and (b) function confusion. The first, sound confusion, was a mismatch between alarms with similar sound qualities. We retrospectively classified the alarms by similarity of sound into seven categories (Table 2). Table 1 is grouped by sound categories to highlight the areas of potential confusion. An example of sound confusion is the 14 responses of "Dinamap" to the Ohmeda Modulus I1 power failure alarm. The Modulus and Dinamap alarms sound similar; both are complex sounds with a burst of patterned tones separated by long periods of silence. Another example is the 33 responses of "electrocautery" to the Nellcor pulse oximeter alarm. This alarm is a constant, broadband sound that is similar to the "buzzing" of electrocautery. There were 327 relevant confusions that we believed could be attributed, at least in part, to sound confusion. The second cause of mismatches was function confusion. This occurred when the subject attributed an alarm to another piece of equipment with a similar clinical function. Each alarm was classified as being in one or more of the folIowing function categories: gas
concentration, pressure, circuit disconnect, power loss, oxygenation, ventilation, circulation, and temperature. Table 3 displays our data grouped by function categories to highlight the areas of potential confusion. Examples are the confusions among the gas analyzers: the Miniox oxygen analyzer, the Ohmeda oxygen analyzer, the Datex airway gas monitor, and the SARA mass spectrometer. Another function confusion, which may be more difficult to appreciate, is the anesthetist's response of "airway pressure" to the Ohmeda Modulus I low-oxygen-supply pressure alarm; however, these are both pressure alarms. There were 202 relevant confusions that we believed could be attributed, at least in part, to function confusion. In this retrospective analysis of relevant confusions, 36% were due to sound confusion, 15% to function confusion, 21% to both, and 28% to neither. Function confusions suggest that clinicians mentally group alarm sounds by monitor function. This may be advantageous in the clinical environment, because attention is directed to the appropriate system. It also reinforces the position that alarm sounds should be grouped by function (13). Our results indicate that the recognition rate for the alarm's function could be readily increased to 55% (the number of correct responses plus the number of relevant confusions due to function confusion) if alarms with similar functions had similar sounds.
504
ANESTH ANALG 1992;75499-505
LOEB ET AL. ALARM RECOGNITION ACCURACY
Table 3. Clinicians' identification of alarm sounds, grouped by function categories. Correct responses are outlined on the main diagonal. Relevant confusions are in bold fype. Shading indicates a functional similarity between the devices on the vertical and horizontal axes.
Line Isolation Monitor
1
Bard Alfenta Pump
3
1
1
3
ValIeyLab Elearocautery
2
6
1
1
1
I
12
I
21
88
4
8
88
25
88
88
Ohmeda Modulus 11 (power failure) Ohmeda Modulus I (low oxygen pressure Ohmeda Volumeter (threshold)
1
3
Ohmeda Volumeter (apnea)
1
1
2 1
7
2
9
I
4
14
88
2
7
2
88
4
11
88
Ohmeda 7000 Ventilator (disconnect)
ohmeda7000 Ventilator (power failure)
18
88
Ohmeda Airway Pressure
5
5
10
88
SARA Mass Spectrometer
4
z
13
88
3
10
88
Datex Airway Gas Monitor
Miniox Oxygen Analyzer Obmeda Oxygen Analyzer
Nellcor Pulse Oximeter
Dinaroap (technical) Dinamap (threshold) PuritanBennett Multifunction
Fisher & Paykel Humidifm
Total
91
71
146
2
1
73
90
Our study has a number of potential design flaws. First, recorded alarms were used rather than the sounds from the actual machines. However, the investigators compared high-quality cassette recordings and digital recordings with the original sounds and believed that the digital system had superior playback sound quality. Even so, any sound recording and playback has the potential for distortion, and 13 of the subjects believed that the quality of the recordings interfered with their ability to recognize at least some of the alarms. Second, the 19 alarms that we recorded would not all be present during a single operative case. Although all of the devices are used at our institution, there are variations in equipment among operating rooms. For example, we tested the alarms from two differentoxygen analyzers, although only one model would be used in any single case. This artificially increased the number of possible alarms and may have interfered with the use of a situational factor in the recognition of alarms. Experienced cIinicians an-
1
107
123 104
60
99
90
32
128
80
120 258 1672
ticipate foreseeable problems better than trainees do (14).Thus, a situational factor may be used by the experienced clinician in that he or she may expect particular alarms at certain phases during the course of an operation or may not consider alarms from equipment not currently in use. There may also be other factors, such as stage of the operation or patient illness, that cue the experienced clinician to particular monitors and alarms. We conclude that current operating room alarms cannot be reliably recognized by sound characteristics alone. Some sounds are better recognized than others, and anesthetists mentally group alarm sounds by function.
References 1. Schreiber PJ, Schreiber J. Structured alarm systems for the
operating room. J Clin Monit 1989;52014. 2. Wilkins PA. Assessing the effectiveness of auditory warnings. Br J Audio1 1981;15:2&74.
ANESTH ANALG 1992;75:499-505
3. Shapiro RA, Berland T. Noise in the operating room. N Engl J Med 1972;28712364. 4. Hodge B, Thompson JF. Noise pollution in the operating theatre. Lancet 1990;335:891-4. 5. Stanford LM, McIntyre JWR, Hogan JT. Audible alarm signals
for anaesthesia monitoring equipment. Int J Clin Monit Comput 1985;1:2514. 6. Samuels SI. An alarming problem (letter). Anesthesiology 1986;M: 128. 7. Schmidt SI, Baysinger CL. Alarms: help or hindrance? (letter). Anesthesiology 1986;64:654-5. 8. Mclntyre JWR. Ergonomics: anaesthetists’ use of auditory alarms in the operating room. Int J Clin Monit Comput 1985;2:47-55.
LOEB ET AL. ALARM RECOGNITION ACCURACY
505
9. Zwaga HJ, Boersema T. Evaluation of a set of graphic symbols. Appl Ergon 1983;16:43-54.
10. Patterson RD, Milroy R. Auditory warnings on civil aircraft: the learning and retention of warnings. Final Contract Report 7D/S/0142, MRC Applied Psychology Unit, Cambridge, 1980. 11. Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 1956;6381-97. 12. Finley GA, Cohen A]. Perceived urgency and the anaesthetist: responses to common operating room monitor alarms. Can J Anaesth 1991;38:958-64. 13. Kerr JH. Warning devices. Br J Anaesth 1985;57696-708. 14. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg 1991;72:30%15.
This prospective study was performed to determine whether anesthesia clinicians (i.e., both anesthesiologists and nurse anesthetists) can identify operating room alarms by their distinctive sounds and to identify factors related to alarm recognition accuracy. Nineteen alarms from 15 commonly used devices were recorded. These sounds were played, in a quiet room, to 44 anesthesia clinicians. The clinicians were asked to choose from a list the device that produced the alarm. After this recognition test, the clinicians rated the importance of each alarm and the frequency with which they heard it in the clinical
M
odern operating rooms are filled with devices that generate audible alarms intended to alert the clinician to potentially hazardous conditions. It is therefore important that the clinician be able to quickly identify any alarm, because the inability to do so can delay or prevent the appropriate corrective action (1). The three criteria for determining the effectiveness of audible alarms are audibility, attention demand, and recognition (2). Audibility refers to whether a sound can be heard in the presence of typical background noise. Attention demand is a measure of whether an unexpected alarm will be perceived. Recognition implies that the sound will be interpreted as an alarm and that its meaning will be understood. Noise-level readings taken during typical operations have established the fact that the operating room is indeed a noisy environment, comparable to that of a freeway (3,4). Even so, audible alarm signals that can be reliably detected above operating room noise levels have been identified (5). Stanford et al. (5) concluded that signals with spectral richness, Presented in part at the Annual Meeting of the Society of Anesthesiologists, Las Vegas, Nevada, October 1990. Accepted for publication May 19, 1992. Address correspondence to Dr. Loeb, Department of Anesthesiology, University of California-Davis, Medical Center, Sacramento, CA 95817. Reprints will not be available from the authors. 01992 by the International Anesthesia Research Society 0003-2999/92$5.00
Medical Center,
situation. Clinicians correctly identified the alarm source 34% of the time. The recognition rate was higher for alarms rated as heard more frequently; however, alarms that were rated as more important were less likely to be correctly identified. Complexity of the sound did not influence accuracy of recognition. Most errors were attributed to similarities in sound or function, or both, among alarms. We conclude that anesthetists cannot reliably identify current operating room alarms by their distinctive sounds. (Anesth Analg 1992;75:499-505)
frequency modulation, and temporal patterning could be reliably detected amid operating room equipment noises in a laboratory setting. There are anecdotal reports of difficulty in identifying the source of an alarm in the clinical environment (6,7). A presumed reason is that many alarms have similar sounds; yet, of 852 Canadian anesthetists responding to a survey, half (422) indicated that the distinctiveness of the alarm sound was more useful in determining its source than was either the direction of the sound source or a visual indicator (8). The aim of our study was to measure anesthesia clinicians’ ability to identify the source of familiar alarms by sound characteristics alone. We also investigated the factors that influence alarm recognition.
Methods Nineteen alarm sounds from 15 devices in our operating rooms were digitally recorded at 22 Hz with 8-bit resolution. The volume of each alarm was also measured with a precision sound-level meter (model 2209, Bruel and Kjaer Instrumentation Co., Marlborough, Mass.) (A scale). Each volume measurement was performed from the head of the operating room table and at a height of 4 ft, with the device in its typical location and set at its default volume level. The volume of each recorded alarm, played through headphones, was then adjusted to that of the original Anesth Analg 1992;75:499-505
499
500
LOEB ET AL. ALARM RECOGNITION ACCURACY
alarm by means of the sound-level meter and a commercial software program (SoundEdit 1.O, Farallon Computing, Berkeley, Calif.). The complexity of each alarm sound was rated by an audiologist who was unfamiliar with the operating room. She classified each as continuous tone, intermittent tones separated by periods of silence, or multitone. Our institutional review board granted approval, and all subjects consented to the study. Twenty-three anesthesia residents, 12 anesthesia attending physicians, and nine certified registered nurse anesthetists (CRNA) participated. Each of the subjects routinely used all of the devices from which the alarms were recorded. Each subject was tested while seated in a quiet room. The subject's hearing was screened at 1000, 2000, 3000, and 5000 Hz, and deficits >30 dB were noted. After the hearing test, the subject heard the recorded alarm sounds through calibrated headphones. The alarms were presented in random order, and each was presented twice during the session. The subject was asked to select from a list the device that generated the alarm sound. The list contained one entry for each of the 15 devices and an entry named "Don't know." Each entry listed the type of device, the manufacturer, and a short description of the device's appearance. Subjects received the following written instructions: Each alarm will sound for 17 seconds and will be followed by 10 seconds of silence. You should verbally identify the monitor which generates the alarm as soon as you can during the 27 seconds after the alarm begins to sound. If you have not responded after the 27 seconds have elapsed, you will be asked to either identify the monitor or indicate that you "don't know".
After the alarm recognition test, each subject was given a list of the 19 alarm sounds and was asked to indicate for each (a) how important the alarm is in clinical practice (critical, important, or advisory); and (b) how frequently, in his or her experience, the alarm occurred (frequently, infrequently, or rarely). Two demographic items were collected for each subject: (a) clinical training (resident, faculty, or CRNA); and (b) clinical experience (number of years since beginning anesthesia training). Data are expressed as frequencies of correct recognitions. Comparisons between groups of alarm importance and frequency of alarm occurrence were by 2 analysis. The correlation between each clinician's rating of alarm importance and frequency of alarm occurrence was analyzed with Spearman rank correlation. Least-squares regression was used to determine the effects of clinical experience and alarm volume on recognition rate. Tests were performed with Data Desk
ANESTH ANALG 1992:75499-505
version 3.0 (Odesta Corp., Northbrook, Ill.). A value of P < 0.05 was considered significant.
Results Forty-four anesthesia clinicians heard each of 19 alarms twice, providing a total of 1672 responses. Table 1 presents the frequencies of responses for each alarm sound. The correct responses are outlined on the main diagonal. The alarm sounds were correctly identified 571 times (34%) and incorrectly identified 843 times (50%);clinicians admitted to not knowing the correct answer 258 times (16%). The clinician with the best performance, a new resident, identified 26 alarms correctly (68%);the worst performance, by a clinician with 26 years' experience, was three correct (8%).There was no correlation between a subject's clinical experience and his or her ability to correctly idenhfy alarms (Figure 1). Hearing deficits were detected in six subjects. The scores of the clinicians with hearing deficits were not different from those with normal hearing. On the questionnaire, clinicians judged the tested alarms as critical 37% of the time, important 34% of the time, and advisory 29% of the time. When clinicians rated an alarm as more important, it was less likely that they would correctly identify it (Figure 2). Alarms were rated as being frequently, infrequently, and rarely heard 30%, 30%, and 41% of the time, respectively. The more frequently clinicians believed that they heard an alarm, the more likely it was that they would identify it (Figure 3). For each clinician, there was a negative correlation between how important he or she thought the alarm was and how frequently he or she thought that it was heard. The alarms believed to be less important were judged to be heard more frequently. The alarm with the highest recognition rate (Fisher & Paykel humidifier) was correctly identified 92% of the time, whereas that with the lowest recognition rate (Ohmeda 7000 Ventilator power failure) was correctly identified only 1% of the time. Six of the alarms were continuous tones, five were intermittent tones separated by periods of silence, and eight were multitone. Sound complexity did not affect recognition rate; however, there was a positive correlation between alarm volume and recognition rate (P < 0.01) (Figure 4). Table 1 presents the frequencies of responses for each alarm sound. The correct responses are outlined on the main diagonal. Entries outside the main diagonal and the "Don't know" column are mismatches; these may have occurred owing to confusion or chance. Mismatches due to random chance would result in a cell frequency of
LOEB ET AL. ALARM RECOGNITION ACCURACY
ANESTH ANALG 1992;75:4%505
501
Table 1. Clinicians' identification of alarm sounds, grouped by sound categories. Correct responses are outlined on the main diagonal. Relevant confusions are in bold type. Shading indicates a sound pattern similarity between the device on the horizontal axis and the one on the vertical axis.
Line Isolation Monitor Bard Alfenta Pump
3
2
1
1
NeUcor Pulse Oximeter
4
4
1
1
ValleyLab Electrmautery
1
2
2
Ohmeda 7000 Ventilator (disconnect)
6
3
1
Miniox Oxygen Analyzer Ohmeda Oxygen Analyzer SARA Mass Spectrometer
5 5
3
1
8
3
1
4
1
2
I 1
4
3
2
3
1
2
I
21
88
4
2
13
88
1
4
14
88
2
7
2 8 8
Datex Airway Gas Monitor Ohmeda Volumeter (threshold)
1
3
3
Ohmeda Volumeter (apnea)
1
1
2
Ohmeda Airway Pressure
5
9
1 1 0 1
2
Ohmeda Modulus I (low oxygen pressure) 10 Ohmeda Modulus II (power failure)
1
Dinamap (technical) Dinamap (threshold)
1
Puritan Bennett Multifunction
2
1
9
6
1
1
10
11
1
2
2
1
71
32
1 1
1
6
Fir & Paykel Humidfir
2
1
60
90 123 73
1
1
91
Tntal
A
146 107
Faculty and CRNAs
99
90 104
1
25
88
3
24
88
2
18
88
2
128
80
120 258 1672
(I)
Q
incorrectly identified
0
correc(ly identified
f
600
L
a
0
8 400
10
w-
0
z
J
0
2
3
5 10 15 20 25 30 35
Years of experience Figure 1. Number of alarm sounds correctly identified (of 38) versus the clinician's years of anesthesia experience.
200
5 =
o Critical
Important
Advisory
Clinical importance of alarm
Figure 2. Number of correctly and incorrectly identified alarm sounds versus the clinician's rating of the alarm's clinical importance. There is a significant difference between groups (P < 0.Oool).
[(No. of times that each alarm sound was presented) - (No. of "Don't know" responses)] + (No. of possible responses)
or (88 - (No. of "Don't know" responses)] + 15
for each row. Cell frequencies that exceed this threshold indicate a relevant confusion (9). Interpreted in
this way, 571 responses were correct (34%),573 were relevant confusions (34%), 258 were "Don't know" (16%),and 270 were random mismatches (16%).
502
LOEB ET AL. ALARM RECOGNITION ACCURACY
ANESTH ANALG 1992;75:499-505
800 u)
P)
0 C
600 L
3
0
i;
400
r
0 L
2
s z
200
0
sgbl 75%
52%
Frequently
Infrequently
Rarely
Frequency that alarm is heard in clinical practice
Figure 3. Number of correctly and incorrectly identified alarm sounds versus how frequently the clinician hears the alarm in clinical practice. There is a significant difference between groups (P < 0.00Ol).
/
a
I
.~
'"I t
n
f
z
a
-#
201
Y
060
t
65
1
70
75
80
85
I
90
Alarm volume (dB) Figure 4. The number of times a n alarm was correctly identified versus the alarm volume. Each alarm was presented 88 times, twice each to 44 subjects. The linear regression line is significant at P < 0.01.
Discussion The data indicate that anesthesia clinicians cannot determine the source of current commercial operating room alarms solely by sound characteristics. This contradicts the subjective impression (8). The study does not imply that alarms cannot be identified quickly in the clinical setting. During clinical use, identification may be facilitated by visual indicators on the monitor, the direction from which the sound is produced, and interpretation of the clinical scenario at the time of the event. Alarm recognition in the operating room is hampered when each device has a proprietary alarm sound. This is due to a number of factors. First, numerous sounds must be recognized. We identified 19 audible alarms in the operating rooms at our institution. Second, alarms may occur simultaneously. Third, the user may rarely use a particular device and thus infrequently hear a particular alarm. Experiments by Patterson and Milroy (10) indicate that subjects can readily learn to recognize four to six audible alarms, and 1 wk after learning 10 alarms,
subjects can reliably recognize only six. This is consistent with other studies that demonstrate the limitations of humans in differentiating among multiple stimuli (11). It suggests that the number of alarm sounds should be reduced to a maximum of six. The number of sounds can be decreased in two ways: standardization of sounds and limitation of alarms. Standardized alarms could reduce the total number of sounds and would ensure that unfamiliar equipment had familiar alarms. Limiting the use of audible alarms would reduce the total number of sounds and the chance of simultaneous alarms. Eventually, integrated monitoring systems with centralized alarms would be ideal, but in the interim, the situation could be improved by standardizing audible alarms and limiting their use. It is intuitive that the most important alarms should have sounds that are most easily identified by anesthesiologists; however, we found an inverse relationship between alarm importance and recognizability. For example, the Fisher & Paykel humidifier alarm had a low importance rating but was very well recognized. This alarm is loud, continuous, and has a complex sound pattern. These features may elicit a strong affective response in the anesthetist that may increase recognition. A related study by Finley and Cohen (12) found no correlation between the perceived urgency of commercial operating room alarm sounds and the urgency of the clinical situation. Thus, there is accumulating evidence that current commercial auditory alarms are poorly correlated to the situations of which they are warning. Although our study demonstrates a problem with the current unstructured approach to alarms, it provides little data to guide manufacturers in developing more recognizable alarms. There was a positive correlation between alarm volume and recognition rate; however, we do not recommend that alarm volumes be increased indiscriminately. An anesthesiologist's initial response might be to silence a louder alarm rather than address the clinical problem that caused it. Also, louder alarms would be more disruptive to the rest of the operating room team. We presumed that the more complex alarm sounds would be better recognized, but our data do not support this conclusion. Additional research is needed to develop alarm sounds that are recognizable and detectable but not disruptive. The more urgent and recognizable sounds should be reserved for the more important alarms, and before general use, the sounds should be evaluated in operating rooms to determine their effect on personnel performance. We looked for patterns of recognition errors in our data to understand the processes involved in recognizing alarm sounds. This method has been used previously to investigate factors that affect other
LOEB ET AL. ALARM RECOGNlTlON ACCURACY
ANESTH ANALG 1992;75:499-505
503
Table 2. Alarm Sound Categories Category description Continuous Broadband
Distinct tone Broadband, siren Intermittent tone Approximate cycle time of 1 s
Approximate cycle time of 0.3 s Burst of intermittent tone separated by long pause
Greater than two tones (these two not similar to each other)
Alarm category Line Isolation Monitor Nellcor Pulse Oximeter Bard Alfenta Pump Buzzing of Electrocautery" Ohmeda 7000 Ventilator (power failure) Miniox Oxygen Analyzer Ohmeda Modulus I (low oxygen pressure) Ohmeda Oxygen Analyzer Ohmeda Volumeter (threshold) Ohmeda Volumeter (apnea) Ohmeda Airway Pressure SARA Mass Spectrometer Datex Airway Gas Monitor Ohmeda 7000 Ventilator (disconnect) ValleyLab electrocautery Dinamap (threshold) Dinamap (technical) Ohmeda Modulus I1 (power failure) Puritan Bennett Multifunction Fisher & Paykel Humidifier
UEventhough it is not an alarm, the "buzzing" sound of electrocautery was included in this category. See text for explanation.
recognition tasks (9). We were most interested in understanding the relevant confusions, because these signify systematic errors in the interpretation of the alarm sounds. Retrospective review of the data suggested that many mismatches resulted from two underlying causes: (a) sound confusion and (b) function confusion. The first, sound confusion, was a mismatch between alarms with similar sound qualities. We retrospectively classified the alarms by similarity of sound into seven categories (Table 2). Table 1 is grouped by sound categories to highlight the areas of potential confusion. An example of sound confusion is the 14 responses of "Dinamap" to the Ohmeda Modulus I1 power failure alarm. The Modulus and Dinamap alarms sound similar; both are complex sounds with a burst of patterned tones separated by long periods of silence. Another example is the 33 responses of "electrocautery" to the Nellcor pulse oximeter alarm. This alarm is a constant, broadband sound that is similar to the "buzzing" of electrocautery. There were 327 relevant confusions that we believed could be attributed, at least in part, to sound confusion. The second cause of mismatches was function confusion. This occurred when the subject attributed an alarm to another piece of equipment with a similar clinical function. Each alarm was classified as being in one or more of the folIowing function categories: gas
concentration, pressure, circuit disconnect, power loss, oxygenation, ventilation, circulation, and temperature. Table 3 displays our data grouped by function categories to highlight the areas of potential confusion. Examples are the confusions among the gas analyzers: the Miniox oxygen analyzer, the Ohmeda oxygen analyzer, the Datex airway gas monitor, and the SARA mass spectrometer. Another function confusion, which may be more difficult to appreciate, is the anesthetist's response of "airway pressure" to the Ohmeda Modulus I low-oxygen-supply pressure alarm; however, these are both pressure alarms. There were 202 relevant confusions that we believed could be attributed, at least in part, to function confusion. In this retrospective analysis of relevant confusions, 36% were due to sound confusion, 15% to function confusion, 21% to both, and 28% to neither. Function confusions suggest that clinicians mentally group alarm sounds by monitor function. This may be advantageous in the clinical environment, because attention is directed to the appropriate system. It also reinforces the position that alarm sounds should be grouped by function (13). Our results indicate that the recognition rate for the alarm's function could be readily increased to 55% (the number of correct responses plus the number of relevant confusions due to function confusion) if alarms with similar functions had similar sounds.
504
ANESTH ANALG 1992;75499-505
LOEB ET AL. ALARM RECOGNITION ACCURACY
Table 3. Clinicians' identification of alarm sounds, grouped by function categories. Correct responses are outlined on the main diagonal. Relevant confusions are in bold fype. Shading indicates a functional similarity between the devices on the vertical and horizontal axes.
Line Isolation Monitor
1
Bard Alfenta Pump
3
1
1
3
ValIeyLab Elearocautery
2
6
1
1
1
I
12
I
21
88
4
8
88
25
88
88
Ohmeda Modulus 11 (power failure) Ohmeda Modulus I (low oxygen pressure Ohmeda Volumeter (threshold)
1
3
Ohmeda Volumeter (apnea)
1
1
2 1
7
2
9
I
4
14
88
2
7
2
88
4
11
88
Ohmeda 7000 Ventilator (disconnect)
ohmeda7000 Ventilator (power failure)
18
88
Ohmeda Airway Pressure
5
5
10
88
SARA Mass Spectrometer
4
z
13
88
3
10
88
Datex Airway Gas Monitor
Miniox Oxygen Analyzer Obmeda Oxygen Analyzer
Nellcor Pulse Oximeter
Dinaroap (technical) Dinamap (threshold) PuritanBennett Multifunction
Fisher & Paykel Humidifm
Total
91
71
146
2
1
73
90
Our study has a number of potential design flaws. First, recorded alarms were used rather than the sounds from the actual machines. However, the investigators compared high-quality cassette recordings and digital recordings with the original sounds and believed that the digital system had superior playback sound quality. Even so, any sound recording and playback has the potential for distortion, and 13 of the subjects believed that the quality of the recordings interfered with their ability to recognize at least some of the alarms. Second, the 19 alarms that we recorded would not all be present during a single operative case. Although all of the devices are used at our institution, there are variations in equipment among operating rooms. For example, we tested the alarms from two differentoxygen analyzers, although only one model would be used in any single case. This artificially increased the number of possible alarms and may have interfered with the use of a situational factor in the recognition of alarms. Experienced cIinicians an-
1
107
123 104
60
99
90
32
128
80
120 258 1672
ticipate foreseeable problems better than trainees do (14).Thus, a situational factor may be used by the experienced clinician in that he or she may expect particular alarms at certain phases during the course of an operation or may not consider alarms from equipment not currently in use. There may also be other factors, such as stage of the operation or patient illness, that cue the experienced clinician to particular monitors and alarms. We conclude that current operating room alarms cannot be reliably recognized by sound characteristics alone. Some sounds are better recognized than others, and anesthetists mentally group alarm sounds by function.
References 1. Schreiber PJ, Schreiber J. Structured alarm systems for the
operating room. J Clin Monit 1989;52014. 2. Wilkins PA. Assessing the effectiveness of auditory warnings. Br J Audio1 1981;15:2&74.
ANESTH ANALG 1992;75:499-505
3. Shapiro RA, Berland T. Noise in the operating room. N Engl J Med 1972;28712364. 4. Hodge B, Thompson JF. Noise pollution in the operating theatre. Lancet 1990;335:891-4. 5. Stanford LM, McIntyre JWR, Hogan JT. Audible alarm signals
for anaesthesia monitoring equipment. Int J Clin Monit Comput 1985;1:2514. 6. Samuels SI. An alarming problem (letter). Anesthesiology 1986;M: 128. 7. Schmidt SI, Baysinger CL. Alarms: help or hindrance? (letter). Anesthesiology 1986;64:654-5. 8. Mclntyre JWR. Ergonomics: anaesthetists’ use of auditory alarms in the operating room. Int J Clin Monit Comput 1985;2:47-55.
LOEB ET AL. ALARM RECOGNITION ACCURACY
505
9. Zwaga HJ, Boersema T. Evaluation of a set of graphic symbols. Appl Ergon 1983;16:43-54.
10. Patterson RD, Milroy R. Auditory warnings on civil aircraft: the learning and retention of warnings. Final Contract Report 7D/S/0142, MRC Applied Psychology Unit, Cambridge, 1980. 11. Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 1956;6381-97. 12. Finley GA, Cohen A]. Perceived urgency and the anaesthetist: responses to common operating room monitor alarms. Can J Anaesth 1991;38:958-64. 13. Kerr JH. Warning devices. Br J Anaesth 1985;57696-708. 14. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg 1991;72:30%15.
