The Laryngoscope C 2014 The American Laryngological, V
Rhinological and Otological Society, Inc.
Reliability and Validity Testing of Automated Scoring in Obstructive Sleep Apnea Diagnosis With the Embletta X100 Do Yang Park, MD; Hyun Jun Kim, MD, PhD; Chang-Hoon Kim, MD, PhD; Yoo Suk Kim, MD; Ji Ho Choi, MD, PhD; Sang Young Hong, MD; Jin Ji Jung, RPSGT; Kang Il Lee, RPSGT; Han Sang Lee, RPSGT Objectives/Hypothesis: To verify the reliability and validity of automated scoring and compare it to that of manual scoring for diagnosing obstructive sleep apnea using an Embletta X100 level 2 portable device. Study Design: Retrospective study. Methods: A total of 116 patients with suspected obstructive sleep apnea who had successfully received portable polysomnography with the Embletta X100 were examined. All polysomnography data were analyzed by automated and manual methods. Manual scoring was performed according to the revised American Academy of Sleep Medicine 2012 criteria. Automated scoring was analyzed using the automatic algorithm, which was updated with the American Academy of Sleep Medicine 2012 criteria. All parameters were evaluated statistically using correlation analysis and paired t tests. Results: The apnea-hypopnea index for automated scoring and manual scoring with the Embletta X100 were moderately correlated (r 5 0.76, P 15 is 11.4% in men and 4.7% in women.1,2 OSA can lead to daytime sleepiness, fatigue, memory problems, and cognitive dysfunction. Several studies suggest a connection between OSA and adverse critical cardiorespiratory disease.3–5 Notably, approximately 80% of men and 93% of women remain undiag-
From the Department of Otorhinolaryngology (D.Y.P., C.H.K.), Yonsei University College of Medicine, Seoul, Korea; Department of Otolaryngology (H.J.K., Y.S.K., S.Y.H., J.J.J., K.I.L., H.S.L.), Ajou University School of Medicine, Suwon, Korea; Department of Otorhinolaryngology–Head and Neck Surgery (J.H.C.), Korea University College of Medicine, Seoul, Korea. Editor’s Note: This Manuscript was accepted for publication July 21, 2014. This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (contract grant number: NRF-2012R1A5A2048183). The authors have no other funding, financial relationships, or conflicts of interest to disclose. Send correspondence to Hyun Jun Kim, MD, Department of Otolaryngology, Ajou University School of Medicine, 164 Worldcup Street, Wonchon-Dong, Yeongtong-Gu, Suwon, Korea. E-mail: [email protected] DOI: 10.1002/lary.24878
Laryngoscope 00: Month 2014
nosed.6 In-laboratory polysomnography (PSG) is the gold standard to diagnose OSA and its severity. However, its limitations (time consuming, expensive, long waiting periods, and unfit for specific situations) have become an obstacle for patients with suspected OSA. Therefore, variable automated portable sleep monitoring devices and automated analyzing systems have been developed and widely used as an alternative to laboratory PSG.7 Portable PSG and automated scoring provide an easy approach for OSA diagnosis. The Embletta X100 (Natus Medical Inc., San Carlos, CA) is an unattended 11-channel portable PSG device (level 2) with an automated scoring system. Level 2 portable monitoring devices provide full sleep monitoring, similar to laboratory PSG, and enable home-based testing without a technician, lowering the cost, decreasing time, eliminating the waiting period, and increasing adaptability to specific situations such as diagnosing postoperative patients. In several studies, portable monitoring devices have been validated against laboratory PSG. However, in most of these studies, PSG data with the portable monitoring devices were scored manually. The routine use of automated scoring clinically is practical because of the limited availability of sleep specialists. However, this more convenient method requires a validation study. Therefore, the present study assesses the reliability and validity of automated scoring with the Park et al.: Automated Scoring With the Embletta X100
1
TABLE I. Comparison Between Automated and Manual Scoring of Polysomnography Data. Pearson Correlation Coefficient
Mean 6 SD and Difference (Auto Scoring/Manual Scoring)
TRT Sleep period
Auto Scoring
Manual Scoring
471.3 6 65.5 418.4 6 101.3
503.2 6 350.2 461.6 6 68.7
Difference
231.9 6 348.3 243.2 6 86.2
P
r
P
.319 .000
0.122 0.543*
.188 .000
WASO
199.5 6 111.2
55.6 6 45.5
143.9 6 106.5
.000
0.306*
.001
TST Sleep onset
220.5 6 135.3 42.8 6 61.8
403.4 6 81.4 9.6 6 17.8
2182.9 6 120.8 33.1 6 62.4
.000 .000
0.470* 0.112
.000 .227
Sleep efficiency (%)
46.3 6 26.8
239.8 6 25.6
.000
0.306*
.001
REM AHI
85.6 6 103.9 7.2 6 12.1
122.9 6 209.4 18 6 17
86 6 10.2
237.3 6 228.7 210.8 6 11.1
.099 .000
0.054 0.761*
.589 .000
RDI
7.2 6 12.1
23.7 6 16.9
216.5 6 11.6
.000
0.725*
.000
Obstructive index Central index
3.6 6 9.2 0.1 6 0.6
7.7 6 12.1 0.4 6 0.7
24.1 6 7.4 20.3 6 0.9
.000 .000
0.791* 0.152
.000 .101
Mixed index
0.2 6 1.2
0.8 6 2.1
20.5 6 1.4
.000
0.755*
.000
Hypopnea index ODI
3.2 6 4.8 7.6 6 12.5
9.1 6 8 16.8 6 16.4
25.9 6 7.2 29.2 6 10.2
.000 .000
0.451* 0.785*
.000 .000
Relative snoring time, %
0.1 6 0.6
22 6 20.1
222 6 20.1
.000
0.053
.570
Respiratory arousals Spontaneous arousals
0.1 6 1.2 0 6 0.3
10 6 12.5 3.5 6 2.7
29.8 6 12.6 23.5 6 2.7
.000 .000
0.005 0.055
.958 .555
RERA Total arousals Mean oxygen saturation, %
0.1 6 0.5
5.2 6 3.8
25.1 6 3.9
.000
0.014
.883
0.2 6 1.7 95.7 6 2
18.6 6 12 95.4 6 3.6
218.4 6 12.1 0.3 6 3
.000 .32
0.031 0.529*
.738 .000
Lowest oxygen saturation, %
87.3 6 6.5
84 6 10.3
Supine AHI
12.3 6 19
27.1 6 23.5
3.3 6 8.3 214.8 6 16.2
.000
0.590*
.000
.000
0.728*
.000
*Statistical significance
Rhinological and Otological Society, Inc.
Reliability and Validity Testing of Automated Scoring in Obstructive Sleep Apnea Diagnosis With the Embletta X100 Do Yang Park, MD; Hyun Jun Kim, MD, PhD; Chang-Hoon Kim, MD, PhD; Yoo Suk Kim, MD; Ji Ho Choi, MD, PhD; Sang Young Hong, MD; Jin Ji Jung, RPSGT; Kang Il Lee, RPSGT; Han Sang Lee, RPSGT Objectives/Hypothesis: To verify the reliability and validity of automated scoring and compare it to that of manual scoring for diagnosing obstructive sleep apnea using an Embletta X100 level 2 portable device. Study Design: Retrospective study. Methods: A total of 116 patients with suspected obstructive sleep apnea who had successfully received portable polysomnography with the Embletta X100 were examined. All polysomnography data were analyzed by automated and manual methods. Manual scoring was performed according to the revised American Academy of Sleep Medicine 2012 criteria. Automated scoring was analyzed using the automatic algorithm, which was updated with the American Academy of Sleep Medicine 2012 criteria. All parameters were evaluated statistically using correlation analysis and paired t tests. Results: The apnea-hypopnea index for automated scoring and manual scoring with the Embletta X100 were moderately correlated (r 5 0.76, P 15 is 11.4% in men and 4.7% in women.1,2 OSA can lead to daytime sleepiness, fatigue, memory problems, and cognitive dysfunction. Several studies suggest a connection between OSA and adverse critical cardiorespiratory disease.3–5 Notably, approximately 80% of men and 93% of women remain undiag-
From the Department of Otorhinolaryngology (D.Y.P., C.H.K.), Yonsei University College of Medicine, Seoul, Korea; Department of Otolaryngology (H.J.K., Y.S.K., S.Y.H., J.J.J., K.I.L., H.S.L.), Ajou University School of Medicine, Suwon, Korea; Department of Otorhinolaryngology–Head and Neck Surgery (J.H.C.), Korea University College of Medicine, Seoul, Korea. Editor’s Note: This Manuscript was accepted for publication July 21, 2014. This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (contract grant number: NRF-2012R1A5A2048183). The authors have no other funding, financial relationships, or conflicts of interest to disclose. Send correspondence to Hyun Jun Kim, MD, Department of Otolaryngology, Ajou University School of Medicine, 164 Worldcup Street, Wonchon-Dong, Yeongtong-Gu, Suwon, Korea. E-mail: [email protected] DOI: 10.1002/lary.24878
Laryngoscope 00: Month 2014
nosed.6 In-laboratory polysomnography (PSG) is the gold standard to diagnose OSA and its severity. However, its limitations (time consuming, expensive, long waiting periods, and unfit for specific situations) have become an obstacle for patients with suspected OSA. Therefore, variable automated portable sleep monitoring devices and automated analyzing systems have been developed and widely used as an alternative to laboratory PSG.7 Portable PSG and automated scoring provide an easy approach for OSA diagnosis. The Embletta X100 (Natus Medical Inc., San Carlos, CA) is an unattended 11-channel portable PSG device (level 2) with an automated scoring system. Level 2 portable monitoring devices provide full sleep monitoring, similar to laboratory PSG, and enable home-based testing without a technician, lowering the cost, decreasing time, eliminating the waiting period, and increasing adaptability to specific situations such as diagnosing postoperative patients. In several studies, portable monitoring devices have been validated against laboratory PSG. However, in most of these studies, PSG data with the portable monitoring devices were scored manually. The routine use of automated scoring clinically is practical because of the limited availability of sleep specialists. However, this more convenient method requires a validation study. Therefore, the present study assesses the reliability and validity of automated scoring with the Park et al.: Automated Scoring With the Embletta X100
1
TABLE I. Comparison Between Automated and Manual Scoring of Polysomnography Data. Pearson Correlation Coefficient
Mean 6 SD and Difference (Auto Scoring/Manual Scoring)
TRT Sleep period
Auto Scoring
Manual Scoring
471.3 6 65.5 418.4 6 101.3
503.2 6 350.2 461.6 6 68.7
Difference
231.9 6 348.3 243.2 6 86.2
P
r
P
.319 .000
0.122 0.543*
.188 .000
WASO
199.5 6 111.2
55.6 6 45.5
143.9 6 106.5
.000
0.306*
.001
TST Sleep onset
220.5 6 135.3 42.8 6 61.8
403.4 6 81.4 9.6 6 17.8
2182.9 6 120.8 33.1 6 62.4
.000 .000
0.470* 0.112
.000 .227
Sleep efficiency (%)
46.3 6 26.8
239.8 6 25.6
.000
0.306*
.001
REM AHI
85.6 6 103.9 7.2 6 12.1
122.9 6 209.4 18 6 17
86 6 10.2
237.3 6 228.7 210.8 6 11.1
.099 .000
0.054 0.761*
.589 .000
RDI
7.2 6 12.1
23.7 6 16.9
216.5 6 11.6
.000
0.725*
.000
Obstructive index Central index
3.6 6 9.2 0.1 6 0.6
7.7 6 12.1 0.4 6 0.7
24.1 6 7.4 20.3 6 0.9
.000 .000
0.791* 0.152
.000 .101
Mixed index
0.2 6 1.2
0.8 6 2.1
20.5 6 1.4
.000
0.755*
.000
Hypopnea index ODI
3.2 6 4.8 7.6 6 12.5
9.1 6 8 16.8 6 16.4
25.9 6 7.2 29.2 6 10.2
.000 .000
0.451* 0.785*
.000 .000
Relative snoring time, %
0.1 6 0.6
22 6 20.1
222 6 20.1
.000
0.053
.570
Respiratory arousals Spontaneous arousals
0.1 6 1.2 0 6 0.3
10 6 12.5 3.5 6 2.7
29.8 6 12.6 23.5 6 2.7
.000 .000
0.005 0.055
.958 .555
RERA Total arousals Mean oxygen saturation, %
0.1 6 0.5
5.2 6 3.8
25.1 6 3.9
.000
0.014
.883
0.2 6 1.7 95.7 6 2
18.6 6 12 95.4 6 3.6
218.4 6 12.1 0.3 6 3
.000 .32
0.031 0.529*
.738 .000
Lowest oxygen saturation, %
87.3 6 6.5
84 6 10.3
Supine AHI
12.3 6 19
27.1 6 23.5
3.3 6 8.3 214.8 6 16.2
.000
0.590*
.000
.000
0.728*
.000
*Statistical significance
