pii: sp-00800-13
http://dx.doi.org/10.5665/sleep.4076
VALIDATION OF SOFTWARE FOR THE DETECTION OF REM SLEEP BEHAVIOR DISORDER
Validation of an Integrated Software for the Detection of Rapid Eye Movement Sleep Behavior Disorder Birgit Frauscher, MD; David Gabelia, MD; Marlene Biermayr; Ambra Stefani, MD; Heinz Hackner; Thomas Mitterling, MD; Werner Poewe, MD; Birgit Högl, MD Department of Neurology, Innsbruck Medical University, Innsbruck, Austria
Study Objectives and Design: Rapid eye movement sleep without atonia (RWA) is the polysomnographic hallmark of REM sleep behavior disorder (RBD). To partially overcome the disadvantages of manual RWA scoring, which is time consuming but essential for the accurate diagnosis of RBD, we aimed to validate software specifically developed and integrated with polysomnography for RWA detection against the gold standard of manual RWA quantification. Setting: Academic referral center sleep laboratory. Participants: Polysomnographic recordings of 20 patients with RBD and 60 healthy volunteers were analyzed. Interventions: N/A. Measurements and Results: Motor activity during REM sleep was quantified manually and computer assisted (with and without artifact detection) according to Sleep Innsbruck Barcelona (SINBAR) criteria for the mentalis (“any,” phasic, tonic electromyographic [EMG] activity) and the flexor digitorum superficialis (FDS) muscle (phasic EMG activity). Computer-derived indices (with and without artifact correction) for “any,” phasic, tonic mentalis EMG activity, phasic FDS EMG activity, and the SINBAR index (“any” mentalis + phasic FDS) correlated well with the manually derived indices (all Spearman rhos 0.66–0.98). In contrast with computerized scoring alone, computerized scoring plus manual artifact correction (median duration 5.4 min) led to a significant reduction of false positives for “any” mentalis (40%), phasic mentalis (40.6%), and the SINBAR index (41.2%). Quantification of tonic mentalis and phasic FDS EMG activity was not influenced by artifact correction. Conclusion: The computer algorithm used here appears to be a promising tool for REM sleep behavior disorder detection in both research and clinical routine. A short check for plausibility of automatic detection should be a basic prerequisite for this and all other available computer algorithms. Keywords: computer algorithm, detection, polysomnography, REM sleep behavior disorder, scoring Citation: Frauscher B, Gabelia D, Biermayr M, Stefani A, Hackner H, Mitterling T, Poewe W, Högl B. Validation of an Integrated Software for the Detection of Rapid Eye Movement Sleep Behavior Disorder. SLEEP 2014;37(10):1663-1671.
INTRODUCTION Rapid eye movement sleep without atonia (RWA) is the polysomnographic (PSG) hallmark of REM sleep behavior disorder (RBD), and current International Classification of Sleep Disorders criteria thus require PSG evaluation for a definite diagnosis of RBD.1 An attempt to diagnose RBD based on history or questionnaires alone may result in false-positive diagnoses because of the challenging differential diagnostic spectrum.2 An accurate diagnosis of RBD has gained further importance because of the eventual development of neurodegenerative diseases such as Parkinson disease, Lewy body dementia, or multiple system atrophy. Long-term follow-up investigations of idiopathic RBD cohorts have shown that neurodegenerative disease developed in up to 80% of subjects, with a mean delay between RBD onset and the respective neurodegenerative condition of approximately 15 y.3–5 Assessment of RWA is observer based and requires manual quantitative scoring of a subject’s PSG recording.6,7 Only recently, cutoff values for the PSG diagnosis of RBD have been introduced.8,9 Although manual quantification remains the gold standard for RWA scoring, the procedure is time intensive and
difficult to use in clinical routine. To overcome this major limitation, computer-assisted algorithms for the detection of RWA have been developed over the past several years,10–14 but none of these software solutions has been integrated in existing PSG systems. This, however, would be essential for clinical routine. In addition, software solutions focused only on the chin, but not the limbs which was shown to be of importance for RBD evaluation.9,15,16 Moreover, validation against the gold standard of manual RWA quantification is lacking for some algorithms.14 The aim of the current study was to validate an individually tailored and now commercially available computer algorithm for the detection of different types of RWA activity measures, including the Sleep Innsbruck Barcelona (SINBAR) criteria against the gold standard of manual RWA quantification. To emphasize the importance of artifact correction for any automatic detector, we provided both the uncorrected and the manually artifact corrected validation results. The assessment of normative values for REM-related electromyographic (EMG) activity for the differentiation between patients with RBD and controls was not within the scope of the current study. METHODS
Submitted for publication December, 2013 Submitted in final revised form March, 2014 Accepted for publication April, 2014 Address correspondence to: Birgit Högl, MD, Innsbruck Medical University, Department of Neurology, Anichstrasse 35, A-6020 Innsbruck, Austria: Tel: +43 512 504-23811; Fax: +43 512 504-23842; E-mail: [email protected] SLEEP, Vol. 37, No. 10, 2014
Selection of Study Participants All patients with either idiopathic or symptomatic RBD caused by neurodegenerative disease as well as all healthy volunteers who underwent PSG at the sleep center of the Department of Neurology of Innsbruck Medical University between September 2011 and July 2013 were screened for this study. 1663
Validation of an Integrated Software for RBD Detection—Frauscher et al.
RBD was diagnosed according to International Classification of Sleep Disorders 2 criteria1 based on (1) a history suggestive of RBD, and (2) a qualitative finding of RWA in the video-PSG. All healthy volunteers were selected from the Innsbruck sleep laboratory normative sample.17 This sample was recruited from an existing representative population sample after undergoing a two-step screening process (telephone interview, personal expert interview including a complete neurological examination). Approximately four subjects had to be screened to identify one willing and eligible subject fulfilling the strict inclusion and exclusion criteria. In brief, any relevant sleep disorder, any neurological or psychiatric disease, presence of internal diseases such as pulmonary, renal, cardiac, and liver disease, diabetes mellitus, a body mass index higher than 30 kg/ m2, pregnancy, regular alcohol consumption of three or more glasses of wine or three bottles of beer per day, use of pain medication more than three times per week, or use of central nervous system active medication including hypnotics during the year prior to study participation were exclusion criteria. Because the goal of the project for which we originally recruited these healthy sleepers was the investigation of motor activity in physiological sleep, careful attention was paid to the presence of RBD or other movement disorders during sleep, which was assessed by expert interview. For further details see the full paper on motor activity in healthy sleep published by Frauscher et al.17 None of the healthy volunteers had a history suggestive for RBD, and the thorough neurological investigation that was performed revealed no signs of parkinsonism in any of the healthy volunteers. The cutoff date of September 2011 was selected, as in September 2011 the Brain RT PSG system of the company OSG (address: Bussestraat 17, 2840 Rumst, Belgium. Contact person: Sabine Wuytens; website: http://www.osg.be) was introduced at the Innsbruck sleep laboratory, which feature of computerized detection of RWA was validated in the current study. This computer algorithm was individually tailored by the authors together with the software development department of the OSG company in several sessions for the detection of RWA based on the previous works of the SINBAR group.9,15,16 This algorithm is now commercially available, and integrated in the Brain RT PSG system of OSG. This software can perform the same computer algorithm on signals from other PSG devices in EDF, EDF+, Embla file format (titanium), and Brainlab format. Exclusion criteria were an apnea-hypopnea index higher than 10/h, use of any psychiatric or neurological medication for patients with idiopathic RBD and healthy volunteers, or use of melatonin or clonazepam for RBD-specific treatment. For symptomatic patients with RBD, the exclusion criterion was use of clonazepam or melatonin for RBD-specific treatment. All subjects underwent 1 night of 8-h video-PSG according to American Academy of Sleep Medicine (AASM) 2007 standards.18 This study was approved by the local ethical committee of Innsbruck Medical University. All participants granted written informed consent prior to study participation.
(vertical and horizontal eye movements), electromyography (EMG) (mental, both flexor digitorum superficialis muscles (FDS), both anterior tibialis muscles), and cardiorespiratory recording [single channel electrocardiography, nasal airflow (thermocouple), pneumoflow, tracheal microphone, thoracic and abdominal respiratory movements (piezo), transcutaneous oxygen saturation]. Sleep was scored according to the 2007 AASM criteria.18 For scoring of EMG activity, bipolar surface EMG was recorded with the low frequency filter at 50 Hz, the high frequency filter at 300 Hz, and a sampling rate of 1000 Hz. Amplification was set at 5 µV per mm for scoring of REMrelated EMG activity. Impedance of surface EMG electrodes had to be lower than 10 kΩ.
Video-PSG Video-PSG included electroencephalography (F3, F4, C3, C4, O1, O2, M1, and M2 electrodes), electrooculography
The computerized software algorithm was developed by OSG in cooperation with the authors to quantify EMG activity reflecting the SINBAR criteria.9 It detects the rate of “any,”
SLEEP, Vol. 37, No. 10, 2014
Description of Manual and Computer-Assisted EMG Analysis with and without Manual Artifact Correction Manual Quantification of REM Sleep Related EMG Activity (“Any,” Phasic, Tonic )
Before beginning the analysis of EMG activity, REM sleep was carefully examined for artifacts (e.g., snoring, background noise) and transitory increases in EMG tone caused by arousals. All artifacts and transitory increases in EMG tone caused by arousals were excluded from the quantitative scoring of REM sleep related EMG activity. Quantification of “any,” phasic and tonic EMG activity was performed manually during REM sleep according to SINBAR criteria by a single scorer.9 “Any” and tonic EMG activity were assessed for the mentalis muscle only, whereas phasic EMG activity was assessed for both the mentalis and the FDS.9 For scoring “any” and phasic EMG activity, the 30-sec epoch was divided into 10 3-sec miniepochs. “Any” EMG activity was defined as the presence of any EMG activity with an amplitude more than twice the background EMG and a duration of 0.1 sec or greater irrespective of its duration. Phasic EMG activity was defined as any burst of EMG activity lasting between 0.1 and 5.0 sec with an amplitude exceeding twice the background EMG activity irrespective of its morphology. Phasic EMG activity was scored for the chin and the FDS. Tonic EMG activity was scored for the mentalis muscle only in 30-sec epochs, and is defined as a more than 50% increase of sustained EMG activity of the total epoch with an amplitude of at least twice the background EMG muscle tone or more than 10 µV. We calculated the “any” EMG activity index, the phasic EMG activity index, and the tonic EMG activity index in the mentalis muscle, the phasic EMG activity index for the FDS as well as the SINBAR EMG activity index for the combination of “any” EMG activity in the mentalis and phasic EMG activity in the FDS. The “any,” phasic, and SINBAR EMG activity indices are the percentages of 3-sec miniepochs containing “any” or phasic EMG activity divided by all evaluated miniepochs. The tonic EMG index is the percentage of 30-sec epochs, which has tonic EMG activity. Computer-Assisted Scoring Algorithm for RWA Detection (“Any,” Phasic, Tonic ) – without Artifact Correction
1664
Validation of an Integrated Software for RBD Detection—Frauscher et al.
phasic, or tonic EMG activity in the mentalis muscles and phasic EMG activity in all other Table 1—Polysomnographic characteristics of the study samples. extremity muscles. According to a recent Sleep variables RBD sample Healthy volunteer sample P value recommendation,9 the rate of phasic EMG Time in bed, min 484.4 ± 24.3 481.1 ± 9.8 0.672 activity in the FDS was reported. Derived inSPT, min 464.1 ± 29.8 455.4 ± 25.8 0.633 dices are the “any,” phasic, and tonic mentalis Sleep efficiency, % SPT 80.1 ± 10.8 79.6 ± 11.6 0.991 EMG activity index, the phasic FDS EMG Wake after sleep onset, min 74.2 ± 44.9 72.0 ± 46.2 0.735 activity index, as well as the SINBAR EMG Sleep stage, % SPT activity index. 1 11.0 ± 4.7 10.3 ± 4.0 0.601 For phasic EMG activity, a default setting 2 45.8 ± 11.2 44.6 ± 9.2 0.846 with a duration between 0.1 and 5 sec and an 3 11.2 ± 4.8 16.8 ± 7.9 0.001 amplitude exceeding twice the background REM 15.8 ± 5.9 12.2 ± 4.6 0.020 EMG was chosen. The background EMG was Sleep onset latency, min 14.6 ± 15.3 22.6 ± 21.8 0.039 adaptively calculated over 15 sec. The onset REM sleep latency, min 102.7 ± 47.6 143.3 ± 74.0 0.014 criterion is selected with twice the background EMG or 2 µV, whichever is higher; Apnea-hypopnea index, /h 2.9 ± 2.3 3.1 ± 4.2 0.504 the offset criterion is selected when the amREM periods, n 3.5 ± 0.6 3.3 ± 1.0 0.657 plitude falls below 10% of the average EMG 3-sec REM miniepochs, n 1426.2 ± 510.7 975.3 ± 392.3 < 0.001 signal amplitude or below 2 µV, whichever is higher. For example, if the average ampliRBD, REM sleep behavior disorder; REM, rapid eye movement; SPT, sleep period time. tude of the EMG activity is 50 µV, the offset amplitude is when the EMG amplitude drops below 5 µV (10% of the average amplitude). When the average RESULTS amplitude is 15 µV, then the offset amplitude is 2 µV (2 µV is the minimum EMG amplitude). The intermovement distance is Demographic, Clinical, and Sleep Characteristics of the Patients given with 0.1 sec. For “any” EMG activity, the duration was with RBD and the Healthy Volunteer Sample increased to 15 sec. In line with the manual quantification of PSG registrations of 20 patients with RBD (18 men, 2 REM sleep, 3-sec miniepochs were evaluated. women) with a mean age of 65.1 ± 11.6 y were analyzed. RBD The tonic activity of the mentalis muscle was evaluated in etiology was idiopathic in 10 cases (9 men, 1 woman) and 1-sec timeframes over the entire night and divided into four symptomatic in 10 cases (9 men, 1 woman) because of neucategories ranging from low level to high level EMG activity. rodegenerative disease. Five of the 10 subjects had probable Tonic EMG activity is detected when at least 50% of the reParkinson disease, three probable multiple system atrophy, one spective 30-sec epoch contains a background tone higher than probable Lewy body dementia, and one patient had suspected the range of the low level background category augmented neurodegenerative disease that was not further specified. The by 1 µV. median Hoehn and Yahr stage in “on” was 2.5 (range 2–4). Idiopathic RBD subjects had no neurological or psychiatric medication, whereas in the symptomatic RBD group, five patients Computer-Assisted Scoring Algorithm for RWA Detection ( “Any,” Phasic, Tonic ) – with Artifact Correction were on antiparkinsonian medication: three patients were on 500–700 mg levodopa/day, one of these three patients had a For this analysis, the aforementioned software default settings were used and a manual artifact correction was performed combination with 8 mg rotigotine TTS and 300 mg amantadine, two patients were on pramipexole (1.05 and 2.1 mg/day), after running the computerized quantification by a trained scorer and three patients were on 1 mg rasagiline. Two of the 10 subnot involved in the manual EMG scoring process. Manual arjects were treated with the selective serotonin reuptake inhibtifact correction consisted of the exclusion of false positive epitor sertraline. One patient was taking the atypical neuroleptic ochs because of snoring artifacts, electrocardiographic artifacts, olanzapine. or EMG activity in the context of arousals. The healthy volunteer sample consisted of 18 men and 42 women with a mean age of 50.9 ± 13.8 y. None of these subjects Statistics had any central nervous system active medication. Table 1 proStatistics were calculated with SPSS 19.0 for Windows. vides the PSG characteristics of the two study samples. For this All data were tested for normal distribution using the Shaanalysis, a total of 87,042 REM sleep 3-sec miniepochs (mean piro-Wilks test. Because data were not normally distributed, number of analyzed REM sleep 3-sec miniepochs per subject: nonparametric statistics were applied. Spearman correlation 1088 ± 465.1) were quantified. coefficients were assessed to investigate correlations between manual and computerized quantification of REM-related EMG Manual Quantification of REM-Related EMG Activity activity. To follow the methodology of the previous SINBAR 9,15,16 Table 2 provides the mean values of all analyzed EMG acpapers, EMG activity variables are reported as means and tivity measures (“any”, phasic, tonic) obtained by manual EMG standard deviations. Sensitivities and specificities were calcuquantification according to SINBAR EMG criteria for the menlated using JAVA applets provided by the University of Münster. talis muscle, the FDS, and the combined SINBAR EMG meaA P value less than 0.05 was considered to indicate statistical sure. As expected, mean values for all evaluated EMG activity significance. SLEEP, Vol. 37, No. 10, 2014
1665
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Table 2—Comparison of manual and computer-based quantification of rapid eye movement-related electromyographic activity. Computer-assisted quantification without artifact correction
Manual quantification Any mentalis EMG activity index Phasic mentalis EMG activity index Tonic mentalis EMG activity index Phasic FDS EMG activity index SINBAR EMG activity index
RBD cases 55.7 ± 25.8 41.7 ± 15.6 28.3 ± 30.5 44.0 ± 18.1 68.9 ± 22.5
Healthy controls 14.5 ± 6.1 14.0 ± 6.0 0.8 ± 1.1 11.1 ± 8.5 23.6 ± 10.8
RBD cases 56.2 ± 24.1 32.8 ± 15.3 30.4 ± 28.9 45.6 ± 17.7 71.4 ± 21.1
Healthy controls 19.1 ± 6.7 16.9 ± 6.1 1.1 ± 2.4 12.2 ± 8.6 28.4 ± 10.6
Computer-assisted quantification with artifact correction RBD cases 53.8 ± 24.3 30.2 ± 11.9 30.3 ± 28.7 42.4 ± 15.3 68.1 ± 20.6
Healthy controls 15.8 ± 6.7 14.0 ± 5.6 1.1 ± 2.4 10.5 ± 7.9 24.3 ± 10.9
Indices are expressed as means ± standard deviations. EMG, electromyographic; FDS, flexor digitorum superficialis; RBD, REM sleep behavior disorder.
Table 3—Diagnostic test indicators of manual versus computer-assisted electromyographic quantification. Manual quantification
Computer-assisted quantification without artifact correction
Computer-assisted quantification with artifact correction
Any mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.85 (0.62-0.97) 0.73 (0.60-0.84)
0.90 (0.68-0.99) 0.45 (0.32-0.58)
0.90 (0.68-0.99) 0.67 (0.63-0.78)
Phasic mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.90 (0.68-0.99) 0.68 (0.55-0.79)
0.90 (0.68-0.99) 0.47 (0.33-0.60)
0.90 (0.68-0.99) 0.68 (0.55-0.80)
Tonic mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.64 (0.38-0.82) 1.00 (0.92-1.00)
0.75 (0.51-0.91) 0.98 (0.91-1.00)
0.75 (0.51-0.91) 0.98 (0.91-1.00)
Phasic FDS Sensitivity, (95% CI) Specificity, (95% CI)
0.94 (0.73-1.00) 0.87 (0.75-0.94)
0.95 (0.72-1.00) 0.81 (0.69-0.90)
0.95 (0.73-1.00) 0.83 (0.71-0.92)
SINBAR Sensitivity, (95% CI) Specificity, (95% CI)
0.83 (0.59-0.96) 0.87 (0.75-0.94)
0.94 (0.73-0.10) 0.72 (0.59-0.83)
0.89 (0.65-0.98) 0.83 (0.71-0.92)
CI, confidence interval; FDS, flexor digitorum superficialis.
measures were much higher in RBD patients compared to controls, although there was some overlap between cases and controls. Ninety-five percent of RBD subjects exceeded at least in one EMG activity measure the recently recommended cutoff levels of the SINBAR group9: 85% of subjects exceeded the 18% cutoff value for “any” mentalis EMG activity, 90% the 16% cutoff value for phasic mentalis EMG activity, 65% the tonic mentalis EMG cutoff index of 9.6%, 95% the phasic FDS EMG cutoff index of 17%, and 89% scored positive on the SINBAR EMG montage with a cutoff value of 32%. In contrast, 33% of the healthy volunteers scored positive on at least one of the EMG activity measures: 27% scored positive for “any” mentalis EMG activity, 32% for phasic mentalis EMG activity, 0% for tonic mentalis EMG activity, 13.3% for phasic FDS EMG activity, and 13.3% for the SINBAR EMG montage. The respective sensitivities and specificities are provided in Table 3.
computerized quantification were very similar for “any” mentalis EMG activity, tonic mentalis EMG activity, phasic FDS EMG activity, and the SINBAR EMG montage, whereas phasic mentalis EMG activity as assessed by the computer algorithm was lower for phasic mentalis EMG activity in patients but not controls. Table 3 provides the sensitivities and specificities for the computerized quantification. The measures tonic mentalis EMG activity and phasic FDS EMG activity were very similar for both manual and computer-assisted scoring, whereas for all other measures (“any” mentalis, phasic mentalis, SINBAR) the sensitivity was comparatively similar, but specificity was reduced with the computer-assisted approach alone. Nevertheless, all correlation coefficients were between 0.66 and 0.98 (Figure 1a through 1e). Figure 2a through 2e show that the rate of false positives in the healthy volunteer sample increases when computerized quantification was performed.
Comparison of Manual Versus Computer-Assisted Quantification of REM-Related EMG Activity Table 2 provides mean values of the computer-assisted quantified EMG activity measures for RBD patients and healthy volunteers. Of note, mean indices gained by manual analysis and
Further Improvement of the Computerized Quantification of REM-Related EMG Activity by Manual Artifact Correction To further improve the results of the computerized quantification of REM-related EMG activity, manual artifact correction was performed by a trained PSG scorer (DG). Manual
SLEEP, Vol. 37, No. 10, 2014
1666
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Figure 1—Comparison between manual versus computer-assisted quantification of different electromyographic (EMG) activity measures in the mentalis and flexor digitorum superficialis muscles. (a) “Any” EMG activity in the mentalis muscle. (b) Phasic EMG activity in the mentalis muscle. (c) Tonic EMG activity in the mentalis muscle. (d) Phasic EMG activity in the flexor digitorum superficialis muscles. (e) SINBAR EMG activity index (combined “any” EMG activity in the chin and phasic EMG activity in the flexor digitorum superficialis muscles). Gray dots represent healthy volunteers, black dots patients with REM sleep behavior disorder. SLEEP, Vol. 37, No. 10, 2014
1667
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Figure 2—Individual data for different electromyographic (EMG) activity measures in the mentalis or flexor digitorum superficialis muscles as assessed by manual versus computer-assisted EMG quantification. (a) “Any” EMG activity in the mentalis muscle. (b) Phasic EMG activity in the mentalis muscle. (c) Tonic EMG act ivity in the mentalis muscle. (d) Phasic EMG activity in the flexor digitorum superficialis muscles. (e) SINBAR EMG activity index (combined “any” EMG activity in the chin and phasic EMG activity in the flexor digitorum superficialis muscles). SLEEP, Vol. 37, No. 10, 2014
1668
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Table 4—Overview of false positive cases as assessed by manual versus computer-assisted quantification of different electromyographic activity measures.
Any mentalis EMG activity index Phasic mentalis EMG activity index Tonic mentalis EMG activity index Phasic FDS EMG activity index SINBAR EMG activity index
Manual quantification 16 (26.7 %) 19 (31.7 %) 0 (0 %) 8 (13.3 %) 8 (13.3 %)
Computer-assisted quantification without artifact correction 33 (50 %) 32 (53.3 %) 1 (1.7 %) 11 (18.3 %) 17 (28.3 %)
Computer-assisted quantification with artifact correction 20 (33.3 %) 19 (31.7 %) 1 (1.7 %) 10 (16.7 %) 10 (16.7 %)
EMG, electromyographic; FDS, flexor digitorum superficialis.
artifact correction took a median of 5.4 min (1.4–20). Mean EMG activity indices were similar to those of computerized scoring alone (Table 2). In line with this finding, correlation coefficients were also only slightly higher than with computerized quantification alone (all Spearman rhos: 0.70–0.98, see Figure 1a through 1e). The most striking difference was that manual artifact correction led to an improvement of specificities compared to computerized scoring alone (see Table 3). Moreover, computerized scoring plus manual artifact correction led to a significant reduction of false positives for “any” mentalis (40%), phasic mentalis (40.6%), and the SINBAR index (41.2%). Tonic mentalis EMG activity and phasic FDS EMG activity was not improved by artifact correction (Table 4). DISCUSSION The current study validated a fully integrated and now commercially available software for the detection of RWA, which was developed in cooperation with the authors. This software is the first algorithm which quantifies different types of EMG activity (“any”, phasic, tonic) according to SINBAR criteria not only in the mentalis but also in the extremity muscles. The correlation between the computerized and the standard manual EMG quantification was demonstrated to be high. In addition, this is the first study providing the results of the automatic detection before standardly applied manual artifact correction, which is a basic prerequisite when applying any automatic detector. Manual artifact correction of the computerized quantification, which took only a few minutes and which can be performed by a sleep technician during routine PSG scoring, led as expected to an important reduction of the number of false positives which, however, is mandatory in a disorder such as RBD. Computerized Versus Manual RWA Quantification RWA quantification performed with the computer algorithm used here correlated highly with the gold standard of manual EMG quantification. Furthermore, overall sensitivities were very similar, suggesting that time-saving computerized quantification is an excellent screening tool for RBD detection. These findings therefore confirm the notion of the few existing studies on computer-assisted quantification of RWA that computerized algorithms are time saving and useful for EMG quantification.10–14 The novelty of the current algorithm to the published algorithms is that it presents the first computerized algorithm fully implemented in a routine PSG system. This, however, is SLEEP, Vol. 37, No. 10, 2014
1669
essential when using computerized quantification in everyday clinical routine. Another advantage of the current software is that the different EMG activity measures are visibly marked after running the software on the PSG screen and therefore can be continuously checked for plausibility online during standard PSG scoring by a routinely trained PSG technician. None of the other software has been integrated in standard PSG systems, and none has fully entered clinical routine so far. Moreover, the algorithm validated in this paper reflects not only a single type of EMG activity,10–14 but all of them.9 The use of several EMG activity measures, however, might be useful as relying on one measure alone might result in RBD underdiagnosis. The current study showed that all subjects who exceeded the tonic EMG cutoff value of 9.6% had RBD, but, sensitivity was only moderate because a significant portion of true RBD subjects did not reach this cutoff level. In contrast, most RBD subjects were detected when the various measures were combined. Of note, the rate of false positives could be reduced when applying a combined measure such as the SINBAR EMG activity index in comparison to assessing phasic EMG activity in the chin alone. In addition, it is the first algorithm that detects EMG activity not only in the chin but the limbs. Previous studies of our group that manually investigated multiple muscles of the human body in patients with RBD demonstrated that a combination of the mentalis and the FDS muscle on the upper extremities is very suitable, with an area under the curve of 0.998 to discriminate between RBD and non-RBD.9 Therefore, in this study EMG activity of the mentalis and the FDS was assessed; however, assessment of EMG activity in any other extremity muscle is feasible. Computer-Assisted RWA Quantification with Versus without Manual Artifact Correction Based on the assumption that computerized RWA quantification might be prone to false positives because of confounding by artifacts (e.g., snoring, transient EMG increases caused by respiratory event-related arousals), we directly compared the results of the computer-assisted RWA quantification without artifact correction against the computer-assisted RWA quantification with artifact correction. This manual artifact correction is a basic prerequisite for the use of any automatic detector. None of the published studies has reported on these data yet, as only data after manual artifact correction were provided.10–14 Correlation analyses of all evaluated EMG activity measures revealed that correlation is high for both approaches, however, that specificities were lower without manual artifact correction Validation of an Integrated Software for RBD Detection—Frauscher et al.
for all 3-sec miniepoch mentalis EMG activity measures. The decrease in specificities might be even more pronounced in real-life routine conditions, as the recording quality of the current research recordings was excellent and none of the patients or controls had relevant respiratory event associated arousals with arousal-associated EMG activity. As specificity, however, might be even more important than sensitivity in RBD because of its major long-term effect, manual artifact correction, which only takes a few minutes and can be performed during routine PSG scoring by a sleep technician, is crucial when quantifying EMG activity with a computer algorithm. Results of automatically detected tonic mentalis EMG activity and phasic FDS EMG activity seemed at first glance not to be improved by manual artifact correction. When interpreting these findings, however, care should be exercised because, especially for tonic EMG activity, a floor effect might be responsible for the lack of a significant improvement of detection after manual artifact correction. Healthy Subjects Exceeding the SINBAR EMG Activity Cutoff Values Although this was not within the scope of the study, there was some overlap in the presence of EMG activity between subjects with RBD and healthy volunteers. Indeed, 33 % of controls exceeded at least one of the proposed SINBAR cutoffs. The explanation for this finding is purely speculative. Of note, the subjects of the current study were healthy controls, whereas in the previous study the control subjects were recruited from patients under sufficient nasal continuous positive airway pressure (nCPAP) therapy.9 Whether nCPAP therapy can mask phasic EMG activity in the mentalis muscle presents one potential explanation. Another possibility is that some of the healthy subjects are correctly classified as having REM sleep without atonia and RBD will eventually develop. This hypothesis awaits longitudinal follow-up studies. Apart from this, we cannot exclude that some of the increase in phasic EMG activity seen in this study in comparison with the previous ones of the SINBAR group might be related to the different frequency filters employed, low frequency filter 10 Hz–high frequency filter 100 Hz before versus 50 Hz–300 Hz now. Although both settings follow the current AASM recommendations for EMG recording, the wider filter bandpass of the current study likely might have allowed more EMG activity to be recorded, and as a result higher phasic EMG activity indices. In summary, the current study validated the first software algorithm for RWA detection that is now integrated into a PSG system and commercially available against the gold standard of manual EMG quantification according to SINBAR scoring criteria. Our data demonstrated that this computer algorithm is a promising tool for RBD detection in both research and clinical routine. A brief check for plausibility of automatic detection should be a basic prerequisite for this and all other available computer algorithms. ABBREVIATIONS AASM, American Academy of Sleep Medicine EMG, electromyography FDS, flexor digitorum superficialis muscle min, minutes SLEEP, Vol. 37, No. 10, 2014
1670
PSG, polysomnography RBD, REM sleep behavior disorder RWA, REM sleep without atonia sec, seconds SPT, sleep period time DISCLOSURE STATEMENT This was not an industry supported study. This study was supported by the Austrian Science Fund (KLI 236). Dr. Frauscher has participated in speaking engagements/consultancies for UCB and Mundipharma and has received travel support from Habel and Vivisol. Dr. Poewe has received consultancy and lecture fees from AbbVie, Teva-Lundbeck, Novartis, GSK, Boehringer-Ingelheim, UCB, Orion Pharma, Merck Serono and Merz Pharmaceuticals in relation to clinical drug development programs for PD. Dr. Högl has participated in speaking engagements/advisory boards and/or consulted for UCB, Pfizer, Sanofi, GSK, BI, Mundipharma, and Respironics; has received travel support from Habel Medizintechnik, Vivisol; and a grant to her institution from UCB. The other authors have indicated no financial conflicts of interest. The work was performed at the Department of Neurology, Innsbruck Medical University, Innsbruck, Austria. REFERENCES
1. American Academy of Sleep Medicine. International classification of sleep disorders: diagnostic and coding manual. 2nd ed. Westchester, IL: American Academy of Sleep Medicine, 2005. 2. Frauscher B, Högl B. REM sleep behavior disorder: discover of REM sleep behavior disorder, clinical and laboratory diagnosis, and treatment. In: Chokroverty S, Allen RP, Walters AS, Montagna P, eds. Sleep and movement disorders. New York: Oxford University Press, 2013:406–22. 3. Schenck CH, Boeve BF, Mahowald MW. Delayed emergence of a parkinsonian disorder or dementia in 81% of older men initially diagnosed with idiopathic rapid eye movement sleep behavior disorder: a 16-year update on a previously reported series. Sleep Med 2013;14:744–8. 4. Iranzo A, Tolosa E, Gelpi E, et al. Neurodegenerative disease status and post-mortem pathology in idiopathic rapid-eye-movement sleep behaviour disorder: an observational cohort study. Lancet Neurol 2013;12:443–53. 5. Postuma RB, Gagnon JF, Vendette M, Fantini ML, Massicotte-Marquez J, Montplaisir J. Quantifying the risk of neurodegenerative disease in idiopathic REM sleep behaviour disorder. Neurology 2009;72:1296–300. 6. Lapierre O, Montplaisir J. Polysomnographic features of REM sleep behavior disorder: development of a scoring method. Neurology 1992;42:1371–4. 7. Frauscher B, Ehrmann L, Högl B. Defining muscle activities for assessment of rapid eye movement sleep behavior disorder: from a qualitative to a quantitative diagnostic level. Sleep Med 2013;14:729–33. 8. Montplaisir J, Gagnon JF, Fantini ML, et al. Polysomnographic diagnosis of idiopathic REM sleep behaviour disorder. Mov Disord 2010;25:2044–51. 9. Frauscher B, Iranzo A, Gaig C, et al. Normative EMG values during REM sleep for the diagnosis of REM sleep behavior disorder. Sleep 2012;35:835–47. 10. Burns JW, Consens FB, Little RJ, Angell KJ, Gilman S, Chervin RD. EMG variance during polysomnography as an assessment for REM sleep behavior disorder. Sleep 2007;30:1771–8. 11. Ferri R, Manconi M, Plazzi G, et al. A quantitative statistical analysis of the submentalis muscle EMG amplitude during sleep in normal controls and patients with REM sleep behavior disorder. J Sleep Res 2008;17:89–100. 12. Mayer G, Kesper K, Ploch T, et al. Quantification of tonic and phasic muscle activity in REM sleep behavior disorder. J Clin Neurophysiol 2008;25:48–55. 13. Ferri R, Rundo F, Manconi M, et al. Improved computation of the atonia index in normal controls and patients with REM sleep behavior disorder. Sleep Med 2010;11:947–9.
Validation of an Integrated Software for RBD Detection—Frauscher et al.
14. Kempfner J, Sorensen G, Zoetmulder M, Jennum P, Sorensen HB. REM behaviour disorder detection associated with neurodegenerative diseases. Conf Proc IEEE Eng Med Biol Soc 2010;2010:5093–6. 15. Frauscher B, Iranzo A, Hogl B, et al. Quantification of electromyographic activity during REM sleep in multiple muscles in REM sleep behavior disorder. Sleep 2008;31:724–31. 16. Iranzo A, Frauscher B, Santos H, et al. Usefulness of the SINBAR electromyographic montage to detect the motor and vocal manifestations occurring in REM sleep behavior disorder. Sleep Med 2011;12:284–8.
SLEEP, Vol. 37, No. 10, 2014
17. Frauscher B, Gabelia D, Mitterling T, et al. Motor events during healthy sleep: a quantitative polysomnographic study. Sleep 2014;37:763–73. 18. Iber C, Ancoli-Israel S, Chesson A, Quan SF, for the American Academy of Sleep Medicine. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications, 1st ed. Westchester, IL: American Academy of Sleep Medicine, 2007.
1671
Validation of an Integrated Software for RBD Detection—Frauscher et al.
http://dx.doi.org/10.5665/sleep.4076
VALIDATION OF SOFTWARE FOR THE DETECTION OF REM SLEEP BEHAVIOR DISORDER
Validation of an Integrated Software for the Detection of Rapid Eye Movement Sleep Behavior Disorder Birgit Frauscher, MD; David Gabelia, MD; Marlene Biermayr; Ambra Stefani, MD; Heinz Hackner; Thomas Mitterling, MD; Werner Poewe, MD; Birgit Högl, MD Department of Neurology, Innsbruck Medical University, Innsbruck, Austria
Study Objectives and Design: Rapid eye movement sleep without atonia (RWA) is the polysomnographic hallmark of REM sleep behavior disorder (RBD). To partially overcome the disadvantages of manual RWA scoring, which is time consuming but essential for the accurate diagnosis of RBD, we aimed to validate software specifically developed and integrated with polysomnography for RWA detection against the gold standard of manual RWA quantification. Setting: Academic referral center sleep laboratory. Participants: Polysomnographic recordings of 20 patients with RBD and 60 healthy volunteers were analyzed. Interventions: N/A. Measurements and Results: Motor activity during REM sleep was quantified manually and computer assisted (with and without artifact detection) according to Sleep Innsbruck Barcelona (SINBAR) criteria for the mentalis (“any,” phasic, tonic electromyographic [EMG] activity) and the flexor digitorum superficialis (FDS) muscle (phasic EMG activity). Computer-derived indices (with and without artifact correction) for “any,” phasic, tonic mentalis EMG activity, phasic FDS EMG activity, and the SINBAR index (“any” mentalis + phasic FDS) correlated well with the manually derived indices (all Spearman rhos 0.66–0.98). In contrast with computerized scoring alone, computerized scoring plus manual artifact correction (median duration 5.4 min) led to a significant reduction of false positives for “any” mentalis (40%), phasic mentalis (40.6%), and the SINBAR index (41.2%). Quantification of tonic mentalis and phasic FDS EMG activity was not influenced by artifact correction. Conclusion: The computer algorithm used here appears to be a promising tool for REM sleep behavior disorder detection in both research and clinical routine. A short check for plausibility of automatic detection should be a basic prerequisite for this and all other available computer algorithms. Keywords: computer algorithm, detection, polysomnography, REM sleep behavior disorder, scoring Citation: Frauscher B, Gabelia D, Biermayr M, Stefani A, Hackner H, Mitterling T, Poewe W, Högl B. Validation of an Integrated Software for the Detection of Rapid Eye Movement Sleep Behavior Disorder. SLEEP 2014;37(10):1663-1671.
INTRODUCTION Rapid eye movement sleep without atonia (RWA) is the polysomnographic (PSG) hallmark of REM sleep behavior disorder (RBD), and current International Classification of Sleep Disorders criteria thus require PSG evaluation for a definite diagnosis of RBD.1 An attempt to diagnose RBD based on history or questionnaires alone may result in false-positive diagnoses because of the challenging differential diagnostic spectrum.2 An accurate diagnosis of RBD has gained further importance because of the eventual development of neurodegenerative diseases such as Parkinson disease, Lewy body dementia, or multiple system atrophy. Long-term follow-up investigations of idiopathic RBD cohorts have shown that neurodegenerative disease developed in up to 80% of subjects, with a mean delay between RBD onset and the respective neurodegenerative condition of approximately 15 y.3–5 Assessment of RWA is observer based and requires manual quantitative scoring of a subject’s PSG recording.6,7 Only recently, cutoff values for the PSG diagnosis of RBD have been introduced.8,9 Although manual quantification remains the gold standard for RWA scoring, the procedure is time intensive and
difficult to use in clinical routine. To overcome this major limitation, computer-assisted algorithms for the detection of RWA have been developed over the past several years,10–14 but none of these software solutions has been integrated in existing PSG systems. This, however, would be essential for clinical routine. In addition, software solutions focused only on the chin, but not the limbs which was shown to be of importance for RBD evaluation.9,15,16 Moreover, validation against the gold standard of manual RWA quantification is lacking for some algorithms.14 The aim of the current study was to validate an individually tailored and now commercially available computer algorithm for the detection of different types of RWA activity measures, including the Sleep Innsbruck Barcelona (SINBAR) criteria against the gold standard of manual RWA quantification. To emphasize the importance of artifact correction for any automatic detector, we provided both the uncorrected and the manually artifact corrected validation results. The assessment of normative values for REM-related electromyographic (EMG) activity for the differentiation between patients with RBD and controls was not within the scope of the current study. METHODS
Submitted for publication December, 2013 Submitted in final revised form March, 2014 Accepted for publication April, 2014 Address correspondence to: Birgit Högl, MD, Innsbruck Medical University, Department of Neurology, Anichstrasse 35, A-6020 Innsbruck, Austria: Tel: +43 512 504-23811; Fax: +43 512 504-23842; E-mail: [email protected] SLEEP, Vol. 37, No. 10, 2014
Selection of Study Participants All patients with either idiopathic or symptomatic RBD caused by neurodegenerative disease as well as all healthy volunteers who underwent PSG at the sleep center of the Department of Neurology of Innsbruck Medical University between September 2011 and July 2013 were screened for this study. 1663
Validation of an Integrated Software for RBD Detection—Frauscher et al.
RBD was diagnosed according to International Classification of Sleep Disorders 2 criteria1 based on (1) a history suggestive of RBD, and (2) a qualitative finding of RWA in the video-PSG. All healthy volunteers were selected from the Innsbruck sleep laboratory normative sample.17 This sample was recruited from an existing representative population sample after undergoing a two-step screening process (telephone interview, personal expert interview including a complete neurological examination). Approximately four subjects had to be screened to identify one willing and eligible subject fulfilling the strict inclusion and exclusion criteria. In brief, any relevant sleep disorder, any neurological or psychiatric disease, presence of internal diseases such as pulmonary, renal, cardiac, and liver disease, diabetes mellitus, a body mass index higher than 30 kg/ m2, pregnancy, regular alcohol consumption of three or more glasses of wine or three bottles of beer per day, use of pain medication more than three times per week, or use of central nervous system active medication including hypnotics during the year prior to study participation were exclusion criteria. Because the goal of the project for which we originally recruited these healthy sleepers was the investigation of motor activity in physiological sleep, careful attention was paid to the presence of RBD or other movement disorders during sleep, which was assessed by expert interview. For further details see the full paper on motor activity in healthy sleep published by Frauscher et al.17 None of the healthy volunteers had a history suggestive for RBD, and the thorough neurological investigation that was performed revealed no signs of parkinsonism in any of the healthy volunteers. The cutoff date of September 2011 was selected, as in September 2011 the Brain RT PSG system of the company OSG (address: Bussestraat 17, 2840 Rumst, Belgium. Contact person: Sabine Wuytens; website: http://www.osg.be) was introduced at the Innsbruck sleep laboratory, which feature of computerized detection of RWA was validated in the current study. This computer algorithm was individually tailored by the authors together with the software development department of the OSG company in several sessions for the detection of RWA based on the previous works of the SINBAR group.9,15,16 This algorithm is now commercially available, and integrated in the Brain RT PSG system of OSG. This software can perform the same computer algorithm on signals from other PSG devices in EDF, EDF+, Embla file format (titanium), and Brainlab format. Exclusion criteria were an apnea-hypopnea index higher than 10/h, use of any psychiatric or neurological medication for patients with idiopathic RBD and healthy volunteers, or use of melatonin or clonazepam for RBD-specific treatment. For symptomatic patients with RBD, the exclusion criterion was use of clonazepam or melatonin for RBD-specific treatment. All subjects underwent 1 night of 8-h video-PSG according to American Academy of Sleep Medicine (AASM) 2007 standards.18 This study was approved by the local ethical committee of Innsbruck Medical University. All participants granted written informed consent prior to study participation.
(vertical and horizontal eye movements), electromyography (EMG) (mental, both flexor digitorum superficialis muscles (FDS), both anterior tibialis muscles), and cardiorespiratory recording [single channel electrocardiography, nasal airflow (thermocouple), pneumoflow, tracheal microphone, thoracic and abdominal respiratory movements (piezo), transcutaneous oxygen saturation]. Sleep was scored according to the 2007 AASM criteria.18 For scoring of EMG activity, bipolar surface EMG was recorded with the low frequency filter at 50 Hz, the high frequency filter at 300 Hz, and a sampling rate of 1000 Hz. Amplification was set at 5 µV per mm for scoring of REMrelated EMG activity. Impedance of surface EMG electrodes had to be lower than 10 kΩ.
Video-PSG Video-PSG included electroencephalography (F3, F4, C3, C4, O1, O2, M1, and M2 electrodes), electrooculography
The computerized software algorithm was developed by OSG in cooperation with the authors to quantify EMG activity reflecting the SINBAR criteria.9 It detects the rate of “any,”
SLEEP, Vol. 37, No. 10, 2014
Description of Manual and Computer-Assisted EMG Analysis with and without Manual Artifact Correction Manual Quantification of REM Sleep Related EMG Activity (“Any,” Phasic, Tonic )
Before beginning the analysis of EMG activity, REM sleep was carefully examined for artifacts (e.g., snoring, background noise) and transitory increases in EMG tone caused by arousals. All artifacts and transitory increases in EMG tone caused by arousals were excluded from the quantitative scoring of REM sleep related EMG activity. Quantification of “any,” phasic and tonic EMG activity was performed manually during REM sleep according to SINBAR criteria by a single scorer.9 “Any” and tonic EMG activity were assessed for the mentalis muscle only, whereas phasic EMG activity was assessed for both the mentalis and the FDS.9 For scoring “any” and phasic EMG activity, the 30-sec epoch was divided into 10 3-sec miniepochs. “Any” EMG activity was defined as the presence of any EMG activity with an amplitude more than twice the background EMG and a duration of 0.1 sec or greater irrespective of its duration. Phasic EMG activity was defined as any burst of EMG activity lasting between 0.1 and 5.0 sec with an amplitude exceeding twice the background EMG activity irrespective of its morphology. Phasic EMG activity was scored for the chin and the FDS. Tonic EMG activity was scored for the mentalis muscle only in 30-sec epochs, and is defined as a more than 50% increase of sustained EMG activity of the total epoch with an amplitude of at least twice the background EMG muscle tone or more than 10 µV. We calculated the “any” EMG activity index, the phasic EMG activity index, and the tonic EMG activity index in the mentalis muscle, the phasic EMG activity index for the FDS as well as the SINBAR EMG activity index for the combination of “any” EMG activity in the mentalis and phasic EMG activity in the FDS. The “any,” phasic, and SINBAR EMG activity indices are the percentages of 3-sec miniepochs containing “any” or phasic EMG activity divided by all evaluated miniepochs. The tonic EMG index is the percentage of 30-sec epochs, which has tonic EMG activity. Computer-Assisted Scoring Algorithm for RWA Detection (“Any,” Phasic, Tonic ) – without Artifact Correction
1664
Validation of an Integrated Software for RBD Detection—Frauscher et al.
phasic, or tonic EMG activity in the mentalis muscles and phasic EMG activity in all other Table 1—Polysomnographic characteristics of the study samples. extremity muscles. According to a recent Sleep variables RBD sample Healthy volunteer sample P value recommendation,9 the rate of phasic EMG Time in bed, min 484.4 ± 24.3 481.1 ± 9.8 0.672 activity in the FDS was reported. Derived inSPT, min 464.1 ± 29.8 455.4 ± 25.8 0.633 dices are the “any,” phasic, and tonic mentalis Sleep efficiency, % SPT 80.1 ± 10.8 79.6 ± 11.6 0.991 EMG activity index, the phasic FDS EMG Wake after sleep onset, min 74.2 ± 44.9 72.0 ± 46.2 0.735 activity index, as well as the SINBAR EMG Sleep stage, % SPT activity index. 1 11.0 ± 4.7 10.3 ± 4.0 0.601 For phasic EMG activity, a default setting 2 45.8 ± 11.2 44.6 ± 9.2 0.846 with a duration between 0.1 and 5 sec and an 3 11.2 ± 4.8 16.8 ± 7.9 0.001 amplitude exceeding twice the background REM 15.8 ± 5.9 12.2 ± 4.6 0.020 EMG was chosen. The background EMG was Sleep onset latency, min 14.6 ± 15.3 22.6 ± 21.8 0.039 adaptively calculated over 15 sec. The onset REM sleep latency, min 102.7 ± 47.6 143.3 ± 74.0 0.014 criterion is selected with twice the background EMG or 2 µV, whichever is higher; Apnea-hypopnea index, /h 2.9 ± 2.3 3.1 ± 4.2 0.504 the offset criterion is selected when the amREM periods, n 3.5 ± 0.6 3.3 ± 1.0 0.657 plitude falls below 10% of the average EMG 3-sec REM miniepochs, n 1426.2 ± 510.7 975.3 ± 392.3 < 0.001 signal amplitude or below 2 µV, whichever is higher. For example, if the average ampliRBD, REM sleep behavior disorder; REM, rapid eye movement; SPT, sleep period time. tude of the EMG activity is 50 µV, the offset amplitude is when the EMG amplitude drops below 5 µV (10% of the average amplitude). When the average RESULTS amplitude is 15 µV, then the offset amplitude is 2 µV (2 µV is the minimum EMG amplitude). The intermovement distance is Demographic, Clinical, and Sleep Characteristics of the Patients given with 0.1 sec. For “any” EMG activity, the duration was with RBD and the Healthy Volunteer Sample increased to 15 sec. In line with the manual quantification of PSG registrations of 20 patients with RBD (18 men, 2 REM sleep, 3-sec miniepochs were evaluated. women) with a mean age of 65.1 ± 11.6 y were analyzed. RBD The tonic activity of the mentalis muscle was evaluated in etiology was idiopathic in 10 cases (9 men, 1 woman) and 1-sec timeframes over the entire night and divided into four symptomatic in 10 cases (9 men, 1 woman) because of neucategories ranging from low level to high level EMG activity. rodegenerative disease. Five of the 10 subjects had probable Tonic EMG activity is detected when at least 50% of the reParkinson disease, three probable multiple system atrophy, one spective 30-sec epoch contains a background tone higher than probable Lewy body dementia, and one patient had suspected the range of the low level background category augmented neurodegenerative disease that was not further specified. The by 1 µV. median Hoehn and Yahr stage in “on” was 2.5 (range 2–4). Idiopathic RBD subjects had no neurological or psychiatric medication, whereas in the symptomatic RBD group, five patients Computer-Assisted Scoring Algorithm for RWA Detection ( “Any,” Phasic, Tonic ) – with Artifact Correction were on antiparkinsonian medication: three patients were on 500–700 mg levodopa/day, one of these three patients had a For this analysis, the aforementioned software default settings were used and a manual artifact correction was performed combination with 8 mg rotigotine TTS and 300 mg amantadine, two patients were on pramipexole (1.05 and 2.1 mg/day), after running the computerized quantification by a trained scorer and three patients were on 1 mg rasagiline. Two of the 10 subnot involved in the manual EMG scoring process. Manual arjects were treated with the selective serotonin reuptake inhibtifact correction consisted of the exclusion of false positive epitor sertraline. One patient was taking the atypical neuroleptic ochs because of snoring artifacts, electrocardiographic artifacts, olanzapine. or EMG activity in the context of arousals. The healthy volunteer sample consisted of 18 men and 42 women with a mean age of 50.9 ± 13.8 y. None of these subjects Statistics had any central nervous system active medication. Table 1 proStatistics were calculated with SPSS 19.0 for Windows. vides the PSG characteristics of the two study samples. For this All data were tested for normal distribution using the Shaanalysis, a total of 87,042 REM sleep 3-sec miniepochs (mean piro-Wilks test. Because data were not normally distributed, number of analyzed REM sleep 3-sec miniepochs per subject: nonparametric statistics were applied. Spearman correlation 1088 ± 465.1) were quantified. coefficients were assessed to investigate correlations between manual and computerized quantification of REM-related EMG Manual Quantification of REM-Related EMG Activity activity. To follow the methodology of the previous SINBAR 9,15,16 Table 2 provides the mean values of all analyzed EMG acpapers, EMG activity variables are reported as means and tivity measures (“any”, phasic, tonic) obtained by manual EMG standard deviations. Sensitivities and specificities were calcuquantification according to SINBAR EMG criteria for the menlated using JAVA applets provided by the University of Münster. talis muscle, the FDS, and the combined SINBAR EMG meaA P value less than 0.05 was considered to indicate statistical sure. As expected, mean values for all evaluated EMG activity significance. SLEEP, Vol. 37, No. 10, 2014
1665
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Table 2—Comparison of manual and computer-based quantification of rapid eye movement-related electromyographic activity. Computer-assisted quantification without artifact correction
Manual quantification Any mentalis EMG activity index Phasic mentalis EMG activity index Tonic mentalis EMG activity index Phasic FDS EMG activity index SINBAR EMG activity index
RBD cases 55.7 ± 25.8 41.7 ± 15.6 28.3 ± 30.5 44.0 ± 18.1 68.9 ± 22.5
Healthy controls 14.5 ± 6.1 14.0 ± 6.0 0.8 ± 1.1 11.1 ± 8.5 23.6 ± 10.8
RBD cases 56.2 ± 24.1 32.8 ± 15.3 30.4 ± 28.9 45.6 ± 17.7 71.4 ± 21.1
Healthy controls 19.1 ± 6.7 16.9 ± 6.1 1.1 ± 2.4 12.2 ± 8.6 28.4 ± 10.6
Computer-assisted quantification with artifact correction RBD cases 53.8 ± 24.3 30.2 ± 11.9 30.3 ± 28.7 42.4 ± 15.3 68.1 ± 20.6
Healthy controls 15.8 ± 6.7 14.0 ± 5.6 1.1 ± 2.4 10.5 ± 7.9 24.3 ± 10.9
Indices are expressed as means ± standard deviations. EMG, electromyographic; FDS, flexor digitorum superficialis; RBD, REM sleep behavior disorder.
Table 3—Diagnostic test indicators of manual versus computer-assisted electromyographic quantification. Manual quantification
Computer-assisted quantification without artifact correction
Computer-assisted quantification with artifact correction
Any mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.85 (0.62-0.97) 0.73 (0.60-0.84)
0.90 (0.68-0.99) 0.45 (0.32-0.58)
0.90 (0.68-0.99) 0.67 (0.63-0.78)
Phasic mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.90 (0.68-0.99) 0.68 (0.55-0.79)
0.90 (0.68-0.99) 0.47 (0.33-0.60)
0.90 (0.68-0.99) 0.68 (0.55-0.80)
Tonic mentalis Sensitivity, (95% CI) Specificity, (95% CI)
0.64 (0.38-0.82) 1.00 (0.92-1.00)
0.75 (0.51-0.91) 0.98 (0.91-1.00)
0.75 (0.51-0.91) 0.98 (0.91-1.00)
Phasic FDS Sensitivity, (95% CI) Specificity, (95% CI)
0.94 (0.73-1.00) 0.87 (0.75-0.94)
0.95 (0.72-1.00) 0.81 (0.69-0.90)
0.95 (0.73-1.00) 0.83 (0.71-0.92)
SINBAR Sensitivity, (95% CI) Specificity, (95% CI)
0.83 (0.59-0.96) 0.87 (0.75-0.94)
0.94 (0.73-0.10) 0.72 (0.59-0.83)
0.89 (0.65-0.98) 0.83 (0.71-0.92)
CI, confidence interval; FDS, flexor digitorum superficialis.
measures were much higher in RBD patients compared to controls, although there was some overlap between cases and controls. Ninety-five percent of RBD subjects exceeded at least in one EMG activity measure the recently recommended cutoff levels of the SINBAR group9: 85% of subjects exceeded the 18% cutoff value for “any” mentalis EMG activity, 90% the 16% cutoff value for phasic mentalis EMG activity, 65% the tonic mentalis EMG cutoff index of 9.6%, 95% the phasic FDS EMG cutoff index of 17%, and 89% scored positive on the SINBAR EMG montage with a cutoff value of 32%. In contrast, 33% of the healthy volunteers scored positive on at least one of the EMG activity measures: 27% scored positive for “any” mentalis EMG activity, 32% for phasic mentalis EMG activity, 0% for tonic mentalis EMG activity, 13.3% for phasic FDS EMG activity, and 13.3% for the SINBAR EMG montage. The respective sensitivities and specificities are provided in Table 3.
computerized quantification were very similar for “any” mentalis EMG activity, tonic mentalis EMG activity, phasic FDS EMG activity, and the SINBAR EMG montage, whereas phasic mentalis EMG activity as assessed by the computer algorithm was lower for phasic mentalis EMG activity in patients but not controls. Table 3 provides the sensitivities and specificities for the computerized quantification. The measures tonic mentalis EMG activity and phasic FDS EMG activity were very similar for both manual and computer-assisted scoring, whereas for all other measures (“any” mentalis, phasic mentalis, SINBAR) the sensitivity was comparatively similar, but specificity was reduced with the computer-assisted approach alone. Nevertheless, all correlation coefficients were between 0.66 and 0.98 (Figure 1a through 1e). Figure 2a through 2e show that the rate of false positives in the healthy volunteer sample increases when computerized quantification was performed.
Comparison of Manual Versus Computer-Assisted Quantification of REM-Related EMG Activity Table 2 provides mean values of the computer-assisted quantified EMG activity measures for RBD patients and healthy volunteers. Of note, mean indices gained by manual analysis and
Further Improvement of the Computerized Quantification of REM-Related EMG Activity by Manual Artifact Correction To further improve the results of the computerized quantification of REM-related EMG activity, manual artifact correction was performed by a trained PSG scorer (DG). Manual
SLEEP, Vol. 37, No. 10, 2014
1666
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Figure 1—Comparison between manual versus computer-assisted quantification of different electromyographic (EMG) activity measures in the mentalis and flexor digitorum superficialis muscles. (a) “Any” EMG activity in the mentalis muscle. (b) Phasic EMG activity in the mentalis muscle. (c) Tonic EMG activity in the mentalis muscle. (d) Phasic EMG activity in the flexor digitorum superficialis muscles. (e) SINBAR EMG activity index (combined “any” EMG activity in the chin and phasic EMG activity in the flexor digitorum superficialis muscles). Gray dots represent healthy volunteers, black dots patients with REM sleep behavior disorder. SLEEP, Vol. 37, No. 10, 2014
1667
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Figure 2—Individual data for different electromyographic (EMG) activity measures in the mentalis or flexor digitorum superficialis muscles as assessed by manual versus computer-assisted EMG quantification. (a) “Any” EMG activity in the mentalis muscle. (b) Phasic EMG activity in the mentalis muscle. (c) Tonic EMG act ivity in the mentalis muscle. (d) Phasic EMG activity in the flexor digitorum superficialis muscles. (e) SINBAR EMG activity index (combined “any” EMG activity in the chin and phasic EMG activity in the flexor digitorum superficialis muscles). SLEEP, Vol. 37, No. 10, 2014
1668
Validation of an Integrated Software for RBD Detection—Frauscher et al.
Table 4—Overview of false positive cases as assessed by manual versus computer-assisted quantification of different electromyographic activity measures.
Any mentalis EMG activity index Phasic mentalis EMG activity index Tonic mentalis EMG activity index Phasic FDS EMG activity index SINBAR EMG activity index
Manual quantification 16 (26.7 %) 19 (31.7 %) 0 (0 %) 8 (13.3 %) 8 (13.3 %)
Computer-assisted quantification without artifact correction 33 (50 %) 32 (53.3 %) 1 (1.7 %) 11 (18.3 %) 17 (28.3 %)
Computer-assisted quantification with artifact correction 20 (33.3 %) 19 (31.7 %) 1 (1.7 %) 10 (16.7 %) 10 (16.7 %)
EMG, electromyographic; FDS, flexor digitorum superficialis.
artifact correction took a median of 5.4 min (1.4–20). Mean EMG activity indices were similar to those of computerized scoring alone (Table 2). In line with this finding, correlation coefficients were also only slightly higher than with computerized quantification alone (all Spearman rhos: 0.70–0.98, see Figure 1a through 1e). The most striking difference was that manual artifact correction led to an improvement of specificities compared to computerized scoring alone (see Table 3). Moreover, computerized scoring plus manual artifact correction led to a significant reduction of false positives for “any” mentalis (40%), phasic mentalis (40.6%), and the SINBAR index (41.2%). Tonic mentalis EMG activity and phasic FDS EMG activity was not improved by artifact correction (Table 4). DISCUSSION The current study validated a fully integrated and now commercially available software for the detection of RWA, which was developed in cooperation with the authors. This software is the first algorithm which quantifies different types of EMG activity (“any”, phasic, tonic) according to SINBAR criteria not only in the mentalis but also in the extremity muscles. The correlation between the computerized and the standard manual EMG quantification was demonstrated to be high. In addition, this is the first study providing the results of the automatic detection before standardly applied manual artifact correction, which is a basic prerequisite when applying any automatic detector. Manual artifact correction of the computerized quantification, which took only a few minutes and which can be performed by a sleep technician during routine PSG scoring, led as expected to an important reduction of the number of false positives which, however, is mandatory in a disorder such as RBD. Computerized Versus Manual RWA Quantification RWA quantification performed with the computer algorithm used here correlated highly with the gold standard of manual EMG quantification. Furthermore, overall sensitivities were very similar, suggesting that time-saving computerized quantification is an excellent screening tool for RBD detection. These findings therefore confirm the notion of the few existing studies on computer-assisted quantification of RWA that computerized algorithms are time saving and useful for EMG quantification.10–14 The novelty of the current algorithm to the published algorithms is that it presents the first computerized algorithm fully implemented in a routine PSG system. This, however, is SLEEP, Vol. 37, No. 10, 2014
1669
essential when using computerized quantification in everyday clinical routine. Another advantage of the current software is that the different EMG activity measures are visibly marked after running the software on the PSG screen and therefore can be continuously checked for plausibility online during standard PSG scoring by a routinely trained PSG technician. None of the other software has been integrated in standard PSG systems, and none has fully entered clinical routine so far. Moreover, the algorithm validated in this paper reflects not only a single type of EMG activity,10–14 but all of them.9 The use of several EMG activity measures, however, might be useful as relying on one measure alone might result in RBD underdiagnosis. The current study showed that all subjects who exceeded the tonic EMG cutoff value of 9.6% had RBD, but, sensitivity was only moderate because a significant portion of true RBD subjects did not reach this cutoff level. In contrast, most RBD subjects were detected when the various measures were combined. Of note, the rate of false positives could be reduced when applying a combined measure such as the SINBAR EMG activity index in comparison to assessing phasic EMG activity in the chin alone. In addition, it is the first algorithm that detects EMG activity not only in the chin but the limbs. Previous studies of our group that manually investigated multiple muscles of the human body in patients with RBD demonstrated that a combination of the mentalis and the FDS muscle on the upper extremities is very suitable, with an area under the curve of 0.998 to discriminate between RBD and non-RBD.9 Therefore, in this study EMG activity of the mentalis and the FDS was assessed; however, assessment of EMG activity in any other extremity muscle is feasible. Computer-Assisted RWA Quantification with Versus without Manual Artifact Correction Based on the assumption that computerized RWA quantification might be prone to false positives because of confounding by artifacts (e.g., snoring, transient EMG increases caused by respiratory event-related arousals), we directly compared the results of the computer-assisted RWA quantification without artifact correction against the computer-assisted RWA quantification with artifact correction. This manual artifact correction is a basic prerequisite for the use of any automatic detector. None of the published studies has reported on these data yet, as only data after manual artifact correction were provided.10–14 Correlation analyses of all evaluated EMG activity measures revealed that correlation is high for both approaches, however, that specificities were lower without manual artifact correction Validation of an Integrated Software for RBD Detection—Frauscher et al.
for all 3-sec miniepoch mentalis EMG activity measures. The decrease in specificities might be even more pronounced in real-life routine conditions, as the recording quality of the current research recordings was excellent and none of the patients or controls had relevant respiratory event associated arousals with arousal-associated EMG activity. As specificity, however, might be even more important than sensitivity in RBD because of its major long-term effect, manual artifact correction, which only takes a few minutes and can be performed during routine PSG scoring by a sleep technician, is crucial when quantifying EMG activity with a computer algorithm. Results of automatically detected tonic mentalis EMG activity and phasic FDS EMG activity seemed at first glance not to be improved by manual artifact correction. When interpreting these findings, however, care should be exercised because, especially for tonic EMG activity, a floor effect might be responsible for the lack of a significant improvement of detection after manual artifact correction. Healthy Subjects Exceeding the SINBAR EMG Activity Cutoff Values Although this was not within the scope of the study, there was some overlap in the presence of EMG activity between subjects with RBD and healthy volunteers. Indeed, 33 % of controls exceeded at least one of the proposed SINBAR cutoffs. The explanation for this finding is purely speculative. Of note, the subjects of the current study were healthy controls, whereas in the previous study the control subjects were recruited from patients under sufficient nasal continuous positive airway pressure (nCPAP) therapy.9 Whether nCPAP therapy can mask phasic EMG activity in the mentalis muscle presents one potential explanation. Another possibility is that some of the healthy subjects are correctly classified as having REM sleep without atonia and RBD will eventually develop. This hypothesis awaits longitudinal follow-up studies. Apart from this, we cannot exclude that some of the increase in phasic EMG activity seen in this study in comparison with the previous ones of the SINBAR group might be related to the different frequency filters employed, low frequency filter 10 Hz–high frequency filter 100 Hz before versus 50 Hz–300 Hz now. Although both settings follow the current AASM recommendations for EMG recording, the wider filter bandpass of the current study likely might have allowed more EMG activity to be recorded, and as a result higher phasic EMG activity indices. In summary, the current study validated the first software algorithm for RWA detection that is now integrated into a PSG system and commercially available against the gold standard of manual EMG quantification according to SINBAR scoring criteria. Our data demonstrated that this computer algorithm is a promising tool for RBD detection in both research and clinical routine. A brief check for plausibility of automatic detection should be a basic prerequisite for this and all other available computer algorithms. ABBREVIATIONS AASM, American Academy of Sleep Medicine EMG, electromyography FDS, flexor digitorum superficialis muscle min, minutes SLEEP, Vol. 37, No. 10, 2014
1670
PSG, polysomnography RBD, REM sleep behavior disorder RWA, REM sleep without atonia sec, seconds SPT, sleep period time DISCLOSURE STATEMENT This was not an industry supported study. This study was supported by the Austrian Science Fund (KLI 236). Dr. Frauscher has participated in speaking engagements/consultancies for UCB and Mundipharma and has received travel support from Habel and Vivisol. Dr. Poewe has received consultancy and lecture fees from AbbVie, Teva-Lundbeck, Novartis, GSK, Boehringer-Ingelheim, UCB, Orion Pharma, Merck Serono and Merz Pharmaceuticals in relation to clinical drug development programs for PD. Dr. Högl has participated in speaking engagements/advisory boards and/or consulted for UCB, Pfizer, Sanofi, GSK, BI, Mundipharma, and Respironics; has received travel support from Habel Medizintechnik, Vivisol; and a grant to her institution from UCB. The other authors have indicated no financial conflicts of interest. The work was performed at the Department of Neurology, Innsbruck Medical University, Innsbruck, Austria. REFERENCES
1. American Academy of Sleep Medicine. International classification of sleep disorders: diagnostic and coding manual. 2nd ed. Westchester, IL: American Academy of Sleep Medicine, 2005. 2. Frauscher B, Högl B. REM sleep behavior disorder: discover of REM sleep behavior disorder, clinical and laboratory diagnosis, and treatment. In: Chokroverty S, Allen RP, Walters AS, Montagna P, eds. Sleep and movement disorders. New York: Oxford University Press, 2013:406–22. 3. Schenck CH, Boeve BF, Mahowald MW. Delayed emergence of a parkinsonian disorder or dementia in 81% of older men initially diagnosed with idiopathic rapid eye movement sleep behavior disorder: a 16-year update on a previously reported series. Sleep Med 2013;14:744–8. 4. Iranzo A, Tolosa E, Gelpi E, et al. Neurodegenerative disease status and post-mortem pathology in idiopathic rapid-eye-movement sleep behaviour disorder: an observational cohort study. Lancet Neurol 2013;12:443–53. 5. Postuma RB, Gagnon JF, Vendette M, Fantini ML, Massicotte-Marquez J, Montplaisir J. Quantifying the risk of neurodegenerative disease in idiopathic REM sleep behaviour disorder. Neurology 2009;72:1296–300. 6. Lapierre O, Montplaisir J. Polysomnographic features of REM sleep behavior disorder: development of a scoring method. Neurology 1992;42:1371–4. 7. Frauscher B, Ehrmann L, Högl B. Defining muscle activities for assessment of rapid eye movement sleep behavior disorder: from a qualitative to a quantitative diagnostic level. Sleep Med 2013;14:729–33. 8. Montplaisir J, Gagnon JF, Fantini ML, et al. Polysomnographic diagnosis of idiopathic REM sleep behaviour disorder. Mov Disord 2010;25:2044–51. 9. Frauscher B, Iranzo A, Gaig C, et al. Normative EMG values during REM sleep for the diagnosis of REM sleep behavior disorder. Sleep 2012;35:835–47. 10. Burns JW, Consens FB, Little RJ, Angell KJ, Gilman S, Chervin RD. EMG variance during polysomnography as an assessment for REM sleep behavior disorder. Sleep 2007;30:1771–8. 11. Ferri R, Manconi M, Plazzi G, et al. A quantitative statistical analysis of the submentalis muscle EMG amplitude during sleep in normal controls and patients with REM sleep behavior disorder. J Sleep Res 2008;17:89–100. 12. Mayer G, Kesper K, Ploch T, et al. Quantification of tonic and phasic muscle activity in REM sleep behavior disorder. J Clin Neurophysiol 2008;25:48–55. 13. Ferri R, Rundo F, Manconi M, et al. Improved computation of the atonia index in normal controls and patients with REM sleep behavior disorder. Sleep Med 2010;11:947–9.
Validation of an Integrated Software for RBD Detection—Frauscher et al.
14. Kempfner J, Sorensen G, Zoetmulder M, Jennum P, Sorensen HB. REM behaviour disorder detection associated with neurodegenerative diseases. Conf Proc IEEE Eng Med Biol Soc 2010;2010:5093–6. 15. Frauscher B, Iranzo A, Hogl B, et al. Quantification of electromyographic activity during REM sleep in multiple muscles in REM sleep behavior disorder. Sleep 2008;31:724–31. 16. Iranzo A, Frauscher B, Santos H, et al. Usefulness of the SINBAR electromyographic montage to detect the motor and vocal manifestations occurring in REM sleep behavior disorder. Sleep Med 2011;12:284–8.
SLEEP, Vol. 37, No. 10, 2014
17. Frauscher B, Gabelia D, Mitterling T, et al. Motor events during healthy sleep: a quantitative polysomnographic study. Sleep 2014;37:763–73. 18. Iber C, Ancoli-Israel S, Chesson A, Quan SF, for the American Academy of Sleep Medicine. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications, 1st ed. Westchester, IL: American Academy of Sleep Medicine, 2007.
1671
Validation of an Integrated Software for RBD Detection—Frauscher et al.
