International Journal of Audiology 2015; 54: 347–352
Technical Report
Evaluation of new technology for intraoperative evoked compound action potential threshold measurements George Tavartkiladze*, Vigen Bakhshinyan* & Colin Irwin† *National Research Centre for Audiology and Hearing Rehabilitation, Moscow, Russia, and †Cochlear AG, Basel, Switzerland
Abstract Objective: To determine whether new technology for intraoperative evoked compound action potential (ECAP) threshold measurements—the CR120 Intraoperative Remote Assistant device—is comparable to the existing Custom Sound® clinical system, as well as assess test-retest accuracy of the systems. Design: Within subject, repeated measures comparative design. Study sample: ECAP data were collected from 81 pediatric subjects (41 females and 40 males). Results: The average ECAP threshold measurement time for the Custom Sound system was 6.2 minutes (SD 1.0) versus 4.8 minutes (SD 0.7) for the CR120 device. ECAP thresholds measured with Custom Sound and the CR120 device had an average Pearson product-moment correlation coefficient for all electrodes (N 1724) of 0.92, p 0.01; an average mean absolute difference of 6 CLs (SD 6); and a 95% confidence interval of 1 to 1 CLs. The test-retest results for Custom Sound and the CR120 device are also included. Conclusion: A significant reduction in the measurement time for ECAP thresholds was noted with the CR120 device. Furthermore, ECAP thresholds measured with the CR120 device are equivalent to Custom Sound within a clinically acceptable range.
Key Words: Cochlear implant; compound action potential; electrically evoked; electrode; impedance; threshold
The ability to measure directly the peripheral electrophysiological responses from electrical stimulation delivered by a cochlear implant is available in the Advanced Bionics (Stäfa, Switzerland), Cochlear Limited (Macquarie University, Australia), and MED-EL GmbH (Innsbruck, Austria) cochlear implant systems. Respectively for each manufacturer, these technologies are named neural response imaging (NRI), neural response telemetry (NRT) and auditory nerve response telemetry (ART). The electrical evoked compound action potential (ECAP) can be measured with these implant-integrated systems both intraoperatively and postoperatively and is done so for different purposes. Intraoperative threshold measurements are made in order to confirm device integrity, to confirm electrode placement, to establish a baseline set of thresholds for postoperative comparison, etc. On the other hand, postoperative threshold measurements are made primarily for the purpose of guidance in the programming procedure for the cochlear implant system. The Nucleus cochlear implant system (Cochlear Limited, Macquarie University, Australia) is capable of automatically determining the electrical ECAP threshold of the auditory nerve—the minimum stimulation level which evokes a synchronized neural response. The automatic threshold measurement algorithm known as AutoNRT provides for significantly easier use to detect this threshold than
manually (as before), either intraoperatively or postoperatively, and was initially described by Botros et al (2006), with clinical results described by van Dijk et al (2007). The system/algorithm is based on a decision tree analysis, machine learning technique (Quinlan, 1993, 2004). This can be considered as a difference among the different NRI/NRT/ART systems. Each cochlear implant manufacturer offers a system which at a basic level performs the same stimulation and measurement operation. The NRI and ART systems present a predefined “series” of stimuli and then makes the electrophysiological measurements (with automated “peak picking” of both the negative/ positive neural responses). The AutoNRT system meanwhile iteratively changes the stimulation parameters in an attempt to derive the best ECAP threshold response, though also using automated “peak picking” capabilities. Nonetheless, to date, in order to measure the electrical ECAP threshold with the Nucleus cochlear implant system, automatically or manually, it has been required to use the Custom Sound clinical cochlear implant programming software/system, consisting of a computer with associated software, a programming interface (providing electrical isolation) and a sound processor. Performing intraoperative measurements using this system can be cumbersome intraoperatively as a separate, dedicated laptop/programming
Correspondence: George Tavartkiladze, MD, PhD. National Research Centre for Audiology and Hearing Rehabilitation, Leninsky Prospect, 123, 117513, Moscow, Russia. E-mail: [email protected] (Received 23 June 2014; accepted 1 October 2014 ) ISSN 1499-2027 print/ISSN 1708-8186 online © 2014 British Society of Audiology, International Society of Audiology, and Nordic Audiological Society DOI: 10.3109/14992027.2014.973537
348
G. Tavartkiladze et al.
Abbreviations ART CL ECAP NRI NRT
Auditory nerve response telemetry Current level Evoked compound action potential Neural response imaging Neural response telemetry
• • • • •
interface/sound processor (for example on a wheeled trolley) is required. It also has been typical for an experienced audiologist to be present to conduct the measurements—even with the automatic detection system—due to the software being the same as that used during postoperative clinical sessions. (The “in-operating-theatre” presence of an experienced audiologist to operate the Custom Sound system may not be absolutely necessary; the authors are familiar with examples of seemingly equally competent testing via remote use by an experienced audiologist or employment in-situ of a suitably trained operating-theatre staff member.) Recently Cochlear Limited has integrated this same automatic electrical ECAP threshold measurement capability into a novel handheld device with a simpler-to-use interface—the CR120 Intraoperative Remote Assistant. A front view of the device is shown in Figure 1, with the dimensions of the device being 45 130 15 mm (width height depth) and with a screen size of 30 30 mm. The display shown in the figure illustrates the ECAP measurement screen, being one amongst three screens available (the other screens are an electrode impedance display and an ECAP threshold across electrode array display). The stated intent of this device is to address the two needs of less bulky equipment in the operating theatre and a more convenient, easier to use system. The size of the device is smaller than most mobile phones and communicates wirelessly to the sound processor. The device’s user interface has been designed to complement the automated nature of the AutoNRT threshold detection algorithm. As an overview, the CR120 device incorporates the following features:
Impedances measured continuously during electrode array insertion. ECAP thresholds measured on all electrodes via the AutoNRT algorithm. Live visibility of the ECAP measurements and then the threshold profile. Presentation of conditioning stimuli (at 230 current levels (CLs)) to achieve an accurate ECAP threshold measurement. Audible beeps during the conditioning stimuli to enable a surgeon to observe the stapedius muscle reflex.
These features are the same as those offered with the Custom Sound system, with the exceptions of the first item, which is newly introduced with the CR120 device, and the last item, for which the Custom Sound system has the flexibility with which to determine the actual stapedius reflex threshold. Similar to the AutoNRT algorithm implementation in the Custom Sound software, there is no facility for modifying the default stimulation/recording parameters with the CR120 device. (The Custom Sound system provides for modifying the stimulation/recording parameters, but then the system is working in “manual” mode.) The evoked responses are displayed on the screen of the CR120 device as the measurements are being made, but identification of the ECAP thresholds is automatic, as also controlled by the AutoNRT algorithm. The study reported on here was conducted to investigate the equivalency of the CR120 device compared to the Custom Sound system, intraoperatively, for the purposes of establishing electrical ECAP thresholds. When initially designing the study, it was also found that there exists very little published data on the test-retest reliability of the AutoNRT algorithm. Explicitly, van Dijk et al (2007), when looking at clinical results, did not investigate this aspect. Therefore, an evaluation of test-retest reliability of each system was added to the research design. Test-retest reliability of ECAP threshold measurements has been investigated previously by Lai et al (2004), Holstad at al (2009), and Brown et al (2010). This research investigated the longitudinal change of the ECAP thresholds in consideration of utility for guiding clinical activities, i.e. determination of minimum and maximum stimulation levels. Therefore only the mean change (with standard deviation) was reported. However, Holstad et al (2009) also reported the Pearson product-moment correlation coefficient. These instances of test-retest reliability measurements were of manually determined ECAP thresholds performed longitudinally over many months, where physiological changes could be expected. Therefore the analysis techniques—with the exception of reporting the Pearson product-moment correlation coefficient—are not comparable, nor do they provide insight into the test-retest reliability question under consideration here. Müller-Deile (2009), however, did report on short-term test-retest reliability of automatically determined ECAP thresholds intraoperatively. The author used mean absolute differences and standard deviations in their analysis of the two sets of measurements reported. This method fundamentally was followed for this study, but also with the aim to investigate statistical equivalence of the two systems under study. Therefore, additional methods were used.
Materials and Methods
Figure 1. CR120 Intraoperative Remote Assistant, with dimensions: 45 130 15 mm (width height depth) and screen-size: 30 30 mm.
The procedures followed were in accordance with the standards of the ethics committee of the National Research Centre for Audiology and Hearing Rehabilitation, Moscow, Russia, and with the Helsinki Declaration. The CR120 Intraoperative Remote Assistant is commercially available with CE mark approval.
Evaluation of new technology for intraoperative ECAP threshold measurements Patients were enrolled with the following inclusion criteria: • • •
Adult and paediatric subjects. Unilateral and bilaterally (simultaneously or sequentially) implanted subjects. Nucleus CI24RE/CI422 type cochlear implants.
The exclusion criteria were: • • •
Cochlear implant failure (detected during surgery). Serious adverse event(s) leading to non-implantation or discontinuation of the measurement procedure with the cochlear implant. Non-full cochlear implant electrode insertion.
One hundred cochlear implant recipients were enrolled in the study in the institution between June and December 2012. Two of the recipients were excluded from the study due to partial insertion of the cochlear implant electrode during surgery. Furthermore, the data for 17 recipients were lost due to a technical issue with the CR120 Intraoperative Remote Assistant, whereby incorrect data transfer occurred between the sound processor, device, and computer. This technical issue which caused the loss of the data for approximately 20% of the recipients was relayed to Cochlear Limited. The explanation returned to the authors as to the cause of the problem referred to the ability of the CR120 device to operate correctly when a communication error occurs between the sound processor and device. The data collected as part of this study is valid, as “all or nothing” measurements were made/data transferred. If the measurement data was available, then it was accurate. If the data could not be transferred, then the measurement data was not available in the measurement set. Cochlear Limited has stated that this issue has been resolved in the next generation CR220 Intraoperative Remote Assistant device, now currently available. The portion of the study investigating CR120 device versus Custom Sound system equivalency reports on a total of 81 recipients: 41 females: mean age 10.1 years (SD 10.9 years); and 40 males: mean age 5.8 years (SD 7.7 years). Both measurement time and electrical ECAP thresholds were compared between systems. The system measurement order was altered in a controlled randomized fashion across patients. The resulting data sets are from 44 recipients tested with Custom Sound first and then the CR120 device, and 37 recipients tested in the reverse order. As discussed in the introduction, there is little published evidence of test-retest reliability for the AutoNRT algorithm, regardless of system. The initial study design was therefore modified to make multiple measurements with each system, when it was deemed as acceptable by the operating surgeon. Different patient and surgically related factors were involved in this determination and made during each individual surgery; for example time under anaesthesia up to the ECAP threshold measurement point was considered. For this test-retest component of the study, the number of recipients for which electrical ECAP threshold data could be collected was N 16 for the Custom Sound system (5 females: mean age 5.4 years (SD 4.2 years); and 11 males: mean age 7.1 years (SD 11.6 years)); and N 20 for the CR120 device (10 females: mean age 11.0 years (SD 11.2 years); and 10 males: mean age 3.1 years (SD 1.3 years)). All recipients received the Nucleus CI24RE cochlear implant with Contour Advance electrode type, implanted using the Advance OffStylet (AOS) surgical technique. The results of this investigation are reported as the 95% confidence intervals of equivalency, in addition to the Pearson product-moment
349
correlation coefficient and mean difference, for purposes of referring to past investigations. Both per-electrode and combined-electrodes results are reported – the per electrode results to understand whether an electrode-specific effect occurred and the combined electrodes’ results to understand the overall equivalency of the two systems. The software R (R Development Core Team, 2008), version 3.0.2, was used for all statistical analyses. A listing of the R commands is included in Appendix 1 to be found at online http://informahealth care.com/doi/abs/10.3109/14992027.2014.973537.
Results A significant difference in the evaluation time interval for the ECAP threshold measurements was observed between the Custom Sound System (M 6.2 minutes, SD 1.0 minutes) and the CR120 Intraoperative Remote Assistant (M 4.8 minutes, SD 0.7 minutes), t(142.02) 10.23, p 0.001. Upon review, the measured ECAP thresholds, both on a perelectrode and an all-electrode basis, and for both inter and intra system, were found to be non-normally distributed. This was done using basic statistical exploration methods, e.g. a qqplot and a residual plot, and also more advanced normality criteria, e.g. the Shapiro Wilks test. Therefore the paired Wilcoxon Rank Sum and Signed Rank non-parametric test type was used to establish the 95% confidence intervals. The results of evaluation of the CR120 device versus Custom Sound ECAP threshold reliability (N 81; 1724 electrodes) are provided in Table 1 (Appendix 2 to be found at online http:// informahealthcare.com/doi/abs/10.3109/14992027.2014.973537), with the all electrode correlation displayed in Figure 2, a. The average Pearson product-moment correlation coefficient for all electrodes was 0.92 (p 0.001), ranging between 0.83 and 0.95 on a per electrode basis (all p 0.001). The average mean absolute difference was 6 CLs (SD 6 CLs), ranging between 4 and 8 CLs with the standard deviations as noted. The 95% confidence interval was 1 to 1 CLs, with the per electrode confidence intervals also as noted in Table 1. Figure 2, b shows a line plot for the mean ECAP thresholds for both systems on a per electrode basis (error bars indicate a 2 Standard Error Means (SEMs)). This plot illustrates that there appears to be no significant system or electrode bias in regard to the ECAP threshold means measured. In comparison, the results of the Custom Sound test-retest ECAP threshold reliability (N 16; 334 electrodes) are provided in Table 2 (Appendix 2 to be found at online http://informahealthcare. com/doi/abs/10.3109/14992027.2014.973537), with the all electrode correlation displayed in Figure 3, a. The Pearson product-moment correlation coefficient for all electrodes was 0.95 (p 0.001), ranging between 0.71 and 0.98 on a per electrode basis (all p 0.001). The average mean absolute difference was 3 CLs (SD 5 CLs), ranging between 2 and 6 CLs with the standard deviations as noted. The 95% confidence interval was 1 to 3 CLs, with the per-electrode confidence intervals as noted in Table 2. Also in comparison, the results of the CR120 device test-retest ECAP threshold reliability (N 20; 440 electrodes) are provided in Table 3 (Appendix 2 to be found at online http://informahealthcare.com/doi/abs/10.3109/ 14992027.2014.973537), with the all electrode correlation displayed in Figure 3, b. The Pearson product-moment correlation coefficient for all electrodes was 0.96 (p 0.001), ranging between 0.89 and 0.98 on a per-electrode basis (all p 0.001). The average mean absolute difference was 5 CLs (SD 5 CLs), ranging between 4 and 10 CLs with the standard deviations as noted. The 95% confidence
350
G. Tavartkiladze et al.
(a)
CR120 vs. Custom Sound ECAP threshold comparison 250
r = 0.92 , p < 0.001
Custom Sound (CLs)
200
150
100
50
0 0
50
(b)
200
250
Mean ECAP thresholds per electrode System CR120 Custom Sound
210
ECAP threshold (CLs)
100 150 CR120 (CLs)
200
190
180
170 22
21
20
19
18
17
16
15
14
13
12 11 10 Electrode
9
8
7
6
5
4
3
2
1
Figure 2. (a: upper image) All electrode correlation of ECAP thresholds: CR120 versus Custom Sound; and (b: lower image) Mean ECAP thresholds on a per electrode basis for each system (error bars are 2 standard error of means.).
interval was 2 to 3 CLs, with the per electrode confidence intervals as noted in Table 3.
Discussion The ECAP threshold measurement duration was found to be less with the CR120 Intraoperative Remote Assistant compared to the Custom Sound system: 4.8 minutes (SD 0.7 minutes) versus 6.2 minutes (SD 1.0 minutes), an amount of approximately 23%. The durations involved, and the comparative difference, only relate to the electrode impedance and ECAP threshold measurement process (staff travel time, equipment set-up/pack-up time, etc. are not included in these values). The authors are not fully aware of the implementation details for either system, but, based on a general understanding of the AutoNRT algorithm, it can be assumed that this reduction in time is due to a reduced need for computer to sound processor communication. With the CR120 device there is still device to sound processor
communication, but this is via a higher bandwidth communication channel (at 2.2 GHz) compared to the wired computer to sound processor connection (bandwidth: 115.2 kbps). The results of the comparison of ECAP thresholds measured in this study provide both a reference for the expected test-retest reliability of ECAP thresholds determined by the AutoNRT algorithm and highlight that the CR120 device is perhaps functionally equivalent to the Custom Sound system. No discernable difference of ECAP thresholds between electrodes was observed, as illustrated in Figure 2, b, where the mean ECAP thresholds are not significantly different between the two systems. The analyses performed as part of this study indicates that simply considering the Pearson productmoment correlation coefficient between two sets of measurements may not be the best indication of system equivalence. Nor does it provide any guidance as to what limits the two systems (or the single system in the test-retest case) are equivalent, for reasons explained below.
Evaluation of new technology for intraoperative ECAP threshold measurements
(a)
Custom Sound test-retest ECAP threshold comparison 250
r = 0.95 , p < 0.001
CS_2 (CLs)
200
150
100
50
0 0
(b)
50
100 150 CS_1 (CLs)
200
250
CR120 test-retest ECAP threshold comparison 250
r = 0.96 , p < 0.001
CR120_2 (CLs)
200
150
100
50
0 0
50
100 150 CR120_1 (CLs)
200
250
Figure 3. All electrode correlation of ECAP thresholds: (a: upper image) Custom Sound test-retest; and (b: lower image) CR120 test-retest. When considering the results, the fact that ECAP threshold measurements were made intraoperatively mean that there is inherent variability in the threshold values (Müller-Deile (2009)). In particular there were a number of outlying values for the measured ECAP thresholds which contributed to the non-normal distribution of the data as a whole. For the comparison between the CR120 device and the Custom Sound system, this variability didn’t pose a significant problem in the analysis, likely due to the large number of ECAP threshold values which were obtained. Referring to the absolute results, however, this variability would need to be considered, both as representative for the intraoperative case, and likely unrepresentative for the postoperative case (where it’s known that the ECAP thresholds are more stable in nature; e.g. Müller-Deile (2009) and Hughes, et al (2001)). Certainly some consideration would need to be given to using ECAP thresholds measured intraoperatively for postoperative purposes given this variability, although the actual clinical application may be tolerant to it. The observed difference in ECAP thresholds for the outlying cases, mentioned above, can’t be explained by the authors. There may have
351
been surgical aspects which prevented accurate ECAP threshold values from being obtained, e.g. an “air bubble” being present in the location of either the stimulation or measurement electrode(s). Alternatively there may have been physiological reasons for these discrepancies, i.e. neurons only newly being stimulated. A different reason could be the sensitivity or specificity of the AutoNRT algorithm to detect the true ECAP threshold. Based on the measurements observed, shown in Figure 2, a, and Figure 3, a and b, the spread of data and outliers seem to be fairly distributed and therefore maybe the latter reason could be considered as the most likely. For analysing these equivalency type results, to date, a number of statistical characteristics have been reported, including the Pearson product-moment correlation coefficient and the absolute mean difference (with standard deviation). Based on the ECAP thresholds measured as part of this study, the Pearson product-moment correlation coefficient would seem to be a poor indicator of equivalence. That is, the coefficients on either a per-electrode or an all-electrodes basis are typically close to 1.0 with a high statistical probability, as might be expected, yet some variability exists. Furthermore, the coefficient values do not provide any guidance for day-to-day clinical practice. Again, Müller-Deile (2009) previously reported the results of comparing two measurement systems with absolute mean differences and their standard deviations. This provides an indication of the observed variability of the measurements, but not an indication of the statistical probability that the measurements were similar, or equivalent, given a defined limit in difference. An alternative to this approach is to report the 95% confidence intervals from a suitable parametric, or non-parametric, two one-sided test (TOST) using the approach originally described by Schuirmann (1981). Using confidence intervals provides both a measure of statistical probability and equivalence. Defining the problem, the question investigated here is: does a statistically significant difference exist for measurements made with the systems and what is the clinical implication? As an example, based on private correspondence with the engineers at Cochlear Limited, it is known that there is an approximate two-to-three current level (CL) offset inherent in the AutoNRT algorithm implemented on the CR120 device. The reasoning for this offset is unknown to the authors—other than being an implementation detail/issue—but is well demonstrated in the 95% confidence interval (CI) results for the all electrodes ECAP thresholds, in the Custom Sound and the CR120 device comparison. This is not obvious in the absolute mean difference value (with associated standard deviation value). This said, the absolute mean difference and standard deviation of 6 CLs (SD 6) for the all electrode comparison of Custom Sound and the CR120 device ECAP thresholds does match well that reported by Müller-Deile (2009): five CLs (SD 6). This can be considered an interesting outcome for this study, i.e. in regard to two-system equivalence, because Müller-Deile investigated the test-retest of the single Custom Sound system only. Therefore this difference would seem to be the limit of equivalence for the AutoNRT algorithm in general, at least intraoperatively, as investigated by Müller-Deile (2009) and here. The system test-retest results—both for Custom Sound and the CR120 device—provide additional information in understanding the general accuracy of the AutoNRT algorithm. For example, the 95% confidence intervals are within 1–2 current levels for the system test-retest measurements, compared with a similar 2 current level 95% confidence interval for the inter-system comparison. However there is a noted positive bias—that is, the second ECAP threshold measurements are higher—for both the Custom Sound
352
G. Tavartkiladze et al.
and CR120 device systems. While there is “pre-conditioning” stimulation delivered prior to the ECAP threshold measurements being made, with both systems, it would still appear that there is a stimulation order effect present. Might this be due to a reduction in the electrode impedance(s), given the stimulation delivered? Then, given the magnitude of this difference, i.e. 1–2 current levels, this shouldn’t be a significant hurdle for the points of establishing device integrity, confirming electrode placement and establishing a baseline set of thresholds for postoperative comparison (as described in the introduction). Whatever the case, this is an interesting result of this investigation. In view of results of this study confirming the comparability and reliability of the ECAP measures assessed via the hand-held CR120 device versus the Custom Sound system, it may be concluded that the CR120 device is a reliable evaluation tool in the operating theatre during cochlear implant surgery. Additional potential advantages for clinical application of the CR120 device also may be suggested for further investigation, such as: efficiency gained overall, given reduction of complexity/bulk of instrumentation of the ECAP evaluation; and efficacy and value of use of personnel working hands-on in the surgery other than audiologists; and by what model of technical/ professional oversight of such testing would be needed. These issues have relevance to problems of the ever-rising costs of health care at any medical center and/or the potential for providing safe treatment via cochlear implants in remote settings (apropos developments in telemedicine in general and outreach efforts to more underdeveloped areas of the globe).
Conclusion This was a study of the accuracy and reliability of using the CR120 Intraoperative Remote Assistant as an intraoperative tool, relative to the Custom Sound system presently and broadly in use clinically. The CR120 device upon its introduction was purported to lessen the task for ECAP threshold measurements in the operating theatre. The adoption of any given device for ECAP testing is predicated on the ability to make response threshold measurements with high reliability and accuracy. Results of this study have shown that the CR120 device yields ECAP threshold measures with the same level of accuracy and reliability as the Custom Sound clinical system, in an automated fashion, and in a compact instrument and thus highly attractive for use in the surgical arena. In addition, measurement time with the CR120 device was 23% faster than with the Custom Sound system.
Supplementary material available online Supplementary Appendix 1–2.
Acknowledgements The intermediate results of this study were presented at the 7th International Symposium on Objective Measures on Auditory Implants, Amsterdam, the Netherlands, 19–22 September 2012; and the 11th European Symposium on Paediatric Cochlear Implantation, Istanbul, Turkey, 23–26 May 2013. Declaration of interest: Colin Irwin is an employee of Cochlear AG, Basel, Switzerland.
References Botros A., van Dijk B. & Killian M. 2006. AutoNRT: An automated system that measures ECAP thresholds with the Nucleus Freedom cochlear implant via machine intelligence. Artificial Intelligence in Medicine, 40, 15–28. Brown C.J., Abbas P.J. et al. 2010. Effects of long-term use of a cochlear implant on the electrically evoked compound action potential. J Am Acad Audiol, 21, 5–15. Holstad B.A., Sonneveldt V.G., Fears B.T., Davidson L.S., Aaron R.J. et al. 2009. Relation of electrically evoked compound action potential thresholds to behavioral T- and C-levels in children with cochlear implants. Ear Hear, 30, 115–127. Hughes M.L., Vander Werff K.R., Brown C.J., Abbas P.J., Kelsay D.M.R. et al. 2001. A longitudinal study of electrode impedance, the electrically evoked compound action potential, and behavioral measures in Nucleus 24 cochlear implant users. Ear Hear, 22, 471–486. Lai W.K., Aksit M., Akdas F. & Dillier N. 2004. Longitudinal behaviour of neural response telemetry (NRT) data and clinical implications. Int J Audiol, 43, 252–263. Müller-Deile J. (ed.) 2009. Verfahren zur Anpassung und Evaluation von Cochlear Implant Sprachprozessoren: First Edition. Heidelberg, Germany: Median Verlag von Killisch-Horn GmbH, pp. 40–69. ISBN 978-3-941146-01-3. Quinlan J. R. 1993. C4.5: Programs for Machine Learning. San Mateo: Morgan Kaufmann. Quinlan J. R. 2004. An Informal Tutorial, Rulequest Research. Available at: http://www.rulequest.com/see5-win.html. Accessed November 5, 2014. R Development Core Team. 2008. R: A Language and Environment for Statistical Computing: 1st Edn. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. URL http://www.R-project.org. Schuirmann D.J. 1981. On hypothesis testing to determine if the mean of a normal distribution is contained in a known interval. Biometrics, 37, 617. van Dijk B., Botros A.M., Battmer R.D., Begall K., Dillier N. et al. 2007. Clinical results of AutoNRT, a completely automatic ECAP recording system for cochlear implants. Ear Hear, 28, 558–570.
Copyright of International Journal of Audiology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Technical Report
Evaluation of new technology for intraoperative evoked compound action potential threshold measurements George Tavartkiladze*, Vigen Bakhshinyan* & Colin Irwin† *National Research Centre for Audiology and Hearing Rehabilitation, Moscow, Russia, and †Cochlear AG, Basel, Switzerland
Abstract Objective: To determine whether new technology for intraoperative evoked compound action potential (ECAP) threshold measurements—the CR120 Intraoperative Remote Assistant device—is comparable to the existing Custom Sound® clinical system, as well as assess test-retest accuracy of the systems. Design: Within subject, repeated measures comparative design. Study sample: ECAP data were collected from 81 pediatric subjects (41 females and 40 males). Results: The average ECAP threshold measurement time for the Custom Sound system was 6.2 minutes (SD 1.0) versus 4.8 minutes (SD 0.7) for the CR120 device. ECAP thresholds measured with Custom Sound and the CR120 device had an average Pearson product-moment correlation coefficient for all electrodes (N 1724) of 0.92, p 0.01; an average mean absolute difference of 6 CLs (SD 6); and a 95% confidence interval of 1 to 1 CLs. The test-retest results for Custom Sound and the CR120 device are also included. Conclusion: A significant reduction in the measurement time for ECAP thresholds was noted with the CR120 device. Furthermore, ECAP thresholds measured with the CR120 device are equivalent to Custom Sound within a clinically acceptable range.
Key Words: Cochlear implant; compound action potential; electrically evoked; electrode; impedance; threshold
The ability to measure directly the peripheral electrophysiological responses from electrical stimulation delivered by a cochlear implant is available in the Advanced Bionics (Stäfa, Switzerland), Cochlear Limited (Macquarie University, Australia), and MED-EL GmbH (Innsbruck, Austria) cochlear implant systems. Respectively for each manufacturer, these technologies are named neural response imaging (NRI), neural response telemetry (NRT) and auditory nerve response telemetry (ART). The electrical evoked compound action potential (ECAP) can be measured with these implant-integrated systems both intraoperatively and postoperatively and is done so for different purposes. Intraoperative threshold measurements are made in order to confirm device integrity, to confirm electrode placement, to establish a baseline set of thresholds for postoperative comparison, etc. On the other hand, postoperative threshold measurements are made primarily for the purpose of guidance in the programming procedure for the cochlear implant system. The Nucleus cochlear implant system (Cochlear Limited, Macquarie University, Australia) is capable of automatically determining the electrical ECAP threshold of the auditory nerve—the minimum stimulation level which evokes a synchronized neural response. The automatic threshold measurement algorithm known as AutoNRT provides for significantly easier use to detect this threshold than
manually (as before), either intraoperatively or postoperatively, and was initially described by Botros et al (2006), with clinical results described by van Dijk et al (2007). The system/algorithm is based on a decision tree analysis, machine learning technique (Quinlan, 1993, 2004). This can be considered as a difference among the different NRI/NRT/ART systems. Each cochlear implant manufacturer offers a system which at a basic level performs the same stimulation and measurement operation. The NRI and ART systems present a predefined “series” of stimuli and then makes the electrophysiological measurements (with automated “peak picking” of both the negative/ positive neural responses). The AutoNRT system meanwhile iteratively changes the stimulation parameters in an attempt to derive the best ECAP threshold response, though also using automated “peak picking” capabilities. Nonetheless, to date, in order to measure the electrical ECAP threshold with the Nucleus cochlear implant system, automatically or manually, it has been required to use the Custom Sound clinical cochlear implant programming software/system, consisting of a computer with associated software, a programming interface (providing electrical isolation) and a sound processor. Performing intraoperative measurements using this system can be cumbersome intraoperatively as a separate, dedicated laptop/programming
Correspondence: George Tavartkiladze, MD, PhD. National Research Centre for Audiology and Hearing Rehabilitation, Leninsky Prospect, 123, 117513, Moscow, Russia. E-mail: [email protected] (Received 23 June 2014; accepted 1 October 2014 ) ISSN 1499-2027 print/ISSN 1708-8186 online © 2014 British Society of Audiology, International Society of Audiology, and Nordic Audiological Society DOI: 10.3109/14992027.2014.973537
348
G. Tavartkiladze et al.
Abbreviations ART CL ECAP NRI NRT
Auditory nerve response telemetry Current level Evoked compound action potential Neural response imaging Neural response telemetry
• • • • •
interface/sound processor (for example on a wheeled trolley) is required. It also has been typical for an experienced audiologist to be present to conduct the measurements—even with the automatic detection system—due to the software being the same as that used during postoperative clinical sessions. (The “in-operating-theatre” presence of an experienced audiologist to operate the Custom Sound system may not be absolutely necessary; the authors are familiar with examples of seemingly equally competent testing via remote use by an experienced audiologist or employment in-situ of a suitably trained operating-theatre staff member.) Recently Cochlear Limited has integrated this same automatic electrical ECAP threshold measurement capability into a novel handheld device with a simpler-to-use interface—the CR120 Intraoperative Remote Assistant. A front view of the device is shown in Figure 1, with the dimensions of the device being 45 130 15 mm (width height depth) and with a screen size of 30 30 mm. The display shown in the figure illustrates the ECAP measurement screen, being one amongst three screens available (the other screens are an electrode impedance display and an ECAP threshold across electrode array display). The stated intent of this device is to address the two needs of less bulky equipment in the operating theatre and a more convenient, easier to use system. The size of the device is smaller than most mobile phones and communicates wirelessly to the sound processor. The device’s user interface has been designed to complement the automated nature of the AutoNRT threshold detection algorithm. As an overview, the CR120 device incorporates the following features:
Impedances measured continuously during electrode array insertion. ECAP thresholds measured on all electrodes via the AutoNRT algorithm. Live visibility of the ECAP measurements and then the threshold profile. Presentation of conditioning stimuli (at 230 current levels (CLs)) to achieve an accurate ECAP threshold measurement. Audible beeps during the conditioning stimuli to enable a surgeon to observe the stapedius muscle reflex.
These features are the same as those offered with the Custom Sound system, with the exceptions of the first item, which is newly introduced with the CR120 device, and the last item, for which the Custom Sound system has the flexibility with which to determine the actual stapedius reflex threshold. Similar to the AutoNRT algorithm implementation in the Custom Sound software, there is no facility for modifying the default stimulation/recording parameters with the CR120 device. (The Custom Sound system provides for modifying the stimulation/recording parameters, but then the system is working in “manual” mode.) The evoked responses are displayed on the screen of the CR120 device as the measurements are being made, but identification of the ECAP thresholds is automatic, as also controlled by the AutoNRT algorithm. The study reported on here was conducted to investigate the equivalency of the CR120 device compared to the Custom Sound system, intraoperatively, for the purposes of establishing electrical ECAP thresholds. When initially designing the study, it was also found that there exists very little published data on the test-retest reliability of the AutoNRT algorithm. Explicitly, van Dijk et al (2007), when looking at clinical results, did not investigate this aspect. Therefore, an evaluation of test-retest reliability of each system was added to the research design. Test-retest reliability of ECAP threshold measurements has been investigated previously by Lai et al (2004), Holstad at al (2009), and Brown et al (2010). This research investigated the longitudinal change of the ECAP thresholds in consideration of utility for guiding clinical activities, i.e. determination of minimum and maximum stimulation levels. Therefore only the mean change (with standard deviation) was reported. However, Holstad et al (2009) also reported the Pearson product-moment correlation coefficient. These instances of test-retest reliability measurements were of manually determined ECAP thresholds performed longitudinally over many months, where physiological changes could be expected. Therefore the analysis techniques—with the exception of reporting the Pearson product-moment correlation coefficient—are not comparable, nor do they provide insight into the test-retest reliability question under consideration here. Müller-Deile (2009), however, did report on short-term test-retest reliability of automatically determined ECAP thresholds intraoperatively. The author used mean absolute differences and standard deviations in their analysis of the two sets of measurements reported. This method fundamentally was followed for this study, but also with the aim to investigate statistical equivalence of the two systems under study. Therefore, additional methods were used.
Materials and Methods
Figure 1. CR120 Intraoperative Remote Assistant, with dimensions: 45 130 15 mm (width height depth) and screen-size: 30 30 mm.
The procedures followed were in accordance with the standards of the ethics committee of the National Research Centre for Audiology and Hearing Rehabilitation, Moscow, Russia, and with the Helsinki Declaration. The CR120 Intraoperative Remote Assistant is commercially available with CE mark approval.
Evaluation of new technology for intraoperative ECAP threshold measurements Patients were enrolled with the following inclusion criteria: • • •
Adult and paediatric subjects. Unilateral and bilaterally (simultaneously or sequentially) implanted subjects. Nucleus CI24RE/CI422 type cochlear implants.
The exclusion criteria were: • • •
Cochlear implant failure (detected during surgery). Serious adverse event(s) leading to non-implantation or discontinuation of the measurement procedure with the cochlear implant. Non-full cochlear implant electrode insertion.
One hundred cochlear implant recipients were enrolled in the study in the institution between June and December 2012. Two of the recipients were excluded from the study due to partial insertion of the cochlear implant electrode during surgery. Furthermore, the data for 17 recipients were lost due to a technical issue with the CR120 Intraoperative Remote Assistant, whereby incorrect data transfer occurred between the sound processor, device, and computer. This technical issue which caused the loss of the data for approximately 20% of the recipients was relayed to Cochlear Limited. The explanation returned to the authors as to the cause of the problem referred to the ability of the CR120 device to operate correctly when a communication error occurs between the sound processor and device. The data collected as part of this study is valid, as “all or nothing” measurements were made/data transferred. If the measurement data was available, then it was accurate. If the data could not be transferred, then the measurement data was not available in the measurement set. Cochlear Limited has stated that this issue has been resolved in the next generation CR220 Intraoperative Remote Assistant device, now currently available. The portion of the study investigating CR120 device versus Custom Sound system equivalency reports on a total of 81 recipients: 41 females: mean age 10.1 years (SD 10.9 years); and 40 males: mean age 5.8 years (SD 7.7 years). Both measurement time and electrical ECAP thresholds were compared between systems. The system measurement order was altered in a controlled randomized fashion across patients. The resulting data sets are from 44 recipients tested with Custom Sound first and then the CR120 device, and 37 recipients tested in the reverse order. As discussed in the introduction, there is little published evidence of test-retest reliability for the AutoNRT algorithm, regardless of system. The initial study design was therefore modified to make multiple measurements with each system, when it was deemed as acceptable by the operating surgeon. Different patient and surgically related factors were involved in this determination and made during each individual surgery; for example time under anaesthesia up to the ECAP threshold measurement point was considered. For this test-retest component of the study, the number of recipients for which electrical ECAP threshold data could be collected was N 16 for the Custom Sound system (5 females: mean age 5.4 years (SD 4.2 years); and 11 males: mean age 7.1 years (SD 11.6 years)); and N 20 for the CR120 device (10 females: mean age 11.0 years (SD 11.2 years); and 10 males: mean age 3.1 years (SD 1.3 years)). All recipients received the Nucleus CI24RE cochlear implant with Contour Advance electrode type, implanted using the Advance OffStylet (AOS) surgical technique. The results of this investigation are reported as the 95% confidence intervals of equivalency, in addition to the Pearson product-moment
349
correlation coefficient and mean difference, for purposes of referring to past investigations. Both per-electrode and combined-electrodes results are reported – the per electrode results to understand whether an electrode-specific effect occurred and the combined electrodes’ results to understand the overall equivalency of the two systems. The software R (R Development Core Team, 2008), version 3.0.2, was used for all statistical analyses. A listing of the R commands is included in Appendix 1 to be found at online http://informahealth care.com/doi/abs/10.3109/14992027.2014.973537.
Results A significant difference in the evaluation time interval for the ECAP threshold measurements was observed between the Custom Sound System (M 6.2 minutes, SD 1.0 minutes) and the CR120 Intraoperative Remote Assistant (M 4.8 minutes, SD 0.7 minutes), t(142.02) 10.23, p 0.001. Upon review, the measured ECAP thresholds, both on a perelectrode and an all-electrode basis, and for both inter and intra system, were found to be non-normally distributed. This was done using basic statistical exploration methods, e.g. a qqplot and a residual plot, and also more advanced normality criteria, e.g. the Shapiro Wilks test. Therefore the paired Wilcoxon Rank Sum and Signed Rank non-parametric test type was used to establish the 95% confidence intervals. The results of evaluation of the CR120 device versus Custom Sound ECAP threshold reliability (N 81; 1724 electrodes) are provided in Table 1 (Appendix 2 to be found at online http:// informahealthcare.com/doi/abs/10.3109/14992027.2014.973537), with the all electrode correlation displayed in Figure 2, a. The average Pearson product-moment correlation coefficient for all electrodes was 0.92 (p 0.001), ranging between 0.83 and 0.95 on a per electrode basis (all p 0.001). The average mean absolute difference was 6 CLs (SD 6 CLs), ranging between 4 and 8 CLs with the standard deviations as noted. The 95% confidence interval was 1 to 1 CLs, with the per electrode confidence intervals also as noted in Table 1. Figure 2, b shows a line plot for the mean ECAP thresholds for both systems on a per electrode basis (error bars indicate a 2 Standard Error Means (SEMs)). This plot illustrates that there appears to be no significant system or electrode bias in regard to the ECAP threshold means measured. In comparison, the results of the Custom Sound test-retest ECAP threshold reliability (N 16; 334 electrodes) are provided in Table 2 (Appendix 2 to be found at online http://informahealthcare. com/doi/abs/10.3109/14992027.2014.973537), with the all electrode correlation displayed in Figure 3, a. The Pearson product-moment correlation coefficient for all electrodes was 0.95 (p 0.001), ranging between 0.71 and 0.98 on a per electrode basis (all p 0.001). The average mean absolute difference was 3 CLs (SD 5 CLs), ranging between 2 and 6 CLs with the standard deviations as noted. The 95% confidence interval was 1 to 3 CLs, with the per-electrode confidence intervals as noted in Table 2. Also in comparison, the results of the CR120 device test-retest ECAP threshold reliability (N 20; 440 electrodes) are provided in Table 3 (Appendix 2 to be found at online http://informahealthcare.com/doi/abs/10.3109/ 14992027.2014.973537), with the all electrode correlation displayed in Figure 3, b. The Pearson product-moment correlation coefficient for all electrodes was 0.96 (p 0.001), ranging between 0.89 and 0.98 on a per-electrode basis (all p 0.001). The average mean absolute difference was 5 CLs (SD 5 CLs), ranging between 4 and 10 CLs with the standard deviations as noted. The 95% confidence
350
G. Tavartkiladze et al.
(a)
CR120 vs. Custom Sound ECAP threshold comparison 250
r = 0.92 , p < 0.001
Custom Sound (CLs)
200
150
100
50
0 0
50
(b)
200
250
Mean ECAP thresholds per electrode System CR120 Custom Sound
210
ECAP threshold (CLs)
100 150 CR120 (CLs)
200
190
180
170 22
21
20
19
18
17
16
15
14
13
12 11 10 Electrode
9
8
7
6
5
4
3
2
1
Figure 2. (a: upper image) All electrode correlation of ECAP thresholds: CR120 versus Custom Sound; and (b: lower image) Mean ECAP thresholds on a per electrode basis for each system (error bars are 2 standard error of means.).
interval was 2 to 3 CLs, with the per electrode confidence intervals as noted in Table 3.
Discussion The ECAP threshold measurement duration was found to be less with the CR120 Intraoperative Remote Assistant compared to the Custom Sound system: 4.8 minutes (SD 0.7 minutes) versus 6.2 minutes (SD 1.0 minutes), an amount of approximately 23%. The durations involved, and the comparative difference, only relate to the electrode impedance and ECAP threshold measurement process (staff travel time, equipment set-up/pack-up time, etc. are not included in these values). The authors are not fully aware of the implementation details for either system, but, based on a general understanding of the AutoNRT algorithm, it can be assumed that this reduction in time is due to a reduced need for computer to sound processor communication. With the CR120 device there is still device to sound processor
communication, but this is via a higher bandwidth communication channel (at 2.2 GHz) compared to the wired computer to sound processor connection (bandwidth: 115.2 kbps). The results of the comparison of ECAP thresholds measured in this study provide both a reference for the expected test-retest reliability of ECAP thresholds determined by the AutoNRT algorithm and highlight that the CR120 device is perhaps functionally equivalent to the Custom Sound system. No discernable difference of ECAP thresholds between electrodes was observed, as illustrated in Figure 2, b, where the mean ECAP thresholds are not significantly different between the two systems. The analyses performed as part of this study indicates that simply considering the Pearson productmoment correlation coefficient between two sets of measurements may not be the best indication of system equivalence. Nor does it provide any guidance as to what limits the two systems (or the single system in the test-retest case) are equivalent, for reasons explained below.
Evaluation of new technology for intraoperative ECAP threshold measurements
(a)
Custom Sound test-retest ECAP threshold comparison 250
r = 0.95 , p < 0.001
CS_2 (CLs)
200
150
100
50
0 0
(b)
50
100 150 CS_1 (CLs)
200
250
CR120 test-retest ECAP threshold comparison 250
r = 0.96 , p < 0.001
CR120_2 (CLs)
200
150
100
50
0 0
50
100 150 CR120_1 (CLs)
200
250
Figure 3. All electrode correlation of ECAP thresholds: (a: upper image) Custom Sound test-retest; and (b: lower image) CR120 test-retest. When considering the results, the fact that ECAP threshold measurements were made intraoperatively mean that there is inherent variability in the threshold values (Müller-Deile (2009)). In particular there were a number of outlying values for the measured ECAP thresholds which contributed to the non-normal distribution of the data as a whole. For the comparison between the CR120 device and the Custom Sound system, this variability didn’t pose a significant problem in the analysis, likely due to the large number of ECAP threshold values which were obtained. Referring to the absolute results, however, this variability would need to be considered, both as representative for the intraoperative case, and likely unrepresentative for the postoperative case (where it’s known that the ECAP thresholds are more stable in nature; e.g. Müller-Deile (2009) and Hughes, et al (2001)). Certainly some consideration would need to be given to using ECAP thresholds measured intraoperatively for postoperative purposes given this variability, although the actual clinical application may be tolerant to it. The observed difference in ECAP thresholds for the outlying cases, mentioned above, can’t be explained by the authors. There may have
351
been surgical aspects which prevented accurate ECAP threshold values from being obtained, e.g. an “air bubble” being present in the location of either the stimulation or measurement electrode(s). Alternatively there may have been physiological reasons for these discrepancies, i.e. neurons only newly being stimulated. A different reason could be the sensitivity or specificity of the AutoNRT algorithm to detect the true ECAP threshold. Based on the measurements observed, shown in Figure 2, a, and Figure 3, a and b, the spread of data and outliers seem to be fairly distributed and therefore maybe the latter reason could be considered as the most likely. For analysing these equivalency type results, to date, a number of statistical characteristics have been reported, including the Pearson product-moment correlation coefficient and the absolute mean difference (with standard deviation). Based on the ECAP thresholds measured as part of this study, the Pearson product-moment correlation coefficient would seem to be a poor indicator of equivalence. That is, the coefficients on either a per-electrode or an all-electrodes basis are typically close to 1.0 with a high statistical probability, as might be expected, yet some variability exists. Furthermore, the coefficient values do not provide any guidance for day-to-day clinical practice. Again, Müller-Deile (2009) previously reported the results of comparing two measurement systems with absolute mean differences and their standard deviations. This provides an indication of the observed variability of the measurements, but not an indication of the statistical probability that the measurements were similar, or equivalent, given a defined limit in difference. An alternative to this approach is to report the 95% confidence intervals from a suitable parametric, or non-parametric, two one-sided test (TOST) using the approach originally described by Schuirmann (1981). Using confidence intervals provides both a measure of statistical probability and equivalence. Defining the problem, the question investigated here is: does a statistically significant difference exist for measurements made with the systems and what is the clinical implication? As an example, based on private correspondence with the engineers at Cochlear Limited, it is known that there is an approximate two-to-three current level (CL) offset inherent in the AutoNRT algorithm implemented on the CR120 device. The reasoning for this offset is unknown to the authors—other than being an implementation detail/issue—but is well demonstrated in the 95% confidence interval (CI) results for the all electrodes ECAP thresholds, in the Custom Sound and the CR120 device comparison. This is not obvious in the absolute mean difference value (with associated standard deviation value). This said, the absolute mean difference and standard deviation of 6 CLs (SD 6) for the all electrode comparison of Custom Sound and the CR120 device ECAP thresholds does match well that reported by Müller-Deile (2009): five CLs (SD 6). This can be considered an interesting outcome for this study, i.e. in regard to two-system equivalence, because Müller-Deile investigated the test-retest of the single Custom Sound system only. Therefore this difference would seem to be the limit of equivalence for the AutoNRT algorithm in general, at least intraoperatively, as investigated by Müller-Deile (2009) and here. The system test-retest results—both for Custom Sound and the CR120 device—provide additional information in understanding the general accuracy of the AutoNRT algorithm. For example, the 95% confidence intervals are within 1–2 current levels for the system test-retest measurements, compared with a similar 2 current level 95% confidence interval for the inter-system comparison. However there is a noted positive bias—that is, the second ECAP threshold measurements are higher—for both the Custom Sound
352
G. Tavartkiladze et al.
and CR120 device systems. While there is “pre-conditioning” stimulation delivered prior to the ECAP threshold measurements being made, with both systems, it would still appear that there is a stimulation order effect present. Might this be due to a reduction in the electrode impedance(s), given the stimulation delivered? Then, given the magnitude of this difference, i.e. 1–2 current levels, this shouldn’t be a significant hurdle for the points of establishing device integrity, confirming electrode placement and establishing a baseline set of thresholds for postoperative comparison (as described in the introduction). Whatever the case, this is an interesting result of this investigation. In view of results of this study confirming the comparability and reliability of the ECAP measures assessed via the hand-held CR120 device versus the Custom Sound system, it may be concluded that the CR120 device is a reliable evaluation tool in the operating theatre during cochlear implant surgery. Additional potential advantages for clinical application of the CR120 device also may be suggested for further investigation, such as: efficiency gained overall, given reduction of complexity/bulk of instrumentation of the ECAP evaluation; and efficacy and value of use of personnel working hands-on in the surgery other than audiologists; and by what model of technical/ professional oversight of such testing would be needed. These issues have relevance to problems of the ever-rising costs of health care at any medical center and/or the potential for providing safe treatment via cochlear implants in remote settings (apropos developments in telemedicine in general and outreach efforts to more underdeveloped areas of the globe).
Conclusion This was a study of the accuracy and reliability of using the CR120 Intraoperative Remote Assistant as an intraoperative tool, relative to the Custom Sound system presently and broadly in use clinically. The CR120 device upon its introduction was purported to lessen the task for ECAP threshold measurements in the operating theatre. The adoption of any given device for ECAP testing is predicated on the ability to make response threshold measurements with high reliability and accuracy. Results of this study have shown that the CR120 device yields ECAP threshold measures with the same level of accuracy and reliability as the Custom Sound clinical system, in an automated fashion, and in a compact instrument and thus highly attractive for use in the surgical arena. In addition, measurement time with the CR120 device was 23% faster than with the Custom Sound system.
Supplementary material available online Supplementary Appendix 1–2.
Acknowledgements The intermediate results of this study were presented at the 7th International Symposium on Objective Measures on Auditory Implants, Amsterdam, the Netherlands, 19–22 September 2012; and the 11th European Symposium on Paediatric Cochlear Implantation, Istanbul, Turkey, 23–26 May 2013. Declaration of interest: Colin Irwin is an employee of Cochlear AG, Basel, Switzerland.
References Botros A., van Dijk B. & Killian M. 2006. AutoNRT: An automated system that measures ECAP thresholds with the Nucleus Freedom cochlear implant via machine intelligence. Artificial Intelligence in Medicine, 40, 15–28. Brown C.J., Abbas P.J. et al. 2010. Effects of long-term use of a cochlear implant on the electrically evoked compound action potential. J Am Acad Audiol, 21, 5–15. Holstad B.A., Sonneveldt V.G., Fears B.T., Davidson L.S., Aaron R.J. et al. 2009. Relation of electrically evoked compound action potential thresholds to behavioral T- and C-levels in children with cochlear implants. Ear Hear, 30, 115–127. Hughes M.L., Vander Werff K.R., Brown C.J., Abbas P.J., Kelsay D.M.R. et al. 2001. A longitudinal study of electrode impedance, the electrically evoked compound action potential, and behavioral measures in Nucleus 24 cochlear implant users. Ear Hear, 22, 471–486. Lai W.K., Aksit M., Akdas F. & Dillier N. 2004. Longitudinal behaviour of neural response telemetry (NRT) data and clinical implications. Int J Audiol, 43, 252–263. Müller-Deile J. (ed.) 2009. Verfahren zur Anpassung und Evaluation von Cochlear Implant Sprachprozessoren: First Edition. Heidelberg, Germany: Median Verlag von Killisch-Horn GmbH, pp. 40–69. ISBN 978-3-941146-01-3. Quinlan J. R. 1993. C4.5: Programs for Machine Learning. San Mateo: Morgan Kaufmann. Quinlan J. R. 2004. An Informal Tutorial, Rulequest Research. Available at: http://www.rulequest.com/see5-win.html. Accessed November 5, 2014. R Development Core Team. 2008. R: A Language and Environment for Statistical Computing: 1st Edn. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. URL http://www.R-project.org. Schuirmann D.J. 1981. On hypothesis testing to determine if the mean of a normal distribution is contained in a known interval. Biometrics, 37, 617. van Dijk B., Botros A.M., Battmer R.D., Begall K., Dillier N. et al. 2007. Clinical results of AutoNRT, a completely automatic ECAP recording system for cochlear implants. Ear Hear, 28, 558–570.
Copyright of International Journal of Audiology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
