Clinical Study Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
Received: November 8, 2012 Accepted after revision: May 7, 2013 Published online: November 8, 2013
Use of an Artificial Neural Network for Diagnosis of Facial Pain Syndromes: An Update Shirley McCartney a Markus Weltin b Kim J. Burchiel a a Department of Neurological Surgery, Oregon Health & Science University, and b Department of Academic and Research Computing, Portland State University, Portland, Oreg., USA
Abstract Background: Based on a classification scheme for facial pain syndromes and a binomial (yes/no) facial pain questionnaire, we previously reported on the ability of an artificial neural network (ANN) to recognize and correctly diagnose patients with different facial pain syndromes. Objectives: We now report on an updated questionnaire, the development of a secure web-based neural network application and details of ANNs trained to diagnose patients with different facial pain syndromes. Methods: Online facial pain questionnaire responses collected from 607 facial pain patients (395 female, 65%, ratio F/M 1.86/1) over 5 years and 7 months were used for ANN training. Results: Sensitivity and specificity of the currently running ANN for trigeminal neuralgia type 1 and trigeminal neuralgia type 2 are 92.4 and 62.5% and 87.8 and 96.4%, respectively. Sensitivity and specificity are 86.7 and 95.2% for trigeminal neuropathic pain, 0 and 100% for trigeminal deafferentation pain and 100% for symptomatic trigeminal neuralgia and postherpetic neuralgia. Sensitivity is 50% for nervus intermedius neuralgia (NIN) and 0% for atypical facial pain (AFP), glossopharyngeal neu-
© 2013 S. Karger AG, Basel 1011–6125/14/0921–0044$39.50/0 E-Mail [email protected] www.karger.com/sfn
ralgia (GPN) and temporomandibular joint disorder (TMJ). Specificity for AFP, NIN and TMJ is 99% and for GPN, 100%. Conclusions: We demonstrate the utilization of questionbased historical self-assessment responses used as inputs to design an ANN for the purpose of diagnosing facial pain syndromes (outputs) with high accuracy. © 2013 S. Karger AG, Basel
Introduction
The diagnostic term ‘facial pain’ includes an extensive list of clinical conditions such as headache and migraine syndromes, central deafferentation syndromes and cranial neuralgias such as trigeminal neuralgia (TN) [1–3]. The International Association for the Study of Pain defines TN as ‘sudden, usually unilateral, severe, brief, stabbing, recurrent episodes of pain in the distribution of one or more branches of the trigeminal nerve’ [4]. An important feature of facial pain that labels it as TN is a positive relief response to an anticonvulsant such as carbamazepine, gabapentin or phenytoin [5, 6]. This can be helpful in differentiating TN from other orofacial pain syndromes that typically do not respond, such as dental pain, sinusitis or temporomandibular disorders. The annual incidence of TN was reported in 1972 as 4.3 per 100,000 Shirley McCartney, PhD Department of Neurological Surgery, Oregon Health & Science University Mail code CH8N, 3303 SW Bond Ave Portland, OR 97239-3098 (USA) E-Mail mccartns @ ohsu.edu
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Key Words Trigeminal neuralgia · Facial pain · Neural networks · Artificial intelligence
Methods Facial Pain Questionnaire We have refined the original 18-question binomial (yes/no) questionnaire and added 4 questions (table 1). The questions are still formulated to provide a concise patient history; 2 questions were added (Q1 and 2) to specifically ask if patients had pain in their face and if they recalled the precise timing of that pain, and a further 2 questions (Q5 and 6) were added to account for the diagnoses of geniculate neuralgia or nervus intermedius neuralgia (NIN), and glossopharyngeal neuralgia (GPN), respectively. Diagnoses currently included as ANN outputs are listed in table 2. Facial Pain Patients All patients with facial pain who were seen by author K.J.B. at the Department of Neurological Surgery, Oregon Health & Science University (OHSU), were prospectively included. The study was
Artificial Neural Network and Facial Pain
Fig. 1. Example of ANN layers.
approved by the OHSU Institutional Review Board. Patient selfadministered response data were verified and confirmed when patients consented (by face-to-face interview and medical history review), and yes/no responses were prospectively collected for future training and refinement of ANNs. The actual diagnoses of the patients as determined by expert diagnosis (author K.J.B.) and documented in the electronic medical record were also collected. Neural Network Application An automated web-based application developed in PHP (a widely used open-source general-purpose scripting language) and C++ (an ‘object-oriented’ programming language created by Bjarne Stroustrup and released in 1985) and dynamically linked to an ANN that we developed was installed on an OHSU secure network. The neural network application is divided into three distinct sites, i.e. public, private and administrative. The public and private sites are similar; both present users with the latest version of the web-based questionnaire and yes/no responses are processed by an ANN to provide a diagnosis. The freely available public (self-diagnosis) site collects no data (https://neurosurgery. ohsu.edu/tgn.php). The private site involves a registration and login, and final collection of yes/no response data from consenting patients. The administration site controls all other aspects of the application. Artificial Neural Network We previously reported on an ANN designed with a highly distributed interconnection of adaptive nonlinear processing elements (PEs) to diagnose a patient’s facial pain type [16]. The current ANN implements a feed-forward neural network trained by the back propagation of error algorithm. This type of neural network is built in layers (fig. 1). Each layer consists of one or more PEs. The output of each PE serves as an input to all the PEs in the layer that precedes it. PEs connected in this manner are said to be ‘fully connected’.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
45
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
persons, with a higher incidence for women (5.9 per 100,000) compared to men (3.4 per 100,000) [7]. A UK study in 2006 reported annual incidence as high as 27 per 100,000 [8]. The exact nature of the mechanism of TN remains unknown; however, pathophysiology is thought to be due to vascular compression of the trigeminal nerve, generally related to age, when atherosclerosis can cause elongation of vessels and compression [9–15]. A patient’s history is central to a facial pain diagnosis and a diagnosis can, in almost every case, be made on history alone. We previously reported on the design of an artificial neural network (ANN) that was capable of recognizing a pattern in a condensed patient history derived from an 18-question self-administered binomial (yes/no) questionnaire [16] based on a classification of some facial pain syndromes [1, 3] that resulted in rendering a correct diagnosis of the type of facial pain syndrome. We now report on updates to the history-based questionnaire, the development of a web-based neural network application and the design of an ANN able to better recognize and correctly diagnose patients with different facial pain syndromes, in particular TN type 1 (TN1), trigeminal neuropathic pain (TNP), symptomatic TN (STN) and postherpetic neuralgia (PHN). The online diagnostic application (https://neurosurgery.ohsu.edu/tgn.php) is freely available to any patient and/or physician. We maintain that a questionnaire such as the one we previously developed and have now refined, in combination with a well-trained ANN, provides facial pain patients access to a highly accurate online self-diagnosis system. With improved diagnosis, better-educated patients will be able to avoid inappropriate treatment modalities and be directed to more appropriate resources.
Table 1. Facial pain questionnaire: TN – diagnostic questionnaire
Diagnostic questions
(4) (5) (6) (7) (8) (9) (10) (11) (12)
(13)
(14) (15) (16)
(17)
(18) (19) (20) (21) (22)
Do you have facial pain? Do you remember exactly where you were the moment your facial pain started? When you have pain, is it predominantly in your face (i.e. forehead, eye, cheek, nose, upper/lower jaw, teeth, lips, etc.)? Do you have pain just on one side of your face? When you have pain, is it predominantly deep in your ear? When you have pain, is it predominantly in the back of your throat or tongue, near the area of your tonsil? Is your pain either entirely or mostly brief (seconds to minutes) and unpredictable sensations (electrical, shocking, stabbing, shooting)? Do you have any constant background facial pain (e.g. aching, burning, throbbing, stinging)? Do you have constant background facial pain (aching, burning, throbbing, stinging) for more than half of your waking hours? Do you have any constant facial numbness? Can your pain start by something touching your face (e.g. by eating, washing your face, shaving, brushing teeth, wind)? Since your pain began have you ever experienced periods of weeks, months or years when you were painfree? (This would not include periods after any pain-relieving surgery or while you were on medications for your pain) Have you ever taken Tegretol® (carbamazepine), Neurontin® (gabapentin), Lioresal® (baclofen), Treleptal® (oxcarbazepine), Topamax® (topiramate), Zonegran® (zonisamide), or any other anticonvulsant medication for your pain? Did you ever experience any major reduction in facial pain (partial or complete) from taking any of the medications listed in Question 9, or any anticonvulsant medication? Have you ever had trigeminal nerve surgery for your pain? (e.g. neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife) Have you ever experienced any major reduction in facial pain (partial or complete) from trigeminal nerve surgery for your pain? (e.g. neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife) Did your current pain start only after trigeminal nerve surgery (neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife)? (If this is a recurrence of your original pain after a successful trigeminal nerve surgery, answer “no”) Did your pain start after facial zoster or “shingles” rash (herpes zoster – not to be confused with “fever blisters” around the mouth)? Do you have multiple sclerosis? Did your pain start after a facial injury? Did your pain start only after facial surgery (oral surgery, ENT surgery, plastic surgery)? When you place your index finger right in front of your ears on both sides at once and feel your jaw open and close does the area under your fingers on either side hurt?
This facial pain questionnaire was taken from the world-wide web universal resource locator: https://neurosurgery.ohsu.edu/ tgn.php. At OHSU’s Department of Neurological Surgery we have developed a helpful questionnaire for the diagnosis and treatment of patients suffering from various types of TN. Italicized questions (Q1 and 2 and Q5 and 6) were added to the original 18-question binomial (yes/no) questionnaire. Adapted from Limonadi et al. [16], 2006. Disclaimer: Any medical or surgical advice provided in this website, even if intended to be accurate to the best of our knowledge, should be discussed with your medical or surgical practi-
46
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
yes yes yes
no no no
yes yes yes yes
no no no no
yes yes
no no
yes yes
no no
yes
no
yes
no
yes
no
yes
no
yes
no
yes
no
yes
no
yes yes yes yes
no no no no
tioner. This website will serve to help direct you (the patient) to appropriate informational resources, and should not be considered a diagnosis. A diagnosis can only be given by an appropriate and experienced physician, after interviewing and examining you (the patient). Only your physician or surgeon knows what is best for you. Always seek direct advice from your physician before embarking on any treatment, medication or therapy. Copyright 2002–2013©, Oregon Health & Science University. This document and the information contained within the document are not to be used or reproduced without written consent from Kim Burchiel, MD.
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
(1) (2) (3)
Table 2. Facial pain diagnoses (outputs) used in ANN development
Diagnosis
Abbreviation
Trigeminal neuralgia, type 1 Trigeminal neuralgia, type 2
TN1 TN2
Trigeminal neuropathic pain
TNP
Trigeminal injury unintentional Facial pain resulting from unintentional injury to the trigeminal system from facial trauma, oral surgery, ear, nose and throat surgery, root injury from posterior fossa or skull base surgery, stroke
Trigeminal deafferentation pain
TDP
Trigeminal injury intentional Facial pain in a region of trigeminal numbness resulting from intentional injury to the trigeminal system from neurectomy, gangliolysis, rhizotomy, nucleotomy, tractotomy, or other denervating procedures
Symptomatic trigeminal neuralgia
STN
Pain resulting from multiple sclerosis
Postherpetic neuralgia
PHN
Pain resulting from trigeminal herpes zoster outbreak
AFP
Pain predominantly having a psychological rather than a physiological origin
Nervus intermedius neuralgia, sometimes known as geniculate neuralgia
NIN
Recurring attacks of severe pain deep in the ear that may spread to the ear canal, outer ear, mastoid or eye regions and involves the nervus intermedius, which is the somatic sensory branch of the 7th cranial nerve
Glossopharyngeal neuralgia
GPN
Recurring attacks of severe pain in the back of the throat, the area near the tonsils, the back of the tongue, and part of the ear and is believed to be caused by irritation of the 9th cranial nerve
Temporomandibular joint disorder
TMJ
Pain due primarily to temporomandibular joint dysfunction
Atypical facial
Facial pain of spontaneous onset of pain >50% episodic >50% constant
be diagnosed by history alone.
Table 3. Data sets available for ANN training (n = 813)
To maintain data integrity, the following 2 conditions apply. a Data that are entered multiple times by patients or patients whose verbal responses at clinic visit do not match responses checked in questionnaire are deemed not reliable and hence ‘unusable’. b Not defined by network; therefore, data not useable to train network
Artificial Neural Network and Facial Pain
Diagnosis in clinical chart
TN1 TN2 TNP TDP STN PHN AFP Not facial pain Occipital neuralgia NIN GPN Bilateral No diagnosis TMJ Anesthesia dolorosa Not TN Poststroke neuropathic facial pain
Number of cases defined by ANN
useable (n = 607)
unusable (n = 206)
yes yes yes yes yes yes yes no no yes yes no no yes no no no
379 75 81 9 35 10 5
27a 5a 11a 1a 1a 1a 4a 10b 9b 1a 2a 7b 26b 13a 7b 75b 3b
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
6 3 4
47
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
a Cannot
paina
History
and testing
Patients, n Male patients, n Female patients, n Facial pain diagnosis, n TN1 TN2 TNP TDP STN PHN AFP NIN GPN TMJ
Total data sets
Testing data sets
Training data sets
364 130 234
120 43 77
244 87 157
239 26 47 3 26 6 3 6 4 4
79 8 15 1 8 2 1 2 2 2
160 18 32 2 18 4 2 4 2 2
The randomly generated data sets (from the available 607 data sets; see table 2) were also randomly split, 1 for training the neural network and 1 for testing. Training involves finding the neural network that best minimizes error. Testing evaluates the neural network that best minimizes error, using a measure of accuracy.
ANN Training Testing and Evaluation A web-based interface allows an administrator to create or evolve a network based on input size (two input sizes are currently available: 18 and 22), hidden layer specifications, selected diagnosis (outputs) and additional hidden layers (fig. 1). Random training/testing sets from the available data sets are generated. The network is retrained and performance evaluated as a measure of evolution of fitness (how accurate the diagnosis represented by each individual data set is relative to the population of data sets; 200 evolutions is the default), a confusion matrix (the accuracy of correct diagnosis vs. actual diagnosis) and sensitivity/specificity (probability of the ANN to identify positive and negative results) analysis. Finally, if the generated and evaluated network is sensitive and specific for outputs it can be installed. This ANN then functions in private (data collected and verified, and patients consented) and public (no data collected) environments.
Results
Data Collection From December 2006 to July 2012 (5 years and 7 months), a total of 813 patients with facial pain consented and responded to the online facial pain questionnaire at the time of their initial clinic visit. Of the 813 consenting subjects, it was determined that 607 of the patient re48
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
sponses were ‘useable’ data sets. A breakdown of the reasoning behind data determined as ‘unusable’ (206 data sets) and ‘useable’ can be found in table 3. Of the 607 respondents, 395 were female (65%; ratio F/M 1.86/1) and the overall average age was 57.3 ± 14.99 years. Of the 607 patients with a complaint of facial pain who were seen in the clinic and using the classification system for facial pain syndromes previously described by Burchiel and Eller [1, 3] as a basis for the ANN output (i.e. diagnosis), 379 were diagnosed with TN1, 75 with TN type 2 (TN2), 81 with TNP, 9 with trigeminal deafferentation pain (TDP), 35 with STN, 10 with PHN, 5 with atypical facial pain (AFP), 6 with NIN, 3 with GPN and 4 with temporomandibular joint (TMJ) pain. Based on stored raw data (output, i.e. ANN diagnosis) and the physician’s diagnosis (as documented in the electronic medical record), which are stored in the neural network application database, the ANN matched the diagnosis of the physician in 529 of the 607 patients (87.1%). Train and Test Data The neural network application could be considered a dynamic, ever-evolving application which allows for retraining, testing and installation of a new ANN at any time or based on the number of data sets collected. The current ANN was retrained, tested and installed in July 2012. A summary of the randomly generated data used to train and test an ANN is shown in table 4. Sensitivity and specificity examples (with either 18 or 22 question inputs and 6, 7 or 10 diagnosis outputs) are presented in table 5. The current sensitivity and specificity of the ANN for TN1 are 92.4 and 87.8%, respectively, and 62.5 and 96.4%, respectively, for TN2. Example test data confirm that the ANNs are still not sensitive in diagnosing TN2 (range 44–89%) or TDP (range 0–100%), although the ANNs trend toward being more sensitive to a TN2 diagnosis (up to 89%) if TN1 is removed as diagnostic output. The current sensitivity and specificity of ANN for TDP are 0 and 100%, respectively, and 86.7 and 95.2% for TNP. Sensitivity and specificity for STN and PHN are 100%. Sensitivity is 50% for NIN and 0% for AFP, GPN and TMJ. Specificity is 99% for AFP, NIN and TMJ and 100% for GPN. A confusion matrix example showing the desired (correct) versus real output (diagnosis) of the ANN created with randomly generated data from the data that are ‘useable’ and currently running in our production environment is shown in table 6.
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Table 4. Randomly generated data sets for neural network training
Table 5. ANN (current network and example networks) sensitivity and specificity data for diagnoses: TN1, TN2, TNP, TDP, STM, PHN,
AFP, NIN, GPN and TMJ from randomly generated train and test data sets TN1
TN2
TNP
TDP
STN
PHN
AFP
NIN
GPN
TMJ
Current ANN (installed 07/24/12) sensitivity specificity
0.924 0.878
0.625 0.964
0.867 0.952
0 1
1 1
1 1
0 0.99
0.5 0.99
0 1
0 0.99
ANN example 1 specificity sensitivity
0.833 0.857
0.444 0.964
0.8 0.9143
0 0.9916
1 1
1 1
1 1
0.5 0.9831
0.5 0.966
0 1
ANN example 2 sensitivity specificity
0.961 0.861
0.667 0.9524
0.733 1
0 1
1 1
1 1
1 0.991
NA NA
NA NA
NA NA
ANN example 3 sensitivity specificity
NA NA
0.889 0.963
0.8 0.952
1 0.971
1 1
1 1
1 0.971
NA NA
NA NA
NA NA
ANN example 4 sensitivity specificity
0.894 0.914
0.5 0.940
0.889 0.9337
0 1
1 0.994
1 0.989
1 0.995
NA NA
NA NA
NA NA
ANN example 5 sensitivity specificity
NA NA
0.792 0.869
0.74 0.884
0.333 0.970
1 1
0.667 1
1 0.971
NA NA
NA NA
NA NA
diagnoses; ANN example 2: input size = 22 questions, output selection = 7 diagnoses; ANN example 3: input size = 18 questions, output selection = 7 diagnoses (TN1 not selected as diagnosis); ANN example 4: input size = 18 questions, output selection = 7 diagnoses; ANN example 5: input size = 18 questions, output selection = 6 diagnoses (TN1 not selected as diagnosis).
Input size determines which question responses are used for train/test data set selection (historically, an 18- or 22-question input is available; see table 1). Output selection determines which diagnoses are evolved as outputs. Current ANN (installed July 24, 2012): input size = 22 questions, output selection = 10 diagnoses; ANN example 1: input size = 22 questions, output selection = 10
Table 6. A confusion matrix example, which visually shows a representation of the accuracy of desired (correct) vs. actual output (diag-
nosis) of the ANN when challenged with randomly generated and tested data sets
Actual diagnosis
TN1 TN2 TNP TDP STN PHN AFP NIN GPN TMJ
TN1
TN2
TNP
TDP
STN
PHN
AFP
NIN
GPN
TMJ
73 2 3 0 0 0 0 1 0 0
1 5 2 0 0 0 0 0 0 0
1 1 13 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 0 0 8 0 0 0 0 0
0 0 0 0 0 2 0 0 0 0
0 1 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 1
2 0 0 0 0 0 0 0 0 0
1 0 1 0 0 0 1 1 0 0
Each column of the matrix represents the predicted/desired facial pain diagnosis and each row represents the actual defined facial pain diagnosis. In the above matrix, values in the diagonal are correctly classified and are italicized, e.g. 73 TN1 diagnoses in the test set were correctly classified as TN1. The off-diagonal row values represent misclassified diagnoses, e.g. 1, 1, 2 and 1 TN1 in the
test set were misclassified as TN2, TNP, GPN and TMJ, respectively. Off-diagonal column values represent diagnoses that were inaccurately included in a diagnosis class, e.g. 2, 3 and 1 TN1 in the test set were included as TN2, TNP and NIN, respectively. The above confusion matrix example represents the ANN currently running in production and was installed on July 24th, 2012.
Artificial Neural Network and Facial Pain
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
49
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Desired diagnosis
Application of Neural Network Modeling Since the original concept of neural computing in 1943 [17], the application of neural network modeling has been used in various disciplines for various tasks including game playing and decision making (chess), pattern recognition (face and object), sequence recognition (speech, handwriting, gestures), financial applications and medical diagnosis such as the presence of breast cancer based on mammography [18–20]. Many diverse applications in medicine have been developed and recent medical applications include carpal tunnel syndrome (based on history) [21], predicting recurrence of kidney stones [22], predicting symptomatic cerebral vasospasm after aneurysmal subarachnoid hemorrhage [23], identification of critical factors in patient falls [24], medical disease estimation [25], thrombotic formation in mechanical heart valves [26], decision making for fineneedle aspiration diagnosis of thyroid malignancy [27], transmandibular joint internal derangements [28], and endoscopic ultrasound in the diagnosis of pancreatic masses [29]. Artificial Neural Networks ANNs are a specific example of the broader concept of artificial intelligence. An ANN is composed of several interconnected layers of ‘neurons’ or PEs. In the ANN we demonstrate, each PE accepts one or more inputs (yes/no responses) and one output (diagnosis; fig. 1). The PEs are connected to the elements in the layers adjacent to them. Each connection is defined by a weight, which is a multiplier applied before the input to the PE. Weights become the actual memory of the network. The ‘learning’ takes place by presenting a network that is initially constructed with random values for weights as an input vector (in our case the 18 or 22 binary question inputs or historical facial pain features). The information is allowed to flow through the network and produce a diagnosis (table 2). The diagnosis produced by the network is compared to expert diagnosis (author K.J.B.). The difference or error between the network and expert diagnosis is used (by a complex algorithm) to make adjustments to the weight values such that the next time it is presented with that same input vector its output (diagnosis) will more likely be identical to the expert diagnosis. A more detailed description of the concepts of layers, interconnections and training algorithms can be found at wikipedia.org (search term ‘artificial neural network’) [30]. 50
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
Key Results Most facial pain syndromes can be categorized into distinct diagnoses as previously proposed [1, 3]. We first reported in 2006 that, using a specific classification of facial pain syndromes and a condensed facial pain history (yes/no responses to a facial pain questionnaire) in combination with an ANN, the ANN could correctly diagnose TN1 with fairly high sensitivity (84%) and specificity (83%) [16]. We now report on the development of a secure webbased neural network application and randomly trained and tested ANNs to accurately diagnose the facial pain syndrome of patients. Various ANNs were able to recognize a pattern in the answers to a facial pain questionnaire that resulted in an output of a correct diagnosis. Interpretation The current sensitivity and specificity of ANN for TN1 are 92.4 and 87.8%, respectively, which is an improvement over our previous results of 84 and 83%. As in our original publication the current ANN is less sensitive at determining an accurate TN2 diagnosis (62.5% sensitive), which could be considered a trend toward improvement over our previous result of 50% sensitivity. As previously reported, this could be because TN2 patients often describe having symptoms that were initially TN1like that evolve with time to symptoms more TN2-like, hence similar questions are checked as yes, making a distinction between TN1 and TN2 more difficult [16]. Sensitivity and specificity results for TNP did improve to 86.7 and 95.2% compared to our previous results of 35 and 85%, respectively. This could be because a larger number of data sets were available for training and testing than previously reported. Sensitivity results for TDP are poor at 0%. Sensitivity did trend higher if a diagnosis of TN1 was not included in the ANN output; however, this was not consistent (see table 5). It is possible that patients who have previously undergone a procedure to treat TN1 retain features of TN1 and similar questions are checked as yes, making a distinction between TN1 and TDP more difficult. Sensitivity and specificity results for STN and PHN are 100%. However, removing TN1 as a diagnosis with an 18-question input lowers the sensitivity of diagnosing PHN from 100 to 67%. We previously conceded that AFP presents the treating physician with complex diagnostic and treatment challenges and as in the previous study ANNs were not optimum for diagnosing AFP. With so few data sets available (5) for training and testing, the AFP results should be interpreted with caution. We predict an improvement over 50% sensitivMcCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Discussion
ity for NIN, GPN and TMJ will be forthcoming with additional data sets. The ability of our ANN in determining the correct diagnosis of TN1 remains of most significance because it is the most treatable form of facial pain. In patients who are refractory to anticonvulsant medications or have side effects, different surgical approaches such as microvascular decompression, percutaneous rhizotomy (glycerol, radiofrequency), balloon compression, stereotactic radiosurgery, trigeminal stimulation or motor cortex stimulation can be performed. Avoiding unnecessary consultations, investigations, procedures or treatments is of costbenefit both fiscally and in terms of the patient’s overall health and well-being. Limitations ANNs depend on the population used to train them, which means they must be trained with a wide variety of data. A weakness of this study is the relatively large number of patients with a diagnosis of TN1 (65.4%). This reflects the referral base pattern at OHSU and as such does result in ‘biasing’ of the neural network program in favor of more commonly predicting a diagnosis of TN1. Removal of TN1 as a diagnosis clearly influences outcome, especially regarding the positive impact on TN2 and TDP and the negative impact on PHN. An important question is whether our proposed questionnaire is optimal for all of the facial pain diagnostic categories described other than TN1, TNP, PHN and STN. Until we have encountered and are able to include a larger number of consenting patients with TN2, TDP, AFP, NIN and GPN in the study, followed by ANN training and testing, a definitive answer to this question remains elusive. An additional weakness/bias that warrants acknowledgment is that the results of the ANN are compared to and built on the evaluation a single expert (K.J.B.). The ANN cannot be fully validated until results can be independently confirmed
(or not) by the collection of external data obtained from other centers and other facial pain experts. Of note, an additional surgeon at OHSU has begun to administer the questionnaire to their facial pain patient population. At a future date this data can be extracted and used for comparison and validation. Data collection, analysis, coordination of IRB approvals and data validation from external centers are an essential next step. A strength of the study lies in that the internet has changed patients’ views of the medical profession and the delivery and cost of their health care. A review of our server logs (a recent 5-year period) reveals that the public website (https://neurosurgery.ohsu.edu/tgn.php) served results approximately 16,000 times (ranging from twice a day to 12 times a day). However, since no data are collected, there is no way to authenticate use. An online selfdiagnosis tool such as we describe provides an avenue to become more actively involved in one’s own health care and highlights the importance of seeking appropriate medical assistance as quickly as possible.
Conclusion
We have refined a binomial facial pain questionnaire and developed a web-based neural network application and an ANN able to recognize and correctly diagnose patients with different facial pain syndromes. Sensitivity and specificity results are improved over our previous report, particularly for TN1, TNP, STN and PHN. Essentially, the facial pain questionnaire in combination with an ANN allows any facial pain patient to self-diagnose. Ultimately, this self-diagnosis tool allows for easy and early identification of TN1 patients suitable for surgical treatment and allows those with other facial pain syndromes to form a better-educated judgment on an appropriate course of treatment and use of resources.
References
Artificial Neural Network and Facial Pain
4 Merskey H, Bogduk N: Classification of Chronic Pain: Descriptions of Chronic Pain Syndromes and Definitions of Pain Terms, ed 2. Seattle, IASP, 1994. 5 Loeser J: Tic douloureux and atypical facial pain; in Wall P, Melzack R (eds): Textbook of Pain. London, Churchill Livingstone, 1994, pp 699–710.
6 Wiffen P, Collins S, McQuay H, Carroll D, Jadad A, Moore A: Anticonvulsant drugs for acute and chronic pain. Cochrane Database Syst Rev 2005:CD001133. 7 Yoshimasu F, Kurland LT, Elveback LR: Tic douloureux in Rochester, Minnesota, 1945– 1969. Neurology 1972;22:952–956. 8 Hall GC, Carroll D, Parry D, McQuay HJ: Epidemiology and treatment of neuropathic pain: the UK primary care perspective. Pain 2006;122:156–162.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
51
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
1 Eller J, Raslan A, Burchiel K: Trigeminal neuralgia: definition and classification. Neurosurg Focus 2005;18:E3. 2 Slavin KV, Burchiel KJ: Surgical options for facial pain; in Burchiel KJ (ed): Surgical Management of Pain. New York, Thieme, 2002, pp 849–864. 3 Burchiel KJ: A new classification for facial pain. Neurosurgery 2003;53:1164–1166.
52
17 McCulloch WS, Pitts W: A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 1943;5:115–133. 18 Parmeggiani D, Avenia N, Sanguinetti A, Ruggiero R, Docimo G, Siciliano M, Ambrosino P, Madonna I, Peltrini R, Parmeggiani U: Artificial intelligence against breast cancer (ANNES-BC Project). Ann Ital Chir 2012;83: 1–5. 19 Orr RK: Use of an artificial neural network to quantitate risk of malignancy for abnormal mammograms. Surgery 2001;129:459–466. 20 Wu Y, Giger ML, Doi K, Vyborny CJ, Schmidt RA, Metz CE: Artificial neural networks in mammography: application to decision making in the diagnosis of breast cancer. Radiology 1993;187:81–87. 21 Bland JDP, Weller P, Rudolfer S: Questionnaire tools for the diagnosis of carpal tunnel syndrome from the patient history. Muscle Nerve 2011;44:757–762. 22 Caudarella R, Tonello L, Rizzoli E, Vescini F: Predicting five-year recurrence rates of kidney stones: an artificial neural network model. Arch Ital Urol Androl 2011;83:14–19. 23 Dumont TM, Rughani AI, Tranmer BI: Prediction of symptomatic cerebral vasospasm after aneurysmal subarachnoid hemorrhage with an artificial neural network: feasibility and comparison with logistic regression models. World Neurosurg 2011;75:57–63.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
24 Lee T-T, Liu C-Y, Kuo Y-H, Mills ME, Fong J-G, Hung C: Application of data mining to the identification of critical factors in patient falls using a web-based reporting system. Int J Med Inform 2011;80:141–150. 25 Mantzaris D, Anastassopoulos G, Adamopoulos A: Genetic algorithm pruning of probabilistic neural networks in medical disease estimation. Neural Netw 2011;24:831–835. 26 Susin FM, Tarzia V, Bottio T, Pengo V, Bagno A, Gerosa G: In vitro detection of thrombotic formation on bileaflet mechanical heart valves. J Heart Valve Dis 2011;20:378–386. 27 Zoulias EA, Asvestas PA, Matsopoulos GK, Tseleni-Balafouta S: A decision support system for assisting fine needle aspiration diagnosis of thyroid malignancy. Anal Quant Cytol Histol 2011;33:215–222. 28 Bas B, Ozgonenel O, Ozden B, Bekcioglu B, Bulut E, Kurt M: Use of artificial neural network in differentiation of subgroups of temporomandibular internal derangements: a preliminary study. J Oral Maxillofac Surg 2012;70:51–59. 29 Saftoiu A, Vilmann P, Gorunescu F, Janssen J, Hocke M, Larsen M, Iglesias-Garcia J, Arcidiacono P, Will U, Giovannini M, Dietrich CF, Havre R, Gheorghe C, McKay C, Gheonea DI, Ciurea T, European EUSEMSG: Efficacy of an artificial neural network-based approach to endoscopic ultrasound elastography in diagnosis of focal pancreatic masses. Clin Gastroenterol Hepatol 2012;10:84–90. 30 http://en.wikipedia.org/wiki/Artificial_neural_network (accessed May 21st, 2012).
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
9 Hamlyn PJ: Neurovascular relationships in the posterior cranial fossa, with special reference to trigeminal neuralgia. 1. Review of the literature and development of a new method of vascular injection-filling in cadaveric controls. Clin Anat 1997;10:371–379. 10 Hamlyn PJ: Neurovascular relationships in the posterior cranial fossa, with special reference to trigeminal neuralgia. 2. Neurovascular compression of the trigeminal nerve in cadaveric controls and patients with trigeminal neuralgia: quantification and influence of method. Clin Anat 1997;10:380–388. 11 Hamlyn PJ, King TT: Neurovascular compression in trigeminal neuralgia: a clinical and anatomical study. J Neurosurg 1992; 76: 948– 954. 12 Hilton DA, Love S, Gradidge T, Coakham HB: Pathological findings associated with trigeminal neuralgia caused by vascular compression. Neurosurgery 1994;35:299–303. 13 Jannetta PJ: Arterial compression of the trigeminal nerve at the pons in patients with trigeminal neuralgia. J Neurosurg 1967; 26 (suppl):159–162. 14 Klun B, Prestor B: Microvascular relations of the trigeminal nerve: an anatomical study. Neurosurgery 1986;19:535–539. 15 Love S, Hilton DA, Coakham HB: Central demyelination of the Vth nerve root in trigeminal neuralgia associated with vascular compression. Brain Pathol 1998;8:1–11. 16 Limonadi FM, McCartney S, Burchiel KJ: Design of an artificial neural network for diagnosis of facial pain syndromes. Stereotact Funct Neurosurg 2006;84:212–220.
Received: November 8, 2012 Accepted after revision: May 7, 2013 Published online: November 8, 2013
Use of an Artificial Neural Network for Diagnosis of Facial Pain Syndromes: An Update Shirley McCartney a Markus Weltin b Kim J. Burchiel a a Department of Neurological Surgery, Oregon Health & Science University, and b Department of Academic and Research Computing, Portland State University, Portland, Oreg., USA
Abstract Background: Based on a classification scheme for facial pain syndromes and a binomial (yes/no) facial pain questionnaire, we previously reported on the ability of an artificial neural network (ANN) to recognize and correctly diagnose patients with different facial pain syndromes. Objectives: We now report on an updated questionnaire, the development of a secure web-based neural network application and details of ANNs trained to diagnose patients with different facial pain syndromes. Methods: Online facial pain questionnaire responses collected from 607 facial pain patients (395 female, 65%, ratio F/M 1.86/1) over 5 years and 7 months were used for ANN training. Results: Sensitivity and specificity of the currently running ANN for trigeminal neuralgia type 1 and trigeminal neuralgia type 2 are 92.4 and 62.5% and 87.8 and 96.4%, respectively. Sensitivity and specificity are 86.7 and 95.2% for trigeminal neuropathic pain, 0 and 100% for trigeminal deafferentation pain and 100% for symptomatic trigeminal neuralgia and postherpetic neuralgia. Sensitivity is 50% for nervus intermedius neuralgia (NIN) and 0% for atypical facial pain (AFP), glossopharyngeal neu-
© 2013 S. Karger AG, Basel 1011–6125/14/0921–0044$39.50/0 E-Mail [email protected] www.karger.com/sfn
ralgia (GPN) and temporomandibular joint disorder (TMJ). Specificity for AFP, NIN and TMJ is 99% and for GPN, 100%. Conclusions: We demonstrate the utilization of questionbased historical self-assessment responses used as inputs to design an ANN for the purpose of diagnosing facial pain syndromes (outputs) with high accuracy. © 2013 S. Karger AG, Basel
Introduction
The diagnostic term ‘facial pain’ includes an extensive list of clinical conditions such as headache and migraine syndromes, central deafferentation syndromes and cranial neuralgias such as trigeminal neuralgia (TN) [1–3]. The International Association for the Study of Pain defines TN as ‘sudden, usually unilateral, severe, brief, stabbing, recurrent episodes of pain in the distribution of one or more branches of the trigeminal nerve’ [4]. An important feature of facial pain that labels it as TN is a positive relief response to an anticonvulsant such as carbamazepine, gabapentin or phenytoin [5, 6]. This can be helpful in differentiating TN from other orofacial pain syndromes that typically do not respond, such as dental pain, sinusitis or temporomandibular disorders. The annual incidence of TN was reported in 1972 as 4.3 per 100,000 Shirley McCartney, PhD Department of Neurological Surgery, Oregon Health & Science University Mail code CH8N, 3303 SW Bond Ave Portland, OR 97239-3098 (USA) E-Mail mccartns @ ohsu.edu
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Key Words Trigeminal neuralgia · Facial pain · Neural networks · Artificial intelligence
Methods Facial Pain Questionnaire We have refined the original 18-question binomial (yes/no) questionnaire and added 4 questions (table 1). The questions are still formulated to provide a concise patient history; 2 questions were added (Q1 and 2) to specifically ask if patients had pain in their face and if they recalled the precise timing of that pain, and a further 2 questions (Q5 and 6) were added to account for the diagnoses of geniculate neuralgia or nervus intermedius neuralgia (NIN), and glossopharyngeal neuralgia (GPN), respectively. Diagnoses currently included as ANN outputs are listed in table 2. Facial Pain Patients All patients with facial pain who were seen by author K.J.B. at the Department of Neurological Surgery, Oregon Health & Science University (OHSU), were prospectively included. The study was
Artificial Neural Network and Facial Pain
Fig. 1. Example of ANN layers.
approved by the OHSU Institutional Review Board. Patient selfadministered response data were verified and confirmed when patients consented (by face-to-face interview and medical history review), and yes/no responses were prospectively collected for future training and refinement of ANNs. The actual diagnoses of the patients as determined by expert diagnosis (author K.J.B.) and documented in the electronic medical record were also collected. Neural Network Application An automated web-based application developed in PHP (a widely used open-source general-purpose scripting language) and C++ (an ‘object-oriented’ programming language created by Bjarne Stroustrup and released in 1985) and dynamically linked to an ANN that we developed was installed on an OHSU secure network. The neural network application is divided into three distinct sites, i.e. public, private and administrative. The public and private sites are similar; both present users with the latest version of the web-based questionnaire and yes/no responses are processed by an ANN to provide a diagnosis. The freely available public (self-diagnosis) site collects no data (https://neurosurgery. ohsu.edu/tgn.php). The private site involves a registration and login, and final collection of yes/no response data from consenting patients. The administration site controls all other aspects of the application. Artificial Neural Network We previously reported on an ANN designed with a highly distributed interconnection of adaptive nonlinear processing elements (PEs) to diagnose a patient’s facial pain type [16]. The current ANN implements a feed-forward neural network trained by the back propagation of error algorithm. This type of neural network is built in layers (fig. 1). Each layer consists of one or more PEs. The output of each PE serves as an input to all the PEs in the layer that precedes it. PEs connected in this manner are said to be ‘fully connected’.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
45
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
persons, with a higher incidence for women (5.9 per 100,000) compared to men (3.4 per 100,000) [7]. A UK study in 2006 reported annual incidence as high as 27 per 100,000 [8]. The exact nature of the mechanism of TN remains unknown; however, pathophysiology is thought to be due to vascular compression of the trigeminal nerve, generally related to age, when atherosclerosis can cause elongation of vessels and compression [9–15]. A patient’s history is central to a facial pain diagnosis and a diagnosis can, in almost every case, be made on history alone. We previously reported on the design of an artificial neural network (ANN) that was capable of recognizing a pattern in a condensed patient history derived from an 18-question self-administered binomial (yes/no) questionnaire [16] based on a classification of some facial pain syndromes [1, 3] that resulted in rendering a correct diagnosis of the type of facial pain syndrome. We now report on updates to the history-based questionnaire, the development of a web-based neural network application and the design of an ANN able to better recognize and correctly diagnose patients with different facial pain syndromes, in particular TN type 1 (TN1), trigeminal neuropathic pain (TNP), symptomatic TN (STN) and postherpetic neuralgia (PHN). The online diagnostic application (https://neurosurgery.ohsu.edu/tgn.php) is freely available to any patient and/or physician. We maintain that a questionnaire such as the one we previously developed and have now refined, in combination with a well-trained ANN, provides facial pain patients access to a highly accurate online self-diagnosis system. With improved diagnosis, better-educated patients will be able to avoid inappropriate treatment modalities and be directed to more appropriate resources.
Table 1. Facial pain questionnaire: TN – diagnostic questionnaire
Diagnostic questions
(4) (5) (6) (7) (8) (9) (10) (11) (12)
(13)
(14) (15) (16)
(17)
(18) (19) (20) (21) (22)
Do you have facial pain? Do you remember exactly where you were the moment your facial pain started? When you have pain, is it predominantly in your face (i.e. forehead, eye, cheek, nose, upper/lower jaw, teeth, lips, etc.)? Do you have pain just on one side of your face? When you have pain, is it predominantly deep in your ear? When you have pain, is it predominantly in the back of your throat or tongue, near the area of your tonsil? Is your pain either entirely or mostly brief (seconds to minutes) and unpredictable sensations (electrical, shocking, stabbing, shooting)? Do you have any constant background facial pain (e.g. aching, burning, throbbing, stinging)? Do you have constant background facial pain (aching, burning, throbbing, stinging) for more than half of your waking hours? Do you have any constant facial numbness? Can your pain start by something touching your face (e.g. by eating, washing your face, shaving, brushing teeth, wind)? Since your pain began have you ever experienced periods of weeks, months or years when you were painfree? (This would not include periods after any pain-relieving surgery or while you were on medications for your pain) Have you ever taken Tegretol® (carbamazepine), Neurontin® (gabapentin), Lioresal® (baclofen), Treleptal® (oxcarbazepine), Topamax® (topiramate), Zonegran® (zonisamide), or any other anticonvulsant medication for your pain? Did you ever experience any major reduction in facial pain (partial or complete) from taking any of the medications listed in Question 9, or any anticonvulsant medication? Have you ever had trigeminal nerve surgery for your pain? (e.g. neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife) Have you ever experienced any major reduction in facial pain (partial or complete) from trigeminal nerve surgery for your pain? (e.g. neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife) Did your current pain start only after trigeminal nerve surgery (neurectomy, RF rhizotomy/gangliolysis, glycerol injection, balloon compression, rhizotomy, MVD, gamma knife)? (If this is a recurrence of your original pain after a successful trigeminal nerve surgery, answer “no”) Did your pain start after facial zoster or “shingles” rash (herpes zoster – not to be confused with “fever blisters” around the mouth)? Do you have multiple sclerosis? Did your pain start after a facial injury? Did your pain start only after facial surgery (oral surgery, ENT surgery, plastic surgery)? When you place your index finger right in front of your ears on both sides at once and feel your jaw open and close does the area under your fingers on either side hurt?
This facial pain questionnaire was taken from the world-wide web universal resource locator: https://neurosurgery.ohsu.edu/ tgn.php. At OHSU’s Department of Neurological Surgery we have developed a helpful questionnaire for the diagnosis and treatment of patients suffering from various types of TN. Italicized questions (Q1 and 2 and Q5 and 6) were added to the original 18-question binomial (yes/no) questionnaire. Adapted from Limonadi et al. [16], 2006. Disclaimer: Any medical or surgical advice provided in this website, even if intended to be accurate to the best of our knowledge, should be discussed with your medical or surgical practi-
46
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
yes yes yes
no no no
yes yes yes yes
no no no no
yes yes
no no
yes yes
no no
yes
no
yes
no
yes
no
yes
no
yes
no
yes
no
yes
no
yes yes yes yes
no no no no
tioner. This website will serve to help direct you (the patient) to appropriate informational resources, and should not be considered a diagnosis. A diagnosis can only be given by an appropriate and experienced physician, after interviewing and examining you (the patient). Only your physician or surgeon knows what is best for you. Always seek direct advice from your physician before embarking on any treatment, medication or therapy. Copyright 2002–2013©, Oregon Health & Science University. This document and the information contained within the document are not to be used or reproduced without written consent from Kim Burchiel, MD.
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
(1) (2) (3)
Table 2. Facial pain diagnoses (outputs) used in ANN development
Diagnosis
Abbreviation
Trigeminal neuralgia, type 1 Trigeminal neuralgia, type 2
TN1 TN2
Trigeminal neuropathic pain
TNP
Trigeminal injury unintentional Facial pain resulting from unintentional injury to the trigeminal system from facial trauma, oral surgery, ear, nose and throat surgery, root injury from posterior fossa or skull base surgery, stroke
Trigeminal deafferentation pain
TDP
Trigeminal injury intentional Facial pain in a region of trigeminal numbness resulting from intentional injury to the trigeminal system from neurectomy, gangliolysis, rhizotomy, nucleotomy, tractotomy, or other denervating procedures
Symptomatic trigeminal neuralgia
STN
Pain resulting from multiple sclerosis
Postherpetic neuralgia
PHN
Pain resulting from trigeminal herpes zoster outbreak
AFP
Pain predominantly having a psychological rather than a physiological origin
Nervus intermedius neuralgia, sometimes known as geniculate neuralgia
NIN
Recurring attacks of severe pain deep in the ear that may spread to the ear canal, outer ear, mastoid or eye regions and involves the nervus intermedius, which is the somatic sensory branch of the 7th cranial nerve
Glossopharyngeal neuralgia
GPN
Recurring attacks of severe pain in the back of the throat, the area near the tonsils, the back of the tongue, and part of the ear and is believed to be caused by irritation of the 9th cranial nerve
Temporomandibular joint disorder
TMJ
Pain due primarily to temporomandibular joint dysfunction
Atypical facial
Facial pain of spontaneous onset of pain >50% episodic >50% constant
be diagnosed by history alone.
Table 3. Data sets available for ANN training (n = 813)
To maintain data integrity, the following 2 conditions apply. a Data that are entered multiple times by patients or patients whose verbal responses at clinic visit do not match responses checked in questionnaire are deemed not reliable and hence ‘unusable’. b Not defined by network; therefore, data not useable to train network
Artificial Neural Network and Facial Pain
Diagnosis in clinical chart
TN1 TN2 TNP TDP STN PHN AFP Not facial pain Occipital neuralgia NIN GPN Bilateral No diagnosis TMJ Anesthesia dolorosa Not TN Poststroke neuropathic facial pain
Number of cases defined by ANN
useable (n = 607)
unusable (n = 206)
yes yes yes yes yes yes yes no no yes yes no no yes no no no
379 75 81 9 35 10 5
27a 5a 11a 1a 1a 1a 4a 10b 9b 1a 2a 7b 26b 13a 7b 75b 3b
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
6 3 4
47
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
a Cannot
paina
History
and testing
Patients, n Male patients, n Female patients, n Facial pain diagnosis, n TN1 TN2 TNP TDP STN PHN AFP NIN GPN TMJ
Total data sets
Testing data sets
Training data sets
364 130 234
120 43 77
244 87 157
239 26 47 3 26 6 3 6 4 4
79 8 15 1 8 2 1 2 2 2
160 18 32 2 18 4 2 4 2 2
The randomly generated data sets (from the available 607 data sets; see table 2) were also randomly split, 1 for training the neural network and 1 for testing. Training involves finding the neural network that best minimizes error. Testing evaluates the neural network that best minimizes error, using a measure of accuracy.
ANN Training Testing and Evaluation A web-based interface allows an administrator to create or evolve a network based on input size (two input sizes are currently available: 18 and 22), hidden layer specifications, selected diagnosis (outputs) and additional hidden layers (fig. 1). Random training/testing sets from the available data sets are generated. The network is retrained and performance evaluated as a measure of evolution of fitness (how accurate the diagnosis represented by each individual data set is relative to the population of data sets; 200 evolutions is the default), a confusion matrix (the accuracy of correct diagnosis vs. actual diagnosis) and sensitivity/specificity (probability of the ANN to identify positive and negative results) analysis. Finally, if the generated and evaluated network is sensitive and specific for outputs it can be installed. This ANN then functions in private (data collected and verified, and patients consented) and public (no data collected) environments.
Results
Data Collection From December 2006 to July 2012 (5 years and 7 months), a total of 813 patients with facial pain consented and responded to the online facial pain questionnaire at the time of their initial clinic visit. Of the 813 consenting subjects, it was determined that 607 of the patient re48
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
sponses were ‘useable’ data sets. A breakdown of the reasoning behind data determined as ‘unusable’ (206 data sets) and ‘useable’ can be found in table 3. Of the 607 respondents, 395 were female (65%; ratio F/M 1.86/1) and the overall average age was 57.3 ± 14.99 years. Of the 607 patients with a complaint of facial pain who were seen in the clinic and using the classification system for facial pain syndromes previously described by Burchiel and Eller [1, 3] as a basis for the ANN output (i.e. diagnosis), 379 were diagnosed with TN1, 75 with TN type 2 (TN2), 81 with TNP, 9 with trigeminal deafferentation pain (TDP), 35 with STN, 10 with PHN, 5 with atypical facial pain (AFP), 6 with NIN, 3 with GPN and 4 with temporomandibular joint (TMJ) pain. Based on stored raw data (output, i.e. ANN diagnosis) and the physician’s diagnosis (as documented in the electronic medical record), which are stored in the neural network application database, the ANN matched the diagnosis of the physician in 529 of the 607 patients (87.1%). Train and Test Data The neural network application could be considered a dynamic, ever-evolving application which allows for retraining, testing and installation of a new ANN at any time or based on the number of data sets collected. The current ANN was retrained, tested and installed in July 2012. A summary of the randomly generated data used to train and test an ANN is shown in table 4. Sensitivity and specificity examples (with either 18 or 22 question inputs and 6, 7 or 10 diagnosis outputs) are presented in table 5. The current sensitivity and specificity of the ANN for TN1 are 92.4 and 87.8%, respectively, and 62.5 and 96.4%, respectively, for TN2. Example test data confirm that the ANNs are still not sensitive in diagnosing TN2 (range 44–89%) or TDP (range 0–100%), although the ANNs trend toward being more sensitive to a TN2 diagnosis (up to 89%) if TN1 is removed as diagnostic output. The current sensitivity and specificity of ANN for TDP are 0 and 100%, respectively, and 86.7 and 95.2% for TNP. Sensitivity and specificity for STN and PHN are 100%. Sensitivity is 50% for NIN and 0% for AFP, GPN and TMJ. Specificity is 99% for AFP, NIN and TMJ and 100% for GPN. A confusion matrix example showing the desired (correct) versus real output (diagnosis) of the ANN created with randomly generated data from the data that are ‘useable’ and currently running in our production environment is shown in table 6.
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Table 4. Randomly generated data sets for neural network training
Table 5. ANN (current network and example networks) sensitivity and specificity data for diagnoses: TN1, TN2, TNP, TDP, STM, PHN,
AFP, NIN, GPN and TMJ from randomly generated train and test data sets TN1
TN2
TNP
TDP
STN
PHN
AFP
NIN
GPN
TMJ
Current ANN (installed 07/24/12) sensitivity specificity
0.924 0.878
0.625 0.964
0.867 0.952
0 1
1 1
1 1
0 0.99
0.5 0.99
0 1
0 0.99
ANN example 1 specificity sensitivity
0.833 0.857
0.444 0.964
0.8 0.9143
0 0.9916
1 1
1 1
1 1
0.5 0.9831
0.5 0.966
0 1
ANN example 2 sensitivity specificity
0.961 0.861
0.667 0.9524
0.733 1
0 1
1 1
1 1
1 0.991
NA NA
NA NA
NA NA
ANN example 3 sensitivity specificity
NA NA
0.889 0.963
0.8 0.952
1 0.971
1 1
1 1
1 0.971
NA NA
NA NA
NA NA
ANN example 4 sensitivity specificity
0.894 0.914
0.5 0.940
0.889 0.9337
0 1
1 0.994
1 0.989
1 0.995
NA NA
NA NA
NA NA
ANN example 5 sensitivity specificity
NA NA
0.792 0.869
0.74 0.884
0.333 0.970
1 1
0.667 1
1 0.971
NA NA
NA NA
NA NA
diagnoses; ANN example 2: input size = 22 questions, output selection = 7 diagnoses; ANN example 3: input size = 18 questions, output selection = 7 diagnoses (TN1 not selected as diagnosis); ANN example 4: input size = 18 questions, output selection = 7 diagnoses; ANN example 5: input size = 18 questions, output selection = 6 diagnoses (TN1 not selected as diagnosis).
Input size determines which question responses are used for train/test data set selection (historically, an 18- or 22-question input is available; see table 1). Output selection determines which diagnoses are evolved as outputs. Current ANN (installed July 24, 2012): input size = 22 questions, output selection = 10 diagnoses; ANN example 1: input size = 22 questions, output selection = 10
Table 6. A confusion matrix example, which visually shows a representation of the accuracy of desired (correct) vs. actual output (diag-
nosis) of the ANN when challenged with randomly generated and tested data sets
Actual diagnosis
TN1 TN2 TNP TDP STN PHN AFP NIN GPN TMJ
TN1
TN2
TNP
TDP
STN
PHN
AFP
NIN
GPN
TMJ
73 2 3 0 0 0 0 1 0 0
1 5 2 0 0 0 0 0 0 0
1 1 13 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 0 0 8 0 0 0 0 0
0 0 0 0 0 2 0 0 0 0
0 1 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 1
2 0 0 0 0 0 0 0 0 0
1 0 1 0 0 0 1 1 0 0
Each column of the matrix represents the predicted/desired facial pain diagnosis and each row represents the actual defined facial pain diagnosis. In the above matrix, values in the diagonal are correctly classified and are italicized, e.g. 73 TN1 diagnoses in the test set were correctly classified as TN1. The off-diagonal row values represent misclassified diagnoses, e.g. 1, 1, 2 and 1 TN1 in the
test set were misclassified as TN2, TNP, GPN and TMJ, respectively. Off-diagonal column values represent diagnoses that were inaccurately included in a diagnosis class, e.g. 2, 3 and 1 TN1 in the test set were included as TN2, TNP and NIN, respectively. The above confusion matrix example represents the ANN currently running in production and was installed on July 24th, 2012.
Artificial Neural Network and Facial Pain
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
49
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Desired diagnosis
Application of Neural Network Modeling Since the original concept of neural computing in 1943 [17], the application of neural network modeling has been used in various disciplines for various tasks including game playing and decision making (chess), pattern recognition (face and object), sequence recognition (speech, handwriting, gestures), financial applications and medical diagnosis such as the presence of breast cancer based on mammography [18–20]. Many diverse applications in medicine have been developed and recent medical applications include carpal tunnel syndrome (based on history) [21], predicting recurrence of kidney stones [22], predicting symptomatic cerebral vasospasm after aneurysmal subarachnoid hemorrhage [23], identification of critical factors in patient falls [24], medical disease estimation [25], thrombotic formation in mechanical heart valves [26], decision making for fineneedle aspiration diagnosis of thyroid malignancy [27], transmandibular joint internal derangements [28], and endoscopic ultrasound in the diagnosis of pancreatic masses [29]. Artificial Neural Networks ANNs are a specific example of the broader concept of artificial intelligence. An ANN is composed of several interconnected layers of ‘neurons’ or PEs. In the ANN we demonstrate, each PE accepts one or more inputs (yes/no responses) and one output (diagnosis; fig. 1). The PEs are connected to the elements in the layers adjacent to them. Each connection is defined by a weight, which is a multiplier applied before the input to the PE. Weights become the actual memory of the network. The ‘learning’ takes place by presenting a network that is initially constructed with random values for weights as an input vector (in our case the 18 or 22 binary question inputs or historical facial pain features). The information is allowed to flow through the network and produce a diagnosis (table 2). The diagnosis produced by the network is compared to expert diagnosis (author K.J.B.). The difference or error between the network and expert diagnosis is used (by a complex algorithm) to make adjustments to the weight values such that the next time it is presented with that same input vector its output (diagnosis) will more likely be identical to the expert diagnosis. A more detailed description of the concepts of layers, interconnections and training algorithms can be found at wikipedia.org (search term ‘artificial neural network’) [30]. 50
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
Key Results Most facial pain syndromes can be categorized into distinct diagnoses as previously proposed [1, 3]. We first reported in 2006 that, using a specific classification of facial pain syndromes and a condensed facial pain history (yes/no responses to a facial pain questionnaire) in combination with an ANN, the ANN could correctly diagnose TN1 with fairly high sensitivity (84%) and specificity (83%) [16]. We now report on the development of a secure webbased neural network application and randomly trained and tested ANNs to accurately diagnose the facial pain syndrome of patients. Various ANNs were able to recognize a pattern in the answers to a facial pain questionnaire that resulted in an output of a correct diagnosis. Interpretation The current sensitivity and specificity of ANN for TN1 are 92.4 and 87.8%, respectively, which is an improvement over our previous results of 84 and 83%. As in our original publication the current ANN is less sensitive at determining an accurate TN2 diagnosis (62.5% sensitive), which could be considered a trend toward improvement over our previous result of 50% sensitivity. As previously reported, this could be because TN2 patients often describe having symptoms that were initially TN1like that evolve with time to symptoms more TN2-like, hence similar questions are checked as yes, making a distinction between TN1 and TN2 more difficult [16]. Sensitivity and specificity results for TNP did improve to 86.7 and 95.2% compared to our previous results of 35 and 85%, respectively. This could be because a larger number of data sets were available for training and testing than previously reported. Sensitivity results for TDP are poor at 0%. Sensitivity did trend higher if a diagnosis of TN1 was not included in the ANN output; however, this was not consistent (see table 5). It is possible that patients who have previously undergone a procedure to treat TN1 retain features of TN1 and similar questions are checked as yes, making a distinction between TN1 and TDP more difficult. Sensitivity and specificity results for STN and PHN are 100%. However, removing TN1 as a diagnosis with an 18-question input lowers the sensitivity of diagnosing PHN from 100 to 67%. We previously conceded that AFP presents the treating physician with complex diagnostic and treatment challenges and as in the previous study ANNs were not optimum for diagnosing AFP. With so few data sets available (5) for training and testing, the AFP results should be interpreted with caution. We predict an improvement over 50% sensitivMcCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
Discussion
ity for NIN, GPN and TMJ will be forthcoming with additional data sets. The ability of our ANN in determining the correct diagnosis of TN1 remains of most significance because it is the most treatable form of facial pain. In patients who are refractory to anticonvulsant medications or have side effects, different surgical approaches such as microvascular decompression, percutaneous rhizotomy (glycerol, radiofrequency), balloon compression, stereotactic radiosurgery, trigeminal stimulation or motor cortex stimulation can be performed. Avoiding unnecessary consultations, investigations, procedures or treatments is of costbenefit both fiscally and in terms of the patient’s overall health and well-being. Limitations ANNs depend on the population used to train them, which means they must be trained with a wide variety of data. A weakness of this study is the relatively large number of patients with a diagnosis of TN1 (65.4%). This reflects the referral base pattern at OHSU and as such does result in ‘biasing’ of the neural network program in favor of more commonly predicting a diagnosis of TN1. Removal of TN1 as a diagnosis clearly influences outcome, especially regarding the positive impact on TN2 and TDP and the negative impact on PHN. An important question is whether our proposed questionnaire is optimal for all of the facial pain diagnostic categories described other than TN1, TNP, PHN and STN. Until we have encountered and are able to include a larger number of consenting patients with TN2, TDP, AFP, NIN and GPN in the study, followed by ANN training and testing, a definitive answer to this question remains elusive. An additional weakness/bias that warrants acknowledgment is that the results of the ANN are compared to and built on the evaluation a single expert (K.J.B.). The ANN cannot be fully validated until results can be independently confirmed
(or not) by the collection of external data obtained from other centers and other facial pain experts. Of note, an additional surgeon at OHSU has begun to administer the questionnaire to their facial pain patient population. At a future date this data can be extracted and used for comparison and validation. Data collection, analysis, coordination of IRB approvals and data validation from external centers are an essential next step. A strength of the study lies in that the internet has changed patients’ views of the medical profession and the delivery and cost of their health care. A review of our server logs (a recent 5-year period) reveals that the public website (https://neurosurgery.ohsu.edu/tgn.php) served results approximately 16,000 times (ranging from twice a day to 12 times a day). However, since no data are collected, there is no way to authenticate use. An online selfdiagnosis tool such as we describe provides an avenue to become more actively involved in one’s own health care and highlights the importance of seeking appropriate medical assistance as quickly as possible.
Conclusion
We have refined a binomial facial pain questionnaire and developed a web-based neural network application and an ANN able to recognize and correctly diagnose patients with different facial pain syndromes. Sensitivity and specificity results are improved over our previous report, particularly for TN1, TNP, STN and PHN. Essentially, the facial pain questionnaire in combination with an ANN allows any facial pain patient to self-diagnose. Ultimately, this self-diagnosis tool allows for easy and early identification of TN1 patients suitable for surgical treatment and allows those with other facial pain syndromes to form a better-educated judgment on an appropriate course of treatment and use of resources.
References
Artificial Neural Network and Facial Pain
4 Merskey H, Bogduk N: Classification of Chronic Pain: Descriptions of Chronic Pain Syndromes and Definitions of Pain Terms, ed 2. Seattle, IASP, 1994. 5 Loeser J: Tic douloureux and atypical facial pain; in Wall P, Melzack R (eds): Textbook of Pain. London, Churchill Livingstone, 1994, pp 699–710.
6 Wiffen P, Collins S, McQuay H, Carroll D, Jadad A, Moore A: Anticonvulsant drugs for acute and chronic pain. Cochrane Database Syst Rev 2005:CD001133. 7 Yoshimasu F, Kurland LT, Elveback LR: Tic douloureux in Rochester, Minnesota, 1945– 1969. Neurology 1972;22:952–956. 8 Hall GC, Carroll D, Parry D, McQuay HJ: Epidemiology and treatment of neuropathic pain: the UK primary care perspective. Pain 2006;122:156–162.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
51
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
1 Eller J, Raslan A, Burchiel K: Trigeminal neuralgia: definition and classification. Neurosurg Focus 2005;18:E3. 2 Slavin KV, Burchiel KJ: Surgical options for facial pain; in Burchiel KJ (ed): Surgical Management of Pain. New York, Thieme, 2002, pp 849–864. 3 Burchiel KJ: A new classification for facial pain. Neurosurgery 2003;53:1164–1166.
52
17 McCulloch WS, Pitts W: A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 1943;5:115–133. 18 Parmeggiani D, Avenia N, Sanguinetti A, Ruggiero R, Docimo G, Siciliano M, Ambrosino P, Madonna I, Peltrini R, Parmeggiani U: Artificial intelligence against breast cancer (ANNES-BC Project). Ann Ital Chir 2012;83: 1–5. 19 Orr RK: Use of an artificial neural network to quantitate risk of malignancy for abnormal mammograms. Surgery 2001;129:459–466. 20 Wu Y, Giger ML, Doi K, Vyborny CJ, Schmidt RA, Metz CE: Artificial neural networks in mammography: application to decision making in the diagnosis of breast cancer. Radiology 1993;187:81–87. 21 Bland JDP, Weller P, Rudolfer S: Questionnaire tools for the diagnosis of carpal tunnel syndrome from the patient history. Muscle Nerve 2011;44:757–762. 22 Caudarella R, Tonello L, Rizzoli E, Vescini F: Predicting five-year recurrence rates of kidney stones: an artificial neural network model. Arch Ital Urol Androl 2011;83:14–19. 23 Dumont TM, Rughani AI, Tranmer BI: Prediction of symptomatic cerebral vasospasm after aneurysmal subarachnoid hemorrhage with an artificial neural network: feasibility and comparison with logistic regression models. World Neurosurg 2011;75:57–63.
Stereotact Funct Neurosurg 2014;92:44–52 DOI: 10.1159/000353188
24 Lee T-T, Liu C-Y, Kuo Y-H, Mills ME, Fong J-G, Hung C: Application of data mining to the identification of critical factors in patient falls using a web-based reporting system. Int J Med Inform 2011;80:141–150. 25 Mantzaris D, Anastassopoulos G, Adamopoulos A: Genetic algorithm pruning of probabilistic neural networks in medical disease estimation. Neural Netw 2011;24:831–835. 26 Susin FM, Tarzia V, Bottio T, Pengo V, Bagno A, Gerosa G: In vitro detection of thrombotic formation on bileaflet mechanical heart valves. J Heart Valve Dis 2011;20:378–386. 27 Zoulias EA, Asvestas PA, Matsopoulos GK, Tseleni-Balafouta S: A decision support system for assisting fine needle aspiration diagnosis of thyroid malignancy. Anal Quant Cytol Histol 2011;33:215–222. 28 Bas B, Ozgonenel O, Ozden B, Bekcioglu B, Bulut E, Kurt M: Use of artificial neural network in differentiation of subgroups of temporomandibular internal derangements: a preliminary study. J Oral Maxillofac Surg 2012;70:51–59. 29 Saftoiu A, Vilmann P, Gorunescu F, Janssen J, Hocke M, Larsen M, Iglesias-Garcia J, Arcidiacono P, Will U, Giovannini M, Dietrich CF, Havre R, Gheorghe C, McKay C, Gheonea DI, Ciurea T, European EUSEMSG: Efficacy of an artificial neural network-based approach to endoscopic ultrasound elastography in diagnosis of focal pancreatic masses. Clin Gastroenterol Hepatol 2012;10:84–90. 30 http://en.wikipedia.org/wiki/Artificial_neural_network (accessed May 21st, 2012).
McCartney /Weltin /Burchiel
Downloaded by: Lund University Libraries 130.235.136.21 - 5/12/2015 4:11:45 PM
9 Hamlyn PJ: Neurovascular relationships in the posterior cranial fossa, with special reference to trigeminal neuralgia. 1. Review of the literature and development of a new method of vascular injection-filling in cadaveric controls. Clin Anat 1997;10:371–379. 10 Hamlyn PJ: Neurovascular relationships in the posterior cranial fossa, with special reference to trigeminal neuralgia. 2. Neurovascular compression of the trigeminal nerve in cadaveric controls and patients with trigeminal neuralgia: quantification and influence of method. Clin Anat 1997;10:380–388. 11 Hamlyn PJ, King TT: Neurovascular compression in trigeminal neuralgia: a clinical and anatomical study. J Neurosurg 1992; 76: 948– 954. 12 Hilton DA, Love S, Gradidge T, Coakham HB: Pathological findings associated with trigeminal neuralgia caused by vascular compression. Neurosurgery 1994;35:299–303. 13 Jannetta PJ: Arterial compression of the trigeminal nerve at the pons in patients with trigeminal neuralgia. J Neurosurg 1967; 26 (suppl):159–162. 14 Klun B, Prestor B: Microvascular relations of the trigeminal nerve: an anatomical study. Neurosurgery 1986;19:535–539. 15 Love S, Hilton DA, Coakham HB: Central demyelination of the Vth nerve root in trigeminal neuralgia associated with vascular compression. Brain Pathol 1998;8:1–11. 16 Limonadi FM, McCartney S, Burchiel KJ: Design of an artificial neural network for diagnosis of facial pain syndromes. Stereotact Funct Neurosurg 2006;84:212–220.
