SPECIAL SECTION Original Article
R. Mack Harrell, MD, FACP, FACE, ECNU; David N. Bimston, MD, FACS, FSSO ABSTRACT Objective: The Afirma Gene Expression Classifier (GEC) molecular marker assay was developed for the purpose of improving surgical decision-making with indeterminate fine-needle aspiration (FNA) biopsies of thyroid nodules. In this paper, we analyze the performance of the GEC over 27 months in a community hospital-based thyroid surgery practice. Methods: We began using GEC and Thyroid Cytopathology Partners (TCP) exclusively for thyroid FNA analysis in January 2011, shortly after the Afirma GEC became commercially available. In this paper, we focus on patients with indeterminate FNA results and the outcomes of GEC analysis, with particular attention paid to the calculation of the negative predictive value (NPV) of the Afirma test. Results: We performed 645 FNAs in 519 patients over 27 months. Overall, 58 FNAs (9%) were read as indeterminate, with 36 of these classified as suspicious by GEC (62%), 20 characterized as GEC benign (34%), and 2 determined to be inadequate due to low mRNA content. Of the 36 suspicious GEC patients, 30 underwent thyroidectomy, and 21 of the 30 had malignant final pathology. Of the 20 benign GEC patients, 5 underwent thyroid surgery, and 2 were discovered to have malignancies. The NPV for the Afirma GEC in our practice environment was 89.6%.
Submitted for publication August 3, 2013 Accepted for publication October 24, 2013 From the Nova Southeastern University College of Osteopathic Medicine, Davie, Florida. Address correspondence to Dr. R. Mack Harrell, 3287 NW 56th Street, Boca Raton, FL 33496. E-mail: [email protected] Published as a Rapid Electronic Article in Press at http://www.endocrine practice.org on November 18, 2013. DOI:10.4158/EP13330.OR To purchase reprints of this article, please visit: www.aace.com/reprints. Copyright © 2014 AACE.
364 ENDOCRINE PRACTICE Vol 20 No. 4 April 2014
Conclusion: In a practice with a high incidence of thyroid cancer in patients with indeterminate FNAs (33% for our practice), the NPV of the Afirma GEC test may not be as robust as suggested in the existing literature. (Endocr Pract. 2014;20:364-369) Abbreviations AACE = American Association of Clinical Endocrinologists; AUS-FLUS = atypia of undetermined significance or follicular lesion of undetermined significance; FN = follicular neoplasm; FNA = fine-needle aspiration; GEC = gene expression classifier; NPV = negative predictive value; TCP = Thyroid Cytopathology Partners INTRODUCTION Since the 2004 landmark report by Kimura et al linking the BRAF mutation with papillary thyroid cancer (1), endocrinologists and endocrine surgeons have held great expectations that genetic analysis of thyroid fine needle aspiration biopsy specimens might decrease the number of unnecessary surgeries performed for patients with cytologically “indeterminate” thyroid nodules. While the presence of the V600E BRAF mutation proved to be highly predictive of papillary cancer, the fact that 40 to 60% of patients with a tissue diagnosis of papillary cancer do not carry a BRAF mutation has limited the diagnostic applicability of this genetic test (2). Researchers have subsequently attempted to identify other genetic mutations and translocations that could predict thyroid cancer. One after another, mutations in RAS and translocations in RET/PTC and PAX8/PPARγ have proven useful when present in patient cytological samples but unhelpful for patients whose thyroid cancers do not exhibit known genetic alterations (2). In 2010, Chudova et al took a different approach by analyzing the amplified transcriptome (mRNA) of fine-needle aspiration biopsy specimens (FNAs) from patients undergoing thyroid surgery, with the intention
365
of developing a gene expression test that could predict benignity. Mathematical analysis led to the development of a 142-gene cDNA Affymetrix cassette (Afirma GEC; Veracyte, Inc., South San Francisco, CA) capable of separating “benign” from “suspicious” FNA specimens in a small independent cohort of prospectively collected samples. Initial statistical analysis was promising, with a “benign” GEC result yielding a negative predictive value (NPV) of 96% (3). Further testing led to the 2012 Alexander et al paper that further validated the Afirma GEC in FNAs from 265 patients with cytologically indeterminate nodules, with the authors reporting an overall NPV of 94% in patients with pre-operative negative GECs (4). However, NPV has an inverse relationship with the overall prevalence of thyroid cancer in the indeterminate cytology group studied (5). We reasoned that the incidence of thyroid cancer in patient groups with indeterminate thyroid cytology might vary according to geographic region and type of thyroid cancer practice. Therefore, we chose to evaluate our south Florida (Memorial Center for Integrative Endocrine Surgery) Afirma GEC experience over the past 27 months and to calculate our own practicespecific NPV. In addition, we examined cytological and pathological idiosyncrasies in our patient group that might potentially affect GEC performance. METHODS From January 2011 through April 2013, all thyroid FNA cytology specimens from our south Florida endocrine surgery practice were sent to Thyroid Cytopathology Partners (TCP, Austin, TX) along with an Afirma washout specimen of RNA preserved in FNA-protect. Patients were selected for thyroid nodule FNA based on American Association of Clinical Endocrinologists (AACE) Thyroid Nodule Guideline criteria, and all biopsies were performed under GE Logiq E9 or S8 ultrasound (GE Healthcare, Waukesha, WI) guidance using a parallel-to-the-beam technique. Each nodule was sampled with three to five 25 gauge needle passes and slides were fixed in 95% ethanol and dried for transport. Each needle’s contents were washed into FNA-protect after the initial slide creation. After cytologic characterization at TCP, FNA-protect washes from all 58 indeterminate biopsies were sent to Veracyte for Afirma GEC testing. The nomenclature for indeterminate FNA result classification was divided into 2 groups including 1) atypia of undetermined significance or follicular lesion of undetermined significance (AUS-FLUS), either follicular or Hürthle cell predominant, and 2) follicular neoplasm (FN), either follicular or Hürthle cell predominant. Throughout the text of this report, the terms Hürthle cell and oncocyte are used interchangeably. FNAs read as “suspicious for malignancy” were not included in our indeterminate FNA analysis because
Veracyte has excluded them from Afirma GEC testing since the summer of 2012. Over the course of the study, only 4 FNAs were interpreted as “suspicious for malignancy” by TCP, and all proved to be cancers at surgery. All surgery was performed by 1 surgeon (D.N.B.), and the final pathologic reports were rendered by the Memorial Health Systems Department of Pathology in consultation with Dr. Virginia LiVolsi at the University of Pennsylvania, Dr. Jennifer Hunt at the University of Arkansas, and the Pathology Staff of the University of Florida in Gainesville. NPV was estimated using the standard equation for NPV and our data was presented using a conventional 2 × 2 table with pre-surgical Afirma GEC results, benign and suspicious, plotted versus clinically significant surgical pathologic diagnoses, benign and malignant (Table 1) (5). Four subcentimeter papillary microcarcinomas, contralateral from the actual nodule biopsied, were not considered to be clinically significant malignancies. RESULTS Nodule and Patient Characteristics We performed 645 FNAs in 519 patients over 27 months. The 58 biopsies read as indeterminate by TCP ranged in size from 0.6 to 5 cm, averaging 1.5 cm. We sampled 6 subcentimeter nodules that were found to have indeterminate cytology. These nodules were biopsied because of suspicious ultrasound findings or strong family histories of thyroid cancer and constituted only 10.3% of the entire indeterminate group. The 58 indeterminate nodules represented 9% of the total FNAs sent to TCP over the study time frame. Overall, 36 of these indeterminate biopsies were classified as suspicious by Afirma GEC (62%), 20 were characterized as GEC benign (34%) and 2 were determined to be inadequate due to low mRNA content (3%). The 58 patients whose thyroid FNAs were sent for Afirma GEC analysis averaged 53 years of age and were predominantly female (76%). The group’s ethnicity was largely Caucasian (67%) with Hispanic (16%), African American (14%), and Asian (3%) minority representation. Surgical Results Of the 36 suspicious GEC patients, 30 (83%) underwent thyroidectomy. Twenty-one of the 30 had malignant final pathologies, and 4 of the 21 patients with malignant final pathologic diagnoses had isolated coincidental subcentimeter papillary microcarcinomas in the contralateral thyroid lobe from their biopsied Afirma GEC-suspicious nodules (the false-positive GEC lesions ranged from 1.0 to 1.6 cm in size). Among the GEC-suspicious patients who did not undergo surgery, 5 refused surgical intervention, and 1 is in the process of completing chemotherapy for breast cancer prior to her thyroidectomy.
366
Table 1 Afirma GEC Results Versus Final Pathologic Results
Afirma GEC
Suspicious Benign n = 35
Final Pathologic Diagnosis Malignant Benign 17 13 1 4 Sensitivity 0.944444
Specificity 0.2352941
NPV PPV Max cancer prevalence (assuming 51.4% prevalence in indeterminates) 0.514a 0.800a 0.567a Min cancer prevalence (no further cancer in 22 unoperated patients) 0.33b 0.896b 0.378b a The
first assumption is that the cancer prevalence in the 22 unoperated patients will be identical to that seen in the 35 unoperated patients (51.4%) b The second assumption (best case scenario) assumes that there will be no further cancers detected in the 22 unoperated patients (33%).
Of the 20 benign GEC patients, 5 underwent thyroid surgery, and 2 were discovered to have malignancies. The false-negative results in these 2 malignant thyroid nodules were likely due to sampling error, with the first nodule being a 0.64-cm papillary cancer, and the second a 2.8cm cystic papillary cancer. Surgery was performed in the first patient because of a frankly malignant aspirate in the contralateral thyroid lobe and in the second case because of a positive family history and recent nodule growth. All GEC-benign patients were encouraged to return for yearly follow-up with sequential neck ultrasound evaluation. One of the 2 patients with non-diagnostic GECs (inadequate RNA) underwent lobectomy for a benign thyroid cyst. NPV Calculation In an effort to construct a “best-case scenario” for the NPV calculation of Afirma GEC in our practice, we excluded the 4 incidental and remote papillary microcarcinomas and assumed a minimum cancer prevalence of 17 cases in the Afirma-suspicious group and 2 cases in the Afirma-benign group, for a total of 19 cancers in 58 patients. Thus, our minimum cancer prevalence in the indeterminate FNA group was 33% (Table 1). This adjustment assumes that no more relevant thyroid cancers will be found in the 22 patients who have yet to undergo a thyroid surgical procedure. In addition, we counted the false-negative GEC 0.64-cm papillary microcarcinoma as a true negative. Technically, the nodule was too small to biopsy, and a large malignant lesion in the contralateral thyroid lobe drove our decision. Based on these assumptions, we then
Fig. 1. Plot of NPV/PPV versus malignancy prevalence in the Afirma GEC-tested group. NPV and PPV are shown in blue and red, respectively. Graph courtesy of Dr. Bryan McIver and the Moffitt Cancer Center. The “best-case scenario” thyroid cancer prevalence in our indeterminate cytology group was 33% with a calculated Afirma GEC NPV of 89.6% (white arrow). GEC = gene expression classifier; NPV = negative predictive value; PPV = positive predictive value.
calculated the “best case” NPV for Afirma in our practice to be 89.6% (Table 1, Fig. 1). If we project that a minimum of 2 more relevant cancers would be uncovered at final pathologic evaluation (consistent with the cancer incidence in the already operated group) from the 6 as yet unoperated GEC-suspicious patients, our indeterminate FNA cancer prevalence rises to 36%, and the NPV falls to 88.3%
367
FNA Cellular Typology Cytologically, 21 out of 58 (41%) of our indeterminate FNAs showed prominent Hürthle cell populations. In the Afirma GEC-benign group, only 2 of 20 aspirates were noted by the TCP cytologists to contain significant Hürthle cell (oncocytic) populations. The 18 non-oncocytic FNAs included 16 AUS-FLUS lesions featuring follicular cells with mild atypia and 2 FNs. In contrast, 19 out of 36 FNAs (53%) in the Afirma GEC-suspicious group showed oncocytic predominance, with 12 Hürthle cell-dominant AUS-FLUS FNAs and 7 oncocytic FNs. Note that biopsies demonstrating Hürthle cells in proximity to copious colloid and lymphocytes are not included in this discussion because TCP interprets such FNAs as benign by Bethesda criteria. Pathologic Correlates A total of 36 surgeries were performed in our 58 patients with indeterminate FNAs (30 on GEC-suspicious FNA patients, 5 on GEC-benign FNA patients, and 1 on a non-diagnostic GEC patient with insufficient RNA) (Table 2). Ten oncocytic tumors were diagnosed at pathological evaluation (10/36 patients, 28%) including 7 Hürthle cell adenomas, 2 multinodular goiters with Hürthle cell metaplasia, and 1 Hürthle cell carcinoma (Table 2). If we include the 4 contralateral thyroid lobe papillary microcarcinoma patients as false-positive GECs (Table 2), 9 of 13 false-positive Afirma GEC patients (69%) were subsequently determined to have benign oncocytic pathology. Thyroidectomy for patients with GEC-suspicious oncocytic FNAs demonstrated a clinically relevant cancer on final pathology in 6 out of 17 surgeries (35%, Table 2). Once again, for the purposes of this analysis, we classified 4 contralateral thyroid lobe incidental papillary microcarcinomas as benign (i.e., not clinically significant) based on the fact that these lesions are routinely present in 6 to 36% of the healthy adult population (6,7). Using these classification criteria, a patient with a suspicious, oncocytic Afirma GEC result has roughly the same risk for harboring a clinically relevant thyroid cancer as a patient with a Bethesda 4 FNA result. Thus, for a patient with an oncocytic indeterminate FNA, Afirma GEC adds little information to enhance surgical decision making. Thyroidectomy in patients with Afirma GECsuspicious, non-oncocytic, indeterminate FNAs produced a clinically relevant cancer on final pathology in 10 out 13 surgeries (77%, Table 2). Thus, the GEC-suspicious, non-oncocytic, indeterminate FNA group appeared to be enriched with relevant thyroid cancer. None of the 5 surgeries performed on patients with benign Afirma GECs produced an oncocytic tumor, nor did the surgery on the patient with a nondiagnostic GEC (Table 2).
DISCUSSION In this paper, we analyzed 27 successive months of Veracyte Afirma GEC use in our endocrine surgical practice and compared our GEC NPV to that reported by Alexander et al in a much larger study involving 49 sites (mostly non-university endocrinology practices) in 26 states over roughly the same duration. Unlike the participants in the Alexander et al study, we used TCP for all cytologic interpretation and we insisted on ethanol-fixation and air-drying of all our slides prior to transport. We did not use Thin-Prep liquid technology. Because the Papanicolaou (Pap) technique requires wet staining, our slides were not examined with Pap staining, and this could have resulted in Bethesda classification differences. In addition, we did not use 1 to 2 dedicated passes for Afirma GEC RNA collection, choosing instead to wash all our needle passes into FNA-protect solution after creating a cytology slide. This technique did not appear to compromise the quality of the RNA specimens we provided, as only 2 of 58 Afirma GEC’s (3%) were found to contain inadequate RNA. Indeterminate Group Cancer Prevalence and NPV While the sex distribution and average age of our Afirma patient population are similar to that described by Alexander et al, our cancer prevalence and final pathologic diagnosis distribution are different (4). The cancer prevalence in our indeterminate group was at least 33% (probably closer to 36% once all Afirma GEC-suspicious surgeries are complete), while the calculated cancer prevalence (with “suspicious for malignancy” cytology patients excluded) in the Alexander et al paper was almost 9% lower at 24.3%. The increased prevalence of thyroid cancer in our indeterminate group results in a significant reduction in the NPV of the Afirma GEC in our practice population. When cancer prevalence is plotted on a cancer prevalence versus NPV curve constructed from our practice sensitivity and specificity data (curve-fitting algorithm supplied by Dr. Bryan McIver of the Moffitt Cancer Center), our calculated NPV falls to 89.6% (Table 1, Figure 1). Thus, for our practice, more than 1 in 10 of our benign GEC Afirma patients are likely to be misclassified. Although our current data is limited by the fact that 15 out of 20 patients in our Afirma GEC-benign group have not yet undergone surgery, our experience still clearly demonstrates that the higher the prevalence of thyroid cancer in the tested population, the more likely that Afirma GEC will misclassify patients with malignant disease. Oncocytic Case Mix In another divergence from the Alexander et al data, our indeterminate FNA patient group demonstrated a
368
Table 2 Cytologic and Pathologic Correlates in 36 Patients Undergoing Surgery After Afirma GEC Pt # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
FNA Bethesda category 3 3 3 4 3 3 4-HC 3-HC 3-HC 3-HC 3-HC 4-HC 3-HC 4-HC 3 3-HC 4-HC 3 3 3 4 3 3 3 3 3-HC 3-HC 4-HC 3 3 3 4-HC 3-HC 3 3-HC 4-HC
GEC Dx Benign Benign Benign Benign Benign Inadequate RNA Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Susp/Medullary
Pathology Pap Ca- multifocal MNG MNG Pap Ca MNG Benign Cyst HC Carcinoma HC Adenoma MNG with HC changes Pap Ca- multifocal HC Adenoma HC Adenoma CLT w/ micro Pap HC Adenoma Pap Ca- multifocal Pap Ca HC Adenoma w/ micro Pap Pap Ca Pap Ca- Follicular Variant Pap Ca Pap Ca Pap Ca Pap Ca- multifocal Pap Ca Follicular nodule Pap Ca HC Adenoma w/ micro Pap MNG with HC changes Pap Ca- Follicular Variant Pap Ca- Follicular Variant CLT w/ micro Pap HC Adenoma Pap Ca- multifocal Benign reactive tissue Pap Ca- multifocal Medullary Carcinoma
Cancer stagea Stage 3 … … Stage 2 … … Stage 2 … … Stage 1 … … … … Stage 3 Stage 3 … Stage 3 Stage 2 Stage 2 Stage 4A Stage 4A Stage 1 Stage 1 … Stage 3 … … Stage 1 Stage 3 … … Stage 1 … Stage 3 Stage 4A
Abbreviations: CLT= chronic lymphocytic thyroiditis; FNA = fine-needle aspiration; HC = Hürthle cell; MNG = multinodular goiter; Pap Ca = papillary carcinoma Color coding: blue cells = benign Afirma GEC; gray cell = inadequate RNA GEC; yellow cells = suspicious GEC; green cells = suspicious medullary GEC; white cells = false-positive Afirma GECs; black cells = oncocytic final pathologic diagnosis; red text = incidental opposite lobe papillary microcarcinoma a All Stage 1 thyroid cancers are T1N0M0
369
considerably higher percentage of oncocytic final pathologic diagnoses (28%) than did the multicenter trial (12%) (4). Although the reason for our plethora of Hürthle celldominant final diagnoses is unclear, it is apparent that Afirma technology disproportionately distributes oncocytic indeterminate lesions into the suspicious GEC category (19 out of 21 oncocytic biopsies were classified as Afirma GEC suspicious) and that most of these Hürthle cell lesions are benign on final pathologic evaluation (9 of 13 of the GEC false-positive nodules contained dominant oncocytic cell populations on final pathology, including 7 Hürthle cell adenomas and 2 multinodular goiters with Hürthle cell metaplasia (Table 2). It is possible that the high concentration of oncocytic aspirates in our indeterminate specimens (21/58 or 36.2%) could have affected Afirma performance; 90% of our oncocytic aspirates (19 out of 21) were subsequently read as GEC “suspicious.” Alexander et al (4) did not provide a breakdown of their indeterminate cytologies into oncocytic and non-oncocytic subsets, so we cannot directly compare our data to theirs. However, the fact that our prevalence of oncocytic final pathology was more than double theirs (28% vs. 12%) suggests that that our indeterminate cytologic cell type distribution was likely different from theirs. Historically, Hürthle cell adenomas and carcinomas have been cytologically indistinguishable. Recent genome deep sequencing studies provide evidence that Hürthle cell carcinomas are genetically unique thyroid malignancies, distinct from papillary thyroid and follicular thyroid cancers (8). Given the infrequency of Hürthle cell malignancies, it is possible that Afirma GEC is not adequately trained to differentiate benign from malignant oncocytic indeterminate FNAs. Although novel genetic Hürthle cell markers have been described (8,9), no commercially available genetic test has been shown to reliably distinguish benign from malignant oncocytic tumors.
or estimate practice-specific thyroid cancer prevalence rates in their indeterminate cytology patient groups. Using such cancer prevalence data, a practice-based re-estimation of the NPV of a benign Afirma GEC test may prove to be informative. In endocrine surgical practices like ours, the NPV of a benign Afirma GEC test may not be as robust as suggested by Alexander et al (4). The problem of Hürthle cell-rich thyroid aspirates that do not contain copious colloid or cellular evidence for chronic thyroiditis remains a vexing one. Commercially available genetic technology seems incapable of distinguishing benign from malignant Hürthle cell lesions. Traditionally, oncocytic indeterminate cytology patients have been sent to surgery for diagnosis. Over our first 27 months of Afirma GEC use, 19 of 21 patients with indeterminate oncocytic cytologies were deemed suspicious by Afirma and sent for surgical diagnosis. Thus, in our practice, Afirma GEC added little information to the surgical selection process in patients with Hürthle cell-dominant cytopathology. DISCLOSURE Dr. Harrell is a member of the LabCorp Endocrine Advisory Board. Dr. Bimston has no multiplicity of interest to disclose. REFERENCES 1.
2. 3.
CONCLUSION Our 27-month study of Afirma GEC reveals that the NPV for this thyroid surgery-preventing test diminishes considerably in populations where the indeterminate cytology group (excluding suspicious for malignancy patients) contains more than 1 in 4 patients with thyroid cancer. Our study is limited by a relatively small sample size and by the fact that 22 of our patients with indeterminate FNAs (17 in the Afirma GEC-benign group) have not undergone surgery. With such a limited sample size, a small number of misclassified cancers by Afirma testing can dramatically alter the sensitivity and specificity calculations for the GEC test and secondarily change NPV calculations. However, we encourage all thyroidologists and thyroid surgeons who use Afirma GEC technology to calculate
4. 5. 6. 7. 8. 9.
Kimura ET, Nikiforova MN, Zhu Z, Knauf JA, Nikiforov YE, Fagin JA. High prevalence of BRAF mutations in thyroid cancer: genetic evidence for constitutive activation of the RET/PTC-RAS-BRAF signaling pathway in papillary thyroid carcinoma. Cancer Res. 2003;63:1454-1457. Nikiforova MN, Nikiforov YE. Molecular diagnostics and predictors in thyroid cancer. Thyroid. 2009;19:1351-1361. Chudova D, Wilde JI, Wang, ET, et al. Molecular classification of thyroid nodules using the high-dimensionality genomic data. Clin Endocrinol Metab. 2010;95:5296-5304. Alexander EK, Kennedy GC, Baloch ZW, et al. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med. 367:705-715. Fletcher, RH. Clinical Epidemiology: The Essentials. 4th ed. Philadephia, PA: Lippincott Williams & Wilkins;2005. Lang W, Borrusch H, Bauer L. Occult carcinomas of the thyroid: Evaluation of 1,020 sequential autopsies. Am J Clin Pathol. 1988:90:72-76. Harach H, Franssila K, Wasenius V. Occult papillary carcinoma of the thyroid: A “normal” finding in Finland. Cancer. 1985;3:531-538. Ganly I, Ricarte Filho J, Eng S, et al. Genomic dissection of Hurthle cell carcinoma reveals a unique class of thyroid malignancy. J Clin Endocrinol Metab. 2013;98:E962-972. Cerutti, JM, Oler, G, Delcelo R, et al. PVALB, a new Hürthle adenoma diagnostic marker identified through gene expression. J Clin Endocrinol Metab. 2011;96:E151-160.
R. Mack Harrell, MD, FACP, FACE, ECNU; David N. Bimston, MD, FACS, FSSO ABSTRACT Objective: The Afirma Gene Expression Classifier (GEC) molecular marker assay was developed for the purpose of improving surgical decision-making with indeterminate fine-needle aspiration (FNA) biopsies of thyroid nodules. In this paper, we analyze the performance of the GEC over 27 months in a community hospital-based thyroid surgery practice. Methods: We began using GEC and Thyroid Cytopathology Partners (TCP) exclusively for thyroid FNA analysis in January 2011, shortly after the Afirma GEC became commercially available. In this paper, we focus on patients with indeterminate FNA results and the outcomes of GEC analysis, with particular attention paid to the calculation of the negative predictive value (NPV) of the Afirma test. Results: We performed 645 FNAs in 519 patients over 27 months. Overall, 58 FNAs (9%) were read as indeterminate, with 36 of these classified as suspicious by GEC (62%), 20 characterized as GEC benign (34%), and 2 determined to be inadequate due to low mRNA content. Of the 36 suspicious GEC patients, 30 underwent thyroidectomy, and 21 of the 30 had malignant final pathology. Of the 20 benign GEC patients, 5 underwent thyroid surgery, and 2 were discovered to have malignancies. The NPV for the Afirma GEC in our practice environment was 89.6%.
Submitted for publication August 3, 2013 Accepted for publication October 24, 2013 From the Nova Southeastern University College of Osteopathic Medicine, Davie, Florida. Address correspondence to Dr. R. Mack Harrell, 3287 NW 56th Street, Boca Raton, FL 33496. E-mail: [email protected] Published as a Rapid Electronic Article in Press at http://www.endocrine practice.org on November 18, 2013. DOI:10.4158/EP13330.OR To purchase reprints of this article, please visit: www.aace.com/reprints. Copyright © 2014 AACE.
364 ENDOCRINE PRACTICE Vol 20 No. 4 April 2014
Conclusion: In a practice with a high incidence of thyroid cancer in patients with indeterminate FNAs (33% for our practice), the NPV of the Afirma GEC test may not be as robust as suggested in the existing literature. (Endocr Pract. 2014;20:364-369) Abbreviations AACE = American Association of Clinical Endocrinologists; AUS-FLUS = atypia of undetermined significance or follicular lesion of undetermined significance; FN = follicular neoplasm; FNA = fine-needle aspiration; GEC = gene expression classifier; NPV = negative predictive value; TCP = Thyroid Cytopathology Partners INTRODUCTION Since the 2004 landmark report by Kimura et al linking the BRAF mutation with papillary thyroid cancer (1), endocrinologists and endocrine surgeons have held great expectations that genetic analysis of thyroid fine needle aspiration biopsy specimens might decrease the number of unnecessary surgeries performed for patients with cytologically “indeterminate” thyroid nodules. While the presence of the V600E BRAF mutation proved to be highly predictive of papillary cancer, the fact that 40 to 60% of patients with a tissue diagnosis of papillary cancer do not carry a BRAF mutation has limited the diagnostic applicability of this genetic test (2). Researchers have subsequently attempted to identify other genetic mutations and translocations that could predict thyroid cancer. One after another, mutations in RAS and translocations in RET/PTC and PAX8/PPARγ have proven useful when present in patient cytological samples but unhelpful for patients whose thyroid cancers do not exhibit known genetic alterations (2). In 2010, Chudova et al took a different approach by analyzing the amplified transcriptome (mRNA) of fine-needle aspiration biopsy specimens (FNAs) from patients undergoing thyroid surgery, with the intention
365
of developing a gene expression test that could predict benignity. Mathematical analysis led to the development of a 142-gene cDNA Affymetrix cassette (Afirma GEC; Veracyte, Inc., South San Francisco, CA) capable of separating “benign” from “suspicious” FNA specimens in a small independent cohort of prospectively collected samples. Initial statistical analysis was promising, with a “benign” GEC result yielding a negative predictive value (NPV) of 96% (3). Further testing led to the 2012 Alexander et al paper that further validated the Afirma GEC in FNAs from 265 patients with cytologically indeterminate nodules, with the authors reporting an overall NPV of 94% in patients with pre-operative negative GECs (4). However, NPV has an inverse relationship with the overall prevalence of thyroid cancer in the indeterminate cytology group studied (5). We reasoned that the incidence of thyroid cancer in patient groups with indeterminate thyroid cytology might vary according to geographic region and type of thyroid cancer practice. Therefore, we chose to evaluate our south Florida (Memorial Center for Integrative Endocrine Surgery) Afirma GEC experience over the past 27 months and to calculate our own practicespecific NPV. In addition, we examined cytological and pathological idiosyncrasies in our patient group that might potentially affect GEC performance. METHODS From January 2011 through April 2013, all thyroid FNA cytology specimens from our south Florida endocrine surgery practice were sent to Thyroid Cytopathology Partners (TCP, Austin, TX) along with an Afirma washout specimen of RNA preserved in FNA-protect. Patients were selected for thyroid nodule FNA based on American Association of Clinical Endocrinologists (AACE) Thyroid Nodule Guideline criteria, and all biopsies were performed under GE Logiq E9 or S8 ultrasound (GE Healthcare, Waukesha, WI) guidance using a parallel-to-the-beam technique. Each nodule was sampled with three to five 25 gauge needle passes and slides were fixed in 95% ethanol and dried for transport. Each needle’s contents were washed into FNA-protect after the initial slide creation. After cytologic characterization at TCP, FNA-protect washes from all 58 indeterminate biopsies were sent to Veracyte for Afirma GEC testing. The nomenclature for indeterminate FNA result classification was divided into 2 groups including 1) atypia of undetermined significance or follicular lesion of undetermined significance (AUS-FLUS), either follicular or Hürthle cell predominant, and 2) follicular neoplasm (FN), either follicular or Hürthle cell predominant. Throughout the text of this report, the terms Hürthle cell and oncocyte are used interchangeably. FNAs read as “suspicious for malignancy” were not included in our indeterminate FNA analysis because
Veracyte has excluded them from Afirma GEC testing since the summer of 2012. Over the course of the study, only 4 FNAs were interpreted as “suspicious for malignancy” by TCP, and all proved to be cancers at surgery. All surgery was performed by 1 surgeon (D.N.B.), and the final pathologic reports were rendered by the Memorial Health Systems Department of Pathology in consultation with Dr. Virginia LiVolsi at the University of Pennsylvania, Dr. Jennifer Hunt at the University of Arkansas, and the Pathology Staff of the University of Florida in Gainesville. NPV was estimated using the standard equation for NPV and our data was presented using a conventional 2 × 2 table with pre-surgical Afirma GEC results, benign and suspicious, plotted versus clinically significant surgical pathologic diagnoses, benign and malignant (Table 1) (5). Four subcentimeter papillary microcarcinomas, contralateral from the actual nodule biopsied, were not considered to be clinically significant malignancies. RESULTS Nodule and Patient Characteristics We performed 645 FNAs in 519 patients over 27 months. The 58 biopsies read as indeterminate by TCP ranged in size from 0.6 to 5 cm, averaging 1.5 cm. We sampled 6 subcentimeter nodules that were found to have indeterminate cytology. These nodules were biopsied because of suspicious ultrasound findings or strong family histories of thyroid cancer and constituted only 10.3% of the entire indeterminate group. The 58 indeterminate nodules represented 9% of the total FNAs sent to TCP over the study time frame. Overall, 36 of these indeterminate biopsies were classified as suspicious by Afirma GEC (62%), 20 were characterized as GEC benign (34%) and 2 were determined to be inadequate due to low mRNA content (3%). The 58 patients whose thyroid FNAs were sent for Afirma GEC analysis averaged 53 years of age and were predominantly female (76%). The group’s ethnicity was largely Caucasian (67%) with Hispanic (16%), African American (14%), and Asian (3%) minority representation. Surgical Results Of the 36 suspicious GEC patients, 30 (83%) underwent thyroidectomy. Twenty-one of the 30 had malignant final pathologies, and 4 of the 21 patients with malignant final pathologic diagnoses had isolated coincidental subcentimeter papillary microcarcinomas in the contralateral thyroid lobe from their biopsied Afirma GEC-suspicious nodules (the false-positive GEC lesions ranged from 1.0 to 1.6 cm in size). Among the GEC-suspicious patients who did not undergo surgery, 5 refused surgical intervention, and 1 is in the process of completing chemotherapy for breast cancer prior to her thyroidectomy.
366
Table 1 Afirma GEC Results Versus Final Pathologic Results
Afirma GEC
Suspicious Benign n = 35
Final Pathologic Diagnosis Malignant Benign 17 13 1 4 Sensitivity 0.944444
Specificity 0.2352941
NPV PPV Max cancer prevalence (assuming 51.4% prevalence in indeterminates) 0.514a 0.800a 0.567a Min cancer prevalence (no further cancer in 22 unoperated patients) 0.33b 0.896b 0.378b a The
first assumption is that the cancer prevalence in the 22 unoperated patients will be identical to that seen in the 35 unoperated patients (51.4%) b The second assumption (best case scenario) assumes that there will be no further cancers detected in the 22 unoperated patients (33%).
Of the 20 benign GEC patients, 5 underwent thyroid surgery, and 2 were discovered to have malignancies. The false-negative results in these 2 malignant thyroid nodules were likely due to sampling error, with the first nodule being a 0.64-cm papillary cancer, and the second a 2.8cm cystic papillary cancer. Surgery was performed in the first patient because of a frankly malignant aspirate in the contralateral thyroid lobe and in the second case because of a positive family history and recent nodule growth. All GEC-benign patients were encouraged to return for yearly follow-up with sequential neck ultrasound evaluation. One of the 2 patients with non-diagnostic GECs (inadequate RNA) underwent lobectomy for a benign thyroid cyst. NPV Calculation In an effort to construct a “best-case scenario” for the NPV calculation of Afirma GEC in our practice, we excluded the 4 incidental and remote papillary microcarcinomas and assumed a minimum cancer prevalence of 17 cases in the Afirma-suspicious group and 2 cases in the Afirma-benign group, for a total of 19 cancers in 58 patients. Thus, our minimum cancer prevalence in the indeterminate FNA group was 33% (Table 1). This adjustment assumes that no more relevant thyroid cancers will be found in the 22 patients who have yet to undergo a thyroid surgical procedure. In addition, we counted the false-negative GEC 0.64-cm papillary microcarcinoma as a true negative. Technically, the nodule was too small to biopsy, and a large malignant lesion in the contralateral thyroid lobe drove our decision. Based on these assumptions, we then
Fig. 1. Plot of NPV/PPV versus malignancy prevalence in the Afirma GEC-tested group. NPV and PPV are shown in blue and red, respectively. Graph courtesy of Dr. Bryan McIver and the Moffitt Cancer Center. The “best-case scenario” thyroid cancer prevalence in our indeterminate cytology group was 33% with a calculated Afirma GEC NPV of 89.6% (white arrow). GEC = gene expression classifier; NPV = negative predictive value; PPV = positive predictive value.
calculated the “best case” NPV for Afirma in our practice to be 89.6% (Table 1, Fig. 1). If we project that a minimum of 2 more relevant cancers would be uncovered at final pathologic evaluation (consistent with the cancer incidence in the already operated group) from the 6 as yet unoperated GEC-suspicious patients, our indeterminate FNA cancer prevalence rises to 36%, and the NPV falls to 88.3%
367
FNA Cellular Typology Cytologically, 21 out of 58 (41%) of our indeterminate FNAs showed prominent Hürthle cell populations. In the Afirma GEC-benign group, only 2 of 20 aspirates were noted by the TCP cytologists to contain significant Hürthle cell (oncocytic) populations. The 18 non-oncocytic FNAs included 16 AUS-FLUS lesions featuring follicular cells with mild atypia and 2 FNs. In contrast, 19 out of 36 FNAs (53%) in the Afirma GEC-suspicious group showed oncocytic predominance, with 12 Hürthle cell-dominant AUS-FLUS FNAs and 7 oncocytic FNs. Note that biopsies demonstrating Hürthle cells in proximity to copious colloid and lymphocytes are not included in this discussion because TCP interprets such FNAs as benign by Bethesda criteria. Pathologic Correlates A total of 36 surgeries were performed in our 58 patients with indeterminate FNAs (30 on GEC-suspicious FNA patients, 5 on GEC-benign FNA patients, and 1 on a non-diagnostic GEC patient with insufficient RNA) (Table 2). Ten oncocytic tumors were diagnosed at pathological evaluation (10/36 patients, 28%) including 7 Hürthle cell adenomas, 2 multinodular goiters with Hürthle cell metaplasia, and 1 Hürthle cell carcinoma (Table 2). If we include the 4 contralateral thyroid lobe papillary microcarcinoma patients as false-positive GECs (Table 2), 9 of 13 false-positive Afirma GEC patients (69%) were subsequently determined to have benign oncocytic pathology. Thyroidectomy for patients with GEC-suspicious oncocytic FNAs demonstrated a clinically relevant cancer on final pathology in 6 out of 17 surgeries (35%, Table 2). Once again, for the purposes of this analysis, we classified 4 contralateral thyroid lobe incidental papillary microcarcinomas as benign (i.e., not clinically significant) based on the fact that these lesions are routinely present in 6 to 36% of the healthy adult population (6,7). Using these classification criteria, a patient with a suspicious, oncocytic Afirma GEC result has roughly the same risk for harboring a clinically relevant thyroid cancer as a patient with a Bethesda 4 FNA result. Thus, for a patient with an oncocytic indeterminate FNA, Afirma GEC adds little information to enhance surgical decision making. Thyroidectomy in patients with Afirma GECsuspicious, non-oncocytic, indeterminate FNAs produced a clinically relevant cancer on final pathology in 10 out 13 surgeries (77%, Table 2). Thus, the GEC-suspicious, non-oncocytic, indeterminate FNA group appeared to be enriched with relevant thyroid cancer. None of the 5 surgeries performed on patients with benign Afirma GECs produced an oncocytic tumor, nor did the surgery on the patient with a nondiagnostic GEC (Table 2).
DISCUSSION In this paper, we analyzed 27 successive months of Veracyte Afirma GEC use in our endocrine surgical practice and compared our GEC NPV to that reported by Alexander et al in a much larger study involving 49 sites (mostly non-university endocrinology practices) in 26 states over roughly the same duration. Unlike the participants in the Alexander et al study, we used TCP for all cytologic interpretation and we insisted on ethanol-fixation and air-drying of all our slides prior to transport. We did not use Thin-Prep liquid technology. Because the Papanicolaou (Pap) technique requires wet staining, our slides were not examined with Pap staining, and this could have resulted in Bethesda classification differences. In addition, we did not use 1 to 2 dedicated passes for Afirma GEC RNA collection, choosing instead to wash all our needle passes into FNA-protect solution after creating a cytology slide. This technique did not appear to compromise the quality of the RNA specimens we provided, as only 2 of 58 Afirma GEC’s (3%) were found to contain inadequate RNA. Indeterminate Group Cancer Prevalence and NPV While the sex distribution and average age of our Afirma patient population are similar to that described by Alexander et al, our cancer prevalence and final pathologic diagnosis distribution are different (4). The cancer prevalence in our indeterminate group was at least 33% (probably closer to 36% once all Afirma GEC-suspicious surgeries are complete), while the calculated cancer prevalence (with “suspicious for malignancy” cytology patients excluded) in the Alexander et al paper was almost 9% lower at 24.3%. The increased prevalence of thyroid cancer in our indeterminate group results in a significant reduction in the NPV of the Afirma GEC in our practice population. When cancer prevalence is plotted on a cancer prevalence versus NPV curve constructed from our practice sensitivity and specificity data (curve-fitting algorithm supplied by Dr. Bryan McIver of the Moffitt Cancer Center), our calculated NPV falls to 89.6% (Table 1, Figure 1). Thus, for our practice, more than 1 in 10 of our benign GEC Afirma patients are likely to be misclassified. Although our current data is limited by the fact that 15 out of 20 patients in our Afirma GEC-benign group have not yet undergone surgery, our experience still clearly demonstrates that the higher the prevalence of thyroid cancer in the tested population, the more likely that Afirma GEC will misclassify patients with malignant disease. Oncocytic Case Mix In another divergence from the Alexander et al data, our indeterminate FNA patient group demonstrated a
368
Table 2 Cytologic and Pathologic Correlates in 36 Patients Undergoing Surgery After Afirma GEC Pt # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
FNA Bethesda category 3 3 3 4 3 3 4-HC 3-HC 3-HC 3-HC 3-HC 4-HC 3-HC 4-HC 3 3-HC 4-HC 3 3 3 4 3 3 3 3 3-HC 3-HC 4-HC 3 3 3 4-HC 3-HC 3 3-HC 4-HC
GEC Dx Benign Benign Benign Benign Benign Inadequate RNA Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Suspicious Susp/Medullary
Pathology Pap Ca- multifocal MNG MNG Pap Ca MNG Benign Cyst HC Carcinoma HC Adenoma MNG with HC changes Pap Ca- multifocal HC Adenoma HC Adenoma CLT w/ micro Pap HC Adenoma Pap Ca- multifocal Pap Ca HC Adenoma w/ micro Pap Pap Ca Pap Ca- Follicular Variant Pap Ca Pap Ca Pap Ca Pap Ca- multifocal Pap Ca Follicular nodule Pap Ca HC Adenoma w/ micro Pap MNG with HC changes Pap Ca- Follicular Variant Pap Ca- Follicular Variant CLT w/ micro Pap HC Adenoma Pap Ca- multifocal Benign reactive tissue Pap Ca- multifocal Medullary Carcinoma
Cancer stagea Stage 3 … … Stage 2 … … Stage 2 … … Stage 1 … … … … Stage 3 Stage 3 … Stage 3 Stage 2 Stage 2 Stage 4A Stage 4A Stage 1 Stage 1 … Stage 3 … … Stage 1 Stage 3 … … Stage 1 … Stage 3 Stage 4A
Abbreviations: CLT= chronic lymphocytic thyroiditis; FNA = fine-needle aspiration; HC = Hürthle cell; MNG = multinodular goiter; Pap Ca = papillary carcinoma Color coding: blue cells = benign Afirma GEC; gray cell = inadequate RNA GEC; yellow cells = suspicious GEC; green cells = suspicious medullary GEC; white cells = false-positive Afirma GECs; black cells = oncocytic final pathologic diagnosis; red text = incidental opposite lobe papillary microcarcinoma a All Stage 1 thyroid cancers are T1N0M0
369
considerably higher percentage of oncocytic final pathologic diagnoses (28%) than did the multicenter trial (12%) (4). Although the reason for our plethora of Hürthle celldominant final diagnoses is unclear, it is apparent that Afirma technology disproportionately distributes oncocytic indeterminate lesions into the suspicious GEC category (19 out of 21 oncocytic biopsies were classified as Afirma GEC suspicious) and that most of these Hürthle cell lesions are benign on final pathologic evaluation (9 of 13 of the GEC false-positive nodules contained dominant oncocytic cell populations on final pathology, including 7 Hürthle cell adenomas and 2 multinodular goiters with Hürthle cell metaplasia (Table 2). It is possible that the high concentration of oncocytic aspirates in our indeterminate specimens (21/58 or 36.2%) could have affected Afirma performance; 90% of our oncocytic aspirates (19 out of 21) were subsequently read as GEC “suspicious.” Alexander et al (4) did not provide a breakdown of their indeterminate cytologies into oncocytic and non-oncocytic subsets, so we cannot directly compare our data to theirs. However, the fact that our prevalence of oncocytic final pathology was more than double theirs (28% vs. 12%) suggests that that our indeterminate cytologic cell type distribution was likely different from theirs. Historically, Hürthle cell adenomas and carcinomas have been cytologically indistinguishable. Recent genome deep sequencing studies provide evidence that Hürthle cell carcinomas are genetically unique thyroid malignancies, distinct from papillary thyroid and follicular thyroid cancers (8). Given the infrequency of Hürthle cell malignancies, it is possible that Afirma GEC is not adequately trained to differentiate benign from malignant oncocytic indeterminate FNAs. Although novel genetic Hürthle cell markers have been described (8,9), no commercially available genetic test has been shown to reliably distinguish benign from malignant oncocytic tumors.
or estimate practice-specific thyroid cancer prevalence rates in their indeterminate cytology patient groups. Using such cancer prevalence data, a practice-based re-estimation of the NPV of a benign Afirma GEC test may prove to be informative. In endocrine surgical practices like ours, the NPV of a benign Afirma GEC test may not be as robust as suggested by Alexander et al (4). The problem of Hürthle cell-rich thyroid aspirates that do not contain copious colloid or cellular evidence for chronic thyroiditis remains a vexing one. Commercially available genetic technology seems incapable of distinguishing benign from malignant Hürthle cell lesions. Traditionally, oncocytic indeterminate cytology patients have been sent to surgery for diagnosis. Over our first 27 months of Afirma GEC use, 19 of 21 patients with indeterminate oncocytic cytologies were deemed suspicious by Afirma and sent for surgical diagnosis. Thus, in our practice, Afirma GEC added little information to the surgical selection process in patients with Hürthle cell-dominant cytopathology. DISCLOSURE Dr. Harrell is a member of the LabCorp Endocrine Advisory Board. Dr. Bimston has no multiplicity of interest to disclose. REFERENCES 1.
2. 3.
CONCLUSION Our 27-month study of Afirma GEC reveals that the NPV for this thyroid surgery-preventing test diminishes considerably in populations where the indeterminate cytology group (excluding suspicious for malignancy patients) contains more than 1 in 4 patients with thyroid cancer. Our study is limited by a relatively small sample size and by the fact that 22 of our patients with indeterminate FNAs (17 in the Afirma GEC-benign group) have not undergone surgery. With such a limited sample size, a small number of misclassified cancers by Afirma testing can dramatically alter the sensitivity and specificity calculations for the GEC test and secondarily change NPV calculations. However, we encourage all thyroidologists and thyroid surgeons who use Afirma GEC technology to calculate
4. 5. 6. 7. 8. 9.
Kimura ET, Nikiforova MN, Zhu Z, Knauf JA, Nikiforov YE, Fagin JA. High prevalence of BRAF mutations in thyroid cancer: genetic evidence for constitutive activation of the RET/PTC-RAS-BRAF signaling pathway in papillary thyroid carcinoma. Cancer Res. 2003;63:1454-1457. Nikiforova MN, Nikiforov YE. Molecular diagnostics and predictors in thyroid cancer. Thyroid. 2009;19:1351-1361. Chudova D, Wilde JI, Wang, ET, et al. Molecular classification of thyroid nodules using the high-dimensionality genomic data. Clin Endocrinol Metab. 2010;95:5296-5304. Alexander EK, Kennedy GC, Baloch ZW, et al. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med. 367:705-715. Fletcher, RH. Clinical Epidemiology: The Essentials. 4th ed. Philadephia, PA: Lippincott Williams & Wilkins;2005. Lang W, Borrusch H, Bauer L. Occult carcinomas of the thyroid: Evaluation of 1,020 sequential autopsies. Am J Clin Pathol. 1988:90:72-76. Harach H, Franssila K, Wasenius V. Occult papillary carcinoma of the thyroid: A “normal” finding in Finland. Cancer. 1985;3:531-538. Ganly I, Ricarte Filho J, Eng S, et al. Genomic dissection of Hurthle cell carcinoma reveals a unique class of thyroid malignancy. J Clin Endocrinol Metab. 2013;98:E962-972. Cerutti, JM, Oler, G, Delcelo R, et al. PVALB, a new Hürthle adenoma diagnostic marker identified through gene expression. J Clin Endocrinol Metab. 2011;96:E151-160.
