Evidence of clinical utility: an unmet need in molecular diagnostics for patients with cancer.

Evidence of Clinical Utility: An Unmet Need in Molecular Diagnostics for Patients with Cancer David R. Parkinson, Robert T. McCormack, Susan M. Keating, et al. Clin Cancer Res 2014;20:1428-1444.

Updated version

Access the most recent version of this article at: http://clincancerres.aacrjournals.org/content/20/6/1428

Cited Articles

This article cites by 74 articles, 28 of which you can access for free at: http://clincancerres.aacrjournals.org/content/20/6/1428.full.html#ref-list-1

Citing articles

This article has been cited by 3 HighWire-hosted articles. Access the articles at: http://clincancerres.aacrjournals.org/content/20/6/1428.full.html#related-urls

E-mail alerts Reprints and Subscriptions Permissions

Sign up to receive free email-alerts related to this article or journal. To order reprints of this article or to subscribe to the journal, contact the AACR Publications Department at [email protected]. To request permission to re-use all or part of this article, contact the AACR Publications Department at [email protected].

Downloaded from clincancerres.aacrjournals.org on March 26, 2014. © 2014 American Association for Cancer Research.

CCR FOCUS Evidence of Clinical Utility: An Unmet Need in Molecular Diagnostics for Patients with Cancer David R. Parkinson1, Robert T. McCormack4, Susan M. Keating2, Steven I. Gutman3, Stanley R. Hamilton5, Elizabeth A. Mansfield6, Margaret A. Piper10, Patricia DeVerka7, Felix W. Frueh8, J. Milburn Jessup9, Lisa M. McShane9, Sean R. Tunis7, Caroline C. Sigman2, and Gary J. Kelloff9

Abstract This article defines and describes best practices for the academic and business community to generate evidence of clinical utility for cancer molecular diagnostic assays. Beyond analytical and clinical validation, successful demonstration of clinical utility involves developing sufficient evidence to demonstrate that a diagnostic test results in an improvement in patient outcomes. This discussion is complementary to theoretical frameworks described in previously published guidance and literature reports by the U.S. Food and Drug Administration, Centers for Disease Control and Prevention, Institute of Medicine, and Center for Medical Technology Policy, among others. These reports are comprehensive and specifically clarify appropriate clinical use, adoption, and payer reimbursement for assay manufacturers, as well as Clinical Laboratory Improvement Amendments–certified laboratories, including those that develop assays (laboratory developed tests). Practical criteria and steps for establishing clinical utility are crucial to subsequent decisions for reimbursement without which high-performing molecular diagnostics will have limited availability to patients with cancer and fail to translate scientific advances into high-quality and cost-effective cancer care. See all articles in this CCR Focus section, "The Precision Medicine Conundrum: Approaches to Companion Diagnostic Co-development." Clin Cancer Res; 20(6); 1428–44. 2014 AACR.

Introduction The widely accepted definition of clinical utility for an assay is that the results of the assay lead to a clinical decision that has been shown with a high level of evidence to improve patient outcomes (1). The demonstration of clinical utility is commonly a two-step process designed to show first that the test provides accurate diagnostic information and second that the diagnostic information, when used in managing patients, given the benefits and harms of the assay, improves health outcomes in a clinically perceivable way compared with alternative management strategies (for at least some number of subjects in the applicable population; refs. 1–6). A test may be considered useful if its results are actionable, driving a treatment decision that leads to a better outcome. These criteria may apply when there is no preexisting test, Authors' Affiliations: 1New Enterprise Associates, Inc., Menlo Park; 2CCS Associates, Mountain View; 3Myraqa, Redwood Shores, California; 4Johnson & Johnson/Veridex, LLC, Raritan, New Jersey; 5University of Texas, MD Anderson Cancer Center, Houston, Texas; 6Center for Diagnostics and Radiologic Health, Office of In Vitro Diagnostics, Personalized Medicine Program, Silver Spring; 7Center for Medical Technology Policy, Baltimore; 8 Opus Three LLC; 9National Cancer Institute, Division of Cancer Treatment and Diagnosis, Rockville, Maryland; and 10Kaiser Permanente Research Affiliates Evidence-Based Practice Center, Kaiser Permanente Center for Health Research, Portland, Oregon Corresponding Author: David R. Parkinson, New Enterprise Associates, Menlo Park, CA 94025. Phone: 858-345-0923; Fax: 650-513-1647; E-mail: [email protected] doi: 10.1158/1078-0432.CCR-13-2961 2014 American Association for Cancer Research.

1428

particularly in comparison to current clinical test results that may not be sufficient to make management decisions. In a broad sense, demonstrating utility may be as simple as showing that a test provides equivalent or increased sensitivity and specificity (leading to equivalent or improved management decisions); is less invasive (incurs less patient harm); is less costly (provides the same benefit with fewer resources); or is more widely or easily available (more likely to be used to make management decisions; refs. 1–8). Clinical utility has become important as the number of molecular diagnostic tests has grown substantially in number over the last decade. These include tests that specifically measure genetic variability (DNA), gene expression profiles (RNA), or protein expression of biological targets, including assays to identify signaling pathways that contribute to dysregulation of cell proliferation and apoptosis in cancer. For example, the Tufts Evidence-based Practice Center reported that 50 new genetic assays for cancer-related conditions were introduced into clinical use between 2006 and 2011, for breast, colorectal, lung, prostate, and other cancers, resulting in a total of 112 gene-based tests for solid and hematologic tumors (2). The results of molecular assays may be used to assess risk, disease stage, or prognosis; to predict treatment response; and to guide patient therapy (2, 3, 7, 9–15). Examples of some molecular targets and corresponding assays in use today are shown in Fig. 1. Personalized medicine involving successfully directed targeted therapy depends in large measure on the availability of specific predictive tests to determine which patients express

Clin Cancer Res; 20(6) March 15, 2014


Clinical Utility of Molecular Diagnostics for Cancer

Brain (glioma): 1p/19q, IDH1 Lung (NSCLC): EGFR, EML4-ALK, KRAS, MEK Liver (hepatocellular): AFP Kidney (renal cell): none–LDTs only

Head and neck (SCCHN): EGFR Breast: CYP2D6, CellSearch, ER, HER2, Mammaprint, Oncotype Dx Breast, PIK3CA, PR Pancreatic: CA19-9, MEK

Colorectal: BRAF, CEA, CellSearch EGFR, KRAS, Oncotype Dx Colon, PIK3CA, UGT1A1, MSI/MMR

Gastric: CA19-9, HER2 GIST: KIT

Prostate: AR, CellSearch, Oncotype Dx Prostate, PSA

Ovarian: CA-125, HE4, OVA1, PIK3CA

Hematologic:

Bladder: NMP22

ALL and CML: BCR-ABL AML: CEBPA, FLT3, KIT, NPM1 APL: PML-RARα CLL and NHL: CD20 MPD: JAK2 T-cell lymphoma: CD25

Melanoma: BRAF

© 2014 American Association for Cancer Research

Figure 1. Examples of tumor molecular targets for prognostic and predictive biomarker assays commercially available and used in clinical practice. Biomarker assays that are FDA approved, PMA or 510(k), are indicated in bold. Prognostic markers associated with clinical outcome include CEA, EGFR, ER, HER2, PR, and PSA. Predictive biomarkers used to select a targeted therapy include EGFR, EML-ALK, HER2, KRAS, and PIK3CA (7, 11, 13, 28–30, 40, 56, 58, 61–84). Abbreviations used: 1p/19q, codeletion of chromosomal arms 1p and 19q; ALL, acute lymphocytic leukemia, AFP, a-fetoprotein (AFP); AML, acute myeloid leukemia; APL, acute promyelocytic leukemia, AR; androgen receptor or nuclear receptor subfamily 3, group C, member 4 (AR); B-CLL; B-cell chronic lymphocytic leukemia; BRAF, v-raf murine sarcoma viral oncogene homolog B1 or proto-oncogene B-Raf (BRAF); BCR-ABL, breakpoint cluster region and Abelson murine leukemia viral oncogene homolog 1 gene fusion; CML, chronic myeloid leukemia, CA-125, cancer antigen 125 or mucin 16 (MUC16 ); CA 19-9, cancer antigen 19-9; CEA, carcinoembryonic antigen or glycosyl phosphatidyl inositol (GPI) cell surface anchored glycoprotein, (CEACAM); CEBPA, CCAAT/enhancer binding protein a C/EBPa); CML, chronic myeloid leukemia, CYP2D6, cytochrome P450 2D6 (CYP2D6); DPYD, dihydropyrimidine dehydrogenase (DPYD); EGFR, epidermal growth factor receptor (EGFR); EML4-ALK, echinoderm microtubule-associated protein-like 4 and anaplastic lymphoma kinase fusion (EML4-ALK); ER, estrogen receptor a (ESR1); FLT3, fms-related tyrosine kinase 3(FLT3) ; GIST, gastrointestinal stromal tumor; HE4, human epididymis protein 4 or WAP four-disulfide core domain (WFDC2); HER2, human epidermal growth factor receptor 2 or receptor tyrosine-protein kinase erbB-2, (ERBB2); IDH1, isocitrate dehydrogenase 1 (IDH1); JAK2, janus kinase 2 (JAK2); KIT, C-KIT, (KIT, v-kit Hardy-Zuckerman 4 feline sarcoma viral oncogene homolog); KRAS, (KRAS), Kirsten rat sarcoma viral oncogene homolog); LDT, lab-developed Abelson murine leukemia viral oncogene homolog 1 gene fusion; MPD, myelodysplastic disorder; MSI/MMR, microsatellite instability/mismatch repair proteins; NHL, non-Hodgkin lymphoma; NMP22, nuclear matrix protein 22; NPM1, nucleophosmin 1 (NPM1); NRAS, neuroblastoma RAS viral oncogene homolog (NRAS); NSCLC, non-small cell lung cancer; Oncotype Dx Breast, breast biomarker panel; Oncotype Dx Colon, colon biomarker panel; Oncotype Dx Prostate, prostate biomarker panel; OVA1 , ovarian biomarker panel; PIK3CA, phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit a or PI3K p110a (PIK3CA); PML-RARa, promyelocytic leukemia and retinoic acid receptor a fusion (RARa); PR, progesterone receptor (PGR); PSA, prostate specific antigen, gamma-seminoprotein, kallikrein-3, (KLK3); SCCHN, squamous cell carcinoma of the head and neck; UGT1A1, UDP glucuronosyltransferase 1A1 (UGT1A1).

the drug target (biomarker), and on evidence that the selected patients display a differential response to the drug compared with patients who lack the biomarker (14, 16, 17). These assays can be preexisting tests or tests developed in parallel with or after drug development. It should be mentioned that even if a test proves not to be predictive for response to a new therapeutic, the test may still be useful in development of the therapeutic if it identifies patients who have a relatively poor outcome in response to standard-of-care, and the new therapeutic improves that outcome. Two paths for deployment of clinical assays exist in the United States. The first path uses in vitro diagnostic devices (IVD; ref. 18) cleared or approved by the U.S. Food and Drug

www.aacrjournals.org

Administration (FDA; ref. 19). These devices are distributed to Clinical Laboratory Improvement Amendments (CLIA)– certified laboratories, which set up the tests and confirm their ability to perform the test consistent with the standards that have been established by the manufacturer. FDA review generally assures a new assay has established analytical and clinical validity, the latter being defined as the association between the biomarker and the pathophysiologic state or clinical presentation of interest. FDA review occurs before the marketing of a new test and is highly transparent with reviews and review decisions all posted on FDA websites. However, with the sole exception of companion diagnostics (20), the agency does not generally require outcome studies, and so



1429

CCR FOCUS clinical utility of a new assay is most commonly not established at the time of clearance or approval of a new diagnostic product. The European Union also may be considering a requirement for demonstration of clinical utility for companion diagnostics (21, 22). The second path uses laboratory-developed tests (LDT) that are not cleared or approved by FDA but are developed and offered by laboratories that are accredited by entities that have deemed status with the Center for Medicare and Medicaid Services (CMS), including the College of American Pathologists (CAP) and the Joint Commission (i.e., laboratories that are CLIA certified; refs. 23, 24). Tests are setup and established at the site of a CLIA-certified laboratory and samples sent to the laboratory for testing. For the second path, there is no premarket evaluation and tests are regulated through spot checks of documentation during the course of a routine laboratory inspection. This documentation is rarely available to the public. Of note, CMS (CLIA) has no authority to mandate that either clinical validity or clinical utility be established for LDTs. The availability of two paths to market, one without a requirement for establishment of either clinical validity or utility, has remarkably expedited entry of new and cutting edge diagnostics into medical use. For example, the Tufts Evidence-based Practice Center counted molecular tests being performed in laboratories for clinical use in 2010, including non–FDA-approved LDTs, as well as tests using commercial FDA-approved kits, or analyte-specific reagents (ASR; ref. 25). This group found 145 non–FDA-approved LDTs out of 212 tests for solid cancers, and 221 LDTs out of 388 tests for hematopathology. However, whether or not patient interests have been well served remains a question. It is clear that the performance of many new tests now being marketed has not been well established and/or well documented. In addition, there is no assurance that a LDT offered by a given CLIA laboratory is strictly comparable to that offered by a separate CLIA laboratory; reagents, methodologies, and result interpretations may be laboratory specific and in the absence of external review and comparative studies (except where proficiency testing programs are available) potential discrepancies remain unknown (see also ref. 26). Hence, it is not surprising that a number of publications and reviews by groups such as the IOM, CMTP, Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group (EWG), and the ECRI Institute Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality, have pointed out uneven success in clinical integration of many of the molecular diagnostic assays because of a variety of factors related to both assay performance and clinical evaluation (Table 1; refs. 1, 4, 5, 8, 10, 14, 25, 27–31). Furthermore, independent review has found that many of these assays have not shown sufficient proof of clinical utility (5, 25). Methodologic and clinical challenges in evaluating clinical utility A major challenge in establishing performance for new tests is the fact that frequently different tests for the same

1430


Table 1. Factors contributing to lack of success of molecular diagnostic tests in the clinic *

*

*

*

*

*

*

Method-specific issues Availability of multiple assays and methodologies for one target with a lack of material and method standards Lack of complete understanding of the clinical setting or scientific knowledge of the disease parameters by the developers Insufficient demonstration of analytical and clinical validation before commercialization Inadequate clinical trial designs or infrastructure needed to support assay implementation Lack of direct evidence that use of the assay leads to improved patient outcomes over other available solutions Lack of agreement among stakeholders on definitions and evaluation of evidence of utility

See refs. 1, 4, 5, 8, 10, 14, 25, 27, and 28.

analyte are poorly standardized. It is not uncommon for an assay of a particular type to be offered using different targets, originating from different sources, perhaps using different methodologies, reagents, and/or computational analysis, all of which can contribute to dissimilar results and hinder successful clinical adoption. Measurement of DNA, RNA, or protein may be directed at the same functional molecular target, but when used to test the clinical outcome may exhibit markedly different analytical and/or clinical sensitivities and specificities. For example, as has been well documented, levels of HER2 (ERBB2) DNA, RNA, and protein are routinely measured in patients with invasive breast cancer to determine prognosis and patients’ candidacy for trastuzumab or other related inhibitor therapy (13, 32–34). A wide variety of assay technologies have been applied to this measurement, including gene amplification using FISH or chromogenic in situ hybridization (CISH), RNA overexpression using quantitative reverse transcription PCR (qRT-PCR), protein product overexpression with immunohistochemistry (IHC) assays, and protein-based assays to measure circulating levels of the external receptor domain. Currently, at least 10 HER2 tests encompassing these various methodologies have been approved by FDA as companion diagnostics (ref. 35; Fig. 1 and Table 2). Results from these methods are reported using different scoring systems (some quantitative, others semiquantitative) often based on subjective, user dependent determinations. Each method carries assay-specific issues (specimen requirements, false negative/positive rates, standardization, and cost) that complicate the decision about which assay to use. More importantly, interlaboratory result comparison/concordance for the same method and between different methods has not been ideal. These issues have led to standardization of these methods over time, due in part to guidelines generated by the American Society of Clinical Oncology (ASCO) and CAP (34, 36–38).

Clinical Cancer Research



Table 2. FDA-approved tests (PMAs) for overexpression and amplification of HER2a Device name

Year/manufacturer

Target organ(s)

ONCOR(R) AMPLITECT HER/NEU(ERBB2)GENE AMPLIFI Inform Her2/NEU (FISH)b

1997 (withdrawn 2007) 2000 Ventana Medical Systems, Inc.

Breast

PATHWAY ANTI-Her2/NEU (4B5) Rabbit Monoclonal Primary Antibody

2000 (2013) Ventana Medical Systems, Inc.

Breast

Inform HER2 Dual ISH DNA Probe Cocktail

2011 (2013) Ventana Medical Systems, Inc.

Breast

PathVysion HER2 DNA Probe Kit (PathVysion Kit, FISH)

1998 (2013) Abbott Molecular Inc.

Breast

InSite HER2/NEU Kit (IHC)

2004 (withdrawn 2006) Biogenex Laboratories, Inc.

Breast

SPOT-Light HER2 CISH Kit

2008 (2012) Life Technologies, Inc.

Breast

Bond Oracle Her2 IHC SYSTEM

2012 Leica Biosystems

Breast

HER2 CISH PharmDx Kit

2011 (software 2013) Dako Denmark A/S

Breast

HerceptTest (immunocytochemical assay)

1998 (2013) Dako Denmark A/S

Breast, gastric

HER2 FISH PharmDx Kit

2005 (2013) Dako Denmark A/S

Breast, gastric

a

Information as given on the FDA website (35). Note also, 510(k) approvals for HER2-related diagnostics, including MammaPrint and TargetPrint (Agendia BV) and imaging and software analysis systems can be found in the same place (90). b Two or more year entries indicate original and latest PMA submissions for the product.

Lack of guidelines for evaluating clinical utility In addition to method-specific issues, an important obstacle to the clinical success of molecular diagnostic assays, with few exceptions, has been lack of evidence that use of a particular assay will result in a net improvement in the patient’s condition. This situation is partially because of lack of clear uniform guidelines for assay developers on generating evidence of associations that are clinically useful in a defined patient population, and partially to the implementation barriers for demonstrating clinical test performance cited above (1, 10, 39–41). An assay that demonstrates clinical utility by definition improves patient outcomes when used appropriately. An assay with proven clinical utility provides accurate diagnostic information for the patient population, improves patient management, and/or reduces costs. During development of diagnostic assays, data are typically generated on the assay performance for analytical validation and for clinical validation (the association between the marker and the pathophysiological state of interest; Fig. 2). Data are less likely to be generated to demonstrate the association of the test result to clinical decision making affecting patient outcomes and even less likely to demonstrate an impact on costs. This evidence is often collected only through longer term clinical use and post-market commercial studies, which can be


expensive, time consuming, and require defined patient populations. It is not surprising that those weighing resources may find these studies unattractive and view them as less critical to commercial success. However, earlier demonstration of utility as part of the assay development process would provide a greater chance of assay adoption. The failure to demonstrate clinical utility is documented in a 2013 report on three commercially available genetic or molecular pathology tests used to identify tissue of origin of cancers in patients with unknown primary sites. These tests were reviewed for analytical and clinical validity, and clinical utility. Although the tests (CancerTypeID, miRview, and PathworkDx) seemed to offer similar clinical accuracy, the authors found insufficient evidence to assess the effects of the tests on treatment decisions and outcomes, i.e., clinical utility (42). Another report published in 2012 evaluated literature for commercially available single nucleotide polymorphism panels to determine risk of prostate cancer (43). The authors concluded that not only was there no evidence available about the clinical utility of the panels, but there was also insufficient evidence to assess analytic and clinical validity. None of these commercially available panels had been evaluated for use in routine clinical settings; and there was no indication that FDA approved the tests.



1431

CCR FOCUS

Adequate test performance characteristics in the laboratory setting

Test results track clinical conditions

Test use results in improved clinical outcome for patient

Analytic validation

Clinical validation

Clinical utility

Performance of test measurement

Test associated with outcome of interest

Effectiveness; uptake; ability to deliver demonstrated

© 2014 American Association for Cancer Research

Figure 2. Process for obtaining clinical utility with predictive/prognostic molecular biomarkers and other diagnostic tests. Analytical validation consists of demonstration of assay performance (accuracy, precision, specificity, etc.); clinical validation is the association between the test result and a clinical outcome, including the variability of a marker in subjects associated with a disease condition. Both analytical and clinical validations precede demonstration of clinical utility. Clinical utility is the evidence of improved, measurable clinical outcomes directly related to the use of the test (see text for further discussion).

Implementation barriers for adequate assessment of clinical utility also arise from other issues. These can include lack of sufficient standards and reference materials as well as reference methods that make it impossible to address the issue of cross-assay validation (5), lack of appropriate infrastructure and training to support correct implementation and interpretation of tests, and lack of clarity in the clinical application of results. A high-risk commercialization pathway for developers of diagnostics remains one of the major barriers to assessment of clinical utility. This is because of many factors, including risk of obsolescence from rapidly developing technologies, along with ease of access to the market for these new complex technologies. These also include unclear reimbursement criteria from payers involving nonpayment or reimbursement amounts inadequate to recover test development and/or ongoing costs of performance. In aggregate, these commercial barriers have often led to inadequate assessment of the clinical utility of diagnostic tests. However, this landscape might markedly change with new reimbursement requirements by third-party payers calling for a demonstration of clinical utility before payment. If this concept of pay for performance is sustained and maintained, there will be significant incentives to develop new assays with patient outcomes in mind. A likely consequence is the introduction of fewer but better credentialed tests in the future (31). Recent reports and next steps The Foundation for the National Institutes of Health (FNIH) Biomarkers Consortium is a public–private biomedical research partnership that promotes development and qualification of promising biomarkers for prevention, early detection, diagnosis, and treatment of disease. Consortium partners include patient advocates, clinical researchers, and representatives from the therapeutic and diagnostic industries, NIH/NCI, and FDA. The Cancer Steering Committee (CSC) Clinical Utility Working Group of the Biomarkers Consortium determined that it would be beneficial to provide practical definitions of clinical utility and

1432


guidelines for developers and to outline the information needed by payers to support reimbursement for molecular diagnostic assays used in cancer therapy. Recent work on policy and frameworks to gather and report evidence coming from IOM (1) and CMTP (5) as well as other groups, including the National Cancer Institute (NCI), Division of Cancer Treatment and Diagnosis (DCTD; ref. 4), CDC, the Genomics Resources Technology Resource Center, Blue Cross Blue Shield Association Technology Evaluation Center (TEC-BCBSA), the Center for Health Research (Kaiser Permanente Northwest), and the EWG, have all generated suggestions for implementation (Table 3). There is consensus that if biomarkers are to be translated into clinical practice more effectively, planning for demonstration of clinical utility and addressing the evidence gap should begin earlier in assay development—at least at the clinical study planning stage. This approach would mean incorporating elements in studies to build evidence focused on outcomes when constructing premarket product or clinical development protocols and planning clinical validation trials instead of waiting until postcommercialization. In alignment with this mission, the CMTP has recently developed an Effectiveness Guidance Document containing recommendations for improving the evidence base for molecular diagnostics from the perspective of enduser and policy decision makers such as clinicians, patients, and payers. This project identified gaps and recommended specific steps that can be taken to improve the evidence base (5). Among the nonmethodological recommendations, the report notes that public–private partnerships such as the Biomarkers Consortium can help advance evidence generation of clinical utility for molecular diagnostic tests with expanded access to sources of funding and data that are needed to support welldesigned studies that produce evidence of clinical utility. These studies would be able to leverage the infrastructure and tools that accompany the cancer clinical trials enterprise.




Table 3. Previous comprehensive reviews on clinical utility Authors/organization

Title

References

Olson S, Berger AC: Institute of Medicine of the National Academies Deverka P, Messner D, Dutta T, et al.: Center for Medical Technology Policy (CMTP)

Genome-based Diagnostics: Clarifying Pathways to Clinical Use: Workshop Summary Effectiveness Guidance Document: Evaluation of Clinical Validity and Clinical Utility of Actionable Molecular Diagnostic Tests in Adult Oncology. Assessing the Clinical Utility of Diagnostics Used in Drug Therapy Development and Use of Integral Assays in Clinical Trials

(1)

Woodcock, J. Center for Drug Evaluation and Research, US Food and Drug Administration Schilsky RL, Doroshow JH, Leblanc M, et al.: National Cancer Institute (NCI), Division of Cancer Treatment and Diagnosis (DCTD) The Genomics Resources Technology Resource Center, Centers for Disease Control P: Public Health Gemonics: Genetic Testing: Lin, JS, Thompson M, Goddard KAB, et al.: Center for Health Research Kaiser Permanente Northwest, Blue Cross Blue Shield Association (TEC-BCBSA), Department of Primary Care Health Sciences, University of Oxford *

*

*

Teutsch SM, Bradley LA, Palomaki GE, et al. Botkin JR, Teutsch SM, Kaye, CI, et al. Veenstra DL, Piper M, Haddow JE, et al.: Sponsored by the National Office of Public Health Genomics at the Centers for Disease Control and Prevention

(29) (4)

ACCE Model System for Collecting, Analyzing and Disseminating Information on Genetic Tests

(91)

Evaluating Genomic Tests from Bench to Bedside: A Practical Framework

(54)

*

*

*

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative: Methods of the EGAPP Working Group Outcomes of Interest in Evidence-based Evaluations of Genetic Tests Improving the Efficiency and Relevance of Evidencebased Recommendations in the Era of Whole-Genome Sequencing: An EGAPP Methods Update

Given the recommendations generated to date, it is appropriate to move toward stakeholder collaboration to translate these guidelines into practical realities that actually implement and fund studies to demonstrate clinical utility for diagnostic assays. This document contains a rational guide and evidentiary standards for demonstrating clinical utility for predictive/prognostic diagnostics for patients with cancer. The article is not a review of previous studies, discussion of possible regulatory review, or description of coverage policy issues, but is directed at how to practically demonstrate clinical utility. It promotes practices consistent with FDA and CMS assessments of the level of evidence required, and the minimum level of clinical validation needed for demonstration of clinical utility.

Current Perspectives on Components of Clinical Utility As illustrated in Fig. 2, analytical and clinical validation must precede demonstration of an assay’s clinical utility. Analytical validation demonstrates assay performance in the laboratory setting. As defined above, clinical validation is the association between the test result and a pathophysiological state or clinical presentation of interest. This is commonly described in terms of diagnostic accuracy, a measure of how


(5)

(6, 92, 93)

well a test detects or predicts the clinical presentation. Test performance is commonly reported as clinical sensitivity and specificity and/or positive or negative predictive value (3, 4, 14, 27, 29, 44–47). Evaluation of clinical validity by test performance in relevant clinical samples will uncover the variability and significance of a biomarker in subjects with and without the specific disease associated with the molecular target. Practical guidances, discussions, and checklists for evaluating analytical and clinical validity of clinical molecular can be found in the references cited above, see for example refs. 3, 4, 25, 44, and in the ACCE Model list of 44 Targeted Questions aimed at a Comprehensive Review of Genetic Tests (48). A number of publications discussing clinical validity and utility for cancer molecular diagnostic assays provide detailed tools for evaluation of assay utility, such as those described in Table 4 (see, e.g., ref. 47). A number of these reviews have been sponsored by organizations that have also developed databases of genomic tests and relevant information. Other groups have discussed the advantages and disadvantages of various clinical trial designs for biomarker clinical utility as described in Table 5 (11, 40, 49). All these authors note that the choice of design should be based on the level of confidence that the marker will predict outcome.



1433

CCR FOCUS

Table 4. Resources for evaluating clinical utility Resource Tumor Marker Utility Grading System (TMUGS): a Framework to Evaluate Clinical Utility of Tumor Markers

Tools *

*

*

Task Force Report: Evaluating the Clinical Utility of Tumor Markers in Oncology National Comprehensive Center (NCCN) Quality, Regulation and Clinical Utility of Laboratory-Developed Molecular Tests

*

*

*

Technology Assessment Report

*

Tufts Evidence-based Practice Center

*

Update on Horizon Scans of Genetic Tests Currently Available for Clinical Use in Cancers

*

Technology Assessment Report *

Tufts Evidence-based Practice Center *

ACCE Model System for Collecting, Analyzing and Disseminating Information on Genetic Tests CDC Office of Public Health Genomics and the Foundation for Blood Research (FBR)

*

*

*

*

EGAPP Initiative and Working Group (EWG) CDC Office of Public Health Genomics

*

*

*

*

Genome-Based Diagnostics: Clarifying Pathways to Clinical Use: Workshop Summary

*

References

Guidelines, TMUGS, to consider when evaluating clinical utility of tumor markers Factors that can impact the associated assay method Definition of levels of evidence for grading utility (level 1 (most definitive)–V) and update

(8, 56)

Discussion of TMUGS levels of evidence Level of evidence ratings for selected glioma, breast, and colon cancer tumor markers

(7)

Report and database on website of 630 laboratory performed molecular FDA-approved and non–FDA-approved tests, kits or ASRs performed in laboratories for clinical in 2010 use in oncology for solid and hematological metastasis in the Medicare applicable age group Analysis of processes developed for examining clinical validity and utility of molecular tests Points to 7 reviews of clinical utility of molecular tests for cancer diagnosis or treatment monitoring reporting patient-centered health outcomes, including survival, disease recurrence, and treatment

(25)

Report on genetic and genomic (DNA, RNA, and protein) tests available for clinical practice in 2011 for assessing solid and hematological cancer in the Medicare applicable age group, for diagnostic, prognostic, predictive, monitoring, and recurrence detection uses; database, GeneTestTracker, of 121 tests on website Based on published, gray literature (manufacturers and other commercial and non-commercial, organizational websites (e.g., genetests.org, the Association for Molecular Pathology, Pharm GKB) Contains test descriptions, purpose, clinical use, availability, and FDA approval status

(2)

Model system for analyzing and updating existing data on the safety and effectiveness of DNA-based genetic tests and testing algorithms (diagnostic assays not covered) Evaluations of genomic assays posted on the website Evaluations based on analytical validity, clinical validity, clinical utility, and associated ethical, legal and social implications Evaluation criteria posted on website; address the natural history of the disorder, impact of the test on patient care, quality assurance, financial costs and benefits, facilities, personnel, and educational materials, and long-term performance evaluationa

(91)

Methods for evaluation of evidence of clinical validity and utility of genetic tests; expanded to encompass outcomes; added whole-genome sequencingb Evaluation of clinical utility based on intended use and the chain of evidence demonstrating clinical utility; importance of the quality of components, consistency and generalizability of the findings Description of acceptable evidence Grading system for the quality of evidence of components of clinical utility

(6, 92–96)

Assay development roadmap to establish clinical utility from definition of purpose and technical feasibility, development studies, analytical methods to cost-effectiveness studies, and acceptable clinical trial designs

(1, 97)

(Continued on the following page)

1434





Table 4. Resources for evaluating clinical utility (Cont'd ) Resource Evolution of Translational Omics: Lessons Learned and the Path Forward (Chapter 4) Institute of Medicine Evaluation of Clinical Validity and Clinical Utility of Actionable Molecular Diagnostic Tests in Adult Oncology

Tools *

*

*

*

*

Effectiveness Guidance Document

*

Center for Medical Technology

*

*

Development and Use of Integral Assays in Clinical Trials Criteria for the use of omics-based predictors in clinical trials

*

*

*

References

Recommendations and processes for assessing clinical utility of "omics"based tests Defines a "bright line" between definition, validation and lock-down of a test and evaluation for clinical utility Address communications with the FDA about the IDE process and validation requirements Recommendations on mechanisms that overcome existing barriers to generate evidence to attain clinical utility, including clinical study designs Importance of the context of the test in therapeutic decision making (patient population identification, biomarker prevalence, and assay validation) Assessment of the impact of the test on health outcomes (measures appropriate and validated for the clinical context, patient's perspective) Clinical benefits measures Addresses underlying causes for delay in translation of novel biomarkers into the clinic

(5)

Analytical, clinical and regulatory issues encountered in developing assays to select and stratify patients, assign treatment, and assess outcomes in clinical trials

(4)

30 criteria for the use of "omics"-based predictors in NCI-supported clinical trials Clinical trial design issues

(39, 40)

Abbreviation: ASRs, analyte specific reagents. a See Appendix A for the ACCE model list of questions for review of clinical utility for genetic tests. b See Appendix B for examples of the EGAPP methods for evaluation of clinical utility.

Best Practices for Demonstration of Clinical Utility Historical perspectives on elements of clinical utility, as described above (1, 4–6, 11), lead to the development of recommendations for best practices to establish clinical utility of diagnostics. Given this, clarity about the intended use and context of use in clinical decision making is critical from the beginning of test development, because the intended use and context should inform the design of appropriate test validation and selection of appropriate trial designs and patient populations. Both intended use and context of use are critical in determining the best practices to use across assay development, project, and commercialization planning to lay the foundation for attainment of clinical utility as outlined in Table 6 and stated below. Define the intended use of the assay along with the clinical context of the test early in assay development and project planning Intended use refers to the specific purpose of the test and should include what is being measured and why. For example, the intended use may be for screening, diagnosis, or prognosis (response, progression, or disease monitoring) for a specific disease condition associated with a specific target(s). An important component of intended use is identification of the populations to be tested. If possible, studies should ultimately be conducted in these selected


populations to assure that predictions of performance in real world use will be accurate. Definition of intended use requires understanding the clinical context (clinical need and any preexisting solutions) to develop an assay that is fit for purpose. Considerations include whether or not there is a preexisting clinical solution/ assay in place for the condition, that is, whether or not the test fits an unmet need. If the proposed assay does not address an unmet need, a consideration should be whether or not it offers other advantages over preexisting tests, that is, is the new test more accessible, less expensive, or more convenient. Potential performance characteristics of the new test and those of preexisting tests should be compared, and analytical and clinical validation studies that have been done should be described. Given the potential benefits and harms of each assay, consideration should be given to how the new diagnostic approach will improve patient management strategies in a clinically perceivable and meaningful way and whether or not the new or preexisting assay would be most likely to be effective in real-world clinical and laboratory situations. The fit for purpose concept provides justification for development of a test that may not have adequate performance characteristics (e.g., sensitivity or specificity) in certain applications but fills an unmet medical need, for example, by identifying a subpopulation for which a new therapeutic demonstrates markedly improved outcomes or by identifying a low prevalence population for which a new



1435

CCR FOCUS

Table 5. Clinical trial designs in relation to demonstration of clinical utilitya Trial design

Description

Recommendation

Prospective-randomized clinical trial design without enrichment for marker status.

All comers marker-stratified design; patients separated into marker positive and negative arms Patients randomized to treatment with drug A or B in each arm Biomarker evaluated in all patients, but randomization to treatment restricted to marker positive patients

Preferred

Marker-based strategy (enrichment) b

Marker evaluated in only one arm

Not preferred Inefficient; requires a large number of patients to detect an outcome

Prospective–retrospective analysis of new markers in all patients from a previous trial of a previously developed therapeutic

Requires: Enough appropriate biopsies (archived tissue from at least two thirds of patients) must be available from the original appropriately powered drug trial to assure that patients included in the biomarker evaluation are representative of patients in the source trial Analytical validation of the assay to ensure that results from archived tissues resemble results obtained from real-time tissue collection Prespecified analysis plan used in marker study Results validated in one or more similarly designed studies using the same assay techniques

Acceptable

A single-arm prospective study (companion diagnostic to an approved drug with previously established efficacy)

If sufficient archived tissues are lacking to conduct a prospective–retrospective study, a well-conducted study demonstrating that the companion diagnostic test can distinguish responders from nonresponders would be considered adequate evidence of the clinical utility of the test

Acceptable

Companion in vitro diagnostic assays developed with the therapeutic

Development process should produced sufficient evidence of clinical utility

Acceptable

Modeling

Modeling the clinical utility of a test may take advantage of existing evidence (i.e., from preexisting studies) to form an indirect evidence chain, or a clear surrogate marker linked to a health outcome

Acceptable

Marker enrichmentb

Not preferred Information is not obtained from the group without the marker

*

*

*

*

a

Designs are as outlined in refs. 1, 4, 11, 14, 17, 25, 39, 40, 49, 55, 56, 98, and references therein. Enrichment designs are designs that exclude patients with a specific marker status and designs that use standard of care unevenly between study arms. They are not always preferred; these authors note that from a statistical point of view, study designs should be based on the level of confidence that the marker will predict outcome. Given this, enrichment designs are appropriate when preliminary evidence suggests patients with or without the marker profile will not benefit from the treatment. The unselected design is optimal when preliminary evidence for treatment benefit or assay reproducibility is uncertain. A hybrid of the enrichment and unselected designs may be appropriate if ethical issues are involved. b

therapeutic shows high efficacy. It will also be important to consider any incentives that will affect the adoption of the new test, from ease of test performance outside of the developer’s laboratory, to assay costs and market size. Results of the above analyses will help in developing the correct hypotheses for testing to establish clinical utility and in determining what types of evidence must be generated to make a case for utility. Having this information early in

1436


development of a new assay will help sponsors and other stakeholders make informed decisions about how to prioritize use of resources. Consider regulatory and commercial implications of implementing an LDT versus an FDA-approved IVD As described above, there are currently two regulatory routes for a new diagnostic test to enter the market: as an




Table 6. Key steps toward demonstration of clinical utility *

*

*

*

*

*

*

Define the intended use of the new test and associate the assay result with a measurable clinical outcome. Understand the clinical context of the test in relation to the clinical need and disease. Determine what increased benefit (e.g., effectiveness vs. efficacy, access, cost, and/or performance) a new test would bring to the clinical setting over established procedures. When evaluating regulatory and business strategies, consider the differences and similarities in implementing an FDA-approved test (IVD) vs. an LDT test without FDA clearance, with CLIA certification. Outline what will be needed to demonstrate clinical utility as part of assay implementation (e.g., infrastructure, sample logistics, user training materials, effectiveness/quality measures, and components for long-term performance tracking). Anticipate the demonstration of clinical utility to support regulatory approval and payer clearance, including cost– benefit analysis. Integrate all the steps needed to demonstrate clinical utility into the product development plan, anticipating what evidence will be collected pre- and postapproval and commercialization. Whether developing an LDT to be offered without FDA review or an IVD for FDA approval, carefully design clinical trials and specify an analysis plan to start generating evidence of clinical utility before regulatory approval and commercialization; work with partners to obtain access to appropriate sample and patient populations. Assemble the chain of evidence; study data in context of use/patient population of interest, measured outcomes, published results, economic impacts, and compare to current practices and impacts on physician behavior or changes in clinical practice.

IVD regulated by FDA or as an LDT currently regulated only by CMS through its oversight of clinical laboratories (CLIA). Both routes require that the analytical validity of a test be evaluated before market entry. FDA regulation also calls for demonstration of clinical validity. Although nothing in the current regulatory scheme prevents the premarket demonstration of clinical utility, neither FDA nor CMS (CLIA) specifically requires this and neither routinely looks at clinical outcomes as part of test decision making unless those outcomes are an inherent part of clinical validity (e.g., FDA companion diagnostics). As part of its current regulation of companion diagnostics, FDA requires that a companion diagnostic test be approved if it is essential to the safe and effective use of a drug to be approved. In this unique case of drug-diagnostic development, the FDA does assess diagnostic performance in terms of patient outcomes, specifically


responses to drug treatment. LDTs independently developed and not having FDA review may make similar claims, but will usually not have information assuring comparative performance (information not available or available but not in the public domain). Although FDA-approved drug labels now have specific language recommending use of an FDA-approved test in decision making, clinical users, and third-party payers frequently have difficulty interpreting and implementing the recommendations. In spite of uneven or nonexistent requirements for clinical outcome studies by FDA and CMS (CLIA), units at CMS making coverage or reimbursement decisions, regional Medicare carriers, other third-party payers, and other stakeholders invested in evidence-based medicine do have considerable interest in clinical utility. Recent changes in reimbursement for molecular diagnostics (replacement of stacking codes with test-specific codes) have reinforced use of clinical utility as an important ingredient in decision making about reimbursement of tests. Whether or not a new test is developed and undergoes FDA review or is offered without FDA review as an LDT, an important step in obtaining payment for the test and ensuring market uptake is to develop evidence that the test does change patient outcomes. Although FDA and CMS have initiated a pilot to allow for joint decision making about regulatory approval/clearance and payment, in general these two processes are not connected. Sponsors and other champions of new diagnostic tests should understand this sometimes unappreciated barrier to successful introduction of a new test and establish a plan for addressing this. Both IVDs reviewed by the FDA or offered as LDTs without FDA review are intended for clinical use; hence, can only be performed in laboratories with CLIA certification. As stated above, this certification involves inspections and proficiency testing through CMS, CAP (50), or other designated third parties. Thus, early planning to meet inspection guidelines, which are standard for biomarker laboratory practice, will benefit efficient development of any test, particularly an LDT. In planning for clinical development of a new assay, it is also important to determine if an Investigational Device Exemption (IDE) is required while the assay is under investigation. An IDE is generally not required for studies in which results are masked and not used in clinical decision making, but is required when important patient care decisions are being made based on an investigational test. An exception is phase I studies where the efficacy and safety of a new therapeutic is unknown and the FDA has allowed the use of an assay described in the IND for patient selection. The FDA might require an IDE filing to collect trial safety and effectiveness data depending on the risk of use of the test, often to support premarketing approval (PMA) submissions. Use of an investigational device for clinical management often requires an approved IDE before the study is initiated, again depending on the risk of the



1437

CCR FOCUS device used to direct management, that is, whether or not an incorrect test result or a "bad test" could expose patients to serious harm or death (51). It is helpful to consult with the FDA and consider a pre-IDE meeting if and when appropriate. It may be worthwhile to consider a managed entry scheme or a staged approach to market entry and to work with business partners and FDA to explore nontraditional approaches to test implementation, for example, FDA-CMS parallel review described above, adaptive licensing, and/or performance-based risk-sharing arrangements with a provision of coverage for patients in well-designed clinical trials to gather evidence of clinical utility (5, 52). Outline what components of assay implementation will be needed to demonstrate clinical utility, including logistical infrastructure Test developers should consider how effectiveness (uptake; ability to deliver), as opposed to efficacy, will be demonstrated. This will include determining how test performance will be monitored, how safety signals from use of a new diagnostic will be analyzed, and whether or not formal or informal assessments of test use over time will be performed. Assay performance may be studied using carefully constructed clinical trials, using existing or new registries, and/or monitoring carefully selected safety signals in an ordered descriptive manner. An important issue in addressing new test use is ensuring that interested stakeholders (clinical users, patients, others) have a clear understanding of how the test should be used and order or request it appropriately. In some cases, this may be obvious but in others it may be helpful and important to provide educational (training materials, guidelines for use) or regulatory materials. A practical example of a successful train of documentation to establish a new test is illustrated on the Genomic Health website, documenting clinical studies and literature around use of the OncotypeDx test, which includes multiple collaborations (53). These studies were funded in part by NIH and demonstrate the value of public–private partnerships. Studies demonstrating prognostic performance, predictive performance, and cost-effectiveness of this test all have contributed to its commercial success. Anticipate the importance of costs in future test reimbursement decisions Regardless of whether the test will be reviewed by the FDA or offered without FDA review as an LDT, it is likely that future decision making about incorporating tests into everyday clinical medicine will become increasingly price sensitive. Sponsors and other stakeholders championing new tests should consider sponsoring cost-effectiveness studies to determine the relative value of a new test compared with the preexisting diagnostic pathway without use of the new test. Although the quality of care is paramount, cost is likely to become of increased importance in the future. Demonstrating that a new test is costeffective is likely to be of real value in making a case for

1438


use of a new test to all stakeholders and in particular to third-party payers (54). Create a project plan that includes a timeline for testing and implementing the assay for demonstrating clinical utility Test developers should consider integrating plans for demonstrating clinical utility with the analytical validation, clinical validation, and early clinical trial plans (see refs. 14 and 54, for a discussion of criteria for these stages) and consider that the evidence of clinical utility may vary over time. When appropriate, consider supplementing premarket studies with postmarket studies. Carefully design clinical trials and analyses Clinical studies for the evaluation of biomarkers can have three phases (55). Phase I studies are biomarker assay assessment in normal and tumor tissue and determination if a reliable and analytically valid signal can be generated. Phase II studies might be prospective designs for hypothesis generation and/or retrospective designs to examine biomarker utility; and phase III studies would demonstrate the clinical validity and utility of the biomarker in a prospective confirmatory multicenter studies (55) and/or might use modeling to combine disparate sources of data to create an indirect chain of evidence demonstrating clinical utility. Alternatively, prospective–retrospective designs may be used in phase II when adequate tissue representative of the patients in a trial is available to generate sufficient statistical power and the results can be validated using specimens from separate studies (56). A trial design for a companion diagnostic assay will depend upon whether or not the test is developed concurrently with the therapeutic or after the therapeutic has achieved FDA approval. Prominent recent examples involve adaptive randomization (BATTLE-1; ref. 57) and I-SPY 2 (58) and other enrichment designs such as the Master Protocol in Squamous Cell Lung Cancer (59). Table 5 provides a list of trial designs relevant to demonstration of utility. In planning trials, it is suggested that developers anticipate questions and issues about clinical utility as outlined above. Focus on studies that reflect how the test will be used in clinical practice. Take into account the mechanism linking the biomarker to the disease. In sizing the trials, account for disease prevalence, appropriateness of certain designs (e.g., enrichment designs vs. all comers or stratified), as well as the ethics of drug administration and trial costs versus returns. In addition to outcomes such as improved overall survival (OS) and progression-free survival (PFS), specify outcomes and decision options in the clinical study protocol that apply to clinical utility (better patient-reported outcomes, lower resource utilization or cost, value of the test result, and other biomarker-related interventions). Review the statistical analysis plan and ensure it is appropriate for the proposed use of the new assay and demonstration of clinical utility. Include comparison to standard of care/current practice or as add-on to standard of care/




current practice and ensure recruited patients represent the intended use population. In addition, if the new test use population will be implemented in more than one setting, consider planning to perform the clinical study in 2 or more laboratories. Assemble the chain of evidence (study data) to demonstrate utility The hierarchy of clinical study evidence is important. In cancer therapeutic studies, for example, OS has traditionally been considered the gold standard in disease endpoint measurements but is not always practical, for example, when duration of likely survival is very long, when patients often cross over to other therapies, etc. FDA has approved drugs using a variety of surrogate endpoints, such as PFS, time to progression, reduction in symptoms, etc. Patient centered or patient reported outcomes such as quality of life are also of increasing interest to stakeholders in understanding disease course, treatment effect, and perceived benefit. Studies to establish clinical utility of diagnostics as well as drugs should clearly describe the relative importance of the outcomes measured. It is important to specify clearly whether or not analysis and data interpretation were blinded and whether or not negative results were independently verified. It is also important to show that the data analysis approach adequately assesses the proposed use. Either using clinical trials or using decision modeling, studies should be designed to compare results to current practice to demonstrate the value added for the new assay. This should include an assessment of the assay’s impact on physician behavior and health care. The use of a prospective randomized controlled clinical trial studying relevant outcomes of importance to patients is the ideal method to develop a chain of evidence and allows for a direct measurement of how a test affects outcomes. However, these studies can be expensive and lengthy to perform and hence commonly are not done. Alternatively, it may be possible to acquire a high level of evidence from prospective–retrospective studies. Sometimes the only other options are to use observational studies, nonrandomized clinical trials, other types of descriptive studies, and/or information from practice guidelines to describe clinical utility in a 2-step process: (1) demonstrating a test has an impact on intermediate outcomes of interest and (2) demonstrating the outcome of interest can be shown to produce improvements in clinical outcomes. A good example of this type of approach to establishing clinical utility is the 2008 TEC-BCBSA Genetic Testing in Long QT Syndrome. At any rate, because third-party payers most frequently rely on the results of published data, it is important that studies of clinical utility be subject to peer review and published in the medical literature, regardless of the statistical significance of the study findings. These publications should be presented according to guidelines that provide standards for reporting such studies (e.g., REMARK, STARD; see Appendix B for references to these guidelines).


Conclusions As outlined in this article and in cited references, there is growing recognition and discussion of the importance of establishing clinical utility in ensuring the successful development and clinical integration of molecular diagnostic assays. Practical translation of the factors involved in demonstrating clinical utility can be accomplished by planning from the early stages of assay formulation and development and by anticipating what types of questions will need to be answered and what kinds of data need to be generated to supplement routine analytical and clinical validation studies in demonstrating the clinical utility of a new test. Critical to this long-term plan is the ability of teams of development, regulatory, clinical, and commercialization stakeholders to work together in a timely and collaborative manner. An assay development project plan by necessity requires early evaluation of the context of the assay. A central goal of such a plan is to examine the clinical usefulness of the test result relative to existing disease and clinical standards of care, to determine how to translate the test to a clinical laboratory setting, and to identify potential patient benefits from the test result. Early attention looking beyond CLIA or FDA regulatory review requirements in order to satisfy the interests of third-party payers is likely to make market entry smoother and more successful and to allow sponsors to plan early to ensure that the infrastructure and training are in place for effective use of the assay outside the R&D environment. By definition, regulatory requirements to receive approval for companion diagnostics include demonstration of clinical utility. Therefore, the approval in recent years of companion diagnostics developed with targeted therapies serve as the best available examples of successful clinical utility efforts. These successes include the KRAS test for EGFR antibody therapy (cetuximab and panitumumab) in metastatic colorectal cancer (60–65), EGFR exon testing for EGFR kinase inhibitor treatment in metastatic non–small cell lung cancer (NSCLC; refs. 66–69), and the BRAF mutation test for mutant BRAF kinase inhibitor therapy in patients with metastatic melanoma (70). Of note, the association of wild type but not mutant KRAS with positive treatment response in colorectal and lung cancer was established by retrospective analysis using archived tissues from the efficacy trials (60–63). It should be recognized that with the ever-increasing availability of relevant data and advanced technologies new tests are being developed continually, many of which may be improvements over existing diagnostics, but lack sufficient evidence to demonstrate clinical utility. Therefore, the reality is that many patient decisions today are made on the basis of tests for which clinical utility may not be well established. This can be true for both FDA-reviewed tests and those offered without FDA review. Because the consequences, if any, of this situation are currently uncertain, collaborations meant to acquire this evidence clearly would be beneficial.



1439

CCR FOCUS Finally, rapid evolution of sequencing technology and genomic panels used to characterize individual patient cancers for treatment decisions presents an unresolved clinical utility question. Reference laboratories and small biotechnology companies are marketing genotyping panels for drug targets to patients, even when a therapeutic is not approved for that target/patient class. This rapidly developing technology continually provides challenges for regulatory science and ethics issues. The situation also underlines the issue of what level of evidence is needed to decide on a treatment based on genomics or histologic characteristics, without evidence that treatments determined by genomics work. These questions are also dependent on whether or not these tests or panels have established adequate analytical and clinical validation before attempting to establish clinical utility. Given that demonstration of clinical utility is currently often a missing element in the evaluation of new laboratory tests, there is a critical need to raise the consciousness about its importance, to encourage the use of better planning and methodologies in addressing the search for utility, and to create guidance that will be of value

to all stakeholders working to medical care more effective. Recognition of the importance of establishing the clinical utility of a test can be reinforced by payer resistance to tests lacking this level of evidence, or by a tiered reimbursement system which rewards clinical tests with superior levels of clinical utility information. The requirement for high levels of evidence for a developed test presents considerable challenges with respect to business models for diagnostic companies. The development of such high levels of evidence requires time, resources, and skill sets not traditionally found within diagnostic companies, and often high levels of collaboration with investigators and clinical trial sponsors. Absent a rewards system that recognizes the value of this information, complex tests of high predictive value will not be developed with the needed frequency. If the practice of clinical medicine is to benefit from the biological and technological advances resulting from the new "omics" era, regulatory and reimbursement expectations need to be more aligned. Clinical tests of high-potential clinical utility should be prioritized by payers; improved reimbursement would stimulate private investment in the

Appendix A. ACCE model list of questions for review of clinical utility for genetic tests Component

Specific question

Disorder/setting

What is the specific clinical disorder to be studied? What are the clinical findings defining this disorder? What is the clinical setting in which the test is to be performed? What DNA test(s) are associated with this disorder? Are preliminary screening questions employed? Is it a stand-alone test or is it one of a series of tests? If it is part of a series of screening tests, are all tests performed in all instances (parallel) or are only some tests performed on the basis of other results (series)? What is the natural history of the disorder? What is the impact of a positive (or negative) test on patient care? If applicable, are diagnostic tests available? Is there an effective remedy, acceptable action, or other measurable benefit? Is there general access to that remedy or action? Is the test being offered to a socially vulnerable population? What quality assurance measures are in place? What are the results of pilot trials? What health risks can be identified for follow-up testing and/or intervention? What are the financial costs associated with testing? What are the economic benefits associated with actions resulting from testing? What facilities/personnel are available or easily put in place? What educational materials have been developed and validated and which of these are available? Are there informed consent requirements? What methods exist for long term monitoring? What guidelines have been developed for evaluating program performance?

Intervention

Quality assurance Pilot trials Health risks Economic Facilities Education Monitoring

Adapted from the ACCE Model List of 44 Targeted Questions Aimed at a Comprehensive Review of Genetic Testing. See the entire list (48) for questions related to analytic validity and clinical validity.

1440





Appendix B. Components of the EGAPP methods for evaluation of clinical utility for genetic testsa Component

Characteristics and questions addressed

Evaluation methods

*

*

*

*

*

Acceptable evidence (clinical studies)

*

*

Important study design characteristics

*

*

*

*

Grading system for the quality of evidence of components

*

*

*

Additional criteria used to assess quality

*

*

*

*

*

*

Additional components evaluated

*

*

*

*

*

*

*

a

Whether use of the assay improves clinical and health outcomes based on subsequent diagnosis and intervention Availability of information useful for decision-making Is this the last possible assay to inform a clinical decision? Balance of benefits and harms when the test results are considered in patient management Chain of evidence demonstrating that the test results can change patient management decisions and improve net health outcomes Peer-reviewed publications of data Review of a meta-analysis of multiple studies with the assay Quality of the individual clinical studies Quality of the overall body of evidence Quality of the relevant data Consistency and generalizability of the findings, and whether or not implementation of tests in different settings could lead to variability in health outcomes Convincing (results from well-designed and conducted studies in representative populations that assess specified health outcomes, systematic review or meta-analysis of randomized controlled trials that show consistent results, or at least one large randomized controlled trial) Adequate (results that might be influenced by flawed study methodology; systematic review with heterogeneity; one or more nonrandomized, peer-reviewed, but not well-designed controlled trials; or systematic review of cohort studies with consistent results) Inadequate (results that are likely to come from flawed study methodology, as in systematic review of studies with heterogeneity, single-level cohort or case–control studies, or data that was neither published nor peer-reviewed) Clear presentation of the study design; clear description and definitions of outcomes of interest or decision options Relative importance of the outcomes measured Blinded interpretation of outcomes Verification of negative results Use of prospective or retrospective data collection Appropriateness of data analysis Randomization of subjects in experimental designs Study inclusion of comparison with current practice to demonstrate value added Interventions used and criteria needed for the intervention Information on data analysis sufficient to rate the study quality Relevance of data to identified outcomes Adequate and appropriate description of the analysis or modeling for the study design and proposed use Consideration of losses and potential bias

Cited in refs. 6, 92, and 93.

development of such tests. Additional mechanisms that would allow for testing, evaluation, and refinement of the methodologies being proposed here can include public/ private activities such as those carried out by the FNIH Biomarkers Consortium. Disclosure of Potential Conflicts of Interest D.R. Parkinson is a CEO of Nodality, Inc. No potential conflicts of interest were disclosed by the other authors.


Authors' Contributions Conception and design: D.R. Parkinson, R.T. McCormack, S.M. Keating, F. W. Frueh, J.M. Jessup, S.R. Tunis, C.C. Sigman, G.J. Kelloff Development of methodology: S.R. Tunis, G.J. Kelloff Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): R.T. McCormack, C.C. Sigman, G.J. Kelloff Writing, review, and/or revision of the manuscript: D.R. Parkinson, R.T. McCormack, S.M. Keating, S.I. Gutman, S.R. Hamilton, E.A. Mansfield, M.A. Piper, P. DeVerka, F.W. Frueh, J.M. Jessup, L.M. McShane, S.R. Tunis, C.C. Sigman, G.J. Kelloff Study supervision: G.J. Kelloff



1441

CCR FOCUS Acknowledgments The authors wish to gratefully acknowledge the support of the Foundation for the National Institutes of Health Biomarkers Consortium in providing a venue for carrying out this project.

Received October 29, 2013; revised January 7, 2014; accepted January 10, 2014; published online March 14, 2014.

References 1.

2.

3.

4.

5.

6.

7.

8.

9.

10. 11. 12.

13.

14. 15. 16.

17.

18.

1442

Olson S, Berger AC. Genome-based diagnostics: clarifying pathways to clinical use: workshop summary. Institute of Medicine, National Academies Press; 2012. Raman G, Wallace B, Patel K, Lau J, Trikalinos TA, Tufts evidencebased practice center. Update on horizon scans of genetic tests currently available for clinical use in cancers. AHRQ Technology Assessment Report 2011. Available from: http://www.cms.gov/ Medicare/Coverage/DeterminationProcess/downloads/id81TA.pdf [Accessed October 23, 2013]. Goodman S, Dickerson K, Wilson R, Center for Medical Technology Policy (CMTP). Effectiveness guidance document. Gene expression profile tests for early stage breast cancer. Version 1.0. June 2009. Available from: http://cmtpnet.org/wp-content/uploads/downloads/ 2011/09/GeneExpressionEGD.pdf [Accessed October 23, 2013]. Schilsky RL, Doroshow JH, Leblanc M, Conley BA. Development and use of integral assays in clinical trials. Clin Cancer Res 2012;18: 1540–6. Deverka P, Messner D, Dutta T, Center for medical technology policy (CMTP). Effectiveness guidance document: evaluation of clinical validity and clinical utility of actionable molecular diagnostic tests in adult oncology. Available from: http://www.cmtpnet.org/ wp-content/uploads/downloads/2013/05/CMTP_MDx_EGD05-012013.pdf [Accessed October 23, 2013]. Teutsch SM, Bradley LA, Palomaki GE, Haddow JE, Piper M, Calonge N, et al. The evaluation of genomic applications in practice and prevention (EGAPP) initiative: methods of the EGAPP working group. Genet Med 2009;11:3–14. Febbo PG, Ladanyi M, Aldape KD, De Marzo AM, Hammond ME, Hayes DF, et al. NCCN task force report: evaluating the clinical utility of tumor markers in oncology. J Natl Compr Canc Netw 2011;9:S1–32; quiz S3. Hayes DF, Bast RC, Desch CE, Fritsche H Jr, Kemeny NE, Jessup JM, et al. Tumor marker utility grading system: a framework to evaluate clinical utility of tumor markers. J Natl Cancer Inst 1996;88:1456–66. U.S. Food DA. Guidance for industry: pharmacogenomic data submissions. March 2005. Available from: http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm126957.pdf [Accessed October 23, 2013]. Majewski IJ, Bernards R. Taming the dragon: genomic biomarkers to individualize the treatment of cancer. Nat Med 2011;17:304–12. Gerard B, Betard M-C, Smith B, Denyer M. Development of targeted agents and companion diagnostics. Drug Inf J 2012;46:405–19. Sikorski R, Yao B. Visualizing the landscape of selection biomarkers in current phase III oncology clinical trials. Sci Transl Med 2010;2: 34ps27. Patel JN, Mandock K, McLeod HL. Clinically relevant cancer biomarkers and pharmacogenetic assays. J Oncol Pharm Pract. 2013 Jan 24. [Epub ahead of print]. Kelloff GJ, Sigman CC. Cancer biomarkers: selecting the right drug for the right patient. Nat Rev Drug Discov 2012;11:201–14. Fan Y-S. Companion diagnostic testing for targeted cancer therapies: an overview. Genet Test Mol Biomarkers 2013;17:515–23. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther 2001;69:89–95. U.S. Food and Drug Administration. Guidance for industry: qualification process for drug development tools. Draft guidance. October 2010. Available from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM230597.pdf [Accessed October 23, 2013]. U.S. Food and Drug Administration. Draft guidance for industry and food and drug administration staff. In vitro companion diagnostic


19.

20. 21.

22.

23.

24. 25.

26.

27. 28.

29. 30.

31.

32.

33.

34. 35.

36.

37.

devices. July 14, 2011. Available from: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM262327.pdf [Accessed October 23, 2013]. Tezak Z, Kondratovich MV, Mansfield E. US FDA and personalized medicine: in vitro diagnostic regulatory perspective. Personalized Med 2010;7:517–30. Mansfield EA. FDA perspective on companion diagnostics: an evolving paradigm. Clin Cancer Res 2014;20:1453–7. Pignatti F, Ehmann F, Hemmings R, Jonsson B, Nuebling M, PapalucaAmati M, et al. Cancer drug development and the evolving regulatory framework for companion diagnostics in the European Union. Clin Cancer Res 2014;20:1458–68. Senderowicz AM, Pfaff O. Similarities and differences in the oncology drug approval process between FDA and European Union with emphasis on in vitro companion diagnostics. Clin Cancer Res 2014;20:1445– 52. Centers for Medicare & Medicaid Services. CLIA Regulations and federal register documents. Available from: http://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/CLIA_Regulations_and_Federal_Register_Documents.html; 2012. Weiss RL. The long and winding regulatory road for laboratory-developed tests. Am J Clin Pathol 2012;138:20–6. Sun F, Bruening W, Uhl S, et al. Quality, regulation and clinical utility of laboratory-developed molecular tests. AHRQ Technology Assessment Report 2010. Available from: http://www.cms.gov/Medicare/ Coverage/DeterminationProcess/downloads/id72TA.pdf [Accessed October 23, 2013]. Byron SK, Crabb N, George E, Marlow M, Newland A. The health technology assessment of companion diagnostics: experience of NICE. Clin Cancer Res 2014;20:1469–76. Simon R, Roychowdhury S. Implementing personalized cancer genomics in clinical trials. Nat Rev Drug Discov 2013;12:358–69. Faruki H, Lai-Goldman M. Application of a pharmacogenetic test adoption model to six oncology biomarkers. Personalized Med 2010; 7:441–50. Woodcock J. Assessing the clinical utility of diagnostics used in drug therapy. Clin Pharmacol Ther 2010;88:765–73. McShane LM, Hayes DF. Publication of tumor marker research results: the necessity for complete and transparent reporting. J Clin Oncol 2013;30:4223–32. Hayes DF, Allen J, Compton C, Gustavsen G, Leonard DG, McCormack R, et al. Breaking a vicious cycle. Sci Transl Med 2013;5: 196cm6. Slamon DJ, Clark GM, Wong SG, Levin WJ, Ullrich A, McGuire WL. Human breast cancer: correlation of relapse and survival with amplification of the HER-2/neu oncogene. Science 1987;235:177–82. Pegram MD, Pauletti G, Slamon DJ. HER-2/neu as a predictive marker of response to breast cancer therapy. Breast Cancer Res Treat 1998;52:65–77. De P, Smith BR, Leyland-Jones B. Human epidermal growth factor receptor 2 testing: where are we? J Clin Oncol 2010;28:4289–92. U.S. Food and Drug Administration. Companion diagnostic devices: in vitro and imaging tools. [cited 2013 Jul 25]. Available from: http:// www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/InVitroDiagnostics/ucm301431.htm. Wolff AC, Hammond ME, Schwartz JN, Hagerty KL, Allred DC, Cote RJ, et al. American society of clinical oncology/college of American pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. J Clin Oncol 2007;25: 118–45. Wolff AC, Hammond ME, Schwartz JN, Hagerty KL, Allred DC, Cote RJ , et al. American society of clinical oncology/college of American




38.

39.

40.

41. 42.

43.

44.

45.

46.

47.

48.

49. 50. 51.

52.

53.

54.

55.

56.

57. 58.

pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. Arch Pathol Lab Med 2007;131:18–43. Wolff AC, Hammond ME, Hicks DG, Dowsett M, McShane LM, Allison KH, et al. Recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical oncology/college of American pathologists clinical practice guideline update. J Clin Oncol 2013;31:3997–4013. McShane LM, Cavenagh MM, Lively TG, Eberhard DA, Bigbee WL, Williams PM, et al. Criteria for the use of omics-based predictors in clinical trials. Nature 2013;502:317–20. McShane LM, Cavenagh MM, Lively TG, Eberhard DA, Bigbee WL, Williams PM, et al. Criteria for the use of omics-based predictors in clinical trials: explanation and elaboration. BMC Med 2013; 11:220. Hayes DF. OMICS-based personalized oncology: if is worth doing, it is worth doing well! BMC Med 2013;11:221. Meleth S, Whitehead N, Evans TS, Lux L. Technology assessment on genetic testing or molecular pathology testing of cancers with unknown primary site to determine origin. AHRQ Technology Assessment Program 2013;20:209. Little J, Wilson B, Carter R, Walker K, Santaguida P, Tomiak E, et al. Multigene panels in prostate cancer risk assessment. Evidence report/ technology assessment No. 209. AHRQ Publication No.12-E020-EF. Rockville, MD: Agency for Healthcare Research and Quality. N/A 2012. p 166. Jennings L, Van Deerlin VM, Gulley ML. Recommended principles and practices for validating clinical molecular pathology tests. Arch Pathol Lab Med 2009;133:743–55. Williams PM, Lively TG, Jessup JM, Conley BA. Bridging the gap: moving predictive and prognostic assays from research to clinical use. Clin Cancer Res 2012;18:1531–9. Chau CH, Rixe O, McLeod H, Figg WD. Validation of analytic methods for biomarkers used in drug development. Clin Cancer Res 2008;14: 5967–76. Poste G, Carbone DP, Parkinson DR, Verweij J, Hewitt SM, Jessup JM. Leveling the playing field: bringing development of biomarkers and molecular diagnostics up to the standards for drug development. Clin Cancer Res 2012;18:1515–23. Centers for Disease Control and Prevention. Public health genomics: genetic testing: ACCE model list of 44 targeted questions aimed at a comprehensive review of genetic testing. [cited 2013 Jun 6] Available from: http://www.cdc.gov/genomics/gtesting/ACCE/acce_proj.htm. Freidlin B, McShane LM, Korn EL. Randomized clinical trials with biomarkers: design issues. J Natl Cancer Inst 2010;102:152–60. College of American Pathologists. About the CAP laboratory accreditation program. [cited 2013 Jun 19] Accessed from: www.cap.org. U.S. Food and Drug Administration. Device advice: investigational device exemption (IDE). [cited 2012 Jul 20]. Accessed from: http:// www.fda.gov /MedicalDevices / DeviceRegulationandGuidance / HowtoMarketYourDevice/InvestigationalDeviceExemptionIDE/default.htm. Wookey A. FDA-approved LDT: accelerating the path to commercialization while mitigating risk and cost. Accelerating Development & Advancing Personal Therapy Congress (ADAPT); 2012. Paik S. Development and clinical utility of a 21-gene recurrence score prognostic assay in patients with early breast cancer treated with tamoxifen. Oncologist 2007;12:631–5. Lin JS, Thompson M, Goddard KA, Piper MA, Heneghan C, Whitlock EP. Evaluating genomic tests from bench to bedside: a practical framework. BMC Med Inform Decis Mak 2012;12:117. Mandrekar SJ, Grothey A, Goetz MP, Sargent DJ. Clinical trial designs for prospective validation of biomarkers. Am J Pharmacogenomics 2005;5:317–25. Simon RM, Paik S, Hayes DF. Use of archived specimens in evaluation of prognostic and predictive biomarkers. J Natl Cancer Inst 2009;101: 1446–52. Sequist LV, Muzikansky A, Engelman JA. A new BATTLE in the evolving war on cancer. Cancer Discov 2010;1:14–6. Barker AD, Sigman CC, Kelloff GJ, Hylton NM, Berry DA, Esserman LJ. I-SPY 2: an adaptive breast cancer trial design in the setting


59. 60.

61.

62.

63.

64. 65.

66.

67.

68. 69.

70.

71.

72.

73.

74.

75.

76.

77.

78.

79.

of neoadjuvant chemotherapy. Clin Pharmacol Ther 2009;86: 97–100. Ledford H. 'Master protocol' aims to revamp cancer trials. Nature 2013;498:1467. Kamel-Reid S, Zhang T, Persons DL, Nikiforova MN, Halling KC. Validation of KRAS testing for anti-EGFR therapeutic decisions for patients with metastatic colorectal carcinoma. Arch Pathol Lab Med 2012;136:26–32. Karapetis CS, Khambata-Ford S, Jonker DJ, O'Callaghan CJ, Tu D, Tebbutt NC , et al. K-ras mutations and benefit from cetuximab in advanced colorectal cancer. N Engl J Med 2008;359:1757–65. Amado RG, Wolf M, Peeters M, Van Cutsem E, Siena S, Freeman DJ, et al. Wild-type KRAS is required for panitumumab efficacy in patients with metastatic colorectal cancer. J Clin Oncol 2008;26:1626–34. Ruzzo A, Graziano F, Canestrari E, Magnani M. Molecular predictors of efficacy to anti-EGFR agents in colorectal cancer patients. Curr Cancer Drug Targets 2010;10:68–79. Fakih MM. KRAS mutation screening in colorectal cancer: from paper to practice. Clin Colorectal Cancer 2010;9:22–30. Normanno N, Tejpar S, Morgillo F, De Luca A, Van Cutsem E, Ciardiello F. Implications for KRAS status and EGFR-targeted therapies in metastatic CRC. Nat Rev Clin Oncol 2009;6:519–27. Pao W, Wang TY, Riely GJ, Miller VA, Pan Q, Ladanyi M, et al. KRAS mutations and primary resistance of lung adenocarcinomas to gefitinib or erlotinib. PLoS Med 2005;2:e17. Zhang Z, Stiegler AL, Boggon TJ, Kobayashi S, Halmos B. EGFRmutated lung cancer: a paradigm of molecular oncology. Oncotarget 2010;1:497–514. Pao W, Chmielecki J. Rational, biologically based treatment of EGFRmutant non-small-cell lung cancer. Nat Rev Cancer 2010;10:760–74. Lindeman NI, Cagle PT, Beasley MB, Chitale DA, Dacic S, Giaccone G, et al. Molecular testing guideline for selection of lung cancer patients for EGFR and ALK tyrosine kinase inhibitors: guideline from the College of American Pathologists, International Association for the Study of Lung Cancer, and Association for Molecular Pathology. Arch Pathol Lab Med 2013 Apr 3. [Epub ahead of print]. Flaherty KT, Puzanov I, Kim KB, Ribas A, McArthur GA, Sosman JA, et al. Inhibition of mutated, activated BRAF in metastatic melanoma. N Engl J Med 2010;363:809–19. Andrews J, Yeh P, Pao W, Horn L. Molecular predictors of response to chemotherapy in non-small cell lung cancer. Cancer J 2011;17: 104–13. U.S. Food and Drug Administration. Table of pharmacogenomic biomarkers in drug labels. [cited 2013 Mar 04]. Available from: http://www. fda.gov/Drugs/ScienceResearch/ResearchAreas/Pharmacogenetics/ ucm083378.htm. Maemondo M, Inoue A, Kobayashi K, Sugawara S, Oizumi S, Isobe H, et al. Gefitinib or chemotherapy for non-small-cell lung cancer with mutated EGFR. N Engl J Med 2010;362:2380–8. Box C, Zimmermann M, Eccles S. Molecular markers of response and resistance to EGFR inhibitors in head and neck cancers. Front Biosci 2013;18:520–42. Yu D, Zhang X, Liu J, Yuan P, Tan W, Guo Y, et al. Characterization of functional excision repair cross-complementation group 1 variants and their association with lung cancer risk and prognosis. Clin Cancer Res 2008;14:2878–86. Bepler G, Gautam A, McIntyre LM, Beck AF, Chervinsky DS, Kim YC, et al. Prognostic significance of molecular genetic aberrations on chromosome segment 11p15.5 in non-small-cell lung cancer. J Clin Oncol 2002;20:1353–60. Kwak EL, Bang Y-J, Camidge DR, Shaw AT, Solomon B, Maki RG, et al. Anaplastic lymphoma kinase inhibition in non-small-cell lung cancer. N Engl J Med 2010;363:1693–703. Baehner FL, Achacoso N, Maddala T, Shak S, Quesenberry CP Jr, Goldstein LC, et al. Human epidermal growth factor receptor 2 assessment in a case-control study: comparison of fluorescence in situ hybridization and quantitative reverse transcription polymerase chain reaction performed by central laboratories. J Clin Oncol 2010;28:4300–6. Horbinski C. Something old and something new about molecular diagnostics in gliomas. Surg Pathol Clin 2012;5:919–39.



1443

CCR FOCUS 80. van de Vijver MJ, He YD, van't Veer LJ, Dai H, Hart AA, Voskuil DW, et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med 2002;347:1999–2009. 81. Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, et al. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 2004;351:2817–26. 82. Gray RG, Quirke P, Handley K, Lopatin M, Magill L, Baehner FL, et al. Validation study of a quantitative multigene reverse transcriptasepolymerase chain reaction assay for assessment of recurrence risk in patients with stage II colon cancer. J Clin Oncol 2011;29:4611–9. 83. Webber EM, Lin JS, Whitlock EP. Oncotype DX tumor gene expression profiling in stage II colon cancer. Application: prognostic, risk prediction. PLoS Curr 2010;2. pii: RRN1177. doi: 10.1371/currents.RRN1177. 84. Cooperberg M, Simko J, Falzarano S, et al. Development and validation of the biopsy-based genomic prostate score (GPS) as a predictor of high grade or extracapsular prostate cancer to improve patient selection for active surveillance. AUA Annual Meeting 2013;N/A: abst. No. 2131. 85. Allard WJ, Matera J, Miller MC, Repollet M, Connelly MC, Rao C, et al. Tumor cells circulate in the peripheral blood of all major carcinomas but not in healthy subjects or patients with nonmalignant diseases. Clin Cancer Res 2004;10:6897–904. 86. Miller MC, Doyle GV, Terstappen LW. Significance of circulating tumor cells detected by the CellSearch system in patients with metastatic breast colorectal and prostate cancer. J Oncol 2010; 2010:617421. 87. Monzon FA, Dumur CI. Diagnosis of uncertain primary tumors with the pathwork tissue-of-origin test. Expert Rev Mol Diagn 2010;10: 17–25. 88. Ross JS, Ali SM, Wang K, Palmer G, Yelensky R, Lipson D, et al. Comprehensive genomic profiling of epithelial ovarian cancer by next generation sequencing-based diagnostic assay reveals new routes to targeted therapies. Gynecol Oncol 2013 Jun 20. [Epub ahead of print].

1444


89. Giulino-Roth L, Wang K, MacDonald TY, Mathew S, Tam Y, Cronin MT, et al. Targeted genomic sequencing of pediatric Burkitt lymphoma identifies recurrent alterations in antiapoptotic and chromatin-remodeling genes. Blood 2012;120:5181–4. 90. U.S. Food and Drug Administration. Devices@FDA. [cited 2013 Aug 05. Available from: http://www.accessdata.fda.gov/scripts/cdrh/devicesatfda/index.cfm. 91. Centers for Disease Control and Prevention. Public health genomics: genetic testing: ACCE model system for collecting, analyzing and disseminating information on genetic tests. Available from: http:// www.cdc.gov/genomics/gtesting/ACCE/fbr; 2010 [Accessed October 23, 2013]. 92. Botkin JR, Teutsch SM, Kaye CI, Hayes M, Haddow JE, Bradley LA, et al. Outcomes of interest in evidence-based evaluations of genetic tests. Genet Med 2010;12:228–35. 93. Veenstra DL, Piper M, Haddow JE, Pauker SG, Klein R, Richards CS, et al. Improving the efficiency and relevance of evidence-based recommendations in the era of whole-genome sequencing: an EGAPP methods update. Genet Med 2013;15:14–24. 94. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM. Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 2005;97:1180–4. 95. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996;276:637–9. 96. Meyer GJ. Guidelines for reporting information in studies of diagnostic test accuracy: the STARD initiative. J Pers Assess 2003;81: 191–3. 97. Institute of Medicine. Evolution of translational omics: lessons learned and the path forward. Washington, DC: The National Academies Press; 2012, 354 pp. 98. Sargent DJ, Conley BA, Allegra C, Collette L. Clinical trial designs for predictive marker validation in cancer treatment trials. J Clin Oncol 2005;23:2020–7.



Impact of venous thromboembolism on survival in patients with advanced cancer: an unmet clinical need.

'Toxgnostics': an unmet need in cancer medicine.

Clinical perspective - cough: an unmet need.

IV iron in pregnancy: an unmet clinical need.

Biomarkers and Microscopic Colitis: An Unmet Need in Clinical Practice.

Molecular diagnostics clinical utility strategy: a six-part framework.

Unmet clinical need in autoimmune liver diseases.

Pharmacometrics in pregnancy: An unmet need.

Determinants of need and unmet need among cancer patients residing at home.

Fall-risk prediction in older adults with cancer: an unmet need.

Sustained Remission: An Unmet Need in Patients with Giant-cell Arteritis.

Clinical Characteristics and Unmet Need Among Patients with Atherosclerotic Cardiovascular Disease Stratified by Statin Use.

An unmet need: Feeding for critically ill vegans.

Management of pulmonary exacerbations in cystic fibrosis: still an unmet medical need in clinical practice.

Nivolumab for refractory metastatic squamous non-small-cell lung cancer: fulfilling an unmet need.

Reducing unmet need by supporting women with met need.

Cachexia vs obesity: where is the real unmet clinical need?

Dry age-related macular degeneration: A currently unmet clinical need.

Editorial: healing of refractory reflux oesophagitis--an ongoing unmet clinical need; authors' reply.

Access to care--an unmet need in headache management?

Unmet need for contraception: issues and challenges.

Clinical ethics consultation: a need for evidence.

Editorial: healing of refractory reflux oesophagitis--an ongoing unmet clinical need.

Assessing cardiovascular risk in hepatitis C: An unmet need.