Appl Health Econ Health Policy DOI 10.1007/s40258-014-0123-8

REVIEW ARTICLE

A Review of the Economic Tools for Assessing New Medical Devices Joyce A. Craig • Louise Carr • John Hutton Julie Glanville • Cynthia P. Iglesias • Andrew J. Sims

•

 Springer International Publishing Switzerland 2014

Abstract Whereas the economic evaluation of pharmaceuticals is an established practice within international health technology assessment (HTA) and is often produced with the support of comprehensive methodological guidance, the equivalent procedure for medical devices is less developed. Medical devices, including diagnostic products, are a rapidly growing market in healthcare, with over 10,000 medical technology patent applications filed in Europe in 2012—nearly double the number filed for pharmaceuticals. This increase in the market place, in combination with the limited, or constricting, budgets that healthcare decision makers face, has led to a greater level of examination with respect to the economic evaluation of

Electronic supplementary material The online version of this article (doi:10.1007/s40258-014-0123-8) contains supplementary material, which is available to authorized users. J. A. Craig (&)
L. Carr
J. Glanville York Health Economics Consortium, University of York, Level 2 Market Square, Vanbrugh Way, Heslington, York YO10 5NH, UK e-mail: [email protected] J. Hutton Department of Health Sciences, University of York, York, UK C. P. Iglesias Centre for Health Economics, University of York, Level 2 Market Square, Vanbrugh Way, Heslington, York YO10 5NH, UK A. J. Sims Clinical Measurement and Engineering Unit, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK

medical devices. However, methodological questions that arise due to the unique characteristics of medical devices have yet to be addressed fully. This review of journal publications and HTA guidance identified these characteristics and the challenges they may subsequently pose from an economic evaluation perspective. These unique features of devices can be grouped into four categories: (1) data quality issues; (2) learning curve; (3) measuring longterm outcomes from diagnostic devices; and (4) wider impact from organisational change. We review the current evaluation toolbox available to researchers and explore potential future approaches to improve the economic evaluation of medical devices.

J. A. Craig et al.

Key Points for Decision Makers The economic evaluation of medical devices is less developed than the equivalent processes used for pharmaceuticals in the context of healthcare decision making. We identified four unique characteristics of medical devices that may pose challenges to economic evaluation: (1) data quality issues; (2) learning curve; (3) measuring long-term outcomes from diagnostic devices; (4) wider impact of organisational change. Often, published economic evaluations are insufficient to address the unique characteristics of medical devices. Enhancing purchasers’ and suppliers’ knowledge of economic evaluation methods may help the National Health Service adopt more efficient and costeffective medical devices and diagnostics more rapidly and consistently. Some device characteristics can be accounted for by using existing techniques within economic modelling. However, there is a need to adopt new modelling approaches to incorporate and/or assess the unique characteristics of medical devices, which are currently often unaddressed. Such approaches include the use of early models or ‘coverage with evidence development’ initiatives to fund studies of the device in a clinical setting.

1 Introduction A medical device is an instrument, apparatus, implant, in vitro reagent, or similar or related object that is used to diagnose, prevent, or treat disease or other conditions, and that does not achieve its purpose(s) through chemical action within or on the body (which would make it a pharmaceutical product) [1]. Medical devices vary greatly in complexity and application. Demand for, and the cost associated with, medical devices in healthcare is large at 7.5 % of total European healthcare expenditure [2], compared with 17 % for pharmaceuticals. Following policy and regulatory pressures, companies are increasingly required to submit comprehensive economic evaluations of new pharmaceuticals to health technology assessment (HTA) bodies to provide robust evidence of cost effectiveness. Established HTA bodies employing this approach to inform decision making

include, but are not limited to, the National Institute for Health and Care Excellence (NICE) in England and Wales, the Canadian Agency for Drugs and Technologies in Health, the Haute Autorite´ de Sante´ in France and the Pharmaceutical Benefits Advisory Committee in Australia. However, a disparity exists with respect to the rigour applied in economic evaluations of pharmaceuticals and medical devices. Udvarhelyi et al. [3] highlighted problems with economic evaluations of pharmaceuticals over 2 decades ago, and concluded that greater attention was required to ensure the appropriate use of analytical methods. While economic evaluation processes applied to pharmaceuticals are now considered robust, this is seldom the case for medical devices [4], and scrutiny by regulators of the economic benefit of devices is relatively new. For example, companies who submit a novel device to the NICE Medical Technologies Evaluation Programme (MTEP) must now include economic evidence [5]; however, the programme has only operated since November 2009 [6]. The MTEP is the single point of entry into NICE for medical devices, and the Medical Technologies Advisory Committee may route suitable technologies for evaluation to one of several programmes, including the Technology Appraisal Programme [7]. Considering that NICE was launched in 1999 [8], the lag in developing a medical device-specific process highlights the delay in developing robust economic evaluations for devices in the context of healthcare decision making. Differences in the regulatory framework can introduce additional considerations (e.g. the requirement for more sophisticated statistical methods) [9] to economic evaluations of devices compared with those of pharmaceuticals. In addition, the unique characteristics of medical devices may require specific methodological approaches [10]. The more dynamic pricing policies associated with medical devices [10, 11], reliance on operator skills, and/or the need for operator training for a device to function optimally, can pose further methodological hurdles. These unique characteristics are compounded by the broad array of device functionalities, with over 500,000 devices currently marketed [2]. The number of different devices is rapidly increasing, with nearly double the number of patent applications filed for medical technologies (10,412) in Europe in 2012 compared with pharmaceuticals (5,364) [2]. With restricted healthcare budgets, demand for accurate information on the economic value of medical devices is growing. This review, therefore, had four objectives: •

Objective 1: Determine the unique characteristics of medical devices that may need to be addressed in an economic evaluation.

Economic Tools for New Devices

•

•

•

Objective 2: Describe how the unique characteristics are assessed and the suitability of the techniques to address them. Objective 3: Determine additional techniques not identified in published economic evaluations of medical devices that might be appropriate to address the unique characteristics of devices. Objective 4: Recommend future approaches for the economic evaluation of medical devices.

Econlit (detailed in the Electronic Supplementary Material [ESM] 1), supplemented by hand searching. Two types of searches were used: one using terms for specific methods, and the other including papers by known authors. Date limitations were from 1961 to December 2011. In addition, searches of the following resources were undertaken: •

2 Methodology We undertook a selective review involving (Fig. 1) a literature search, data extraction of selected studies using an assessment framework and critical analysis of selected papers to address the four objectives. The definitions of ‘medical devices’ or ‘medical technology’ used in this review are those adopted by NICE for its MTEP [12]. Throughout this report, the term ‘medical device(s)’ is used to mean diagnostic and non-diagnostic devices, unless otherwise specified. Unique characteristics of medical devices that need to be addressed in economic evaluation were defined as those that introduce additional considerations to economic evaluation of devices compared with pharmaceuticals. 2.1 Literature Searches, Selection and Data Extraction Individual search strategies were used to search international bibliographic databases, including MEDLINE and

•

International HTA Guidelines in Australia [13–15], Canada [16, 17], France [18–20], Denmark [21] and Sweden to identify whether (1) separate guidance on medical devices exists, and (b) reference was made to any unique characteristics. The countries were selected on the basis of the research team’s knowledge of international HTA. NICE MTEP and Diagnostics Assessment Programme (DAP) guidance for medical devices (as published by 10 February 2012) [22].

Following the searches, the results were assessed for relevance to the following themes: • •

Methodological papers that recognise the unique characteristics of medical devices. Published economic evaluations of medical devices that identified unique characteristics of the device under evaluation.

A random sampling strategy was used to select economic evaluations; this retrieved a representative sample (100 abstracts) of the results where high numbers (i.e. [400 results) were obtained, since the strategy used sensitive search terms. Published methodological papers were selected based on relevance of title and abstract to the

Fig. 1 Overview of the methodologies used to inform findings on each review objective. DAP Diagnostics Assessment Programme, HTA Health Technology Assessment, MTEP Medical Technologies Evaluation Programme

J. A. Craig et al.

objectives, with studies included if they related directly to a medical device or diagnostic test, or were considered an intervention that used a specific medical device. Studies were excluded if they related to diseases, pharmaceuticals, animals, ecology, clinical interventions not using a medical device, or methods to improve measuring public preferences for health states unless these related specifically to devices. A data extraction sheet was developed, tested, and completed for each paper (see ESM 1). The data extraction form recorded study features, medical device characteristics, type of economic analysis conducted and whether any unique characteristics identified were accounted for. Any methodological issues relevant to evaluating medical devices were also extracted along with suggested ways to address these issues.

3 Objective 1 Results: Unique Characteristics Identified 3.1 International Health Technology Assessment (HTA) Recognition of Unique Characteristic First, international HTA guidance was reviewed to identify characteristics of medical devices that may need to be addressed in the context of economic evaluations. The search results identified that Australia and France have separate guidance on medical devices. Medical devices were incorporated into the general economic guidelines in Canada and Denmark, while no specific mention of devices was found on the Swedish HTA website. The guidance provided a range of detail on medical devices and assessment methods. French guidelines offered the most extensive consideration of characteristics [17]; they highlighted that short product life cycles, the reliance of outcomes on operator skill, the role of operator learning curves and organisational structures in which a device functions must be taken into account [20]. Both the Canadian and Danish guidelines referred to operator learning curve effects, the issue of linking results from diagnostic devices to changes in treatment outcomes for patients, and the impact of devices on organisational change [16, 21]. The Australian guidelines noted that establishing clinical effectiveness and safety using adequate data is necessary for all devices, but that the evaluation may be a cost-consequences, cost-effectiveness, costminimisation or cost-utility analysis, depending on the purpose of the evaluation [10]. None of the guidelines referred to specific new methods or research priorities to address characteristics unique to devices. These findings highlight the relative under-development of medical device economic evaluations in the context of international HTA.

3.2 Unique Characteristics of Medical Devices Of 43 studies retrieved, 11 papers met the selection criteria [4, 9–11, 23–29]. These included perspectives from industry and academia. A large number of medical devices’ characteristics have been proposed as presenting relevant methodological issues in the context of economic evaluation. These were assigned to one or more of the four broad categories in Table 1, being (1) practical difficulties in undertaking outcomes research; (2) difficulties in assessing economic costs and benefits; (3) aspects of device adoption; and (4) characteristics of device manufacturing. To assist identification of unique characteristics, we assessed whether the issues identified as specific to medical devices might be accounted for by better use of existing techniques within economic evaluation; or whether they are a result of the regulatory framework; or a characteristic of the adoption of devices within the market; or, perhaps, whether the issue is a potentially unique characteristic that requires alternative techniques. This assessment is listed in Table 1. Issues that require changes to regulation are outside the scope of this research and were not considered further. Similarly, issues identified as not unique to medical devices and that might be accounted for by better use of existing techniques within economic evaluation are not discussed further. Four characteristics (data quality issues, learning curve, wider impact of organisational change and measuring longterm outcomes from diagnostic devices) were identified as unique and presenting methodological challenges, such that they cannot be accommodated within existing, established techniques adopted for economic evaluations. 3.2.1 Data Quality Issues Difficulties in undertaking outcomes research: Whereas randomised controlled trials (RCTs) are the gold standard for the evaluation of pharmaceuticals, such trials may not be possible for medical devices [11]. Of the RCTs completed for devices, many have small sample sizes and short follow-up periods [11]. Patients may be unwilling to enter RCTs if they risk being assigned to an invasive surgical procedure, rather than to a minimally invasive one [10]. Moreover, the design of a device can make it impossible to blind clinicians and/or researchers [10, 23, 25]. To address poor recruitment, larger, multicentre trials could be employed; however, such efforts could introduce treatment centre effects from different patient volumes, and different local care protocols and training. The magnitude of such effects is difficult to isolate and evaluate. Hence, the clinical efficacy of many medical devices is informed by observational studies [9]. These can provide evidence about how a device operates in clinical practice and are

Economic Tools for New Devices Table 1 Device characteristics presenting methodological issues and methods employed to address these in the context of economic evaluations Characteristic/issue

Assessment of characteristic in the context of economic evaluation

Unique characteristic

Difficulties in undertaking randomized controlled trials

Unique characteristic for some devices, particularly invasive devices, make blinding virtually impossible. Placebo may be unethical, e.g. for implants

Potentially yes

Incremental innovation Class effects

Use modelling techniques Product differentiation can be addressed in economic evaluation

No No

Observational and post-launch data

Data can be used in economic evaluation

No

Impact on process not outcomes

Pathway between new diagnostic and patient outcome requires modelling

Potentially yes

Practical difficulties in undertaking research

Difficulties in how to assess economic costs and benefits Separating value of diagnostic tests from treatment

Use economic evaluation if appropriate pathway and outcomes adopted and robust data collection

Potentially yes

Variation in price

Many devices do not have a list price and have volume discounts. For high capital cost devices, unit cost sensitive to throughput and hence learning curve. Requires modelling

No

Multiple applications for use

Typical overhead cost allocation problem—need consistent rules

No

Accounting for all benefits of diagnostics

Need pathway analysis; challenge in data availability not economic evaluation

No

Prevalence relevant for evaluation of diagnostic devices

Issue for clinical evaluation not economic evaluation

No

More confounding factors (e.g. learning curve)

Devices used as part of a series of activities can be modelled. However, this overlaps with user competence

Yes

Issues concern how devices are adopted and are after the evaluation has been carried out

No

Devices may have wider organisational impacts

Organisational issues can be accounted for within an economic model; however, availability of data to populate the model may be challenging. Such factors can be addressed in ongoing post-implementation evaluation

Potentially yes

Effectiveness of devices is linked to user competence (i.e. learning curve)

This is a unique characteristic for user-operated devices but can be modelled

Potentially yes

Time lapse between costs and benefits

Often, challenges in data availability, but if data available can be addressed using modelling

No

These issues can be addressed within the regulation framework and partly by the manufacturers responding to the regulations

Regulatory

Aspects of how devices are adopted Rapid diffusion of devices Post-implementation evaluations difficult to conduct and may lead to suboptimal use

Manufacturing issues Different perceptions of safety concerns Shorter life cycles Lower usage Size of device manufacturers

often appropriate to measure health outcomes, particularly when RCTs are not feasible [30]. However, the hierarchical pyramid of evidence approach to judging the risk of bias of studies suggests such studies are of poorer methodological quality than RCTs [31]. Concerns with observational studies include selection bias and confounding factors, which can require the use of more sophisticated statistical methods to inform a plausible model. Device manufacturing: Several characteristics of device manufacturing contribute to the, often incomplete, evidence available to inform their economic evaluation. One major factor is the short lifespan of devices, which typically only last 18 months before requiring replacement [4,

23, 32]. Other issues include the potential number of comparators, intermediate outcomes, difficulties in blinding operators and patients, and cost. For such reasons, manufacturers are often hesitant to conduct lengthy and costly RCTs or other robust studies for devices, which can make post-implementation studies difficult to complete [4]. Moreover, many device manufacturers are small businesses for whom undertaking large comparative studies such as RCTs is difficult to manage. Moreover, the latter may be prohibitive relative to the potential benefit. This compounds the lack of ‘gold standard’ data upon which to build robust economic evaluations [23]. This characteristic of devices is unlikely to disappear in the short term, with

J. A. Craig et al.

recent figures showing that small and medium-sized enterprises (SMEs) make up almost 95 % of the medical device industry in Europe [2]. Potential consequences of rapid device adoption: Under current European regulatory requirements, devices can enter clinical practice upon receipt of conformite europeenne (CE) approval, which indicates that the product meets the requirements of the applicable European Community directives. No formal requirement for RCTs or indeed completion of any particular study design to support the application exists [11]. This lower threshold for effectiveness evidence, compared with equivalent processes for pharmaceuticals, can result in the rapid diffusion of devices before substantial evaluation has taken place [11]. The consequences of rapid diffusion for device evaluation are twofold. First, HTAs for devices typically occur 2–5 years after the device enters general use and, thus, opportunities to inform best practice in terms of cost effectiveness appear late in a device’s lifecycle [23]. Second, if post-implementation evaluations for implantable devices indicate adverse events, additional invasive procedures for device removal may be required and could be widespread [32]. 3.2.2 Learning Curve The efficacy of a medical device is a function of the device itself and the skill of users, or ‘operator-dependent’ effectiveness [4, 9, 11]. In particular, a learning curve may be associated with using a new device, unlike with pharmaceuticals, where usage (dosage, etc.) is normally defined by manufacturers on the basis of RCTs [9–11]. Trials of medical devices may be conducted with expert users who are not representative of healthcare professionals in routine clinical practice [11]. Device adoption not only requires user education, but may also necessitate organisational changes, such as training or physical alterations to settings, to achieve maximum benefits [10, 11].

3.2.4 Measuring Long-Term Outcomes from Diagnostic Devices Accounting fully for the benefits of diagnostic tests can be challenging, with the gains from certain devices potentially going beyond patients’ health outcomes [26]. The result of a diagnostic test may offer increased certainty on diagnosis, which can inform prognosis. Another characteristic specific to diagnostic tests is the challenge of separating the value of these tests in reducing uncertainty from that of subsequent investigations (such as confirmatory tests) and treatment(s) [10, 26, 32]. In other words, economic evaluations should capture not only the value of reducing the uncertainty around diagnosis but also the better patient outcomes that result from subsequent investigations and treatment(s) informed by the use of the diagnostic test. Furthermore, assessing the value of the accuracy of diagnostic tests requires accurate information on the prevalence of the underlying disease in the population of interest if the predictive value of a device is to be correctly calculated and incorporated into economic evaluations [24].

4 Objective 2 Results: How Have Unique Characteristics Been Addressed in the Published Literature? 4.1 Addressing Device-Related Issues in Economic Evaluations Published economic evaluations of medical devices and guidance from the NICE MTEP and DAP were searched to identify the methods that researchers have employed when trying to accommodate the four unique characteristics described in Sect. 3.2. Of the 60 studies identified, 43 were assessed as relevant, and their application to the four characteristics identified above is discussed.

3.2.3 Wider Impact of Organisational Change

4.1.1 Accounting for the Learning Curve

Typically, many new devices have lower budgetary impacts than new pharmaceuticals and, as a result, purchasers may be less concerned about obtaining robust costeffectiveness data [4]. However, for larger, more complex devices, such as electromechanical devices (e.g. X-ray and magnetic resonance imaging machines), significant initial investments are required to achieve long-term benefits [23], and the capital investment may be associated with material annual operating costs. Robust profiling of initial costs, annual costs and benefits, and their variation over time, is necessary to overcome any reluctance from healthcare payers to incur the initial investment [23].

Of the five papers addressing the learning curve effect, two accounted for this characteristic but did not report robust methods. One study incorporated the costs of training general practitioners (GPs) to use an electronic decision support system to order laboratory tests in primary care, but the model used did not account for system improvements as the operators became more familiar with the system [33]. In the other study, the assessment of learning curves was hampered by a lack of consideration of potential differences between study centres and normal clinical practice [34]. An indirect approach of accounting for the learning curve issue was achieved by not explicitly defining the need for

Economic Tools for New Devices

learning but, instead, measuring total costs before and after the introduction of drug-eluting stents. Hence, any learning curve improvements were incorporated in the overall costs [35]. Notably, neither paper that reported on new devices addressed the issue of a learning curve [36, 37], indicating a weakness in current economic evaluations.

published models either did not account for long-term outcomes [49] or only measured cost per diagnosis [50].

4.1.2 Addressing Data Quality Issues

Published NICE MTEP and DAP guidance were reviewed to identify economic evaluation techniques and any associated insufficiencies, and to address the four identified unique characteristics of medical devices. From the 11 medical devices (including three diagnostics) evaluated by NICE, a key theme that emerged was the lack of credible clinical evidence to assess the devices [51–61]. Four of the 11 assessments specifically noted the lack of relevant data to help inform decision makers. In each case, the recommendation was that more research should be undertaken to inform future assessments [51, 57–59]. These findings underline the theme of the lack of available data for new medical devices. The potential organisational and economic impact of the assessed medical devices on, for example, technician staffing, new buildings and extra training was considered in the NICE assessments and within the models, as required by the MTEP. This finding reinforces the view that current economic modelling techniques can account for some specific characteristics that are unique to medical devices.

Four papers address data quality issues. These papers frequently noted that economic models included limited published data/evidence, often of poor quality, taken from underpowered studies or from secondary sources, with costs drawn largely from assumptions and previously published studies [38, 39]. However, some studies did use data from RCTs, national cost databases and validated quality of life (QoL) measures. In one study, for example, data were drawn from several large trials, with costs incorporated from the Department of Health reference costs [40] and from a previous study accepted by NICE, thus providing a high-quality dataset [41]. These findings indicate that data quality issues frequently affect the clinical and cost evidence available for device evaluations/ models. 4.1.3 Measuring the Effect of Organisational Change Five studies were identified as relevant to this question. The organisational impact of introducing a new medical device may include effects on space and staff, possible resistance to its adoption and approval of the capital investment [33, 42–44]. Even if no direct reorganisation is required—for example, if the new device is replacing a similar one—cost savings may be achieved due to the improved efficacy of the new device [35]. Despite some efforts to address organisational change, one literature review noted that more subtle impacts, such as annual maintenance costs, are not always addressed within economic evaluations [45].

4.2 Information from Published Medical Technologies Evaluation Programme (MTEP) and Diagnostics Assessment Programme (DAP) Guidance

5 Objective 3 Results: Suggested Analytic Methods for Future Economic Evaluations of Medical Devices 5.1 Overview and Literature Selection A total of 94 records were identified by the searches; 23 papers were selected from the formal literature searching, supplemented by hand searching for further publications [24, 26, 62–82]. The studies were grouped into the eight approaches described below. 5.1.1 Bayesian Techniques

4.1.4 Accounting for Distance from Outcomes Six diagnostic papers indicated that Markov modelling is a useful approach to estimate transitions between different health states depending on diagnosis and treatment [38, 46], with one study also including failure to diagnose as a decision outcome [47]. O’Connor and Knoblauch [48] described the use of reports, detailing the proportion of people diagnosed and the anticipated life expectancy as a result of diagnosis, to model cost effectiveness. However, the use of simplifying assumptions to link changes in diagnosis to changes in subsequent management and outcomes also appeared within the literature, where several

Eight studies provided evidence on this topic [66–70, 72, 73, 77]. The Bayesian approach enables relevant, existing information to be incorporated formally into statistical analyses, using ‘prior distributions’, which are a summary of the pre-existing understanding of uncertain parameters [66, 69]. This approach differs from the classical or frequentist approach, which uses data from clinical trials to test a pre-specified, null hypothesis of no difference between the effects of treatment and control [69]. In the identified studies, Bayesian methods were used primarily to inform clinical trial design or data analysis and were used less within economic evaluations [66, 68–70].

J. A. Craig et al.

Bayesian methods have been recommended for use in medical device pre-market submissions to the US FDA since the late 1990s. Campbell [67] explained that this stems from many medical devices being associated with prior information that can be used to inform a new trial and that the device’s mechanism of action is usually well understood [67, 70]. Adopting a Bayesian approach enables the conduct of adaptive trials that use prior information explicitly and in which parameters are updated as the trial progresses [66, 69, 70]. Thus, researchers ‘learn’ from evidence as it accumulates, enabling robust results to be reported from smaller-sized or shorter-duration trials [66]. While the use of Bayesian methods has chiefly featured in the context of clinical trial design, several reports evaluating their use within the HTA process were identified [73, 77]. Spiegelhalter et al. [77] suggested that Bayesian methods could be used in HTAs whenever evidence synthesis is required; for example, when pooling studies of different designs in the assessment of medical devices. However, ‘further research’ issues were highlighted, including methodological developments to enable the synthesis of all evidence sources and the need for standards for researchers to follow when performing and reporting Bayesian analyses. Claxton [73] has described two applications of Bayesian theory to HTA: one measures the maximum value of additional information and a second offers a sequential method to adopt when commissioning further research. In a further paper, Bayesian methods were explored to assess how these could select probability distributions for parameters in a probabilistic sensitivity analysis for costeffectiveness studies [72]. 5.1.2 Multi-Parameter Evidence Synthesis Only Ades and Sutton [71] provided evidence on this subject. They reviewed methods used to synthesise complex evidence on multiple parameters within HTA. The proposed approach, using hierarchical models, was based on Bayesian decision modelling and the expected value of information theory. Their aim was to find a method that used all available evidence from multiple sources to (1) calculate more precise estimates of model parameters; (2) provide more accurate assessments of uncertainties in the evidence and inform further research priorities; and (3) allow evidence to be validated for consistency and reliability. 5.1.3 Early Models Several studies identified early models as useful tools to inform device development programmes, including early

market assessment, research and development (R&D) portfolio management, and first estimations of pricing and reimbursement scenarios [83–86]. Two studies provided applied examples of using modelling to inform value [37, 84]. Two studies recommended using a Bayesian approach to incorporate all available evidence to help companies manage uncertainties and identify major risks and hence make better informed choices at each decision point [76, 87]. 5.1.4 Other Methodological Aspects of Using Observational Studies Observational studies have the potential to be confounded by patient selection and observer bias. This issue has been addressed by Haro et al. [74], who applied various methods to control for selection bias in estimating treatment effects and proposed ways to assess the presence of observer bias. The study concluded that such methods could improve the validity of results from observational studies and, hence, enhance their value to HTA decision makers [74]. 5.1.5 Coverage with Evidence Development An increasingly common barrier to effective decision making in healthcare systems is the conclusion of HTAs that ‘further research is needed’ [75]. Post-implementation studies can increase the available evidence on clinical and cost effectiveness for the technology under scrutiny. ‘Coverage with Evidence Development’ (CED) recommendations can be made [32] by HTA bodies where there is uncertainty in relation to promising technologies. Under a CED approval, a device can be adopted by an HTA body provided that additional evidence is gathered to address sources of uncertainty. If such evidence is gathered satisfactorily, the device can receive HTA approval for ongoing or extended use. CED projects have been conducted by Medicare in the USA, by the Dental and Pharmaceutical Benefits Agency in Sweden and by the Ontario Health Technology Advisory Committee [75]. CED has also featured within UK HTA, specifically for a transcatheter aortic valve implantation (TAVI), which, following the publication of the NICE interventional procedure guidance in March 2012, allowed the use of TAVI under special arrangements [82]. 5.1.6 Constructive Technology Assessment Constructive Technology Assessment (CTA) [79] aims to produce better technology through the early involvement of a range of stakeholders to facilitate social learning about the technology and potential impacts. It claims to be an

Economic Tools for New Devices

alternative to a Bayesian approach to manage the dynamic nature of medical devices. It is described as complementary to HTA, especially for devices introduced in an early stage of development. CTA adopts a different approach to evaluation, depending on the stage of product take-up. It focuses on the technology and its associated environment. With CTA, a cost-effectiveness study could be performed when a sufficient number of users have experience of the technology. 5.1.7 Valuing Diagnostic Information Current approaches to economic evaluation may fail to capture the value of the reduced uncertainty from additional diagnostic information, and further research is required to ‘measure’ or account for these neglected attributes [26]. Mushlin et al. [81] conducted a trial to measure the value of stress testing in patients with chest pain suggestive of coronary artery disease. At 1 week after the stress test, patients experienced a statistically significant increase in perceived life expectancy and a reduction in anxiety [88]. The short-form 36 QoL questionnaire was unable to detect such benefits. The authors concluded that these measurable psychological benefits should be standard components of diagnostic test evaluation [81]. 5.1.8 Eliciting User Requirements for Devices and Valuing the Learning Curve It is well recognised that the efficacy and safety of a medical device is operator dependent [4, 9, 11]. To account for the value that users, rather than patients, place on medical devices, Martin et al. [80] used open-ended, semistructured interviews with potential clinical users of a new imaging device to investigate the clinical need and potential benefits to them. The study identified a number of significant clinical needs that required the device design to be changed and also additional potential barriers (e.g. staffing time pressures) to safe and effective adoption of the device that were not considered by the device developers prior to the interviews [80]. The learning curve can cause variation in clinical responses to a new medical device [9–11]. Cook et al. [78] undertook a literature review to establish whether a learning curve can be quantified. The 21 studies gave explicit information concerning previous experience of the operator(s), with 32 different definitions of procedure time. The approach allowed learning to be quantified for the technology, although no formal statistical estimation of the curve was possible. The authors concluded that standardised reporting of devices, with adequate learning curve details was necessary to inform trial design and cost-effectiveness analysis [78].

6 Discussion 6.1 Overview This review identified the numerous and complex characteristics of medical devices and the associated issues they may pose when conducting economic evaluations. A key characteristic of medical devices is that their successful adoption does not just impact or rely on the patient. Instead, factors such as operator-dependent effectiveness, the learning curve and wider organisational changes may influence their effectiveness [4, 9–11]. From the published economic evaluations reviewed, it appeared that such factors were often either not addressed or insufficiently addressed. Device design alterations over time can also introduce incremental innovation [9, 11], which should be taken into account, for example, by using iterative modelling approaches. Although many of the unique characteristics we have identified are shared among diagnostic and non-diagnostic devices, the former also have additional features that warrant further consideration. For example, the additional benefits a diagnosis may bring, such as reassurance or reduced anxiety for patients, should be considered, and an effective way of quantifying these gains is required [26, 81]. This review suggests that current models fail to incorporate these benefits fully and often over-simplify the diagnostic chain of benefits for the purposes of economic evaluation. While many of the features identified are intrinsic to the device, some reflect a regulatory framework that appears to facilitate market entry for some devices without outcome data from clinical studies. Commercial pressures facing SMEs, which make up the majority (95 %) of device manufacturers [2], typically prohibit the commissioning of RCTs, which are currently the key source of robust clinical and safety data for pharmaceuticals. Furthermore, the less stringent regulatory requirements faced by devices compared with pharmaceuticals contributes to the lack of datasets to inform economic evaluations prior to devices entering the market. Although the literature reviewed provided an extensive overview of the characteristics of devices, it contained little or no explicit debate on the comparative importance of individual characteristics in the context of economic evaluations. This finding suggests that greater attention is required to establish whether a particular medical device feature can be accounted for by appropriate use of existing techniques, or whether it requires the adoption of alternative methods. We have become aware of two relevant papers published since December 2011 [89, 90]. These confirm that methodological difficulties exist in the evaluation of

J. A. Craig et al.

medical devices. The complicating factors identified are similar to those in this study, including specific characteristics of devices, the regulatory framework for devices and the structure of the medical device industry [90]. 6.2 Recommendations Evaluations of medical devices are likely to benefit from a collaborative effort between stakeholders from HTA, government health bodies, regulatory bodies and manufacturers to agree specific improvements to data collection. These improvements could include a greater use of early modelling by manufacturers to identify the outcomes the product needs to achieve. Other developments could include further evaluation of the Bayesian methods developed by the FDA [70], and their application to inform clinical trials for medical devices. Moreover, greater transparency of the clinical and cost-effectiveness analysis of medical devices needs to be encouraged, for example, by promoting publication of device economic evaluations and increasing availability of education on methodologies. HTA agencies should encourage the development of existing modelling techniques to accommodate device characteristics more completely and stipulate the modelling approach required in the economic evaluation of medical devices. With flexible model structures and appropriate data capture, it should be possible to measure incremental innovation, the learning curve and organisational aspects of introducing new technologies [83]. Techniques already exist to account for these aspects; however, training in those techniques may be required. There are specific related cost issues, which include the allocation of initial investment costs over competing uses that change over time and which may include periods of non-use. Such costing issues can be addressed through the production of standard guidelines by HTA bodies to ensure a common approach. Early modelling should also be a focus of further research. These techniques have a range of mathematical complexity, and the circumstances in which each approach is appropriate should be explored. Key features for exploration could include the number of, and precision required for, data inputs; potential sources of such data; the ability to model variation and uncertainty; and the acceptability of the approach to those with no in-depth knowledge of economic modelling. The creation of case studies to demonstrate utilisation of modelling in planning outputs would be of benefit. Greater use of CED may also aid device evaluation [32], permitting the new device to be funded while evidence is collected. However, given the complexity of organising such projects, CED is likely to apply in only a limited

number of cases. Guidance on how to establish and operate such projects is required from funding agencies. The process of HTA could apply to all new medical devices, with decisions available at the launch of the device; however, time and commercial and resource pressures currently seem to preclude this. Only a small proportion of devices currently entering the market in England and Wales are evaluated by the NICE MTEP, with most devices entering healthcare systems directly at a local purchasing level. Healthcare purchasers should be encouraged to increase their knowledge of economic evaluation methods through training activities, so that they can apply, or ask, for economic analyses when buying medical devices. Similar educational support may be required for the device manufacturers. This group comprises businesses of widely varying sizes, with some having little knowledge of, and minimal experience in, collecting and evaluating clinical and economic data. Creating a wider appreciation of the issues of evidencebased healthcare and economic evaluation in the context of medical devices may help the NHS adopt more efficient and cost-effective medical devices and diagnostics more rapidly and consistently.

7 Conclusion From this review, it is clear that using economic evaluation to inform decisions on medical devices is at an earlier stage of development than the equivalent processes employed for pharmaceuticals. Many of the characteristics unique to medical devices can be addressed by existing economic evaluation techniques. Both HTA bodies and local-level decision makers should be encouraged to require and expect manufacturers to make better use of the economic evaluation toolbox in their submissions for approval. However, as is often the case in healthcare, a collaborative approach between HTA, decision makers, regulatory bodies, researchers and manufacturers is likely to create an environment that will generate more economic evaluations. Changes to the ways in which data are collected, both prior to a device’s entry to market and during post-implementation stages, will enable the development of more robust economic evaluations. Acknowledgments All authors contributed to the conception and design of the work and agreed overall methodologies. Julie Glanville and Andrew Sims devised and ran the literature search strategies. Louise Carr and Joyce Craig selected included studies, synthesised findings and prepared the initial draft of the full report. All authors commented on drafts, revising these to improve intellectual content. All approved the final version for publication. Hayward Medical Communications has contributed both medical writing and editorial support in the development of this manuscript. Joyce Craig is the guarantor for the overall content.

Economic Tools for New Devices Disclosures This work was developed as part of a contract with the NICE MTEP for evidence preparation and assessment services. The authors were solely responsible for the design of the methods and retained editorial control throughout the development and publication of the report. At the time of review, Cynthia Iglesias held a personal fellowship in Health Services Research and Health of the Public funded by the Medical Research Council. The authors have no conflicts of interest that are directly relevant to the content of this article.

References 1. U. S. Food Drug Administration. Medical devices: is the product a medical device? U. S. Food Drug Administration, Maryland. 2013. www.fda.gov/medicaldevices/deviceregulationandguidance/ overview/classifyyourdevice/ucm051512.htm. Accessed 2 April 2014. 2. MedTech Europe. The European Medical Technology Industry in figures. Brussels: MedTech Europe; 2013. 3. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM. Cost-effectiveness and cost-benefit analyses in the medical literature. Are the methods being used correctly? Ann Intern Med. 1992;116:238–44. 4. Cookson R, Hutton J. Regulating the economic evaluation of pharmaceuticals and medical devices: a European perspective. Health Policy. 2003;63:167–78. 5. National Institute for Health Care Excellence. Medical technologies evaluation programme. London: NICE; 2013. 6. French-Mowat E, Burnett J. How are medical devices regulated in the European Union? J R Soc Med. 2012;105(Suppl 1):S22–8. 7. National Institute for Health and Clinical Excellence. Medical technologies evaluation programme: methods guide. London: NICE; 2011. 8. National Institute for Health and Clinical Excellence. 1999–2009 a decade of excellence. London: NICE; 2009. 9. Taylor RS, Iglesias CP. Assessing the clinical and cost-effectiveness of medical devices and drugs: are they that different? Value Health. 2009;12:404–6. 10. Drummond M, Griffin A, Tarricone R. Economic evaluation for devices and drugs—same or different? Value Health. 2009;12:402–4. 11. Sorenson C, Tarricone R, Siebert M, Drummond M. Applying health economics for policy decision making: do devices differ from drugs? Europace. 2011;13(Suppl 2):54–8. 12. National Institute for Health and Care Excellence. Medical technology evaluation programme: definition of technologies which fall within the remit of the programme. London: NICE; 2013. 13. Medical Services Advisory Committee. Funding for new medical technologies and procedures: application and assessment guidelines. Canberra: Medical Services Advisory Committee; 2005. 14. Medical Services Advisory Committee. Guidelines for the assessment of diagnostic technologies. Canberra: Medical Services Advisory Committee; 2005. 15. Medical Services Advisory Committee. Economics section of the MSAC guidelines. Canberra: Medical Services Advisory Committee; 2009. 16. Canadian Agency for Drugs Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. Ottowa: CADTH; 2006. 17. Canadian Agency for Drugs Technologies in Health. Indirect evidence: indirect treatment comparisons in meta-analysis. Ottowa: CADTH; 2009.

18. Haute Autorite´ de Sante´. Rapid assessment method for assessing medical and surgical procedures. Paris: Haute Autorite´ de Sante´; 2007. 19. Haute Autorite´ de Sante´. General method for assessing health technologies. Paris: Haute Autorite´ de Sante´; 2007. 20. Haute Autorite´ de Sante´. Medical device assessment in France: guidebook. Paris: Haute Autorite´ de Sante´; 2009. 21. Kristensen F, He S. Health technology assessment handbook. Copenhagen: Danish Centre for Health Technology Assessment, National Board of Health; 2007. 22. National Institute for Health and Care Excellence. Published medical technologies guidance. NICE, London. 2014. http:// www.nice.org.uk/guidance/mt/published/index.jsp. Accessed 5 Jan 2014. 23. Ferrusi IL, Ames D, Lim ME, Goeree R. Health technology assessment from a Canadian device industry perspective. J Am Coll Radiol. 2009;6:353–9. 24. Foley K, Lim S, Schulman KL. Methodological considerations in modeling the economic value of diagnostic accuracy. In: ISPOR 16th Annual International Meeting; Baltimore; 21–25 May 2011. 25. Gimenes F, de O Machado F, Quiroz ME. Health technology assessment applied to medical devices in Latin America: what must be assessed. In: ISPOR 3rd Latin America Conference; Mexico City; 8–10 September 2011. 26. Hutton J, Trueman P, Henshall C. Coverage with evidence development: an examination of conceptual and policy issues. Int J Technol Assess Health Care. 2007;23:425–32. 27. Manca A. Economic evaluation of medical devices and drugs— same or different? Value Health. 2009;12(4):401. 28. Tesar T, Babela R. Quality assurance of fourth hurdle concerning medical devices. In: ISPOR 13th Annual European Congress; Prague; 6–9 November 2010. 29. Drummond M, Weatherly H, Ferguson B. Economic evaluation of health interventions. BMJ. 2008;337:a1204. 30. Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid hierarchy? Circulation. 2008;118:1675–84. 31. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. London: NICE; 2013. 32. Sassi F, McKee M, Roberts JA. Economic evaluation of diagnostic technology. Methodological challenges and viable solutions. Int J Technol Assess Health Care. 1998;13:613–30. 33. Poley MJ, Edelenbos KI, Mosseveld M, van Wijk MA, de Bakker DH, van der Lei J, et al. Cost consequences of implementing an electronic decision support system for ordering laboratory tests in primary care: evidence from a controlled prospective study in the Netherlands. Clin Chem. 2006;53:213–9. 34. Cole JH, Chunn VM, Morrow JA, Buckley RS, Phillips GM. Cost implications of initial computed tomography angiography as opposed to catheterization in patients with mildly abnormal or equivocal myocardial perfusion scans. J Cardiovasc Comput Tomogr. 2007;1:21–6. 35. Rihal CS, Ryan JL, Singh M, Lennon RJ, Bresnahan JF, Liesinger JT, et al. Clinical and economic outcomes after introduction of drug-eluting stents. Am J Manag Care. 2010;16:580–7. 36. Clegg JP, Guest JF. Modelling the cost-utility of bio-electric stimulation therapy compared to standard care in the treatment of elderly patients with chronic non-healing wounds in the UK. Curr Med Res Opin. 2007;23:871–83. 37. Regier DA, Petrou S, Henderson J, Eddama O, Patel N, Strohm B, et al. Cost-effectiveness of therapeutic hypothermia to treat neonatal encephalopathy. Value Health. 2010;13:695–702. 38. Palmer AJ, Valentine WJ, Chen R, Mehin N, Gabriel S, Bregman B, et al. A health economic analysis of screening and optimal treatment of nephropathy in patients with type 2 diabetes and hypertension in the USA. Nephrol Dial Transpl. 2008;23:1216–23.

J. A. Craig et al. 39. Watkins RG, Gupta A, Watkins RG. Cost-effectiveness of imageguided spine surgery. Open Orthop J. 2011;4:228–33. 40. Department of Health. Reference costs 2005–06. London: Department of Health; 2006. 41. Remak E, Manson S, Hutton J, Brasseur P, Olivier E, Gershlick A. Cost-effectiveness of the Endeavor stent in de novo native coronary artery lesions updated with contemporary data. EuroIntervention. 2010;5(7):826–32. 42. Bradley SM, Levy WC, Veenstra DL. Cost-consequences of ultrafiltration for acute heart failure: a decision model analysis. Circ Cardiovasc Qual Outcomes. 2009;2:566–73. 43. Krug B, Van Zanten A, Pirson AS, Crott R, Borght TV. Activitybased costing evaluation of a [(18)F]-fludeoxyglucose positron emission tomography study. Health Policy. 2009;92:234–43. 44. Tarricone R, Aguzzi G, Musi F, Fariselli L, Casasco A. Costeffectiveness analysis for trigeminal neuralgia: cyberknife vs microvascular decompression. Neuropsychiatr Dis Treat. 2008;4:647–52. 45. Lowrance WT, Tarin TV, Shariat SF. Evidence-based comparison of robotic and open radical prostatectomy. Sci World J. 2010;10:2228–37. 46. Meads CA, Cnossen JS, Meher S, Juarez-Garcia A, ter Riet G, Duley L, et al. Methods of prediction and prevention of preeclampsia: systematic reviews of accuracy and effectiveness literature with economic modelling. Health Technol Assess. 2008;12:1–270. 47. Mueller D, Gandjour A. Cost-effectiveness of using clinical risk factors with and without DXA for osteoporosis screening in postmenopausal women. Value Health. 2009;12:1106–17. 48. O’Connor DP, Knoblauch MA. Electrocardiogram testing during athletic preparticipation physical examinations. J Athl Train. 2010;45:265–72. 49. Collins R, Cranny G, Burch J, Aguiar-Ibanez R, Craig D, Wright K, et al. A systematic review of duplex ultrasound, magnetic resonance angiography and computed tomography angiography for the diagnosis and assessment of symptomatic, lower limb peripheral arterial disease. Health Technol Assess. 2007;11: 1–184. 50. Wordsworth S, Buchanan J, Regan R, Davison V, Smith K, Dyer S, et al. Diagnosing idiopathic learning disability: a cost-effectiveness analysis of microarray technology in the National Health Service of the United Kingdom. Genomic Med. 2008;1:35–45. 51. National Institute for Health and Clinical Excellence. The EOS 2D/3D imaging system (NICE diagnostic guidance 1). London: NICE; 2011. 52. National Institute for Health and Clinical Excellence. Elucigene FH20 and LIPOchip for the diagnosis of familial hypercholesterolaemia (NICE diagnostic guidance 2). London: NICE; 2011. 53. National Institute for Health and Care Excellence. New generation cardiac CT scanners (Aquilion ONE, Brilliance iCT, Discovery CT750 HD and Somatom Definition Flash) for cardiac imaging in people with suspected or known coronary artery disease in whom imaging is difficult with earlier generation CT scanners (NICE diagnostic guidance 3). London: NICE; 2012. 54. National Institute for Health and Clinical Excellence. SeQuent please balloon catheter for in-stent coronary restenosis (NICE medical technology guidance 1). London: NICE; 2010. 55. National Institute for Health and Clinical Excellence. moorLDI2BI: a laser doppler blood flow imager for burn wound assessment (NICE medical technology guidance 2). London: NICE; 2011. 56. National Institute for Health and Clinical Excellence. CardioQODM oesophageal Doppler monitor (NICE medical technology guidance 3). London: NICE; 2011. 57. National Institute for Health and Clinical Excellence. BRAHMS copeptin assay to rule out myocardial infarction in patients with

58.

59.

60.

61.

62.

63.

64.

65.

66.

67. 68. 69. 70.

71.

72.

73.

74.

75.

76.

77.

acute chest pain (NICE medical technology guidance 4). London: NICE; 2011. National Institute for Health and Clinical Excellence. MIST therapy system for the promotion of wound healing in chronic and acute wounds (NICE medical technology guidance 5). London: NICE; 2011. National Institute for Health and Clinical Excellence. Ambulight PDT for the treatment of non-melanoma skin cancer (NICE medical technology guidance 6). London: NICE; 2011. National Institute for Health and Clinical Excellence. Inditherm patient warming mattress for the prevention of inadvertent hypothermia (NICE medical technology guidance 7). London: NICE; 2011. National Institute for Health and Clinical Excellence. The VeriQ system for assessing graft flow during coronary artery bypass graft surgery (NICE medical technology guidance 8). London: NICE; 2011. Claxton K. Bayesian approaches to the value of information: implications for the regulation of new pharmaceuticals. Health Econ. 1999;8:269–74. Henshall C, Mardhani-Bayne L, Fronsdal KB, Klemp M. Interactions between health technology assessment, coverage, and regulatory processes: emerging issues, goals, and opportunities. Int J Technol Assess Health Care. 2011;27:253–60. Mauskopf JA, Paul JE, Grant DM, Stergachis A. The role of costconsequence analysis in healthcare decision-making. Pharmacoeconomics. 1998;13:277–88. Schreyogg J, Baumler M, Busse R. Balancing adoption and affordability of medical devices in Europe. Health Policy. 2009;92:218–24. Bonangelino P, Irony T, Liang S, Li X, Mukhi V, Ruan S, et al. Bayesian approaches in medical device clinical trials: a discussion with examples in the regulatory setting. J Biopharm Stat. 2011;21:938–53. Campbell G. Bayesian statistics in medical devices: innovation sparked by the FDA. J Biopharm Stat. 2011;21:871–87. Grunkemeier GL, Payne N. Bayesian analysis: a new statistical paradigm for new technology. Ann Thorac Surg. 2003;74:1901–8. Hobbs BP, Carlin BP. Practical Bayesian design and analysis for drug and device clinical trials. J Biopharm Stat. 2007;18:54–80. US Food and Drug Administration. Guidance for the use of Bayesian statistics in medical device clinical trials. Maryland: US Food and Drug Administration; 2010. Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. J R Stat Soc Ser A (Statistics in Society). 2006;169:5–35. Armero C, Garcia-Donato G, Lopez-Quilez A. Bayesian methods in cost-effectiveness studies: objectivity, computation and other relevant aspects. Health Econ. 2009;19:629–43. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18:341–64. Haro JM, Kontodimas S, Negrin MA, Ratcliffe M, Suarez D, Windmeijer F. Methodological aspects in the assessment of treatment effects in observational health outcomes studies. Appl Health Econ Health Policy. 2006;5:11–25. Healthcare Improvement Scotland. Coverage with evidence development in NHS Scotland. Edinburgh: Healthcare Improvement Scotland; 2008. Pietzsch JB, Pate-Cornell ME. Early technology assessment of new medical devices. Int J Technol Assess Health Care. 2008;24:36–44. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology assessment: a review. Health Technol Assess. 2001;4:1–130.

Economic Tools for New Devices 78. Cook JA, Ramsay CR, Fayers P. Using the literature to quantify the learning curve: a case study. Int J Technol Assess Health Care. 2007;23:255–60. 79. Douma KF, Karsenberg K, Hummel MJ, Bueno-de-Mesquita JM, van Harten WH. Methodology of constructive technology assessment in health care. Int J Technol Assess Health Care. 2007;23:162–8. 80. Martin JL, Clark DJ, Morgan SP, Crowe JA, Murphy E. A usercentred approach to requirements elicitation in medical device development: a case study from an industry perspective. Appl Ergon. 2011;43:184–90. 81. Mushlin AI, Kern LM, Paris M, Lambert DR, Williams G. The value of diagnostic information to patients with chest pain suggestive of coronary artery disease. Med Decis Making. 2005;25:149–57. 82. National Institute for Health Clinical and Excellence. Transcatheter aortic valve implantation for aortic stenosis (NICE interventional procedures guidance 421). London: NICE; 2012. 83. Girling A, Young T, Brown C, Lilford R. Early-stage valuation of medical devices: the role of developmental uncertainty. Value Health. 2010;13(5):585–91. 84. McAteer H, Cosh E, Freeman G, Pandit G, Wood P, Lilford R. Cost-effectiveness analysis at the development phase of a potential health technology: examples based on tissue

85.

86.

87.

88.

89.

90.

engineering of bladder and urethra. J Tissue Eng Regen Med. 2007;1:343–9. Ijzerman MJ, Steuten LMG. Early assessment of medical technologies to inform product development and market access: a review of methods and applications. Appl Health Econ Health Policy. 2011;9(5):331–47. Hartz S, John J. Contribution of economic evaluation to decision making in early phases of product development: a methodological and empirical review. Int J Technol Assess Health Care. 2008;24(4):465–72. Vallejo-Torres L, Steuten LMG, Buxton MJ, Girling AJ, Lilford RJ, Young T. Integrating health economics modeling in the product development cycle of medical devices: a Bayesian approach. Int J Technol Assess Health Care. 2008;24(4):459–64. Mushlin AI, Kern LM, Paris M, Lambert DR, Williams G. The value of diagnostic information to patients with chest pain suggestive of coronary artery disease. Med Decis Making. 2005;25(2):149–57. Chapman AM, Taylor CA, Girling AJ. Are the UK systems of innovation and evaluation of medical devices compatible? The role of NICE’s medical technologies evaluation programme (MTEP). Appl Health Econ Health Policy. 2014;12(4):347–57. Kirisits A, Redekop WK. The economic evaluation of medical devices: challenges ahead. Appl Health Econ Health Policy. 2013;11(1):15–26. doi:10.1007/s40258-012-0006-9.