Research
Evaluating the quality and use of economic data in decisions about essential medicines Corrina Moucheraud,a Veronika J Wirtzb & Michael R Reicha Objective To evaluate the quality of economic data provided in applications to the World Health Organization (WHO) Model List of Essential Medicines and to evaluate the role of these data in decision-making by the expert committee that considers the applications. Methods We analysed applications submitted to the WHO Expert Committee on the Selection and Use of Essential Medicines between 2002 and 2013. The completeness of data on the price and cost–effectiveness of medicines was extracted from application documents and coded using a four-point scale. We recorded whether or not the expert committee discussed economic information and the outcomes of each application. Associations between the completeness of economic data and application outcomes were assessed using χ2 tests. Findings The expert committee received 134 applications. Only eight applications (6%) included complete price data and economic evaluation data. Many applicants omitted or misinterpreted the economic evaluation section of the application form. Despite the lack of economic data, all applications were reviewed by the committee. There was no significant association between the completeness of economic information and application outcomes. The expert committee tried to address information gaps in applications by further review and analysis of data related to the application. Conclusion The World Health Organization should revise the instructions to applicants on economic data requirements; develop new mechanisms to assist applicants in completing the application process; and define methods for the use of economic data in decision-making.
Introduction Essential medicines are those that satisfy the priority health care needs of the population.1 The World Health Organization (WHO) introduced the first Model List of Essential Medicines in 1977. Countries are encouraged to use the model list as a guide for their decisions on pharmaceutical selection and procurement. Between 1977 and 2007, over 130 countries introduced national lists of essential medicines, modifying the WHO model list for their national context.2 The model list is updated every two years, following an application and review process by an expert committee. Any individual or institution may submit an application. Before each committee meeting, applications are published on a website for public comment and experts (committee members and external advisors) review each application.3 These reviews and any public comments are also published on the website before the committee meeting. Since 2002, essential medicines have been selected via an evidence-based process, with due regard to public health relevance, evidence on efficacy and safety and comparative cost–effectiveness.1 Detailed information on the decision criteria can be found at http://www.who.int/selec�tion_medicines/en/. The model list is divided into a core list and a complementary list. The core list includes medicines that meet the criteria of efficacy, safety and comparative cost–effectiveness,1 but expensive patented medicines are not necessarily excluded.3,4 The complementary list includes medicines that may require specialized facilities, that are consistently more costly, or less cost–effective in a variety of settings.5 Additions to the model list can have a major impact on global and national decisions, with significant budgetary, ethical and health implications.6,7 Other studies have examined application processes and criteria for national decisions on
adopting new vaccines,8 national health technology assessment programmes9,10 and financing decisions for health technologies.11 A recent study found that applications to the model list for mental health medicines generally provided low-quality and incomplete evidence across several required dimensions.12 There is no global consensus on how to use economic data in decision-making for medicines and health technologies.13–17 Here, we evaluate the extent to which applicants and the expert committee adhere to the instructions and procedures on economic data and analysis for applications to the model list. We assess the application process rather than the substance of decisions (specifically, compliance with instructions for applicants and the quality of data on price and on cost–effectiveness). We also assess whether economic considerations are included in the final report by the expert committee. The overall goal of this study is to improve the quality, transparency and clarity of the process for reviewing applications to the model list.
Methods Data set We included applications to the model list for medicines intended for use in adults. We analysed final reports from the twelfth (2002) to nineteenth (2013) meetings of the WHO expert committee; the sixteenth meeting reviewed paediatric medicines only and was not included. We did not review applications for new formulations of existing medicines or applications for reinstatement.
Definition of variables We assessed the extent to which applicants complied with the instructions provided. The instructions ask applicants to provide “a range of costs of the proposed medicine” that
Harvard TH Chan School of Public Health, Boston, Department of Global Health and Population, 665 Huntington Avenue, Building 1, Boston, Massachusetts 02115, United States of America (USA). b Boston University School of Public Health, Department of Global Health, Boston, USA. Correspondence to Corrina Moucheraud (email: [email protected]). (Submitted: 10 November 2014 – Revised version received: 15 June 2015 – Accepted: 19 June 2015 – Published online: 14 August 2015 ) a
Bull World Health Organ 2015;93:693–699 | doi: http://dx.doi.org/10.2471/BLT.14.149914
693
Research Corrina Moucheraud et al.
Essential medicines
reflect “average generic world market prices”, with a clearly specified source of data. The WHO-recommended sources for these data include the International Drug Price Indicator Guide, the United Nations Children’s Fund, Médecins Sans Frontières or WHO itself. We allocated one point for each of the following: price given; offered range of price or average or median price; clear source; WHO-recommended source. The application was classified as complete if the application attained a score of four; applications with scores between one and three points (inclusive) were classified as incomplete. If no pricing data were provided, this was considered missing (zero points). If the applicant explicitly mentioned that no pricing data exist, this was classified as not available. Completeness of economic evaluation data was scored based on WHO instructions for comparative cost–effectiveness presented as range of cost per routine outcome, and a clear citation, preferably of WHO-recommended sources. We also accepted citations from the published scientific literature as one of the recommended data sources. Adherence was classified as complete if applications met all of the following criteria: economic evaluation value provided; comparator given; clear source; WHO-recommended source. As with price, applications scoring between one and three inclusive were classified as incomplete. Omitted economic evaluation sections were classified as missing; cases where the applicant explicitly noted the lack of available data on economic evaluation were classified as not available. There were instances in which the applicant provided administration costs for medicines, without an outcomesbased denominator, under the heading of cost–effectiveness. These applications were classified as having provided economic evaluation data that were not applicable. Based on the final reports from each meeting, the outcome was coded as a dichotomous variable: either rejected or deferred (coded 0) versus accepted to the core or complementary lists (coded 1). Applicant type and therapeutic class were categorized based on information provided in the application. We recorded the proportion of applications reviewed by the committee that were added to the model list, by therapeutic class. We also recorded whether
694
the discussion and decisions mentioned price or economic evaluation.
Data entry and analysis A data entry form was generated based on the variables listed above. There were two independent data reviewers, each of whom extracted data from the applications; discrepancies were resolved by consensus. Application data were entered using CSPro version 5 (United States Census Bureau, Washington, United States of America). Qualitative information from discussions and application decision categories were coded by hand. We extracted the narrative sections about the reviewed medicines from the final report for each committee meeting; text concerning economic considerations was collated in an Excel spreadsheet. We identified whether the report mentioned economic data or the absence of such data, as well as which data sources were cited (i.e. the application itself or new additional data). Associations between the completeness of economic data and the application outcomes were assessed using χ2 tests in Stata version 12.1 (StataCorp. LP, College Station, USA). We conducted sensitivity analyses that dropped each scoring criterion in turn.
Results We analysed 134 applications for new medicines from 2002 to 2013 (Table 1). Application authors varied by year, but were most commonly submitted by WHO departments (25.4%; 34/134), academic institutions (22.4%; 30/134) and nongovernmental organizations (21.6%; 29/134). Therapeutic classes of medicines also varied between meetings; applications for cancer and diabetes medicines were more common in recent years. Of 134 applications analysed, 52 were rejected and decisions were deferred for an additional 10. Among the 72 applications added to the model list, 67 were added to the core list and five to the complementary list. Across all meeting years, only 6.0% of applications (8/134) provided complete price and economic evaluation data (Table 2). Since no time trends were apparent, we report totals for all years; analyses by year are available from the corresponding author. If we include applications that reported economic data were not available, assuming that these
applicants made an effort to include data but that none were available, 13.4% (18/134) of applications fully followed the instructions on price and economic evaluation. Only 20.9% (28/134) and 18.7% (25/134) of applications provided complete price and economic evaluation information, respectively. Over a third (36.6%; 49/134) included no economic evaluation information, one-fifth (20.9%; 28/139) stated that economic evaluation information was unavailable and 17.9% (24/134) presented financial information that was unrelated to economic evaluation (e.g. total treatment cost). All applications were reviewed by the committee. For the applications with complete economic data, 52.0% (13/25) was added to the model list. When we included applications that stated that economic data were not available, 50.9% (27/53) was added to the model list. Among applications with no price data or no economic evaluation, over one-third and over half, respectively, of these applications were added to the model list. There was variation in the extent of economic information provided by applicant type. Academic authors were most likely to provide complete price and economic evaluation information. Applicants from WHO did not include economic evaluation information in half of their applications and industry applicants very rarely provided complete information on price or economic evaluation. Detailed data on the completeness of economic information by applicant type and therapeutic class are available from the corresponding author. For applications with complete price data, 64.3% (18/28) were added to the model list, a percentage that was not significantly higher than for applications with incomplete or absent price information (50.5%; 50/99; P = 0.2). For applications with complete economic evaluation information 52.0%, (13/25) were added to the model list, compared to 54.1% for those with incomplete, missing, inapplicable or unavailable information (59/109; P = 0.9). We explored whether the committee’s discussions of economic factors reflected the content of the applications. As shown in Table 3, price data or economic evaluations were discussed in some cases, even where this information had not been included in the application. Our qualitative analysis suggests four
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al.
Table 1. Applications to the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Application characteristic
No. of applications (%) Meeting of the WHO Expert committee (year)
Total
12th (2002) 13th (2003) 14th (2005) 15th (2007) 17th (2009) 18th (2011) Applications Additions to the list Applicant WHO (internal) Academia NGO Industry Othera Class of medicine HIV Infections Mental health Tuberculosis and malaria Cancer RH and MCH CVD and diabetes Otherb
19th (2013)
16 (100) 11 (68.8)
7 (100) 3 (42.9)
17 (100) 7 (41.2)
22 (100) 19 (86.4)
35 (100) 12 (34.3)
17 (100) 10 (58.8)
20 (100) 10 (50.0)
134 (100) 72 (53.7)
12 (75.0) 0 (0.0) 0 (0.0) 3 (18.8) 1 (6.2)
4 (57.1) 1 (14.3) 2 (28.6) 0 (0.0) 0 (0.0)
2 (11.8) 1 (5.9) 5 (29.4) 5 (29.4) 4 (23.5)
7 (31.8) 3 (13.6) 3 (13.6) 6 (27.3) 3 (13.6)
4 (11.4) 15 (42.9) 11 (31.4) 0 (0.0) 5 (14.3)
3 (17.6) 6 (35.3) 0 (0.0) 6 (35.3) 2 (11.8)
2 (10.0) 4 (20.0) 8 (40.0) 1 (5.0) 5 (25.0)
34 (25.4) 30 (22.4) 29 (21.6) 21 (15.7) 20 (14.9)
10 (62.5) 1 (6.3) 0 (0.0) 4 (25.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (6.3)
0 (0.0) 4 (57.1) 0 (0.0) 1 (14.3) 0 (0.0) 1 (14.3) 1 (14.3) 0 (0.0)
3 (17.6) 3 (17.6) 1 (5.9) 0 (0.0) 0 (0.0) 7 (41.2) 0 (0.0) 3 (17.6)
7 (31.8) 4 (18.2) 1 (4.5) 2 (9.1) 1 (4.5) 2 (9.1) 1 (4.5) 4 (18.2)
2 (5.7) 5 (14.3) 14 (40.0) 1 (2.9) 4 (11.4) 2 (5.7) 1 (2.9) 6 (17.1)
3 (17.6) 2 (11.8) 0 (0.0) 3 (17.6) 3 (17.6) 0 (0.0) 2 (11.8) 4 (23.5)
0 (0.0) 2 (10.0) 2 (10.0) 2 (10.0) 4 (20.0) 0 (0.0) 1 (5.0) 9 (45.0)
25 (18.7) 21 (15.7) 18 (13.4) 13 (9.7) 12 (9.0) 12 (9.0) 6 (4.5) 27 (20.2)
CVD: cardiovascular disease; HIV: human immunodeficiency virus; MCH: maternal and child health; NGO: nongovernmental organization; RH: reproductive health; WHO: World Health Organization. a Government groups and co-applicants of different types (e.g. NGO-academic partnerships). b Includes analgesics, antidotes, gastrointestinal medicines, antivirals, anaesthesia and sedatives, ophthalmology preparations and nutritional supplements including vitamins and minerals.
ways in which this happened. First, the committee sometimes noted the lack of information in the application (two applications with missing price data, seven applications with missing economic evaluation); in three of these cases, the medicines were nonetheless added to the core list. Second, the committee sometimes referenced economic information, suggesting an alternative, although unspecified, source of supplementary data (six applications). Third, the committee sometimes did further research: for example, the report mentioned a review of cost–effectiveness data prepared by the secretariat, or indicated that the secre-
tariat (usually comprised of WHO staff who support the committee process) performed its own economic evaluation (four applications). Fourth, in two instances where an economic evaluation was missing, the committee discussed price data included in the application but referred to it as “cost–effectiveness,” suggesting a misinterpretation. We explored whether the distinction between core and complementary lists was followed in the decision process. A feature that distinguishes the complementary list is its inclusion of medicines requiring specialized diagnostic or monitoring facilities and/or
specialist medical care and/or specialist training.5 According to our analysis, 50% (46/92) of applications that we classified as belonging to the complementary list were nonetheless added to the core list by the committee. This included all psychiatric medicines and antiretroviral medicines for treating human immunodeficiency virus infections. We conducted sensitivity analyses, to examine the relative importance of each scoring criterion. The most influential variables were: for price data, the average generic world price and for economic evaluations, the choice of comparator (i.e. presenting relative or
Table 2. Completeness of economic information provided in applications to the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Application status
No. of applications (%)a Price (n = 134) Complete
Applied Added to list
28 (20.9) 18 (64.3)
Incomplete Not available 79 (59.0) 43 (54.4)
7 (5.2) 4 (57.1)
Economic evaluation (n = 134) Missing
Complete Incomplete
20 (14.9) 7 (35.0)
25 (18.7) 13 (52.0)
8 (6.0) 5 (62.5)
Not available Not applicable 28 (20.9) 14 (50.0)
24 (17.9) 12 (50.0)
Missing 49 (36.6) 28 (57.1)
WHO: World Health Organization. a If applicant explicitly noted that price and/or economic evaluation data did not exist for the medicine, these were classified as not available. To be classified as missing, the applicant did not provide any price or economic evaluation information. Applications classified as not applicable were those in which the economic evaluation sections did not include outcomes-based measures (i.e. neither cost–effectiveness, cost–benefit nor cost–utility analysis).
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
695
Research Corrina Moucheraud et al.
Essential medicines
Table 3. Essential medicine list applications where price or economic evaluation are mentioned in final reports from the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Type
Price Not mentioned Mentioned Total Economic evaluation Not mentioned Mentioned Total
No. of applications (%) Complete information
Incomplete information
No informationa
14 (50) 14 (50) 28 (100)
44 (56) 35 (44) 79 (100)
20 (74) 7 (26) 27 (100)
78 (58) 56 (42) 134 (100)
10 (40) 15 (60) 25 (100)
4 (50) 4 (50) 8 (100)
79 (78) 22 (22) 101 (100)
93 (69) 41 (31) 134 (100)
Total
WHO: World Health Organization. a Applications where no information was provided for price and economic evaluation, plus those where the applicant explicitly noted that price and economic evaluation data did not exist for the medicine and those earlier classified as not applicable.
incremental data). The fourth criterion, use of a WHO-recommended source for economic data, might be seen as an optional point: dropping this criterion from the analysis did not substantially increase the number of applications scored as complete (from 20.9% complete on price to 28.4% and from 18.7% complete for economic evaluation to 19.4%).
Discussion Very few applications complied fully with the instructions on providing price data and economic evaluations. The quality of information provided in applications to the Model List of Essential Medicines can be improved substantially. Despite the majority of applications being submitted without the required economic information, the committee reviewed all applications. There was no significant association between the completeness of economic information and the outcome of applications: provision of economic data in an application is not necessary for a positive decision by the committee. Our qualitative findings suggest that in some cases, the committee found alternative ways to address information gaps – such as internal literature reviews and analyses. However, such data collection and analysis requires additional time on the part of the committee and reviewers. Since the committee still considered and approved incomplete applications, this may have reinforced applicants’ decisions to not comply with instructions. There is little evidence that 696
applicants have improved their efforts to submit requested information over time. There are several possible reasons why applicants often did not provide complete or robust economic data. First, this information may be hard to collect: high-quality price and economic evaluation data that are relevant for resource-poor settings may be difficult to find in the literature; applicants may lack the technical skills to conduct their own economic evaluation. Such issues should, however, be less of a barrier for applicants from private companies, who have access to proprietary data, but applications from companies had some of the lowest rates of provision of economic information. Second, the application instructions may be unclear or insufficiently detailed, as suggested at a recent committee meeting. Application instructions for the recent twentieth committee meeting were revised,18 but still do not fully incorporate recent recommendations, such as asking applicants to provide comprehensive search strategies or stating that only complete applications would be reviewed. Third, the committee itself may not be clear about whether economic studies should be a necessary component of the decision process. Our qualitative analysis suggests that other considerations – such as safety and tolerability – are sometimes given higher priority even in cases of unfavourable price or economic evaluation data. However, it is often unclear which criteria were emphasized during the committee’s review and decision process. There are also chal-
lenges in using economic criteria, such as comparative cost–effectiveness, in decision-making for medicines. These challenges include information deficiencies and a lack of universal standards on appropriate thresholds. Our study has some limitations. We did not assess the accuracy of information provided and when an applicant noted that no data were available, we did not attempt to verify this. Our analysis of committee discussions and decisions was limited to the publicly-available meeting reports, which may not capture all aspects of the meeting. To overcome the various data limitations, we relaxed certain criteria in the application instructions to allow more flexibility in assessing adherence to the instructions: for example, we expanded the average generic world market price criterion to permit inclusion of median prices and prices per country-group (e.g. low-income countries); we allowed other published data sources (e.g. peer-reviewed manuscripts) or internal sources (e.g. clinical trial dossiers) for economic evaluation. In conclusion, we have three recommendations to improve the review and decision-making process for the Model List of Essential Medicines. WHO should: (i) provide clear and detailed instructions about how much and what kind of economic data are required; (ii) develop mechanisms to assist applicants in completing an application; and (iii) provide clear rules about how economic data will be used in making decisions about applications and about the consequences of not providing economic data (Box 1). To implement these recommendations, the current application form needs to be revised and examples of high-quality applications and an interactive, guided, application process should be provided. Many countries are debating how to use economic data and analysis in decision-making processes about essential medicines. WHO could assist these countries by providing clear examples of how applications should be prepared and how decisions should be reached. The model list is an important global tool and decisions about new additions have significant implications for national policies and budgets, as well as clinical decision-making. ■ Competing interests: VJW is affiliated with a World Health Organization Collaborating Centre in Pharmaceutical Policy.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al.
Box 1. Detailed recommendations for the WHO Expert Committee on the Selection and Use of Essential Medicines 1. The World Health Organization (WHO) should provide clear and detailed application instructions on how much and what kind of economic data (price data and economic evaluation information) should be presented, based on accepted standards.19 a) For price data, the current instructions ask applicants to provide a “range of costs of the proposed medicine and to show medicine prices from a range of settings where the product is registered.” This should be clarified: a range of unit prices (not costs) should be provided for specific countries and for specific sellers and data should include low- and/or middle-income countries whenever possible. The application should clearly identify the source of price data, the years for the data and conditions that apply to the prices (bulk purchasing, payment method, etc.). In addition to unit prices, the applicant should provide meaningful per-patient prices, for example, per treatment duration or per full vaccination course. If no price data exist, this should be explicitly noted as such and the efforts to find price data should be documented. b) For economic evaluations (cost–effectiveness, cost–benefit and cost–utility analyses), applications should provide clear units (for both costs and outcomes), all comparisons (ideally reported as incremental costs and outcomes) and citations and the relevant country or countries. The economic evaluation should include information on administration requirements (human resources, supplies) and costs for these if possible. If any data components are lacking in the literature, this should be explicitly noted. For published economic evaluations, applicants should provide details of their search strategy (sources and keywords used). All results of this search should be presented in full. For applications reporting new economic evaluation analyses, applicants should include a full accounting of all cost and outcome data components and sources, plus information on the modelling approach and sensitivity and uncertainty analyses. Applicants should provide a discussion and interpretation of the results presented. c) For both price data and economic evaluation information, the distinction between required and optional information should be explicitly stated in the application instructions. For price data, required items might include unit and clinical prices (e.g. per course of treatment); and for economic evaluation, clear outcomes and comparative, incremental analyses (e.g. incremental cost–effectiveness ratios). d) For price data reported directly in the application and for any price data used in de-novo economic evaluation calculations, WHO should seek to extend its partnerships with agencies that collect economic information – currently WHO recommends data sources such as the International Drug Price Indicator Guide and the United Nations Children’s Fund. This is a longer-term goal, but would provide a standardized, high-quality source for applications that present price data. 2. WHO should develop mechanisms to assist prospective applicants in completing an application. We recommend four new mechanisms, which may also require additional financing. a) The application instructions should be reviewed using focus groups and feedback from past applicants and reviewers, to improve the clarity of instructions around data types and formats (as suggested in recommendation 1a–d). b) Examples of high-quality, complete applications from the past should be provided to prospective applicants, to illustrate appropriate responses to specific sections of the application instructions. c) An interactive, guided, form-based application process, with detailed application sections and mandatory fields, could be developed to improve the process of applying and assure that all sections of an application are completed before submission. d) WHO should develop an online training course and require that this course be completed by applicants before submission of a new application. This would include information about the model list, its objectives and role, as well as technical guidance in completing a high-quality application, including identifying, interpreting and using economic data. The World Health Assembly passed a resolution to improve policies and practices for adopting national-level model lists, including promoting collaboration and information-sharing about best practices for selection procedures20; WHO could extend such technical assistance and education programmes to aid national decision processes as well. 3. WHO should provide clear rules about how economic data will be used in making decisions and about the consequences of not providing economic data. a) Incomplete applications should be identified as such and should not be reviewed. Applicants should be informed that their application is not complete and will not be reviewed unless all instructions are met, including providing specific price data and economic evaluation studies as identified via recommendation 1c. b) All criteria used to guide application assessments should be accounted for in documentation of the committee’s decisions: just as a structured application form/ process could help ensure uniform provision of information, a template of the process for committee decisions could increase transparency. For instance, the committee report should provide a brief snapshot summary of information provided in terms of the medicine’s safety, efficacy, quality and economic profile. A model for this already exists, in the expert review process before the committee meeting – and this type of form could be expanded and incorporated into the committee decision process. There is a short timeline after each meeting in which the report is generated (with decision justifications, etc.); standardized reporting might facilitate a quicker and more efficient process for writing the report, but rapid turnaround should be considered as an important factor in adopting such a change. c) WHO should provide clear decision rules about what information and criteria are used to assign an application for consideration under the core list or the complementary list, including the role of economic data in doing so. These determinations should also be clearly accounted for in documentation of the committee’s decisions. d) All documents used in the application process before the committee meeting should be publicly available on a WHO website, including initial application, all expert reviews, any application revisions, all public comments and all additional information sought from applicants or others. All information used in the process should be fully documented and publicly available, including details of the committee’s decision process, including assessment of information provided as per items 3a–b and any additional data or analyses prepared by the Secretariat, to assure full transparency about how decisions are made and to strengthen the evidence base. The committee should continue to require only publicly-available information in applications.
ملخص
تقييم مستوى جودة البيانات االقتصادية واستخدامها الختاذ القرارات املتعلقة باألدوية األساسية
الطريقة لقد قمنا بتحليل الطلبات املقدمة للجنة اخلرباء بمنظمة الصحة العاملية بشأن اختيار األدوية األساسية واستخدامها يف وتم استخالص درجة.2013 و2002 الفرتة ما بني عامي اكتامل البيانات املتعلقة بسعر األدوية ومعقولية تكلفتها من خالل
الغرض تقييم مستوى جودة البيانات االقتصادية املقدمة يف طلبات اإلدراج ضمن القائمة النموذجية لألدوية األساسية بمنظمة وتقييم الدور الذي تلعبه هذه البيانات،)WHO( الصحة العاملية .يف اختاذ القرارات من جانب جلنة اخلرباء التي تنظر يف الطلبات
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
697
Research Corrina Moucheraud et al.
Essential medicines
ومل. قامت اللجنة بمراجعة مجيع الطلبات،البيانات االقتصادية ،يوجد أي ارتباط واضح بني درجة اكتامل املعلومات االقتصادية بل حاولت جلنة اخلرباء سد فجوات البيانات يف،ونتائج الطلب الطلبات املقدمة عن طريق إجراء املزيد من املراجعة والتحليل .للبيانات املتعلقة بالطلب االستنتاج جيب عىل منظمة الصحة العاملية مراجعة التعليامت واملتعلقة بمتطلبات البيانات،اخلاصة بمقدمي الطلبات كام جيب عليها وضع آليات جديدة ملساعدة مقدمي،االقتصادية وكذلك حتديد بعض،الطلبات يف إكامل إجراءات تقديم الطلب .الطرق الستخدام البيانات االقتصادية يف اختاذ القرارات
، كام تم التعبري عن ذلك بالرتميز،املستندات التي تضمنتها الطلبات وسجلنا ما إذا كانت جلنة.مع االستعانة بمعيار من أربع نقاط اخلرباء قد قامت بمناقشة املعلومات االقتصادية والنتائج فيام يتعلق وتم تقييم حاالت االرتباط بني درجة اكتامل. أم ال،بكل طلب ، ونتائج الطلبات من جهة أخرى،البيانات االقتصادية من جهة .χ2 وذلك باستخدام اختبارات خي مربع وتضمنت ثامنية طلبات. طلب ًا134 النتائج تسلمت جلنة اخلرباء ) بيانات مكتملة عن السعر وبيانات عن التقييم%6( فقط وقد حذف الكثري من مقدمي الطلبات قسم التقييم.االقتصادي وبرغم نقص.االقتصادي بنموذج الطلب أو أساءوا تفسريه
摘要 评估经济数据在基本药物决策中的特性和作用 目的 评估向世界卫生组织 (WHO) 申请的基本药物标 准清单中所提供的经济数据的特性。同时也评估专家 委员会审议申请时，这些数据在他们制定决策中所起 的作用。 方 法 我 们 分 析 了 在 2002 年 至 2013 年 间 提 交 给 WHO 专家委员会的关于选择和使用基本药物的申请。 药物价格和成本效益的数据完整性从申请文件中提 取，并且用四点量表编码。 我们记录了专家委员会是 否讨论了经济信息以及每份申请的结果。 我们用 χ2 检验对经济数据的完整性和申请结果之间的关联性进 行了评估。
结果 专家委员会共收到 134 份申请。 只有 8 份申 请 (6%) 包含完整的价格数据和经济评估数据。 许多 申请都遗漏或曲解了申请表的经济评估部分。 尽管缺 少经济数据，但委员会还是审查了所有的申请。 经济 信息的完整性和申请结果之间并无重大关联。 专家委 员会尽力通过进一步审查和分析申请相关数据来处理 申请中的信息缺失。 结论 世界卫生组织应该修改供申请人查阅的经济 数据要求说明，制定有助于申请人完成申请过程的新 机制，规定决策中经济数据起作用的方法。
Résumé Évaluer la qualité et l’utilisation des données économiques dans les décisions sur les médicaments essentiels Objectif Évaluer la qualité des données économiques fournies dans les candidatures à la Liste Modèle des Médicaments Essentiels de l’Organisation Mondiale de la Santé (OMS) et évaluer le rôle de ces données dans la prise de décision du comité d’experts chargé d’examiner les candidatures. Méthodes Nous avons analysé les candidatures soumises au comité d’experts de l’OMS concernant la sélection et l’utilisation des médicaments essentiels entre 2002 et 2013. Toutes les données relatives aux prix et au rapport coût-efficacité des médicaments ont été tirées des dossiers de candidature et leur exhaustivité a été codée à l’aide d’une échelle à quatre points. Nous avons déterminé si le comité d’experts examinait les informations économiques et l’issue de chaque candidature. La relation entre le caractère exhaustif des données économiques et l’issue des candidatures a été évaluée par des tests du χ2. Résultats Le comité d’experts a reçu 134 candidatures. Seules huit d’entre elles (6%) incluaient des données complètes sur les prix et
des données d’évaluation économique. De nombreux candidats ont négligé ou mal compris la section du formulaire de candidature relative à l’évaluation économique. Malgré le manque de données économiques, toutes les candidatures ont été examinées par le comité. Aucune relation significative entre l’exhaustivité des informations économiques et l’issue des candidatures n’a été établie. Le comité d’experts a tenté de remédier à l’insuffisance des informations fournies dans les candidatures en procédant à un examen et à une analyse plus poussés des données relatives à la candidature. Conclusion L’Organisation Mondiale de la Santé doit revoir les indications données aux candidats en ce qui concerne les exigences relatives aux données économiques, élaborer de nouveaux mécanismes pour aider les candidats à aller au bout du processus de candidature et définir des méthodes d’utilisation des données économiques dans la prise de décision.
Резюме Оценка качества и применения экономических данных при принятии решений об основных лекарствах Цель Оценить качество экономических данных, предоставленных в заявках на примерный список основных лекарств, составленный Всемирной организацией здравоохранения (ВОЗ), и оценить роль этих данных при принятии комитетом экспертов решений по этим заявкам. Методы Мы проанализировали заявки, подаваемые в комитет экспертов ВОЗ по отбору и использованию основных лекарств за период между 2002 и 2013 гг. Полный объем данных о ценах 698
и соотношении эффективности и расходов для конкретных лекарств были получены по документации, прилагаемой к заявкам. Данный объем оценивается по четырехбалльной шкале. Мы также отметили наличие или отсутствие обсуждения экономической информации комитетом экспертов и то, какое решение было принято по каждой из заявок. Связь между полнотой экономических данных и результатом рассмотрения заявки была оценена по критерию χ2.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al. Результаты Комитет экспертов получил за это время 134 заявки. Из них только в восьми (6%) имелись полные данные о ценах и экономическая оценка. Во многих заявках раздел экономической оценки был не заполнен или заполнен неправильно. Несмотря на недостаток экономической информации, комитет рассмотрел каждую из заявок. Связи между полнотой экономической информации и результатом рассмотрения заявки выявлено не было. Комитет экспертов пытался заполнить пробелы в заявках
путем дальнейшего изучения и анализа данных, связанных с отдельной заявкой. Вывод Комитету экспертов следует пересмотреть инструкции, выдаваемые заявителям, в части требований, касающихся экономических данных, разработать новые механизмы помощи заявителям в заполнении заявок, а также определить методы использования экономических данных при принятии решений.
Resumen Evaluación de la calidad y el uso de datos económicos en las decisiones sobre medicamentos esenciales Objetivo Evaluar la calidad de los datos económicos proporcionados en las solicitudes para la Lista Modelo de Medicamentos Esenciales de la Organización Mundial de la Salud (OMS) y valorar el papel que desempeña dicha información en el proceso de toma de decisiones del Comité de Expertos que tiene en cuenta las solicitudes. Métodos Se analizaron las solicitudes entregadas al Comité de Expertos de la OMS en relación con la selección y el uso de medicamentos esenciales entre 2002 y 2013. La exactitud de los datos sobre el precio y la costoeficacia de los medicamentos se extrajo de los documentos de solicitud y se calificó mediante una escala de cuatro puntos. Se registró si en el Comité de Expertos se discutió o no la información económica y los resultados de cada solicitud. Las asociaciones entre la exactitud de los datos económicos y los resultados de las solicitudes se evaluaron usando pruebas χ2. Resultados El comité de expertos recibió 134 solicitudes. Solamente
ocho solicitudes (6%) incluían datos completos relativos al precio y datos relativos a la evaluación económica. Muchos solicitantes omitieron o malinterpretaron la sección de evaluación económica del formulario de solicitud. A pesar de la falta de datos económicos, el Comité revisó todas las solicitudes. No hubo ningún vínculo significativo entre la exactitud de la información económica y los resultados de la solicitud. El Comité de Expertos trató de remediar la falta de información en las solicitudes mediante una revisión más exhaustiva y un análisis de los datos relacionados con dicha solicitud. Conclusión La Organización Mundial de la Salud deberá revisar las instrucciones para los solicitantes que figuran en los requisitos de los datos económicos, desarrollar nuevas formas de asistir a los solicitantes a la hora de completar el proceso de solicitud y definir métodos para el uso de datos económicos en el proceso de toma de decisiones.
References 1. Essential medicines [Internet]. Geneva: World Health Organization; 2014. Available from: http://www.who.int/topics/essential_medicines/en/ [cited 2014 August 10]. 2. The world medicines situation 2011. Geneva: World Health Organization; 2011. 3. EB109/8. Provisional agenda item 3.6. WHO medicines strategy. In: 109th Executive Board Meeting, Geneva, 7 December 2001. Geneva: World Health Organization; 2001. Available from: http://www.who.int/ selection_medicines/committees/subcommittee/2/eeb1098%5B1%5D.pdf [cited 2015 Jul 13]. 4. Hogerzeil HV. The concept of essential medicines: lessons for rich countries. BMJ. 2004 Nov 13;329(7475):1169–72. doi: http://dx.doi.org/10.1136/ bmj.329.7475.1169 PMID: 15539676 5. World Health Organization. The selection and use of essential medicines. World Health Organ Tech Rep Ser. 2014;985(985):i–xiv, 1–219, back cover. PMID: 25080778 6. Schiff GD, Galanter WL, Duhig J, Koronkowski MJ, Lodolce AE, Pontikes P, et al. A prescription for improving drug formulary decision making. PLoS Med. 2012;9(5):1–7. doi: http://dx.doi.org/10.1371/journal.pmed.1001220 PMID: 22629233 7. Sinclair D, Gyansa-Lutterodt M, Asare B, Koduah A, Andrews E, Garner P. Integrating global and national knowledge to select medicines for children: the Ghana National Drugs Programme. PLoS Med. 2013;10(5):e1001449. doi: http://dx.doi.org/10.1371/journal.pmed.1001449 PMID: 23704834 8. Burchett HED, Mounier-Jack S, Griffiths UK, Mills AJ. National decisionmaking on adopting new vaccines: a systematic review. Health Policy Plan. 2012 May;27 Suppl 2:ii62–76. doi: http://dx.doi.org/10.1093/heapol/czr049 PMID: 21733989 9. Golan O, Hansen P, Kaplan G, Tal O. Health technology prioritization: which criteria for prioritizing new technologies and what are their relative weights? Health Policy. 2011 Oct;102(2-3):126–35. doi: http://dx.doi. org/10.1016/j.healthpol.2010.10.012 PMID: 21071107 10. Neumann PJ, Drummond MF, Jönsson B, Luce BR, Schwartz JS, Siebert U, et al.; International Working Group for HTA Advancement. Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations? Int J Technol Assess Health Care. 2010 Jan;26(1):71–8. doi: http://dx.doi.org/10.1017/S0266462309990833 PMID: 20059783
11. Stafinski T, Menon D, Philippon DJ, McCabe C. Health technology funding decision-making processes around the world: the same, yet different. Pharmacoeconomics. 2011 Jun;29(6):475–95. doi: http://dx.doi. org/10.2165/11586420-000000000-00000 PMID: 21568357 12. Barbui C, Purgato M. Decisions on WHO’s essential medicines need more scrutiny. BMJ. 2014;349 jul31 1:g4798. doi: http://dx.doi.org/10.1136/bmj. g4798 PMID: 25081250 13. Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness–the curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014 Aug 28;371(9):796–7. doi: http://dx.doi.org/10.1056/NEJMp1405158 PMID: 25162885 14. Newall AT, Jit M, Hutubessy R. Are current cost-effectiveness thresholds for low- and middle-income countries useful? Examples from the world of vaccines. Pharmacoeconomics. 2014 Jun;32(6):525–31. doi: http://dx.doi. org/10.1007/s40273-014-0162-x PMID: 24791735 15. Shillcutt SD, Walker DG, Goodman CA, Mills AJ. Cost effectiveness in low- and middle-income countries: a review of the debates surrounding decision rules. Pharmacoeconomics. 2009;27(11):903–17. doi: http://dx.doi. org/10.2165/10899580-000000000-00000 PMID: 19888791 16. Fox-Rushby JA, Hanson K. Calculating and presenting disability adjusted life years (DALYs) in cost-effectiveness analysis. Health Policy Plan. 2001 Sep;16(3):326–31. doi: http://dx.doi.org/10.1093/heapol/16.3.326 PMID: 11527874 17. Stolk P, Willemen MJ, Leufkens HG. Rare essentials: drugs for rare diseases as essential medicines. Bull World Health Organ. 2006 Sep;84(9):745–51. doi: http://dx.doi.org/10.2471/BLT.06.031518 PMID: 17128345 18. 20th Expert Committee on the Selection and Use of Essential Medicines [Internet]. Geneva: World Health Organization; 2015. Available from: http:// www.who.int/selection_medicines/committees/expert/20/en/ [cited 2015 July 1]. 19. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the Economic Evaluation of Health Care Programmes. Third edition. Oxford: Oxford University Press; 2005. 20. Resolution WHA67.22. Access to essential medicines. In: Sixty-seventh World Health Assembly, Geneva, 24 May 2014. Agenda item 15.4. Geneva: World Health Organization; 2014.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
699
Evaluating the quality and use of economic data in decisions about essential medicines Corrina Moucheraud,a Veronika J Wirtzb & Michael R Reicha Objective To evaluate the quality of economic data provided in applications to the World Health Organization (WHO) Model List of Essential Medicines and to evaluate the role of these data in decision-making by the expert committee that considers the applications. Methods We analysed applications submitted to the WHO Expert Committee on the Selection and Use of Essential Medicines between 2002 and 2013. The completeness of data on the price and cost–effectiveness of medicines was extracted from application documents and coded using a four-point scale. We recorded whether or not the expert committee discussed economic information and the outcomes of each application. Associations between the completeness of economic data and application outcomes were assessed using χ2 tests. Findings The expert committee received 134 applications. Only eight applications (6%) included complete price data and economic evaluation data. Many applicants omitted or misinterpreted the economic evaluation section of the application form. Despite the lack of economic data, all applications were reviewed by the committee. There was no significant association between the completeness of economic information and application outcomes. The expert committee tried to address information gaps in applications by further review and analysis of data related to the application. Conclusion The World Health Organization should revise the instructions to applicants on economic data requirements; develop new mechanisms to assist applicants in completing the application process; and define methods for the use of economic data in decision-making.
Introduction Essential medicines are those that satisfy the priority health care needs of the population.1 The World Health Organization (WHO) introduced the first Model List of Essential Medicines in 1977. Countries are encouraged to use the model list as a guide for their decisions on pharmaceutical selection and procurement. Between 1977 and 2007, over 130 countries introduced national lists of essential medicines, modifying the WHO model list for their national context.2 The model list is updated every two years, following an application and review process by an expert committee. Any individual or institution may submit an application. Before each committee meeting, applications are published on a website for public comment and experts (committee members and external advisors) review each application.3 These reviews and any public comments are also published on the website before the committee meeting. Since 2002, essential medicines have been selected via an evidence-based process, with due regard to public health relevance, evidence on efficacy and safety and comparative cost–effectiveness.1 Detailed information on the decision criteria can be found at http://www.who.int/selec�tion_medicines/en/. The model list is divided into a core list and a complementary list. The core list includes medicines that meet the criteria of efficacy, safety and comparative cost–effectiveness,1 but expensive patented medicines are not necessarily excluded.3,4 The complementary list includes medicines that may require specialized facilities, that are consistently more costly, or less cost–effective in a variety of settings.5 Additions to the model list can have a major impact on global and national decisions, with significant budgetary, ethical and health implications.6,7 Other studies have examined application processes and criteria for national decisions on
adopting new vaccines,8 national health technology assessment programmes9,10 and financing decisions for health technologies.11 A recent study found that applications to the model list for mental health medicines generally provided low-quality and incomplete evidence across several required dimensions.12 There is no global consensus on how to use economic data in decision-making for medicines and health technologies.13–17 Here, we evaluate the extent to which applicants and the expert committee adhere to the instructions and procedures on economic data and analysis for applications to the model list. We assess the application process rather than the substance of decisions (specifically, compliance with instructions for applicants and the quality of data on price and on cost–effectiveness). We also assess whether economic considerations are included in the final report by the expert committee. The overall goal of this study is to improve the quality, transparency and clarity of the process for reviewing applications to the model list.
Methods Data set We included applications to the model list for medicines intended for use in adults. We analysed final reports from the twelfth (2002) to nineteenth (2013) meetings of the WHO expert committee; the sixteenth meeting reviewed paediatric medicines only and was not included. We did not review applications for new formulations of existing medicines or applications for reinstatement.
Definition of variables We assessed the extent to which applicants complied with the instructions provided. The instructions ask applicants to provide “a range of costs of the proposed medicine” that
Harvard TH Chan School of Public Health, Boston, Department of Global Health and Population, 665 Huntington Avenue, Building 1, Boston, Massachusetts 02115, United States of America (USA). b Boston University School of Public Health, Department of Global Health, Boston, USA. Correspondence to Corrina Moucheraud (email: [email protected]). (Submitted: 10 November 2014 – Revised version received: 15 June 2015 – Accepted: 19 June 2015 – Published online: 14 August 2015 ) a
Bull World Health Organ 2015;93:693–699 | doi: http://dx.doi.org/10.2471/BLT.14.149914
693
Research Corrina Moucheraud et al.
Essential medicines
reflect “average generic world market prices”, with a clearly specified source of data. The WHO-recommended sources for these data include the International Drug Price Indicator Guide, the United Nations Children’s Fund, Médecins Sans Frontières or WHO itself. We allocated one point for each of the following: price given; offered range of price or average or median price; clear source; WHO-recommended source. The application was classified as complete if the application attained a score of four; applications with scores between one and three points (inclusive) were classified as incomplete. If no pricing data were provided, this was considered missing (zero points). If the applicant explicitly mentioned that no pricing data exist, this was classified as not available. Completeness of economic evaluation data was scored based on WHO instructions for comparative cost–effectiveness presented as range of cost per routine outcome, and a clear citation, preferably of WHO-recommended sources. We also accepted citations from the published scientific literature as one of the recommended data sources. Adherence was classified as complete if applications met all of the following criteria: economic evaluation value provided; comparator given; clear source; WHO-recommended source. As with price, applications scoring between one and three inclusive were classified as incomplete. Omitted economic evaluation sections were classified as missing; cases where the applicant explicitly noted the lack of available data on economic evaluation were classified as not available. There were instances in which the applicant provided administration costs for medicines, without an outcomesbased denominator, under the heading of cost–effectiveness. These applications were classified as having provided economic evaluation data that were not applicable. Based on the final reports from each meeting, the outcome was coded as a dichotomous variable: either rejected or deferred (coded 0) versus accepted to the core or complementary lists (coded 1). Applicant type and therapeutic class were categorized based on information provided in the application. We recorded the proportion of applications reviewed by the committee that were added to the model list, by therapeutic class. We also recorded whether
694
the discussion and decisions mentioned price or economic evaluation.
Data entry and analysis A data entry form was generated based on the variables listed above. There were two independent data reviewers, each of whom extracted data from the applications; discrepancies were resolved by consensus. Application data were entered using CSPro version 5 (United States Census Bureau, Washington, United States of America). Qualitative information from discussions and application decision categories were coded by hand. We extracted the narrative sections about the reviewed medicines from the final report for each committee meeting; text concerning economic considerations was collated in an Excel spreadsheet. We identified whether the report mentioned economic data or the absence of such data, as well as which data sources were cited (i.e. the application itself or new additional data). Associations between the completeness of economic data and the application outcomes were assessed using χ2 tests in Stata version 12.1 (StataCorp. LP, College Station, USA). We conducted sensitivity analyses that dropped each scoring criterion in turn.
Results We analysed 134 applications for new medicines from 2002 to 2013 (Table 1). Application authors varied by year, but were most commonly submitted by WHO departments (25.4%; 34/134), academic institutions (22.4%; 30/134) and nongovernmental organizations (21.6%; 29/134). Therapeutic classes of medicines also varied between meetings; applications for cancer and diabetes medicines were more common in recent years. Of 134 applications analysed, 52 were rejected and decisions were deferred for an additional 10. Among the 72 applications added to the model list, 67 were added to the core list and five to the complementary list. Across all meeting years, only 6.0% of applications (8/134) provided complete price and economic evaluation data (Table 2). Since no time trends were apparent, we report totals for all years; analyses by year are available from the corresponding author. If we include applications that reported economic data were not available, assuming that these
applicants made an effort to include data but that none were available, 13.4% (18/134) of applications fully followed the instructions on price and economic evaluation. Only 20.9% (28/134) and 18.7% (25/134) of applications provided complete price and economic evaluation information, respectively. Over a third (36.6%; 49/134) included no economic evaluation information, one-fifth (20.9%; 28/139) stated that economic evaluation information was unavailable and 17.9% (24/134) presented financial information that was unrelated to economic evaluation (e.g. total treatment cost). All applications were reviewed by the committee. For the applications with complete economic data, 52.0% (13/25) was added to the model list. When we included applications that stated that economic data were not available, 50.9% (27/53) was added to the model list. Among applications with no price data or no economic evaluation, over one-third and over half, respectively, of these applications were added to the model list. There was variation in the extent of economic information provided by applicant type. Academic authors were most likely to provide complete price and economic evaluation information. Applicants from WHO did not include economic evaluation information in half of their applications and industry applicants very rarely provided complete information on price or economic evaluation. Detailed data on the completeness of economic information by applicant type and therapeutic class are available from the corresponding author. For applications with complete price data, 64.3% (18/28) were added to the model list, a percentage that was not significantly higher than for applications with incomplete or absent price information (50.5%; 50/99; P = 0.2). For applications with complete economic evaluation information 52.0%, (13/25) were added to the model list, compared to 54.1% for those with incomplete, missing, inapplicable or unavailable information (59/109; P = 0.9). We explored whether the committee’s discussions of economic factors reflected the content of the applications. As shown in Table 3, price data or economic evaluations were discussed in some cases, even where this information had not been included in the application. Our qualitative analysis suggests four
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al.
Table 1. Applications to the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Application characteristic
No. of applications (%) Meeting of the WHO Expert committee (year)
Total
12th (2002) 13th (2003) 14th (2005) 15th (2007) 17th (2009) 18th (2011) Applications Additions to the list Applicant WHO (internal) Academia NGO Industry Othera Class of medicine HIV Infections Mental health Tuberculosis and malaria Cancer RH and MCH CVD and diabetes Otherb
19th (2013)
16 (100) 11 (68.8)
7 (100) 3 (42.9)
17 (100) 7 (41.2)
22 (100) 19 (86.4)
35 (100) 12 (34.3)
17 (100) 10 (58.8)
20 (100) 10 (50.0)
134 (100) 72 (53.7)
12 (75.0) 0 (0.0) 0 (0.0) 3 (18.8) 1 (6.2)
4 (57.1) 1 (14.3) 2 (28.6) 0 (0.0) 0 (0.0)
2 (11.8) 1 (5.9) 5 (29.4) 5 (29.4) 4 (23.5)
7 (31.8) 3 (13.6) 3 (13.6) 6 (27.3) 3 (13.6)
4 (11.4) 15 (42.9) 11 (31.4) 0 (0.0) 5 (14.3)
3 (17.6) 6 (35.3) 0 (0.0) 6 (35.3) 2 (11.8)
2 (10.0) 4 (20.0) 8 (40.0) 1 (5.0) 5 (25.0)
34 (25.4) 30 (22.4) 29 (21.6) 21 (15.7) 20 (14.9)
10 (62.5) 1 (6.3) 0 (0.0) 4 (25.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (6.3)
0 (0.0) 4 (57.1) 0 (0.0) 1 (14.3) 0 (0.0) 1 (14.3) 1 (14.3) 0 (0.0)
3 (17.6) 3 (17.6) 1 (5.9) 0 (0.0) 0 (0.0) 7 (41.2) 0 (0.0) 3 (17.6)
7 (31.8) 4 (18.2) 1 (4.5) 2 (9.1) 1 (4.5) 2 (9.1) 1 (4.5) 4 (18.2)
2 (5.7) 5 (14.3) 14 (40.0) 1 (2.9) 4 (11.4) 2 (5.7) 1 (2.9) 6 (17.1)
3 (17.6) 2 (11.8) 0 (0.0) 3 (17.6) 3 (17.6) 0 (0.0) 2 (11.8) 4 (23.5)
0 (0.0) 2 (10.0) 2 (10.0) 2 (10.0) 4 (20.0) 0 (0.0) 1 (5.0) 9 (45.0)
25 (18.7) 21 (15.7) 18 (13.4) 13 (9.7) 12 (9.0) 12 (9.0) 6 (4.5) 27 (20.2)
CVD: cardiovascular disease; HIV: human immunodeficiency virus; MCH: maternal and child health; NGO: nongovernmental organization; RH: reproductive health; WHO: World Health Organization. a Government groups and co-applicants of different types (e.g. NGO-academic partnerships). b Includes analgesics, antidotes, gastrointestinal medicines, antivirals, anaesthesia and sedatives, ophthalmology preparations and nutritional supplements including vitamins and minerals.
ways in which this happened. First, the committee sometimes noted the lack of information in the application (two applications with missing price data, seven applications with missing economic evaluation); in three of these cases, the medicines were nonetheless added to the core list. Second, the committee sometimes referenced economic information, suggesting an alternative, although unspecified, source of supplementary data (six applications). Third, the committee sometimes did further research: for example, the report mentioned a review of cost–effectiveness data prepared by the secretariat, or indicated that the secre-
tariat (usually comprised of WHO staff who support the committee process) performed its own economic evaluation (four applications). Fourth, in two instances where an economic evaluation was missing, the committee discussed price data included in the application but referred to it as “cost–effectiveness,” suggesting a misinterpretation. We explored whether the distinction between core and complementary lists was followed in the decision process. A feature that distinguishes the complementary list is its inclusion of medicines requiring specialized diagnostic or monitoring facilities and/or
specialist medical care and/or specialist training.5 According to our analysis, 50% (46/92) of applications that we classified as belonging to the complementary list were nonetheless added to the core list by the committee. This included all psychiatric medicines and antiretroviral medicines for treating human immunodeficiency virus infections. We conducted sensitivity analyses, to examine the relative importance of each scoring criterion. The most influential variables were: for price data, the average generic world price and for economic evaluations, the choice of comparator (i.e. presenting relative or
Table 2. Completeness of economic information provided in applications to the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Application status
No. of applications (%)a Price (n = 134) Complete
Applied Added to list
28 (20.9) 18 (64.3)
Incomplete Not available 79 (59.0) 43 (54.4)
7 (5.2) 4 (57.1)
Economic evaluation (n = 134) Missing
Complete Incomplete
20 (14.9) 7 (35.0)
25 (18.7) 13 (52.0)
8 (6.0) 5 (62.5)
Not available Not applicable 28 (20.9) 14 (50.0)
24 (17.9) 12 (50.0)
Missing 49 (36.6) 28 (57.1)
WHO: World Health Organization. a If applicant explicitly noted that price and/or economic evaluation data did not exist for the medicine, these were classified as not available. To be classified as missing, the applicant did not provide any price or economic evaluation information. Applications classified as not applicable were those in which the economic evaluation sections did not include outcomes-based measures (i.e. neither cost–effectiveness, cost–benefit nor cost–utility analysis).
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
695
Research Corrina Moucheraud et al.
Essential medicines
Table 3. Essential medicine list applications where price or economic evaluation are mentioned in final reports from the WHO Expert Committee on the Selection and Use of Essential Medicines, 2002–2013 Type
Price Not mentioned Mentioned Total Economic evaluation Not mentioned Mentioned Total
No. of applications (%) Complete information
Incomplete information
No informationa
14 (50) 14 (50) 28 (100)
44 (56) 35 (44) 79 (100)
20 (74) 7 (26) 27 (100)
78 (58) 56 (42) 134 (100)
10 (40) 15 (60) 25 (100)
4 (50) 4 (50) 8 (100)
79 (78) 22 (22) 101 (100)
93 (69) 41 (31) 134 (100)
Total
WHO: World Health Organization. a Applications where no information was provided for price and economic evaluation, plus those where the applicant explicitly noted that price and economic evaluation data did not exist for the medicine and those earlier classified as not applicable.
incremental data). The fourth criterion, use of a WHO-recommended source for economic data, might be seen as an optional point: dropping this criterion from the analysis did not substantially increase the number of applications scored as complete (from 20.9% complete on price to 28.4% and from 18.7% complete for economic evaluation to 19.4%).
Discussion Very few applications complied fully with the instructions on providing price data and economic evaluations. The quality of information provided in applications to the Model List of Essential Medicines can be improved substantially. Despite the majority of applications being submitted without the required economic information, the committee reviewed all applications. There was no significant association between the completeness of economic information and the outcome of applications: provision of economic data in an application is not necessary for a positive decision by the committee. Our qualitative findings suggest that in some cases, the committee found alternative ways to address information gaps – such as internal literature reviews and analyses. However, such data collection and analysis requires additional time on the part of the committee and reviewers. Since the committee still considered and approved incomplete applications, this may have reinforced applicants’ decisions to not comply with instructions. There is little evidence that 696
applicants have improved their efforts to submit requested information over time. There are several possible reasons why applicants often did not provide complete or robust economic data. First, this information may be hard to collect: high-quality price and economic evaluation data that are relevant for resource-poor settings may be difficult to find in the literature; applicants may lack the technical skills to conduct their own economic evaluation. Such issues should, however, be less of a barrier for applicants from private companies, who have access to proprietary data, but applications from companies had some of the lowest rates of provision of economic information. Second, the application instructions may be unclear or insufficiently detailed, as suggested at a recent committee meeting. Application instructions for the recent twentieth committee meeting were revised,18 but still do not fully incorporate recent recommendations, such as asking applicants to provide comprehensive search strategies or stating that only complete applications would be reviewed. Third, the committee itself may not be clear about whether economic studies should be a necessary component of the decision process. Our qualitative analysis suggests that other considerations – such as safety and tolerability – are sometimes given higher priority even in cases of unfavourable price or economic evaluation data. However, it is often unclear which criteria were emphasized during the committee’s review and decision process. There are also chal-
lenges in using economic criteria, such as comparative cost–effectiveness, in decision-making for medicines. These challenges include information deficiencies and a lack of universal standards on appropriate thresholds. Our study has some limitations. We did not assess the accuracy of information provided and when an applicant noted that no data were available, we did not attempt to verify this. Our analysis of committee discussions and decisions was limited to the publicly-available meeting reports, which may not capture all aspects of the meeting. To overcome the various data limitations, we relaxed certain criteria in the application instructions to allow more flexibility in assessing adherence to the instructions: for example, we expanded the average generic world market price criterion to permit inclusion of median prices and prices per country-group (e.g. low-income countries); we allowed other published data sources (e.g. peer-reviewed manuscripts) or internal sources (e.g. clinical trial dossiers) for economic evaluation. In conclusion, we have three recommendations to improve the review and decision-making process for the Model List of Essential Medicines. WHO should: (i) provide clear and detailed instructions about how much and what kind of economic data are required; (ii) develop mechanisms to assist applicants in completing an application; and (iii) provide clear rules about how economic data will be used in making decisions about applications and about the consequences of not providing economic data (Box 1). To implement these recommendations, the current application form needs to be revised and examples of high-quality applications and an interactive, guided, application process should be provided. Many countries are debating how to use economic data and analysis in decision-making processes about essential medicines. WHO could assist these countries by providing clear examples of how applications should be prepared and how decisions should be reached. The model list is an important global tool and decisions about new additions have significant implications for national policies and budgets, as well as clinical decision-making. ■ Competing interests: VJW is affiliated with a World Health Organization Collaborating Centre in Pharmaceutical Policy.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al.
Box 1. Detailed recommendations for the WHO Expert Committee on the Selection and Use of Essential Medicines 1. The World Health Organization (WHO) should provide clear and detailed application instructions on how much and what kind of economic data (price data and economic evaluation information) should be presented, based on accepted standards.19 a) For price data, the current instructions ask applicants to provide a “range of costs of the proposed medicine and to show medicine prices from a range of settings where the product is registered.” This should be clarified: a range of unit prices (not costs) should be provided for specific countries and for specific sellers and data should include low- and/or middle-income countries whenever possible. The application should clearly identify the source of price data, the years for the data and conditions that apply to the prices (bulk purchasing, payment method, etc.). In addition to unit prices, the applicant should provide meaningful per-patient prices, for example, per treatment duration or per full vaccination course. If no price data exist, this should be explicitly noted as such and the efforts to find price data should be documented. b) For economic evaluations (cost–effectiveness, cost–benefit and cost–utility analyses), applications should provide clear units (for both costs and outcomes), all comparisons (ideally reported as incremental costs and outcomes) and citations and the relevant country or countries. The economic evaluation should include information on administration requirements (human resources, supplies) and costs for these if possible. If any data components are lacking in the literature, this should be explicitly noted. For published economic evaluations, applicants should provide details of their search strategy (sources and keywords used). All results of this search should be presented in full. For applications reporting new economic evaluation analyses, applicants should include a full accounting of all cost and outcome data components and sources, plus information on the modelling approach and sensitivity and uncertainty analyses. Applicants should provide a discussion and interpretation of the results presented. c) For both price data and economic evaluation information, the distinction between required and optional information should be explicitly stated in the application instructions. For price data, required items might include unit and clinical prices (e.g. per course of treatment); and for economic evaluation, clear outcomes and comparative, incremental analyses (e.g. incremental cost–effectiveness ratios). d) For price data reported directly in the application and for any price data used in de-novo economic evaluation calculations, WHO should seek to extend its partnerships with agencies that collect economic information – currently WHO recommends data sources such as the International Drug Price Indicator Guide and the United Nations Children’s Fund. This is a longer-term goal, but would provide a standardized, high-quality source for applications that present price data. 2. WHO should develop mechanisms to assist prospective applicants in completing an application. We recommend four new mechanisms, which may also require additional financing. a) The application instructions should be reviewed using focus groups and feedback from past applicants and reviewers, to improve the clarity of instructions around data types and formats (as suggested in recommendation 1a–d). b) Examples of high-quality, complete applications from the past should be provided to prospective applicants, to illustrate appropriate responses to specific sections of the application instructions. c) An interactive, guided, form-based application process, with detailed application sections and mandatory fields, could be developed to improve the process of applying and assure that all sections of an application are completed before submission. d) WHO should develop an online training course and require that this course be completed by applicants before submission of a new application. This would include information about the model list, its objectives and role, as well as technical guidance in completing a high-quality application, including identifying, interpreting and using economic data. The World Health Assembly passed a resolution to improve policies and practices for adopting national-level model lists, including promoting collaboration and information-sharing about best practices for selection procedures20; WHO could extend such technical assistance and education programmes to aid national decision processes as well. 3. WHO should provide clear rules about how economic data will be used in making decisions and about the consequences of not providing economic data. a) Incomplete applications should be identified as such and should not be reviewed. Applicants should be informed that their application is not complete and will not be reviewed unless all instructions are met, including providing specific price data and economic evaluation studies as identified via recommendation 1c. b) All criteria used to guide application assessments should be accounted for in documentation of the committee’s decisions: just as a structured application form/ process could help ensure uniform provision of information, a template of the process for committee decisions could increase transparency. For instance, the committee report should provide a brief snapshot summary of information provided in terms of the medicine’s safety, efficacy, quality and economic profile. A model for this already exists, in the expert review process before the committee meeting – and this type of form could be expanded and incorporated into the committee decision process. There is a short timeline after each meeting in which the report is generated (with decision justifications, etc.); standardized reporting might facilitate a quicker and more efficient process for writing the report, but rapid turnaround should be considered as an important factor in adopting such a change. c) WHO should provide clear decision rules about what information and criteria are used to assign an application for consideration under the core list or the complementary list, including the role of economic data in doing so. These determinations should also be clearly accounted for in documentation of the committee’s decisions. d) All documents used in the application process before the committee meeting should be publicly available on a WHO website, including initial application, all expert reviews, any application revisions, all public comments and all additional information sought from applicants or others. All information used in the process should be fully documented and publicly available, including details of the committee’s decision process, including assessment of information provided as per items 3a–b and any additional data or analyses prepared by the Secretariat, to assure full transparency about how decisions are made and to strengthen the evidence base. The committee should continue to require only publicly-available information in applications.
ملخص
تقييم مستوى جودة البيانات االقتصادية واستخدامها الختاذ القرارات املتعلقة باألدوية األساسية
الطريقة لقد قمنا بتحليل الطلبات املقدمة للجنة اخلرباء بمنظمة الصحة العاملية بشأن اختيار األدوية األساسية واستخدامها يف وتم استخالص درجة.2013 و2002 الفرتة ما بني عامي اكتامل البيانات املتعلقة بسعر األدوية ومعقولية تكلفتها من خالل
الغرض تقييم مستوى جودة البيانات االقتصادية املقدمة يف طلبات اإلدراج ضمن القائمة النموذجية لألدوية األساسية بمنظمة وتقييم الدور الذي تلعبه هذه البيانات،)WHO( الصحة العاملية .يف اختاذ القرارات من جانب جلنة اخلرباء التي تنظر يف الطلبات
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
697
Research Corrina Moucheraud et al.
Essential medicines
ومل. قامت اللجنة بمراجعة مجيع الطلبات،البيانات االقتصادية ،يوجد أي ارتباط واضح بني درجة اكتامل املعلومات االقتصادية بل حاولت جلنة اخلرباء سد فجوات البيانات يف،ونتائج الطلب الطلبات املقدمة عن طريق إجراء املزيد من املراجعة والتحليل .للبيانات املتعلقة بالطلب االستنتاج جيب عىل منظمة الصحة العاملية مراجعة التعليامت واملتعلقة بمتطلبات البيانات،اخلاصة بمقدمي الطلبات كام جيب عليها وضع آليات جديدة ملساعدة مقدمي،االقتصادية وكذلك حتديد بعض،الطلبات يف إكامل إجراءات تقديم الطلب .الطرق الستخدام البيانات االقتصادية يف اختاذ القرارات
، كام تم التعبري عن ذلك بالرتميز،املستندات التي تضمنتها الطلبات وسجلنا ما إذا كانت جلنة.مع االستعانة بمعيار من أربع نقاط اخلرباء قد قامت بمناقشة املعلومات االقتصادية والنتائج فيام يتعلق وتم تقييم حاالت االرتباط بني درجة اكتامل. أم ال،بكل طلب ، ونتائج الطلبات من جهة أخرى،البيانات االقتصادية من جهة .χ2 وذلك باستخدام اختبارات خي مربع وتضمنت ثامنية طلبات. طلب ًا134 النتائج تسلمت جلنة اخلرباء ) بيانات مكتملة عن السعر وبيانات عن التقييم%6( فقط وقد حذف الكثري من مقدمي الطلبات قسم التقييم.االقتصادي وبرغم نقص.االقتصادي بنموذج الطلب أو أساءوا تفسريه
摘要 评估经济数据在基本药物决策中的特性和作用 目的 评估向世界卫生组织 (WHO) 申请的基本药物标 准清单中所提供的经济数据的特性。同时也评估专家 委员会审议申请时，这些数据在他们制定决策中所起 的作用。 方 法 我 们 分 析 了 在 2002 年 至 2013 年 间 提 交 给 WHO 专家委员会的关于选择和使用基本药物的申请。 药物价格和成本效益的数据完整性从申请文件中提 取，并且用四点量表编码。 我们记录了专家委员会是 否讨论了经济信息以及每份申请的结果。 我们用 χ2 检验对经济数据的完整性和申请结果之间的关联性进 行了评估。
结果 专家委员会共收到 134 份申请。 只有 8 份申 请 (6%) 包含完整的价格数据和经济评估数据。 许多 申请都遗漏或曲解了申请表的经济评估部分。 尽管缺 少经济数据，但委员会还是审查了所有的申请。 经济 信息的完整性和申请结果之间并无重大关联。 专家委 员会尽力通过进一步审查和分析申请相关数据来处理 申请中的信息缺失。 结论 世界卫生组织应该修改供申请人查阅的经济 数据要求说明，制定有助于申请人完成申请过程的新 机制，规定决策中经济数据起作用的方法。
Résumé Évaluer la qualité et l’utilisation des données économiques dans les décisions sur les médicaments essentiels Objectif Évaluer la qualité des données économiques fournies dans les candidatures à la Liste Modèle des Médicaments Essentiels de l’Organisation Mondiale de la Santé (OMS) et évaluer le rôle de ces données dans la prise de décision du comité d’experts chargé d’examiner les candidatures. Méthodes Nous avons analysé les candidatures soumises au comité d’experts de l’OMS concernant la sélection et l’utilisation des médicaments essentiels entre 2002 et 2013. Toutes les données relatives aux prix et au rapport coût-efficacité des médicaments ont été tirées des dossiers de candidature et leur exhaustivité a été codée à l’aide d’une échelle à quatre points. Nous avons déterminé si le comité d’experts examinait les informations économiques et l’issue de chaque candidature. La relation entre le caractère exhaustif des données économiques et l’issue des candidatures a été évaluée par des tests du χ2. Résultats Le comité d’experts a reçu 134 candidatures. Seules huit d’entre elles (6%) incluaient des données complètes sur les prix et
des données d’évaluation économique. De nombreux candidats ont négligé ou mal compris la section du formulaire de candidature relative à l’évaluation économique. Malgré le manque de données économiques, toutes les candidatures ont été examinées par le comité. Aucune relation significative entre l’exhaustivité des informations économiques et l’issue des candidatures n’a été établie. Le comité d’experts a tenté de remédier à l’insuffisance des informations fournies dans les candidatures en procédant à un examen et à une analyse plus poussés des données relatives à la candidature. Conclusion L’Organisation Mondiale de la Santé doit revoir les indications données aux candidats en ce qui concerne les exigences relatives aux données économiques, élaborer de nouveaux mécanismes pour aider les candidats à aller au bout du processus de candidature et définir des méthodes d’utilisation des données économiques dans la prise de décision.
Резюме Оценка качества и применения экономических данных при принятии решений об основных лекарствах Цель Оценить качество экономических данных, предоставленных в заявках на примерный список основных лекарств, составленный Всемирной организацией здравоохранения (ВОЗ), и оценить роль этих данных при принятии комитетом экспертов решений по этим заявкам. Методы Мы проанализировали заявки, подаваемые в комитет экспертов ВОЗ по отбору и использованию основных лекарств за период между 2002 и 2013 гг. Полный объем данных о ценах 698
и соотношении эффективности и расходов для конкретных лекарств были получены по документации, прилагаемой к заявкам. Данный объем оценивается по четырехбалльной шкале. Мы также отметили наличие или отсутствие обсуждения экономической информации комитетом экспертов и то, какое решение было принято по каждой из заявок. Связь между полнотой экономических данных и результатом рассмотрения заявки была оценена по критерию χ2.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
Research Essential medicines
Corrina Moucheraud et al. Результаты Комитет экспертов получил за это время 134 заявки. Из них только в восьми (6%) имелись полные данные о ценах и экономическая оценка. Во многих заявках раздел экономической оценки был не заполнен или заполнен неправильно. Несмотря на недостаток экономической информации, комитет рассмотрел каждую из заявок. Связи между полнотой экономической информации и результатом рассмотрения заявки выявлено не было. Комитет экспертов пытался заполнить пробелы в заявках
путем дальнейшего изучения и анализа данных, связанных с отдельной заявкой. Вывод Комитету экспертов следует пересмотреть инструкции, выдаваемые заявителям, в части требований, касающихся экономических данных, разработать новые механизмы помощи заявителям в заполнении заявок, а также определить методы использования экономических данных при принятии решений.
Resumen Evaluación de la calidad y el uso de datos económicos en las decisiones sobre medicamentos esenciales Objetivo Evaluar la calidad de los datos económicos proporcionados en las solicitudes para la Lista Modelo de Medicamentos Esenciales de la Organización Mundial de la Salud (OMS) y valorar el papel que desempeña dicha información en el proceso de toma de decisiones del Comité de Expertos que tiene en cuenta las solicitudes. Métodos Se analizaron las solicitudes entregadas al Comité de Expertos de la OMS en relación con la selección y el uso de medicamentos esenciales entre 2002 y 2013. La exactitud de los datos sobre el precio y la costoeficacia de los medicamentos se extrajo de los documentos de solicitud y se calificó mediante una escala de cuatro puntos. Se registró si en el Comité de Expertos se discutió o no la información económica y los resultados de cada solicitud. Las asociaciones entre la exactitud de los datos económicos y los resultados de las solicitudes se evaluaron usando pruebas χ2. Resultados El comité de expertos recibió 134 solicitudes. Solamente
ocho solicitudes (6%) incluían datos completos relativos al precio y datos relativos a la evaluación económica. Muchos solicitantes omitieron o malinterpretaron la sección de evaluación económica del formulario de solicitud. A pesar de la falta de datos económicos, el Comité revisó todas las solicitudes. No hubo ningún vínculo significativo entre la exactitud de la información económica y los resultados de la solicitud. El Comité de Expertos trató de remediar la falta de información en las solicitudes mediante una revisión más exhaustiva y un análisis de los datos relacionados con dicha solicitud. Conclusión La Organización Mundial de la Salud deberá revisar las instrucciones para los solicitantes que figuran en los requisitos de los datos económicos, desarrollar nuevas formas de asistir a los solicitantes a la hora de completar el proceso de solicitud y definir métodos para el uso de datos económicos en el proceso de toma de decisiones.
References 1. Essential medicines [Internet]. Geneva: World Health Organization; 2014. Available from: http://www.who.int/topics/essential_medicines/en/ [cited 2014 August 10]. 2. The world medicines situation 2011. Geneva: World Health Organization; 2011. 3. EB109/8. Provisional agenda item 3.6. WHO medicines strategy. In: 109th Executive Board Meeting, Geneva, 7 December 2001. Geneva: World Health Organization; 2001. Available from: http://www.who.int/ selection_medicines/committees/subcommittee/2/eeb1098%5B1%5D.pdf [cited 2015 Jul 13]. 4. Hogerzeil HV. The concept of essential medicines: lessons for rich countries. BMJ. 2004 Nov 13;329(7475):1169–72. doi: http://dx.doi.org/10.1136/ bmj.329.7475.1169 PMID: 15539676 5. World Health Organization. The selection and use of essential medicines. World Health Organ Tech Rep Ser. 2014;985(985):i–xiv, 1–219, back cover. PMID: 25080778 6. Schiff GD, Galanter WL, Duhig J, Koronkowski MJ, Lodolce AE, Pontikes P, et al. A prescription for improving drug formulary decision making. PLoS Med. 2012;9(5):1–7. doi: http://dx.doi.org/10.1371/journal.pmed.1001220 PMID: 22629233 7. Sinclair D, Gyansa-Lutterodt M, Asare B, Koduah A, Andrews E, Garner P. Integrating global and national knowledge to select medicines for children: the Ghana National Drugs Programme. PLoS Med. 2013;10(5):e1001449. doi: http://dx.doi.org/10.1371/journal.pmed.1001449 PMID: 23704834 8. Burchett HED, Mounier-Jack S, Griffiths UK, Mills AJ. National decisionmaking on adopting new vaccines: a systematic review. Health Policy Plan. 2012 May;27 Suppl 2:ii62–76. doi: http://dx.doi.org/10.1093/heapol/czr049 PMID: 21733989 9. Golan O, Hansen P, Kaplan G, Tal O. Health technology prioritization: which criteria for prioritizing new technologies and what are their relative weights? Health Policy. 2011 Oct;102(2-3):126–35. doi: http://dx.doi. org/10.1016/j.healthpol.2010.10.012 PMID: 21071107 10. Neumann PJ, Drummond MF, Jönsson B, Luce BR, Schwartz JS, Siebert U, et al.; International Working Group for HTA Advancement. Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations? Int J Technol Assess Health Care. 2010 Jan;26(1):71–8. doi: http://dx.doi.org/10.1017/S0266462309990833 PMID: 20059783
11. Stafinski T, Menon D, Philippon DJ, McCabe C. Health technology funding decision-making processes around the world: the same, yet different. Pharmacoeconomics. 2011 Jun;29(6):475–95. doi: http://dx.doi. org/10.2165/11586420-000000000-00000 PMID: 21568357 12. Barbui C, Purgato M. Decisions on WHO’s essential medicines need more scrutiny. BMJ. 2014;349 jul31 1:g4798. doi: http://dx.doi.org/10.1136/bmj. g4798 PMID: 25081250 13. Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness–the curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014 Aug 28;371(9):796–7. doi: http://dx.doi.org/10.1056/NEJMp1405158 PMID: 25162885 14. Newall AT, Jit M, Hutubessy R. Are current cost-effectiveness thresholds for low- and middle-income countries useful? Examples from the world of vaccines. Pharmacoeconomics. 2014 Jun;32(6):525–31. doi: http://dx.doi. org/10.1007/s40273-014-0162-x PMID: 24791735 15. Shillcutt SD, Walker DG, Goodman CA, Mills AJ. Cost effectiveness in low- and middle-income countries: a review of the debates surrounding decision rules. Pharmacoeconomics. 2009;27(11):903–17. doi: http://dx.doi. org/10.2165/10899580-000000000-00000 PMID: 19888791 16. Fox-Rushby JA, Hanson K. Calculating and presenting disability adjusted life years (DALYs) in cost-effectiveness analysis. Health Policy Plan. 2001 Sep;16(3):326–31. doi: http://dx.doi.org/10.1093/heapol/16.3.326 PMID: 11527874 17. Stolk P, Willemen MJ, Leufkens HG. Rare essentials: drugs for rare diseases as essential medicines. Bull World Health Organ. 2006 Sep;84(9):745–51. doi: http://dx.doi.org/10.2471/BLT.06.031518 PMID: 17128345 18. 20th Expert Committee on the Selection and Use of Essential Medicines [Internet]. Geneva: World Health Organization; 2015. Available from: http:// www.who.int/selection_medicines/committees/expert/20/en/ [cited 2015 July 1]. 19. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the Economic Evaluation of Health Care Programmes. Third edition. Oxford: Oxford University Press; 2005. 20. Resolution WHA67.22. Access to essential medicines. In: Sixty-seventh World Health Assembly, Geneva, 24 May 2014. Agenda item 15.4. Geneva: World Health Organization; 2014.
Bull World Health Organ 2015;93:693–699| doi: http://dx.doi.org/10.2471/BLT.14.149914
699
