RESEARCH ARTICLE Autism Research Funding Allocation: Can Economics Tell Us If We Have Got It Right? Jennifer D. Zwicker and J.C. Herbert Emery There is a concern that the allocation of autism spectrum disorder (ASD) research funding may be misallocating resources, overemphasizing basic science at the expense of translational and clinical research. Anthony Bailey has proposed that an economic evaluation of autism research funding allocations could be beneficial for funding agencies by identifying under- or overfunded areas of research. In response to Bailey, we illustrate why economics cannot provide an objective, technical solution for identifying the “best” allocation of research resources. Economic evaluation has its greatest power as a late-stage research tool for interventions with identified objectives, outcomes, and data. This is not the case for evaluating whether research areas are over- or underfunded. Without an understanding of how research funding influences the likelihood and value of a discovery, or without a statement of the societal objectives for ASD research and level of risk aversion, economic analysis cannot provide a useful normative evaluation of ASD research. Autism Res 2014, 7: 704–711. © 2014 International Society for Autism Research, Wiley Periodicals, Inc. Keywords: autism; research funding; economic evaluation; funding allocation
Introduction Despite sizeable investments in autism research, it is not possible to address all key questions associated with improving the lives of people with autism or preventing persons having autism [Bailey, 2009]. This results in the need to prioritize the use of available research funding for autism spectrum disorder (ASD). With rising reported prevalence of ASD and a high estimated economic burden of the condition, there can be a dramatic difference between the outcomes that the public wants and what science is equipped to provide [Collier, 2010]. As the ultimate success or dividends of autism research effort are probabilistic and likely many years away, the choice of how to allocate finite research budgets is far from obvious. As Bailey [2009] suggests, we really have no idea if research resources deployed across basic and other research areas (or across projects within research areas) are being allocated in the “best” way, even if it is by the categorical metric of being overfunded or underfunded. Bailey then argues that even a basic economic evaluation might be useful for providing this information. In this paper, we show that this bigger question is fundamentally dependent on societal values, which must be specified for an economic model to be useful for normative evaluation of outcomes. This means that there is not an objective,
technical criterion for big questions like the normative evaluation of research funding allocations. In this case, economic modeling cannot substitute for the judgment of experts and broader stakeholders, and is limited in the information that can be provided beyond what is available to decision makers already. The technical approach to funding priority setting is typically used by national funding bodies, such as the Canadian Institute of Health Research (Canada), Medical Research Council (UK), and the National Institutes of Health (US), with the allocation being driven largely by experts and scientific panels of researchers. When trying to choose which research areas, programs, and projects to fund, decision makers consider many different perspectives, including laboratory science and existing scientific opportunities, the future disease burden on society, the current public health needs, and the quality, experience, and sustainability of the proposed research protocols [Coalition to Protect Research, 2007]. This perspective highlights that allocation of funds is influenced by the dissemination of research findings, expectations of a “breakthrough,” new technological or theoretical frameworks, the changing “fashion” of some research areas, and path dependence of research funding reflecting interests in maintaining established research groups and networks [Bailey, 2009]. Perhaps not surprisingly, this
From the School of Public Policy, University of Calgary, Calgary, Alberta, Canada (J.D.Z.); Department of Economics, University of Calgary, Calgary, Alberta, Canada (J.C.H.E.) Received February 19, 2014; accepted for publication September 5, 2014 Address for correspondence and reprints: Jennifer D. Zwicker, School of Public Policy, University of Calgary, Downtown Campus, 906 8th Ave S.W., 5th floor, Calgary, Alberta T2P 1H9, Canada. E-mail: [email protected] All grant information in the following format: Grant sponsor Alberta Innovates Health Solutions; Grant number: 10003547. Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/aur.1423 © 2014 International Society for Autism Research, Wiley Periodicals, Inc.
704
Autism Research 7: 704–711, 2014
INSAR
approach to allocating funding has resulted in the majority of ASD research funds being allocated to the area of basic research. Recently, in the US, increased consultation with stakeholders has led to an allocation of research dollars that is interpreted as reflecting a preference for addressing societal needs, shifting resources from basic research to translational and clinical research [Lenaway et al., 2006]. The rise in ASD public and charitable research funding in the US has been attributed to the political momentum and awareness raised by ASD advocacy groups [Silverman & Brosco, 2007; Singh, Illes, Lazzeroni, & Hallmayer, 2009]. Work by Pellicano, Dinsmore, and Charman [2014] highlights the increasing pressure to represent stakeholder interests in the resource allocation process. However, the concern is that in some situations, the lay public interests impact allocation of scarce ASD research funding, to the point where research with greater probability of success or impact in the findings may receive less funding than the research deemed important by the public (such as the coverage in the media on ASD and vaccines) [Godlee, Smith, & Marcovitch, 2011; Stokstad, 2007]. Given the divergence in views over what the allocation of ASD research resources across research areas should be, Bailey [2009] asks whether there may be a technical resolution to the problem from the application of an economic model to provide decision makers allocating research funding to have access to “quantitative estimates of the value of a specific advance” [Bailey, 2009]. What Bailey is arguing is that the opportunity cost of allocating resources between the alternatives of clinical research vs. basic science research must be considered. Bailey points out that by analyzing the probability of research outcomes and the value of the output, we could determine areas of research that are relatively overfunded or underfunded, or at least identify key questions that remain to be answered. At a very minimum, this discussion would provide incentive for debate around some of these value issues [Bailey, 2009]. In taking up Bailey’s challenge that better informed policy decisions surrounding research funding allocations could be made using an economic approach, we propose a simple model of how a decision maker could “optimally” allocate research funds across two broad areas of ASD research: basic science (B), and translational and clinical research (TC). The decision problem addressed by economic evaluation considers the choice between different types of ASD research given a budget constraint, which is a question of allocative efficiency— are resources applied to their highest value use? What we wish to show is that for big questions like choosing the socially preferred, or efficient, research funding allocation, there is no objective or technical answer that an economic model can provide. Economic models are built on expressed statements of objectives for the resources
INSAR
allocated and the values that the society places on the outcomes. Economists do not specify the societal values but the values wholly determine the models. In the absence of this information, an economic model can only describe differences in funding allocations and possible trade-offs that can be made given an assumed objective, rather than answer questions of optimal allocation. This approach brings into question who should be determining the allocation of ASD research funding and how the benefits and returns to research investment should be determined. Specifically, we discuss some of the practical challenges associated with this rational decision model, such as specifying the preferences of the decision makers for benefits today vs. benefits tomorrow (time preference and discount rates), for more certain research outcomes (risk aversion), and for helping persons with ASD today over preventing future cases of ASD (weighing to account for the relative importance of different members of the population, or to define the population of interest).1 Without first determining what societal preferences are in terms of these concerns, economic modeling cannot substitute for the judgment of experts and broader stakeholders, and is limited in the information that can be provided beyond what is available to decision makers already.
Illustrating the Utility and Futility of Modeling “Optimal Research Funding” for ASD To illustrate the importance of establishing values, we present a simple decision analytic, Bayesian approach to the prioritization of research that accounts for the existing uncertainty in the values of all parameters affecting the cost-effectiveness of the alternative interventions [Ades et al., 2006; Best, 2005]. The underlying premise for funding research consists of the decision maker essentially making a bet on a researcher and his/her proposal by funding the research in the expectation that a breakthrough or advanced knowledge will occur. For example, the effect of a breakthrough in an area, such as genetic determinants of ASD, would mean that a Bayesian decision maker would go from the low likelihood of a cure to believing the probability is higher, resulting in a reevaluation of the probability of payoff that can occur. This may result in the reallocation of research funding in relation to the adjusted probabilities. Similarly, evidence disproving a potential cause (such as discredited studies suggesting vaccinations may cause ASD) will lower the
1 This issue was examined in the context of do we overuse animals in experiments and overconsume meat because we do not include the preferences of the animals in our decisions. See [Blackorby & Donaldson, 1992].
Zwicker and Emery/The economics of funding autism research
705
probability of a payoff from research in causes of ASD [Godlee et al., 2011]. A decision maker can allocate resources in a way that contains costs, manages demand, and ultimately maximizes the expected benefit from research. Suppose that the research funder has R total dollars to allocate between projects in two areas, basic science (B) and translational and clinical research (TC): RB + RTC = R. If there is a breakthrough, then the decision maker gets a payoff that is conditional on which outcomes occur and the value of the outcomes. The social benefit of a breakthrough in basic science and in translational and clinical research can be represented as BB and BTC, respectively. Naively, Bj (where j = B, TC) could represent the aggregate burden of ASD avoided due to the discovery where the burden value combines the costs of ASD to the society for the population of affected individuals alive today and affected individuals to be born in the future. This notation is obviously a gross abstraction of reality for ASD, which is characterized as a range of conditions classified as pervasive developmental disorders that can differ greatly in severity and prevalence [Baron-Cohen, 1997]. Further, RB and RTC investments potentially benefit different populations. One of the most contentious arguments is that research funding allocation toward the cause and prevention of ASD is to the benefit of future generations, whereas the development of treatment addresses the needs of the current population. The value of research breakthroughs is not as simple as calculating Bj. Decision makers may have preferences that result in different valuations of Bj that depend on the riskiness of Bj occurring (risk aversion would mean lower preference on “long shots” vs. “sure things”), over the timing of when Bj will be realized (such as a decision maker with a time preference that values the present more than the future),2 and who in the population is actually getting the benefit Bj (weighing the relative importance of members of the population). In other words, decision makers have preferences that would be represented with a “utility function,” uj(Bj). When we specify in our model that the preferences of the decision maker are represented by uj(Bj) = Bj, we are making strong statements about preferences—the decision maker is riskneutral and has no differential preference for breakthroughs in the treatment of persons with ASD alive now
2 The rate of return of research will critically depend on assumptions regarding time lag from discovery into practice, which is highly variable and often much longer for basic research [Morris, Wooding, & Grant, 2011; Ward, House, & Hamer, 2009]. For example, some studies have used a time lag of research spending of 17 years for the translation of basic research [Health Economics Research Group, 2008]. With the extent to which decision makers are “impatient” and prefer benefits now vs. benefits later, there will be a preference for avoiding investing in long-term research.
706
vs. breakthroughs that may prevent cases of ASD in the future.3 We have no way of knowing if this is the correct utility function. The next challenge is assigning values to the probability of a research breakthrough and specifying the relationship between research funding and the probability of a breakthrough. The success of a research discovery is probabilistic in the sense that we do not know with certainty if or when a research effort, as measured by dollars of funding (R), will yield a result. When deciding which research and researchers to fund, a Bayesian decision maker would start with a naive rule for allocating research funding of ASD based on the information at hand and would update this as more information, such as breakthroughs or disproved hypotheses, evolves. This means that inferences informing probability assignment will logically contribute to one another [Spiegelhalter, 2004]. The probability of success in research (P(θ)) can be subjectively assessed based on past experience and knowledge of a variety of criteria, including previous findings in the area (or number of publications on the topic), preliminary data, promising new findings, personal experience, and specialization or technical expertise in a particular area. This information is used to inform which discoveries are more likely to occur than others. We assume that this information is summarized by a parameter θ that represents the stock of information exogenously available to the decision maker. Next, we need to understand how research funding R might influence the probability of discovery. It seems reasonable that the decision maker could believe that the probability of a “breakthrough” in basic science (B) and translational and clinical research (TC) is non-decreasing in R, meaning that the probability of discovery can be represented as the function:
Pj ( Rj ; θ j ) where j = B, TC
and
∂Pj ( Rj ; θ j ) ≥0 ∂Rj
If the partial derivative of this function with respect to R is greater than zero, then the probability Pj of a breakthrough increases with the level of funding Rj. If this partial derivative is equal to zero, then R has no influence on the probability of success, so more investment in research amounts to placing more bets on a fixed probability gamble.
3 If decision makers are risk-averse in the sense that they need to show returns in the near term to keep their jobs, then they would prefer smaller gambles with more regular payoffs. Operationally, increasing risk aversion of research funders could explain the emerging preference for commercializable innovation over basic science research aimed at discovery.
Zwicker and Emery/The economics of funding autism research
INSAR
Note that our description of probabilities ignores potential spillovers between RB and RTC investments. In other words, we have assumed that B and TC research are substitutes with respect to their activities. The decision maker can choose one or the other. In some cases, the investment in B is necessary prior to TC, making the relevant probabilities dependent, rather than independent, due to the temporal sequencing of research investments. For example, the tools for developing new treatments for ASD are limited by the reality that for the core problems associated with ASD there is currently no biological diagnosis or genetic test, little understanding of the neurological circuits involved, and few practical models in which to study ASD [Bauman & Schumann, 2013; Doyle & McDougle, 2012]. However, only by investing in basic research can advances in these areas be made. It is also likely that investments in B and TC are complementary, reflecting spillover effects and knowledge between research areas. In its most general form, our sketch of the decisions maker’s problem is to choose RB and RTC to maximize the expected social benefit from research subject to the total funds equal R:
Choose R B, R TC to maximize PB (R B ; θB ) u (BB ) + PTC (R TC ; θ TC ) u (BTC ) subject to R B + R TC ≤ R Generally, to maximize the expected social return on investment, the allocation of research dollars should occur in such a way that it equalizes the change in expected benefit from the last dollar allocated to each research area. Mathematically, this condition for maximum social benefit from the research dollar allocation can be expressed as the ratio of preference for benefits [∂PB(RB)/∂RB]/[∂PTC(RTC)/∂ RTC] = uj(BTC)/uj(BB). To say anything further with respect to how much research funding should be allocated across research areas, we need to know more about the functional relationship of Pj and Rj, and the utility function for Bj. It may be possible to evaluate Pj empirically, but the specification of uj(Bj) must be determined outside of the model and stated prior to analysis. Given a statement of what preferences are, an economist can then determine what the allocation of research dollars should be. For example, if the decision maker is “risk-neutral,” hence an expected value maximizer, funds will be allocated to the areas of science with the highest expected benefit for the society.
W = PB (R B ; θB ) BB + PTC (R TC ; θ TC ) BTC The condition necessary for maximizing social benefit from the research allocation simplifies to the ratio of the marginal changes in probabilities = ratio of the benefits, so:
INSAR
[∂PB (R B ) ∂R B ] [∂PTC (R TC ) ∂R TC ] = BTC BB. Suppose we assume the benefit of treatment research (BTC) to be a 50% reduction in the cost/burden of ASD where prevention of future cases arising from a breakthrough in basic science (BB) would be the full cost/ burden avoided. We can assume that BTC/BB = ½, meaning the ratio of marginal changes in the probability of a breakthrough should be ½. In other words, a social planner should move money between basic science (RB) to screening, translational, and clinical research (RTC) until the change in PB is ½ of the change in PTC. Bigger payoff bets require smaller improvements in probabilities of success to justify the incremental spending on them. If the returns to prevention are seen to be larger than the returns to treatment (like early childhood intervention), then it would not be surprising to see disproportionate resources going to basic science research even if the likely increases in the probability of discovery are small. However, we do not actually know by how much research funding today alters the probability of a breakthrough tomorrow (or how funding alters social systems to have the capacity to receive and integrate the research innovations). Simplifying the model further, we get a different criterion for optimal choice of Rj. If Pj(Rj;θj) = Pj(θj), meaning that the probability of research success in a given bet is independent of R, then our decision maker would have a different rule for allocating research funds. A risk-neutral decision maker will allocate all funds to the category with the highest expected value. In other words, rather than seeing where to divide the total funds between areas based on marginal changes in outcomes, in this circumstance the decision maker goes “all in” on the most promising area in terms of probability weighed payoffs. If a risk-averse decision maker or weighing the value of population groups is considered, then funds would be divided across different areas. However, these considerations incorporate a different objective function for the decision maker than simply maximizing value in a risk-neutral manner.
Can We at Least Determine Which Area Is Overfunded? A Case Study Using a Comparison of US and UK ASD Research Funding In theory, the model sketched above is prescriptive, and should be complete and coherent while weighing elements of social welfare to compare health outcomes from different types of research, in this case a comparison of basic science research and treatment/screening ASD research [Little, 2006; Sugden & Williams, 1978]. The advantage of this approach is the ability to incorporate previous information on research in the area or other
Zwicker and Emery/The economics of funding autism research
707
Table 1. UK
Autism Research Funding Allocation in the US and
Biology Risk factors Diagnosis Treatments and interventions Services Infrastructure and surveillance Lifespan issues Total
US (USD spent in 2010—in millions of $)
UK (USD spent from 2007 to 2011—in millions of $)
91.2 81.2 45.6 68.1 64.8 50.8 6.6 408.6
18.4 4.9 1.6 6.0 1.6 N/A 0.5 32.9
The December 2010 conversion rate of 1 GBP = 1.5829 USD was used. For more detailed information about the funding allocation information used, see Interagency Autism Coordinating Committee [2012] and Pellicano et al. [2013].
factors that may influence the decision, as well as update assigned probabilities as new information becomes available so that the most informed choice can be made when comparing the risk associated with alternate decisions. However, without information on the likelihood of a breakthrough and the utility of the financial value of the breakthrough for the society, this model can only describe differences in funding allocations. Normative evaluation in terms of relative over- or underfunding of research areas, or even identification of trade-offs that the society would be prepared to accept, cannot be made unless we know the preferences of the society, or at least the decision maker. A 2010 US funding portfolio analysis of ASD research funding shows that $408 million was spent on ASD research both publicly and privately in 2010, while in the UK a recent analysis of public and private research funding reports total spending at $32.9 million over a 5-year period (between 2007 and 2011) (Table 1) [Interagency Autism Coordinating Committee, 2012; Pellicano, Dinsmore, & Charman, 2013]. When adjusted for population size, in 2010 the amount spent on autism research in the UK was estimated at £0.04 per capita compared with £0.76 in the US [Pellicano et al., 2013]. In our model, we simplistically break the types of research down into basic research (B) and translational and clinical research (TC). Within the top two countries for ASD research spending, the UK has allocated a larger portion of funding to basic science, while the US tends toward more distributed spending across all categories (Fig. 1). Based on the categories outlined in Table 1, we include biology and risk factors into the basic research category (B) as we generally characterized this research as focusing more on mechanistic work resulting in earlier diagnosis and prevention of ASD research. In the translational/
708
clinical research (TC) category, we included diagnosis, treatments and interventions, services, infrastructure and surveillance, and lifespan issues, as we generalized these areas of research to typically have more direct applicability to those currently affected by ASD. These groupings are gross generalizations, and it should be noted that the diagnosis category in particular includes biomarker research that would largely be characterized as basic science. Within this grouping, the UK and the US allocated 71% and 42% of their ASD research funding toward B, respectively, leaving 29% and 58% allocated toward TC, respectively. Following from the simple model above, we take the perspective of a risk-neutral decision maker who places bets on fixed probability gambles. With the state of knowledge on ASD research, we can propose an even simpler perspective of a rational investor decision maker. Suppose that funders of research, like granting agencies, invest so long as they expect the outputs from the research to generate benefits sufficient to cover the opportunity cost of their capital. If that is the case, then RB and RTC are chosen so that:
P B PB BB = (1 + r ) = TC TC RTC RB That is, a rational investor decision maker would invest in research until the expected benefit (PjBj) is sufficient to earn a social rate of return (r). If B generated a higher r than TC, then B would be underfunded and (TC overfunded), meaning the portfolio approach would have resources allocated from TC to B in this example. However, even with this simple decision rule, there are limits to what can be assessed. We do not know PB, PTC, BB, BTC, or r (it must be chosen as well). We only know RB and RTC. So if we take what we know, then we can do the following:
RB PB = B B RTC PTC BTC The ratio of funding between the two areas is equal to the ratio of the expected benefits from the two areas. Consider the allocation of funding between B and TC in the US (42% B, 58% TC) and the UK (71% B, 29% TC). That means that the UK infers the expected benefits of B to be approximately three times that of the TC research where the US sees them as closer to equal. It is not clear, however, if this reflects differences across countries in probabilities of breakthroughs or in values assigned to the breakthrough—that is, neither decision maker is riskneutral. The different funding allocation seen in the UK compared with the US has been suggested to be a reflection of the lack of a high level systematic process for prioritizing
Zwicker and Emery/The economics of funding autism research
INSAR
Figure 1. Autism spectrum disorder research funding distribution in the US and UK. Funding allocation for each country is represented by the colors indicated in the legend. Sections outlined in black are categorized into basic science research (B), while those with no outline are in translational and clinical research. For more detailed information about funding allocation, see Interagency Autism Coordinating Committee [2012] and Pellicano et al. [2013].
and coordinating autism research across public and private organizations [Pellicano et al., 2014]. A recent study by Pellicano et al. [2014] used both focus groups and a large-scale survey to investigate the views of the autism community in the UK on the current funding allocation to discern their priorities for future research. The stakeholders in this study had a preference for a distribution of funding in the UK that is more balanced, emphasizing a need for a greater balance toward translational and clinical research. This would suggest that based on the probability of success attributed to treatment research, autism stakeholders take a social planner perspective where more resources should be allocated to TC research. Ultimately from this proposed model, we can say that the allocations are different, but we have no basis for determining if areas are funded appropriately without a better understanding of the values of the decision maker. We have no criteria for determining if stakeholder preferences would result in the “best” allocation of resources. It is possible that the decision to not incorporate stakeholder preferences for TC research aimed at alleviating burden in the current population with ASD may yield greater returns for the society in the future through the emphasis on basic research reflecting greater patience for outcomes.
Answering Bailey’s Question: Are the Benefits of an Economic Analysis of Autism Research Worth the Effort? Bailey [2009] assessed that: “[i]n a very small number of countries we already know something about the costs of ASDs, and sometimes the potential value of improving services. But, to quantify gains from future research advances we
INSAR
need to be able to estimate both the probability that the desired outcome will be achieved and the value of this advance for affected individuals over specific time intervals, as well as for society more broadly. This is not an easy task, as many parameters will have broad confidence intervals and the delay before a discovery translates into an effective treatment or preventative strategy will vary significantly between different fields of research. Moreover, we are familiar with macro-economists providing conflicting advice because their models differ either in their inputs, their assumptions or both; which begs the question of whether the benefits of an economic analysis of autism research would be worth the effort?” As Bailey [2009] notes, different assumptions across economic models can generate very different results. Bailey has largely raised issues of uncertainty over outcomes and parameters (which would be in any economic model) and issues around different methods. He has not considered the biggest issue for an economic model, the statement of the objective for the decision maker. The first and critical assumption is the statement of objective for ASD research. The choice of objective is neither something that economists should specify nor something in which they would be expected to agree on for reasons other than “analytic convenience.” To assess if an area of research is under- or overfunded, we need more transparency to know what the objective of the research funds is. The statement of the objective will then define how to assign benefits to outcomes and the normative criteria for assessing the values of outcomes. Second, the objective for the research will be based on how the society weighs the importance of the different members of the population, alive today and in future. If decision makers are patient and thinking about longer term outcomes for a population broader than that alive
Zwicker and Emery/The economics of funding autism research
709
today, then they may place higher value on breakthroughs that in the future will reduce the prevalence of ASD. In contrast, a more persistent focus would put more weight on research into breakthroughs that would improve the well-being of persons with ASD alive at this time. The usefulness of economics for answering questions with analytic models is not for big questions like the one Bailey is proposing for evaluating ASD research funding. Economists have been perceived as providing useful answers to narrow questions, addressing comparison of alternative therapies for addressing a given condition. This does not mean that these methods are useful for providing technical solutions to bigger questions. A similar point has been made in another context: “(Economists) use similar analytical tools and come up with similar answers to narrow questions. But when it comes to explaining the behavior of the global economy, economists cannot agree—in fact, most of them no longer seem to believe there is a single correct explanation. Economists rule the world, but they are not quite sure what to do with it” [Fox, 1999]. For evaluation purposes in health, economics is of greatest use as an end-stage consideration of known research outcomes, such as effect sizes and prevalence or risk. Such outcome measures can be monetized or assigned “utility scores” as with quality adjusted life years or disability adjusted life years. Importantly, the objectives for the discovery are defined prior to the evaluation and are understood. For example, in the case of an economic evaluation of a new drug’s cost-effectiveness against an alternative course of action, the effect of the drug is demonstrated in a randomized control trial. In this case, the population of interest is defined, the costs of the drug intervention are known, and the values of the outcomes are estimated. These values are abstracted from extensive literature on the methodological challenges in economic evaluations. The objective of research should be defined by what is best for the “society” and that evaluation is not something that economics can address. For better or for worse, ASD research funding allocation will depend on the judgment of whomever the society delegates to be the decision maker, and the allocations will reflect the decision maker’s preferences. Ultimately, the challenge is to select decision makers and processes that translate societal interests and preferences accurately and effectively into the choices made.
References Ades, A.E., Sculpher, M., Sutton, A., Abrams, K., Cooper, N., et al. (2006). Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics, 24, 1–19. doi:10.2165/00019053-200624010-00001
710
Bailey, A.J. (2009). Where are the autism economists? Autism Research, 2, 245. doi:10.1002/aur.99 Baron-Cohen, S. (1997). Mindblindness an essay on autism and theory of mind (200 p.). Retrieved from http:// cognet.mit.edu/library/books/view?isbn=026252225X Bauman, M.D., & Schumann, C.M. (2013). Is “bench-to-bedside” realistic for autism? An integrative neuroscience approach. Neuropsychiatry (London), 3, 159–168. doi:10.2217/ npy.13.18 Best, M.A. (2005). Bayesian approaches to clinical trials and health-care evaluation. Journal of the Royal Statistical Society. Series A, (Statistics in Society), 168, 261–261. doi:10.1111/j.1467-985X.2004.00347_17.x Blackorby, C., & Donaldson, D. (1992). Pigs and guinea pigs: A note on the ethics of animal exploitation. The Economic Journal, 102, 1345–1369. doi:10.2307/2234794 Coalition to Protect Research. (2007). National Institute of Health research priorities Q & A. Retrieved from http:// www.cossa.org/CPR/CPR_NIH_Priorities_QA.html Collier, R. (2010). Research allocations: The influence of material and social factors. Canadian Medical Association Journal, 182, E619–E620. doi:10.1503/cmaj.109-3280 Doyle, C.A., & McDougle, C.J. (2012). Pharmacologic treatments for the behavioral symptoms associated with autism spectrum disorders across the lifespan. Dialogues in Clinical Neuroscience, 14, 263–279. Fox, J. (1999). What in the world happened to economics? Economists are all finally speaking the same language, but they still can’t answer the big questions. Fortune. Retrieved from http://archive.fortune.com/magazines/fortune/fortune _archive/1999/03/15/256482/index.htm Godlee, F., Smith, J., & Marcovitch, H. (2011). Wakefield’s article linking MMR vaccine and autism was fraudulent. British Medical Journal (Clinical Research Ed.), 342, c7452. doi:10.1136/bmj.c7452 Health Economics Research Group. (2008). Medical research: What’s it worth? Estimating the economic benefits from medical research in the UK. Medical Research Council, The Wellcome Trust and Academy of Medical Sciences. Interagency Autism Coordinating Committee. (2012). 2010 Autism spectrum disorder research portfolio analysis report. Washington: Office of Autism Research Coordination, National Institutes of Health. Lenaway, D., Halverson, P., Sotnikov, S., Tilson, H., Corso, L., & Millington, W. (2006). Public health systems research: Setting a national agenda. American Journal of Public Health, 96, 410–413. doi:10.2105/ajph.2004.046037 Little, R.J. (2006). Calibrated Bayes: A Bayes/frequentist roadmap. The American Statistician, 60, 213–223. Morris, Z.S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: Understanding time lags in translational research. Journal of the Royal Society of Medicine, 104, 510–520. doi:10.1258/jrsm.2011.110180 Pellicano, E., Dinsmore, A., & Charman, T. (2014). What should autism research focus upon? Community views and priorities from the United Kingdom. Autism: The International Journal of Research and Practice, 18(7), 756–770. doi:10.1177/ 1362361314529627
Zwicker and Emery/The economics of funding autism research
INSAR
Pellicano, L., Dinsmore, A., & Charman, T. (2013). A future made together: Shaping autism research in the UK. London: Centre for Research in Autism and Education. Silverman, C., & Brosco, J.P. (2007). Understanding autism: Parents and pediatricians in historical perspective. Archives of Pediatrics and Adolescent Medicine, 161, 392–398. doi:10.1001/archpedi.161.4.392 Singh, J., Illes, J., Lazzeroni, L., & Hallmayer, J. (2009). Trends in US autism research funding. Journal of Autism and Developmental Disorders, 39, 788–795. doi:10.1007/s10803-0080685-0 Spiegelhalter, D.J. (2004). Incorporating Bayesian ideas into health-care evaluation. Statistical Science, 19, 156–174. doi:10.1214/088342304000000080
INSAR
Stokstad, E. (2007). Biomedical policy. New autism law focuses on patients, environment. Science, 315, 27. doi:10.1126/ science.315.5808.27a Sugden, R., & Williams, A. (1978). The principles of practical cost-benefit analysis. Oxford: Oxford University Press. Ward, V., House, A., & Hamer, S. (2009). Developing a framework for transferring knowledge into action: A thematic analysis of the literature. Journal of Health Services Research and Policy, 14, 156–164. doi:10.1258/ jhsrp.2009.008120
Zwicker and Emery/The economics of funding autism research
711
Introduction Despite sizeable investments in autism research, it is not possible to address all key questions associated with improving the lives of people with autism or preventing persons having autism [Bailey, 2009]. This results in the need to prioritize the use of available research funding for autism spectrum disorder (ASD). With rising reported prevalence of ASD and a high estimated economic burden of the condition, there can be a dramatic difference between the outcomes that the public wants and what science is equipped to provide [Collier, 2010]. As the ultimate success or dividends of autism research effort are probabilistic and likely many years away, the choice of how to allocate finite research budgets is far from obvious. As Bailey [2009] suggests, we really have no idea if research resources deployed across basic and other research areas (or across projects within research areas) are being allocated in the “best” way, even if it is by the categorical metric of being overfunded or underfunded. Bailey then argues that even a basic economic evaluation might be useful for providing this information. In this paper, we show that this bigger question is fundamentally dependent on societal values, which must be specified for an economic model to be useful for normative evaluation of outcomes. This means that there is not an objective,
technical criterion for big questions like the normative evaluation of research funding allocations. In this case, economic modeling cannot substitute for the judgment of experts and broader stakeholders, and is limited in the information that can be provided beyond what is available to decision makers already. The technical approach to funding priority setting is typically used by national funding bodies, such as the Canadian Institute of Health Research (Canada), Medical Research Council (UK), and the National Institutes of Health (US), with the allocation being driven largely by experts and scientific panels of researchers. When trying to choose which research areas, programs, and projects to fund, decision makers consider many different perspectives, including laboratory science and existing scientific opportunities, the future disease burden on society, the current public health needs, and the quality, experience, and sustainability of the proposed research protocols [Coalition to Protect Research, 2007]. This perspective highlights that allocation of funds is influenced by the dissemination of research findings, expectations of a “breakthrough,” new technological or theoretical frameworks, the changing “fashion” of some research areas, and path dependence of research funding reflecting interests in maintaining established research groups and networks [Bailey, 2009]. Perhaps not surprisingly, this
From the School of Public Policy, University of Calgary, Calgary, Alberta, Canada (J.D.Z.); Department of Economics, University of Calgary, Calgary, Alberta, Canada (J.C.H.E.) Received February 19, 2014; accepted for publication September 5, 2014 Address for correspondence and reprints: Jennifer D. Zwicker, School of Public Policy, University of Calgary, Downtown Campus, 906 8th Ave S.W., 5th floor, Calgary, Alberta T2P 1H9, Canada. E-mail: [email protected] All grant information in the following format: Grant sponsor Alberta Innovates Health Solutions; Grant number: 10003547. Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/aur.1423 © 2014 International Society for Autism Research, Wiley Periodicals, Inc.
704
Autism Research 7: 704–711, 2014
INSAR
approach to allocating funding has resulted in the majority of ASD research funds being allocated to the area of basic research. Recently, in the US, increased consultation with stakeholders has led to an allocation of research dollars that is interpreted as reflecting a preference for addressing societal needs, shifting resources from basic research to translational and clinical research [Lenaway et al., 2006]. The rise in ASD public and charitable research funding in the US has been attributed to the political momentum and awareness raised by ASD advocacy groups [Silverman & Brosco, 2007; Singh, Illes, Lazzeroni, & Hallmayer, 2009]. Work by Pellicano, Dinsmore, and Charman [2014] highlights the increasing pressure to represent stakeholder interests in the resource allocation process. However, the concern is that in some situations, the lay public interests impact allocation of scarce ASD research funding, to the point where research with greater probability of success or impact in the findings may receive less funding than the research deemed important by the public (such as the coverage in the media on ASD and vaccines) [Godlee, Smith, & Marcovitch, 2011; Stokstad, 2007]. Given the divergence in views over what the allocation of ASD research resources across research areas should be, Bailey [2009] asks whether there may be a technical resolution to the problem from the application of an economic model to provide decision makers allocating research funding to have access to “quantitative estimates of the value of a specific advance” [Bailey, 2009]. What Bailey is arguing is that the opportunity cost of allocating resources between the alternatives of clinical research vs. basic science research must be considered. Bailey points out that by analyzing the probability of research outcomes and the value of the output, we could determine areas of research that are relatively overfunded or underfunded, or at least identify key questions that remain to be answered. At a very minimum, this discussion would provide incentive for debate around some of these value issues [Bailey, 2009]. In taking up Bailey’s challenge that better informed policy decisions surrounding research funding allocations could be made using an economic approach, we propose a simple model of how a decision maker could “optimally” allocate research funds across two broad areas of ASD research: basic science (B), and translational and clinical research (TC). The decision problem addressed by economic evaluation considers the choice between different types of ASD research given a budget constraint, which is a question of allocative efficiency— are resources applied to their highest value use? What we wish to show is that for big questions like choosing the socially preferred, or efficient, research funding allocation, there is no objective or technical answer that an economic model can provide. Economic models are built on expressed statements of objectives for the resources
INSAR
allocated and the values that the society places on the outcomes. Economists do not specify the societal values but the values wholly determine the models. In the absence of this information, an economic model can only describe differences in funding allocations and possible trade-offs that can be made given an assumed objective, rather than answer questions of optimal allocation. This approach brings into question who should be determining the allocation of ASD research funding and how the benefits and returns to research investment should be determined. Specifically, we discuss some of the practical challenges associated with this rational decision model, such as specifying the preferences of the decision makers for benefits today vs. benefits tomorrow (time preference and discount rates), for more certain research outcomes (risk aversion), and for helping persons with ASD today over preventing future cases of ASD (weighing to account for the relative importance of different members of the population, or to define the population of interest).1 Without first determining what societal preferences are in terms of these concerns, economic modeling cannot substitute for the judgment of experts and broader stakeholders, and is limited in the information that can be provided beyond what is available to decision makers already.
Illustrating the Utility and Futility of Modeling “Optimal Research Funding” for ASD To illustrate the importance of establishing values, we present a simple decision analytic, Bayesian approach to the prioritization of research that accounts for the existing uncertainty in the values of all parameters affecting the cost-effectiveness of the alternative interventions [Ades et al., 2006; Best, 2005]. The underlying premise for funding research consists of the decision maker essentially making a bet on a researcher and his/her proposal by funding the research in the expectation that a breakthrough or advanced knowledge will occur. For example, the effect of a breakthrough in an area, such as genetic determinants of ASD, would mean that a Bayesian decision maker would go from the low likelihood of a cure to believing the probability is higher, resulting in a reevaluation of the probability of payoff that can occur. This may result in the reallocation of research funding in relation to the adjusted probabilities. Similarly, evidence disproving a potential cause (such as discredited studies suggesting vaccinations may cause ASD) will lower the
1 This issue was examined in the context of do we overuse animals in experiments and overconsume meat because we do not include the preferences of the animals in our decisions. See [Blackorby & Donaldson, 1992].
Zwicker and Emery/The economics of funding autism research
705
probability of a payoff from research in causes of ASD [Godlee et al., 2011]. A decision maker can allocate resources in a way that contains costs, manages demand, and ultimately maximizes the expected benefit from research. Suppose that the research funder has R total dollars to allocate between projects in two areas, basic science (B) and translational and clinical research (TC): RB + RTC = R. If there is a breakthrough, then the decision maker gets a payoff that is conditional on which outcomes occur and the value of the outcomes. The social benefit of a breakthrough in basic science and in translational and clinical research can be represented as BB and BTC, respectively. Naively, Bj (where j = B, TC) could represent the aggregate burden of ASD avoided due to the discovery where the burden value combines the costs of ASD to the society for the population of affected individuals alive today and affected individuals to be born in the future. This notation is obviously a gross abstraction of reality for ASD, which is characterized as a range of conditions classified as pervasive developmental disorders that can differ greatly in severity and prevalence [Baron-Cohen, 1997]. Further, RB and RTC investments potentially benefit different populations. One of the most contentious arguments is that research funding allocation toward the cause and prevention of ASD is to the benefit of future generations, whereas the development of treatment addresses the needs of the current population. The value of research breakthroughs is not as simple as calculating Bj. Decision makers may have preferences that result in different valuations of Bj that depend on the riskiness of Bj occurring (risk aversion would mean lower preference on “long shots” vs. “sure things”), over the timing of when Bj will be realized (such as a decision maker with a time preference that values the present more than the future),2 and who in the population is actually getting the benefit Bj (weighing the relative importance of members of the population). In other words, decision makers have preferences that would be represented with a “utility function,” uj(Bj). When we specify in our model that the preferences of the decision maker are represented by uj(Bj) = Bj, we are making strong statements about preferences—the decision maker is riskneutral and has no differential preference for breakthroughs in the treatment of persons with ASD alive now
2 The rate of return of research will critically depend on assumptions regarding time lag from discovery into practice, which is highly variable and often much longer for basic research [Morris, Wooding, & Grant, 2011; Ward, House, & Hamer, 2009]. For example, some studies have used a time lag of research spending of 17 years for the translation of basic research [Health Economics Research Group, 2008]. With the extent to which decision makers are “impatient” and prefer benefits now vs. benefits later, there will be a preference for avoiding investing in long-term research.
706
vs. breakthroughs that may prevent cases of ASD in the future.3 We have no way of knowing if this is the correct utility function. The next challenge is assigning values to the probability of a research breakthrough and specifying the relationship between research funding and the probability of a breakthrough. The success of a research discovery is probabilistic in the sense that we do not know with certainty if or when a research effort, as measured by dollars of funding (R), will yield a result. When deciding which research and researchers to fund, a Bayesian decision maker would start with a naive rule for allocating research funding of ASD based on the information at hand and would update this as more information, such as breakthroughs or disproved hypotheses, evolves. This means that inferences informing probability assignment will logically contribute to one another [Spiegelhalter, 2004]. The probability of success in research (P(θ)) can be subjectively assessed based on past experience and knowledge of a variety of criteria, including previous findings in the area (or number of publications on the topic), preliminary data, promising new findings, personal experience, and specialization or technical expertise in a particular area. This information is used to inform which discoveries are more likely to occur than others. We assume that this information is summarized by a parameter θ that represents the stock of information exogenously available to the decision maker. Next, we need to understand how research funding R might influence the probability of discovery. It seems reasonable that the decision maker could believe that the probability of a “breakthrough” in basic science (B) and translational and clinical research (TC) is non-decreasing in R, meaning that the probability of discovery can be represented as the function:
Pj ( Rj ; θ j ) where j = B, TC
and
∂Pj ( Rj ; θ j ) ≥0 ∂Rj
If the partial derivative of this function with respect to R is greater than zero, then the probability Pj of a breakthrough increases with the level of funding Rj. If this partial derivative is equal to zero, then R has no influence on the probability of success, so more investment in research amounts to placing more bets on a fixed probability gamble.
3 If decision makers are risk-averse in the sense that they need to show returns in the near term to keep their jobs, then they would prefer smaller gambles with more regular payoffs. Operationally, increasing risk aversion of research funders could explain the emerging preference for commercializable innovation over basic science research aimed at discovery.
Zwicker and Emery/The economics of funding autism research
INSAR
Note that our description of probabilities ignores potential spillovers between RB and RTC investments. In other words, we have assumed that B and TC research are substitutes with respect to their activities. The decision maker can choose one or the other. In some cases, the investment in B is necessary prior to TC, making the relevant probabilities dependent, rather than independent, due to the temporal sequencing of research investments. For example, the tools for developing new treatments for ASD are limited by the reality that for the core problems associated with ASD there is currently no biological diagnosis or genetic test, little understanding of the neurological circuits involved, and few practical models in which to study ASD [Bauman & Schumann, 2013; Doyle & McDougle, 2012]. However, only by investing in basic research can advances in these areas be made. It is also likely that investments in B and TC are complementary, reflecting spillover effects and knowledge between research areas. In its most general form, our sketch of the decisions maker’s problem is to choose RB and RTC to maximize the expected social benefit from research subject to the total funds equal R:
Choose R B, R TC to maximize PB (R B ; θB ) u (BB ) + PTC (R TC ; θ TC ) u (BTC ) subject to R B + R TC ≤ R Generally, to maximize the expected social return on investment, the allocation of research dollars should occur in such a way that it equalizes the change in expected benefit from the last dollar allocated to each research area. Mathematically, this condition for maximum social benefit from the research dollar allocation can be expressed as the ratio of preference for benefits [∂PB(RB)/∂RB]/[∂PTC(RTC)/∂ RTC] = uj(BTC)/uj(BB). To say anything further with respect to how much research funding should be allocated across research areas, we need to know more about the functional relationship of Pj and Rj, and the utility function for Bj. It may be possible to evaluate Pj empirically, but the specification of uj(Bj) must be determined outside of the model and stated prior to analysis. Given a statement of what preferences are, an economist can then determine what the allocation of research dollars should be. For example, if the decision maker is “risk-neutral,” hence an expected value maximizer, funds will be allocated to the areas of science with the highest expected benefit for the society.
W = PB (R B ; θB ) BB + PTC (R TC ; θ TC ) BTC The condition necessary for maximizing social benefit from the research allocation simplifies to the ratio of the marginal changes in probabilities = ratio of the benefits, so:
INSAR
[∂PB (R B ) ∂R B ] [∂PTC (R TC ) ∂R TC ] = BTC BB. Suppose we assume the benefit of treatment research (BTC) to be a 50% reduction in the cost/burden of ASD where prevention of future cases arising from a breakthrough in basic science (BB) would be the full cost/ burden avoided. We can assume that BTC/BB = ½, meaning the ratio of marginal changes in the probability of a breakthrough should be ½. In other words, a social planner should move money between basic science (RB) to screening, translational, and clinical research (RTC) until the change in PB is ½ of the change in PTC. Bigger payoff bets require smaller improvements in probabilities of success to justify the incremental spending on them. If the returns to prevention are seen to be larger than the returns to treatment (like early childhood intervention), then it would not be surprising to see disproportionate resources going to basic science research even if the likely increases in the probability of discovery are small. However, we do not actually know by how much research funding today alters the probability of a breakthrough tomorrow (or how funding alters social systems to have the capacity to receive and integrate the research innovations). Simplifying the model further, we get a different criterion for optimal choice of Rj. If Pj(Rj;θj) = Pj(θj), meaning that the probability of research success in a given bet is independent of R, then our decision maker would have a different rule for allocating research funds. A risk-neutral decision maker will allocate all funds to the category with the highest expected value. In other words, rather than seeing where to divide the total funds between areas based on marginal changes in outcomes, in this circumstance the decision maker goes “all in” on the most promising area in terms of probability weighed payoffs. If a risk-averse decision maker or weighing the value of population groups is considered, then funds would be divided across different areas. However, these considerations incorporate a different objective function for the decision maker than simply maximizing value in a risk-neutral manner.
Can We at Least Determine Which Area Is Overfunded? A Case Study Using a Comparison of US and UK ASD Research Funding In theory, the model sketched above is prescriptive, and should be complete and coherent while weighing elements of social welfare to compare health outcomes from different types of research, in this case a comparison of basic science research and treatment/screening ASD research [Little, 2006; Sugden & Williams, 1978]. The advantage of this approach is the ability to incorporate previous information on research in the area or other
Zwicker and Emery/The economics of funding autism research
707
Table 1. UK
Autism Research Funding Allocation in the US and
Biology Risk factors Diagnosis Treatments and interventions Services Infrastructure and surveillance Lifespan issues Total
US (USD spent in 2010—in millions of $)
UK (USD spent from 2007 to 2011—in millions of $)
91.2 81.2 45.6 68.1 64.8 50.8 6.6 408.6
18.4 4.9 1.6 6.0 1.6 N/A 0.5 32.9
The December 2010 conversion rate of 1 GBP = 1.5829 USD was used. For more detailed information about the funding allocation information used, see Interagency Autism Coordinating Committee [2012] and Pellicano et al. [2013].
factors that may influence the decision, as well as update assigned probabilities as new information becomes available so that the most informed choice can be made when comparing the risk associated with alternate decisions. However, without information on the likelihood of a breakthrough and the utility of the financial value of the breakthrough for the society, this model can only describe differences in funding allocations. Normative evaluation in terms of relative over- or underfunding of research areas, or even identification of trade-offs that the society would be prepared to accept, cannot be made unless we know the preferences of the society, or at least the decision maker. A 2010 US funding portfolio analysis of ASD research funding shows that $408 million was spent on ASD research both publicly and privately in 2010, while in the UK a recent analysis of public and private research funding reports total spending at $32.9 million over a 5-year period (between 2007 and 2011) (Table 1) [Interagency Autism Coordinating Committee, 2012; Pellicano, Dinsmore, & Charman, 2013]. When adjusted for population size, in 2010 the amount spent on autism research in the UK was estimated at £0.04 per capita compared with £0.76 in the US [Pellicano et al., 2013]. In our model, we simplistically break the types of research down into basic research (B) and translational and clinical research (TC). Within the top two countries for ASD research spending, the UK has allocated a larger portion of funding to basic science, while the US tends toward more distributed spending across all categories (Fig. 1). Based on the categories outlined in Table 1, we include biology and risk factors into the basic research category (B) as we generally characterized this research as focusing more on mechanistic work resulting in earlier diagnosis and prevention of ASD research. In the translational/
708
clinical research (TC) category, we included diagnosis, treatments and interventions, services, infrastructure and surveillance, and lifespan issues, as we generalized these areas of research to typically have more direct applicability to those currently affected by ASD. These groupings are gross generalizations, and it should be noted that the diagnosis category in particular includes biomarker research that would largely be characterized as basic science. Within this grouping, the UK and the US allocated 71% and 42% of their ASD research funding toward B, respectively, leaving 29% and 58% allocated toward TC, respectively. Following from the simple model above, we take the perspective of a risk-neutral decision maker who places bets on fixed probability gambles. With the state of knowledge on ASD research, we can propose an even simpler perspective of a rational investor decision maker. Suppose that funders of research, like granting agencies, invest so long as they expect the outputs from the research to generate benefits sufficient to cover the opportunity cost of their capital. If that is the case, then RB and RTC are chosen so that:
P B PB BB = (1 + r ) = TC TC RTC RB That is, a rational investor decision maker would invest in research until the expected benefit (PjBj) is sufficient to earn a social rate of return (r). If B generated a higher r than TC, then B would be underfunded and (TC overfunded), meaning the portfolio approach would have resources allocated from TC to B in this example. However, even with this simple decision rule, there are limits to what can be assessed. We do not know PB, PTC, BB, BTC, or r (it must be chosen as well). We only know RB and RTC. So if we take what we know, then we can do the following:
RB PB = B B RTC PTC BTC The ratio of funding between the two areas is equal to the ratio of the expected benefits from the two areas. Consider the allocation of funding between B and TC in the US (42% B, 58% TC) and the UK (71% B, 29% TC). That means that the UK infers the expected benefits of B to be approximately three times that of the TC research where the US sees them as closer to equal. It is not clear, however, if this reflects differences across countries in probabilities of breakthroughs or in values assigned to the breakthrough—that is, neither decision maker is riskneutral. The different funding allocation seen in the UK compared with the US has been suggested to be a reflection of the lack of a high level systematic process for prioritizing
Zwicker and Emery/The economics of funding autism research
INSAR
Figure 1. Autism spectrum disorder research funding distribution in the US and UK. Funding allocation for each country is represented by the colors indicated in the legend. Sections outlined in black are categorized into basic science research (B), while those with no outline are in translational and clinical research. For more detailed information about funding allocation, see Interagency Autism Coordinating Committee [2012] and Pellicano et al. [2013].
and coordinating autism research across public and private organizations [Pellicano et al., 2014]. A recent study by Pellicano et al. [2014] used both focus groups and a large-scale survey to investigate the views of the autism community in the UK on the current funding allocation to discern their priorities for future research. The stakeholders in this study had a preference for a distribution of funding in the UK that is more balanced, emphasizing a need for a greater balance toward translational and clinical research. This would suggest that based on the probability of success attributed to treatment research, autism stakeholders take a social planner perspective where more resources should be allocated to TC research. Ultimately from this proposed model, we can say that the allocations are different, but we have no basis for determining if areas are funded appropriately without a better understanding of the values of the decision maker. We have no criteria for determining if stakeholder preferences would result in the “best” allocation of resources. It is possible that the decision to not incorporate stakeholder preferences for TC research aimed at alleviating burden in the current population with ASD may yield greater returns for the society in the future through the emphasis on basic research reflecting greater patience for outcomes.
Answering Bailey’s Question: Are the Benefits of an Economic Analysis of Autism Research Worth the Effort? Bailey [2009] assessed that: “[i]n a very small number of countries we already know something about the costs of ASDs, and sometimes the potential value of improving services. But, to quantify gains from future research advances we
INSAR
need to be able to estimate both the probability that the desired outcome will be achieved and the value of this advance for affected individuals over specific time intervals, as well as for society more broadly. This is not an easy task, as many parameters will have broad confidence intervals and the delay before a discovery translates into an effective treatment or preventative strategy will vary significantly between different fields of research. Moreover, we are familiar with macro-economists providing conflicting advice because their models differ either in their inputs, their assumptions or both; which begs the question of whether the benefits of an economic analysis of autism research would be worth the effort?” As Bailey [2009] notes, different assumptions across economic models can generate very different results. Bailey has largely raised issues of uncertainty over outcomes and parameters (which would be in any economic model) and issues around different methods. He has not considered the biggest issue for an economic model, the statement of the objective for the decision maker. The first and critical assumption is the statement of objective for ASD research. The choice of objective is neither something that economists should specify nor something in which they would be expected to agree on for reasons other than “analytic convenience.” To assess if an area of research is under- or overfunded, we need more transparency to know what the objective of the research funds is. The statement of the objective will then define how to assign benefits to outcomes and the normative criteria for assessing the values of outcomes. Second, the objective for the research will be based on how the society weighs the importance of the different members of the population, alive today and in future. If decision makers are patient and thinking about longer term outcomes for a population broader than that alive
Zwicker and Emery/The economics of funding autism research
709
today, then they may place higher value on breakthroughs that in the future will reduce the prevalence of ASD. In contrast, a more persistent focus would put more weight on research into breakthroughs that would improve the well-being of persons with ASD alive at this time. The usefulness of economics for answering questions with analytic models is not for big questions like the one Bailey is proposing for evaluating ASD research funding. Economists have been perceived as providing useful answers to narrow questions, addressing comparison of alternative therapies for addressing a given condition. This does not mean that these methods are useful for providing technical solutions to bigger questions. A similar point has been made in another context: “(Economists) use similar analytical tools and come up with similar answers to narrow questions. But when it comes to explaining the behavior of the global economy, economists cannot agree—in fact, most of them no longer seem to believe there is a single correct explanation. Economists rule the world, but they are not quite sure what to do with it” [Fox, 1999]. For evaluation purposes in health, economics is of greatest use as an end-stage consideration of known research outcomes, such as effect sizes and prevalence or risk. Such outcome measures can be monetized or assigned “utility scores” as with quality adjusted life years or disability adjusted life years. Importantly, the objectives for the discovery are defined prior to the evaluation and are understood. For example, in the case of an economic evaluation of a new drug’s cost-effectiveness against an alternative course of action, the effect of the drug is demonstrated in a randomized control trial. In this case, the population of interest is defined, the costs of the drug intervention are known, and the values of the outcomes are estimated. These values are abstracted from extensive literature on the methodological challenges in economic evaluations. The objective of research should be defined by what is best for the “society” and that evaluation is not something that economics can address. For better or for worse, ASD research funding allocation will depend on the judgment of whomever the society delegates to be the decision maker, and the allocations will reflect the decision maker’s preferences. Ultimately, the challenge is to select decision makers and processes that translate societal interests and preferences accurately and effectively into the choices made.
References Ades, A.E., Sculpher, M., Sutton, A., Abrams, K., Cooper, N., et al. (2006). Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics, 24, 1–19. doi:10.2165/00019053-200624010-00001
710
Bailey, A.J. (2009). Where are the autism economists? Autism Research, 2, 245. doi:10.1002/aur.99 Baron-Cohen, S. (1997). Mindblindness an essay on autism and theory of mind (200 p.). Retrieved from http:// cognet.mit.edu/library/books/view?isbn=026252225X Bauman, M.D., & Schumann, C.M. (2013). Is “bench-to-bedside” realistic for autism? An integrative neuroscience approach. Neuropsychiatry (London), 3, 159–168. doi:10.2217/ npy.13.18 Best, M.A. (2005). Bayesian approaches to clinical trials and health-care evaluation. Journal of the Royal Statistical Society. Series A, (Statistics in Society), 168, 261–261. doi:10.1111/j.1467-985X.2004.00347_17.x Blackorby, C., & Donaldson, D. (1992). Pigs and guinea pigs: A note on the ethics of animal exploitation. The Economic Journal, 102, 1345–1369. doi:10.2307/2234794 Coalition to Protect Research. (2007). National Institute of Health research priorities Q & A. Retrieved from http:// www.cossa.org/CPR/CPR_NIH_Priorities_QA.html Collier, R. (2010). Research allocations: The influence of material and social factors. Canadian Medical Association Journal, 182, E619–E620. doi:10.1503/cmaj.109-3280 Doyle, C.A., & McDougle, C.J. (2012). Pharmacologic treatments for the behavioral symptoms associated with autism spectrum disorders across the lifespan. Dialogues in Clinical Neuroscience, 14, 263–279. Fox, J. (1999). What in the world happened to economics? Economists are all finally speaking the same language, but they still can’t answer the big questions. Fortune. Retrieved from http://archive.fortune.com/magazines/fortune/fortune _archive/1999/03/15/256482/index.htm Godlee, F., Smith, J., & Marcovitch, H. (2011). Wakefield’s article linking MMR vaccine and autism was fraudulent. British Medical Journal (Clinical Research Ed.), 342, c7452. doi:10.1136/bmj.c7452 Health Economics Research Group. (2008). Medical research: What’s it worth? Estimating the economic benefits from medical research in the UK. Medical Research Council, The Wellcome Trust and Academy of Medical Sciences. Interagency Autism Coordinating Committee. (2012). 2010 Autism spectrum disorder research portfolio analysis report. Washington: Office of Autism Research Coordination, National Institutes of Health. Lenaway, D., Halverson, P., Sotnikov, S., Tilson, H., Corso, L., & Millington, W. (2006). Public health systems research: Setting a national agenda. American Journal of Public Health, 96, 410–413. doi:10.2105/ajph.2004.046037 Little, R.J. (2006). Calibrated Bayes: A Bayes/frequentist roadmap. The American Statistician, 60, 213–223. Morris, Z.S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: Understanding time lags in translational research. Journal of the Royal Society of Medicine, 104, 510–520. doi:10.1258/jrsm.2011.110180 Pellicano, E., Dinsmore, A., & Charman, T. (2014). What should autism research focus upon? Community views and priorities from the United Kingdom. Autism: The International Journal of Research and Practice, 18(7), 756–770. doi:10.1177/ 1362361314529627
Zwicker and Emery/The economics of funding autism research
INSAR
Pellicano, L., Dinsmore, A., & Charman, T. (2013). A future made together: Shaping autism research in the UK. London: Centre for Research in Autism and Education. Silverman, C., & Brosco, J.P. (2007). Understanding autism: Parents and pediatricians in historical perspective. Archives of Pediatrics and Adolescent Medicine, 161, 392–398. doi:10.1001/archpedi.161.4.392 Singh, J., Illes, J., Lazzeroni, L., & Hallmayer, J. (2009). Trends in US autism research funding. Journal of Autism and Developmental Disorders, 39, 788–795. doi:10.1007/s10803-0080685-0 Spiegelhalter, D.J. (2004). Incorporating Bayesian ideas into health-care evaluation. Statistical Science, 19, 156–174. doi:10.1214/088342304000000080
INSAR
Stokstad, E. (2007). Biomedical policy. New autism law focuses on patients, environment. Science, 315, 27. doi:10.1126/ science.315.5808.27a Sugden, R., & Williams, A. (1978). The principles of practical cost-benefit analysis. Oxford: Oxford University Press. Ward, V., House, A., & Hamer, S. (2009). Developing a framework for transferring knowledge into action: A thematic analysis of the literature. Journal of Health Services Research and Policy, 14, 156–164. doi:10.1258/ jhsrp.2009.008120
Zwicker and Emery/The economics of funding autism research
711
