TREE-1808; No. of Pages 2
Letter
Data availability and model complexity, generality, and utility: a reply to Lonergan Matthew R. Evans1, Tim G. Benton2, Volker Grimm3, Catherine M. Lessells4, Maureen A. O’Malley5, Aristides Moustakas1, and Michael Weisberg6 1
School of Biological and Chemical Sciences, Queen Mary University of London, Mile End Road, London E1 4NS, UK Institute of Integrative and Comparative Biology, University of Leeds, Leeds LS2 9JT, UK 3 Helmholtz Center for Environmental Research, Department of Ecological Modelling, Permoserstrasse 15, 04318 Leipzig, Germany 4 Nederlands Institut voor Ecologie, PO Box 50, 6700 AB Wageningen, The Netherlands 5 Department of Philosophy, School of Philosophical and Historical Inquiry, Quadrangle A14, University of Sydney, NSW, Australia 6 Department of Philosophy, University of Pennsylvania, 433 Cohen Hall, Philadelphia, PA 19104, USA 2
Lonergan [1] suggests that the claim made in our recent paper [2], that ‘viewing simple models as the main way to achieve generality may be an obstacle to the progress of ecological research’, will depend on the availability of data. We have discussed this issue in a recent article [3] in which we acknowledge that to obtain more robust predictions and better understanding of ecosystem processes there needs to be ‘greater emphasis on constraining models with data and on hypothesis testing within models’. In that paper we also point out how the ability of ecologists to collect data has been revolutionised by technological advances. For example, in the past few days it has been shown how southern right whales can be surveyed by satellite [4]. Ecologists are taught that the path to generality lies via simple models. Our article [2] was aimed at providing a philosophically informed challenge to this view and suggested that the perceived generality of simple models can be superficial because as models become simpler they lose the ability to provide actual instances of explanation. They become instead demonstrations of possible explanations. In our paper we emphasised that this function of simple models is extremely useful. Our main argument was that the maxim ‘simple = general = good’ should not be accepted unquestioningly by ecological modellers. It will be the case that additional complexity will mean that more data are required for calibration, but one point of agreement among ecologists is that the real world is complex. This complexity does not simply vanish if one reduces multiple processes down to a summary variable: it has only been hidden [5]. For example, a term such as the intrinsic population growth rate in a logistic equation is a single parameter that can be estimated from data and uses up few degrees of freedom (potentially only one). However, such a portmanteau parameter gives little understanding of the processes that underlie population growth. For this it is necessary to decompose this one parameter into several. Both approaches are useful, but let us not pretend that we have fully understood the system if we have reduced everything down to the simplest model we can devise. Corresponding author: Evans, M.R. ([email protected]). 0169-5347/ ß 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.tree.2014.03.004
If we take complexity to be the number of parameters estimated, then as complexity increases model fit to data cannot get worse. However, we dispute the suggestion that this makes our argument a tautology. Our claim was that the reason why simple models are perceived as having more generality than complex models is because the standard of fidelity used to assess model fit is lower for simple models than for complex models. This results in them appearing to have the ability to fit a greater range of systems while not actually fitting any of them well. Note that our use of ‘fidelity’ refers to ‘standards theorists use to evaluate a model’s ability to represent phenomena’ ([6], p. 39) rather than to how close the predictions of each model actually are to the data fitted to it, as claimed by Lonergan [1]. The final test of a model is its ability to make predictions about observations unused in model design and parameterisation. Developing complex models will often be demanding of both logical thought and data. One of the most useful products of a modelling exercise is to demonstrate potentially important mechanisms where data are lacking. A good example of this can be seen in the development of climate models, which have become progressively more complex with time and actually have wider confidence intervals now than in the past [7]. The increasing complexity has resulted from greater understanding of the climate system and a reduction in the number of simplifying assumptions. The desire to make climate models reflect the complexity of the climate system has stimulated research into the terrestrial carbon cycle, the role of aerosols, and the roles of clouds and ground-toatmosphere feedback. Would these phenomena have been subjected to such active research without models that required their quantification? Lonergan offers a counsel of despair: complex models cannot be tested against data unless many data are available. He infers that the search for general patterns across ecology will often be consigned to simple approaches. The result would be stagnation in the progress of ecology. We think that we need to embrace the complexity of the real world and reflect this in our models, and if necessary collect the appropriate data. Thirty years ago Medawar wrote that what ‘sets the genuine sciences apart from those that arrogate to themselves the title without really earning Trends in Ecology & Evolution xx (2014) 1–2
1
TREE-1808; No. of Pages 2
Letter it is their predictive capability’ ([8], p4). If we think this is right then we should also wish ecology to be able to make strong predictions: the current focus on simple models is a hindrance in reaching this goal. References 1 Lonergan, M. (2014) Data availability constrains model complexity, generality, and utility: a response to Evans et al. Trends Ecol. Evol. http://dx.doi.org/10.1016/j.tree.2014.03.005 2 Evans, M.R. et al. (2013) Do simple models lead to generality in ecology? Trends Ecol. Evol. 28, 578–583
2
Trends in Ecology & Evolution xxx xxxx, Vol. xxx, No. x
3 Evans, M.R. et al. (2013) Predictive systems ecology. Proc. R. Soc. B: Biol. Sci. 280, 20131452 4 Fretwell, P.T. et al. (2014) Whales from space: counting southern right whales by satellite. PLoS ONE 9, e88655 5 Donaldson, D.D. and Nisbet, R.M. (1999) Population dynamics and spatial scale: effects of system size on population persistence. Ecology 80, 2492–2507 6 Weisberg, M. (2013) Simulation and Similarity: Using Models to Understand the World, Oxford University Press 7 Maslin, M. and Austin, P. (2014) Climate models at their limit? Nature 486, 183–184 8 Medawar, P. (1984) The Limits of Science, Oxford University Press
Letter
Data availability and model complexity, generality, and utility: a reply to Lonergan Matthew R. Evans1, Tim G. Benton2, Volker Grimm3, Catherine M. Lessells4, Maureen A. O’Malley5, Aristides Moustakas1, and Michael Weisberg6 1
School of Biological and Chemical Sciences, Queen Mary University of London, Mile End Road, London E1 4NS, UK Institute of Integrative and Comparative Biology, University of Leeds, Leeds LS2 9JT, UK 3 Helmholtz Center for Environmental Research, Department of Ecological Modelling, Permoserstrasse 15, 04318 Leipzig, Germany 4 Nederlands Institut voor Ecologie, PO Box 50, 6700 AB Wageningen, The Netherlands 5 Department of Philosophy, School of Philosophical and Historical Inquiry, Quadrangle A14, University of Sydney, NSW, Australia 6 Department of Philosophy, University of Pennsylvania, 433 Cohen Hall, Philadelphia, PA 19104, USA 2
Lonergan [1] suggests that the claim made in our recent paper [2], that ‘viewing simple models as the main way to achieve generality may be an obstacle to the progress of ecological research’, will depend on the availability of data. We have discussed this issue in a recent article [3] in which we acknowledge that to obtain more robust predictions and better understanding of ecosystem processes there needs to be ‘greater emphasis on constraining models with data and on hypothesis testing within models’. In that paper we also point out how the ability of ecologists to collect data has been revolutionised by technological advances. For example, in the past few days it has been shown how southern right whales can be surveyed by satellite [4]. Ecologists are taught that the path to generality lies via simple models. Our article [2] was aimed at providing a philosophically informed challenge to this view and suggested that the perceived generality of simple models can be superficial because as models become simpler they lose the ability to provide actual instances of explanation. They become instead demonstrations of possible explanations. In our paper we emphasised that this function of simple models is extremely useful. Our main argument was that the maxim ‘simple = general = good’ should not be accepted unquestioningly by ecological modellers. It will be the case that additional complexity will mean that more data are required for calibration, but one point of agreement among ecologists is that the real world is complex. This complexity does not simply vanish if one reduces multiple processes down to a summary variable: it has only been hidden [5]. For example, a term such as the intrinsic population growth rate in a logistic equation is a single parameter that can be estimated from data and uses up few degrees of freedom (potentially only one). However, such a portmanteau parameter gives little understanding of the processes that underlie population growth. For this it is necessary to decompose this one parameter into several. Both approaches are useful, but let us not pretend that we have fully understood the system if we have reduced everything down to the simplest model we can devise. Corresponding author: Evans, M.R. ([email protected]). 0169-5347/ ß 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.tree.2014.03.004
If we take complexity to be the number of parameters estimated, then as complexity increases model fit to data cannot get worse. However, we dispute the suggestion that this makes our argument a tautology. Our claim was that the reason why simple models are perceived as having more generality than complex models is because the standard of fidelity used to assess model fit is lower for simple models than for complex models. This results in them appearing to have the ability to fit a greater range of systems while not actually fitting any of them well. Note that our use of ‘fidelity’ refers to ‘standards theorists use to evaluate a model’s ability to represent phenomena’ ([6], p. 39) rather than to how close the predictions of each model actually are to the data fitted to it, as claimed by Lonergan [1]. The final test of a model is its ability to make predictions about observations unused in model design and parameterisation. Developing complex models will often be demanding of both logical thought and data. One of the most useful products of a modelling exercise is to demonstrate potentially important mechanisms where data are lacking. A good example of this can be seen in the development of climate models, which have become progressively more complex with time and actually have wider confidence intervals now than in the past [7]. The increasing complexity has resulted from greater understanding of the climate system and a reduction in the number of simplifying assumptions. The desire to make climate models reflect the complexity of the climate system has stimulated research into the terrestrial carbon cycle, the role of aerosols, and the roles of clouds and ground-toatmosphere feedback. Would these phenomena have been subjected to such active research without models that required their quantification? Lonergan offers a counsel of despair: complex models cannot be tested against data unless many data are available. He infers that the search for general patterns across ecology will often be consigned to simple approaches. The result would be stagnation in the progress of ecology. We think that we need to embrace the complexity of the real world and reflect this in our models, and if necessary collect the appropriate data. Thirty years ago Medawar wrote that what ‘sets the genuine sciences apart from those that arrogate to themselves the title without really earning Trends in Ecology & Evolution xx (2014) 1–2
1
TREE-1808; No. of Pages 2
Letter it is their predictive capability’ ([8], p4). If we think this is right then we should also wish ecology to be able to make strong predictions: the current focus on simple models is a hindrance in reaching this goal. References 1 Lonergan, M. (2014) Data availability constrains model complexity, generality, and utility: a response to Evans et al. Trends Ecol. Evol. http://dx.doi.org/10.1016/j.tree.2014.03.005 2 Evans, M.R. et al. (2013) Do simple models lead to generality in ecology? Trends Ecol. Evol. 28, 578–583
2
Trends in Ecology & Evolution xxx xxxx, Vol. xxx, No. x
3 Evans, M.R. et al. (2013) Predictive systems ecology. Proc. R. Soc. B: Biol. Sci. 280, 20131452 4 Fretwell, P.T. et al. (2014) Whales from space: counting southern right whales by satellite. PLoS ONE 9, e88655 5 Donaldson, D.D. and Nisbet, R.M. (1999) Population dynamics and spatial scale: effects of system size on population persistence. Ecology 80, 2492–2507 6 Weisberg, M. (2013) Simulation and Similarity: Using Models to Understand the World, Oxford University Press 7 Maslin, M. and Austin, P. (2014) Climate models at their limit? Nature 486, 183–184 8 Medawar, P. (1984) The Limits of Science, Oxford University Press
