Advances in Health Sciences Education 1: 175-177, 1997. ( 1997 Kluwer Academic Publishers. Printed in the Netherlands.
175
From the Editors: Research, Theory and Science in Medical Education
The subtitle of the Journal - Theory and Practice has had a checkered career.
Sometimes it was on the masthead, sometimes off; sometimes on the letterhead, sometimes off. It seems that no-one but I, really understands why it's there. So I guess it's time to explain myself. As we mentioned in issue 1, the idea for the title was hatched over a couple of Anchor Steam beers in San Francisco at a late afternoon meeting among Peter de Liefde, Henk Schmidt and myself. Advances is there because it's more upbeat than retreats,Health Sciences is there in recognition of the fact that much research in health professions education occurs in disciplines other than medicine, and much of the research done in medicine applies equally well to all professions. It made little sense for a struggling new journal to be unnecessarily restrictive. But why Theory andPractice?Practiceis easy, as much of educational research is directed to improving educational practices. But why theory? Basically, because we wanted to highlight the research and scientific orientation of the Journal. Naturally, if we only accepted articles which are testing theories, we would likely only put out about one issue every decade. Still, we believe that a theoretical orientation represents a positive direction for research. Which brings us to the more substantive question. What is the relationship between research, theory, methodology, and science anyway? They are clearly not one and the same thing; all research is not scientific research. It's easy to come up with examples of strong research methods directed at goals which are clearly not serving science. For instance, arguably the most methodologically rigorous survey research is done by polling companies. They know more about question design, random sampling, follow up, compliance than most of us so-called experts in universities. But I don't really think that George Gallup, were he alive, would call himself a scientist. In the same vein, many of my friends in clinical epidemiology equate science with methodological rigour, but, as a criterion for good science, that does not hold up under scrutiny. A large, carefully designed, randomized trial of another me-too NSAID may constitute a major contribution to a drug company's bottom line, but it is hardly a major contributor to science. What distinguishes science from non-science is not the rigour of the methodology. Scientific rigour cannot be absent; since clearly one thing which distinguishes
176
FROM THE EDITORS
scientific from lay theories (Cold weather causes colds; women who adopt are more likely to get pregnant, some people have ESP) is the possibility of refutation by sound experimentation. But methodology is not a sufficient prerequisite of good science. Rather, the essence of science lies in the generation of general and enduring knowledge. Scientists are in the business of seeking universals, whether about the decay of a sulphur nucleus after absorbing a neutron or the behaviour of snails. And, in turn, the universality of the phenomena resides in the theoretical explanation which specifies the relationship among the variables under study. Lest the qualitative researchers get up in arms about this admittedly positivist view of the world, I don't think that the perspective has much to do with the qualitative-quantitative debate (which incidentally seems to be getting more strident and less informative with time). It is just as likely (in fact perhaps more likely) to see atheoretical quantitative research as qualitative. And I really do believe that generalization from the particular is an inherent and fundamental characteristic of science - if you can't generalize from the particular situation you studied, then publish it as a memoir or institutional biography, but not as science. This definition, although incomplete as all definitions tend to be, does provide a neat Occam's razor to apply to specific research. For example, the finding of a study that shows that NSAID works (p. 5) is not science, since, although the finding may be generalizable to larger patient populations, it reveals nothing about the mechanisms that make it work. That work was done in the lab (some other company's lab) long before. Similarly, most curriculum evaluations such as the many studies of problem-based learning, are not science, since they too are unable to offer any theoretical explanation for effectiveness. Any finding that females, science majors, mature students, or porpoises do better or worse on a performance exam is not science since there is only the phenomenon, not the explanation (although an attempt to model success on the exam using a range of variables, hopefully guided by a theory of mastery, starts to look like science). In brief, to say that such-and-such a difference has been observed is not science; to say why it was observed, is science. Good science is distinguished, not by its methods but by its questions and answers. As Mary Budd Rowe, a distinguished professor of science education at Stanford said (in an issue of PopularMechanics- of all places):
"Science is a special kind of storytelling, with no right or wrong answers, just better and better stories". The character of the stories also enters in the way we, as scientists, judge our products. Sound methodology is a given, but few get excited about a study because it was so gosh darn rigorous. The words we use tell us something about what we value. When we talk about a study we admire, we call it neat, pretty, elegant, gorgeous, even sexy. It sounds more like a description out of Playboy or Cosmo than the New EnglandJournal. This aesthetic judgment may reflect any number of characteristics: the originality of the question, the cleverness of the design, the universality of the conclusion, and sometimes even the counter-intuitive nature of
FROM THE EDITORS
177
the findings. But central to the appeal of the good studies is that they reveal some enduring truth; a new way to interpret the world, a better theory of the world. So, on this criterion, how are we in health sciences education doing as a scientific community? Not very well. This year, at the American Educational Research Association meeting in Chicago, Ilene Harris reviewed a decade of submissions to Division I (Education in the Professions). She reported that, of about 1000 manuscripts accepted over the years, only 6% had any theoretical justification. So it's pretty clear that relatively little of what we do and what we publish has any theory-building component. And it's equally clear that we, as editors, are hardly in a position to demand a substantive theory in every paper. But the subtitle of the Journal, if not a description of the present, does represent a direction for the future. Geoff Norman Editor
175
From the Editors: Research, Theory and Science in Medical Education
The subtitle of the Journal - Theory and Practice has had a checkered career.
Sometimes it was on the masthead, sometimes off; sometimes on the letterhead, sometimes off. It seems that no-one but I, really understands why it's there. So I guess it's time to explain myself. As we mentioned in issue 1, the idea for the title was hatched over a couple of Anchor Steam beers in San Francisco at a late afternoon meeting among Peter de Liefde, Henk Schmidt and myself. Advances is there because it's more upbeat than retreats,Health Sciences is there in recognition of the fact that much research in health professions education occurs in disciplines other than medicine, and much of the research done in medicine applies equally well to all professions. It made little sense for a struggling new journal to be unnecessarily restrictive. But why Theory andPractice?Practiceis easy, as much of educational research is directed to improving educational practices. But why theory? Basically, because we wanted to highlight the research and scientific orientation of the Journal. Naturally, if we only accepted articles which are testing theories, we would likely only put out about one issue every decade. Still, we believe that a theoretical orientation represents a positive direction for research. Which brings us to the more substantive question. What is the relationship between research, theory, methodology, and science anyway? They are clearly not one and the same thing; all research is not scientific research. It's easy to come up with examples of strong research methods directed at goals which are clearly not serving science. For instance, arguably the most methodologically rigorous survey research is done by polling companies. They know more about question design, random sampling, follow up, compliance than most of us so-called experts in universities. But I don't really think that George Gallup, were he alive, would call himself a scientist. In the same vein, many of my friends in clinical epidemiology equate science with methodological rigour, but, as a criterion for good science, that does not hold up under scrutiny. A large, carefully designed, randomized trial of another me-too NSAID may constitute a major contribution to a drug company's bottom line, but it is hardly a major contributor to science. What distinguishes science from non-science is not the rigour of the methodology. Scientific rigour cannot be absent; since clearly one thing which distinguishes
176
FROM THE EDITORS
scientific from lay theories (Cold weather causes colds; women who adopt are more likely to get pregnant, some people have ESP) is the possibility of refutation by sound experimentation. But methodology is not a sufficient prerequisite of good science. Rather, the essence of science lies in the generation of general and enduring knowledge. Scientists are in the business of seeking universals, whether about the decay of a sulphur nucleus after absorbing a neutron or the behaviour of snails. And, in turn, the universality of the phenomena resides in the theoretical explanation which specifies the relationship among the variables under study. Lest the qualitative researchers get up in arms about this admittedly positivist view of the world, I don't think that the perspective has much to do with the qualitative-quantitative debate (which incidentally seems to be getting more strident and less informative with time). It is just as likely (in fact perhaps more likely) to see atheoretical quantitative research as qualitative. And I really do believe that generalization from the particular is an inherent and fundamental characteristic of science - if you can't generalize from the particular situation you studied, then publish it as a memoir or institutional biography, but not as science. This definition, although incomplete as all definitions tend to be, does provide a neat Occam's razor to apply to specific research. For example, the finding of a study that shows that NSAID works (p. 5) is not science, since, although the finding may be generalizable to larger patient populations, it reveals nothing about the mechanisms that make it work. That work was done in the lab (some other company's lab) long before. Similarly, most curriculum evaluations such as the many studies of problem-based learning, are not science, since they too are unable to offer any theoretical explanation for effectiveness. Any finding that females, science majors, mature students, or porpoises do better or worse on a performance exam is not science since there is only the phenomenon, not the explanation (although an attempt to model success on the exam using a range of variables, hopefully guided by a theory of mastery, starts to look like science). In brief, to say that such-and-such a difference has been observed is not science; to say why it was observed, is science. Good science is distinguished, not by its methods but by its questions and answers. As Mary Budd Rowe, a distinguished professor of science education at Stanford said (in an issue of PopularMechanics- of all places):
"Science is a special kind of storytelling, with no right or wrong answers, just better and better stories". The character of the stories also enters in the way we, as scientists, judge our products. Sound methodology is a given, but few get excited about a study because it was so gosh darn rigorous. The words we use tell us something about what we value. When we talk about a study we admire, we call it neat, pretty, elegant, gorgeous, even sexy. It sounds more like a description out of Playboy or Cosmo than the New EnglandJournal. This aesthetic judgment may reflect any number of characteristics: the originality of the question, the cleverness of the design, the universality of the conclusion, and sometimes even the counter-intuitive nature of
FROM THE EDITORS
177
the findings. But central to the appeal of the good studies is that they reveal some enduring truth; a new way to interpret the world, a better theory of the world. So, on this criterion, how are we in health sciences education doing as a scientific community? Not very well. This year, at the American Educational Research Association meeting in Chicago, Ilene Harris reviewed a decade of submissions to Division I (Education in the Professions). She reported that, of about 1000 manuscripts accepted over the years, only 6% had any theoretical justification. So it's pretty clear that relatively little of what we do and what we publish has any theory-building component. And it's equally clear that we, as editors, are hardly in a position to demand a substantive theory in every paper. But the subtitle of the Journal, if not a description of the present, does represent a direction for the future. Geoff Norman Editor
