Bird Brains: The Evolution of Scientific Misconduct George A. Diamond, MD
ardly a week goesby without a new revelation of scientific fraud, plagiarism, or conflict of interest appearing in Science or Nature, Newsweek or Time. Is there really an epidemic of immoral investigators out there? Is the pressureto publish and the lack of supervision causing some amoral individuals to bend the rules more often? Or is the issue largely illusory, a result of heightened scrutiny of the moral majority in responseto a few notorious transgressions? In attempts to forestall the threatened epidemic, the National Academy of Science, the American Association for the Advancement of Science, the Council of Biology Editors and the American Bar Association have all held symposia on the ethical conduct of science’; the Department of Health and Human Services,2the Institute of Medicine,3 and the Association of American Universities4 have begun to issue guidelines for ethical conduct; a prominent journal editor has called for a formal audit of the scientific literature5; and a member of Congressis considering the introduction of federal legislation.6 Is all this activity necessary?To answer this question, I will propose a simple theoretical model for the evolution of scientific misconduct, and will thereby show that well-intentioned efforts to control the epidemic could actually serve to spread it. The model is based on one first proposed by Maynard Smith to explain the evolution of aggressivebehavior in animals,’ and my description closely parallels that of Dawkins.8 The model assumesthat individual members of the population are competitors for some common resource. In nature (the jungle, not the journal) this resource might be the food necessaryto one’s survival or the sexual partners necessaryto the survival of one’s genes. In science (again, not the journal) the resource might be the research funds, facilities, publications, and prestige necessary to the survival of one’s ideas (including one’s ideas about the best way to promote those ideas). Let us begin with the simplest possible jungle, in which there are only 2 strategies for survival: dove and hawk. Doves engage in long periods of empirical ob-
servation, statistical analysis and intellectual debate. Hawks lie, cheat and steal. Whenever dovesmeet at the watering hole each evening, they argue in a dignified way about the events of the day, and carefully document their arguments in the jungle book for all to see. Eventually, one dove decidesthe other is right, the issue is unimportant, or the argument is uninteresting, and politely backs down. No one gets hurt. When hawks meet, on the other hand, they fight ferociously until one of them is severely injured. Whenever a dove meets a hawk, the dove flies away quickly before being hurt. The survival of one’s ideas is the name of this game. Accordingly, we can score the outcome associatedwith each of these pairings along somearbitrary utility scale representing the likelihood that one’s ideas will spread throughout the jungle: 30 points for a win, 0 points for a loss, -10 points for wasting time, and -480 points for being hurt. (These utilities are highly subjective.I have chosen30 points to representthe modestgain associated with acceptance of one’s idea by another, and -480 points to represent the material loss associatedwith outright exposure as a miscreant, becausethesevalues are generally consistent with my own personal beliefs. Readers are invited to experiment with alternative values representing their beliefs.) Each time a dove meets a dove there will be a long contest resulting in a winner (30 points for the win -10 points for wasting time) and a loser (0 points for the loss - 10 points for wasting time). Becauseeach dove will win half the time, the average scorefor dove/dove encounters is 5 points (20/2 - 10/2). In a population made up entirely of doves,each dove ekesout a meager existence. Supposenow a single hawk enters this idyllic jungle. Becauseit is the only hawk around, eachcontest will be with a dove, and it will win every one of these contests without being hurt, giving it an average score of 30 points compared with an averageof 5 points for doves. This advantageover the competition will quickly attract other hawks to the jungle, and might even entice some of the less dignified doves to becomemore aggressive. As a result, hawks will begin to encounter other hawks. Whenever a hawk meets another hawk, one of them will win (30 points) and the other will be injured From the Division of Cardiology, Cedars-SinaiMedical Center, and the (-480 points). Becauseeach hawk has an even chance School of Medicine, University of California, Los Angeles, California. This work was supported in part by a Specialized Center of Research of winning, the average score for hawk/hawk encoun(SCOR) grant (HL-17651) from the National Institutes of Health, ters is -225 points (30/2 - 480/2). In a population Bethesda,Maryland. Manuscript receivedMarch l&1990, and accept- made up entirely of hawks, each hawk doesvery poorly. ed March 21. A dove will not do very well either, but at least it will Addressfor reprints: George A. Diamond, MD, Cedars-Sinai Medical Center, Division of Cardiology, 8700 Beverly Boulevard, Los Ange- do better than a hawk. Although the dove will lose evles, California 90048. ery one of its encountersin this savagejungle, its aver-
H
372
THE AMERICAN JOURNAL OF CARDIOLOGY VOLUME 66
age scoreof 0 is still much better than the hawk’s average scoreof -225. In a population of hawks, therefore, a dove has the advantage.As before, this advantage too will not go unnoticed for long. Tt wil! eventua!!y attract more doves, and might convince some of the less malicious hawks to mend their hurtful ways. If the behavior of hawks tends to encouragethe proliferation of doves, and the behavior of doves tends to encouragethe proliferation of hawks, is there some stable mixture of hawks and doves? Indeed there is. This stable proportion is that at which the average score for hawks is equal to the average score for doves. If we let p equal the proportion of hawks in the population and 1 - p equal the proportion of doves,then the averagescore for hawks is -225 p + 30 (1 - p) and the averagescorefor dovesis 0 p + 5 (1 - p). If we set these 2 scoresequal to each other, we find p = l/10. Accordingly, a population of hawks and dovesin a ratio of 1:9 is stable, given these utility values. There is no incentive for the immigration of new hawks or doves, or for the behavioral conversion of existing hawks and doves.Actually, theseneed not be pure mixtures; probabilistic behavior on the part of each individual is all that is needed:1 hawk and 9 dovesis equivalent to 10 “dawks” (aggressive doves that fight like hawks 10% of the time). Unfortunately, there’s a price to be paid for this stability. The average score for a 1:9 mix of hawks and doves is only 4.5 points, compared with 5 points for a pure population of doves. Everyone would do better if they would agree to being doves,but the resultant segregated society would be vulnerable to the treachery of a single defector. The integrated society is stable precisely becauseit removes the advantage of defection, and it pays a tithe for this stability. Our common senserebels against the unconditional Procrustean behavior patterns of hawks and doves. We like to think we tailor our behavior in somerational way to the situation at hand. Owls, for example, might chooseto behave like doves in the company of doves, and like hawks in the company of hawks. Hawks could not take advantageof a population composedentirely of owls, becausethey would all seem like hawks to them. Similarly, doves could not take advantage of owls becausethey would all seemlike doves to them. Axelrod subjected the “tit for tat” strategy of the owl to an extensive seriesof computer simulations.’ He showedthat tit for tat successfullyresisted invasion by a variety of other strategies-many of which were substantially more sophisticated-and concluded that it possessed all the characteristics necessaryfor the evolution of cooperation, First, the strategy is nice; it starts out cooperating, and always reciprocates cooperation. Second, the strategy is procokable; it swiftly punishes any breach of cooperation. Third, it is forgiuing; it does not hold a grudge, and reciprocates cooperation again immediately after imposing punishment. Fourth, it is simple; its rules for punishment and reward are readily recognized. A number of real biological and social examples of tit for tat have been identified,9 demonstrating that the
jungle is not inhabited solely by winners and losers. Often, potential competitors share a common set of interests,and lie together in harmony.‘O On the other hand, whenever competitors do not share the same interests, envious strategies (directed more to minimizing the survival of another than to maximizing one’s own survival) can also evolve.” So can outright conflicts of interest.” Suppose, for example, that an editorial referee and a journal editor (you, dear reader, can supply the ornithological metaphors here-1 know who feathers my nest) are considering whether or not to publish a revolutionary manuscript that has some small probability, p, of being correct, and 1 - p of not being correct (the first report of cold fusion, for example). Call a the gain to the editor if she publishes the paper and it is later verified, and call b the loss to the editor if she publishes the paper and it is later refuted. If we define utility as the net gain associated with publication and verification minus the net loss associated with publication and refutation (assuming the lossesassociatedwith rejection are negligible), the editor’s utility is ap - b (1 - p). As long as this utility is positive, the editor should publish the manuscript, and this occurs whenever p > b/(a + 6). Similarly, call c the gain to the referee if he recommends publication and it is later verified, and call d the loss to the referee if he recommendspublication and it is later refuted. By the same reasoning, the referee’s utility is cp - d(1 p). As long as this utility is positive the referee should recommend publication, and this occurs whenever p > d/(c + d). When d/(c + d) > p > b/(a + b), however, the preferencesof the referee and the editor will be in conflict: the editor has more to gain by publishing, while the referee has more to gain by not recommending publication. How can we better manage the ecology of this jungle? We can begin by taking a censusof the number of hawks and doves and owls, and thereby determine if there really is a problem that needsto be managed.“s12 But we would also need to enumerate all the utilities, and this is no easy task. These utilities are highly subjective, probably subconscious,and not likely to be volunteered freely even if they are known. It might be easier to discover a hawk’s sexual inclinations than its ethical motivations. Even if we did conduct a comprehensivecensus,and obtained a full and accurate audit of the utilities, could we use the information effectively? If we knew the utilities, we could calculate the optimal proportion of hawks and doves required for stability, but the result might be too high for us to stomach. Do we really want to live in a society populated by so many hawks, even if it is a stable society? We could legislate changesin the utilities that would tend to drive down the proportion of hawks, but the magnitude of such changes would have to be substantial. To reduce the proportion of hawks from 10%to l%, for example, would require us to increase the cost of losing from -480 points to -4,980 points (changing the reward for winning will not do it). Are we prepared to exact such Draconian penalties?
THE AMERICAN JOURNAL OF CARDIOLOGY AUGUST 1, 1990
373
Even if we could change the numbers, should we REFERENCES change them? Recall that a society of self-satisfied 1. Culliton BJ. Scientists confront misconduct. Science 1988:24/:1748-1749. Department of Health and Human Services. Responsibilities of awardee and dovesis easy prey for a single clever hawk. What effect 2.applicant institutions for dealing with and reporting possible misconduct in sciwould legislated ethical sanctions have on the behavior ence. Fed Reg 1989;54:32446-32451, of the existing flock of doves,and on the likelihood that 3. Institute of Medicine Committee on the Responsible Conduct of Research (Rubinstein AH, Chairman). IOM report of a study on the responsible conduct of new doves and owls will chooseto enter the jungle? research in the health sciences. Clin Res 1989;37:179-191. Until the appropriate environmental impact studies 4. Association of American Universities. Framework for Institutional Policies and are done, I am inclined to think we should leave the Procedures to Deal with Fraud in Research. November 4. 1988. 5. Culliton BJ. Random audit of papers proposed. Science /988;242:657academic jungle pretty much alone, trusting that the 6S8. wildlife there will evolve optimal cooperative strategies 6. Holden C. New rules on misconduct. Science /989;245:593. on its own (the recent flurry of transgressionsnotwith- 7. Maynard Smith J. Evolution and the theory of games. Am Sci 1976,64:41standing). This is not a libertarian call for anarchy, but 45. 8. Dawkins R. The Selfish Gene. New York: Oxford University Press; 1976. an enlightened doctrine of laissez faire. The time-hon- 9. Axelrod R. The Evolution of Cooperation. New York; Basic Books: 1984. ored safeguards-scholarly peer review, the public dis- 10. van Neumann J, Morgenstern 0. Theory of Games and Economic Behavior. Third edition. Princetont Princeton Uniuersify Press, 1953. closure of conflicts and the pointed letter to the editor11. Diamond GA, Rozanski A, Steuer M. Playing doctor: application of game should, of course, continue. But the underlying structure theory to medical decision-making. J Chron Dis 1986:39:669-677, of our jungle always has been-and must continue to 12. Shapiro MF, Charrow RP. Scientific misconduct in investigational drug N Engl .I Med 198$;312:731-736. be-founded on trust, not on hastily written restrictive trials. 13. Palca J. NIH conflict-of-interest guidelines shot down, Science 1990; 247:154-155. regulations that evince a presumption of guilt.r3
374
THE AMERICAN JOURNAL OF CARDIOLOGY VOLUME 66
ardly a week goesby without a new revelation of scientific fraud, plagiarism, or conflict of interest appearing in Science or Nature, Newsweek or Time. Is there really an epidemic of immoral investigators out there? Is the pressureto publish and the lack of supervision causing some amoral individuals to bend the rules more often? Or is the issue largely illusory, a result of heightened scrutiny of the moral majority in responseto a few notorious transgressions? In attempts to forestall the threatened epidemic, the National Academy of Science, the American Association for the Advancement of Science, the Council of Biology Editors and the American Bar Association have all held symposia on the ethical conduct of science’; the Department of Health and Human Services,2the Institute of Medicine,3 and the Association of American Universities4 have begun to issue guidelines for ethical conduct; a prominent journal editor has called for a formal audit of the scientific literature5; and a member of Congressis considering the introduction of federal legislation.6 Is all this activity necessary?To answer this question, I will propose a simple theoretical model for the evolution of scientific misconduct, and will thereby show that well-intentioned efforts to control the epidemic could actually serve to spread it. The model is based on one first proposed by Maynard Smith to explain the evolution of aggressivebehavior in animals,’ and my description closely parallels that of Dawkins.8 The model assumesthat individual members of the population are competitors for some common resource. In nature (the jungle, not the journal) this resource might be the food necessaryto one’s survival or the sexual partners necessaryto the survival of one’s genes. In science (again, not the journal) the resource might be the research funds, facilities, publications, and prestige necessary to the survival of one’s ideas (including one’s ideas about the best way to promote those ideas). Let us begin with the simplest possible jungle, in which there are only 2 strategies for survival: dove and hawk. Doves engage in long periods of empirical ob-
servation, statistical analysis and intellectual debate. Hawks lie, cheat and steal. Whenever dovesmeet at the watering hole each evening, they argue in a dignified way about the events of the day, and carefully document their arguments in the jungle book for all to see. Eventually, one dove decidesthe other is right, the issue is unimportant, or the argument is uninteresting, and politely backs down. No one gets hurt. When hawks meet, on the other hand, they fight ferociously until one of them is severely injured. Whenever a dove meets a hawk, the dove flies away quickly before being hurt. The survival of one’s ideas is the name of this game. Accordingly, we can score the outcome associatedwith each of these pairings along somearbitrary utility scale representing the likelihood that one’s ideas will spread throughout the jungle: 30 points for a win, 0 points for a loss, -10 points for wasting time, and -480 points for being hurt. (These utilities are highly subjective.I have chosen30 points to representthe modestgain associated with acceptance of one’s idea by another, and -480 points to represent the material loss associatedwith outright exposure as a miscreant, becausethesevalues are generally consistent with my own personal beliefs. Readers are invited to experiment with alternative values representing their beliefs.) Each time a dove meets a dove there will be a long contest resulting in a winner (30 points for the win -10 points for wasting time) and a loser (0 points for the loss - 10 points for wasting time). Becauseeach dove will win half the time, the average scorefor dove/dove encounters is 5 points (20/2 - 10/2). In a population made up entirely of doves,each dove ekesout a meager existence. Supposenow a single hawk enters this idyllic jungle. Becauseit is the only hawk around, eachcontest will be with a dove, and it will win every one of these contests without being hurt, giving it an average score of 30 points compared with an averageof 5 points for doves. This advantageover the competition will quickly attract other hawks to the jungle, and might even entice some of the less dignified doves to becomemore aggressive. As a result, hawks will begin to encounter other hawks. Whenever a hawk meets another hawk, one of them will win (30 points) and the other will be injured From the Division of Cardiology, Cedars-SinaiMedical Center, and the (-480 points). Becauseeach hawk has an even chance School of Medicine, University of California, Los Angeles, California. This work was supported in part by a Specialized Center of Research of winning, the average score for hawk/hawk encoun(SCOR) grant (HL-17651) from the National Institutes of Health, ters is -225 points (30/2 - 480/2). In a population Bethesda,Maryland. Manuscript receivedMarch l&1990, and accept- made up entirely of hawks, each hawk doesvery poorly. ed March 21. A dove will not do very well either, but at least it will Addressfor reprints: George A. Diamond, MD, Cedars-Sinai Medical Center, Division of Cardiology, 8700 Beverly Boulevard, Los Ange- do better than a hawk. Although the dove will lose evles, California 90048. ery one of its encountersin this savagejungle, its aver-
H
372
THE AMERICAN JOURNAL OF CARDIOLOGY VOLUME 66
age scoreof 0 is still much better than the hawk’s average scoreof -225. In a population of hawks, therefore, a dove has the advantage.As before, this advantage too will not go unnoticed for long. Tt wil! eventua!!y attract more doves, and might convince some of the less malicious hawks to mend their hurtful ways. If the behavior of hawks tends to encouragethe proliferation of doves, and the behavior of doves tends to encouragethe proliferation of hawks, is there some stable mixture of hawks and doves? Indeed there is. This stable proportion is that at which the average score for hawks is equal to the average score for doves. If we let p equal the proportion of hawks in the population and 1 - p equal the proportion of doves,then the averagescore for hawks is -225 p + 30 (1 - p) and the averagescorefor dovesis 0 p + 5 (1 - p). If we set these 2 scoresequal to each other, we find p = l/10. Accordingly, a population of hawks and dovesin a ratio of 1:9 is stable, given these utility values. There is no incentive for the immigration of new hawks or doves, or for the behavioral conversion of existing hawks and doves.Actually, theseneed not be pure mixtures; probabilistic behavior on the part of each individual is all that is needed:1 hawk and 9 dovesis equivalent to 10 “dawks” (aggressive doves that fight like hawks 10% of the time). Unfortunately, there’s a price to be paid for this stability. The average score for a 1:9 mix of hawks and doves is only 4.5 points, compared with 5 points for a pure population of doves. Everyone would do better if they would agree to being doves,but the resultant segregated society would be vulnerable to the treachery of a single defector. The integrated society is stable precisely becauseit removes the advantage of defection, and it pays a tithe for this stability. Our common senserebels against the unconditional Procrustean behavior patterns of hawks and doves. We like to think we tailor our behavior in somerational way to the situation at hand. Owls, for example, might chooseto behave like doves in the company of doves, and like hawks in the company of hawks. Hawks could not take advantageof a population composedentirely of owls, becausethey would all seem like hawks to them. Similarly, doves could not take advantage of owls becausethey would all seemlike doves to them. Axelrod subjected the “tit for tat” strategy of the owl to an extensive seriesof computer simulations.’ He showedthat tit for tat successfullyresisted invasion by a variety of other strategies-many of which were substantially more sophisticated-and concluded that it possessed all the characteristics necessaryfor the evolution of cooperation, First, the strategy is nice; it starts out cooperating, and always reciprocates cooperation. Second, the strategy is procokable; it swiftly punishes any breach of cooperation. Third, it is forgiuing; it does not hold a grudge, and reciprocates cooperation again immediately after imposing punishment. Fourth, it is simple; its rules for punishment and reward are readily recognized. A number of real biological and social examples of tit for tat have been identified,9 demonstrating that the
jungle is not inhabited solely by winners and losers. Often, potential competitors share a common set of interests,and lie together in harmony.‘O On the other hand, whenever competitors do not share the same interests, envious strategies (directed more to minimizing the survival of another than to maximizing one’s own survival) can also evolve.” So can outright conflicts of interest.” Suppose, for example, that an editorial referee and a journal editor (you, dear reader, can supply the ornithological metaphors here-1 know who feathers my nest) are considering whether or not to publish a revolutionary manuscript that has some small probability, p, of being correct, and 1 - p of not being correct (the first report of cold fusion, for example). Call a the gain to the editor if she publishes the paper and it is later verified, and call b the loss to the editor if she publishes the paper and it is later refuted. If we define utility as the net gain associated with publication and verification minus the net loss associated with publication and refutation (assuming the lossesassociatedwith rejection are negligible), the editor’s utility is ap - b (1 - p). As long as this utility is positive, the editor should publish the manuscript, and this occurs whenever p > b/(a + 6). Similarly, call c the gain to the referee if he recommends publication and it is later verified, and call d the loss to the referee if he recommendspublication and it is later refuted. By the same reasoning, the referee’s utility is cp - d(1 p). As long as this utility is positive the referee should recommend publication, and this occurs whenever p > d/(c + d). When d/(c + d) > p > b/(a + b), however, the preferencesof the referee and the editor will be in conflict: the editor has more to gain by publishing, while the referee has more to gain by not recommending publication. How can we better manage the ecology of this jungle? We can begin by taking a censusof the number of hawks and doves and owls, and thereby determine if there really is a problem that needsto be managed.“s12 But we would also need to enumerate all the utilities, and this is no easy task. These utilities are highly subjective, probably subconscious,and not likely to be volunteered freely even if they are known. It might be easier to discover a hawk’s sexual inclinations than its ethical motivations. Even if we did conduct a comprehensivecensus,and obtained a full and accurate audit of the utilities, could we use the information effectively? If we knew the utilities, we could calculate the optimal proportion of hawks and doves required for stability, but the result might be too high for us to stomach. Do we really want to live in a society populated by so many hawks, even if it is a stable society? We could legislate changesin the utilities that would tend to drive down the proportion of hawks, but the magnitude of such changes would have to be substantial. To reduce the proportion of hawks from 10%to l%, for example, would require us to increase the cost of losing from -480 points to -4,980 points (changing the reward for winning will not do it). Are we prepared to exact such Draconian penalties?
THE AMERICAN JOURNAL OF CARDIOLOGY AUGUST 1, 1990
373
Even if we could change the numbers, should we REFERENCES change them? Recall that a society of self-satisfied 1. Culliton BJ. Scientists confront misconduct. Science 1988:24/:1748-1749. Department of Health and Human Services. Responsibilities of awardee and dovesis easy prey for a single clever hawk. What effect 2.applicant institutions for dealing with and reporting possible misconduct in sciwould legislated ethical sanctions have on the behavior ence. Fed Reg 1989;54:32446-32451, of the existing flock of doves,and on the likelihood that 3. Institute of Medicine Committee on the Responsible Conduct of Research (Rubinstein AH, Chairman). IOM report of a study on the responsible conduct of new doves and owls will chooseto enter the jungle? research in the health sciences. Clin Res 1989;37:179-191. Until the appropriate environmental impact studies 4. Association of American Universities. Framework for Institutional Policies and are done, I am inclined to think we should leave the Procedures to Deal with Fraud in Research. November 4. 1988. 5. Culliton BJ. Random audit of papers proposed. Science /988;242:657academic jungle pretty much alone, trusting that the 6S8. wildlife there will evolve optimal cooperative strategies 6. Holden C. New rules on misconduct. Science /989;245:593. on its own (the recent flurry of transgressionsnotwith- 7. Maynard Smith J. Evolution and the theory of games. Am Sci 1976,64:41standing). This is not a libertarian call for anarchy, but 45. 8. Dawkins R. The Selfish Gene. New York: Oxford University Press; 1976. an enlightened doctrine of laissez faire. The time-hon- 9. Axelrod R. The Evolution of Cooperation. New York; Basic Books: 1984. ored safeguards-scholarly peer review, the public dis- 10. van Neumann J, Morgenstern 0. Theory of Games and Economic Behavior. Third edition. Princetont Princeton Uniuersify Press, 1953. closure of conflicts and the pointed letter to the editor11. Diamond GA, Rozanski A, Steuer M. Playing doctor: application of game should, of course, continue. But the underlying structure theory to medical decision-making. J Chron Dis 1986:39:669-677, of our jungle always has been-and must continue to 12. Shapiro MF, Charrow RP. Scientific misconduct in investigational drug N Engl .I Med 198$;312:731-736. be-founded on trust, not on hastily written restrictive trials. 13. Palca J. NIH conflict-of-interest guidelines shot down, Science 1990; 247:154-155. regulations that evince a presumption of guilt.r3
374
THE AMERICAN JOURNAL OF CARDIOLOGY VOLUME 66
