HHS Public Access Author manuscript Author Manuscript
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23. Published in final edited form as: Good Clin Pract J. 2007 April ; 14(4): 13–15.
Three Steps to Ensure Good Science and Safe Research David B. Resnik, JD, PhD Bioethicist and IRB Vice Chair, NIEHS/NIH, Box 12233, Mail Drop NH06, Research Triangle Park, NC, 27709. Phone: 919 541 5658. Fax: 919 541 3659. Email: [email protected]
Abstract Author Manuscript
This article examines the strengths and weaknesses of three different approaches to reviewing scientific issues related to research with human subjects, and argues that the best strategy is one that combines all three approaches.
Author Manuscript
One of the essential ethical requirements for research with human subjects is that it be scientifically well-designed. Poorly designed research can result in several different ethical problems, including exposing human subjects to risks unnecessarily, undermining the consent process, compromising confidentiality, adversely affecting the benefit/risk ratio, and unfair selection of subjects.[1] Several ethics codes, including the Nuremberg Code, the World Medical Association’s Declaration of Helsinki, the Council for the International Organization of Medical Sciences (CIOMS) Guidelines, and the International Conference on Harmonization’s Good Clinical Practice (GCP) Guidelines, address the importance of sound scientific design in research with human subjects. Though the U.S. federal research regulations do not explicitly state that research must be well-designed, they imply that it should be, since poorly designed research may fail to meet the requirement that risks be minimized or that risks be reasonable in relation to benefits.[2],[3] Canada’s Tri-Council Policy Statement recommends that research protocols conform to scholarly standards, and the European Union’s Clinical Trial Directive states that clinical research should follow GCP guidelines.[4],[5]
Author Manuscript
While various regulations and codes spell out the practical details of ethical review of human research, they have little to say about the scientific review process. This lack of regulatory guidance concerning scientific review creates many practical dilemmas for organizations conducting or sponsoring research with human subjects. Who should review that the scientific issues related to research with human subjects? What qualifies someone to be a reviewer? What policies and procedures should apply to the review process? What are the scientific standards reviewers should use to assess the design of a research protocol? How should reviewers be educated and trained? How many reviewers are needed? When should reviews take place? Can reviews be appealed? Unfortunately, there is not sufficient space in this article to address all of these important issues concerning scientific review. Instead, the article will provide an overview of three approaches to reviewing the scientific issues that can arise in research with human subjects. The first approach is for the committee that reviews the ethical issues related to the research, such as an Institutional Review Board (IRB), Research Ethics Board (REB), or Research Ethics Committee (REC), to also review the scientific issues. Most ethics regulations and
Resnik
Page 2
Author Manuscript
guidelines require or recommend that ethical review committees have sufficient expertise to assess scientific issues.[1] For example, the U.S. federal research regulations require that IRBs be composed of scientists and non-scientists.[2],[3] Although the regulations do not define the term “scientist,” it is reasonable to assume that it includes people from the biomedical, physical, or social sciences, as well as clinicians, mathematicians, or engineers. [6] Committee members can apply their expertise to protocols they are qualified to review. For example, a social scientist on an ethical review committee can assess a social science protocol, a pediatrician can review a protocol involving children in medical research, and so on.
Author Manuscript
There are at least two advantages to this approach. First, it conserves resources, because there is no separate committee to review scientific issues. Second, it makes it easy to integrate the discussion of scientific and ethical issues because scientific review occurs in the context of ethics review. However, there is a major disadvantage of this approach: the committee may lack the expertise needed for the scientific review. For example, if an IRB has two physician members, none of whom are oncologists, it may lack the expertise to review a protocol involving a new cancer drug. If an IRB has no social scientists as members, it may lack the expertise to review a protocol that uses survey research methods. One way for a committee to overcome this problem is to expand its roster to include additional members with scientific expertise. Expanding the roster is not always the best way to solve this problem, however, because the committee may need to become very large (i.e. twenty members or more) in order to review the variety of scientific issues that may arise. If a committee is too large, it may have difficulty achieving a quorum, and the reviews may become inconsistent as different members attend different meetings.
Author Manuscript
To enhance its expertise, an ethical review committee could use outside consultants from time-to-time to review protocols.[7] Although the consultants would not vote protocols (only members can vote), they could provide information and advice to the committee. For example, in the cancer drug trial example discussed above, the IRB could ask an oncologist for his or her expert opinion. While this second approach helps to overcome the problem of lack of scientific expertise, it has some other shortcomings. First, it may be difficult (and awkward) to obtain the services of experts on an ad hoc basis, under the pressure of deadlines. Second, if the committee uses different reviewers each time, there may be some inconsistency in the scientific review. To deal with these problems, an ethical review committee should maintain a large roster of potential consultants and should make sure that they receive appropriate education and training in human research protections.
Author Manuscript
A second approach is to create different ethical review committees that specialize in different types of research. For example, the University of North Carolina at Chapel Hill has a biomedical IRB, a social/behavioral IRB, and a public health/nursing IRB. Some of its biomedical IRBs focus on specific areas of biomedical research, such as oncology or dentistry.[8] There are many advantages to this approach. First, it can enhance the quality of ethical and scientific review by increasing expertise in a particular area. Second, it can increase consistency and efficiency within particular committees because the members become accustomed to reviewing similar protocols. However, this approach also has some drawbacks. First, an institution may lack the personnel or resources to operate a number of
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 3
Author Manuscript
different ethical review committees. Second, there may be some inconsistency across different committees, due to differences in membership and focus. Third, even a special committee may sometimes lack the expertise needed to review particular protocols. To address these problems, institutions must provide these committees with adequate personnel and resources, they should ensure that committee members receive proper education and training, and they should maintain a list of consultants who can participate in the review on an ad hoc basis.
Author Manuscript Author Manuscript
A third approach is to use a scientific review committee(s) to examine scientific issues.[7] Universities, colleges, pharmaceutical companies, research organizations have formed scientific review committees for research with human subjects. New protocols are submitted to the scientific committee(s) for review before they go to the ethical review committee. In some instances, the scientific issues may be reviewed by more than one committee before the ethical issues are reviewed. Scientific review committees can improve the review process by providing the ethical review committee with expert assessment of the scientific issues involved in the research. This can save the ethical review committee time and effort and improve the quality and efficiency of review. However, this approach also has some difficulties. First, an institution may lack the personnel or resources to staff scientific review committees. Second, scientific review committees may need to use consultants from time-totime, when they lack the expertise needed to review a protocol. Third, scientific review committees may focus on technical, scientific issues that are not relevant to the ethical issues, and therefore fail to answer the questions that the ethical review committees need to have addressed. To deal with these potential problems with scientific review, organizations that use scientific review committees should ensure that these committees have adequate personnel and resources (including access to consultants, as needed). Organizations should also provide scientific review committee members with education and guidance concerning human subjects review process, so that committee members can provide ethical review committee with the information and advice they require for ethical review. Scientific review committees should address specific questions that ethical review committees need to have answered for review of the protocol. A scientific review committee should answer the following questions: 1.
Is the protocol well-designed? a.
Does the protocol have a clear hypothesis, aims, or objectives?
b. Will the methods, tests, and procedures help to achieve the aims and objectives or prove the hypothesis? c.
Does the protocol minimize potential bias?
Author Manuscript
d. Are the endpoints and variables appropriate and well-defined?
2.
e.
Does the protocol have sufficient statistical power?
f.
Are they any ways to improve the design?
Are the investigators qualified to conduct this research?
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 4
Author Manuscript Author Manuscript
3.
Have the investigators performed an adequate survey of the background literature, including previous animal or human studies related to this research?
4.
Does the topic of the study have scientific or social value? Will the research contribute to basic science or clinical medicine?
5.
Does the protocol minimize risks to research subjects? Would any changes in the protocol’s procedures, tests, or methods help to minimize risks?
6.
Is the study population appropriate for the research questions the protocol addresses?
7.
Are the inclusion/exclusion criteria appropriate? Do they protect research subjects from risk?
8.
Are there adequate procedures in place for data and safety monitoring?
9.
Is the research feasible? Are there any practical problems with implementing the protocol or recruiting subjects?
Author Manuscript
Organizations conducting or sponsoring research with human subjects can decide which of these three different approaches to follow for scientific review. Since each approach has strengths and weaknesses, the best strategy for reviewing scientific issues in human research would be to combine the approaches. For example, university could sponsor a number of different specialized ethical and scientific review committees, and it could employ expert consultants, as needed. The university could ensure that all employees, volunteers, or consultants who review human research have appropriate training and guidance. When an organization does not have the personnel and resources needed to implement this hybrid approach, it should take appropriate measures to maintain and improve the quality of scientific review of human research, choosing a strategy that best fits with its personnel, resources, research focus, and organizational culture.
Acknowledgments This research was supported by the intramural program of the NIEHS/NIH. It does not represent the views of the NIEHS/NIH.
References
Author Manuscript
1. Emanuel E, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000; 283:2701– 11. [PubMed: 10819955] 2. Department of Health and Human Services. Protection of Human Subjects, 45 Code of Federal Regulation 46. 2005 3. Food and Drug Administration. Institutional Review Boards, 21 Code of Federal Regulations 56. 1998 4. Tri-Council Policy Statement, Ethical Conduct of Research Involving Humans . (http:// www.pre.ethics.gc.ca/english/pdf/TCPS%20October%202005_E.pdf) 5. European Parliament and Council. Directive 2001/20/EC. (http://europa.eu.int/eurlex/pri/en/oj/dat/ 2001/l_121/l_12120010501en00340044.pdf) 6. Amdur, R.; Bankert, E. Institutional Review Board Member Handbook. 2. Boston: Jones and Bartlett; 2005.
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 5
Author Manuscript
7. Amdur, R. Evaluating study design and quality. In: Amdur, R.; Bankert, E., editors. Institutional Review Board Management and Function. Boston: Jones and Bartlett; 2002. p. 152-154. 8. University of North Carolina. Office of Human Research Ethics. (http://ohre.unc.edu/index.php)
Author Manuscript Author Manuscript Author Manuscript Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23. Published in final edited form as: Good Clin Pract J. 2007 April ; 14(4): 13–15.
Three Steps to Ensure Good Science and Safe Research David B. Resnik, JD, PhD Bioethicist and IRB Vice Chair, NIEHS/NIH, Box 12233, Mail Drop NH06, Research Triangle Park, NC, 27709. Phone: 919 541 5658. Fax: 919 541 3659. Email: [email protected]
Abstract Author Manuscript
This article examines the strengths and weaknesses of three different approaches to reviewing scientific issues related to research with human subjects, and argues that the best strategy is one that combines all three approaches.
Author Manuscript
One of the essential ethical requirements for research with human subjects is that it be scientifically well-designed. Poorly designed research can result in several different ethical problems, including exposing human subjects to risks unnecessarily, undermining the consent process, compromising confidentiality, adversely affecting the benefit/risk ratio, and unfair selection of subjects.[1] Several ethics codes, including the Nuremberg Code, the World Medical Association’s Declaration of Helsinki, the Council for the International Organization of Medical Sciences (CIOMS) Guidelines, and the International Conference on Harmonization’s Good Clinical Practice (GCP) Guidelines, address the importance of sound scientific design in research with human subjects. Though the U.S. federal research regulations do not explicitly state that research must be well-designed, they imply that it should be, since poorly designed research may fail to meet the requirement that risks be minimized or that risks be reasonable in relation to benefits.[2],[3] Canada’s Tri-Council Policy Statement recommends that research protocols conform to scholarly standards, and the European Union’s Clinical Trial Directive states that clinical research should follow GCP guidelines.[4],[5]
Author Manuscript
While various regulations and codes spell out the practical details of ethical review of human research, they have little to say about the scientific review process. This lack of regulatory guidance concerning scientific review creates many practical dilemmas for organizations conducting or sponsoring research with human subjects. Who should review that the scientific issues related to research with human subjects? What qualifies someone to be a reviewer? What policies and procedures should apply to the review process? What are the scientific standards reviewers should use to assess the design of a research protocol? How should reviewers be educated and trained? How many reviewers are needed? When should reviews take place? Can reviews be appealed? Unfortunately, there is not sufficient space in this article to address all of these important issues concerning scientific review. Instead, the article will provide an overview of three approaches to reviewing the scientific issues that can arise in research with human subjects. The first approach is for the committee that reviews the ethical issues related to the research, such as an Institutional Review Board (IRB), Research Ethics Board (REB), or Research Ethics Committee (REC), to also review the scientific issues. Most ethics regulations and
Resnik
Page 2
Author Manuscript
guidelines require or recommend that ethical review committees have sufficient expertise to assess scientific issues.[1] For example, the U.S. federal research regulations require that IRBs be composed of scientists and non-scientists.[2],[3] Although the regulations do not define the term “scientist,” it is reasonable to assume that it includes people from the biomedical, physical, or social sciences, as well as clinicians, mathematicians, or engineers. [6] Committee members can apply their expertise to protocols they are qualified to review. For example, a social scientist on an ethical review committee can assess a social science protocol, a pediatrician can review a protocol involving children in medical research, and so on.
Author Manuscript
There are at least two advantages to this approach. First, it conserves resources, because there is no separate committee to review scientific issues. Second, it makes it easy to integrate the discussion of scientific and ethical issues because scientific review occurs in the context of ethics review. However, there is a major disadvantage of this approach: the committee may lack the expertise needed for the scientific review. For example, if an IRB has two physician members, none of whom are oncologists, it may lack the expertise to review a protocol involving a new cancer drug. If an IRB has no social scientists as members, it may lack the expertise to review a protocol that uses survey research methods. One way for a committee to overcome this problem is to expand its roster to include additional members with scientific expertise. Expanding the roster is not always the best way to solve this problem, however, because the committee may need to become very large (i.e. twenty members or more) in order to review the variety of scientific issues that may arise. If a committee is too large, it may have difficulty achieving a quorum, and the reviews may become inconsistent as different members attend different meetings.
Author Manuscript
To enhance its expertise, an ethical review committee could use outside consultants from time-to-time to review protocols.[7] Although the consultants would not vote protocols (only members can vote), they could provide information and advice to the committee. For example, in the cancer drug trial example discussed above, the IRB could ask an oncologist for his or her expert opinion. While this second approach helps to overcome the problem of lack of scientific expertise, it has some other shortcomings. First, it may be difficult (and awkward) to obtain the services of experts on an ad hoc basis, under the pressure of deadlines. Second, if the committee uses different reviewers each time, there may be some inconsistency in the scientific review. To deal with these problems, an ethical review committee should maintain a large roster of potential consultants and should make sure that they receive appropriate education and training in human research protections.
Author Manuscript
A second approach is to create different ethical review committees that specialize in different types of research. For example, the University of North Carolina at Chapel Hill has a biomedical IRB, a social/behavioral IRB, and a public health/nursing IRB. Some of its biomedical IRBs focus on specific areas of biomedical research, such as oncology or dentistry.[8] There are many advantages to this approach. First, it can enhance the quality of ethical and scientific review by increasing expertise in a particular area. Second, it can increase consistency and efficiency within particular committees because the members become accustomed to reviewing similar protocols. However, this approach also has some drawbacks. First, an institution may lack the personnel or resources to operate a number of
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 3
Author Manuscript
different ethical review committees. Second, there may be some inconsistency across different committees, due to differences in membership and focus. Third, even a special committee may sometimes lack the expertise needed to review particular protocols. To address these problems, institutions must provide these committees with adequate personnel and resources, they should ensure that committee members receive proper education and training, and they should maintain a list of consultants who can participate in the review on an ad hoc basis.
Author Manuscript Author Manuscript
A third approach is to use a scientific review committee(s) to examine scientific issues.[7] Universities, colleges, pharmaceutical companies, research organizations have formed scientific review committees for research with human subjects. New protocols are submitted to the scientific committee(s) for review before they go to the ethical review committee. In some instances, the scientific issues may be reviewed by more than one committee before the ethical issues are reviewed. Scientific review committees can improve the review process by providing the ethical review committee with expert assessment of the scientific issues involved in the research. This can save the ethical review committee time and effort and improve the quality and efficiency of review. However, this approach also has some difficulties. First, an institution may lack the personnel or resources to staff scientific review committees. Second, scientific review committees may need to use consultants from time-totime, when they lack the expertise needed to review a protocol. Third, scientific review committees may focus on technical, scientific issues that are not relevant to the ethical issues, and therefore fail to answer the questions that the ethical review committees need to have addressed. To deal with these potential problems with scientific review, organizations that use scientific review committees should ensure that these committees have adequate personnel and resources (including access to consultants, as needed). Organizations should also provide scientific review committee members with education and guidance concerning human subjects review process, so that committee members can provide ethical review committee with the information and advice they require for ethical review. Scientific review committees should address specific questions that ethical review committees need to have answered for review of the protocol. A scientific review committee should answer the following questions: 1.
Is the protocol well-designed? a.
Does the protocol have a clear hypothesis, aims, or objectives?
b. Will the methods, tests, and procedures help to achieve the aims and objectives or prove the hypothesis? c.
Does the protocol minimize potential bias?
Author Manuscript
d. Are the endpoints and variables appropriate and well-defined?
2.
e.
Does the protocol have sufficient statistical power?
f.
Are they any ways to improve the design?
Are the investigators qualified to conduct this research?
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 4
Author Manuscript Author Manuscript
3.
Have the investigators performed an adequate survey of the background literature, including previous animal or human studies related to this research?
4.
Does the topic of the study have scientific or social value? Will the research contribute to basic science or clinical medicine?
5.
Does the protocol minimize risks to research subjects? Would any changes in the protocol’s procedures, tests, or methods help to minimize risks?
6.
Is the study population appropriate for the research questions the protocol addresses?
7.
Are the inclusion/exclusion criteria appropriate? Do they protect research subjects from risk?
8.
Are there adequate procedures in place for data and safety monitoring?
9.
Is the research feasible? Are there any practical problems with implementing the protocol or recruiting subjects?
Author Manuscript
Organizations conducting or sponsoring research with human subjects can decide which of these three different approaches to follow for scientific review. Since each approach has strengths and weaknesses, the best strategy for reviewing scientific issues in human research would be to combine the approaches. For example, university could sponsor a number of different specialized ethical and scientific review committees, and it could employ expert consultants, as needed. The university could ensure that all employees, volunteers, or consultants who review human research have appropriate training and guidance. When an organization does not have the personnel and resources needed to implement this hybrid approach, it should take appropriate measures to maintain and improve the quality of scientific review of human research, choosing a strategy that best fits with its personnel, resources, research focus, and organizational culture.
Acknowledgments This research was supported by the intramural program of the NIEHS/NIH. It does not represent the views of the NIEHS/NIH.
References
Author Manuscript
1. Emanuel E, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000; 283:2701– 11. [PubMed: 10819955] 2. Department of Health and Human Services. Protection of Human Subjects, 45 Code of Federal Regulation 46. 2005 3. Food and Drug Administration. Institutional Review Boards, 21 Code of Federal Regulations 56. 1998 4. Tri-Council Policy Statement, Ethical Conduct of Research Involving Humans . (http:// www.pre.ethics.gc.ca/english/pdf/TCPS%20October%202005_E.pdf) 5. European Parliament and Council. Directive 2001/20/EC. (http://europa.eu.int/eurlex/pri/en/oj/dat/ 2001/l_121/l_12120010501en00340044.pdf) 6. Amdur, R.; Bankert, E. Institutional Review Board Member Handbook. 2. Boston: Jones and Bartlett; 2005.
Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
Resnik
Page 5
Author Manuscript
7. Amdur, R. Evaluating study design and quality. In: Amdur, R.; Bankert, E., editors. Institutional Review Board Management and Function. Boston: Jones and Bartlett; 2002. p. 152-154. 8. University of North Carolina. Office of Human Research Ethics. (http://ohre.unc.edu/index.php)
Author Manuscript Author Manuscript Author Manuscript Good Clin Pract J. Author manuscript; available in PMC 2016 March 23.
