Building Interdisciplinary Research Models Through Interactive Education Amanda J. Hessels, Ph.D., R.N.1, Brian Robinson, Ph.D.2, Michael O’Rourke, Ph.D.2, Melissa D. Begg, Sc.D.3, and Elaine L. Larson, Ph.D., R.N.1 Abstract
Background: Critical interdisciplinary research skills include effective communication with diverse disciplines and cultivating collaborative relationships. Acquiring these skills during graduate education may foster future interdisciplinary research quality and productivity. Objective: The project aim was to develop and evaluate an interactive Toolbox workshop approach within an interprofessional graduate level course to enhance student learning and skill in interdisciplinary research. We sought to examine the student experience of integrating the Toolbox workshop in modular format over the duration of a 14-week course. Methods: The Toolbox Health Sciences Instrument includes six modules that were introduced in a 110-minute dialogue session during the first class and then integrated into the course in a series of six individual workshops in three phases over the course of the semester. Results: Seventeen students participated; the majority were nursing students. Three measures were used to assess project outcomes: pre–post intervention Toolbox survey, competency self-assessment, and a postcourse survey. All measures indicated the objectives were met by a change in survey responses, improved competencies, and favorable experience of the Toolbox modular intervention. Conclusion: Our experience indicates that incorporating this Toolbox modular approach into research curricula can enhance individual level scientific capacity, future interdisciplinary research project success, and ultimately impact on practice and policy. Clin Trans Sci 2015; Volume 8: 793–799
Keywords: interdisciplinary research, graduate education, methodology
Introduction
Team science skills are increasingly important for researchers from all health-related disciplines to assure that knowledge generated and subsequently translated is comprehensive and addresses the complexity of modern healthcare issues. A hallmark of interdisciplinary team science is the incorporation of diverse stakeholder perspectives through effective dialogue and the formation of trusting and collaborative relationships.1 Development of these skills in novice researchers will foster the generation of critical research, capitalizing on diverse epistemological perspectives. Hence, the aim of this project was to evaluate the effectiveness of a novel interactive Toolbox approach to interprofessional graduate education. Methods
Course description With initial funding through the NIH Roadmap for a Center for Interdisciplinary Research to Prevent Infections, we developed and implemented a graduate level course offered through Columbia University, “Building Interdisciplinary Research Models,” cotaught since its inception in 2008 by professors from the schools of nursing and public health. The course was carefully conceived and planned over a period of several years to assure that it was responsive to the issues, concerns, and recommendations raised by the National Academy of Sciences publication “Facilitating Interdisciplinary Research.”2 Development of the course and challenges faced have been fully described elsewhere.3 Important components of the course development included the formation of a working group to develop a shared definition of interdisciplinary research4 and the conduct of a Delphi study of
national interdisciplinary experts to articulate the competencies essential to interprofessional collaboration.5 The course is mandatory for all nursing PhD students and preand postdoctoral fellows in a T32 training grant (T32NR013454), and is available as an elective for other graduate students at the university. To date, the course has been given eight times, and has been well received with approximately 20 students enrolled annually, including physicians (primarily residents and fellows enrolled in public health degree programs), nurses enrolled in a PhD program, masters and doctoral public health trainees, and other students from a variety of graduate programs, such as bioethics, nutrition, social work, and computational biology. The course includes didactic content describing the interdisciplinary research process, competencies essential to the conduct of interdisciplinary research, and findings from studies regarding barriers and facilitators of interdisciplinary research.3 A “Breaking the Barriers” series of presentations and interviews by interdisciplinary researchers is interspersed with the didactic content. Each researcher provides an overview of an interdisciplinary project, discusses how their team was selected and why, and describes barriers and facilitators to the success of their work. These presentations are illustrative of the challenges encountered in specific aspects of interdisciplinary scholarship, and include one or more research critiques and structured interviews presented by students as assigned. While the course has been a success and student evaluations have been mostly positive, students have also expressed the need for more interactive, real-life experiences with actual practice in developing and nurturing a collaborative, interdisciplinary research team. To address this need, specific research competency
School of Nursing, Columbia University, New York, New York, USA; 2Department of Philosophy, Michigan State University, East Lansing, Michigan, USA; 3Office of the Provost, Columbia University, New York, New York, USA. 1
Correspondence: A. Hessels ([email protected]) DOI: 10.1111/cts.12354
WWW.CTSJOURNAL.COM
VOLUME 8 • ISSUE 6
793
Hessels et al. BIRM n
goals for students were developed to: (1) engage in discussion with scholars from other disciplines to gain perspectives, (2) translate research from a disciplinary area into language meaningful to an interdisciplinary team, (3) articulate theories of disciplines other than their own through reading and discussion of issues with members of related disciplines, and (4) collaborate respectfully, equitably, and regularly with colleagues from other disciplines to develop sustainable interdisciplinary research teams. Incorporating workshops and modules into a curriculum has been suggested as a core strategy to enhance the individual and collective capacity for scientific collaboration.6,7 In previous research, the Toolbox approach8 has been demonstrated to be successful at enhancing communication and collaboration in interdisciplinary research.8–10 We therefore adapted this method and integrated it into our course structure. Because we planned to conduct formal evaluations of the curriculum, we obtained institutional review board approval from the study institution. Toolbox workshops Prior to the first class session, the developers of the Toolbox workshops provided faculty training. They then conducted and facilitated a 110-minute introductory dialogue session during the first class, which covered all six modules of the Toolbox Health Sciences Instrument: (1) motivation, (2) confirmation, (3) objectivity, (4) values, (5) reductionism-emergence, and (6) methodology.5 Each module consists of a core question that announces its theme and a series of statements (“prompts”) designed to stimulate discussion on that theme (listed in Table 1). Participants rated their level of agreement or disagreement with each statement on a 5-point Likert scale and then discussed their responses together. The course faculty participated in this session in a train-the-trainer role developing onsite skills. Throughout the semester the individual workshop modules were then incorporated into the course by the course faculty in a series of six modular workshops. Four interdisciplinary groups of four to five students were formed and 20 minutes per class was dedicated to each modular group workshop session. After rating their agreement or disagreement with each prompt in that week’s module, students discussed their responses as a group. Each group then generated new prompts in their own vernacular aimed to address the core question in that module. This was immediately followed by an instructor-led, structured dialogue to perform a deep, rapid dive into the content in consort with principles of adult learning, available time, and group energy level.8 This method was designed to leverage the modular nature of the Toolbox and foster student engagement and active learning. The course instructors regularly consulted with the Toolbox developers to assure congruence of in-class methodological interventions with the overarching Toolbox approach. These module interventions were implemented in three phases over the course of the semester: weekly in-class single modules, team feedback, and reflective self-directed multimodules. Module phase 1 The first three sessions used weekly in-class single modules to stimulate engagement and interaction, with each group tasked to produce a tangible, module-related product at the end of the session. The single modules used during these sessions were Motivation, Research Approaches, and Methodology. This activity was directed to motivate discussion toward achieving a more precise understanding of the issue raised by the core question of each module. These sessions were intentionally designed to 794
VOLUME 8 • ISSUE 6
allow students to create prompts in their technical vernacular that were not already included and that they perceived as best addressing the core questions. The expected value of the exercise was the immersive experience in creating the prompts, not the created prompts per se. An example of how this approach was operationalized using the Motivation module follows. Part 1 (8 minutes) Each group member completed the module (e.g., Motivation) by rating their level of agreement or disagreement on the 5-point Likert scale with each prompt. Each group then tallied responses and discussed the two prompts with the fewest high agreement and fewest low agreement ratings. If there were competing rankings for prompts, the group selected the prompt with the lowest aggregate score and discussed for 3 minutes. Part 2 (3 minutes) Building on dialogue from Part 1, each group created a prompt that elicited thoughtful response to the core question “What is your primary motivation for conducting research” that was not addressed by prompts in the existing instrument (4 minutes). The prompt could be nonspecific and/or have ambiguous terminology to generate discussion among interdisciplinary researchers.9 Part 3 (4 minutes) Each group passed their developed prompt to the next group. Members of that group then discussed the prompt and assessed whether it adequately stimulated dialogue aimed at the core question. Part 4 (5 minutes) A spokesperson for each group presented the team’s assessment of proposed prompt and final recommendation and rationale to retain or reject the prompt. Module phase 2 Following the first three modular Workshops, each student received a group profile created for their working group, which highlighted the key points of agreement, disagreement, and areas for further discussion. These profiles were based on an aggregated group level picture recovered from an analysis of the initial Workshop dialogue. These group profiles included specific recommendations of prompts to use as a starting point for further group discussion. In phase 2, students were advised to read their group profile and initiate discussion beginning with these recommended prompts; they were also encouraged to use the profile in group activities for the remainder of the semester. Module phase 3 The reflective self-directed multimodules phase was designed to enhance sustainability of these newly developed collaborative research team skills. In order to simultaneously capitalize on student engagement in the first half of the module Workshops that were the focus of phase 1, and also recognize the importance of sufficient time to reflect on and incorporate newly developed thoughts into enhanced skills, students in each group used collective decision making to set a pace through the next three modules. Because it is responsive to participant energy level, willingness to dialogue, and availability, this method aligns with Toolbox workshop best practices;8 further, it is an important adaptation to learners’ needs and styles that keeps the process dynamic. WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Domain and items
Predialogue mean (SD)
Postdialogue mean (SD)
Pre–post delta mean*
1. There are strict requirements for determining when empirical data confirm a tested hypothesis.
4.05 (0.89)
3.71 (0.96)
0.35
2. Validation of evidence requires replication.
3.74 (1.07)
3.76 (1.11)
−0.03
3. Unreplicated results can be validated if confirmed by a combination of several different methods.
3.2 (1.08)
4.0 (0.87)
−0.78
4. Standards of confirmation depend on the use to which the research will be put.
3.39 (1.21)
3.38 (0.86)
0.01
5. One way to confirm research results is to establish their usefulness.
3.06 (1.18)
2.76 (1.11)
0.29
6. Efficacy in clinical trials is the best indicator of effectiveness in practice.
1.95 (1.02)
2.41 (1.29)
−0.46
3.4 (1.07)
3.59 (1.09)
−0.19
3.53 (1.09)
3.63 (1.22)
−0.10
Confirmation core question: What types of evidentiary support are required for knowledge?
7. C linical practice should always be based on valid scientific information that has been replicated. 8. Application of scientific information is useful even though it is uncertain. Methodology core question: What are the most important considerations in study design? 1. Scientific research must be hypothesis driven.
3.47 (1.27)
3.24 (1.21)
0.23
2. In order for biomedical research to be valid, it must be quantitative.
1.95 (1.07)
2.29 (1.45)
−0.34
3. Controls in clinical research should be held to the same standard as they are in basic research.
3.17 (1.12)
3.19 (1.13)
−0.02
4. Scientific research can be valid without being experimental.
4.40 (0.58)
4.35 (0.68)
0.05
5. Techniques for managing uncertainty are essentially similar in basic and clinical research.
2.79 (1.15)
3.0 (1.15)
−0.21
1. My research is driven primarily by intellectual curiosity.
3.95 (1.10)
3.39 (1.16)
0.56
2. My research is driven primarily by a desire to contribute directly to the health and wellbeing of our society.
4.79 (0.41)
4.61 (0.49)
0.18
3. Understanding physiological systems is as important as developing treatments for disease.
4.15 (0.73)
4.17 (0.90)
−0.02
4. I am motivated by a desire to translate basic research more quickly into clinical practice.
4.0 (0.89)
3.78 (1.18)
0.22
5. I pursue research that I believe is important, regardless of whether or not it is likely to garner grant support.
3.94 (0.87)
3.5 (1.12)
0.44
6. Application without basic understanding is dangerous
4.42 (0.82)
4.2 (0.79)
0.20
7. I am in academia because I want to do research.
4.0 (1.05)
3.89 (1.15)
0.11
Motivation core question: What is your primary motivation for conducting research?
Reductionism core question: Can the world under investigation be reduced to independent elements for study? 1. The world under investigation is not fully explicable as the assembly of its individual components.
4. 0 (0.88)
4.11 (0.87)
−0.11
2. My research typically isolates the behavior of individual components of a system.
2.88 (1.18)
3.06 (1.14)
−0.18
3. Developing effective treatments depends equally on understanding the biochemical process of disease and the social factors that influence behavior.
4.40 (0.80)
3.61 (1.38)
0.79
4. Clinical research investigates systems too complex to be reduced completely to control and experimental variables.
3.21 (1.24)
3.5 (1.07)
−0.29
5. Diseases are merely states of patients’ biophysical systems.
2.10 (1.09)
2.67 (1.41)
−0.57
Research approaches core question: What things need to be taken into account in identifying a research problem? 1. Social and behavioral science should play a central role in translational research.
4.10 (0.9)
4.18 (1.04)
−0.08
2. Development of questions should always be a collaboration between basic scientists and clinical researchers.
2.95 (1.10)
2.94 (1.30)
0.01
3. The most important consideration in developing a question should be how the results will feed into the translational process.
3.11 (0.99)
3.06 (1.26)
0.05
4. The types of questions we ask should be constrained by the level of certainty we can expect in the answer.
2.15 (1.06)
2.65 (1.13)
−0.50
Table 1. Continued.
WWW.CTSJOURNAL.COM
VOLUME 8 • ISSUE 6
795
Hessels et al. BIRM n
Domain and items
Predialogue mean (SD)
Postdialogue mean (SD)
Pre–post delta mean*
1. Incorporating one's personal perspective in framing a research question always introduces bias.
2.89 (1.17)
2.78 (1.03)
0.11
2. Objective scientific research is not possible.
2.63 (1.27)
2.83 (1.12)
−0.20
3. Determining what constitutes acceptable validation of research data is a value issue.
3.26 (0.91)
3.18 (1.10)
0.08
4. Biomedical researchers should engage in advocacy related to their research.
3.65 (1.13)
3.22 (1.08)
0.43
5. Potential impacts should always influence the course of scientific research.
3.65 (1.11)
3.5 (0.90)
0.15
Values core question: Do values have a legitimate role in scientific research?
6. Sponsored research is biased research.
2.79 (1.0)
2.61 (1.21)
0.18
7. S ociocultural and political values should not have a role in determining the course of biomedical research.
2.30 (1.0)
2.67 (1.25)
−0.37
5-point Likert scale where 1 = Disagree, 5 = Agree. *Negative value indicates direction from disagree to agree.
Table 1. The health sciences toolbox questionnaire items and pre- and postintervention ratings (N = 17).
The four groups collaboratively decided to work among themselves on the development of the remaining three module prompts and collectively present, dialogue, and evaluate these prompts in one session toward the end of the course. Over three consecutive weeks students reflected independently and in groups on the remaining three modules (Values, Confirmation, and Reductionism) and group profiles. This work was facilitated by weekly in-class check-ins to ascertain any barriers, questions, or concerns. Groups dialogued in class as well as outside of class and used email and text messaging to facilitate a learning environment that was collaborative and capitalized on the social nature of learning while fostering socialization into the role of interdisciplinary researcher. Evaluation To assess project outcomes, three measures were used. First, a 36item pre- and postintervention survey of the six Workshop domains and related probes on the practice of team science (Table 1) was administered during the first and last class.9 One indication of a successful Workshop is a shift in responses, suggesting thoughtful reflection supported by the dialogue. This measure provides insight into patterns of responses and change over time in conjunction with other evaluative measures.8 Second, a self-assessment tool to measure students’ perceived levels of competence with identified domains of interdisciplinary collaboration, such as advocacy for interdisciplinary research, interdisciplinary interactions and synthesis of research, and theory of different disciplines was administered pre- and postcourse.2 Third, at the end of the course an anonymous survey was administered electronically using Qualtrics software (Qualtrics, Provo, UT, USA) to assess student experience of the Toolbox in its entirety. The survey included 18 items rated using a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). Five open-ended questions were also posed to assess the impact of the Toolbox on personal insight, professional development, and changes in philosophical views, as well as impressions of the format of the modular intervention. It was sent following the last class two times to all students, 2 weeks apart. Results
Seventeen students participated in the project: 14 doctoral nursing students and 3 graduate public health, epidemiology, 796
VOLUME 8 • ISSUE 6
and biostatistics students. All completed the semester long course and all pre- and post-Workshop evaluation measures. Pre- and postintervention survey A shift in the mean scores of all items was evident, as shown in Table 1. The greatest absolute differences in mean ratings were seen in the following prompts: Reductionism 3, Confirmation 3, Reductionism 5, Motivation 1, and Research Approaches 4. The majority (28/36) of pre- and postintervention item mean ratings remained on the same side of the scale (e.g., disagree = 1 or 2, neutral = 3, or agree = 4 or 5) indicating at most minor changes. Substantive changes, defined as mean ratings changing from one of the three categories to another (e.g., agree to disagree, uncertain to agree or disagree, or agree or disagree to uncertain), were noted in eight prompts. Six of these indicated a tendency toward the neutral response, including four from agree (Motivation 4, Motivation 7, Confirmation 1, and Reductionism 3) and two from disagree (Methodology 5 and Reductionism 2). The remaining two prompt ratings moved from neutral, one each to the agree (Confirmation 3) and disagree (Confirmation 5) sides of the scale. There was no group level reversal in opinion from disagree to agree or agree to disagree. Self-assessment Overall, student ratings of their level of confidence in their ability to perform 17 competencies improved during the course from precourse mean of 3.4 (standard deviation (SD): 0.7) to postcourse 3.9 (SD: 0.7) out of a possible 5 points (Table 2). The greatest changes were in the items of sharing research from their own discipline in language meaningful to interdisciplinary research teams, modifying work as a result of interdisciplinary interactions, integrating concepts and methods, disseminating interdisciplinary research, and presenting interdisciplinary research. The self-assessment competencies that changed the least were those that were rated very high (3.8–4.5) initially and included the ability to express respect for the perspectives of other disciplines, confidence to engage colleagues from other disciplines, or reading journals outside of own discipline. Postcourse survey Ten responses were received; students reported an overall positive course experience with outcomes consistent with the aims of the WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Predialogue modules mean (SD)
Postdialogue modules mean (SD)
Pre–post delta mean*
1. A dvocate interdisciplinary research in developing initiatives within a substantive area of study.
3.4 (1.2)
4.0 (1.0)
−0.6
2. E ngage colleagues from other disciplines to gain their perspectives on research problems.
3.8 (1.0)
3.8 (1.1)
0.0
3. Express respect for the perspectives of other disciplines.
4.5 (0.9)
4.3 (1.3)
0.2
4. Read journals outside of his or her discipline.
3.8 (1.0)
3.8 (1.1)
0.0
5. Interact in training exercises with scholars from other disciplines.
3.8 (0.9)
3.9 (1.0)
−0.1
6. Communicate regularly with scholars from multiple disciplines.
3.4 (1.0)
3.9 (0.9)
−0.4
7. Attend scholarly presentations by members of other disciplines.
3.6 (1.0)
4.1 (1.0)
−0.6
8. U se theories and methods of multiple disciplines in developing integrated theoretical and research frameworks.
3.1 (0.9)
3.6 (1.2)
−0.6
9. I ntegrate concepts and methods from multiple disciplines in designing interdisciplinary research protocols.
2.9 (1.1)
3.6 (1.1)
−0.7
10. Investigate hypotheses through interdisciplinary research.
3.4 (1.0)
3.7 (1.00
−0.3
11. Share research from his or her discipline in language meaningful to an interdisciplinary team.
3.4 (1.0)
4.2 (0.9)
−0.8
12. Collaborate respectfully and equitably with scholars from other disciplines to develop interdisciplinary research frameworks.
3.6 (1.1)
4.1 (1.0)
−0.4
13. Modify his or her own work or research agenda as a result of interactions with colleagues from fields other than his or her own.
3.3 (1.0)
4.1 (1.0)
−0.8
14. Draft funding proposals for interdisciplinary research programs in partnership with scholars from other disciplines.
2.9 (0.9)
3.4 (1.2)
−0.5
15. Disseminate interdisciplinary research results both within and outside his or her discipline.
3.1 (1.1)
3.8 (1.0)
−0.7
16. Author publications with scholars from other disciplines.
3.3 (0.9)
3.6 (1.2)
−0.4
17. Present interdisciplinary research at venues representing more than one discipline.
3.3 (1.0)
4.0 (0.8)
−0.7
Overall competency
3.4 (0.7)
3.9 (0.7)
−0.5
Competency
5-point Likert scale where 1 = Disagree, 5 = Agree. *Negative value indicates direction from disagree to agree. Scale ratings adapted from University of Illinois–Chicago School of Public Health Competency Assessment Tool, accessible at http://128.248.232.71/prepare/personal.asp.
Table 2. Interdisciplinary research competency self-assessment items and ratings (N = 10).
Toolbox intervention. Students responded in agreement that the Toolbox prompts and comments from others stimulated new thoughts about differing research worldviews and they felt free to present a view that differed from others in their group. In so doing, the Toolbox also helped their group identify some of their differing research worldviews and create mutual understanding, which they agreed was important for research productivity. The students overall, however, tended to disagree that they had done additional reading on the topics addressed in the workshop after the initial Toolbox workshop at the start of term, or discussed topics raised in the workshop with others not in the workshop or with other group members outside of class. In the survey, the students indicated that they gained important insights through the Toolbox experience, including: “Sometimes it’s shocking how differently people can interpret something that on the surface appears very straight forward,” and “It was interesting to hear other people’s opinions and world views.” The Toolbox promoted introspection as well: “It was interesting to see how strongly I hold onto my world views.” and “I think the Toolbox was effective at helping to clarify one’s own research perspective/worldview.” WWW.CTSJOURNAL.COM
A few students did not find value in the Toolbox modules implementation and made either skeptical or negative comments about an aspect of the Toolbox workshop such as: “I’m still not sure what the purpose of this Toolbox was. It seemed like our time could have more effectively be spent doing other things.” Comments were mixed on the need and benefit of additional time for the Toolbox intervention: “I thought we were given adequate time as is” to “an additional 15–20 minutes would have allowed us to further develop our conversation” or “No, they were more than enough time spent on the toolbox.” The relationship between the format and amount of time spent on the Toolbox and group composition surfaced in several comments: “I think that the toolbox is only useful when you truly have interdisciplinary groups. My group was primarily nurses and these discussions began to feel very one dimensional, especially over time.” VOLUME 8 • ISSUE 6
797
Hessels et al. BIRM n
and “The discussion was interesting and engaging. However, our group was not very interdisciplinary, so I think we got less out of it than we could have with a more diverse group.” An additional output from the workshop modules included the creation, discussion, and selection of the prompts generated by the students to facilitate discussion of the main Toolbox domains as described in the Methods. These prompts are presented in the Appendix.
Toolbox workshops places philosophical issues and methods front and center for much of the course, a reality is that might not be the same for every guest lecturer or student group. Changes in the mean ratings of these modules may also reflect the novice experience of students in research training. It may be at this developmental stage the ability to answer epistemological core “what” questions is more modifiable than the ability to answer metaphysical core “how” questions. Finally, as some student comments suggest, the approach used might be less effective in this largely monodisciplinary environment. Conclusion
Discussion
In 2012, The Center for Healthcare Research and Transformation conducted a symposium to identify barriers and solutions to increase the impact of health services researchers on both policy and practice.11 An underlying theme of key lessons identified was the importance of communication skills across varying worldviews, that is, that researchers must be able to speak the same vernacular as policymakers, elected officials, payers, purchasers, practice, and health system leaders.11 Integrating the Toolbox workshop modules into the course was a successful translational strategy toward achieving these goals. The necessary academic administrative support, infrastructure, and committed faculty from multiple disciplines were in place to support this project as a priority to improve interprofessional education.12 The Toolbox prompts generated dialogue and insight into central philosophical issues, as evidenced by the variance in responses pre- and postworkshop. Moreover, this is evident in the formation of philosophical commitments following the Workshops in that the number of “don’t know” responses decreased from 33 to 6. These findings indicate that that dialogue was constructive in cocreating philosophical interdisciplinary research underpinnings. The Toolbox workshop modules enhanced team building and team learning, already an expectation of the course, by focusing dialogue on core philosophical values that form the foundation for future interdisciplinary research work. This was evident in both interteam and intrateam communication tone, quality of outputs, and respectful and inclusive attitudes within and among groups indicating a high level of engagement.8 Engagement was further evident in the creation of new prompts, ensuing discussion and “voting” to retain or reject the prompt among course participants (Appendix). A rationale was presented by each group for the decision to reject or retain the prompt and discussed further by the class. This component of the activity, through ongoing sessions, provided the opportunity for all students to develop a desired skill and outcome of the Toolbox, namely, the individual and collaborative ability to explicitly articulate philosophical differences and commonalities.9 Notably, the substantive changes in the mean ratings of the groups occurred predominantly in the epistemological modules (Motivation, Confirmation, and Methodology); only one item from a metaphysical module (Reductionism) substantively shifted. This may reflect the overall congruence of these modules and core questions with content presented during the didactic sessions of the course. Though the design of integrating the
798
VOLUME 8 • ISSUE 6
The aims of this project were to develop and evaluate an interactive Toolbox workshop approach within a graduate level course to enhance student learning and skill in interdisciplinary research. This collaborative project successfully incorporated a structured template to guide students through introspective reflection, interactive dialogue, and explicit articulation of philosophical underpinnings of interdisciplinary research. Our experience indicates that incorporating this approach into research curricula can enhance individual level scientific capacity, future interdisciplinary research project success, and ultimately impact on practice and policy. Acknowledgments
Amanda Hessels was supported as a post-doctoral trainee by the National Institute of Nursing Research, National Institutes of Health (Training in Interdisciplinary Research to Prevent Infections, T32 NR013454). This publication was also supported in part by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant Number UL1 TR000040. References
1. Rosenfield PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Sci Med. 1992; 35: 1343–1357. 2. National Academy of Sciences. Facilitating Interdisciplinary Research. Washington, DC: National Academies Press; 2005. 3. Larson EL, Landers TF, Begg MD. Building interdisciplinary research models: a didactic course to prepare interdisciplinary scholars and faculty. Clin Transl Sci. 2011; 4: 38–41. 4. Aboelela SW, Larson E, Bakken S, Carrasquillo O, Formicola A, Glied SA, Haas J, Gebbie KM. Defining interdisciplinary research: conclusions from a critical review of the literature. Health Serv Res. 2007; 42: 329–346. 5. Gebbie KM, Meier BM, Bakken S, Carrasquillo O, Formicola A, Aboelela SW, Glied S, Larson E. Training for interdisciplinary health research: defining the required competencies. J Allied Health. 2008; 37: 65–70. 6. Eigenbrode S, O’Rourke M, Wulfhorst J, Althoff DM, Goldberg CS, Merrill K, Morse W, NielsenPincus M, Stephens J, Winowieci L, et al. Employing philosophical dialogue in collaborative science. BioScience. 2007; 57: 55–64 7. Stokols D, Hall KL, Taylor BK, Moser RP. The science of team science: overview of the field and introduction to the supplement. Am J Prev Med. 2008; 35(Suppl. 2): S77–S89. 8. O’Rourke M, Crowley S, Eigenbrode S, Wulfhorst J. Enhancing Communication and Collaboration in Interdisciplinary Research. Thousand Oaks, CA: Sage Publications; 2014. 9. Schnapp LM, Rotschy L, Hall TE, Crowley S, O’Rourke M. How to talk to strangers: facilitating knowledge sharing within translational health teams with the Toolbox dialogue method. Transl Behav Med. 2012; 2: 469–479. 10. O’Rourke M, Crowley S. Philosophical intervention and cross-disciplinary science: the story of the Toolbox Project. Synthese. 2013; 190: 1937–1954. 11. Kerr EA, Riba M, Udow-Phillips M. Helping health service researchers and policy makers speak the same language. Health Services Res. 2015; 50(1): 1–11. 12. Farrell K, Payne C, Heye M. Integrating interprofessional collaboration skills into the advanced practice registered nurse socialization process. J Profess Nurs. 31(1): 5–10.
WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Appendix: Health Science Toolbox Instrument Columbia University Developed Prompts
Motivation Core Question: What is your primary motivation for conducting research? 1. I pursue research for personal gain. 2. I would rather contribute a piece to the puzzle than the whole picture. 3. My research primarily aims to add knowledge to my own field. 4. The main reason for conducting research is to fill a gap in existing knowledge in my field. Values Core Question: Do values have a legitimate role in scientific research? 1. When a researcher personally cares about an issue he/she performs better research. 2. It is never fully possible to set aside personal biases during research. 3. It is difficult to work with people who have different values. Reductionism Core Question: Can the world under investigation be reduced to independent elements for study? 1. A holistic approach is the only way to capture the truth. 2. It is more important to study a system as a whole than its individual parts.
WWW.CTSJOURNAL.COM
Research Approaches Core Question: What things need to be taken into account in identifying a research problem? 1. The most important thing when formulating a research question is to address gaps in existing scientific knowledge. 2. Level of innovation is as important as level of certainty when constructing a research question. Methodology Core Question: What are the most important considerations in study design? 1. The most important consideration when choosing your study design is your hypothesis. 2. For every research question there is always one design which is best. 3. Funding should be the primary consideration when designing a study. 4. Study designs that generate both quantitative and qualitative contribute equally to an understanding of the truth. Confirmation Core Question: What types of evidentiary support are required for knowledge? 1. It is safe to apply findings of a study immediately following its completion. 2. Observational or retrospective evidence may be used alone in establishing confirmation.
VOLUME 8 • ISSUE 6
799
Background: Critical interdisciplinary research skills include effective communication with diverse disciplines and cultivating collaborative relationships. Acquiring these skills during graduate education may foster future interdisciplinary research quality and productivity. Objective: The project aim was to develop and evaluate an interactive Toolbox workshop approach within an interprofessional graduate level course to enhance student learning and skill in interdisciplinary research. We sought to examine the student experience of integrating the Toolbox workshop in modular format over the duration of a 14-week course. Methods: The Toolbox Health Sciences Instrument includes six modules that were introduced in a 110-minute dialogue session during the first class and then integrated into the course in a series of six individual workshops in three phases over the course of the semester. Results: Seventeen students participated; the majority were nursing students. Three measures were used to assess project outcomes: pre–post intervention Toolbox survey, competency self-assessment, and a postcourse survey. All measures indicated the objectives were met by a change in survey responses, improved competencies, and favorable experience of the Toolbox modular intervention. Conclusion: Our experience indicates that incorporating this Toolbox modular approach into research curricula can enhance individual level scientific capacity, future interdisciplinary research project success, and ultimately impact on practice and policy. Clin Trans Sci 2015; Volume 8: 793–799
Keywords: interdisciplinary research, graduate education, methodology
Introduction
Team science skills are increasingly important for researchers from all health-related disciplines to assure that knowledge generated and subsequently translated is comprehensive and addresses the complexity of modern healthcare issues. A hallmark of interdisciplinary team science is the incorporation of diverse stakeholder perspectives through effective dialogue and the formation of trusting and collaborative relationships.1 Development of these skills in novice researchers will foster the generation of critical research, capitalizing on diverse epistemological perspectives. Hence, the aim of this project was to evaluate the effectiveness of a novel interactive Toolbox approach to interprofessional graduate education. Methods
Course description With initial funding through the NIH Roadmap for a Center for Interdisciplinary Research to Prevent Infections, we developed and implemented a graduate level course offered through Columbia University, “Building Interdisciplinary Research Models,” cotaught since its inception in 2008 by professors from the schools of nursing and public health. The course was carefully conceived and planned over a period of several years to assure that it was responsive to the issues, concerns, and recommendations raised by the National Academy of Sciences publication “Facilitating Interdisciplinary Research.”2 Development of the course and challenges faced have been fully described elsewhere.3 Important components of the course development included the formation of a working group to develop a shared definition of interdisciplinary research4 and the conduct of a Delphi study of
national interdisciplinary experts to articulate the competencies essential to interprofessional collaboration.5 The course is mandatory for all nursing PhD students and preand postdoctoral fellows in a T32 training grant (T32NR013454), and is available as an elective for other graduate students at the university. To date, the course has been given eight times, and has been well received with approximately 20 students enrolled annually, including physicians (primarily residents and fellows enrolled in public health degree programs), nurses enrolled in a PhD program, masters and doctoral public health trainees, and other students from a variety of graduate programs, such as bioethics, nutrition, social work, and computational biology. The course includes didactic content describing the interdisciplinary research process, competencies essential to the conduct of interdisciplinary research, and findings from studies regarding barriers and facilitators of interdisciplinary research.3 A “Breaking the Barriers” series of presentations and interviews by interdisciplinary researchers is interspersed with the didactic content. Each researcher provides an overview of an interdisciplinary project, discusses how their team was selected and why, and describes barriers and facilitators to the success of their work. These presentations are illustrative of the challenges encountered in specific aspects of interdisciplinary scholarship, and include one or more research critiques and structured interviews presented by students as assigned. While the course has been a success and student evaluations have been mostly positive, students have also expressed the need for more interactive, real-life experiences with actual practice in developing and nurturing a collaborative, interdisciplinary research team. To address this need, specific research competency
School of Nursing, Columbia University, New York, New York, USA; 2Department of Philosophy, Michigan State University, East Lansing, Michigan, USA; 3Office of the Provost, Columbia University, New York, New York, USA. 1
Correspondence: A. Hessels ([email protected]) DOI: 10.1111/cts.12354
WWW.CTSJOURNAL.COM
VOLUME 8 • ISSUE 6
793
Hessels et al. BIRM n
goals for students were developed to: (1) engage in discussion with scholars from other disciplines to gain perspectives, (2) translate research from a disciplinary area into language meaningful to an interdisciplinary team, (3) articulate theories of disciplines other than their own through reading and discussion of issues with members of related disciplines, and (4) collaborate respectfully, equitably, and regularly with colleagues from other disciplines to develop sustainable interdisciplinary research teams. Incorporating workshops and modules into a curriculum has been suggested as a core strategy to enhance the individual and collective capacity for scientific collaboration.6,7 In previous research, the Toolbox approach8 has been demonstrated to be successful at enhancing communication and collaboration in interdisciplinary research.8–10 We therefore adapted this method and integrated it into our course structure. Because we planned to conduct formal evaluations of the curriculum, we obtained institutional review board approval from the study institution. Toolbox workshops Prior to the first class session, the developers of the Toolbox workshops provided faculty training. They then conducted and facilitated a 110-minute introductory dialogue session during the first class, which covered all six modules of the Toolbox Health Sciences Instrument: (1) motivation, (2) confirmation, (3) objectivity, (4) values, (5) reductionism-emergence, and (6) methodology.5 Each module consists of a core question that announces its theme and a series of statements (“prompts”) designed to stimulate discussion on that theme (listed in Table 1). Participants rated their level of agreement or disagreement with each statement on a 5-point Likert scale and then discussed their responses together. The course faculty participated in this session in a train-the-trainer role developing onsite skills. Throughout the semester the individual workshop modules were then incorporated into the course by the course faculty in a series of six modular workshops. Four interdisciplinary groups of four to five students were formed and 20 minutes per class was dedicated to each modular group workshop session. After rating their agreement or disagreement with each prompt in that week’s module, students discussed their responses as a group. Each group then generated new prompts in their own vernacular aimed to address the core question in that module. This was immediately followed by an instructor-led, structured dialogue to perform a deep, rapid dive into the content in consort with principles of adult learning, available time, and group energy level.8 This method was designed to leverage the modular nature of the Toolbox and foster student engagement and active learning. The course instructors regularly consulted with the Toolbox developers to assure congruence of in-class methodological interventions with the overarching Toolbox approach. These module interventions were implemented in three phases over the course of the semester: weekly in-class single modules, team feedback, and reflective self-directed multimodules. Module phase 1 The first three sessions used weekly in-class single modules to stimulate engagement and interaction, with each group tasked to produce a tangible, module-related product at the end of the session. The single modules used during these sessions were Motivation, Research Approaches, and Methodology. This activity was directed to motivate discussion toward achieving a more precise understanding of the issue raised by the core question of each module. These sessions were intentionally designed to 794
VOLUME 8 • ISSUE 6
allow students to create prompts in their technical vernacular that were not already included and that they perceived as best addressing the core questions. The expected value of the exercise was the immersive experience in creating the prompts, not the created prompts per se. An example of how this approach was operationalized using the Motivation module follows. Part 1 (8 minutes) Each group member completed the module (e.g., Motivation) by rating their level of agreement or disagreement on the 5-point Likert scale with each prompt. Each group then tallied responses and discussed the two prompts with the fewest high agreement and fewest low agreement ratings. If there were competing rankings for prompts, the group selected the prompt with the lowest aggregate score and discussed for 3 minutes. Part 2 (3 minutes) Building on dialogue from Part 1, each group created a prompt that elicited thoughtful response to the core question “What is your primary motivation for conducting research” that was not addressed by prompts in the existing instrument (4 minutes). The prompt could be nonspecific and/or have ambiguous terminology to generate discussion among interdisciplinary researchers.9 Part 3 (4 minutes) Each group passed their developed prompt to the next group. Members of that group then discussed the prompt and assessed whether it adequately stimulated dialogue aimed at the core question. Part 4 (5 minutes) A spokesperson for each group presented the team’s assessment of proposed prompt and final recommendation and rationale to retain or reject the prompt. Module phase 2 Following the first three modular Workshops, each student received a group profile created for their working group, which highlighted the key points of agreement, disagreement, and areas for further discussion. These profiles were based on an aggregated group level picture recovered from an analysis of the initial Workshop dialogue. These group profiles included specific recommendations of prompts to use as a starting point for further group discussion. In phase 2, students were advised to read their group profile and initiate discussion beginning with these recommended prompts; they were also encouraged to use the profile in group activities for the remainder of the semester. Module phase 3 The reflective self-directed multimodules phase was designed to enhance sustainability of these newly developed collaborative research team skills. In order to simultaneously capitalize on student engagement in the first half of the module Workshops that were the focus of phase 1, and also recognize the importance of sufficient time to reflect on and incorporate newly developed thoughts into enhanced skills, students in each group used collective decision making to set a pace through the next three modules. Because it is responsive to participant energy level, willingness to dialogue, and availability, this method aligns with Toolbox workshop best practices;8 further, it is an important adaptation to learners’ needs and styles that keeps the process dynamic. WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Domain and items
Predialogue mean (SD)
Postdialogue mean (SD)
Pre–post delta mean*
1. There are strict requirements for determining when empirical data confirm a tested hypothesis.
4.05 (0.89)
3.71 (0.96)
0.35
2. Validation of evidence requires replication.
3.74 (1.07)
3.76 (1.11)
−0.03
3. Unreplicated results can be validated if confirmed by a combination of several different methods.
3.2 (1.08)
4.0 (0.87)
−0.78
4. Standards of confirmation depend on the use to which the research will be put.
3.39 (1.21)
3.38 (0.86)
0.01
5. One way to confirm research results is to establish their usefulness.
3.06 (1.18)
2.76 (1.11)
0.29
6. Efficacy in clinical trials is the best indicator of effectiveness in practice.
1.95 (1.02)
2.41 (1.29)
−0.46
3.4 (1.07)
3.59 (1.09)
−0.19
3.53 (1.09)
3.63 (1.22)
−0.10
Confirmation core question: What types of evidentiary support are required for knowledge?
7. C linical practice should always be based on valid scientific information that has been replicated. 8. Application of scientific information is useful even though it is uncertain. Methodology core question: What are the most important considerations in study design? 1. Scientific research must be hypothesis driven.
3.47 (1.27)
3.24 (1.21)
0.23
2. In order for biomedical research to be valid, it must be quantitative.
1.95 (1.07)
2.29 (1.45)
−0.34
3. Controls in clinical research should be held to the same standard as they are in basic research.
3.17 (1.12)
3.19 (1.13)
−0.02
4. Scientific research can be valid without being experimental.
4.40 (0.58)
4.35 (0.68)
0.05
5. Techniques for managing uncertainty are essentially similar in basic and clinical research.
2.79 (1.15)
3.0 (1.15)
−0.21
1. My research is driven primarily by intellectual curiosity.
3.95 (1.10)
3.39 (1.16)
0.56
2. My research is driven primarily by a desire to contribute directly to the health and wellbeing of our society.
4.79 (0.41)
4.61 (0.49)
0.18
3. Understanding physiological systems is as important as developing treatments for disease.
4.15 (0.73)
4.17 (0.90)
−0.02
4. I am motivated by a desire to translate basic research more quickly into clinical practice.
4.0 (0.89)
3.78 (1.18)
0.22
5. I pursue research that I believe is important, regardless of whether or not it is likely to garner grant support.
3.94 (0.87)
3.5 (1.12)
0.44
6. Application without basic understanding is dangerous
4.42 (0.82)
4.2 (0.79)
0.20
7. I am in academia because I want to do research.
4.0 (1.05)
3.89 (1.15)
0.11
Motivation core question: What is your primary motivation for conducting research?
Reductionism core question: Can the world under investigation be reduced to independent elements for study? 1. The world under investigation is not fully explicable as the assembly of its individual components.
4. 0 (0.88)
4.11 (0.87)
−0.11
2. My research typically isolates the behavior of individual components of a system.
2.88 (1.18)
3.06 (1.14)
−0.18
3. Developing effective treatments depends equally on understanding the biochemical process of disease and the social factors that influence behavior.
4.40 (0.80)
3.61 (1.38)
0.79
4. Clinical research investigates systems too complex to be reduced completely to control and experimental variables.
3.21 (1.24)
3.5 (1.07)
−0.29
5. Diseases are merely states of patients’ biophysical systems.
2.10 (1.09)
2.67 (1.41)
−0.57
Research approaches core question: What things need to be taken into account in identifying a research problem? 1. Social and behavioral science should play a central role in translational research.
4.10 (0.9)
4.18 (1.04)
−0.08
2. Development of questions should always be a collaboration between basic scientists and clinical researchers.
2.95 (1.10)
2.94 (1.30)
0.01
3. The most important consideration in developing a question should be how the results will feed into the translational process.
3.11 (0.99)
3.06 (1.26)
0.05
4. The types of questions we ask should be constrained by the level of certainty we can expect in the answer.
2.15 (1.06)
2.65 (1.13)
−0.50
Table 1. Continued.
WWW.CTSJOURNAL.COM
VOLUME 8 • ISSUE 6
795
Hessels et al. BIRM n
Domain and items
Predialogue mean (SD)
Postdialogue mean (SD)
Pre–post delta mean*
1. Incorporating one's personal perspective in framing a research question always introduces bias.
2.89 (1.17)
2.78 (1.03)
0.11
2. Objective scientific research is not possible.
2.63 (1.27)
2.83 (1.12)
−0.20
3. Determining what constitutes acceptable validation of research data is a value issue.
3.26 (0.91)
3.18 (1.10)
0.08
4. Biomedical researchers should engage in advocacy related to their research.
3.65 (1.13)
3.22 (1.08)
0.43
5. Potential impacts should always influence the course of scientific research.
3.65 (1.11)
3.5 (0.90)
0.15
Values core question: Do values have a legitimate role in scientific research?
6. Sponsored research is biased research.
2.79 (1.0)
2.61 (1.21)
0.18
7. S ociocultural and political values should not have a role in determining the course of biomedical research.
2.30 (1.0)
2.67 (1.25)
−0.37
5-point Likert scale where 1 = Disagree, 5 = Agree. *Negative value indicates direction from disagree to agree.
Table 1. The health sciences toolbox questionnaire items and pre- and postintervention ratings (N = 17).
The four groups collaboratively decided to work among themselves on the development of the remaining three module prompts and collectively present, dialogue, and evaluate these prompts in one session toward the end of the course. Over three consecutive weeks students reflected independently and in groups on the remaining three modules (Values, Confirmation, and Reductionism) and group profiles. This work was facilitated by weekly in-class check-ins to ascertain any barriers, questions, or concerns. Groups dialogued in class as well as outside of class and used email and text messaging to facilitate a learning environment that was collaborative and capitalized on the social nature of learning while fostering socialization into the role of interdisciplinary researcher. Evaluation To assess project outcomes, three measures were used. First, a 36item pre- and postintervention survey of the six Workshop domains and related probes on the practice of team science (Table 1) was administered during the first and last class.9 One indication of a successful Workshop is a shift in responses, suggesting thoughtful reflection supported by the dialogue. This measure provides insight into patterns of responses and change over time in conjunction with other evaluative measures.8 Second, a self-assessment tool to measure students’ perceived levels of competence with identified domains of interdisciplinary collaboration, such as advocacy for interdisciplinary research, interdisciplinary interactions and synthesis of research, and theory of different disciplines was administered pre- and postcourse.2 Third, at the end of the course an anonymous survey was administered electronically using Qualtrics software (Qualtrics, Provo, UT, USA) to assess student experience of the Toolbox in its entirety. The survey included 18 items rated using a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). Five open-ended questions were also posed to assess the impact of the Toolbox on personal insight, professional development, and changes in philosophical views, as well as impressions of the format of the modular intervention. It was sent following the last class two times to all students, 2 weeks apart. Results
Seventeen students participated in the project: 14 doctoral nursing students and 3 graduate public health, epidemiology, 796
VOLUME 8 • ISSUE 6
and biostatistics students. All completed the semester long course and all pre- and post-Workshop evaluation measures. Pre- and postintervention survey A shift in the mean scores of all items was evident, as shown in Table 1. The greatest absolute differences in mean ratings were seen in the following prompts: Reductionism 3, Confirmation 3, Reductionism 5, Motivation 1, and Research Approaches 4. The majority (28/36) of pre- and postintervention item mean ratings remained on the same side of the scale (e.g., disagree = 1 or 2, neutral = 3, or agree = 4 or 5) indicating at most minor changes. Substantive changes, defined as mean ratings changing from one of the three categories to another (e.g., agree to disagree, uncertain to agree or disagree, or agree or disagree to uncertain), were noted in eight prompts. Six of these indicated a tendency toward the neutral response, including four from agree (Motivation 4, Motivation 7, Confirmation 1, and Reductionism 3) and two from disagree (Methodology 5 and Reductionism 2). The remaining two prompt ratings moved from neutral, one each to the agree (Confirmation 3) and disagree (Confirmation 5) sides of the scale. There was no group level reversal in opinion from disagree to agree or agree to disagree. Self-assessment Overall, student ratings of their level of confidence in their ability to perform 17 competencies improved during the course from precourse mean of 3.4 (standard deviation (SD): 0.7) to postcourse 3.9 (SD: 0.7) out of a possible 5 points (Table 2). The greatest changes were in the items of sharing research from their own discipline in language meaningful to interdisciplinary research teams, modifying work as a result of interdisciplinary interactions, integrating concepts and methods, disseminating interdisciplinary research, and presenting interdisciplinary research. The self-assessment competencies that changed the least were those that were rated very high (3.8–4.5) initially and included the ability to express respect for the perspectives of other disciplines, confidence to engage colleagues from other disciplines, or reading journals outside of own discipline. Postcourse survey Ten responses were received; students reported an overall positive course experience with outcomes consistent with the aims of the WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Predialogue modules mean (SD)
Postdialogue modules mean (SD)
Pre–post delta mean*
1. A dvocate interdisciplinary research in developing initiatives within a substantive area of study.
3.4 (1.2)
4.0 (1.0)
−0.6
2. E ngage colleagues from other disciplines to gain their perspectives on research problems.
3.8 (1.0)
3.8 (1.1)
0.0
3. Express respect for the perspectives of other disciplines.
4.5 (0.9)
4.3 (1.3)
0.2
4. Read journals outside of his or her discipline.
3.8 (1.0)
3.8 (1.1)
0.0
5. Interact in training exercises with scholars from other disciplines.
3.8 (0.9)
3.9 (1.0)
−0.1
6. Communicate regularly with scholars from multiple disciplines.
3.4 (1.0)
3.9 (0.9)
−0.4
7. Attend scholarly presentations by members of other disciplines.
3.6 (1.0)
4.1 (1.0)
−0.6
8. U se theories and methods of multiple disciplines in developing integrated theoretical and research frameworks.
3.1 (0.9)
3.6 (1.2)
−0.6
9. I ntegrate concepts and methods from multiple disciplines in designing interdisciplinary research protocols.
2.9 (1.1)
3.6 (1.1)
−0.7
10. Investigate hypotheses through interdisciplinary research.
3.4 (1.0)
3.7 (1.00
−0.3
11. Share research from his or her discipline in language meaningful to an interdisciplinary team.
3.4 (1.0)
4.2 (0.9)
−0.8
12. Collaborate respectfully and equitably with scholars from other disciplines to develop interdisciplinary research frameworks.
3.6 (1.1)
4.1 (1.0)
−0.4
13. Modify his or her own work or research agenda as a result of interactions with colleagues from fields other than his or her own.
3.3 (1.0)
4.1 (1.0)
−0.8
14. Draft funding proposals for interdisciplinary research programs in partnership with scholars from other disciplines.
2.9 (0.9)
3.4 (1.2)
−0.5
15. Disseminate interdisciplinary research results both within and outside his or her discipline.
3.1 (1.1)
3.8 (1.0)
−0.7
16. Author publications with scholars from other disciplines.
3.3 (0.9)
3.6 (1.2)
−0.4
17. Present interdisciplinary research at venues representing more than one discipline.
3.3 (1.0)
4.0 (0.8)
−0.7
Overall competency
3.4 (0.7)
3.9 (0.7)
−0.5
Competency
5-point Likert scale where 1 = Disagree, 5 = Agree. *Negative value indicates direction from disagree to agree. Scale ratings adapted from University of Illinois–Chicago School of Public Health Competency Assessment Tool, accessible at http://128.248.232.71/prepare/personal.asp.
Table 2. Interdisciplinary research competency self-assessment items and ratings (N = 10).
Toolbox intervention. Students responded in agreement that the Toolbox prompts and comments from others stimulated new thoughts about differing research worldviews and they felt free to present a view that differed from others in their group. In so doing, the Toolbox also helped their group identify some of their differing research worldviews and create mutual understanding, which they agreed was important for research productivity. The students overall, however, tended to disagree that they had done additional reading on the topics addressed in the workshop after the initial Toolbox workshop at the start of term, or discussed topics raised in the workshop with others not in the workshop or with other group members outside of class. In the survey, the students indicated that they gained important insights through the Toolbox experience, including: “Sometimes it’s shocking how differently people can interpret something that on the surface appears very straight forward,” and “It was interesting to hear other people’s opinions and world views.” The Toolbox promoted introspection as well: “It was interesting to see how strongly I hold onto my world views.” and “I think the Toolbox was effective at helping to clarify one’s own research perspective/worldview.” WWW.CTSJOURNAL.COM
A few students did not find value in the Toolbox modules implementation and made either skeptical or negative comments about an aspect of the Toolbox workshop such as: “I’m still not sure what the purpose of this Toolbox was. It seemed like our time could have more effectively be spent doing other things.” Comments were mixed on the need and benefit of additional time for the Toolbox intervention: “I thought we were given adequate time as is” to “an additional 15–20 minutes would have allowed us to further develop our conversation” or “No, they were more than enough time spent on the toolbox.” The relationship between the format and amount of time spent on the Toolbox and group composition surfaced in several comments: “I think that the toolbox is only useful when you truly have interdisciplinary groups. My group was primarily nurses and these discussions began to feel very one dimensional, especially over time.” VOLUME 8 • ISSUE 6
797
Hessels et al. BIRM n
and “The discussion was interesting and engaging. However, our group was not very interdisciplinary, so I think we got less out of it than we could have with a more diverse group.” An additional output from the workshop modules included the creation, discussion, and selection of the prompts generated by the students to facilitate discussion of the main Toolbox domains as described in the Methods. These prompts are presented in the Appendix.
Toolbox workshops places philosophical issues and methods front and center for much of the course, a reality is that might not be the same for every guest lecturer or student group. Changes in the mean ratings of these modules may also reflect the novice experience of students in research training. It may be at this developmental stage the ability to answer epistemological core “what” questions is more modifiable than the ability to answer metaphysical core “how” questions. Finally, as some student comments suggest, the approach used might be less effective in this largely monodisciplinary environment. Conclusion
Discussion
In 2012, The Center for Healthcare Research and Transformation conducted a symposium to identify barriers and solutions to increase the impact of health services researchers on both policy and practice.11 An underlying theme of key lessons identified was the importance of communication skills across varying worldviews, that is, that researchers must be able to speak the same vernacular as policymakers, elected officials, payers, purchasers, practice, and health system leaders.11 Integrating the Toolbox workshop modules into the course was a successful translational strategy toward achieving these goals. The necessary academic administrative support, infrastructure, and committed faculty from multiple disciplines were in place to support this project as a priority to improve interprofessional education.12 The Toolbox prompts generated dialogue and insight into central philosophical issues, as evidenced by the variance in responses pre- and postworkshop. Moreover, this is evident in the formation of philosophical commitments following the Workshops in that the number of “don’t know” responses decreased from 33 to 6. These findings indicate that that dialogue was constructive in cocreating philosophical interdisciplinary research underpinnings. The Toolbox workshop modules enhanced team building and team learning, already an expectation of the course, by focusing dialogue on core philosophical values that form the foundation for future interdisciplinary research work. This was evident in both interteam and intrateam communication tone, quality of outputs, and respectful and inclusive attitudes within and among groups indicating a high level of engagement.8 Engagement was further evident in the creation of new prompts, ensuing discussion and “voting” to retain or reject the prompt among course participants (Appendix). A rationale was presented by each group for the decision to reject or retain the prompt and discussed further by the class. This component of the activity, through ongoing sessions, provided the opportunity for all students to develop a desired skill and outcome of the Toolbox, namely, the individual and collaborative ability to explicitly articulate philosophical differences and commonalities.9 Notably, the substantive changes in the mean ratings of the groups occurred predominantly in the epistemological modules (Motivation, Confirmation, and Methodology); only one item from a metaphysical module (Reductionism) substantively shifted. This may reflect the overall congruence of these modules and core questions with content presented during the didactic sessions of the course. Though the design of integrating the
798
VOLUME 8 • ISSUE 6
The aims of this project were to develop and evaluate an interactive Toolbox workshop approach within a graduate level course to enhance student learning and skill in interdisciplinary research. This collaborative project successfully incorporated a structured template to guide students through introspective reflection, interactive dialogue, and explicit articulation of philosophical underpinnings of interdisciplinary research. Our experience indicates that incorporating this approach into research curricula can enhance individual level scientific capacity, future interdisciplinary research project success, and ultimately impact on practice and policy. Acknowledgments
Amanda Hessels was supported as a post-doctoral trainee by the National Institute of Nursing Research, National Institutes of Health (Training in Interdisciplinary Research to Prevent Infections, T32 NR013454). This publication was also supported in part by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant Number UL1 TR000040. References
1. Rosenfield PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Sci Med. 1992; 35: 1343–1357. 2. National Academy of Sciences. Facilitating Interdisciplinary Research. Washington, DC: National Academies Press; 2005. 3. Larson EL, Landers TF, Begg MD. Building interdisciplinary research models: a didactic course to prepare interdisciplinary scholars and faculty. Clin Transl Sci. 2011; 4: 38–41. 4. Aboelela SW, Larson E, Bakken S, Carrasquillo O, Formicola A, Glied SA, Haas J, Gebbie KM. Defining interdisciplinary research: conclusions from a critical review of the literature. Health Serv Res. 2007; 42: 329–346. 5. Gebbie KM, Meier BM, Bakken S, Carrasquillo O, Formicola A, Aboelela SW, Glied S, Larson E. Training for interdisciplinary health research: defining the required competencies. J Allied Health. 2008; 37: 65–70. 6. Eigenbrode S, O’Rourke M, Wulfhorst J, Althoff DM, Goldberg CS, Merrill K, Morse W, NielsenPincus M, Stephens J, Winowieci L, et al. Employing philosophical dialogue in collaborative science. BioScience. 2007; 57: 55–64 7. Stokols D, Hall KL, Taylor BK, Moser RP. The science of team science: overview of the field and introduction to the supplement. Am J Prev Med. 2008; 35(Suppl. 2): S77–S89. 8. O’Rourke M, Crowley S, Eigenbrode S, Wulfhorst J. Enhancing Communication and Collaboration in Interdisciplinary Research. Thousand Oaks, CA: Sage Publications; 2014. 9. Schnapp LM, Rotschy L, Hall TE, Crowley S, O’Rourke M. How to talk to strangers: facilitating knowledge sharing within translational health teams with the Toolbox dialogue method. Transl Behav Med. 2012; 2: 469–479. 10. O’Rourke M, Crowley S. Philosophical intervention and cross-disciplinary science: the story of the Toolbox Project. Synthese. 2013; 190: 1937–1954. 11. Kerr EA, Riba M, Udow-Phillips M. Helping health service researchers and policy makers speak the same language. Health Services Res. 2015; 50(1): 1–11. 12. Farrell K, Payne C, Heye M. Integrating interprofessional collaboration skills into the advanced practice registered nurse socialization process. J Profess Nurs. 31(1): 5–10.
WWW.CTSJOURNAL.COM
Hessels et al. BIRM n
Appendix: Health Science Toolbox Instrument Columbia University Developed Prompts
Motivation Core Question: What is your primary motivation for conducting research? 1. I pursue research for personal gain. 2. I would rather contribute a piece to the puzzle than the whole picture. 3. My research primarily aims to add knowledge to my own field. 4. The main reason for conducting research is to fill a gap in existing knowledge in my field. Values Core Question: Do values have a legitimate role in scientific research? 1. When a researcher personally cares about an issue he/she performs better research. 2. It is never fully possible to set aside personal biases during research. 3. It is difficult to work with people who have different values. Reductionism Core Question: Can the world under investigation be reduced to independent elements for study? 1. A holistic approach is the only way to capture the truth. 2. It is more important to study a system as a whole than its individual parts.
WWW.CTSJOURNAL.COM
Research Approaches Core Question: What things need to be taken into account in identifying a research problem? 1. The most important thing when formulating a research question is to address gaps in existing scientific knowledge. 2. Level of innovation is as important as level of certainty when constructing a research question. Methodology Core Question: What are the most important considerations in study design? 1. The most important consideration when choosing your study design is your hypothesis. 2. For every research question there is always one design which is best. 3. Funding should be the primary consideration when designing a study. 4. Study designs that generate both quantitative and qualitative contribute equally to an understanding of the truth. Confirmation Core Question: What types of evidentiary support are required for knowledge? 1. It is safe to apply findings of a study immediately following its completion. 2. Observational or retrospective evidence may be used alone in establishing confirmation.
VOLUME 8 • ISSUE 6
799
