From the Schools and Programs of Public Health INTEGRATING CLASSROOM, COMMUNITY, MIXED-METHODS RESEARCH, AND COMMUNITY-BASED PARTICIPATORY RESEARCH TO TEACH PUBLIC HEALTH PRACTICE Mudita Upadhyaya, MHA, MPH Marlynn May, PhD Linda Highfield, PhD, MS
Community-based participatory research (CBPR) is an iterative, action-based research approach bridging the gap between theory and practice by engaging community throughout the research process.1 CBPR has been framed as a research protocol, rather than a specific methodology, that provides an overarching approach to research (e.g., qualitative or quantitative methods) seeking to balance power dynamics between academic and community partners.1 Therefore, CBPR studies can be conducted using a variety of methods, including mixed methods.2 Mixed-methods research includes the mixing of quantitative and qualitative data collection and analysis within a single study.3 Theoretically, a mixed-methods approach engages researchers from diverse disciplines in designing and implementing research. In mixed methods, adherence to methodological structure is central to organizing academics from diverse epistemological, theoretical, and methodological backgrounds to cooperate, sometimes collaborate, and maybe even form partnerships.3,4 Creating this structure is a major undertaking that is not easily accomplished. Concomitantly, the CBPR protocol mandates attention to process and guides collaboration within the mixed-methods structure.5 The CBPR process focuses on mixed-methods team formation and integration with people from different backgrounds. Public health master’s and/or doctoral students in a typical curriculum receive limited training in CBPR, and even less in combination with mixed-methods research.3,6–8 Recognizing this shortfall in curricula training, in the fall of 2012, a doctoral-level seminar curriculum was designed and implemented expressly to teach students both mixed-methods research and CBPR. In this article, we report on a practice model in which students, community leaders, and faculty collaborated in a teaching, learning, and research experiment 286
that integrated mixed methods and CBPR. An underlying thesis in this practice model is that mixed-methods research and CBPR have an inherent affinity. The structure of mixed methods calls for a team approach (i.e., team science). The processes encapsulated in the principles and practices of CBPR are tailored for achieving team collaboration. Thus, CBPR creates a crucible for implementing mixed-methods team science in community health practice. As such, the goal of this seminar was to enhance student learning and practice of (1) mixed methods, (2) integration with the principles of CBPR, (3) forming interdisciplinary teams reflecting mixed-methods team science, and (4) building a community-academic partnership. METHODS Project description Key stakeholders for the project partnership included faculty and doctoral students from the University of Texas School of Public Health (UTSPH) in Houston, Texas; faculty from the Texas A&M School of Rural Public Health (SRPH) in College Station, Texas; and organizational leadership from the community partner, the Breast Health Collaborative of Texas (BHCT), in Houston, Texas. Faculty from both universities had a history of collaboration with BHCT on research projects. Dr. Highfield had worked closely with BHCT during a three-year period on research related to access to mammography screening in underserved women and had already established a level of trust with the organization, which served as a springboard for this course. Eight doctoral students enrolled in the class from health promotion, health services research, and community health practice majors. Students also had varied master’s-level educational backgrounds (i.e., in social work, international health, management and policy science, and epidemiology) and work experience (i.e., 1–8 years). Consequently, their epistemological assumptions about knowledge acquisition varied greatly. The community partner for the project, BHCT, is a nonprofit consortium of organizational and individual members with a mission to educate, advocate, and leverage resources to improve access to breast health care for all Texans.9 BHCT was founded in 2005 in response to a need to unite mammogram providers to increase the rates of breast cancer screening among disadvantaged populations.9 In 2007, BHCT expanded
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 287
its collaborative network throughout the state (Personal communication, Lyn Widlaski, BHCT, October 2012). Mixed-methods/CBPR project conceptualization and proposal development The curriculum structure followed a constructivist approach10 requiring students to interact extensively with community and faculty and among themselves to create a collaborative partnership. Project conceptualization and development was conducted in four stages during the seminar: (1) interdisciplinary team formation, (2) relationship building and pre-research design contemplation, (3 ) research design and proposal development, and (4) feedback, as shown in Figure 1.
field-level learning. The faculty taught theory through classroom lectures, exercises, presentations, and facilitated group discussions. Students reviewed peerreviewed literature and theoretical models related to CBPR, implementation, and evaluation science. To develop an adequate research design, and to combine knowledge with action, a CBPR approach was used to engage and integrate BHCT and academic team members equitably throughout the research process. The teams progressed through five steps to enable contextual understanding and relationship building:
Stage 1. During stage 1, interdisciplinary team formation was a key component and served as the foundation for the seminar. Following didactic training in multiple research ontology and epistemology paradigms, students were divided into two interdisciplinary mixed-methods research teams, each team containing diverse ontological and epistemological positions and preferred research methods (i.e., qualitative vs. quantitative). Each team also included one BHCT staff member. The role of the two lead faculty members was to provide guidance to the teams and ensure that course objectives for the curriculum were met as the seminar evolved. Stage 2. Stage 2 focused on relationship building and pre-design deliberations within the teams. Throughout the seminar, students engaged in an iterative learning process at two levels: (1) theoretically grounded classroom learning and (2) practical application and
1. The teams held a preliminary meeting to learn more about BHCT and brainstorm potential research ideas. 2. Students were invited to attend two monthly membership meetings and webinars hosted by BHCT at different locations in the community. Each student was required prior to these meetings to prepare relevant questions to acquire in-depth information about the collaborative. 3. Following the community meetings, each team participated in separate 90-minute, face-to-face research problem/question-generating sessions facilitated by the faculty. Exhaustive lists of concepts were recorded related to the identified research problem. The teams used nontechnical language and terminology to make the process understandable for the community partners, and then regrouped the generated concepts under broader themes or categories. 4. The student members of each team examined peer-reviewed literature to identify existing
Figure 1. Stages used to create a collaborative partnership between public health students and community partners using team science and a CBPR approach: University of Texas School of Public Health, Houston, Texas, 2012
CBPR 5 community-based participatory research
Public Health Reports / May–June 2015 / Volume 130
288 From the Schools and Programs of Public Health
evidence and theoretical models to better understand the relationships between the themes generated and how they were connected to the research problem. Students were responsible for presenting and explaining the reviews for the benefit of their BHCT team member. 5. A second 90-minute, face-to-face, facilitated session allowed each team to focus on integrating the identified themes into a specific, measurable, attainable, realistic, and time-bound research question and develop a table that outlined potential variables, indicators, indicator types (i.e., qualitative and quantitative), data collection methods, and rationale for use. Stage 3. During Stage 3, each team created a draft research proposal detailing its identified research problem, related concepts, and research design. During this stage, each team worked independently to develop an initial design. Faculty monitored and reviewed each team’s design development during several weeks and provided feedback. Each research team developed an independent proposal detailing its rationale, sampling methodology, implementation plan, and a dissemination plan for results. Stage 4. In Stage 4, students from each team shared their draft proposals with the students, faculty, and community members on the other team for review and feedback, both in written and oral formats. Following up, and based on discussion and feedback from this formal presentation session, the interdisciplinary research teams together created one final proposal from the two draft proposals. In the end, a unified, CBPR-based, mixed-methods, interdisciplinary research design was created, integrating formal content and informal expectations of community and academic stakeholders. OUTCOMES The final outcome of this curriculum was a concrete, practicable, and unified research proposal, ready to be implemented in future semester(s). The students were presented with an offer to implement the proposal in the future semester(s) either as a practicum or as a dissertation study. The overall goal and specific research questions were developed with the underlying philosophy and mission of the BHCT collaborative in mind. The main goal of this proposal was to inform BHCT leadership about the overall effectiveness of memberto-member and member-to-collaborative relationships.
The proposal was designed to be implemented in two discrete phases, using an explanatory, sequential, mixed-methods design, in which the quantitative data collection phase is typically followed up with a qualitative data collection phase. The first phase was designed to collect and analyze quantitative data through an initial blitz membership survey to (1) identify members and non-members of the BHCT and their location, (2) describe types of relationships within the BHCT, and (3) indicate the level of resource utilization by individuals and organizations (i.e., members and non-members). The existing data sources from BHCT were not updated and some members’ information was entered twice because the records in the database were not linked properly. Thus, the BHCT database contained more members than those actually enrolled, resulting in an inaccurate count of registered members. Also, the extent to which members considered themselves part of BHCT was unknown. This information was crucial for evaluating the BHCT’s membership. The survey was pilot tested with a small sample of BHCT members to ensure the appropriateness of questions and language. The first phase—the blitz survey—was followed by a second, qualitative phase, mainly designed to answer the how and why of the initial quantitative results. Also, the initial blitz survey could not answer certain questions that were applicable to a subset of BHCT members and the nature of their unique relationship with BHCT. For example, it was made clear through conversations with the BHCT administrative team that certain individuals and organizations affiliated with BHCT had utilized resources from the BHCT but did not consider themselves as BHCT members. Little was known about their involvement within BHCT and why they had used BHCT’s resources. It was also important to collect relevant information and draw comparisons between the affiliated and nonaffiliated organizational members, as this information would have helped to provide the BHCT administrative team information on how to involve inactive affiliates. For the second phase, the key informant interviews of self-identified members and non-members were designated to describe the quantity, quality, and types of relationships that existed among self-identified members, non-members, and the BHCT leadership. Student evaluations The students participating in the course had the opportunity to evaluate and provide feedback at midterm and the end of the semester. The midterm
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 289
evaluation included open-ended questions, and the end-of-semester evaluation was a scaled survey with response categories ranging from 1 (lowest) to 5 (highest). During the midterm evaluation, students noted that they particularly enjoyed the hands-on activities, facilitation sessions, and having a real project. “Having a real project is an awesome addition—in most classes, we do imaginary projects,” one student noted. They also indicated that having to seek out and find their own direction was a challenge. As one student put it, “This course has forced me to seek out my own direction in finding out what’s needed, and I haven’t had much experience with that.” In the final course evaluation, instructor effectiveness was rated higher than the school average, as were several areas of the course. Students cited the following as positives: being encouraged to ask questions and express ideas, interaction in the course facilitating progress, faculty creating a respectful learning environment, and faculty conveying high expectations and expecting students to show a high level of initiative. Some specific areas of the course scored lower than the school average: receiving regular feedback, multiple methods of assessing mastery, and grading differentiation among students who performed and those who did not. LESSONS LEARNED The strength of this seminar’s curriculum was in allowing students to understand the intrinsic interconnectedness of public health theory and community practice, as shown in Figure 211 and as noted by students’ course evaluations. The students gained a broader, deeper understanding and appreciation of CBPR and mixedmethods approaches and their application in real-world settings. Collaboration between student researchers from various disciplines and epistemologies and a community partner for a scientific inquiry is complex and presents many challenges throughout the process. The construction of multidisciplinary teams at an early stage of the curriculum was crucial for this learning process. It gave the students an opportunity to gain insight into others’ epistemological approaches and expand their methodological perspectives, thus making the research process more inclusive. The CBPR approach helped the students gain a deeper understanding of the underlying cross-cultural and contextual issues, which are usually overlooked in favor of the suitability of the research design and resources at hand. For example, community integration as a part of the research teams right from the planning stage was fundamental in building relationships with
the community partner. Initial participation in the webinars and community meetings greatly aided in understanding the existing gaps in the collection and documentation of membership enrollment data. This clarity in understanding basic structures and processes helped the students to sequence the quantitative and qualitative arm of the research design. Furthermore, it enabled students to examine the key research areas from a community perspective as opposed to a purely academic and theoretical standpoint. The use of a mixed-methods approach complemented the limitations of isolated methodologies and made the project design more applicable and meaningful, especially for the community partners.5,12 The integration of didactic and constructivist teaching approaches also promoted student-centered learning.9 Traditional teacher-student roles were reversed, where teachers acted as facilitators and students took a more active role as collaborators and researchers. The students gained invaluable firsthand experience working within a collaborative environment where they were able to translate their theoretical knowledge and understanding of public health into practice and an actionable outcome in the form of one unified proposal for BHCT evaluation. Furthermore, the field project offered a unique opportunity for the doctoral students to realize that while initial relationships with the community were relatively easy to develop, building and maintaining those relationships was harder and more time- and resource-intensive.13–15 Conflicts across employees of the same organization, researchers, and community members are very common in the real world.16 The two 90-minute sessions were an ideal setting for the students to deal with such issues. Interestingly, and importantly for development of the research design, the two community members participating in the course had fundamentally different perspectives and ideas about how to prioritize the research foci; as a result, they identified different research questions that they considered, in their respective roles, best evaluated the collaborative. The two draft proposals were developed to accommodate these conflicting priority areas, thus prioritizing community-owned research over nominal community involvement. The iterative process needed to create a final unified draft from the two drafts compelled students to integrate and practice principles of team science (i.e., mutual respect, power balance, and effective communication) and CBPR (i.e., shared knowledge and resources) to arrive at an actionable solution to the problem at hand. The students experienced the
Public Health Reports / May–June 2015 / Volume 130
290 From the Schools and Programs of Public Health
Figure 2. Public health training framework for integrating public health research, practice, and implementation science: University of Texas School of Public Health, Houston, Texas, 2012a
a Adapted from Evidence Integration Triangle model in: Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med 2012;42:646-54.
CBPR 5 community-based participatory research
advantages of being part of a team that could evaluate a research question from diverse vantage points, facilitate a comprehensive analysis of the situation, and work through conflicts when needed, thereby improving the chances of a holistic solution. The students also learned that, when partnering with community, communication has to be clear to be effective, particularly avoiding the use of technical language.5,14,17,18 Specifically, the students were given an opportunity to express and clarify their thoughts, opinions, and research approaches to a variety of audiences—an experience often absent in a typical classroom setting. All stakeholders came from varied backgrounds and expected results to be disseminated in different forms, offering both quantitative and qualitative methods of investigation and dissemination. The practice of public health thrives on a multidisciplinary, participatory approach.19 Principles of team science, theoretical public health knowledge, multiple research methodologies, and understanding and managing organizational processes are complementary approaches, aiding practitioners in carving out practical solutions to community-based health problems. In a traditional academic setting, these ingredients are more often taught independently of each other and at different career stages for public health students. This curriculum offered an opportunity for public health students to acquire not only theoretical understanding, but also nuances of practical application simultaneously and in a mentored environment.
The faculty also learned important lessons while teaching the course. Managing a course of this nature is paramount, and process takes on unusual importance alongside teaching outcomes. Overseeing the work with the community partner was very time intensive for the faculty members, much more so than a traditional didactic course and more involved than if the faculty partners were working with a community partner without student involvement. Additionally, maintaining the flexibility to mold the project based on community needs and maintaining prescheduled assignments for the students proved demanding. Faculty agreed with the students’ assessment that their learning would have been enriched by taking the project through completion; however, fitting design implementation into a single semester course was not possible. Future work should consider how to effectively transition students from the design of a study through completion, all of which are critical skills for practitioners. One option might be to make the course two semesters long. Finally, future iterations of the course will focus on orienting the students at the beginning of the semester to the flexible and proficiency-oriented perspective of the course. What the students perceived as a lack of clarity related to assignments and grading was perceived by the faculty as the reality of working in a CBPR project with community. It is rare to have a predeveloped schedule of activities and know from the outset exactly what the interactive engagement with the community will require and what adaptations will be needed. This
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 291
approach is radically different from what students usually receive in their training and grading. Limitations This study was subject to several limitations. First, the course outcome was limited to designing the proposal and not implementation, which could have been a much richer experience for the students. Second, as this study was a first of its kind at UTSPH, there was no ready point of reference for the faculty or the students to use. Improvisations were sometimes made on the fly, as when a deviation was required to make the process flow. These lessons were also reflected in the students’ course evaluations. Deviation from the course plan made sticking to a predesigned schedule of assignments challenging. Additionally, the ultimate course objective was student learning and growth in trying to blend research and practice. Students were given the opportunity to submit assignments for feedback and resubmit based on feedback received. They were evaluated based on their growth through the course, and the typical differentiation between high and low performers in the academic grading model was not the focus of this class. Because the course followed a different grading and assessment approach, students may have felt frustrated by their perceived level of mastery as compared with other students. Students also likely did not realize that they were indeed being evaluated by multiple methods somewhat differently from a nontraditional classroom approach. Students were evaluated based on their interactions with each other and the community partner, in-class participation, written assignments, and verbal presentations, although they did not receive a formal grade each week for many of these interactions. Instead, they received detailed feedback from the faculty in the hopes of creating student growth and mastery. Lastly, the experience of the students and the results obtained from this course were contextual to the community partner and problem at hand. Any replications should take into account necessary adaptations based on specific contexts. CONCLUSIONS It is important not to think of public health science, practice, and community in silos. Concerted efforts are mandated from graduate schools and professional associations (e.g., American Public Health Association and Association of Schools and Programs of Public Health) to refocus their efforts to integrate and align curricular training with community-based practical experience. The curricular model presented in this article offers
a replicable solution that can be used as a launch pad to train the future public health workforce to become more community oriented and technically proficient to address both ongoing and emerging health problems. Mudita Upadhyaya is a Doctor of Public Health Candidate in Management, Policy, and Community Health at the University of Texas School of Public Health, Division of Epidemiology, Human Genetics, and Environmental Sciences in Houston, Texas. Marlynn May is a Professor Emeritus at the Texas A&M University System Department of Social and Behavioral Health in College Station, Texas. Linda Highfield is an Assistant Professor at the University of Texas School of Public Health, Division of Management, Policy, and Community Health. Address correspondence to: Mudita Upadhyaya, MHA, MPH, University of Texas School of Public Health, Division of Epidemiology, Human Genetics, and Environmental Sciences, 1200 Herman Pressler, Ste. E637, Houston, TX 77030; tel. 713-4746203; fax 713-500-9264; e-mail . ©2015 Association of Schools and Programs of Public Health
REFERENCES 1. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health 2010;100 Suppl 1:S40-6. 2. Padgett DK. Qualitative methods in social work research. 2nd ed. Thousand Oaks (CA): Sage Publications, Inc.; 2008. 3. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. Washington: National Institutes of Health; 2011. 4. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks (CA): Sage Publications, Inc.; 2010. 5. Johnson CE, Ali SA, Shipp MP. Building community-based participatory research partnerships with a Somali refugee community. Am J Prev Med 2009;37(6 Suppl 1):S230-6. 6. Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream: are researchers prepared? Circulation 2009;119:2633-42. 7. Ahmed SM, Beck B, Maurana CA, Newton G. Overcoming barriers to effective community-based participatory research in US medical schools. Educ Health (Abingdon) 2004;17:141-51. 8. Potter MA, Quill BE, Aglipay GS, Anderson E, Rowitz L, Smith LU, et al. Demonstrating excellence in practice-based research for public health. Public Health Rep 2006;12(Suppl 1):1-16. 9. Breast Health Collaborative of Texas. Who we are [cited 2012 Oct 23]. Available from: URL: http://www.breasthealthcollaborativeoftexas .org 10. Shen T, Wu D, Archhpiliya V, Bieber M, Hiltz R. Participatory learning approach: a research agenda. Newark (NJ): New Jersey Institute of Technology, Information Systems Department; 2004. 11. Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med 2012;42:646-54. 12. Weathers B, Barg FK, Bowman M, Briggs V, Delmoor E, Kumanyika S, et al. Using a mixed-methods approach to identify health concerns in an African American community. Am J Public Health 2011;101:2087-92. 13. Christopher S, Watts V, McCormick AK, Young S. Building and maintaining trust in a community-based participatory research partnership. Am J Public Health 2008;98:1398-406. 14. O’Fallon LR, Dearry A. Community-based participatory research as a tool to advance environmental health sciences. Environ Health Perspect 2002;110 Suppl 2:155-9. 15. Minkler M, Wallerstein N. Community-based participatory research for health: from process to outcomes. 2nd ed. Hoboken (NJ): John Wiley & Sons; 2010.
Public Health Reports / May–June 2015 / Volume 130
292 From the Schools and Programs of Public Health
16. Resnik DB, Kennedy CE. Balancing scientific and community interests in community-based participatory research. Account Res 2010;17:198-210. 17. Israel BA, Parker EA, Rowe Z, Salvatore A, Minkler M, Lopez J, et al. Community-based participatory research: lessons learned from the Centers for Children’s Environmental Health and Disease Prevention Research. Environ Health Perspect 2005;113:1463-71.
18. Rivkin I, Trimble J, Lopez ED, Johnson S, Orr E, Allen J. Disseminating research in rural Yup’ik communities: challenges and ethical considerations in moving from discovery to intervention development. Int J Circumpolar Health 2013;72. 19. Baker EA, Homan S, Schonhoff SR, Kreuter M. Principles of practice for academic/practice/community research partnerships. Am J Prev Med 1999;16(3 Suppl):86-93.
Public Health Reports / May–June 2015 / Volume 130
Community-based participatory research (CBPR) is an iterative, action-based research approach bridging the gap between theory and practice by engaging community throughout the research process.1 CBPR has been framed as a research protocol, rather than a specific methodology, that provides an overarching approach to research (e.g., qualitative or quantitative methods) seeking to balance power dynamics between academic and community partners.1 Therefore, CBPR studies can be conducted using a variety of methods, including mixed methods.2 Mixed-methods research includes the mixing of quantitative and qualitative data collection and analysis within a single study.3 Theoretically, a mixed-methods approach engages researchers from diverse disciplines in designing and implementing research. In mixed methods, adherence to methodological structure is central to organizing academics from diverse epistemological, theoretical, and methodological backgrounds to cooperate, sometimes collaborate, and maybe even form partnerships.3,4 Creating this structure is a major undertaking that is not easily accomplished. Concomitantly, the CBPR protocol mandates attention to process and guides collaboration within the mixed-methods structure.5 The CBPR process focuses on mixed-methods team formation and integration with people from different backgrounds. Public health master’s and/or doctoral students in a typical curriculum receive limited training in CBPR, and even less in combination with mixed-methods research.3,6–8 Recognizing this shortfall in curricula training, in the fall of 2012, a doctoral-level seminar curriculum was designed and implemented expressly to teach students both mixed-methods research and CBPR. In this article, we report on a practice model in which students, community leaders, and faculty collaborated in a teaching, learning, and research experiment 286
that integrated mixed methods and CBPR. An underlying thesis in this practice model is that mixed-methods research and CBPR have an inherent affinity. The structure of mixed methods calls for a team approach (i.e., team science). The processes encapsulated in the principles and practices of CBPR are tailored for achieving team collaboration. Thus, CBPR creates a crucible for implementing mixed-methods team science in community health practice. As such, the goal of this seminar was to enhance student learning and practice of (1) mixed methods, (2) integration with the principles of CBPR, (3) forming interdisciplinary teams reflecting mixed-methods team science, and (4) building a community-academic partnership. METHODS Project description Key stakeholders for the project partnership included faculty and doctoral students from the University of Texas School of Public Health (UTSPH) in Houston, Texas; faculty from the Texas A&M School of Rural Public Health (SRPH) in College Station, Texas; and organizational leadership from the community partner, the Breast Health Collaborative of Texas (BHCT), in Houston, Texas. Faculty from both universities had a history of collaboration with BHCT on research projects. Dr. Highfield had worked closely with BHCT during a three-year period on research related to access to mammography screening in underserved women and had already established a level of trust with the organization, which served as a springboard for this course. Eight doctoral students enrolled in the class from health promotion, health services research, and community health practice majors. Students also had varied master’s-level educational backgrounds (i.e., in social work, international health, management and policy science, and epidemiology) and work experience (i.e., 1–8 years). Consequently, their epistemological assumptions about knowledge acquisition varied greatly. The community partner for the project, BHCT, is a nonprofit consortium of organizational and individual members with a mission to educate, advocate, and leverage resources to improve access to breast health care for all Texans.9 BHCT was founded in 2005 in response to a need to unite mammogram providers to increase the rates of breast cancer screening among disadvantaged populations.9 In 2007, BHCT expanded
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 287
its collaborative network throughout the state (Personal communication, Lyn Widlaski, BHCT, October 2012). Mixed-methods/CBPR project conceptualization and proposal development The curriculum structure followed a constructivist approach10 requiring students to interact extensively with community and faculty and among themselves to create a collaborative partnership. Project conceptualization and development was conducted in four stages during the seminar: (1) interdisciplinary team formation, (2) relationship building and pre-research design contemplation, (3 ) research design and proposal development, and (4) feedback, as shown in Figure 1.
field-level learning. The faculty taught theory through classroom lectures, exercises, presentations, and facilitated group discussions. Students reviewed peerreviewed literature and theoretical models related to CBPR, implementation, and evaluation science. To develop an adequate research design, and to combine knowledge with action, a CBPR approach was used to engage and integrate BHCT and academic team members equitably throughout the research process. The teams progressed through five steps to enable contextual understanding and relationship building:
Stage 1. During stage 1, interdisciplinary team formation was a key component and served as the foundation for the seminar. Following didactic training in multiple research ontology and epistemology paradigms, students were divided into two interdisciplinary mixed-methods research teams, each team containing diverse ontological and epistemological positions and preferred research methods (i.e., qualitative vs. quantitative). Each team also included one BHCT staff member. The role of the two lead faculty members was to provide guidance to the teams and ensure that course objectives for the curriculum were met as the seminar evolved. Stage 2. Stage 2 focused on relationship building and pre-design deliberations within the teams. Throughout the seminar, students engaged in an iterative learning process at two levels: (1) theoretically grounded classroom learning and (2) practical application and
1. The teams held a preliminary meeting to learn more about BHCT and brainstorm potential research ideas. 2. Students were invited to attend two monthly membership meetings and webinars hosted by BHCT at different locations in the community. Each student was required prior to these meetings to prepare relevant questions to acquire in-depth information about the collaborative. 3. Following the community meetings, each team participated in separate 90-minute, face-to-face research problem/question-generating sessions facilitated by the faculty. Exhaustive lists of concepts were recorded related to the identified research problem. The teams used nontechnical language and terminology to make the process understandable for the community partners, and then regrouped the generated concepts under broader themes or categories. 4. The student members of each team examined peer-reviewed literature to identify existing
Figure 1. Stages used to create a collaborative partnership between public health students and community partners using team science and a CBPR approach: University of Texas School of Public Health, Houston, Texas, 2012
CBPR 5 community-based participatory research
Public Health Reports / May–June 2015 / Volume 130
288 From the Schools and Programs of Public Health
evidence and theoretical models to better understand the relationships between the themes generated and how they were connected to the research problem. Students were responsible for presenting and explaining the reviews for the benefit of their BHCT team member. 5. A second 90-minute, face-to-face, facilitated session allowed each team to focus on integrating the identified themes into a specific, measurable, attainable, realistic, and time-bound research question and develop a table that outlined potential variables, indicators, indicator types (i.e., qualitative and quantitative), data collection methods, and rationale for use. Stage 3. During Stage 3, each team created a draft research proposal detailing its identified research problem, related concepts, and research design. During this stage, each team worked independently to develop an initial design. Faculty monitored and reviewed each team’s design development during several weeks and provided feedback. Each research team developed an independent proposal detailing its rationale, sampling methodology, implementation plan, and a dissemination plan for results. Stage 4. In Stage 4, students from each team shared their draft proposals with the students, faculty, and community members on the other team for review and feedback, both in written and oral formats. Following up, and based on discussion and feedback from this formal presentation session, the interdisciplinary research teams together created one final proposal from the two draft proposals. In the end, a unified, CBPR-based, mixed-methods, interdisciplinary research design was created, integrating formal content and informal expectations of community and academic stakeholders. OUTCOMES The final outcome of this curriculum was a concrete, practicable, and unified research proposal, ready to be implemented in future semester(s). The students were presented with an offer to implement the proposal in the future semester(s) either as a practicum or as a dissertation study. The overall goal and specific research questions were developed with the underlying philosophy and mission of the BHCT collaborative in mind. The main goal of this proposal was to inform BHCT leadership about the overall effectiveness of memberto-member and member-to-collaborative relationships.
The proposal was designed to be implemented in two discrete phases, using an explanatory, sequential, mixed-methods design, in which the quantitative data collection phase is typically followed up with a qualitative data collection phase. The first phase was designed to collect and analyze quantitative data through an initial blitz membership survey to (1) identify members and non-members of the BHCT and their location, (2) describe types of relationships within the BHCT, and (3) indicate the level of resource utilization by individuals and organizations (i.e., members and non-members). The existing data sources from BHCT were not updated and some members’ information was entered twice because the records in the database were not linked properly. Thus, the BHCT database contained more members than those actually enrolled, resulting in an inaccurate count of registered members. Also, the extent to which members considered themselves part of BHCT was unknown. This information was crucial for evaluating the BHCT’s membership. The survey was pilot tested with a small sample of BHCT members to ensure the appropriateness of questions and language. The first phase—the blitz survey—was followed by a second, qualitative phase, mainly designed to answer the how and why of the initial quantitative results. Also, the initial blitz survey could not answer certain questions that were applicable to a subset of BHCT members and the nature of their unique relationship with BHCT. For example, it was made clear through conversations with the BHCT administrative team that certain individuals and organizations affiliated with BHCT had utilized resources from the BHCT but did not consider themselves as BHCT members. Little was known about their involvement within BHCT and why they had used BHCT’s resources. It was also important to collect relevant information and draw comparisons between the affiliated and nonaffiliated organizational members, as this information would have helped to provide the BHCT administrative team information on how to involve inactive affiliates. For the second phase, the key informant interviews of self-identified members and non-members were designated to describe the quantity, quality, and types of relationships that existed among self-identified members, non-members, and the BHCT leadership. Student evaluations The students participating in the course had the opportunity to evaluate and provide feedback at midterm and the end of the semester. The midterm
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 289
evaluation included open-ended questions, and the end-of-semester evaluation was a scaled survey with response categories ranging from 1 (lowest) to 5 (highest). During the midterm evaluation, students noted that they particularly enjoyed the hands-on activities, facilitation sessions, and having a real project. “Having a real project is an awesome addition—in most classes, we do imaginary projects,” one student noted. They also indicated that having to seek out and find their own direction was a challenge. As one student put it, “This course has forced me to seek out my own direction in finding out what’s needed, and I haven’t had much experience with that.” In the final course evaluation, instructor effectiveness was rated higher than the school average, as were several areas of the course. Students cited the following as positives: being encouraged to ask questions and express ideas, interaction in the course facilitating progress, faculty creating a respectful learning environment, and faculty conveying high expectations and expecting students to show a high level of initiative. Some specific areas of the course scored lower than the school average: receiving regular feedback, multiple methods of assessing mastery, and grading differentiation among students who performed and those who did not. LESSONS LEARNED The strength of this seminar’s curriculum was in allowing students to understand the intrinsic interconnectedness of public health theory and community practice, as shown in Figure 211 and as noted by students’ course evaluations. The students gained a broader, deeper understanding and appreciation of CBPR and mixedmethods approaches and their application in real-world settings. Collaboration between student researchers from various disciplines and epistemologies and a community partner for a scientific inquiry is complex and presents many challenges throughout the process. The construction of multidisciplinary teams at an early stage of the curriculum was crucial for this learning process. It gave the students an opportunity to gain insight into others’ epistemological approaches and expand their methodological perspectives, thus making the research process more inclusive. The CBPR approach helped the students gain a deeper understanding of the underlying cross-cultural and contextual issues, which are usually overlooked in favor of the suitability of the research design and resources at hand. For example, community integration as a part of the research teams right from the planning stage was fundamental in building relationships with
the community partner. Initial participation in the webinars and community meetings greatly aided in understanding the existing gaps in the collection and documentation of membership enrollment data. This clarity in understanding basic structures and processes helped the students to sequence the quantitative and qualitative arm of the research design. Furthermore, it enabled students to examine the key research areas from a community perspective as opposed to a purely academic and theoretical standpoint. The use of a mixed-methods approach complemented the limitations of isolated methodologies and made the project design more applicable and meaningful, especially for the community partners.5,12 The integration of didactic and constructivist teaching approaches also promoted student-centered learning.9 Traditional teacher-student roles were reversed, where teachers acted as facilitators and students took a more active role as collaborators and researchers. The students gained invaluable firsthand experience working within a collaborative environment where they were able to translate their theoretical knowledge and understanding of public health into practice and an actionable outcome in the form of one unified proposal for BHCT evaluation. Furthermore, the field project offered a unique opportunity for the doctoral students to realize that while initial relationships with the community were relatively easy to develop, building and maintaining those relationships was harder and more time- and resource-intensive.13–15 Conflicts across employees of the same organization, researchers, and community members are very common in the real world.16 The two 90-minute sessions were an ideal setting for the students to deal with such issues. Interestingly, and importantly for development of the research design, the two community members participating in the course had fundamentally different perspectives and ideas about how to prioritize the research foci; as a result, they identified different research questions that they considered, in their respective roles, best evaluated the collaborative. The two draft proposals were developed to accommodate these conflicting priority areas, thus prioritizing community-owned research over nominal community involvement. The iterative process needed to create a final unified draft from the two drafts compelled students to integrate and practice principles of team science (i.e., mutual respect, power balance, and effective communication) and CBPR (i.e., shared knowledge and resources) to arrive at an actionable solution to the problem at hand. The students experienced the
Public Health Reports / May–June 2015 / Volume 130
290 From the Schools and Programs of Public Health
Figure 2. Public health training framework for integrating public health research, practice, and implementation science: University of Texas School of Public Health, Houston, Texas, 2012a
a Adapted from Evidence Integration Triangle model in: Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med 2012;42:646-54.
CBPR 5 community-based participatory research
advantages of being part of a team that could evaluate a research question from diverse vantage points, facilitate a comprehensive analysis of the situation, and work through conflicts when needed, thereby improving the chances of a holistic solution. The students also learned that, when partnering with community, communication has to be clear to be effective, particularly avoiding the use of technical language.5,14,17,18 Specifically, the students were given an opportunity to express and clarify their thoughts, opinions, and research approaches to a variety of audiences—an experience often absent in a typical classroom setting. All stakeholders came from varied backgrounds and expected results to be disseminated in different forms, offering both quantitative and qualitative methods of investigation and dissemination. The practice of public health thrives on a multidisciplinary, participatory approach.19 Principles of team science, theoretical public health knowledge, multiple research methodologies, and understanding and managing organizational processes are complementary approaches, aiding practitioners in carving out practical solutions to community-based health problems. In a traditional academic setting, these ingredients are more often taught independently of each other and at different career stages for public health students. This curriculum offered an opportunity for public health students to acquire not only theoretical understanding, but also nuances of practical application simultaneously and in a mentored environment.
The faculty also learned important lessons while teaching the course. Managing a course of this nature is paramount, and process takes on unusual importance alongside teaching outcomes. Overseeing the work with the community partner was very time intensive for the faculty members, much more so than a traditional didactic course and more involved than if the faculty partners were working with a community partner without student involvement. Additionally, maintaining the flexibility to mold the project based on community needs and maintaining prescheduled assignments for the students proved demanding. Faculty agreed with the students’ assessment that their learning would have been enriched by taking the project through completion; however, fitting design implementation into a single semester course was not possible. Future work should consider how to effectively transition students from the design of a study through completion, all of which are critical skills for practitioners. One option might be to make the course two semesters long. Finally, future iterations of the course will focus on orienting the students at the beginning of the semester to the flexible and proficiency-oriented perspective of the course. What the students perceived as a lack of clarity related to assignments and grading was perceived by the faculty as the reality of working in a CBPR project with community. It is rare to have a predeveloped schedule of activities and know from the outset exactly what the interactive engagement with the community will require and what adaptations will be needed. This
Public Health Reports / May–June 2015 / Volume 130
From the Schools and Programs of Public Health 291
approach is radically different from what students usually receive in their training and grading. Limitations This study was subject to several limitations. First, the course outcome was limited to designing the proposal and not implementation, which could have been a much richer experience for the students. Second, as this study was a first of its kind at UTSPH, there was no ready point of reference for the faculty or the students to use. Improvisations were sometimes made on the fly, as when a deviation was required to make the process flow. These lessons were also reflected in the students’ course evaluations. Deviation from the course plan made sticking to a predesigned schedule of assignments challenging. Additionally, the ultimate course objective was student learning and growth in trying to blend research and practice. Students were given the opportunity to submit assignments for feedback and resubmit based on feedback received. They were evaluated based on their growth through the course, and the typical differentiation between high and low performers in the academic grading model was not the focus of this class. Because the course followed a different grading and assessment approach, students may have felt frustrated by their perceived level of mastery as compared with other students. Students also likely did not realize that they were indeed being evaluated by multiple methods somewhat differently from a nontraditional classroom approach. Students were evaluated based on their interactions with each other and the community partner, in-class participation, written assignments, and verbal presentations, although they did not receive a formal grade each week for many of these interactions. Instead, they received detailed feedback from the faculty in the hopes of creating student growth and mastery. Lastly, the experience of the students and the results obtained from this course were contextual to the community partner and problem at hand. Any replications should take into account necessary adaptations based on specific contexts. CONCLUSIONS It is important not to think of public health science, practice, and community in silos. Concerted efforts are mandated from graduate schools and professional associations (e.g., American Public Health Association and Association of Schools and Programs of Public Health) to refocus their efforts to integrate and align curricular training with community-based practical experience. The curricular model presented in this article offers
a replicable solution that can be used as a launch pad to train the future public health workforce to become more community oriented and technically proficient to address both ongoing and emerging health problems. Mudita Upadhyaya is a Doctor of Public Health Candidate in Management, Policy, and Community Health at the University of Texas School of Public Health, Division of Epidemiology, Human Genetics, and Environmental Sciences in Houston, Texas. Marlynn May is a Professor Emeritus at the Texas A&M University System Department of Social and Behavioral Health in College Station, Texas. Linda Highfield is an Assistant Professor at the University of Texas School of Public Health, Division of Management, Policy, and Community Health. Address correspondence to: Mudita Upadhyaya, MHA, MPH, University of Texas School of Public Health, Division of Epidemiology, Human Genetics, and Environmental Sciences, 1200 Herman Pressler, Ste. E637, Houston, TX 77030; tel. 713-4746203; fax 713-500-9264; e-mail . ©2015 Association of Schools and Programs of Public Health
REFERENCES 1. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health 2010;100 Suppl 1:S40-6. 2. Padgett DK. Qualitative methods in social work research. 2nd ed. Thousand Oaks (CA): Sage Publications, Inc.; 2008. 3. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. Washington: National Institutes of Health; 2011. 4. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks (CA): Sage Publications, Inc.; 2010. 5. Johnson CE, Ali SA, Shipp MP. Building community-based participatory research partnerships with a Somali refugee community. Am J Prev Med 2009;37(6 Suppl 1):S230-6. 6. Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream: are researchers prepared? Circulation 2009;119:2633-42. 7. Ahmed SM, Beck B, Maurana CA, Newton G. Overcoming barriers to effective community-based participatory research in US medical schools. Educ Health (Abingdon) 2004;17:141-51. 8. Potter MA, Quill BE, Aglipay GS, Anderson E, Rowitz L, Smith LU, et al. Demonstrating excellence in practice-based research for public health. Public Health Rep 2006;12(Suppl 1):1-16. 9. Breast Health Collaborative of Texas. Who we are [cited 2012 Oct 23]. Available from: URL: http://www.breasthealthcollaborativeoftexas .org 10. Shen T, Wu D, Archhpiliya V, Bieber M, Hiltz R. Participatory learning approach: a research agenda. Newark (NJ): New Jersey Institute of Technology, Information Systems Department; 2004. 11. Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med 2012;42:646-54. 12. Weathers B, Barg FK, Bowman M, Briggs V, Delmoor E, Kumanyika S, et al. Using a mixed-methods approach to identify health concerns in an African American community. Am J Public Health 2011;101:2087-92. 13. Christopher S, Watts V, McCormick AK, Young S. Building and maintaining trust in a community-based participatory research partnership. Am J Public Health 2008;98:1398-406. 14. O’Fallon LR, Dearry A. Community-based participatory research as a tool to advance environmental health sciences. Environ Health Perspect 2002;110 Suppl 2:155-9. 15. Minkler M, Wallerstein N. Community-based participatory research for health: from process to outcomes. 2nd ed. Hoboken (NJ): John Wiley & Sons; 2010.
Public Health Reports / May–June 2015 / Volume 130
292 From the Schools and Programs of Public Health
16. Resnik DB, Kennedy CE. Balancing scientific and community interests in community-based participatory research. Account Res 2010;17:198-210. 17. Israel BA, Parker EA, Rowe Z, Salvatore A, Minkler M, Lopez J, et al. Community-based participatory research: lessons learned from the Centers for Children’s Environmental Health and Disease Prevention Research. Environ Health Perspect 2005;113:1463-71.
18. Rivkin I, Trimble J, Lopez ED, Johnson S, Orr E, Allen J. Disseminating research in rural Yup’ik communities: challenges and ethical considerations in moving from discovery to intervention development. Int J Circumpolar Health 2013;72. 19. Baker EA, Homan S, Schonhoff SR, Kreuter M. Principles of practice for academic/practice/community research partnerships. Am J Prev Med 1999;16(3 Suppl):86-93.
Public Health Reports / May–June 2015 / Volume 130
