RESEARCH ARTICLE
Community Engagement: Outcomes for Occupational Therapy Students, Faculty and Clients Victoria P. Schindler*† The Richard Stockton College of New Jersey, USA
Abstract Students in health care professions, including occupational therapy, are required to develop knowledge, skills and attitudes in mental health and research. Persons diagnosed with a mental illness, a learning disability or an autism-spectrum disorder desire to achieve goals in higher education and employment. Faculty in health care programmes strives to meet professional goals and accreditation and institution requirements for teaching, service and scholarship. The Bridge Program, a programme based on principles of community engagement, was developed to meet the needs of these three stakeholders. The objective of this paper is to provide programme description and outcomes of the effectiveness of the Bridge Program for all three stakeholders. This uses mixed methods research design including descriptive and quantitative and qualitative one-group pre-test–post-test designs. Instruments consisted of the Occupational Therapy Student and Mental Health Population Scale and the Canadian Occupational Performance Measure. Quantitative results support that graduate occupational therapy students gained research and clinical skills (n = 100; p = .000); clients increased performance and satisfaction toward goals (n = 113; p = .000) and faculty (n = 1) achieved goals related to teaching, service and scholarship. Programmes based on principles of community engagement can address the needs of the community, can provide outcomes that advance knowledge about community practice and can result in benefits for all stakeholders. This paper is limited to generalization and instrumentation and recommends an ongoing evaluation of other community engagement programmes involving all stakeholders in the future research. Copyright © 2014 John Wiley & Sons, Ltd. Received 13 June 2013; Revised 12 November 2013; Accepted 2 January 2014
Keywords community engagement; health care students; faculty scholarship; supported education and employment *Correspondence Victoria P. Schindler, School of Health Sciences, The Richard Stockton College of New Jersey, 101 Vera King Farris Drive, Galloway, NJ 08205, USA. †
Email: [email protected]
Published online 29 January 2014 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/oti.1364
Introduction Community engagement programmes in the health care professions within higher education provide benefits to all stakeholders: college students, clients and faculty. This study addresses a problem/need for all three stakeholders. Students in health care professions, Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
including occupational therapy, are required to develop knowledge, skills and attitudes in mental health and research (Accreditation Council for Occupational Therapy Education, 2013). Persons diagnosed with a mental illness, learning disability or an autism-spectrum disorder desire to achieve goals in higher education and employment (Adreon and Durocher, 2007; Substance 71
Community Engagement: Outcomes
Abuse and Mental Health Services Administration, 2007; Fossey and Harvey, 2010; Boston University Center for Psychiatric Rehabilitation, 2011). However, persons with those disorders often experience difficulty in achieving these goals (Arbesman and Logsdon, 2011). Faculty teaching in health care programmes strives to meet their own professional goals and accreditation and institution requirements for teaching, service and scholarship (Boyer, 1990). This paper provides a description and outcomes of a programme based on principles of community engagement with these three stakeholders. The programme is a supported education/employment programme (entitled the Bridge Program) that was developed in 2005 and was conducted on a college campus. Students are master’s-level occupational therapy students; clients are undergraduate college students or community members diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder who have higher education or employment goals and faculty is the Director of the Bridge Program/associate professor of occupational therapy. The specific purpose of this paper, therefore, is to provide outcomes supporting the effectiveness of the Bridge Program in achieving (1) occupational therapy student goals related to knowledge, skills and attitudes in mental health and research; (2) client goals related to higher education and employment; and (3) faculty goals related to teaching, service and scholarship.
Literature review Community engagement is defined as the collaboration between institutions of higher education and their larger communities (local, regional/state, national, global) for the mutually beneficial exchange of knowledge and resources in a context of partnership and reciprocity. The purpose of community engagement is the partnership of college and university knowledge and resources with those of the public and private sectors to enrich scholarship, research, and creative activity; enhance curriculum, teaching and learning; prepare educated, engaged citizens; strengthen democratic values and community responsibility; address critical societal issues; and contribute to the public good (Carnegie Foundation for the Advancement of Teaching, 2013, para. 4–5.). 72
Schindler
Best practice community engagement programmes address community needs that are supported by outcomes that advance knowledge and community practice and promote and result in mutual benefit for all community and higher-education stakeholders (i.e. students, clients and faculty) (Saltmarsh and Zlotkowski, 2011; Tett et al., 2003). The literature on community engagement programmes also uses the terms civic engagement, service learning (Eyler and Giles, 1999; Saltmarsh and Zlotkowski, 2011) and public scholarship (Scobey, 2011) to describe programmes that include students, faculty and clients; and this literature is vast. Therefore, for the purpose of this paper, the literature search was limited to programmes that met the definition of community engagement as described earlier and addressed the stakeholders represented in this paper. Several combinations of key words including community engagement, public scholarship, service learning, civic engagement, graduate students, health care programmes, research skills, faculty, mental health, psychiatric diagnosis and autism-spectrum disorders were used to search databases including pre-CINAHL, CINAHL, ERIC and PsycINFO between the years 1999 and 2012. This specific search produced a moderate amount of literature. The majority of papers addressed one of the stakeholders (students, clients or faculty), although some papers addressed more than one stakeholder.
Community engagement programmes and college students in health care professions The search in this area of the literature focused on community engagement programmes that addressed the knowledge, skills and attitudes of health care students in mental health and research. The majority of papers found addressed mental health, and although the bulk of these papers provided only programme descriptions (Stocking and Cutforth, 2006; Beltran et al., 2007; Lashley, 2007; Raiz, 2007), some of the publications provided quantitative and/or qualitative outcomes assessing students’ knowledge, skills and attitudes (Coates and Crist, 2004; Flinn et al., 2009; Atler and Gavin, 2010; Schindler, 2010). Fewer publications addressed research skills, and the majority of these were limited to programme descriptions (Brosnan et al., 2005; Rash, 2005; Portney and Applebaum, 2006; Strand, 2000; Schindler, 2010). Two publications Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
indicated that nursing students in community engagement programmes expressed greater motivation and enthusiasm for learning research in comparison with students in a traditional research curriculum (Peterson and Schaffer, 1999; Balakas and Sparks, 2010). Community engagement programmes and outcomes for clients Literature on community engagement programmes with mental health populations is limited. However, these papers suggested that clients perceive students as providing useful services (e.g. assisting clients to learn job search skills) and that the students’ skills, behaviours and attitudes reflect learning of professional work skills such as sensitivity to client needs, positive attitude and dependable (Ferrari and Worrall, 2000; Flecky and Gitlow, 2009; Schindler, 2010). These skills and behaviours addressed clients’ goals and needs that positively impacted client outcomes. Community engagement programmes and outcomes for faculty Community engagement programmes developed and implemented by faculty in the health professions can positively impact faculty teaching, service and scholarship. Community engagement programmes can bring real-life, hands-on learning to illustrate and concepts learned in the classroom. Several papers highlighted the benefits for faculty with goals of tenure and promotion, service and scholarship (Suarez-Balcazar et al., 2005; Driscoll, 2008; O’Meara, 2008; Flecky and Gitlow, 2009; Curry-Stevens, 2011; Hutchinson, 2011; Williams and Sparks, 2011; Stanton, 2012). Some of these benefits included simultaneously addressing teaching, service and scholarship requirements by embedding community engagement service programmes into curriculum and completing presentations and publications on the outcomes of these programmes. Community engagement programmes for all stakeholders: students, clients and faculty A few publications described programmes benefitting all three stakeholders. Velde et al. (2009) described occupational therapy faculty, student and community partnerships and highlighted the benefits for each stakeholder. Other literature described positive impacts Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
for all stakeholders in programmes serving low-income youth (Bazyk et al., 2010), adolescents diagnosed with autism (Gardner et al., 2012) and at-risk, urban adults (Hyde and Meyer, 2004), but they included very limited or no quantitative or qualitative outcomes. Overall, literature on outcomes of community engagement programmes in the health professions has been increasing, but literature on outcomes addressing all stakeholders is limited. This study seeks to add to the literature by reporting outcomes for all stakeholders.
Programme description The Bridge Program Because this study is on the effectiveness of the Bridge Program for all stakeholders, a programme description is provided. The Bridge Program is a supported education/employment programme developed and conducted by a master’s-level occupational therapy programme to assist persons diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder to achieve higher education or employment goals. The Bridge Program is an example of a programme based on community engagement because it addresses real problems (e.g. assisting persons with psychiatric conditions to achieve higher education and employment goals), is supported by outcomes and results in mutual benefits for all stakeholders (Tett et al., 2003; Carnegie Foundation, 2013). The Bridge Program was developed in 2005 and has continued yearly. The theoretical foundation for the Bridge Program is based on a client-centred, occupation-based approach (Law et al., 2005; Schindler, 2010). This paper reports outcomes from 2008 to 2012. The Bridge Program is embedded in two research courses in the second year of the graduate occupational therapy curriculum and was conducted fall and spring semesters, once per week for 3 hours. The programme consisted of academic group modules and one-to-one mentoring under the direction and supervision of occupational therapy faculty. Modules were presentations on academic topics, such as reading, writing and study skills, by occupational therapy student groups. One-to-one mentoring paired an individual occupational therapy student (mentor) with a client (mentee) to develop and achieve client-identified goals. Although the activities to achieve the goals varied according to each client’s needs, the mentoring process followed a standard 73
Community Engagement: Outcomes
procedure that included guidelines for an introduction, a body and a conclusion for each mentoring session. Goals typically consisted of 2–4 goals per semester and were customized to meet the needs of the client. For example, clients who were undergraduate students typically had academic and social goals related to their college experience. A common goal was time management and organization of class attendance, extracurricular activities and completion of homework related to assignments and exams. One activity to address this goal was the weekly use of a calendar to plan, prioritize and schedule completion of homework throughout the semester. The clients were individuals diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder and were undergraduate students striving to succeed in college or community members desiring to begin or return to higher education or employment. Clients were referred from the college’s office for students with disabilities, the local community college or the community mental health agencies. Prior to starting the programme, clients made a commitment to complete one semester. Attendance for a second semester was based on their choice and their availability. The students included all second-year master’s-level occupational therapy students. The students participated in instruction prior to and concurrent with programme implementation. Preparation included instruction in three mental health courses, an introductory research course and in the mentoring process (clinical assessment, goal development and intervention toward goals). This instruction is described in detail elsewhere (Schindler, 2010; Schindler, 2011). Concurrent instruction addressed the mentoring process and various areas of the research process including data collection and analysis (various pre-test and post-test) and the development of a research paper and a professional poster. The faculty was one associate professor who desired to provide a service to the college and community while simultaneously meeting requirements for teaching, service and scholarship.
Methods Participants The study had three groups of participants (1) all second-year occupational therapy students (approximately 20/year, Class of 2008 to Class of 2012); 74
Schindler
(2) approximately 20 clients who had attended the Bridge Program for at least one semester from 2008 to 2012; and (3) the faculty (the Director of the Bridge Program/associate professor of occupational therapy).
Research design The study included descriptive and one-group pre-testpost-test designs. A quantitative one-group pre-test– post-test survey design was used with the clients and the occupational therapy students, and a qualitative pre-test–post-test measure was also used with the occupational therapy students. A post-test descriptive design was used with faculty to quantify outcomes of publications and presentations. Yearly, the study received Institutional Review Board approval, and occupational therapy students and clients provided written informed consent. The informed consent forms explicitly stated a right to withdraw from the study at any time without prejudice. All data collection measures were coded or identified in methods to protect confidentiality. Data were stored on a password-protected computer or in a locked file cabinet. Instruments For the occupational therapy students, Occupational Therapy Student and Mental Health Population Scale was developed because no scale measuring knowledge, skills and attitudes, as applicable to this programme, was found (Gutman et al., 2007). This scale was developed based on factors that influence occupational therapy students’ knowledge, skills and attitudes with a mental health population (Graessle, 1997; Penny, 2001; Tsang et al., 2004), and it included 10 items on a 5-point Likert scale (1, strongly disagree; 5, strongly agree) and three open-ended questions. Test–re-test reliability and internal consistency were conducted on the scale. For test–re-test reliability, the Spearman’s rho correlation coefficient was .857 (p = .001; n = 19). For internal consistency, the Cronbach’s alpha was .799 (n = 19) (Schindler, 2011). For the clients, a self-report pre-test and post-test measuring effectiveness of the mentoring process on performance and satisfaction toward higher education/employment goals were used. This outcome measure is the Canadian Occupational Performance Measure (COPM) (Law et al., 2005), a standardized assessment that uses a semi-structured interview to Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
identify and rate performance and satisfaction in occupational performance problems on a 1–10 Likert scale. These problem areas are then developed into goals. For this study, problems were identified in the areas of higher education, employment and related areas (e.g. poor diet affecting function in higher education or employment). McColl et al. (2006) in Research on the Canadian Occupational Performance Measure: An annotated resource cited 23 studies supporting the reliability and validity of the COPM. Twelve references supported face, convergent, divergent, concurrent, construct, criterion and content validity. Seventeen studies were conducted on the utility of the COPM and supported that the COPM offered significant benefits across settings, with different cultures, and in different languages. Additionally, eight studies used the COPM with individuals diagnosed with mental illness (McColl et al., 2006; Schindler, 2010). For faculty, the number of presentations and publication were quantified.
Procedures Occupational therapy students Because the occupational therapy students began instruction in mental health in the first semester of the academic programme, it was important to assess their knowledge, skills and attitudes with a mental health population prior to and after all instruction in mental health. Therefore, data were collected using the Occupational Therapy Student and Mental Health Population Scale during the students’ first week and the last week of the 2-year academic programme. Clients The COPM (Law et al., 2005) was administered by the occupational therapy student (mentor) to the client (mentee) at the start and end of each semester. From 2008 to 2010, there were a total of 10 semesters. Faculty Faculty presentations and publications were quantified for the years 2008–2012.
Data analysis Quantitative and qualitative data analysis was conducted. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
Quantitative data analysis was conducted using IBM Statistics version 19.0 on the demographic data, the Occupational Therapy Student and Mental Health Population Scale and the COPM. Descriptive data analyses were conducted on the demographic data for the occupational therapy students and the clients. Nonparametric statistical analyses were used to determine if statistically significant differences were found from pre-test to post-test on the ordinal level data in the Occupational Therapy Student and Mental Health Population Scale and the COPM. Qualitative data analysis was completed on the three open-ended questions on the Occupational Therapy Student and Mental Health Population Scale. These questions served as the initial codes, and relevant information pertaining to each question was highlighted. Frequency counts were used for coded items to develop meaningful data “chunks” (Carpenter and Suto, 2008, p. 116; Portney and Watkins, 2009). These data chunks were organized into categories. Next, a table was developed for each category, and the corresponding data chunks were listed for each category. Once the table was completed, it was reviewed to ensure that all responses were included. In the conclusion and verification phase, a number of overarching themes emerged. Each step and decision in this iterative process was documented in the table to create an audit trail. Steps included in the data collection and analysis to increase rigour and validity included triangulation, reflexivity and an audit trail (Carpenter and Suto, 2008). SPSS
Results Graduate occupational therapy students One hundred of 104 occupational therapy students completed pre-test and post-tests on the Occupational Therapy Student and Mental Health Population Scale from 2008 to 2012. The students were 94% women, 94% Caucasian, 2% each Hispanic, African American and Asian. Ages ranged from 22 to 50 years (mean age = 28 years; SD = 6.7). The quantitative aspect of the 10-item, 5-point Likert Occupational Therapy Student and Mental Health Population Scale had a range of 10–50. Pre-test scores ranged from 20 to 47 with a mean of 36.09 (SD = 5.6). Post-test scores ranged from 27 to 50 with a mean of 42.42 (SD = 4.5). A related-sample Wilcoxon signed rank test determined a statistically significant difference between the students’ pre-test and post-test responses (p = .000), 75
Community Engagement: Outcomes
meaning students demonstrated a statistically significant increase in their knowledge, skills and attitudes with this mental health population. The qualitative aspect of the Occupational Therapy Student and Mental Health Population Scale had three open-ended questions. Responses to each question varied from no response, to a few words and up to a few sentences. For Question #1: What are your general impressions of this population? The categories of comments that emerged were positive, negative or neutral. The number of positive comments increased from 42 (pre-test) to 50 (post-test). The number of negative comments decreased from 35(pre-test) to 33 (post-test). Neutral comments decreased from eight (pre-test) to four (post-test). Examples of “Positive” comments: Pre-test: “With therapy and/or medication they can be a fully functioning member of society”. “They are regular people who just need help”. Post-test: “They are no different from other people who have goals, dreams, and desires”. Examples of “Negative” comments: Pre-test: “Their behavior is unpredictable”. “They tend to be violent when frustrated”. Post-test: “They always have something going on whether it be with themselves or their family that makes it harder for them to accomplish their goals”. Example of neutral comments on pre-test and posttest responses: It is difficult to make a generalization about psychiatric populations. For Question #2: What is your biggest concern (if any) about working with this population? The categories of comments that emerged were (1) fear of unpredictable behaviour; (2) lack of knowledge/experience; and (3) other. The number of comments related to fear of unpredictable behaviour decreased from 48 (pre-test) to 21 (post-test). The number of comments related to lack of knowledge/experience decreased from 50 (pre-test) to 34 (post-test). There were no comments in the “other category” at pre-test, but at post-test, there were two subcategories: (1) no concerns (n = 7) and (2) concern about clients’ lack of progress (n = 20). 76
Schindler
Examples of “Fear of unpredictable behavior” comments: Pre-test: “I or other people will be put in danger”. “Unpredictable outburst/violence”. Post-test: “They may become violent if they are not taking their medication on a daily basis”. Examples of “Lack of knowledge/experience” comments: Pre-test: “My ability to help them during a breakdown”. “Diagnosing the problem correctly and coming up with the effective intervention”. Post-test: “My experience and knowledge as a therapist to make sure I was doing enough to help my client”. Example of neutral comments on pre-test and posttest responses: Unsure. For Question #3: How do you feel your previous experiences have shaped your views about working with this population? The categories of comments that emerged were (1) lack of experience; (2) positive experience; and (3) neutral. The number of comments related to lack of experience decreased from 27 (pre-test) to 0 (post-test). The number of comments related to positive experience increased from 43 (pre-test) to 77 (post-test). The number of comments related to “neutral” decreased from 14 (pre-test) to 7 (post-test). Examples of “Lack of experience” comments: Pre-test: “No prior experience”. “Minimal experience with this population”. Post-test: “No comments written for lack of experience”. Examples of “Positive experience” comments: Pre-test: “I have worked with this population in the past and feel more comfortable around them”. Post-test: “I feel more comfortable with this population as a result of coming in direct contact via The Bridge Program”. Example of neutral comments on pre-test and posttest responses: “My past experiences have shown me a wide diversity among people with psychiatric disabilities”. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
For the research aspect of the courses, the students developed a research paper and professional posters that they presented to faculty, first-year students and invited guests.
Clients In this article, 113 clients completed at least one semester of the programme. The clients were 56% men and 44% women, 75% Caucasian, 20% African American and 5% Hispanic. Ages ranged from 18 to 55 years (mean age = 34 years, SD = 8.6). About half of the clients (n = 67) attended the programme for two semesters. Because the COPM was administered to each participant as pre-test/post-test each semester, there were 186 COPMs completed from 2008 to 2012. The COPM consists of two items scored on each pre-test and post-test (performance and satisfaction) according to a 10-point Likert scale, with a range of 2–20 (Law et al., 2005). Individual pre-test scores on performance ranged from 1.0 to 8.0 with a mean of 3.47 (SD = 1.2), and individual pre-test scores on satisfaction ranged from 1.0 to 9.0 with a mean of 2.96 (SD = 1.4). Individual post-test scores on performance ranged from 1.0 to 10.0 with a mean of 6.18 (SD = 1.9), and individual post-test scores on satisfaction ranged from 1.0 to 10.0 with a mean of 6.30 (SD = 2.1). The average change in performance scores from pre-test to post-test was an increase of 2.71. The average change in satisfaction scores from pre-test to post-test was an increase of 3.34. A related-sample Friedman’s two-way analysis of variance by ranks determined a statistically significant difference existed between the participants’ pre-test and post-test scores (p = .000). Additionally, of the 113 participants, 38(34%) were current college students. Thirty-two (84%) of these students remained enrolled at the college upon completion of the programme. More in-depth outcomes for all participants are reported elsewhere (Schindler and Sauerwald, 2013).
Faculty By merging work with the Bridge Program with teaching, service and scholarship responsibilities, from 2008 to 2012, I was able to complete four peer-reviewed journal publications, two book chapters, five peerreviewed presentations and eight invited presentations on this programme. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
Discussion The Bridge Program, which is based on community engagement, provided benefits to all stakeholders involved: college students, clients and faculty. Quantitative and qualitative outcomes on the Occupational Therapy Student and Mental Health Population Scale demonstrated that occupational therapy students improved knowledge and skills and developed increasingly positive attitudes with the population. The quantitative results reached statistical significance, and the qualitative results supported the quantitative results. These results add support to previous reports of community engagement programmes increasing students’ knowledge, skills and attitudes with a mental health population (Coates and Crist, 2004; Flinn et al., 2009; Atler and Gavin, 2010; Schindler, 2010). The qualitative findings support that students gained a more positive view of this population. Comments related to concerns about working with this population showed a significant decrease in fear and in lack of knowledge/experience. In fact, students had a significant increase in comments describing positive experiences and linked this to involvement in instruction, preparation and implementation of the Bridge Program. The Bridge Program was preceded by and conducted concurrently with classroom instruction, so the students were able to transfer classroom learning to implementing the modules and mentoring, including completing evaluations, interventions and re-evaluations. Although the qualitative comments reflected a positive change in knowledge, skills and attitudes, they also reflected a realistic view of the challenges that occur when lives are complicated by the symptoms and sequelae of a mental health diagnosis. Students reported more competence and confidence in their own skills but were also appropriately concerned about the extent of skills needed to address the depth of the problems. For the research aspect of the courses, the papers and poster presentations were hands-on learning activities that applied research principles to the evaluation of the Bridge Program and reflected students’ knowledge and skills in this aspect of the courses. As reported by Balakas and Sparks (2010), Peterson and Schaffer (1999), and as seen in the students in this study, learning research via a community engagement programme has positive effects on learning. The clients reported statistically significant improvement in performance and satisfaction toward 77
Community Engagement: Outcomes
their higher education and employment goals. The authors of the COPM report that an increase of at least 2 points is clinically significant (Law et al., 2005). Overall change in performance increased by 2.71 points, and overall change in satisfaction increased by 3.34 points. Progress toward goals occurred primarily via the mentoring process, whereby each client chose goals on the basis of self-identified problems in higher education or employment and systematically addressed these goals with their occupational therapy student mentor. Clients are motivated to achieve goals when they are active in identifying them and when they relate to their current problems (Law and Mills, 1998). Another important aspect of mentoring is the relationship between mentor and mentee. As reported previously and as seen in this study, clients acknowledged that students provided useful services and that their skills, behaviours and attitudes reflected professional work skills (e.g. occupational therapy students were organized and prepared for the mentoring session) (Flecky and Gitlow, 2009; Ferrari and Worrall, 2000). As the faculty member, the Bridge Program facilitated work in areas that are very important and rewarding: students’ knowledge, skills and attitudes in mental health and research and clients’ achievement in meaningful, personalized goals. It has also productively addressed teaching, service and scholarship by embedding this service programme in courses and through documenting outcomes in presentations and publications. This experience adds to previous reports of authors who highlighted the benefits for faculty with goals of tenure and promotion, service and scholarship (Suarez-Balcazar et al., 2005; Driscoll, 2008; Flecky and Gitlow, 2009; Curry-Stevens, 2011; Hutchinson, 2011; Williams and Sparks, 2011; Stanton, 2012). Limitations include generalization due to the onegroup design and instrumentation due to limited psychometrics of the Occupational Therapy Student and Mental Health Population Scale.
Conclusion Successful programmes based on principles of community engagement that address community needs are supported by outcomes that advance knowledge and community practice and result in mutual benefit for all community and higher-education stakeholders. Programmes satisfying these important areas contain a 78
Schindler
recipe for success (Tett et al., 2003; Carnegie Foundation, 2013). The Bridge Program has the components of a community engagement programme and has provided an important and necessary service to all stakeholders. Although the literature on community engagement programmes is vast, literature on the effectiveness of community engagement programmes in the health professions on stakeholders is limited. This study adds to the literature by providing outcomes for all stakeholders. Recommendations for further research include continued evaluation of community engagement programmes for all stakeholders, use of a control group where ethically indicated and evaluation of whether length of time in community engagement programmes (especially for a client population) has an impact on goal achievement.
Conflicts of interest The author declares no conflict of interest.
REFERENCES Accreditation Council for Occupational Therapy Education (ACOTE). Retrieved January 24, 2013 from http://www.aota.org/Educate/Accredit.aspx Adreon D, Durocher JS (2007). Evaluating the college transition needs of individuals with high-functioning autism spectrum disorders. Intervention in School and Clinic 42(5): 271–279. Arbesman M, Logsdon DW (2011). Occupational therapy interventions for employment and education for adults with serious mental illness: a systematic review. The American Journal of Occupational Therapy 65(3): 238–246. Atler K, Gavin WJ (2010). A mixed methods inquiry into the effect of service-learning on graduate students’ competencies. Occupational Therapy in Health Care 24(1): 23–38. DOI: 10.3109/07380570903410860 Balakas K, Sparks L (2010). Teaching research and evidence-based practice using a service-learning approach. Journal of Nursing Education 49(12): 691. Bazyk S, Gordon R, Haines J, Percociante M, Glorioso M (2010). Service learning: the process of doing and becoming an occupational therapist. Occupational Therapy in Health Care 24(3): 171–187. Beltran RO, Scanlan JN, Hancock N, Luckett T (2007). The effect of first year mental health fieldwork on attitudes of occupational therapy students towards people with mental illness. Australian Occupational Therapy Journal 54(1): 42–48. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
Boston University Center for Psychiatric Rehabilitation Systematic Review of Supported Education Literature 1989–2009 (2011) http://www.bu.edu/drrk/researchsyntheses/psychiatric-disabilities/supported-education/ [12 December 2012] Boyer EL (1990). Scholarship Reconsidered: Priorities of the Professoriate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching. Brosnan CA, Upchurch SL, Meininger JC, Hester LE, Johnsons G, Eissa MA (2005). Student nurses participate in public health research and practice through a school-based screening program. Public Health Nursing 22(3): 260–266. Carnegie Foundation for the Advancement of Teaching. (2013). Communication Description – community engagement. http://classifications.carnegiefoundation.org/ descriptions/community_engagement.php. [1May 2013] Carpenter C, Suto M (2008). Qualitative Research for Occupational and Physical Therapists: A Practical Guide. United Kingdom: Blackwell Publishing Ltd. Coates GF, Crist PA (2004). Brief or new: professional development of fieldwork students: occupational adaptation, clinical reasoning, and client-centeredness. Occupational Therapy in Health Care 18 (1/2): 39–47. Curry-Stevens A (2011). Research for change: transforming policy, scholarship, and the classroom through engaged research with communities of color. Journal of Public Scholarship in Higher Education 1(1): 11–29. Driscoll A (2008). Carnegie’s community engagement classification: intentions and insights. Change, January– February: 39–41. Eyler J, Giles D (1999). Where’s the Learning in Service Learning? San Francisco, CA: John Wiley & Sons. Ferrari J R, Worrall L (2000). Assessments by community agencies: how “the other side” sees service-learning. Michigan Journal of Community Service Learning 7: 35–40. Flecky K, Gitlow L (2009). Service-Learning in Occupational Therapy Education: Philosophy & Practice. Sudbury, MA: Jones & Bartlett Publishers. Flinn S, Kloos A, Teaford M, Clark K, Szucs K (2009). Helping hands for healthy living: a collaborative service-learning project with occupational and physical therapy students. Occupational Therapy in Health Care 23(2): 146–167. Fossey EM, Harvey CA (2010). Finding and sustaining employment: a qualitative meta-synthesis of mental health consumer views. Canadian Journal of Occupational Therapy 77(5): 303–314. Gardner J, Mulry CM, Chalik S (2012). Considering college?: adolescents with autism and learning disorders Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
participate in an on-campus service-learning program. Occupational Therapy in Health Care 26(4): 25–269. Graessle EA (1997). Influences on occupational therapy students’ attitudes about mental illness. Occupational Therapy in Mental Health 13(3): 41–61. Gutman SA, Schindler VP, Furphy KA, Klein K, Lisak JM, Durham DP (2007). The effectiveness of a supported education program: the Bridge Program. Occupational Therapy in Mental Health 23(1): 21–38. Hutchinson MC (2011). Outside the margins: promotion and tenure with a public scholarship platform. The Journal of Public Scholarship in Higher Education 1(1): 133–152. Hyde CA, Meyer M (2004). A collaborative approach to service, learning, and scholarship: a community-based research course. Journal of Community Practice 12(1/2): 71–88. Lashley M (2007). Nurses on a mission: a professional service learning experience with the inner city homeless. Nursing Education Perspectives 28(1): 24–26. Law M, Mills J (1998). Client-centred occupational therapy. In: Law M (ed). Client-Centred Occupational Therapy (37–58). Thorofare, NJ: Slack. Law M, Baptiste S, Carswell A, McColl M, Polatajko H, Pollock N (2005). Canadian Occupational Performance Measure (COPM) (4th edn). Ottawa, ON: CAOT Publications, ACE. McColl MA, Carswell A, Law M, Baptiste S, Pollock N, Polatajko H (2006). Research on the Canadian Occupational Performance Measure: An Annotated Resource. Ottawa, ON: Canadian Association of Occupational Therapists. O’Meara K (2008, Spring). Graduate education and community engagement. New Directions for Teaching and Learning 113: 27–42. Penny NH (2001). Longitudinal study of student attitudes toward people with mental illness. Occupational Therapy in Mental Health 17(2): 49–80. Peterson SJ, Schaffer MJ (1999). Service-learning: a strategy to develop group collaboration and research skills. Journal of Nursing Education 38(5): 208–214. Portney LG, Applebaum DL (2006). Integrating objectives for clinical education, research, and service-learning into community health promotion projects. Journal of Physical Therapy Education 20(3): 18–24. Portney LG, Watkins MP (2009). Foundations of Clinical Research: Application to Practice (3rd edn). Upper Saddle River, NJ: Pearson/Prentice Hall. Raiz A (2007). Service-learning in an occupation-based curriculum: student commentary. Occupational Therapy in Health Care 21(1/2): 61–69. Rash EM (2005). A service-learning research methods course. Journal of Nursing Education 44(10): 477–478. 79
Community Engagement: Outcomes
Saltmarsh J, Zlotkowski E (eds) (2011). Higher Education and Democracy: Essays on Service-Learning and Community Engagement. Philadelphia: Temple University Press. Schindler VP (2010). A client-centered, occupation-based occupational therapy programme for adults with psychiatric diagnoses. Occupational Therapy International 17(3): 105–112. Schindler VP (2011). Using service-learning to teach mental health and research skills. Occupational Therapy in Heath Care 25(1): 54–64. Schindler VP, Sauerwald C (2013). Outcomes of a 4-year program with higher education and employment goals for individuals diagnosed with mental illness. Work: A Journal of Prevention, Assessment and Rehabilitation 46(3): 325–336. Scobey D (2011). Journals and journeys: new horizons in public scholarship. The Journal of Public Scholarship in Higher Education 1 (1): 1–10. Stanton TK (2012). New times demand new scholarship II: research universities and community engagement: opportunities and challenges. Journal of Higher Education Outreach and Engagement 16(4): 271–304. Stocking VB, Cutforth N (2006). Managing the challenges of teaching community-based research courses: insights from two instructors. Michigan Journal of Community Service Learning 13(1): 56–65. Strand KJ (2000). Community-based research as pedagogy. Michigan Journal of Community Service Learning 7: 85–96.
80
Schindler
Suarez-Balcazar Y, Hammel J, Helfrich C, Thomas J, Wilson T, Head-Ball D (2005). A model of universitycommunity partnerships for occupational therapy scholarship and practice. Occupational Therapy in Health Care 21(1-2): 47–70. Substance Abuse and Mental Health Services Administration, Center for Mental Health Services (2007). Promoting independence and recovery through work: employment for people with psychiatric disabilities: 1–8. Tett L, Crowther J, O’Hara P (2003). Collaborative partnerships in community education. Journal of Education Policy 18(1): 37–51. Tsang HWH, Chan F, Chan CCH (2004). Factors influencing occupational therapy students’ attitudes towards persons with disabilities: a conjoint analysis. American Journal of Occupational Therapy 58(5): 426–434. Velde BP, Davis DT, Grant GR (2009). Who sits at the head of the table? a reflection on community, student, and faculty partnerships. In Flecky K, Gitlow L (eds) Service-Learning in Occupational Therapy Education: Philosophy & Practice Sudbury (37–58). MA: Jones & Bartlett Publishers. Williams PH, Sparks J (2011). Collaborative inquiry at a children’s museum: benefits for student learning, museum outcomes, and faculty scholarship. Journal of Public Scholarship in Higher Education 1(1): 31–46.
Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes for Occupational Therapy Students, Faculty and Clients Victoria P. Schindler*† The Richard Stockton College of New Jersey, USA
Abstract Students in health care professions, including occupational therapy, are required to develop knowledge, skills and attitudes in mental health and research. Persons diagnosed with a mental illness, a learning disability or an autism-spectrum disorder desire to achieve goals in higher education and employment. Faculty in health care programmes strives to meet professional goals and accreditation and institution requirements for teaching, service and scholarship. The Bridge Program, a programme based on principles of community engagement, was developed to meet the needs of these three stakeholders. The objective of this paper is to provide programme description and outcomes of the effectiveness of the Bridge Program for all three stakeholders. This uses mixed methods research design including descriptive and quantitative and qualitative one-group pre-test–post-test designs. Instruments consisted of the Occupational Therapy Student and Mental Health Population Scale and the Canadian Occupational Performance Measure. Quantitative results support that graduate occupational therapy students gained research and clinical skills (n = 100; p = .000); clients increased performance and satisfaction toward goals (n = 113; p = .000) and faculty (n = 1) achieved goals related to teaching, service and scholarship. Programmes based on principles of community engagement can address the needs of the community, can provide outcomes that advance knowledge about community practice and can result in benefits for all stakeholders. This paper is limited to generalization and instrumentation and recommends an ongoing evaluation of other community engagement programmes involving all stakeholders in the future research. Copyright © 2014 John Wiley & Sons, Ltd. Received 13 June 2013; Revised 12 November 2013; Accepted 2 January 2014
Keywords community engagement; health care students; faculty scholarship; supported education and employment *Correspondence Victoria P. Schindler, School of Health Sciences, The Richard Stockton College of New Jersey, 101 Vera King Farris Drive, Galloway, NJ 08205, USA. †
Email: [email protected]
Published online 29 January 2014 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/oti.1364
Introduction Community engagement programmes in the health care professions within higher education provide benefits to all stakeholders: college students, clients and faculty. This study addresses a problem/need for all three stakeholders. Students in health care professions, Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
including occupational therapy, are required to develop knowledge, skills and attitudes in mental health and research (Accreditation Council for Occupational Therapy Education, 2013). Persons diagnosed with a mental illness, learning disability or an autism-spectrum disorder desire to achieve goals in higher education and employment (Adreon and Durocher, 2007; Substance 71
Community Engagement: Outcomes
Abuse and Mental Health Services Administration, 2007; Fossey and Harvey, 2010; Boston University Center for Psychiatric Rehabilitation, 2011). However, persons with those disorders often experience difficulty in achieving these goals (Arbesman and Logsdon, 2011). Faculty teaching in health care programmes strives to meet their own professional goals and accreditation and institution requirements for teaching, service and scholarship (Boyer, 1990). This paper provides a description and outcomes of a programme based on principles of community engagement with these three stakeholders. The programme is a supported education/employment programme (entitled the Bridge Program) that was developed in 2005 and was conducted on a college campus. Students are master’s-level occupational therapy students; clients are undergraduate college students or community members diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder who have higher education or employment goals and faculty is the Director of the Bridge Program/associate professor of occupational therapy. The specific purpose of this paper, therefore, is to provide outcomes supporting the effectiveness of the Bridge Program in achieving (1) occupational therapy student goals related to knowledge, skills and attitudes in mental health and research; (2) client goals related to higher education and employment; and (3) faculty goals related to teaching, service and scholarship.
Literature review Community engagement is defined as the collaboration between institutions of higher education and their larger communities (local, regional/state, national, global) for the mutually beneficial exchange of knowledge and resources in a context of partnership and reciprocity. The purpose of community engagement is the partnership of college and university knowledge and resources with those of the public and private sectors to enrich scholarship, research, and creative activity; enhance curriculum, teaching and learning; prepare educated, engaged citizens; strengthen democratic values and community responsibility; address critical societal issues; and contribute to the public good (Carnegie Foundation for the Advancement of Teaching, 2013, para. 4–5.). 72
Schindler
Best practice community engagement programmes address community needs that are supported by outcomes that advance knowledge and community practice and promote and result in mutual benefit for all community and higher-education stakeholders (i.e. students, clients and faculty) (Saltmarsh and Zlotkowski, 2011; Tett et al., 2003). The literature on community engagement programmes also uses the terms civic engagement, service learning (Eyler and Giles, 1999; Saltmarsh and Zlotkowski, 2011) and public scholarship (Scobey, 2011) to describe programmes that include students, faculty and clients; and this literature is vast. Therefore, for the purpose of this paper, the literature search was limited to programmes that met the definition of community engagement as described earlier and addressed the stakeholders represented in this paper. Several combinations of key words including community engagement, public scholarship, service learning, civic engagement, graduate students, health care programmes, research skills, faculty, mental health, psychiatric diagnosis and autism-spectrum disorders were used to search databases including pre-CINAHL, CINAHL, ERIC and PsycINFO between the years 1999 and 2012. This specific search produced a moderate amount of literature. The majority of papers addressed one of the stakeholders (students, clients or faculty), although some papers addressed more than one stakeholder.
Community engagement programmes and college students in health care professions The search in this area of the literature focused on community engagement programmes that addressed the knowledge, skills and attitudes of health care students in mental health and research. The majority of papers found addressed mental health, and although the bulk of these papers provided only programme descriptions (Stocking and Cutforth, 2006; Beltran et al., 2007; Lashley, 2007; Raiz, 2007), some of the publications provided quantitative and/or qualitative outcomes assessing students’ knowledge, skills and attitudes (Coates and Crist, 2004; Flinn et al., 2009; Atler and Gavin, 2010; Schindler, 2010). Fewer publications addressed research skills, and the majority of these were limited to programme descriptions (Brosnan et al., 2005; Rash, 2005; Portney and Applebaum, 2006; Strand, 2000; Schindler, 2010). Two publications Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
indicated that nursing students in community engagement programmes expressed greater motivation and enthusiasm for learning research in comparison with students in a traditional research curriculum (Peterson and Schaffer, 1999; Balakas and Sparks, 2010). Community engagement programmes and outcomes for clients Literature on community engagement programmes with mental health populations is limited. However, these papers suggested that clients perceive students as providing useful services (e.g. assisting clients to learn job search skills) and that the students’ skills, behaviours and attitudes reflect learning of professional work skills such as sensitivity to client needs, positive attitude and dependable (Ferrari and Worrall, 2000; Flecky and Gitlow, 2009; Schindler, 2010). These skills and behaviours addressed clients’ goals and needs that positively impacted client outcomes. Community engagement programmes and outcomes for faculty Community engagement programmes developed and implemented by faculty in the health professions can positively impact faculty teaching, service and scholarship. Community engagement programmes can bring real-life, hands-on learning to illustrate and concepts learned in the classroom. Several papers highlighted the benefits for faculty with goals of tenure and promotion, service and scholarship (Suarez-Balcazar et al., 2005; Driscoll, 2008; O’Meara, 2008; Flecky and Gitlow, 2009; Curry-Stevens, 2011; Hutchinson, 2011; Williams and Sparks, 2011; Stanton, 2012). Some of these benefits included simultaneously addressing teaching, service and scholarship requirements by embedding community engagement service programmes into curriculum and completing presentations and publications on the outcomes of these programmes. Community engagement programmes for all stakeholders: students, clients and faculty A few publications described programmes benefitting all three stakeholders. Velde et al. (2009) described occupational therapy faculty, student and community partnerships and highlighted the benefits for each stakeholder. Other literature described positive impacts Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
for all stakeholders in programmes serving low-income youth (Bazyk et al., 2010), adolescents diagnosed with autism (Gardner et al., 2012) and at-risk, urban adults (Hyde and Meyer, 2004), but they included very limited or no quantitative or qualitative outcomes. Overall, literature on outcomes of community engagement programmes in the health professions has been increasing, but literature on outcomes addressing all stakeholders is limited. This study seeks to add to the literature by reporting outcomes for all stakeholders.
Programme description The Bridge Program Because this study is on the effectiveness of the Bridge Program for all stakeholders, a programme description is provided. The Bridge Program is a supported education/employment programme developed and conducted by a master’s-level occupational therapy programme to assist persons diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder to achieve higher education or employment goals. The Bridge Program is an example of a programme based on community engagement because it addresses real problems (e.g. assisting persons with psychiatric conditions to achieve higher education and employment goals), is supported by outcomes and results in mutual benefits for all stakeholders (Tett et al., 2003; Carnegie Foundation, 2013). The Bridge Program was developed in 2005 and has continued yearly. The theoretical foundation for the Bridge Program is based on a client-centred, occupation-based approach (Law et al., 2005; Schindler, 2010). This paper reports outcomes from 2008 to 2012. The Bridge Program is embedded in two research courses in the second year of the graduate occupational therapy curriculum and was conducted fall and spring semesters, once per week for 3 hours. The programme consisted of academic group modules and one-to-one mentoring under the direction and supervision of occupational therapy faculty. Modules were presentations on academic topics, such as reading, writing and study skills, by occupational therapy student groups. One-to-one mentoring paired an individual occupational therapy student (mentor) with a client (mentee) to develop and achieve client-identified goals. Although the activities to achieve the goals varied according to each client’s needs, the mentoring process followed a standard 73
Community Engagement: Outcomes
procedure that included guidelines for an introduction, a body and a conclusion for each mentoring session. Goals typically consisted of 2–4 goals per semester and were customized to meet the needs of the client. For example, clients who were undergraduate students typically had academic and social goals related to their college experience. A common goal was time management and organization of class attendance, extracurricular activities and completion of homework related to assignments and exams. One activity to address this goal was the weekly use of a calendar to plan, prioritize and schedule completion of homework throughout the semester. The clients were individuals diagnosed with a mental illness, a learning disability and/or an autism-spectrum disorder and were undergraduate students striving to succeed in college or community members desiring to begin or return to higher education or employment. Clients were referred from the college’s office for students with disabilities, the local community college or the community mental health agencies. Prior to starting the programme, clients made a commitment to complete one semester. Attendance for a second semester was based on their choice and their availability. The students included all second-year master’s-level occupational therapy students. The students participated in instruction prior to and concurrent with programme implementation. Preparation included instruction in three mental health courses, an introductory research course and in the mentoring process (clinical assessment, goal development and intervention toward goals). This instruction is described in detail elsewhere (Schindler, 2010; Schindler, 2011). Concurrent instruction addressed the mentoring process and various areas of the research process including data collection and analysis (various pre-test and post-test) and the development of a research paper and a professional poster. The faculty was one associate professor who desired to provide a service to the college and community while simultaneously meeting requirements for teaching, service and scholarship.
Methods Participants The study had three groups of participants (1) all second-year occupational therapy students (approximately 20/year, Class of 2008 to Class of 2012); 74
Schindler
(2) approximately 20 clients who had attended the Bridge Program for at least one semester from 2008 to 2012; and (3) the faculty (the Director of the Bridge Program/associate professor of occupational therapy).
Research design The study included descriptive and one-group pre-testpost-test designs. A quantitative one-group pre-test– post-test survey design was used with the clients and the occupational therapy students, and a qualitative pre-test–post-test measure was also used with the occupational therapy students. A post-test descriptive design was used with faculty to quantify outcomes of publications and presentations. Yearly, the study received Institutional Review Board approval, and occupational therapy students and clients provided written informed consent. The informed consent forms explicitly stated a right to withdraw from the study at any time without prejudice. All data collection measures were coded or identified in methods to protect confidentiality. Data were stored on a password-protected computer or in a locked file cabinet. Instruments For the occupational therapy students, Occupational Therapy Student and Mental Health Population Scale was developed because no scale measuring knowledge, skills and attitudes, as applicable to this programme, was found (Gutman et al., 2007). This scale was developed based on factors that influence occupational therapy students’ knowledge, skills and attitudes with a mental health population (Graessle, 1997; Penny, 2001; Tsang et al., 2004), and it included 10 items on a 5-point Likert scale (1, strongly disagree; 5, strongly agree) and three open-ended questions. Test–re-test reliability and internal consistency were conducted on the scale. For test–re-test reliability, the Spearman’s rho correlation coefficient was .857 (p = .001; n = 19). For internal consistency, the Cronbach’s alpha was .799 (n = 19) (Schindler, 2011). For the clients, a self-report pre-test and post-test measuring effectiveness of the mentoring process on performance and satisfaction toward higher education/employment goals were used. This outcome measure is the Canadian Occupational Performance Measure (COPM) (Law et al., 2005), a standardized assessment that uses a semi-structured interview to Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
identify and rate performance and satisfaction in occupational performance problems on a 1–10 Likert scale. These problem areas are then developed into goals. For this study, problems were identified in the areas of higher education, employment and related areas (e.g. poor diet affecting function in higher education or employment). McColl et al. (2006) in Research on the Canadian Occupational Performance Measure: An annotated resource cited 23 studies supporting the reliability and validity of the COPM. Twelve references supported face, convergent, divergent, concurrent, construct, criterion and content validity. Seventeen studies were conducted on the utility of the COPM and supported that the COPM offered significant benefits across settings, with different cultures, and in different languages. Additionally, eight studies used the COPM with individuals diagnosed with mental illness (McColl et al., 2006; Schindler, 2010). For faculty, the number of presentations and publication were quantified.
Procedures Occupational therapy students Because the occupational therapy students began instruction in mental health in the first semester of the academic programme, it was important to assess their knowledge, skills and attitudes with a mental health population prior to and after all instruction in mental health. Therefore, data were collected using the Occupational Therapy Student and Mental Health Population Scale during the students’ first week and the last week of the 2-year academic programme. Clients The COPM (Law et al., 2005) was administered by the occupational therapy student (mentor) to the client (mentee) at the start and end of each semester. From 2008 to 2010, there were a total of 10 semesters. Faculty Faculty presentations and publications were quantified for the years 2008–2012.
Data analysis Quantitative and qualitative data analysis was conducted. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
Quantitative data analysis was conducted using IBM Statistics version 19.0 on the demographic data, the Occupational Therapy Student and Mental Health Population Scale and the COPM. Descriptive data analyses were conducted on the demographic data for the occupational therapy students and the clients. Nonparametric statistical analyses were used to determine if statistically significant differences were found from pre-test to post-test on the ordinal level data in the Occupational Therapy Student and Mental Health Population Scale and the COPM. Qualitative data analysis was completed on the three open-ended questions on the Occupational Therapy Student and Mental Health Population Scale. These questions served as the initial codes, and relevant information pertaining to each question was highlighted. Frequency counts were used for coded items to develop meaningful data “chunks” (Carpenter and Suto, 2008, p. 116; Portney and Watkins, 2009). These data chunks were organized into categories. Next, a table was developed for each category, and the corresponding data chunks were listed for each category. Once the table was completed, it was reviewed to ensure that all responses were included. In the conclusion and verification phase, a number of overarching themes emerged. Each step and decision in this iterative process was documented in the table to create an audit trail. Steps included in the data collection and analysis to increase rigour and validity included triangulation, reflexivity and an audit trail (Carpenter and Suto, 2008). SPSS
Results Graduate occupational therapy students One hundred of 104 occupational therapy students completed pre-test and post-tests on the Occupational Therapy Student and Mental Health Population Scale from 2008 to 2012. The students were 94% women, 94% Caucasian, 2% each Hispanic, African American and Asian. Ages ranged from 22 to 50 years (mean age = 28 years; SD = 6.7). The quantitative aspect of the 10-item, 5-point Likert Occupational Therapy Student and Mental Health Population Scale had a range of 10–50. Pre-test scores ranged from 20 to 47 with a mean of 36.09 (SD = 5.6). Post-test scores ranged from 27 to 50 with a mean of 42.42 (SD = 4.5). A related-sample Wilcoxon signed rank test determined a statistically significant difference between the students’ pre-test and post-test responses (p = .000), 75
Community Engagement: Outcomes
meaning students demonstrated a statistically significant increase in their knowledge, skills and attitudes with this mental health population. The qualitative aspect of the Occupational Therapy Student and Mental Health Population Scale had three open-ended questions. Responses to each question varied from no response, to a few words and up to a few sentences. For Question #1: What are your general impressions of this population? The categories of comments that emerged were positive, negative or neutral. The number of positive comments increased from 42 (pre-test) to 50 (post-test). The number of negative comments decreased from 35(pre-test) to 33 (post-test). Neutral comments decreased from eight (pre-test) to four (post-test). Examples of “Positive” comments: Pre-test: “With therapy and/or medication they can be a fully functioning member of society”. “They are regular people who just need help”. Post-test: “They are no different from other people who have goals, dreams, and desires”. Examples of “Negative” comments: Pre-test: “Their behavior is unpredictable”. “They tend to be violent when frustrated”. Post-test: “They always have something going on whether it be with themselves or their family that makes it harder for them to accomplish their goals”. Example of neutral comments on pre-test and posttest responses: It is difficult to make a generalization about psychiatric populations. For Question #2: What is your biggest concern (if any) about working with this population? The categories of comments that emerged were (1) fear of unpredictable behaviour; (2) lack of knowledge/experience; and (3) other. The number of comments related to fear of unpredictable behaviour decreased from 48 (pre-test) to 21 (post-test). The number of comments related to lack of knowledge/experience decreased from 50 (pre-test) to 34 (post-test). There were no comments in the “other category” at pre-test, but at post-test, there were two subcategories: (1) no concerns (n = 7) and (2) concern about clients’ lack of progress (n = 20). 76
Schindler
Examples of “Fear of unpredictable behavior” comments: Pre-test: “I or other people will be put in danger”. “Unpredictable outburst/violence”. Post-test: “They may become violent if they are not taking their medication on a daily basis”. Examples of “Lack of knowledge/experience” comments: Pre-test: “My ability to help them during a breakdown”. “Diagnosing the problem correctly and coming up with the effective intervention”. Post-test: “My experience and knowledge as a therapist to make sure I was doing enough to help my client”. Example of neutral comments on pre-test and posttest responses: Unsure. For Question #3: How do you feel your previous experiences have shaped your views about working with this population? The categories of comments that emerged were (1) lack of experience; (2) positive experience; and (3) neutral. The number of comments related to lack of experience decreased from 27 (pre-test) to 0 (post-test). The number of comments related to positive experience increased from 43 (pre-test) to 77 (post-test). The number of comments related to “neutral” decreased from 14 (pre-test) to 7 (post-test). Examples of “Lack of experience” comments: Pre-test: “No prior experience”. “Minimal experience with this population”. Post-test: “No comments written for lack of experience”. Examples of “Positive experience” comments: Pre-test: “I have worked with this population in the past and feel more comfortable around them”. Post-test: “I feel more comfortable with this population as a result of coming in direct contact via The Bridge Program”. Example of neutral comments on pre-test and posttest responses: “My past experiences have shown me a wide diversity among people with psychiatric disabilities”. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
For the research aspect of the courses, the students developed a research paper and professional posters that they presented to faculty, first-year students and invited guests.
Clients In this article, 113 clients completed at least one semester of the programme. The clients were 56% men and 44% women, 75% Caucasian, 20% African American and 5% Hispanic. Ages ranged from 18 to 55 years (mean age = 34 years, SD = 8.6). About half of the clients (n = 67) attended the programme for two semesters. Because the COPM was administered to each participant as pre-test/post-test each semester, there were 186 COPMs completed from 2008 to 2012. The COPM consists of two items scored on each pre-test and post-test (performance and satisfaction) according to a 10-point Likert scale, with a range of 2–20 (Law et al., 2005). Individual pre-test scores on performance ranged from 1.0 to 8.0 with a mean of 3.47 (SD = 1.2), and individual pre-test scores on satisfaction ranged from 1.0 to 9.0 with a mean of 2.96 (SD = 1.4). Individual post-test scores on performance ranged from 1.0 to 10.0 with a mean of 6.18 (SD = 1.9), and individual post-test scores on satisfaction ranged from 1.0 to 10.0 with a mean of 6.30 (SD = 2.1). The average change in performance scores from pre-test to post-test was an increase of 2.71. The average change in satisfaction scores from pre-test to post-test was an increase of 3.34. A related-sample Friedman’s two-way analysis of variance by ranks determined a statistically significant difference existed between the participants’ pre-test and post-test scores (p = .000). Additionally, of the 113 participants, 38(34%) were current college students. Thirty-two (84%) of these students remained enrolled at the college upon completion of the programme. More in-depth outcomes for all participants are reported elsewhere (Schindler and Sauerwald, 2013).
Faculty By merging work with the Bridge Program with teaching, service and scholarship responsibilities, from 2008 to 2012, I was able to complete four peer-reviewed journal publications, two book chapters, five peerreviewed presentations and eight invited presentations on this programme. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
Discussion The Bridge Program, which is based on community engagement, provided benefits to all stakeholders involved: college students, clients and faculty. Quantitative and qualitative outcomes on the Occupational Therapy Student and Mental Health Population Scale demonstrated that occupational therapy students improved knowledge and skills and developed increasingly positive attitudes with the population. The quantitative results reached statistical significance, and the qualitative results supported the quantitative results. These results add support to previous reports of community engagement programmes increasing students’ knowledge, skills and attitudes with a mental health population (Coates and Crist, 2004; Flinn et al., 2009; Atler and Gavin, 2010; Schindler, 2010). The qualitative findings support that students gained a more positive view of this population. Comments related to concerns about working with this population showed a significant decrease in fear and in lack of knowledge/experience. In fact, students had a significant increase in comments describing positive experiences and linked this to involvement in instruction, preparation and implementation of the Bridge Program. The Bridge Program was preceded by and conducted concurrently with classroom instruction, so the students were able to transfer classroom learning to implementing the modules and mentoring, including completing evaluations, interventions and re-evaluations. Although the qualitative comments reflected a positive change in knowledge, skills and attitudes, they also reflected a realistic view of the challenges that occur when lives are complicated by the symptoms and sequelae of a mental health diagnosis. Students reported more competence and confidence in their own skills but were also appropriately concerned about the extent of skills needed to address the depth of the problems. For the research aspect of the courses, the papers and poster presentations were hands-on learning activities that applied research principles to the evaluation of the Bridge Program and reflected students’ knowledge and skills in this aspect of the courses. As reported by Balakas and Sparks (2010), Peterson and Schaffer (1999), and as seen in the students in this study, learning research via a community engagement programme has positive effects on learning. The clients reported statistically significant improvement in performance and satisfaction toward 77
Community Engagement: Outcomes
their higher education and employment goals. The authors of the COPM report that an increase of at least 2 points is clinically significant (Law et al., 2005). Overall change in performance increased by 2.71 points, and overall change in satisfaction increased by 3.34 points. Progress toward goals occurred primarily via the mentoring process, whereby each client chose goals on the basis of self-identified problems in higher education or employment and systematically addressed these goals with their occupational therapy student mentor. Clients are motivated to achieve goals when they are active in identifying them and when they relate to their current problems (Law and Mills, 1998). Another important aspect of mentoring is the relationship between mentor and mentee. As reported previously and as seen in this study, clients acknowledged that students provided useful services and that their skills, behaviours and attitudes reflected professional work skills (e.g. occupational therapy students were organized and prepared for the mentoring session) (Flecky and Gitlow, 2009; Ferrari and Worrall, 2000). As the faculty member, the Bridge Program facilitated work in areas that are very important and rewarding: students’ knowledge, skills and attitudes in mental health and research and clients’ achievement in meaningful, personalized goals. It has also productively addressed teaching, service and scholarship by embedding this service programme in courses and through documenting outcomes in presentations and publications. This experience adds to previous reports of authors who highlighted the benefits for faculty with goals of tenure and promotion, service and scholarship (Suarez-Balcazar et al., 2005; Driscoll, 2008; Flecky and Gitlow, 2009; Curry-Stevens, 2011; Hutchinson, 2011; Williams and Sparks, 2011; Stanton, 2012). Limitations include generalization due to the onegroup design and instrumentation due to limited psychometrics of the Occupational Therapy Student and Mental Health Population Scale.
Conclusion Successful programmes based on principles of community engagement that address community needs are supported by outcomes that advance knowledge and community practice and result in mutual benefit for all community and higher-education stakeholders. Programmes satisfying these important areas contain a 78
Schindler
recipe for success (Tett et al., 2003; Carnegie Foundation, 2013). The Bridge Program has the components of a community engagement programme and has provided an important and necessary service to all stakeholders. Although the literature on community engagement programmes is vast, literature on the effectiveness of community engagement programmes in the health professions on stakeholders is limited. This study adds to the literature by providing outcomes for all stakeholders. Recommendations for further research include continued evaluation of community engagement programmes for all stakeholders, use of a control group where ethically indicated and evaluation of whether length of time in community engagement programmes (especially for a client population) has an impact on goal achievement.
Conflicts of interest The author declares no conflict of interest.
REFERENCES Accreditation Council for Occupational Therapy Education (ACOTE). Retrieved January 24, 2013 from http://www.aota.org/Educate/Accredit.aspx Adreon D, Durocher JS (2007). Evaluating the college transition needs of individuals with high-functioning autism spectrum disorders. Intervention in School and Clinic 42(5): 271–279. Arbesman M, Logsdon DW (2011). Occupational therapy interventions for employment and education for adults with serious mental illness: a systematic review. The American Journal of Occupational Therapy 65(3): 238–246. Atler K, Gavin WJ (2010). A mixed methods inquiry into the effect of service-learning on graduate students’ competencies. Occupational Therapy in Health Care 24(1): 23–38. DOI: 10.3109/07380570903410860 Balakas K, Sparks L (2010). Teaching research and evidence-based practice using a service-learning approach. Journal of Nursing Education 49(12): 691. Bazyk S, Gordon R, Haines J, Percociante M, Glorioso M (2010). Service learning: the process of doing and becoming an occupational therapist. Occupational Therapy in Health Care 24(3): 171–187. Beltran RO, Scanlan JN, Hancock N, Luckett T (2007). The effect of first year mental health fieldwork on attitudes of occupational therapy students towards people with mental illness. Australian Occupational Therapy Journal 54(1): 42–48. Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Schindler
Boston University Center for Psychiatric Rehabilitation Systematic Review of Supported Education Literature 1989–2009 (2011) http://www.bu.edu/drrk/researchsyntheses/psychiatric-disabilities/supported-education/ [12 December 2012] Boyer EL (1990). Scholarship Reconsidered: Priorities of the Professoriate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching. Brosnan CA, Upchurch SL, Meininger JC, Hester LE, Johnsons G, Eissa MA (2005). Student nurses participate in public health research and practice through a school-based screening program. Public Health Nursing 22(3): 260–266. Carnegie Foundation for the Advancement of Teaching. (2013). Communication Description – community engagement. http://classifications.carnegiefoundation.org/ descriptions/community_engagement.php. [1May 2013] Carpenter C, Suto M (2008). Qualitative Research for Occupational and Physical Therapists: A Practical Guide. United Kingdom: Blackwell Publishing Ltd. Coates GF, Crist PA (2004). Brief or new: professional development of fieldwork students: occupational adaptation, clinical reasoning, and client-centeredness. Occupational Therapy in Health Care 18 (1/2): 39–47. Curry-Stevens A (2011). Research for change: transforming policy, scholarship, and the classroom through engaged research with communities of color. Journal of Public Scholarship in Higher Education 1(1): 11–29. Driscoll A (2008). Carnegie’s community engagement classification: intentions and insights. Change, January– February: 39–41. Eyler J, Giles D (1999). Where’s the Learning in Service Learning? San Francisco, CA: John Wiley & Sons. Ferrari J R, Worrall L (2000). Assessments by community agencies: how “the other side” sees service-learning. Michigan Journal of Community Service Learning 7: 35–40. Flecky K, Gitlow L (2009). Service-Learning in Occupational Therapy Education: Philosophy & Practice. Sudbury, MA: Jones & Bartlett Publishers. Flinn S, Kloos A, Teaford M, Clark K, Szucs K (2009). Helping hands for healthy living: a collaborative service-learning project with occupational and physical therapy students. Occupational Therapy in Health Care 23(2): 146–167. Fossey EM, Harvey CA (2010). Finding and sustaining employment: a qualitative meta-synthesis of mental health consumer views. Canadian Journal of Occupational Therapy 77(5): 303–314. Gardner J, Mulry CM, Chalik S (2012). Considering college?: adolescents with autism and learning disorders Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
Community Engagement: Outcomes
participate in an on-campus service-learning program. Occupational Therapy in Health Care 26(4): 25–269. Graessle EA (1997). Influences on occupational therapy students’ attitudes about mental illness. Occupational Therapy in Mental Health 13(3): 41–61. Gutman SA, Schindler VP, Furphy KA, Klein K, Lisak JM, Durham DP (2007). The effectiveness of a supported education program: the Bridge Program. Occupational Therapy in Mental Health 23(1): 21–38. Hutchinson MC (2011). Outside the margins: promotion and tenure with a public scholarship platform. The Journal of Public Scholarship in Higher Education 1(1): 133–152. Hyde CA, Meyer M (2004). A collaborative approach to service, learning, and scholarship: a community-based research course. Journal of Community Practice 12(1/2): 71–88. Lashley M (2007). Nurses on a mission: a professional service learning experience with the inner city homeless. Nursing Education Perspectives 28(1): 24–26. Law M, Mills J (1998). Client-centred occupational therapy. In: Law M (ed). Client-Centred Occupational Therapy (37–58). Thorofare, NJ: Slack. Law M, Baptiste S, Carswell A, McColl M, Polatajko H, Pollock N (2005). Canadian Occupational Performance Measure (COPM) (4th edn). Ottawa, ON: CAOT Publications, ACE. McColl MA, Carswell A, Law M, Baptiste S, Pollock N, Polatajko H (2006). Research on the Canadian Occupational Performance Measure: An Annotated Resource. Ottawa, ON: Canadian Association of Occupational Therapists. O’Meara K (2008, Spring). Graduate education and community engagement. New Directions for Teaching and Learning 113: 27–42. Penny NH (2001). Longitudinal study of student attitudes toward people with mental illness. Occupational Therapy in Mental Health 17(2): 49–80. Peterson SJ, Schaffer MJ (1999). Service-learning: a strategy to develop group collaboration and research skills. Journal of Nursing Education 38(5): 208–214. Portney LG, Applebaum DL (2006). Integrating objectives for clinical education, research, and service-learning into community health promotion projects. Journal of Physical Therapy Education 20(3): 18–24. Portney LG, Watkins MP (2009). Foundations of Clinical Research: Application to Practice (3rd edn). Upper Saddle River, NJ: Pearson/Prentice Hall. Raiz A (2007). Service-learning in an occupation-based curriculum: student commentary. Occupational Therapy in Health Care 21(1/2): 61–69. Rash EM (2005). A service-learning research methods course. Journal of Nursing Education 44(10): 477–478. 79
Community Engagement: Outcomes
Saltmarsh J, Zlotkowski E (eds) (2011). Higher Education and Democracy: Essays on Service-Learning and Community Engagement. Philadelphia: Temple University Press. Schindler VP (2010). A client-centered, occupation-based occupational therapy programme for adults with psychiatric diagnoses. Occupational Therapy International 17(3): 105–112. Schindler VP (2011). Using service-learning to teach mental health and research skills. Occupational Therapy in Heath Care 25(1): 54–64. Schindler VP, Sauerwald C (2013). Outcomes of a 4-year program with higher education and employment goals for individuals diagnosed with mental illness. Work: A Journal of Prevention, Assessment and Rehabilitation 46(3): 325–336. Scobey D (2011). Journals and journeys: new horizons in public scholarship. The Journal of Public Scholarship in Higher Education 1 (1): 1–10. Stanton TK (2012). New times demand new scholarship II: research universities and community engagement: opportunities and challenges. Journal of Higher Education Outreach and Engagement 16(4): 271–304. Stocking VB, Cutforth N (2006). Managing the challenges of teaching community-based research courses: insights from two instructors. Michigan Journal of Community Service Learning 13(1): 56–65. Strand KJ (2000). Community-based research as pedagogy. Michigan Journal of Community Service Learning 7: 85–96.
80
Schindler
Suarez-Balcazar Y, Hammel J, Helfrich C, Thomas J, Wilson T, Head-Ball D (2005). A model of universitycommunity partnerships for occupational therapy scholarship and practice. Occupational Therapy in Health Care 21(1-2): 47–70. Substance Abuse and Mental Health Services Administration, Center for Mental Health Services (2007). Promoting independence and recovery through work: employment for people with psychiatric disabilities: 1–8. Tett L, Crowther J, O’Hara P (2003). Collaborative partnerships in community education. Journal of Education Policy 18(1): 37–51. Tsang HWH, Chan F, Chan CCH (2004). Factors influencing occupational therapy students’ attitudes towards persons with disabilities: a conjoint analysis. American Journal of Occupational Therapy 58(5): 426–434. Velde BP, Davis DT, Grant GR (2009). Who sits at the head of the table? a reflection on community, student, and faculty partnerships. In Flecky K, Gitlow L (eds) Service-Learning in Occupational Therapy Education: Philosophy & Practice Sudbury (37–58). MA: Jones & Bartlett Publishers. Williams PH, Sparks J (2011). Collaborative inquiry at a children’s museum: benefits for student learning, museum outcomes, and faculty scholarship. Journal of Public Scholarship in Higher Education 1(1): 31–46.
Occup. Ther. Int. 21 (2014) 71–80 © 2014 John Wiley & Sons, Ltd.
