Research and Services Partnerships
A Partnership to Develop a UtilizationFocused Evaluation Plan for a Criminal Justice Diversion Program Maureen Rubin, Ph.D., M.S.W. Leanne Fiftal Alarid, Ph.D. Mary Jo Rodriguez, Ph.D.
More than 100,000 offenders in the Texas correctional system have a mental illness, which represents about 19% of the state’s correctional population, including offenders in prison and under community correctional supervision. To reduce these numbers, a leading mental health service provider established the Community Reintegration Program (CRP) to divert misdemeanor offenders with mental illness to communitybased treatment. This column describes a collaborative process between the University of Texas at San Antonio and community partners to develop a utilizationfocused evaluation plan for the CRP that would benefit all stakeholders. The evaluation used data collected as part of routine clinical work. (Psychiatric Services 65:8–10, 2014; doi: 10.1176/appi. ps.201300411)
A
mong adult offenders in the U.S. correctional system, the prevalence of diagnosable mental illnesses is disproportionately higher than the rate in the general population. In the Dr. Rubin is with the School of Social Work, University of Nevada, Reno (e-mail: [email protected]). Dr. Alarid is with the Criminal Justice Department, University of Texas at El Paso. Dr. Rodriguez is with University Health System in San Antonio. Lisa B. Dixon, M.D., M.P.H., and Brian Hepburn, M.D., are editors of this column. 8
Texas correctional system, more than 100,000 offenders have a mental illness, which represents 18.8% of the Texas correctional population, including offenders in prisons and under community correctional supervision. The Texas Correctional Office on Offenders with Medical or Mental Impairments has estimated that 9.7% of prisoners, 7.7% of parolees, and 4.4% of probationers have a severe mental illness, such as schizophrenia, bipolar disorder, or major depression that requires prescription medication (1). Because of the large number of individuals with mental illness in correctional facilities, the multiple needs of this population, and the desire to maintain individuals with serious mental illness in the community, it is important to collaborate with a range of service providers and develop a system of care to provide less fragmented services. Collaboration can happen at various levels—county, state, or federal—to help provide the best care for this population. Osher and colleagues (2) pointed out that a coordinating body in a local community is a key ingredient in developing an integrated system of care. They further suggested that system integration should be viewed as a process— a course of action involving ongoing “communication, goal setting, assigning accountability, evaluating and reforming” rather than “an event, a document, or position.” Given the need to reduce the number of individuals with mental illness in jails and prisons, a program to provide PSYCHIATRIC SERVICES
misdemeanor arrestees with communitybased treatment as an alternative to incarceration was developed and implemented in Bexar County, Texas. This column highlights the partnership established between University of Texas at San Antonio (UTSA) and community partners to develop a utilization-focused evaluation plan for the new program.
Bexar County and the CRP The Center for Health Care Services (CHCS) is the state-approved mental health authority for Bexar County. CHCS has a strong history of developing and implementing creative programs to foster community-based services for individuals with serious mental illness in San Antonio. The Bexar County Jail Diversion Program, established in 2002, was awarded the Gold Achievement Award in 2006 by the American Psychiatric Association (3) for its innovation in developing collaborative community-based services to keep individuals with serious mental illness in the community. With experience gained from this program, CHCS established the Community Reintegration Program (CRP) for arrested individuals who were waiting in detention cells and who showed signs of mental illness or had a history of mental illness. University Health System (UHS), the public district hospital for the area, and CHCS partnered to implement the CRP. UHS plays a vital role in the early identification, screening, and assessment of arrested persons and in appropriate diversion
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
of individuals to community-based treatment. The CRP is an outpatient treatment clinic for offenders with mental illness who have been arrested for a misdemeanor offense. The CRP opened in March 2010 and began accepting clients from a variety of sources until a suitable referral program from the magistrate court was institutionalized in July 2010. Currently, the CRP is able to handle between 50 and 55 clients at a time who have either an axis I or an axis II mental disorder. The primary program goals are to ensure that the client remains mentally stable, attends his or her next court date, and pays all court fines as applicable. Clients are served by the CRP for three to 12 months, and staffing meetings are held twice weekly. The CRP staff consists of a program director, a licensed counselor who practices cognitive-behavioral therapy, and two case managers, each with a caseload of 25 to 28 clients. After program completion, the client is transferred to a resiliency and disease management clinic at a lower level of service. Four major steps are involved in diversion of an arrested person to mental health treatment: identification, screening for eligibility, assessment, and diversion. In the CRP program, these steps are carried out by UHS clinicians located at the magistrate court facility. During the identification stage, UHS staff examine data from the Criminal Justice Information System to determine whether an offender has a history of mental illness. UHS staff also visit offenders in their cells to describe how the CRP can help them gain access to services rather than enter the jail or prison system. The next step is screening for program eligibility, which is based on evidence of a mental disorder in the offender’s initial self-report. If such evidence is found, UHS staff submit an application for assessment of the offender by mental health treatment providers. Upon completion of the assessment, UHS clinicians notify the CRP whether the offender has been accepted to the program. The most important step in the diversion process is a personal recognizance bond, which is obtained from the magistrate court PSYCHIATRIC SERVICES
and which permits the individual to be released to the community to obtain mental health services.
The CRP evaluation The partnership The UTSA Department of Social Work offers a master’s degree in social work and collaborates with CHCS to place students in internships in community agencies. The first author (who at the time was in the UTSA Department of Social Work) served as the faculty field liaison for internship students and learned about the CRP and about the agency’s interest in evaluating the program. The interest was at many levels: serving clients better, reducing incarceration, saving taxpayers money by retaining individuals with mental illness in the community, and, most of all, treating the illness. Because the CRP has both a mental health and a criminal justice component, a faculty member (the second author, who at the time was in the Criminal Justice Department at UTSA) joined the collaboration. To learn more about the CRP, we met with CHCS research administrators and data maintenance staff. Before undertaking an evaluation of the CRP, we sought to understand what data were collected as part of routine clinical work in order to ensure feasibility of data extraction from CHCS databases and accuracy of the data. Utilization-focused evaluation Program evaluation—in particular, utilization-focused evaluation—has gained momentum in the social services arena in the past decade. Utilizationfocused evaluation is defined as “evaluation done for and with specific intended primary users for specific, intended uses” (4). A unique aspect of utilization-focused evaluation is its emphasis on “intended users” and “intended uses” of the finished product. Narrowing the evaluation’s purpose and use directs attention to key features of the evaluative process that should be considered when developing and executing the evaluation plan. Evaluators who undertake a utilization-focused evaluation must gain a comprehensive understanding of the program and work with key personnel involved in program development and implementation. Evaluators gather
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
knowledge for two purposes: to understand the rationale for development of the program and the developers’ expectations of the evaluation process and outcomes. Evaluation process and plan To conduct the evaluation, we (MR and LA) worked with staff at CHCS, the magistrate court, and UHS. Decision making was focused on what information would be useful for CHCS and how information available in the system could address these information needs. A major task was to find a systematic way to identify and extrapolate data from various databases that CHCS maintains or has access to (only deidentified data were used in the evaluation). Information from various data sets— clinical, criminal justice, and service delivery data sets—was needed, along with demographic information. To help us understand the data, the CRP program director provided information on the program’s daily functioning and on its scope and magnitude. On the basis of information gathered at these meetings and from other collaborative discussions, a pragmatic evaluation plan was developed based on the partners’ agreement about what was possible, feasible, and realistic in the given timeframe and what would benefit stakeholders and the clients served by CRP. The proposal was reviewed by the research team at CHCS, and institutional review board approval was obtained from UTSA. The CRP evaluation plan targeted three specific outcomes: reduction in number of days of incarceration, medication adherence and follow-up at the clinic, and amount and type of services received. Because the CRP program was under way when the evaluation team got involved, only data already collected from clients enrolled in the program were utilized.
Conclusions The development of a utilizationfocused evaluation plan for the CRP has been an important learning experience for all partners. CHCS has continued to demonstrate a commitment to engaging in research, even though the agency’s focus is on serving individuals with mental illness. 9
One lesson learned is that service providers like CHCS gather volumes of data that researchers can use to help improve the lives of individuals with mental illness. The partnership started as an internship placement program and grew to create research opportunities for university faculty. Bexar County’s CRP screening, referral, assessment, and diversion process serves as a model for other communities, as does the university-community partnership created to develop the CRP evaluation.
10
Acknowledgments and disclosures The authors thank Linda Lopez, M.A., and Aaron Diaz, L.M.S.W.–A.P., for assistance in gaining data access and programmatic support. The authors report no competing interests.
References 1. Biennial Report of the Texas Correctional Office on Offenders With Medical or Mental Impairments. Huntsville, Texas Department of Criminal Justice, 2011. Available at www.tdcj.state.tx.us/documents/ rid/TCOOMMI_Biennial_Report_2011.pdf
PSYCHIATRIC SERVICES
2. Osher F, Steadman HJ, Barr H: A Best Practice Approach to Community Re-entry From Jails for Inmates With Co-occurring Disorders: The APIC Model. Delmar, NY, National GAINS Center for People With Co-occurring Disorders in the Justice System, Sept 2002 3. Evans L: Providing jail diversion for people with mental illness: Bexar County Jail Diversion Program, the Center for Health Care Services, San Antonio, Texas. Psychiatric Services 57:1521–1523, 2006 4. Patton MQ: Utilization-Focused Evaluation, 4th ed. Thousand Oaks, Calif, Sage, 2008
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
A Partnership to Develop a UtilizationFocused Evaluation Plan for a Criminal Justice Diversion Program Maureen Rubin, Ph.D., M.S.W. Leanne Fiftal Alarid, Ph.D. Mary Jo Rodriguez, Ph.D.
More than 100,000 offenders in the Texas correctional system have a mental illness, which represents about 19% of the state’s correctional population, including offenders in prison and under community correctional supervision. To reduce these numbers, a leading mental health service provider established the Community Reintegration Program (CRP) to divert misdemeanor offenders with mental illness to communitybased treatment. This column describes a collaborative process between the University of Texas at San Antonio and community partners to develop a utilizationfocused evaluation plan for the CRP that would benefit all stakeholders. The evaluation used data collected as part of routine clinical work. (Psychiatric Services 65:8–10, 2014; doi: 10.1176/appi. ps.201300411)
A
mong adult offenders in the U.S. correctional system, the prevalence of diagnosable mental illnesses is disproportionately higher than the rate in the general population. In the Dr. Rubin is with the School of Social Work, University of Nevada, Reno (e-mail: [email protected]). Dr. Alarid is with the Criminal Justice Department, University of Texas at El Paso. Dr. Rodriguez is with University Health System in San Antonio. Lisa B. Dixon, M.D., M.P.H., and Brian Hepburn, M.D., are editors of this column. 8
Texas correctional system, more than 100,000 offenders have a mental illness, which represents 18.8% of the Texas correctional population, including offenders in prisons and under community correctional supervision. The Texas Correctional Office on Offenders with Medical or Mental Impairments has estimated that 9.7% of prisoners, 7.7% of parolees, and 4.4% of probationers have a severe mental illness, such as schizophrenia, bipolar disorder, or major depression that requires prescription medication (1). Because of the large number of individuals with mental illness in correctional facilities, the multiple needs of this population, and the desire to maintain individuals with serious mental illness in the community, it is important to collaborate with a range of service providers and develop a system of care to provide less fragmented services. Collaboration can happen at various levels—county, state, or federal—to help provide the best care for this population. Osher and colleagues (2) pointed out that a coordinating body in a local community is a key ingredient in developing an integrated system of care. They further suggested that system integration should be viewed as a process— a course of action involving ongoing “communication, goal setting, assigning accountability, evaluating and reforming” rather than “an event, a document, or position.” Given the need to reduce the number of individuals with mental illness in jails and prisons, a program to provide PSYCHIATRIC SERVICES
misdemeanor arrestees with communitybased treatment as an alternative to incarceration was developed and implemented in Bexar County, Texas. This column highlights the partnership established between University of Texas at San Antonio (UTSA) and community partners to develop a utilization-focused evaluation plan for the new program.
Bexar County and the CRP The Center for Health Care Services (CHCS) is the state-approved mental health authority for Bexar County. CHCS has a strong history of developing and implementing creative programs to foster community-based services for individuals with serious mental illness in San Antonio. The Bexar County Jail Diversion Program, established in 2002, was awarded the Gold Achievement Award in 2006 by the American Psychiatric Association (3) for its innovation in developing collaborative community-based services to keep individuals with serious mental illness in the community. With experience gained from this program, CHCS established the Community Reintegration Program (CRP) for arrested individuals who were waiting in detention cells and who showed signs of mental illness or had a history of mental illness. University Health System (UHS), the public district hospital for the area, and CHCS partnered to implement the CRP. UHS plays a vital role in the early identification, screening, and assessment of arrested persons and in appropriate diversion
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
of individuals to community-based treatment. The CRP is an outpatient treatment clinic for offenders with mental illness who have been arrested for a misdemeanor offense. The CRP opened in March 2010 and began accepting clients from a variety of sources until a suitable referral program from the magistrate court was institutionalized in July 2010. Currently, the CRP is able to handle between 50 and 55 clients at a time who have either an axis I or an axis II mental disorder. The primary program goals are to ensure that the client remains mentally stable, attends his or her next court date, and pays all court fines as applicable. Clients are served by the CRP for three to 12 months, and staffing meetings are held twice weekly. The CRP staff consists of a program director, a licensed counselor who practices cognitive-behavioral therapy, and two case managers, each with a caseload of 25 to 28 clients. After program completion, the client is transferred to a resiliency and disease management clinic at a lower level of service. Four major steps are involved in diversion of an arrested person to mental health treatment: identification, screening for eligibility, assessment, and diversion. In the CRP program, these steps are carried out by UHS clinicians located at the magistrate court facility. During the identification stage, UHS staff examine data from the Criminal Justice Information System to determine whether an offender has a history of mental illness. UHS staff also visit offenders in their cells to describe how the CRP can help them gain access to services rather than enter the jail or prison system. The next step is screening for program eligibility, which is based on evidence of a mental disorder in the offender’s initial self-report. If such evidence is found, UHS staff submit an application for assessment of the offender by mental health treatment providers. Upon completion of the assessment, UHS clinicians notify the CRP whether the offender has been accepted to the program. The most important step in the diversion process is a personal recognizance bond, which is obtained from the magistrate court PSYCHIATRIC SERVICES
and which permits the individual to be released to the community to obtain mental health services.
The CRP evaluation The partnership The UTSA Department of Social Work offers a master’s degree in social work and collaborates with CHCS to place students in internships in community agencies. The first author (who at the time was in the UTSA Department of Social Work) served as the faculty field liaison for internship students and learned about the CRP and about the agency’s interest in evaluating the program. The interest was at many levels: serving clients better, reducing incarceration, saving taxpayers money by retaining individuals with mental illness in the community, and, most of all, treating the illness. Because the CRP has both a mental health and a criminal justice component, a faculty member (the second author, who at the time was in the Criminal Justice Department at UTSA) joined the collaboration. To learn more about the CRP, we met with CHCS research administrators and data maintenance staff. Before undertaking an evaluation of the CRP, we sought to understand what data were collected as part of routine clinical work in order to ensure feasibility of data extraction from CHCS databases and accuracy of the data. Utilization-focused evaluation Program evaluation—in particular, utilization-focused evaluation—has gained momentum in the social services arena in the past decade. Utilizationfocused evaluation is defined as “evaluation done for and with specific intended primary users for specific, intended uses” (4). A unique aspect of utilization-focused evaluation is its emphasis on “intended users” and “intended uses” of the finished product. Narrowing the evaluation’s purpose and use directs attention to key features of the evaluative process that should be considered when developing and executing the evaluation plan. Evaluators who undertake a utilization-focused evaluation must gain a comprehensive understanding of the program and work with key personnel involved in program development and implementation. Evaluators gather
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
knowledge for two purposes: to understand the rationale for development of the program and the developers’ expectations of the evaluation process and outcomes. Evaluation process and plan To conduct the evaluation, we (MR and LA) worked with staff at CHCS, the magistrate court, and UHS. Decision making was focused on what information would be useful for CHCS and how information available in the system could address these information needs. A major task was to find a systematic way to identify and extrapolate data from various databases that CHCS maintains or has access to (only deidentified data were used in the evaluation). Information from various data sets— clinical, criminal justice, and service delivery data sets—was needed, along with demographic information. To help us understand the data, the CRP program director provided information on the program’s daily functioning and on its scope and magnitude. On the basis of information gathered at these meetings and from other collaborative discussions, a pragmatic evaluation plan was developed based on the partners’ agreement about what was possible, feasible, and realistic in the given timeframe and what would benefit stakeholders and the clients served by CRP. The proposal was reviewed by the research team at CHCS, and institutional review board approval was obtained from UTSA. The CRP evaluation plan targeted three specific outcomes: reduction in number of days of incarceration, medication adherence and follow-up at the clinic, and amount and type of services received. Because the CRP program was under way when the evaluation team got involved, only data already collected from clients enrolled in the program were utilized.
Conclusions The development of a utilizationfocused evaluation plan for the CRP has been an important learning experience for all partners. CHCS has continued to demonstrate a commitment to engaging in research, even though the agency’s focus is on serving individuals with mental illness. 9
One lesson learned is that service providers like CHCS gather volumes of data that researchers can use to help improve the lives of individuals with mental illness. The partnership started as an internship placement program and grew to create research opportunities for university faculty. Bexar County’s CRP screening, referral, assessment, and diversion process serves as a model for other communities, as does the university-community partnership created to develop the CRP evaluation.
10
Acknowledgments and disclosures The authors thank Linda Lopez, M.A., and Aaron Diaz, L.M.S.W.–A.P., for assistance in gaining data access and programmatic support. The authors report no competing interests.
References 1. Biennial Report of the Texas Correctional Office on Offenders With Medical or Mental Impairments. Huntsville, Texas Department of Criminal Justice, 2011. Available at www.tdcj.state.tx.us/documents/ rid/TCOOMMI_Biennial_Report_2011.pdf
PSYCHIATRIC SERVICES
2. Osher F, Steadman HJ, Barr H: A Best Practice Approach to Community Re-entry From Jails for Inmates With Co-occurring Disorders: The APIC Model. Delmar, NY, National GAINS Center for People With Co-occurring Disorders in the Justice System, Sept 2002 3. Evans L: Providing jail diversion for people with mental illness: Bexar County Jail Diversion Program, the Center for Health Care Services, San Antonio, Texas. Psychiatric Services 57:1521–1523, 2006 4. Patton MQ: Utilization-Focused Evaluation, 4th ed. Thousand Oaks, Calif, Sage, 2008
' ps.psychiatryonline.org ' January 2014 Vol. 65 No. 1
