Local Health Department Translation Processes: Potential of Machine Translation Technologies to Help Meet Needs Anne M. Turner, MD, MLIS, MPH1 Hannah Mandel1, Daniel Capurro MD, PhD1,2 1
University of Washington, Seattle, WA Pontificia Universidad Católica de Chile. Santiago, Chile
2
Abstract Limited English proficiency (LEP), defined as a limited ability to read, speak, write, or understand English, is associated with health disparities. Despite federal and state requirements to translate health information, the vast majority of health materials are solely available in English. This project investigates barriers to translation of health information and explores new technologies to improve access to multilingual public health materials. We surveyed all 77 local health departments (LHDs) in the Northwest about translation needs, practices, barriers and attitudes towards machine translation (MT). We received 67 responses from 45 LHDs. Translation of health materials is the principle strategy used by LHDs to reach LEP populations. Cost and access to qualified translators are principle barriers to producing multilingual materials. Thirteen LHDs have used online MT tools. Many respondents expressed concerns about the accuracy of MT. Overall, respondents were positive about its potential use, if low costs and quality could be assured. Introduction The ten essential services of public health include informing, educating and empowering people about public health issues (1). The provision of these services involves making health information available to the public. However, for the 25 million people in the US who do not speak English well, these health messages may be lost. Limited English proficiency (LEP), defined as not speaking English as a primary language and having a limited ability to read, speak, write, or understand English (2), has been associated with poorer access to health care and poorer health outcomes (3–7). For this reason, both state and federal regulations require translation of health related materials (8). Under Title VI of the Civil Rights Act of 1964, a recipient of federal financial assistance must take reasonable steps to provide meaningful access for persons with LEP to covered programs and activities. Meaningful access includes provision of language appropriate written materials considered vital to health department services and emergency preparedness (9).. Despite these requirements, the vast majority of health information is provided solely in English. The TransPHorm project—funded by the National Library of Medicine—investigates current barriers to translation of health promotion materials and explores the use of new information technologies to improve access to and distribution of multilingual health promotion materials (10).
Background Machine Translation is defined as the “fully automated translation of text or speech from one language (the source language) into a different language (the target language)” (11). However, machine translation is not perfect and frequently generates text with errors (12). A human-mediated post-editing phase is frequently needed to correct those errors. Despite these imprecisions, it is possible that a combination of machine translation plus human postediting might reduce the cost of producing multilingual materials and improve access of LEP individuals to health promotion materials. As part of the TransPHorm project, we set out to better understand translation practices at local health departments (LHDs) across the Northwest, and to identify how new technologies might fit into existing health department workflows. We previously investigated the translation processes of the two largest LHDs in Washington State and found that a significant proportion of translations were outsourced to commercial vendors, with minimal use of currently available machine translation services (13). Based on this preliminary work, we developed an online survey to better understand the translation activities of a wider variety of health departments across the region. We
1378
describe here the results of a survey of LHDs in the Department of Health and Human Services (DHHS) Region X. We explored current translation practices, and assessed the attitudes of health department directors and translators towards technologies, including machine translation, to assist with creation and sharing of translated health promotion materials for LEP groups. Methods Study population All 77 LHDs in Washington, Oregon, Idaho and Alaska [DHHS Region X (14)] were included for participation in this survey. We obtained a current list of all LHDs and contact information through health department websites. Additionally, we used a database from the Northwest Center for Public Health Practice (NWCPHP)—a University of Washington School of Public Health-affiliated organization that works with LHDs in the Northwest—to complete information not accessible through the aforementioned websites (15). We obtained health department director contact information for all 77 LHDs in the four state region. Survey We developed the survey through an iterative process of question design and evaluation. Modifications to survey question wording and order were made in response to feedback. Questions were designed to obtain baseline information from all LHDs as well as information on current translation practices and barriers. To comprehensively cover all translation stages we divided question categories into pre-translation, translation, and post-translation tasks, according to our previous findings (13). Each question was pretested and modified based on feedback provided by multiple public health practitioners currently involved in performing public health translations. The online survey was created and distributed using SurveyMonkey (www.surveymonkey.com). Participants were asked to provide information about their department, their job title and role in the translation process. The survey covered questions regarding communication strategies, current translation processes, policies and barriers, how well the department is addressing its translation needs, and experience with and attitudes towards machine translation technology. Most questions required a selection of responses that were often frequencies or proportions; however, there were several open-ended questions that allowed participants to enter in free-text answers. The open-ended questions included the following: strategies used by the department to reach non-English speaking populations; details of a recent translation project, including factors such as cost; and main concerns about using a machine translation system for translating public health promotion materials. We sent email messages with survey invitations to all 77 LHD directors. Two weeks after the initial message, a follow-up email was sent as a reminder. Participants were given the opportunity to enter into an Amazon gift card drawing for completing the study. The survey and study protocol were reviewed and approved by the University of Washington Institutional Review Board (IRB). Analysis Survey data collected through SurveyMonkey was converted to an Excel spreadsheet. Duplicate responses were eliminated from the survey, and where possible, free-text entries in the ‘other’ category were classified with a predefined response. For the purpose of this analysis, and depending upon whether the question related to individual practices or departmental policies, questions answered by several employees of the same health department were either analyzed per respondent, using the responses of all individuals, or per department. In the latter case, we selected one answer for each of the 13 health departments where multiple employees replied to the survey. The answer was based on consensus between the multiple respondents (i.e. which answer was more frequent) or the answers of the department director if there was no consensus.
1379
We used descriptive statistics to characterize the surveyed population. Due to small sample sizes we used Fisher’s exact test to test for statistical significance. Analysis was performed using Stata 12 (StataCorp LP, College Station, TX). Results The survey was distributed in April 2012. We received 67 responses to our survey, with 57 respondents completing the entire survey. We received multiple responses from 13 health departments. In total, we received responses from 45 unique health departments in the Northwest (58% response rate). The make-up of the participants, including roles and associated health departments, is presented in Table 1. All respondents were involved in the translation process in some capacity, directly or indirectly, for departments that performed translation. Responding health departments were located in Washington State (53%), Oregon (38%) and Idaho (9%). Eighty-two percent of the health departments that responded were classified as either small or medium sized health departments based on the number of full-time employees (FTEs). Table 1. Overview of participants and departments. Number Health department position (responded = 67 staff members) Health Department Director
39 (58.2%)
Program Manager
8 (11.9%)
Health Educator
6 (9.0%)
Administrative Staff
5 (7.5%)
Other
3 (4.5%)
Communications Officer
3 (4.5%)
Public Health Nurse
2 (3.0%)
Health Officer
1 (1.55)
Health Department Size (responded = 45 departments) Small (100 FTEs)
8 (17.8%)
Based on the survey responses, the top three non-English languages spoken within the surveyed jurisdictions were Spanish (93.3% of departments), Chinese (35.6%), and Russian (31.1%). A minority of departments reported having materials available in languages other than English and Spanish. The top three methods used to disseminate health promotion messages were Internet (i.e. health department websites) (75.6%), printed materials (68.9%), and direct person-to-person contact (53.3%). Because of the heavy reliance on written (Internet and printed materials) communication for health promotion among the participating health departments, we assessed their translation practices and the availability of translated health promotion materials. Almost all (96%) health departments reported that some or most of their health promotion materials were available online. This did not vary with the size of the health department (p = .31) or by state (p = .52). 81.4% of health departments reported that they currently translated at least some health promotion materials into languages other than English. Eight health departments, the majority of which were in Washington State, did not translate any materials, some citing a lack of need. In response to an open-ended response question, most respondents mentioned that translation of health materials was a key strategy to reach LEP populations. Fifty-seven percent of departments reported that they only translated “some” or “few” of the available health promotion materials. However, when
1380
asked what proportion of health department materials were being translated that needed to be translated, over half (61%) felt that their department was translating “most” or “everything” that they felt needed to be translated. This perception that the department’s translation needs were being met did not significantly vary by size (p=.217), existence of a budget (p=.827), or position of the staff member (p=.687). In response to a question regarding the number of translation projects performed, estimates ranged from 0 to over 20 translations per year. Table 2. Availability of translated materials. Number Does department currently translate materials? (responded = 43 departments) Yes
35 (81.4%)
No
8 (18.6%)
Proportion of materials that are translated (responded = 45 departments) A few
6 (17.1%)
Some
14 (40.0%)
A great majority
11 (31.4%)
All
4 (11.4%)
Only a fifth of LHDs reported having a budget for translation (Table 2). This finding did not vary significantly by size of the department (p = .416) or its location (p = .817). Relatedly, financial costs were the most-cited barrier to translation, with over a third of respondents listing it as the main barrier. In response to an open-ended question, most respondents noted that their translations had been performed by in-house bilingual staff, so the cost was dependent on the hourly wage and time spent by the staff. Where respondents listed a dollar amount, three translations cost less than $100, two were in the $200-$300 range, and one cost $5,000. Perceived barriers did not vary by size (p=.555) or existence of a budget (p=.688). Although several employees mentioned that they were able to borrow and distribute translated materials from other agencies, such as the Washington State Department of Health and the Oregon Health Authority, the majority of health departments did not share documents with other local health agencies. Table 3. Facilitators and barriers to translation. Does department have a budget for translation? (responded = 34 departments) Yes
7 (20.6%)
No
24 (70.6%)
I don't know
3 (8.8%)
Perceived barriers to translation (responded = 55 staff members) Financial costs
20 (36.4%)
Lack of qualified translators
11 (20.0%)
Lack of time
10 (18.2%)
Competing health department priorities
9 (16.4%)
Other
5 (9.1%)
Does department share documents with LHAs? (responded = 34 departments) Yes
7 (20.6%)
No
24 (70.6%)
I don't know
3 (8.8%)
1381
We introduced the concept of machine translation (MT) technology (such as Google Translate) to ask about the respondent’s experiences with and perception of MT for public health materials and health promotion documents (Table 4). In response to these questions, less than one third of respondents mentioned using MT to assist in translation. Of those who did use MT, only one respondent rated its quality as “good.” However, 79% of respondents remarked that they would use MT that was specifically tailored to public health, if available. Regarding concerns about using MT for translating public health promotion materials, a few respondents mentioned that general errors or low quality of MT was of concern to them. More specifically, accuracy of the translation was mentioned as a concern most frequently, followed by the cultural appropriateness of the translation. Respondents were also concerned about the translation being too literal, or not using colloquialisms. Several mentioned that MT fails to recognize different dialects. Less often, respondents noted that there is no “second pass” and that MT does not adjust literacy levels. Related to this, one participant mentioned that “loss of control” was a concern. Several respondents remarked that the cost of MT was a major concern to them. This response suggests some participants may have been unaware of the many free online translation tools, such as Google Translate (http://translate.google.com ) and Microsoft Translator (http://www.bing.com/translator). Table 4. Perceptions of MT. Have you used MT to assist in translation? (responded = 49 staff members) Yes
15 (30.6%)
No 34 (69.4%) How often do you use MT to translate materials? (responded = 15 staff members) Frequently
1 (6.7%)
Sometimes
5 (33.3%)
Rarely
9 (60.0%)
Rating of MT quality (responded = 15 staff members) Good (some simple corrections needed)
1 (6.7%)
Fair (several corrections needed)
8 (53.3%)
Poor (many corrections needed)
6 (40.0%)
Would you be willing to use tailored MT? (responded = 56 staff members) Yes
44 (78.6%)
No
6 (10.7%)
I don't know
6 (10.7%)
Conclusion Our survey results indicate that LHDs in the Northwest are trying to meet their translation needs through the use of in-house bilingual staff. For the vast majority of health departments, designated funds are not available for producing health department materials in multiple languages and the availability of bilingual staff is limited. Although most health departments are currently not meeting legal requirements for translating health materials in their communities, many reported that they were meeting their translation needs. This may indicate that health department personnel are not aware of current laws requiring translation of materials. Prior research investigating the presence of multilingual health materials on LHD websites in Washington State indicated that less than 10% of the health promotion materials online were available in a language other than English (16).
1382
A major barrier to producing multilingual health materials is the cost of translation and finding qualified translators. This finding is similar to what we found in a case study with the two largest health departments in Washington State – Public Health Seattle & King County (PHSKC) and the Department of Health— as well as results from an inhouse study performed by PHSKC in 2011-2012 (17). Despite reported concerns about translation costs, there is little evidence that translated materials are shared across health departments. Although a state-run collaborative archive for public health materials exists in Washington State (http://here.doh.wa.gov), few health departments appear to be adding to or using this online tool for locating or sharing multilingual health promotion materials. This finding was similar to what our research group found when investigating the availability translated health promotion materials on LHD websites in Washington State (18). While our survey did not specifically address this archive, this lack of use may be due to the limited number of translated documents available on a limited number of subjects, as well as a lack of awareness about its existence. However, such an archive could be a valuable resource for translation reuse. Only one LHD reported using MT as a first step in performing translations. Staff from 13 health departments reported using freely available online translation software at some point in the translation process. Respondents expressed concern over the quality of current online translators. This concern appears to be well founded. Our prior research (11) and the research of others (19,20) indicates that freely available online translators, when used without post-editing, perform poorly when translating health information. Machine translation systems trained on specifically in the domain of health, and having a human quality assurance step, are necessary to ensure quality translations. Our TransPHorm team is developing and testing informatics solutions on both fronts. Overall, health department directors had a positive attitude towards using low cost alternatives to current translation processes, such as Google Translate, if the costs were low and the quality could be assured. In response to the need for decreasing the costs of producing quality translations of public health information, TransPHorm researchers are currently exploring the use of online MT technologies in conjunction with collaborative tools. These technologies would allow LHD staff to utilize MT and post-editing to translate health promotion materials, and to easily share these translated materials with other health departments (21) with the goal of improving access to languageappropriate health information. The hope is that application of new technologies will help reduce the health disparities that result from the lack of multilingual health information. Acknowledgments The authors would like to thank the public health department staff that participated in this study. The research described here was supported by the National Library of Medicine Grant 1R01LM010811-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Library Of Medicine or the National Institutes of Health. In addition Dr. Capurro received support from a Fulbright-MECESUP Faculty Development Grant. The authors wish to thank Kate Cole for review and preparation of this manuscript. References 1.
National Public Health Performance Standards Program. Centers for Disease Control and Prevention: 10 Essential Public Health Services [Internet]. 2010. Available from: www.cdc.gov/nphpsp/essentialservices.html
2.
Federal Interagency Working Group on Limited English Proficiency. Limited English Proficiency [Internet]. 2013 [cited 2013 Dec 3]. Available from: http://www.lep.gov/faqs/faqs.html
3.
Betancourt JR, Green AR, Carrillo JE, Ananeh-Firempong O. Defining cultural competence: a practical framework for addressing racial/ethnic disparities in health and health care. Public health reports (Washington, D.C. : 1974) [Internet]. [cited 2013 Mar 1];118(4):293–302. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1497553&tool=pmcentrez&rendertype=abstract
1383
4.
Ponce NA, Hays RD, Cunningham WE. Linguistic disparities in health care access and health status among older adults. Journal of general internal medicine [Internet]. 2006 Jul [cited 2013 Feb 12];21(7):786–91. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1924691&tool=pmcentrez&rendertype=abstract
5.
Sentell T, Shumway M, Snowden L. Access to mental health treatment by English language proficiency and race/ethnicity. Journal of general internal medicine [Internet]. 2007 Nov [cited 2013 Mar 11];22 Suppl 2:289–93. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2150610&tool=pmcentrez&rendertype=abstract
6.
Cheng EM, Chen A, Cunningham W. Primary language and receipt of recommended health care among Hispanics in the United States. Journal of general internal medicine [Internet]. 2007 Nov [cited 2013 Mar 11];22 Suppl 2:283–8. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2078546&tool=pmcentrez&rendertype=abstract
7.
Pippins JR, Alegría M, Haas JS. Association between language proficiency and the quality of primary care among a national sample of insured Latinos. Medical care [Internet]. 2007 Nov [cited 2013 Mar 11];45(11):1020–5. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2836911&tool=pmcentrez&rendertype=abstract
8.
Perkins J, Youdelman M. Summary of State Law Requirements: Addressing Language Needs in Health Care [Internet]. Washington, DC; 2008 p. 1–136. Available from: http://sites.lawhelp.org/documents/383231nhelp.lep.state.law.chart.final.pdf
9.
US Department of Health and Human Services. Language Access Plan 2013 [Internet]. 2013. Available from: http://www.hhs.gov/open/execorders/2013-hhs-language-access-plan.pdf
10.
Northwest Center for Public Health Practice. TransPHorm: Translation of Public Health Information for a Diverse Public [Internet]. 2013. Available from: http://www.nwcphp.org/research/projects/current/machinetranslation-for-health-information
11.
Kirchhoff K, Turner AM, Axelrod A, Saavedra F. Application of statistical machine translation to public health information: a feasibility study. Journal of the American Medical Informatics Association : JAMIA [Internet]. [cited 2013 Mar 14];18(4):473–8. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3128406&tool=pmcentrez&rendertype=abstract
12.
Kirchhoff K, Capurro D, Turner A. Evaluating user preferences in machine translation using conjoint analysis. Proceedings of EAMT. 2012.
13.
Brownstein M, Capurro D, Cole K, Karaz H, Turner A. Improving access to translated materials through machine translation. Joint Conference on Health. Vancouver, WA; 2011.
14.
US Department of Health and Human Services. Regional Offices [Internet]. 2013 [cited 2013 Dec 3]. Available from: http://www.hhs.gov/iea/regional/index.html
15.
University of Washington School of Public Health. Northwest Center for Public Health Practice [Internet]. 2013 [cited 2013 Dec 3]. Available from: www.nwcphp.org
16.
Capurro D, Turner A, Chaudhuri S. The online availability of multilingual health promotion materials produced by local health departments: A statewide assessment. AMIA Annual Symposium Proceedings. Washington, DC; 2013.
1384
17.
Valenzuela M, Beebe A, Beleford J, Apa J, Holland A, Wong E. Internal and External Translation Assessment Summary - October 2008. Seattle, WA; 2008.
18.
Capurro D, Chaudhuri S, Brownstein M, Turner AM. Assessment of Translated Public Health Materials in Washington State. University of Washington Department of Health Services Annual Research Conference. Seattle, WA; 2013.
19.
Zeng-Treitler Q, Kim H, Rosemblat G, Keselman A. Can multilingual machine translation help make medical record content more comprehensible to patients? Studies in health technology and informatics [Internet]. 2010 Jan [cited 2013 Jul 10];160(Pt 1):73–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20841653
20.
Nguyen-Lu N, Reide P, Yentis SM. “Do you have a stick in your mouth?”-- use of Google Translate as an aid to anaesthetic pre-assessment. Anaesthesia [Internet]. 2010 Jan [cited 2013 Jul 10];65(1):96–7. Available from: http://doi.wiley.com/10.1111/j.1365-2044.2009.06184_4.x
21.
Laurenzi A, Brownstein M, Turner A, Kirchhoff K, Kientz J. An Online Collaborative Translation Management System for Public Health Workers. CHI. Paris, France; 2013.
1385
University of Washington, Seattle, WA Pontificia Universidad Católica de Chile. Santiago, Chile
2
Abstract Limited English proficiency (LEP), defined as a limited ability to read, speak, write, or understand English, is associated with health disparities. Despite federal and state requirements to translate health information, the vast majority of health materials are solely available in English. This project investigates barriers to translation of health information and explores new technologies to improve access to multilingual public health materials. We surveyed all 77 local health departments (LHDs) in the Northwest about translation needs, practices, barriers and attitudes towards machine translation (MT). We received 67 responses from 45 LHDs. Translation of health materials is the principle strategy used by LHDs to reach LEP populations. Cost and access to qualified translators are principle barriers to producing multilingual materials. Thirteen LHDs have used online MT tools. Many respondents expressed concerns about the accuracy of MT. Overall, respondents were positive about its potential use, if low costs and quality could be assured. Introduction The ten essential services of public health include informing, educating and empowering people about public health issues (1). The provision of these services involves making health information available to the public. However, for the 25 million people in the US who do not speak English well, these health messages may be lost. Limited English proficiency (LEP), defined as not speaking English as a primary language and having a limited ability to read, speak, write, or understand English (2), has been associated with poorer access to health care and poorer health outcomes (3–7). For this reason, both state and federal regulations require translation of health related materials (8). Under Title VI of the Civil Rights Act of 1964, a recipient of federal financial assistance must take reasonable steps to provide meaningful access for persons with LEP to covered programs and activities. Meaningful access includes provision of language appropriate written materials considered vital to health department services and emergency preparedness (9).. Despite these requirements, the vast majority of health information is provided solely in English. The TransPHorm project—funded by the National Library of Medicine—investigates current barriers to translation of health promotion materials and explores the use of new information technologies to improve access to and distribution of multilingual health promotion materials (10).
Background Machine Translation is defined as the “fully automated translation of text or speech from one language (the source language) into a different language (the target language)” (11). However, machine translation is not perfect and frequently generates text with errors (12). A human-mediated post-editing phase is frequently needed to correct those errors. Despite these imprecisions, it is possible that a combination of machine translation plus human postediting might reduce the cost of producing multilingual materials and improve access of LEP individuals to health promotion materials. As part of the TransPHorm project, we set out to better understand translation practices at local health departments (LHDs) across the Northwest, and to identify how new technologies might fit into existing health department workflows. We previously investigated the translation processes of the two largest LHDs in Washington State and found that a significant proportion of translations were outsourced to commercial vendors, with minimal use of currently available machine translation services (13). Based on this preliminary work, we developed an online survey to better understand the translation activities of a wider variety of health departments across the region. We
1378
describe here the results of a survey of LHDs in the Department of Health and Human Services (DHHS) Region X. We explored current translation practices, and assessed the attitudes of health department directors and translators towards technologies, including machine translation, to assist with creation and sharing of translated health promotion materials for LEP groups. Methods Study population All 77 LHDs in Washington, Oregon, Idaho and Alaska [DHHS Region X (14)] were included for participation in this survey. We obtained a current list of all LHDs and contact information through health department websites. Additionally, we used a database from the Northwest Center for Public Health Practice (NWCPHP)—a University of Washington School of Public Health-affiliated organization that works with LHDs in the Northwest—to complete information not accessible through the aforementioned websites (15). We obtained health department director contact information for all 77 LHDs in the four state region. Survey We developed the survey through an iterative process of question design and evaluation. Modifications to survey question wording and order were made in response to feedback. Questions were designed to obtain baseline information from all LHDs as well as information on current translation practices and barriers. To comprehensively cover all translation stages we divided question categories into pre-translation, translation, and post-translation tasks, according to our previous findings (13). Each question was pretested and modified based on feedback provided by multiple public health practitioners currently involved in performing public health translations. The online survey was created and distributed using SurveyMonkey (www.surveymonkey.com). Participants were asked to provide information about their department, their job title and role in the translation process. The survey covered questions regarding communication strategies, current translation processes, policies and barriers, how well the department is addressing its translation needs, and experience with and attitudes towards machine translation technology. Most questions required a selection of responses that were often frequencies or proportions; however, there were several open-ended questions that allowed participants to enter in free-text answers. The open-ended questions included the following: strategies used by the department to reach non-English speaking populations; details of a recent translation project, including factors such as cost; and main concerns about using a machine translation system for translating public health promotion materials. We sent email messages with survey invitations to all 77 LHD directors. Two weeks after the initial message, a follow-up email was sent as a reminder. Participants were given the opportunity to enter into an Amazon gift card drawing for completing the study. The survey and study protocol were reviewed and approved by the University of Washington Institutional Review Board (IRB). Analysis Survey data collected through SurveyMonkey was converted to an Excel spreadsheet. Duplicate responses were eliminated from the survey, and where possible, free-text entries in the ‘other’ category were classified with a predefined response. For the purpose of this analysis, and depending upon whether the question related to individual practices or departmental policies, questions answered by several employees of the same health department were either analyzed per respondent, using the responses of all individuals, or per department. In the latter case, we selected one answer for each of the 13 health departments where multiple employees replied to the survey. The answer was based on consensus between the multiple respondents (i.e. which answer was more frequent) or the answers of the department director if there was no consensus.
1379
We used descriptive statistics to characterize the surveyed population. Due to small sample sizes we used Fisher’s exact test to test for statistical significance. Analysis was performed using Stata 12 (StataCorp LP, College Station, TX). Results The survey was distributed in April 2012. We received 67 responses to our survey, with 57 respondents completing the entire survey. We received multiple responses from 13 health departments. In total, we received responses from 45 unique health departments in the Northwest (58% response rate). The make-up of the participants, including roles and associated health departments, is presented in Table 1. All respondents were involved in the translation process in some capacity, directly or indirectly, for departments that performed translation. Responding health departments were located in Washington State (53%), Oregon (38%) and Idaho (9%). Eighty-two percent of the health departments that responded were classified as either small or medium sized health departments based on the number of full-time employees (FTEs). Table 1. Overview of participants and departments. Number Health department position (responded = 67 staff members) Health Department Director
39 (58.2%)
Program Manager
8 (11.9%)
Health Educator
6 (9.0%)
Administrative Staff
5 (7.5%)
Other
3 (4.5%)
Communications Officer
3 (4.5%)
Public Health Nurse
2 (3.0%)
Health Officer
1 (1.55)
Health Department Size (responded = 45 departments) Small (100 FTEs)
8 (17.8%)
Based on the survey responses, the top three non-English languages spoken within the surveyed jurisdictions were Spanish (93.3% of departments), Chinese (35.6%), and Russian (31.1%). A minority of departments reported having materials available in languages other than English and Spanish. The top three methods used to disseminate health promotion messages were Internet (i.e. health department websites) (75.6%), printed materials (68.9%), and direct person-to-person contact (53.3%). Because of the heavy reliance on written (Internet and printed materials) communication for health promotion among the participating health departments, we assessed their translation practices and the availability of translated health promotion materials. Almost all (96%) health departments reported that some or most of their health promotion materials were available online. This did not vary with the size of the health department (p = .31) or by state (p = .52). 81.4% of health departments reported that they currently translated at least some health promotion materials into languages other than English. Eight health departments, the majority of which were in Washington State, did not translate any materials, some citing a lack of need. In response to an open-ended response question, most respondents mentioned that translation of health materials was a key strategy to reach LEP populations. Fifty-seven percent of departments reported that they only translated “some” or “few” of the available health promotion materials. However, when
1380
asked what proportion of health department materials were being translated that needed to be translated, over half (61%) felt that their department was translating “most” or “everything” that they felt needed to be translated. This perception that the department’s translation needs were being met did not significantly vary by size (p=.217), existence of a budget (p=.827), or position of the staff member (p=.687). In response to a question regarding the number of translation projects performed, estimates ranged from 0 to over 20 translations per year. Table 2. Availability of translated materials. Number Does department currently translate materials? (responded = 43 departments) Yes
35 (81.4%)
No
8 (18.6%)
Proportion of materials that are translated (responded = 45 departments) A few
6 (17.1%)
Some
14 (40.0%)
A great majority
11 (31.4%)
All
4 (11.4%)
Only a fifth of LHDs reported having a budget for translation (Table 2). This finding did not vary significantly by size of the department (p = .416) or its location (p = .817). Relatedly, financial costs were the most-cited barrier to translation, with over a third of respondents listing it as the main barrier. In response to an open-ended question, most respondents noted that their translations had been performed by in-house bilingual staff, so the cost was dependent on the hourly wage and time spent by the staff. Where respondents listed a dollar amount, three translations cost less than $100, two were in the $200-$300 range, and one cost $5,000. Perceived barriers did not vary by size (p=.555) or existence of a budget (p=.688). Although several employees mentioned that they were able to borrow and distribute translated materials from other agencies, such as the Washington State Department of Health and the Oregon Health Authority, the majority of health departments did not share documents with other local health agencies. Table 3. Facilitators and barriers to translation. Does department have a budget for translation? (responded = 34 departments) Yes
7 (20.6%)
No
24 (70.6%)
I don't know
3 (8.8%)
Perceived barriers to translation (responded = 55 staff members) Financial costs
20 (36.4%)
Lack of qualified translators
11 (20.0%)
Lack of time
10 (18.2%)
Competing health department priorities
9 (16.4%)
Other
5 (9.1%)
Does department share documents with LHAs? (responded = 34 departments) Yes
7 (20.6%)
No
24 (70.6%)
I don't know
3 (8.8%)
1381
We introduced the concept of machine translation (MT) technology (such as Google Translate) to ask about the respondent’s experiences with and perception of MT for public health materials and health promotion documents (Table 4). In response to these questions, less than one third of respondents mentioned using MT to assist in translation. Of those who did use MT, only one respondent rated its quality as “good.” However, 79% of respondents remarked that they would use MT that was specifically tailored to public health, if available. Regarding concerns about using MT for translating public health promotion materials, a few respondents mentioned that general errors or low quality of MT was of concern to them. More specifically, accuracy of the translation was mentioned as a concern most frequently, followed by the cultural appropriateness of the translation. Respondents were also concerned about the translation being too literal, or not using colloquialisms. Several mentioned that MT fails to recognize different dialects. Less often, respondents noted that there is no “second pass” and that MT does not adjust literacy levels. Related to this, one participant mentioned that “loss of control” was a concern. Several respondents remarked that the cost of MT was a major concern to them. This response suggests some participants may have been unaware of the many free online translation tools, such as Google Translate (http://translate.google.com ) and Microsoft Translator (http://www.bing.com/translator). Table 4. Perceptions of MT. Have you used MT to assist in translation? (responded = 49 staff members) Yes
15 (30.6%)
No 34 (69.4%) How often do you use MT to translate materials? (responded = 15 staff members) Frequently
1 (6.7%)
Sometimes
5 (33.3%)
Rarely
9 (60.0%)
Rating of MT quality (responded = 15 staff members) Good (some simple corrections needed)
1 (6.7%)
Fair (several corrections needed)
8 (53.3%)
Poor (many corrections needed)
6 (40.0%)
Would you be willing to use tailored MT? (responded = 56 staff members) Yes
44 (78.6%)
No
6 (10.7%)
I don't know
6 (10.7%)
Conclusion Our survey results indicate that LHDs in the Northwest are trying to meet their translation needs through the use of in-house bilingual staff. For the vast majority of health departments, designated funds are not available for producing health department materials in multiple languages and the availability of bilingual staff is limited. Although most health departments are currently not meeting legal requirements for translating health materials in their communities, many reported that they were meeting their translation needs. This may indicate that health department personnel are not aware of current laws requiring translation of materials. Prior research investigating the presence of multilingual health materials on LHD websites in Washington State indicated that less than 10% of the health promotion materials online were available in a language other than English (16).
1382
A major barrier to producing multilingual health materials is the cost of translation and finding qualified translators. This finding is similar to what we found in a case study with the two largest health departments in Washington State – Public Health Seattle & King County (PHSKC) and the Department of Health— as well as results from an inhouse study performed by PHSKC in 2011-2012 (17). Despite reported concerns about translation costs, there is little evidence that translated materials are shared across health departments. Although a state-run collaborative archive for public health materials exists in Washington State (http://here.doh.wa.gov), few health departments appear to be adding to or using this online tool for locating or sharing multilingual health promotion materials. This finding was similar to what our research group found when investigating the availability translated health promotion materials on LHD websites in Washington State (18). While our survey did not specifically address this archive, this lack of use may be due to the limited number of translated documents available on a limited number of subjects, as well as a lack of awareness about its existence. However, such an archive could be a valuable resource for translation reuse. Only one LHD reported using MT as a first step in performing translations. Staff from 13 health departments reported using freely available online translation software at some point in the translation process. Respondents expressed concern over the quality of current online translators. This concern appears to be well founded. Our prior research (11) and the research of others (19,20) indicates that freely available online translators, when used without post-editing, perform poorly when translating health information. Machine translation systems trained on specifically in the domain of health, and having a human quality assurance step, are necessary to ensure quality translations. Our TransPHorm team is developing and testing informatics solutions on both fronts. Overall, health department directors had a positive attitude towards using low cost alternatives to current translation processes, such as Google Translate, if the costs were low and the quality could be assured. In response to the need for decreasing the costs of producing quality translations of public health information, TransPHorm researchers are currently exploring the use of online MT technologies in conjunction with collaborative tools. These technologies would allow LHD staff to utilize MT and post-editing to translate health promotion materials, and to easily share these translated materials with other health departments (21) with the goal of improving access to languageappropriate health information. The hope is that application of new technologies will help reduce the health disparities that result from the lack of multilingual health information. Acknowledgments The authors would like to thank the public health department staff that participated in this study. The research described here was supported by the National Library of Medicine Grant 1R01LM010811-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Library Of Medicine or the National Institutes of Health. In addition Dr. Capurro received support from a Fulbright-MECESUP Faculty Development Grant. The authors wish to thank Kate Cole for review and preparation of this manuscript. References 1.
National Public Health Performance Standards Program. Centers for Disease Control and Prevention: 10 Essential Public Health Services [Internet]. 2010. Available from: www.cdc.gov/nphpsp/essentialservices.html
2.
Federal Interagency Working Group on Limited English Proficiency. Limited English Proficiency [Internet]. 2013 [cited 2013 Dec 3]. Available from: http://www.lep.gov/faqs/faqs.html
3.
Betancourt JR, Green AR, Carrillo JE, Ananeh-Firempong O. Defining cultural competence: a practical framework for addressing racial/ethnic disparities in health and health care. Public health reports (Washington, D.C. : 1974) [Internet]. [cited 2013 Mar 1];118(4):293–302. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1497553&tool=pmcentrez&rendertype=abstract
1383
4.
Ponce NA, Hays RD, Cunningham WE. Linguistic disparities in health care access and health status among older adults. Journal of general internal medicine [Internet]. 2006 Jul [cited 2013 Feb 12];21(7):786–91. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1924691&tool=pmcentrez&rendertype=abstract
5.
Sentell T, Shumway M, Snowden L. Access to mental health treatment by English language proficiency and race/ethnicity. Journal of general internal medicine [Internet]. 2007 Nov [cited 2013 Mar 11];22 Suppl 2:289–93. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2150610&tool=pmcentrez&rendertype=abstract
6.
Cheng EM, Chen A, Cunningham W. Primary language and receipt of recommended health care among Hispanics in the United States. Journal of general internal medicine [Internet]. 2007 Nov [cited 2013 Mar 11];22 Suppl 2:283–8. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2078546&tool=pmcentrez&rendertype=abstract
7.
Pippins JR, Alegría M, Haas JS. Association between language proficiency and the quality of primary care among a national sample of insured Latinos. Medical care [Internet]. 2007 Nov [cited 2013 Mar 11];45(11):1020–5. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2836911&tool=pmcentrez&rendertype=abstract
8.
Perkins J, Youdelman M. Summary of State Law Requirements: Addressing Language Needs in Health Care [Internet]. Washington, DC; 2008 p. 1–136. Available from: http://sites.lawhelp.org/documents/383231nhelp.lep.state.law.chart.final.pdf
9.
US Department of Health and Human Services. Language Access Plan 2013 [Internet]. 2013. Available from: http://www.hhs.gov/open/execorders/2013-hhs-language-access-plan.pdf
10.
Northwest Center for Public Health Practice. TransPHorm: Translation of Public Health Information for a Diverse Public [Internet]. 2013. Available from: http://www.nwcphp.org/research/projects/current/machinetranslation-for-health-information
11.
Kirchhoff K, Turner AM, Axelrod A, Saavedra F. Application of statistical machine translation to public health information: a feasibility study. Journal of the American Medical Informatics Association : JAMIA [Internet]. [cited 2013 Mar 14];18(4):473–8. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3128406&tool=pmcentrez&rendertype=abstract
12.
Kirchhoff K, Capurro D, Turner A. Evaluating user preferences in machine translation using conjoint analysis. Proceedings of EAMT. 2012.
13.
Brownstein M, Capurro D, Cole K, Karaz H, Turner A. Improving access to translated materials through machine translation. Joint Conference on Health. Vancouver, WA; 2011.
14.
US Department of Health and Human Services. Regional Offices [Internet]. 2013 [cited 2013 Dec 3]. Available from: http://www.hhs.gov/iea/regional/index.html
15.
University of Washington School of Public Health. Northwest Center for Public Health Practice [Internet]. 2013 [cited 2013 Dec 3]. Available from: www.nwcphp.org
16.
Capurro D, Turner A, Chaudhuri S. The online availability of multilingual health promotion materials produced by local health departments: A statewide assessment. AMIA Annual Symposium Proceedings. Washington, DC; 2013.
1384
17.
Valenzuela M, Beebe A, Beleford J, Apa J, Holland A, Wong E. Internal and External Translation Assessment Summary - October 2008. Seattle, WA; 2008.
18.
Capurro D, Chaudhuri S, Brownstein M, Turner AM. Assessment of Translated Public Health Materials in Washington State. University of Washington Department of Health Services Annual Research Conference. Seattle, WA; 2013.
19.
Zeng-Treitler Q, Kim H, Rosemblat G, Keselman A. Can multilingual machine translation help make medical record content more comprehensible to patients? Studies in health technology and informatics [Internet]. 2010 Jan [cited 2013 Jul 10];160(Pt 1):73–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20841653
20.
Nguyen-Lu N, Reide P, Yentis SM. “Do you have a stick in your mouth?”-- use of Google Translate as an aid to anaesthetic pre-assessment. Anaesthesia [Internet]. 2010 Jan [cited 2013 Jul 10];65(1):96–7. Available from: http://doi.wiley.com/10.1111/j.1365-2044.2009.06184_4.x
21.
Laurenzi A, Brownstein M, Turner A, Kirchhoff K, Kientz J. An Online Collaborative Translation Management System for Public Health Workers. CHI. Paris, France; 2013.
1385
