This article was downloaded by: [McGill University Library] On: 26 February 2015, At: 17:51 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Health Communication: International Perspectives Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uhcm20
Development and Evaluation of the Korean Health Literacy Instrument a
b
c
Soo Jin Kang , Tae Wha Lee , Michael K. Paasche-Orlow , Gwang b
Suk Kim & Hee Kwan Won
d
a
Health and Nursing Science , Daegu University , Daegu , Republic of Korea b
College of Nursing , Yonsei University , Seoul , Republic of Korea
c
Section of General Internal Medicine, Department of Medicine , Boston University School of Medicine , Boston , Massachusetts , USA
Click for updates
d
College of Medicine , Konyang University , Daejeon , Republic of Korea Published online: 14 Oct 2014.
To cite this article: Soo Jin Kang , Tae Wha Lee , Michael K. Paasche-Orlow , Gwang Suk Kim & Hee Kwan Won (2014) Development and Evaluation of the Korean Health Literacy Instrument, Journal of Health Communication: International Perspectives, 19:sup2, 254-266, DOI: 10.1080/10810730.2014.946113 To link to this article: http://dx.doi.org/10.1080/10810730.2014.946113
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [McGill University Library] at 17:51 26 February 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Health Communication, 19:254–266, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.946113
Development and Evaluation of the Korean Health Literacy Instrument SOO JIN KANG
Downloaded by [McGill University Library] at 17:51 26 February 2015
Health and Nursing Science, Daegu University, Daegu, Republic of Korea
TAE WHA LEE College of Nursing, Yonsei University, Seoul, Republic of Korea
MICHAEL K. PAASCHE-ORLOW Section of General Internal Medicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
GWANG SUK KIM College of Nursing, Yonsei University, Seoul, Republic of Korea
HEE KWAN WON College of Medicine, Konyang University, Daejeon, Republic of Korea The purpose of this study is to develop and validate the Korean Health Literacy Instrument, which measures the capacity to understand and use health-related information and make informed health decisions in Korean adults. In Phase 1, 33 initial items were generated to measure functional, interactive, and critical health literacy with prose, document, and numeracy tasks. These items included content from health promotion, disease management, and health navigation contexts. Content validity assessment was conducted by an expert panel, and 11 items were excluded. In Phase 2, the 22 remaining items were administered to a convenience sample of 292 adults from community and clinical settings. Exploratory factor and item difficulty and discrimination analyses were conducted and four items with low discrimination were deleted. In Phase 3, the remaining 18 items were administered to a convenience sample of 315 adults 40–64 years of age from community and clinical settings. A confirmatory factor analysis was performed to test the construct validity of the instrument. The Korean Health Literacy Instrument has a range of 0 to 18. The mean score in our validation study was 11.98. The instrument exhibited an internal consistency reliability coefficient of 0.82, and a test–retest reliability of 0.89. The instrument is suitable for screening individuals who have limited health literacy skills. Future studies are needed to further define the psychometric properties and predictive validity of the Korean Health Literacy Instrument.
Health literacy is ‘‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services that are needed Address correspondence to Tae Wha Lee, College of Nursing, Yonsei University, 250 Seongsanro, Seodaemun-gu, Seoul, Republic of Korea. E-mail: [email protected]
254
Downloaded by [McGill University Library] at 17:51 26 February 2015
Development of Korean Health Literacy Instrument
255
to make appropriate health decisions’’ (Institute of Medicine, 2004). Although the importance of health literacy has been well documented in several countries (Agency for Healthcare Research and Quality, 2004); Berkman, Sheridan, Donahue, & Halpem, 2011; Institute of Medicine, 2004; World Health Organization, 1998), little data exist regarding the prevalence of low health literacy in Korea. Several initial studies have reported the prevalence of health literacy among elderly Koreans (Jeong & Kim, 2014; S. H. Kim & Lee, 2008; Lee, Kang, Lee, & Hyun, 2009). A considerable portion, 68.7%, of Korean adults older than 65 years of age had not even completed elementary school (National Institute of the Korean Language, 2008). On the contrary, 89% of 25–64-year-old Korean adults reported to have completed at least a middle school level education (Organisation for Economic Co-operation and Development, 2009). Nevertheless, 32.7% of adults were unable to understand health information, including drug directions (National Institute of the Korean Language, 2008). S. S. Kim, Kim, and Lee (2005) created a Korean language version of the Rapid Estimate of Adult Literacy in Medicine (T. C. Davis et al., 1991). These studies showed that 20–35% of elderly Koreans have limited health literacy. However, research conducted with tools like the Rapid Estimate of Adult Literacy in Medicine is inherently limited (McCormack et al., 2010; Nutbeam, 2009; Pleasant, McKinney, & Rikard, 2011; Sørensen et al., 2012). The most commonly used health literacy instruments, the Test of Functional Health Literacy in Adults (Parker, Baker, Williams, & Nurss, 1995), the Rapid Estimate of Adult Literacy in Medicine, and the Newest Vital Sign (Weiss et al., 2005) evaluate a very narrow band of health literacy skills. The Test of Functional Health Literacy in Adults measures the patient’s ability to read and comprehend text passages (i.e., prose literacy) and passages containing numerical information (i.e., numeracy) using real materials from health care settings. The Rapid Estimate of Adult Literacy in Medicine exclusively measures the ability to correctly pronounce a list of medical terms. The Newest Vital Sign is an effective screening tool; however, it is focused on document and numeracy skills, and there is not validated Korean version of this instrument. Conceptual models of health literacy define multiple additional dimensions and recognize the importance of context (Frisch, Camerini, Diviani, & Schulz, 2012; Nutbeam, 2000; Paasche-Orlow & Wolf, 2007). Measures of health literacy are needed that can capture such complexities in order to be able to differentiate which aspects of health literacy may be appropriately be targeted for intervention. Furthermore, direct translation of individual health literacy instruments from other languages is inappropriate (Han, Kim, Kim, & Kim, 2011; Lee, Kang, Lee, & Hyun, 2009). This is a result of both linguistic and cultural factors. Korean has a much higher degree of phoneme-grapheme correspondence than English; therefore, a broad array of Korean words can be pronounced by people with limited Korean literacy skills with no preexisting knowledge or comprehension for what is being read. In English the grapheme may represent a number of alternative phonemes (e.g., e-a in neat vs. head). The relative low frequency of phoneme-grapheme correspondence in English is precisely the linguistic mechanism by which tools such as the Rapid Estimate of Adult Literacy in Medicine rapidly differentiate levels of literacy skill. As phonological decoding is not as demanding a task in Korean, this linguistic attribute is not a sound basis for a Korean health literacy test. In addition, the health and public health systems in Korea have their own distinct structure, history, and culture. Items in a test like the Test of Functional Health Literacy in Adults are taken from the cultural context of the target population for the test. For example, one of the test passages relates to a patient’s rights and responsibilities in the U.S. government program Medicare. Measures developed for alternative target populations should have culturally appropriate test items.
256
S. J. Kang et al.
The lack of a reliable, valid, and comprehensive skills-based health literacy instrument impedes empirical research in Korea. The purpose of the current study was to develop and conduct initial validation of the Korean Heath Literacy Instrument (KHLI) to fill this void. However, we restricted participants to middleaged adults for this study because of the highly significant gap in educational attainment between younger and elderly Koreans.
Method
Downloaded by [McGill University Library] at 17:51 26 February 2015
This study was conducted in three phases: instrument development and expert review of content validity; (b) preliminary test; and (c) instrument evaluation. Instrument Development and Content Validation After scanning the literature on definitions and conceptual frameworks for health literacy (American Medical Association, 1999; Berkman, Davis, & McCormack, 2010; Freedman et al., 2009; Institute of Medicine, 2004; Jordan, Buchbinder, & Osborne, 2010; Kickbusch, Wait, & Magg, 2005; Nutbeam, 2000; Paasche-Orlow & Wolf, 2007; Paasche-Orlow, Wilson, McCormack, 2010; Parker et al., 1995; Zarcadoolas, Pleasant, & Greer, 2003), we defined that health literacy is a personal skill that determines the ability to understand, interpret, and apply information in order to manage health. We adapted Nutbeam’s conceptual model as a basis for the skills depicted in our taxonomy. Nutbeam (2000) argued that health-literacy skills should be expanded from cognitive skills to social skills and suggested that these skills can be categorized as functional, interactive, and critical skills; in this study, we defined functional health literacy as the ability to read and understand written materials to use with patients in health care. We defined interactive health literacy as ability to extract information, and to derive meaning for communication with health providers. For these items we aimed to verify whether the consumer possesses a prior understanding of the essential terms and basic concepts that are required to achieve proper communication. We defined critical literacy as the ability to appraise health-related information to manage one’s health (Chin & McCarthy, 2013; Ishikawa, Tacheuchi, & Yano, 2008). The skills of the KHLI are based on three types of literacy tasks: prose (understanding text presented in sentences), document (understanding forms and tables), and quantitative (applying arithmetic operations and understanding quantitative information). In addition, these literacy tasks are encountered in the everyday health-related activities of adults in the situation (context) including health promotion, disease management, and health navigation. Figure 1 presents the framework for instrument development. Test item stimuli were selected from health educational materials for chronic diseases used in clinical settings and from other prose and non-prose documents used in clinical and public health settings. For each item, the respondent is presented with instructions and a test stimulus (e.g., excerpt from patient education material), followed by a multiple choice answer set. Initially, 33 items were generated with an additional question about instructions for taking medicine from the national literacy survey (National Institute of the Korean Language, 2008). Functional health literacy items were based on the ability to make an appointment with a doctor, understand medicine instructions, understand nutritional labels, and billing. Interactive health literacy items were composed of the ability to understand health-related terms and concepts, such as estimations (e.g., the concept of a basic unit volumes), as well as understanding criteria and national health insurance terms. Critical literacy items involved the ability to apply information to situations.
Downloaded by [McGill University Library] at 17:51 26 February 2015
Development of Korean Health Literacy Instrument
257
Figure 1. Framework of the study.
Content validity was evaluated by an expert panel of 12 health care professionals, including clinical, academic, and school nurses, a dietician, and practicing physicians. These experts rated each item for relevance, difficulty, and clarity, on a 4-point scale ranging from 1 (not relevant) to 4 (very relevant) and suggested edits. The content validity index was computed with the relevance and appropriateness of each item. Items that did not meet the content validity index of 0.8 (Knapp, 1985; Lynn, 1986), were either eliminated or revised by the expert panel. On the basis of this process, 11 items were excluded. The remaining set of 22 items included 7 functional, 9 interactive, and 6 critical items.
Preliminary Test The preliminary test was conducted in a convenience sample of adult Korean participants at community and hospital settings in Seoul, Korea from November 5 to November 23, 2010. The questionnaires were administered with face-to-face interviews. Informed consent forms were obtained from participants before the survey. Among the 345 questionnaires distributed, 307 were collected and 292 (85% overall response rate) were completed. These data were analyzed with exploratory factor analysis (EFA) and item difficulty and discrimination evaluation. A three-factor model was extracted from EFA on these 22 items. An EFA was conducted to examine the factor structure and to evaluate unidimensionality of this data set. The chi-square test can inflate type 1 errors (Bollen, 1989); thus, other indices were used to judge the model fit. In this study, the model fit was assessed by root mean square error of approximation (RMSEA), which is amenable for analyses with large sample sizes. RMSEA values below .06 indicated a satisfactory model fit (Browne & Cudeck, 1993; Hu & Bentler, 1999). There was no significant difference between models; and one-, two-, and three-factor models had a good fit (RMSEA criterion 0.06 or less). The three-factor model was chosen as the best solution because the model fit index did not show significant differences among the three models and the KHLI was developed based on the three-factor model. After item analysis, 4 items out of 22 items were eliminated from the instrument after preliminary test. Item 5 and Item 18 was deleted because of similar item
258
S. J. Kang et al.
Table 1. Item characteristics Health context Item no. 2 3
Downloaded by [McGill University Library] at 17:51 26 February 2015
4 8 9 10 12 13 15 16 19 21 24 27 28 29 31 33
Contents of questions
Task
Fhl 2 Medication label use in the community pharmacy Fhl 3 Selective medical treatment Fhl 4 Clinic schedules Fhl 8 Nutritional labels for serving sizes Fhl 9 Medication times Fhl 10 Standard weight Ihl 1 Calculation of body mass index Ihl 2 Diet management Ihl 4 Body related terms Ihl 5 National health insurance Ihl 8 Hypertension criteria Ihl 10 Complication risks Chl 2 Medication directions Chl 5 Obesity Chl 6 Colon cancer screening Chl 7 Hot and cold packs Chl 9 Nutritional labels 2 Chl 11 Risk perception
Document
Health promotion
Disease management X
Document
X
Document Numeracy
X
Numeracy Numeracy Numeracy
X X
X X
Prose Prose Prose
X X
Numeracy Numeracy Prose Document Document
X X X
Prose Document Numeracy
Health navigation
X
X X X X X
Note. Fhl ¼ functional health literacy; Ihl ¼ interactive health literacy; Chl ¼ critical health literacy.
difficulty (Item 4 and Item 13). Item 17 and 22 was eliminated due to low item difficulty and discrimination. On the basis of the item analyses four items were excluded, and additional modifications were made (Table 1). Last, a total of 18 items (functional health literacy: 6 items, interactive health literacy: 6 items, and critical health literacy: 6 items) were retained for the next instrument refinement process.
Instrument Evaluation The evaluation study was conducted with a convenience sample of 40–64-year-old Korean adults from community centers, hospitals, and clinics, welfare centers, and workplaces located in Seoul. Potential subjects were excluded if they had (a) severe hearing or vision problems or (b) if they were unable to give informed consent. On the basis of Everitt’s (1975) recommendations for factor analysis sample size (10:1 ratio of participants to variables), a minimum sample size of 240 was needed for this study. In addition, when the 2-Parameter Logistic Model (2 PLM) is used, 200–400 participants are required to achieve a stable estimation (Henning, 1987). Therefore, a sample size of 315 in this study was considered adequate for the 2 PLM model and factor analysis. In addition, to assess the test retest reliability, a sample of 63 participants was selected to retake the Korean Health Literacy Instrument 10 days after the first interview.
Development of Korean Health Literacy Instrument
259
Data Collection
Downloaded by [McGill University Library] at 17:51 26 February 2015
Informed consent, as approved by the Institutional Review Board of Yonsei Medical Center (IRB No. 4-2010-0520) in Seoul, was conducted before subject participation in the survey. The survey for instrument evaluation was conducted with face-to-face interviews with 336 older adults from December 5 to 24, 2010, for instrument evaluation. Of these interviews, 21 participants did not complete data collection and were excluded. Five interviewers were trained with a standardized protocol. Before data collection, interviewers explained directions for the multiple-choice formats. In addition, the interviewers explained that if a question was skipped, it would be regarded as an incorrect answer and that there was no time limit for the test. A total of 315 surveys were used for the statistical analysis. Statistical Analysis Descriptive analyses were computed for participants’ demographics and health literacy scores using the SPSS 17.0. To apply the item response theory, the unidimensionality (the assumption that items are measuring a single latent trait) and the local independence (the assumption that responses to items are unrelated) of the KHLI was tested. A confirmatory factor analysis (CFA) was performed using the M-plus 2.1 program to examine dimensionality (Muthe´n & Muthe´n, 1998). Weighted least square estimation with robust standard errors was determined using Mplus. Having confirmed the required prerequisite, item response theory was applied. To estimate the item discrimination and item difficulty, two-parameter logistic model for item analysis was performed using the BILOG 3.0 program (SSI Inc.). Corrected item-total correlations, inter-item correlations, and internal consistency (KR 20) were examined.
Results General Characteristics Table 2 shows the general characteristics of the study participants. Of the 315 participants, 55% were female, 41% had completed university or college-level education, and the mean age was 52 years (SD ¼ 7.4 years). The mean time for test administration was 24.29 minutes (range ¼ 10–51 minutes, SD ¼ 8.48 minutes). A total of 246 (78.1%) participants completed the questionnaires without assistance, 59 (18.7%) required assistance, which consisted of having to read aloud the questions. CFA A CFA was conducted to establish the unidimensionality of the three-factor model of the KHLI (Kang, 2011). Three fit indices were used to examine model fit: comparative fit index (CFI), Tucker-Lewis Index (TLI), and RMSEA. The RMSEA value was 0.04 (criterion 0.05 or less), CFI was 0.96 (criterion larger than 0.95), and TLI was 0.97 (criterion larger than 0.95) for the three-factor model in this study. The overall result of the fit indices in CFA was interpreted to support the three dimensions (functional, interactive, and critical health literacy) of the KHLI. Item Response Theory Approach Item response theory was performed to estimate the difficulty and discrimination of the 18 items in the KHLI. Item response theory is a model-based, item-oriented approach to psychometric test development (Embretson & Reise, 2000; Zickar,
260
S. J. Kang et al. Table 2. General characteristics of participants (N ¼ 315)
Downloaded by [McGill University Library] at 17:51 26 February 2015
Characteristic Gender Male Female Age (years) 40–49 50–59 60–64 Marital status Married Nonmarried Education Less than elementary school Middle school High school University or college Graduate school Monthly household income $999 $1,000–$2,999 $3,000–$4,999 >$ 5,000 Health problem Yes No
n
%
173 142
54.9 45.1
134 122 59
42.5 38.7 18.7
260 55
82.8 8.0
22 31 132 114 16
7.0 9.8 41.9 36.2 5.1
45 101 99 75
14.9 33.4 32.8 18.9
174 126
58.0 42.0
1998). It assumes that an examinee’s response to an item on a test is related to a latent trait, which the test is presumed to measure, and that the relation can be represented by a mathematical function (usually an s-shaped logistic function). Table 3 shows the item response theory parameter estimates of location (b) and slope (a). The (b) parameter refers to a point on the latent trait (h) at which there is a 50% probability; high (b) parameter indicates the difficulty of the item (Streiner & Norman, 2008). Overall, in this instrument, (b) parameter ranges from 1.694 to 0.510, and (a) parameter ranges from 0.482 to 1.178. Only one item was classified as being very difficult (2.0 > b 0.5), eight items were moderately difficult (0.5 > b 0.5), and the remaining nine items were classified as easy (2.0 > b 0.5; Baker, 1985). Larger (a) indicates more effective items that are able to discriminate between high and low ability examinees (Ellis & Mead, 2002). In terms of discrimination values (a), 13 items had moderate discrimination (1.34 a 0.65) and five items had a low level of discrimination (0.64 a 0.35). Additional results of the item response theory test information function are presented in Figures 2 through 4.
Reliability The KHLI exhibits good internal reliability (KR ¼ 0.80; DeVellis, 1991). Cronbach’s alpha for the functional, interactive, and critical literacy subscales were 0.663, 0.635, and 0.536, respectively (Table 3). Corrected item to total correlations ranged from 0.204 to 0.481; the majority of items had a correlation with the total scores that was higher than 0.30. Interitem correlations ranged from 0.013 to 0.342. The KHLI showed a good test–retest reliability (Pearson’s r ¼ .89).
Development of Korean Health Literacy Instrument
261
Table 3. Item characteristics: Item difficulty and item discrimination by using item response theory (N ¼ 315)
Downloaded by [McGill University Library] at 17:51 26 February 2015
No.
Variables
2
Fhl 2
3
Fhl 3
4 8
Fhl 4 Fhl 8
9 10
Fhl 9 Fhl10
12
Ihl 1
13 15 16 19 21
Ihl 2 Ihl 4 Ihl 5 Ihl 8 Ihl10
24 27 28 29 31 33
Chl Chl Chl Chl Chl Chl
2 5 6 7 9 11
Questions Medication label use in community pharmacy Selective medical treatment Clinic schedules Nutritional label for serving size Medication time Standard weight Submean Calculation of body mass index Diet management Body-related terms National health insurance Hypertension risk Complication risk Submean Medication direction Obesity Colon cancer screening Hot and cold packs Nutritional facts label 2 Risk perception Submean
Item difficulty
Item discrimination
Item-to-total correlation
–0.076
0.836
.377
–0.258
1.178
.396
–0.795 0.149
1.119 0.841
.481 .400
–1.069 –1.510 –0.593 –0.134
0.648 1.015 0.939 0.860
.320 .388
–1.484 –1.141 0.510 0.233 –0.554 –0.428 –0.832 –0.201 –0.176 –1.694 –1.585 –1.168 –0.942
0.607 0.692 0.482 0.654 0.836 0.688 0.775 0.640 0.814 0.881 0.998 0.542 0.775
.330 .250 .235 .240 .312 .283 .367 .305 .282 .268 .311 .204
Note. Fhl ¼ functional health literacy; Ihl ¼ interactive health literacy; Chl ¼ critical health literacy.
Figure 2. Test information function for functional health literacy. Solid line ¼ information; dashed line ¼ standard error. The test information function indicates how well each ability level is being estimated by the test (Thorpe et al., 2007). Functional health literacy has the most information between 1 and 0 points.
Downloaded by [McGill University Library] at 17:51 26 February 2015
262
S. J. Kang et al.
Figure 3. Test information function for interactive health literacy. Solid line ¼ information; dashed line ¼ standard error.
Figure 4. Test information function for critical health literacy. Solid line ¼ information; dashed line ¼ standard error. Critical health literacy has the most information between 1 and 2 points.
Descriptive Statistics of the KHLI Table 4 shows the association of the KHLI scores with socioeconomic status. The mean score of the KHLI was 11.98 (SD ¼ 3.88) with a range of 0 to 18. Younger, more educated, and wealthier subjects had significantly higher scores.
Discussion Health literacy has been linked to a broad array of health and public health phenomena. To date, such research has not yet blossomed in Korea because of the absence of a culturally appropriate valid health literacy measurement tool. We developed a theory-based comprehensive Korean health literacy item bank which we have taken through content validity, preliminary testing, and instrument evaluation activities with item response theory to create the 18-item KHLI. The test exhibits a good reliability profile and can be used to study health literacy. Our aim was to create a comprehensive measure. Although we did start with a comprehensive item bank of 33 items representing the various aspects of leading
Development of Korean Health Literacy Instrument
263
Table 4. The health literacy scores of the Korean Health Literacy Instrument by general characteristics
Downloaded by [McGill University Library] at 17:51 26 February 2015
Characteristic Gender Male Female Age (years) 40–49 50–59 60–64 Marital status Married Nonmarried (others) Education Elementary school (a) Middle school (b) High school (c) University or college (d) Graduate school (e) Monthly household income $500 (a) $510–1,000 (b) $1,010–2,000 (c) $2,010–3,000 (d) $3,010–4,000 (e) $4,010–5,000 (f) >$5,010 (g) Health problemsb Yes No
n
M
SD
F=t
173 142
11.86 12.11
3.99 3.75
–0.572
.568
134 122 59
13.37 11.53 9.73
3.40 3.80 3.88
21.896
.000
260 55
12.13 11.27
3.90 3.69
1.558
.12
22 31 131 113 16
9.41 8.77 11.39 13.71 14.56
3.59 4.12 3.46 3.23 3.42
20.106
.000
9 36 48 53 57 42 57
9.0 9.72 11.67 11.79 12.72 12.69 13.44
4.56 3.60 3.69 3.53 3.45 3.67 3.82
5.520
.000
179 121
11.74 12.50
0.07 0.10
9.476
.078
Scheffea
p
a, b < c < d, e
a
Journal of Health Communication: International Perspectives Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uhcm20
Development and Evaluation of the Korean Health Literacy Instrument a
b
c
Soo Jin Kang , Tae Wha Lee , Michael K. Paasche-Orlow , Gwang b
Suk Kim & Hee Kwan Won
d
a
Health and Nursing Science , Daegu University , Daegu , Republic of Korea b
College of Nursing , Yonsei University , Seoul , Republic of Korea
c
Section of General Internal Medicine, Department of Medicine , Boston University School of Medicine , Boston , Massachusetts , USA
Click for updates
d
College of Medicine , Konyang University , Daejeon , Republic of Korea Published online: 14 Oct 2014.
To cite this article: Soo Jin Kang , Tae Wha Lee , Michael K. Paasche-Orlow , Gwang Suk Kim & Hee Kwan Won (2014) Development and Evaluation of the Korean Health Literacy Instrument, Journal of Health Communication: International Perspectives, 19:sup2, 254-266, DOI: 10.1080/10810730.2014.946113 To link to this article: http://dx.doi.org/10.1080/10810730.2014.946113
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [McGill University Library] at 17:51 26 February 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Health Communication, 19:254–266, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.946113
Development and Evaluation of the Korean Health Literacy Instrument SOO JIN KANG
Downloaded by [McGill University Library] at 17:51 26 February 2015
Health and Nursing Science, Daegu University, Daegu, Republic of Korea
TAE WHA LEE College of Nursing, Yonsei University, Seoul, Republic of Korea
MICHAEL K. PAASCHE-ORLOW Section of General Internal Medicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
GWANG SUK KIM College of Nursing, Yonsei University, Seoul, Republic of Korea
HEE KWAN WON College of Medicine, Konyang University, Daejeon, Republic of Korea The purpose of this study is to develop and validate the Korean Health Literacy Instrument, which measures the capacity to understand and use health-related information and make informed health decisions in Korean adults. In Phase 1, 33 initial items were generated to measure functional, interactive, and critical health literacy with prose, document, and numeracy tasks. These items included content from health promotion, disease management, and health navigation contexts. Content validity assessment was conducted by an expert panel, and 11 items were excluded. In Phase 2, the 22 remaining items were administered to a convenience sample of 292 adults from community and clinical settings. Exploratory factor and item difficulty and discrimination analyses were conducted and four items with low discrimination were deleted. In Phase 3, the remaining 18 items were administered to a convenience sample of 315 adults 40–64 years of age from community and clinical settings. A confirmatory factor analysis was performed to test the construct validity of the instrument. The Korean Health Literacy Instrument has a range of 0 to 18. The mean score in our validation study was 11.98. The instrument exhibited an internal consistency reliability coefficient of 0.82, and a test–retest reliability of 0.89. The instrument is suitable for screening individuals who have limited health literacy skills. Future studies are needed to further define the psychometric properties and predictive validity of the Korean Health Literacy Instrument.
Health literacy is ‘‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services that are needed Address correspondence to Tae Wha Lee, College of Nursing, Yonsei University, 250 Seongsanro, Seodaemun-gu, Seoul, Republic of Korea. E-mail: [email protected]
254
Downloaded by [McGill University Library] at 17:51 26 February 2015
Development of Korean Health Literacy Instrument
255
to make appropriate health decisions’’ (Institute of Medicine, 2004). Although the importance of health literacy has been well documented in several countries (Agency for Healthcare Research and Quality, 2004); Berkman, Sheridan, Donahue, & Halpem, 2011; Institute of Medicine, 2004; World Health Organization, 1998), little data exist regarding the prevalence of low health literacy in Korea. Several initial studies have reported the prevalence of health literacy among elderly Koreans (Jeong & Kim, 2014; S. H. Kim & Lee, 2008; Lee, Kang, Lee, & Hyun, 2009). A considerable portion, 68.7%, of Korean adults older than 65 years of age had not even completed elementary school (National Institute of the Korean Language, 2008). On the contrary, 89% of 25–64-year-old Korean adults reported to have completed at least a middle school level education (Organisation for Economic Co-operation and Development, 2009). Nevertheless, 32.7% of adults were unable to understand health information, including drug directions (National Institute of the Korean Language, 2008). S. S. Kim, Kim, and Lee (2005) created a Korean language version of the Rapid Estimate of Adult Literacy in Medicine (T. C. Davis et al., 1991). These studies showed that 20–35% of elderly Koreans have limited health literacy. However, research conducted with tools like the Rapid Estimate of Adult Literacy in Medicine is inherently limited (McCormack et al., 2010; Nutbeam, 2009; Pleasant, McKinney, & Rikard, 2011; Sørensen et al., 2012). The most commonly used health literacy instruments, the Test of Functional Health Literacy in Adults (Parker, Baker, Williams, & Nurss, 1995), the Rapid Estimate of Adult Literacy in Medicine, and the Newest Vital Sign (Weiss et al., 2005) evaluate a very narrow band of health literacy skills. The Test of Functional Health Literacy in Adults measures the patient’s ability to read and comprehend text passages (i.e., prose literacy) and passages containing numerical information (i.e., numeracy) using real materials from health care settings. The Rapid Estimate of Adult Literacy in Medicine exclusively measures the ability to correctly pronounce a list of medical terms. The Newest Vital Sign is an effective screening tool; however, it is focused on document and numeracy skills, and there is not validated Korean version of this instrument. Conceptual models of health literacy define multiple additional dimensions and recognize the importance of context (Frisch, Camerini, Diviani, & Schulz, 2012; Nutbeam, 2000; Paasche-Orlow & Wolf, 2007). Measures of health literacy are needed that can capture such complexities in order to be able to differentiate which aspects of health literacy may be appropriately be targeted for intervention. Furthermore, direct translation of individual health literacy instruments from other languages is inappropriate (Han, Kim, Kim, & Kim, 2011; Lee, Kang, Lee, & Hyun, 2009). This is a result of both linguistic and cultural factors. Korean has a much higher degree of phoneme-grapheme correspondence than English; therefore, a broad array of Korean words can be pronounced by people with limited Korean literacy skills with no preexisting knowledge or comprehension for what is being read. In English the grapheme may represent a number of alternative phonemes (e.g., e-a in neat vs. head). The relative low frequency of phoneme-grapheme correspondence in English is precisely the linguistic mechanism by which tools such as the Rapid Estimate of Adult Literacy in Medicine rapidly differentiate levels of literacy skill. As phonological decoding is not as demanding a task in Korean, this linguistic attribute is not a sound basis for a Korean health literacy test. In addition, the health and public health systems in Korea have their own distinct structure, history, and culture. Items in a test like the Test of Functional Health Literacy in Adults are taken from the cultural context of the target population for the test. For example, one of the test passages relates to a patient’s rights and responsibilities in the U.S. government program Medicare. Measures developed for alternative target populations should have culturally appropriate test items.
256
S. J. Kang et al.
The lack of a reliable, valid, and comprehensive skills-based health literacy instrument impedes empirical research in Korea. The purpose of the current study was to develop and conduct initial validation of the Korean Heath Literacy Instrument (KHLI) to fill this void. However, we restricted participants to middleaged adults for this study because of the highly significant gap in educational attainment between younger and elderly Koreans.
Method
Downloaded by [McGill University Library] at 17:51 26 February 2015
This study was conducted in three phases: instrument development and expert review of content validity; (b) preliminary test; and (c) instrument evaluation. Instrument Development and Content Validation After scanning the literature on definitions and conceptual frameworks for health literacy (American Medical Association, 1999; Berkman, Davis, & McCormack, 2010; Freedman et al., 2009; Institute of Medicine, 2004; Jordan, Buchbinder, & Osborne, 2010; Kickbusch, Wait, & Magg, 2005; Nutbeam, 2000; Paasche-Orlow & Wolf, 2007; Paasche-Orlow, Wilson, McCormack, 2010; Parker et al., 1995; Zarcadoolas, Pleasant, & Greer, 2003), we defined that health literacy is a personal skill that determines the ability to understand, interpret, and apply information in order to manage health. We adapted Nutbeam’s conceptual model as a basis for the skills depicted in our taxonomy. Nutbeam (2000) argued that health-literacy skills should be expanded from cognitive skills to social skills and suggested that these skills can be categorized as functional, interactive, and critical skills; in this study, we defined functional health literacy as the ability to read and understand written materials to use with patients in health care. We defined interactive health literacy as ability to extract information, and to derive meaning for communication with health providers. For these items we aimed to verify whether the consumer possesses a prior understanding of the essential terms and basic concepts that are required to achieve proper communication. We defined critical literacy as the ability to appraise health-related information to manage one’s health (Chin & McCarthy, 2013; Ishikawa, Tacheuchi, & Yano, 2008). The skills of the KHLI are based on three types of literacy tasks: prose (understanding text presented in sentences), document (understanding forms and tables), and quantitative (applying arithmetic operations and understanding quantitative information). In addition, these literacy tasks are encountered in the everyday health-related activities of adults in the situation (context) including health promotion, disease management, and health navigation. Figure 1 presents the framework for instrument development. Test item stimuli were selected from health educational materials for chronic diseases used in clinical settings and from other prose and non-prose documents used in clinical and public health settings. For each item, the respondent is presented with instructions and a test stimulus (e.g., excerpt from patient education material), followed by a multiple choice answer set. Initially, 33 items were generated with an additional question about instructions for taking medicine from the national literacy survey (National Institute of the Korean Language, 2008). Functional health literacy items were based on the ability to make an appointment with a doctor, understand medicine instructions, understand nutritional labels, and billing. Interactive health literacy items were composed of the ability to understand health-related terms and concepts, such as estimations (e.g., the concept of a basic unit volumes), as well as understanding criteria and national health insurance terms. Critical literacy items involved the ability to apply information to situations.
Downloaded by [McGill University Library] at 17:51 26 February 2015
Development of Korean Health Literacy Instrument
257
Figure 1. Framework of the study.
Content validity was evaluated by an expert panel of 12 health care professionals, including clinical, academic, and school nurses, a dietician, and practicing physicians. These experts rated each item for relevance, difficulty, and clarity, on a 4-point scale ranging from 1 (not relevant) to 4 (very relevant) and suggested edits. The content validity index was computed with the relevance and appropriateness of each item. Items that did not meet the content validity index of 0.8 (Knapp, 1985; Lynn, 1986), were either eliminated or revised by the expert panel. On the basis of this process, 11 items were excluded. The remaining set of 22 items included 7 functional, 9 interactive, and 6 critical items.
Preliminary Test The preliminary test was conducted in a convenience sample of adult Korean participants at community and hospital settings in Seoul, Korea from November 5 to November 23, 2010. The questionnaires were administered with face-to-face interviews. Informed consent forms were obtained from participants before the survey. Among the 345 questionnaires distributed, 307 were collected and 292 (85% overall response rate) were completed. These data were analyzed with exploratory factor analysis (EFA) and item difficulty and discrimination evaluation. A three-factor model was extracted from EFA on these 22 items. An EFA was conducted to examine the factor structure and to evaluate unidimensionality of this data set. The chi-square test can inflate type 1 errors (Bollen, 1989); thus, other indices were used to judge the model fit. In this study, the model fit was assessed by root mean square error of approximation (RMSEA), which is amenable for analyses with large sample sizes. RMSEA values below .06 indicated a satisfactory model fit (Browne & Cudeck, 1993; Hu & Bentler, 1999). There was no significant difference between models; and one-, two-, and three-factor models had a good fit (RMSEA criterion 0.06 or less). The three-factor model was chosen as the best solution because the model fit index did not show significant differences among the three models and the KHLI was developed based on the three-factor model. After item analysis, 4 items out of 22 items were eliminated from the instrument after preliminary test. Item 5 and Item 18 was deleted because of similar item
258
S. J. Kang et al.
Table 1. Item characteristics Health context Item no. 2 3
Downloaded by [McGill University Library] at 17:51 26 February 2015
4 8 9 10 12 13 15 16 19 21 24 27 28 29 31 33
Contents of questions
Task
Fhl 2 Medication label use in the community pharmacy Fhl 3 Selective medical treatment Fhl 4 Clinic schedules Fhl 8 Nutritional labels for serving sizes Fhl 9 Medication times Fhl 10 Standard weight Ihl 1 Calculation of body mass index Ihl 2 Diet management Ihl 4 Body related terms Ihl 5 National health insurance Ihl 8 Hypertension criteria Ihl 10 Complication risks Chl 2 Medication directions Chl 5 Obesity Chl 6 Colon cancer screening Chl 7 Hot and cold packs Chl 9 Nutritional labels 2 Chl 11 Risk perception
Document
Health promotion
Disease management X
Document
X
Document Numeracy
X
Numeracy Numeracy Numeracy
X X
X X
Prose Prose Prose
X X
Numeracy Numeracy Prose Document Document
X X X
Prose Document Numeracy
Health navigation
X
X X X X X
Note. Fhl ¼ functional health literacy; Ihl ¼ interactive health literacy; Chl ¼ critical health literacy.
difficulty (Item 4 and Item 13). Item 17 and 22 was eliminated due to low item difficulty and discrimination. On the basis of the item analyses four items were excluded, and additional modifications were made (Table 1). Last, a total of 18 items (functional health literacy: 6 items, interactive health literacy: 6 items, and critical health literacy: 6 items) were retained for the next instrument refinement process.
Instrument Evaluation The evaluation study was conducted with a convenience sample of 40–64-year-old Korean adults from community centers, hospitals, and clinics, welfare centers, and workplaces located in Seoul. Potential subjects were excluded if they had (a) severe hearing or vision problems or (b) if they were unable to give informed consent. On the basis of Everitt’s (1975) recommendations for factor analysis sample size (10:1 ratio of participants to variables), a minimum sample size of 240 was needed for this study. In addition, when the 2-Parameter Logistic Model (2 PLM) is used, 200–400 participants are required to achieve a stable estimation (Henning, 1987). Therefore, a sample size of 315 in this study was considered adequate for the 2 PLM model and factor analysis. In addition, to assess the test retest reliability, a sample of 63 participants was selected to retake the Korean Health Literacy Instrument 10 days after the first interview.
Development of Korean Health Literacy Instrument
259
Data Collection
Downloaded by [McGill University Library] at 17:51 26 February 2015
Informed consent, as approved by the Institutional Review Board of Yonsei Medical Center (IRB No. 4-2010-0520) in Seoul, was conducted before subject participation in the survey. The survey for instrument evaluation was conducted with face-to-face interviews with 336 older adults from December 5 to 24, 2010, for instrument evaluation. Of these interviews, 21 participants did not complete data collection and were excluded. Five interviewers were trained with a standardized protocol. Before data collection, interviewers explained directions for the multiple-choice formats. In addition, the interviewers explained that if a question was skipped, it would be regarded as an incorrect answer and that there was no time limit for the test. A total of 315 surveys were used for the statistical analysis. Statistical Analysis Descriptive analyses were computed for participants’ demographics and health literacy scores using the SPSS 17.0. To apply the item response theory, the unidimensionality (the assumption that items are measuring a single latent trait) and the local independence (the assumption that responses to items are unrelated) of the KHLI was tested. A confirmatory factor analysis (CFA) was performed using the M-plus 2.1 program to examine dimensionality (Muthe´n & Muthe´n, 1998). Weighted least square estimation with robust standard errors was determined using Mplus. Having confirmed the required prerequisite, item response theory was applied. To estimate the item discrimination and item difficulty, two-parameter logistic model for item analysis was performed using the BILOG 3.0 program (SSI Inc.). Corrected item-total correlations, inter-item correlations, and internal consistency (KR 20) were examined.
Results General Characteristics Table 2 shows the general characteristics of the study participants. Of the 315 participants, 55% were female, 41% had completed university or college-level education, and the mean age was 52 years (SD ¼ 7.4 years). The mean time for test administration was 24.29 minutes (range ¼ 10–51 minutes, SD ¼ 8.48 minutes). A total of 246 (78.1%) participants completed the questionnaires without assistance, 59 (18.7%) required assistance, which consisted of having to read aloud the questions. CFA A CFA was conducted to establish the unidimensionality of the three-factor model of the KHLI (Kang, 2011). Three fit indices were used to examine model fit: comparative fit index (CFI), Tucker-Lewis Index (TLI), and RMSEA. The RMSEA value was 0.04 (criterion 0.05 or less), CFI was 0.96 (criterion larger than 0.95), and TLI was 0.97 (criterion larger than 0.95) for the three-factor model in this study. The overall result of the fit indices in CFA was interpreted to support the three dimensions (functional, interactive, and critical health literacy) of the KHLI. Item Response Theory Approach Item response theory was performed to estimate the difficulty and discrimination of the 18 items in the KHLI. Item response theory is a model-based, item-oriented approach to psychometric test development (Embretson & Reise, 2000; Zickar,
260
S. J. Kang et al. Table 2. General characteristics of participants (N ¼ 315)
Downloaded by [McGill University Library] at 17:51 26 February 2015
Characteristic Gender Male Female Age (years) 40–49 50–59 60–64 Marital status Married Nonmarried Education Less than elementary school Middle school High school University or college Graduate school Monthly household income $999 $1,000–$2,999 $3,000–$4,999 >$ 5,000 Health problem Yes No
n
%
173 142
54.9 45.1
134 122 59
42.5 38.7 18.7
260 55
82.8 8.0
22 31 132 114 16
7.0 9.8 41.9 36.2 5.1
45 101 99 75
14.9 33.4 32.8 18.9
174 126
58.0 42.0
1998). It assumes that an examinee’s response to an item on a test is related to a latent trait, which the test is presumed to measure, and that the relation can be represented by a mathematical function (usually an s-shaped logistic function). Table 3 shows the item response theory parameter estimates of location (b) and slope (a). The (b) parameter refers to a point on the latent trait (h) at which there is a 50% probability; high (b) parameter indicates the difficulty of the item (Streiner & Norman, 2008). Overall, in this instrument, (b) parameter ranges from 1.694 to 0.510, and (a) parameter ranges from 0.482 to 1.178. Only one item was classified as being very difficult (2.0 > b 0.5), eight items were moderately difficult (0.5 > b 0.5), and the remaining nine items were classified as easy (2.0 > b 0.5; Baker, 1985). Larger (a) indicates more effective items that are able to discriminate between high and low ability examinees (Ellis & Mead, 2002). In terms of discrimination values (a), 13 items had moderate discrimination (1.34 a 0.65) and five items had a low level of discrimination (0.64 a 0.35). Additional results of the item response theory test information function are presented in Figures 2 through 4.
Reliability The KHLI exhibits good internal reliability (KR ¼ 0.80; DeVellis, 1991). Cronbach’s alpha for the functional, interactive, and critical literacy subscales were 0.663, 0.635, and 0.536, respectively (Table 3). Corrected item to total correlations ranged from 0.204 to 0.481; the majority of items had a correlation with the total scores that was higher than 0.30. Interitem correlations ranged from 0.013 to 0.342. The KHLI showed a good test–retest reliability (Pearson’s r ¼ .89).
Development of Korean Health Literacy Instrument
261
Table 3. Item characteristics: Item difficulty and item discrimination by using item response theory (N ¼ 315)
Downloaded by [McGill University Library] at 17:51 26 February 2015
No.
Variables
2
Fhl 2
3
Fhl 3
4 8
Fhl 4 Fhl 8
9 10
Fhl 9 Fhl10
12
Ihl 1
13 15 16 19 21
Ihl 2 Ihl 4 Ihl 5 Ihl 8 Ihl10
24 27 28 29 31 33
Chl Chl Chl Chl Chl Chl
2 5 6 7 9 11
Questions Medication label use in community pharmacy Selective medical treatment Clinic schedules Nutritional label for serving size Medication time Standard weight Submean Calculation of body mass index Diet management Body-related terms National health insurance Hypertension risk Complication risk Submean Medication direction Obesity Colon cancer screening Hot and cold packs Nutritional facts label 2 Risk perception Submean
Item difficulty
Item discrimination
Item-to-total correlation
–0.076
0.836
.377
–0.258
1.178
.396
–0.795 0.149
1.119 0.841
.481 .400
–1.069 –1.510 –0.593 –0.134
0.648 1.015 0.939 0.860
.320 .388
–1.484 –1.141 0.510 0.233 –0.554 –0.428 –0.832 –0.201 –0.176 –1.694 –1.585 –1.168 –0.942
0.607 0.692 0.482 0.654 0.836 0.688 0.775 0.640 0.814 0.881 0.998 0.542 0.775
.330 .250 .235 .240 .312 .283 .367 .305 .282 .268 .311 .204
Note. Fhl ¼ functional health literacy; Ihl ¼ interactive health literacy; Chl ¼ critical health literacy.
Figure 2. Test information function for functional health literacy. Solid line ¼ information; dashed line ¼ standard error. The test information function indicates how well each ability level is being estimated by the test (Thorpe et al., 2007). Functional health literacy has the most information between 1 and 0 points.
Downloaded by [McGill University Library] at 17:51 26 February 2015
262
S. J. Kang et al.
Figure 3. Test information function for interactive health literacy. Solid line ¼ information; dashed line ¼ standard error.
Figure 4. Test information function for critical health literacy. Solid line ¼ information; dashed line ¼ standard error. Critical health literacy has the most information between 1 and 2 points.
Descriptive Statistics of the KHLI Table 4 shows the association of the KHLI scores with socioeconomic status. The mean score of the KHLI was 11.98 (SD ¼ 3.88) with a range of 0 to 18. Younger, more educated, and wealthier subjects had significantly higher scores.
Discussion Health literacy has been linked to a broad array of health and public health phenomena. To date, such research has not yet blossomed in Korea because of the absence of a culturally appropriate valid health literacy measurement tool. We developed a theory-based comprehensive Korean health literacy item bank which we have taken through content validity, preliminary testing, and instrument evaluation activities with item response theory to create the 18-item KHLI. The test exhibits a good reliability profile and can be used to study health literacy. Our aim was to create a comprehensive measure. Although we did start with a comprehensive item bank of 33 items representing the various aspects of leading
Development of Korean Health Literacy Instrument
263
Table 4. The health literacy scores of the Korean Health Literacy Instrument by general characteristics
Downloaded by [McGill University Library] at 17:51 26 February 2015
Characteristic Gender Male Female Age (years) 40–49 50–59 60–64 Marital status Married Nonmarried (others) Education Elementary school (a) Middle school (b) High school (c) University or college (d) Graduate school (e) Monthly household income $500 (a) $510–1,000 (b) $1,010–2,000 (c) $2,010–3,000 (d) $3,010–4,000 (e) $4,010–5,000 (f) >$5,010 (g) Health problemsb Yes No
n
M
SD
F=t
173 142
11.86 12.11
3.99 3.75
–0.572
.568
134 122 59
13.37 11.53 9.73
3.40 3.80 3.88
21.896
.000
260 55
12.13 11.27
3.90 3.69
1.558
.12
22 31 131 113 16
9.41 8.77 11.39 13.71 14.56
3.59 4.12 3.46 3.23 3.42
20.106
.000
9 36 48 53 57 42 57
9.0 9.72 11.67 11.79 12.72 12.69 13.44
4.56 3.60 3.69 3.53 3.45 3.67 3.82
5.520
.000
179 121
11.74 12.50
0.07 0.10
9.476
.078
Scheffea
p
a, b < c < d, e
a
