Revision of the Osteoporosis Knowledge Test: Reliability and Validity.

537565 research-article2014

WJNXXX10.1177/0193945914537565Western Journal of Nursing ResearchGendler et al.

Article

Revision of the Osteoporosis Knowledge Test: Reliability and Validity

Western Journal of Nursing Research 1–21 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0193945914537565 wjn.sagepub.com

Phyllis Ellen Gendler1, Cynthia Peltier Coviak1, Jean Thomas Martin1, Katherine K. Kim1, Jennifer K. Dankers1,2, Julieanne Marie Barclay1, and Thomas A. Sanchez1

Abstract The purpose of this study was to revise the Osteoporosis Knowledge Test (OKT) and evaluate its reliability and validity. The original OKT, developed in the early 1990s, needed updating based on current research. A convenience sample of 105 adults completed the draft revised OKT. A subsample (n = 27) completed the questionnaire 2 weeks later to determine stability. The sample was recruited from diverse sites in western and northern Michigan over a year. The 32-item Revised OKT (2012) demonstrated internal consistency (total scale Kuder–Richardson-20 = .85, Nutrition subscale = .83, and Exercise subscale = .81). Test–retest analysis resulted in a Pearson correlation coefficient of .87. Validity was evaluated by content validity. Questions were examined for difficulty, effectiveness of distracters, and discrimination. In addition, measures of point-biserial, internal consistency and stability were determined. The Revised OKT (2012) is a comprehensive instrument reflecting current research and assesses osteoporosis knowledge of adults. 1Grand 2Heart

Valley State University, Grand Rapids, MI, USA Failure Clinic at MidMichigan Physicians Group Cardiology, Midland, USA

Corresponding Author: Phyllis Ellen Gendler, Professor Emerita of Nursing, Kirkhof College of Nursing, Cook-DeVos Center for Health Sciences, Grand Valley State University, 301 Michigan St. N.E., Grand Rapids, MI 49503-3314, USA. Email: [email protected]

Downloaded from wjn.sagepub.com at MOUNT ALLISON UNIV on June 15, 2015

2

Western Journal of Nursing Research

Keywords instrument development, osteoporosis knowledge, validity, reliability

Osteoporosis is a disease of bone architecture that is currently estimated to affect 10 million Americans (National Osteoporosis Foundation [NOF], 2010). As a cause of bone weakness, it is thought that half of all women and one quarter of men will suffer a fracture because of this disease. Costs of osteoporosis were estimated at US$17 billion in 2005, particularly because of the number of fractures that occur as a result. It is estimated that by 2040, costs associated with osteoporotic fractures and their consequences will double or triple up to US$51 billion. These consequences include pain and disability, which affect quality of life (NOF, 2010). Because of the costs and consequences of osteoporosis, it is important for health professionals to develop interventions to prevent and treat this disease. Encouraging behaviors that promote bone health starts with ensuring adequate osteoporosis knowledge.

Development of Osteoporosis Knowledge Test (OKT) Over 20 years ago, a set of instruments addressing osteoporosis health beliefs was developed and tested through interview administration. Using the health belief model (HBM; Rosenstock, Strecher, & Becker, 1988) as the framework for the instruments, these researchers created self-reports of osteoporosis health beliefs (Kim, Horan, Gendler, & Patel, 1991) and osteoporosis self-efficacy (Horan, Kim, Gendler, Froman, & Patel, 1998). These questionnaires are now self-administered in most studies. The third instrument developed (OKT) assessed knowledge regarding risks of the disease, intake of foods and drinks that provide calcium, and appropriate exercise for the prevention of osteoporosis (Kim, Horan, & Gendler, 1991). The OKT consists of 24 dichotomous items: 9 items on general osteoporosis risk, 8 items on calcium intake, and 7 items on exercise behaviors. Content of the test was based on literature concerned with osteoporosis. In addition, a panel of six judges consisting of nursing and physical therapy faculty and registered dieticians determined whether the questions on the test reflected the kind of information the study participants needed to know to prevent osteoporosis. The OKT was used in its entirety, or as a calcium scale (risk factors and calcium items) or exercise scale (risk factors and exercise items); thus, reliability coefficients for internal


3

Gendler et al.

consistency were calculated separately yielding a Kuder–Richardson-20 (KR20) of .72 for the calcium scale and .69 for the exercise scale. The validity of the OKT has been supported by its widespread use in various cultures. Multiple researchers have requested the OKT and used it in their studies. For example, in addition to the United States (Doheny, Sedlak, Estok, & Zeller, 2007; Qi, Resnick, Smeltzer, & Bausell, 2011), it has been used in other English-speaking countries such as Australia, England, and New Zealand (von Hurst & Wham, 2006). It has also been used and translated into multiple languages, for example, for Asian subjects (Chen, Liu, & Cai, 2005) and Spanish-speaking subjects (Elliott, Seals, & Jacobson, 2008). These researchers have demonstrated sound psychometrics of the OKT in these varied samples with minimal modifications of the instrument. Their modifications have addressed wording and food choices that were more familiar to the culture (see Table 1). When the OKT (Kim, Horan, & Gendler, 1991) was originally developed in the early 1990s, research regarding the process of bone accretion and remodeling in relation to bone density and total bone mass was in its infancy. It was clear that the OKT was in need of updating, and testing of reliability and validity of the updated version was necessary. Therefore, the purpose of this study was to determine the reliability and validity of a revised version of the OKT, which included new items. In addition, evidence was needed that the Revised OKT (2012) would be psychometrically sound when self-administered.

Theoretical Framework The OKT (Kim, Horan, & Gendler, 1991) was originally one of three instruments created to measure concepts of the HBM (Rosenstock et al., 1988) in relation to the behaviors and risk factors associated with osteoporosis. The other two instruments addressed osteoporosis health beliefs and osteoporosis self-efficacy for exercise and calcium intake. As a test of knowledge, the OKT represented the concept of a modifying factor in the model. Although knowledge and skills gained from health education do not always translate into subsequent desirable behavior (Becker & Janz, 1985), one must have the requisite knowledge to develop realistic health beliefs. When mediated by health beliefs and self-efficacy, research has shown that osteoporosis knowledge positively influences health behaviors (Chang, 2008; Sedlak, Doheny, & Jones, 2000). It is therefore important to evaluate research methods of measuring osteoporosis knowledge to ensure that the most accurate assessment is made.


4


622 women Ages 20-49

English

Added: two questions on vitamin D Web-based

KR20 Total = .78 Calcium = .74 Exercise = .7

Correlation: Calcium r = .91 Exercise r = .98

a. Personal communication, June 1, 2005.

Note. List of other researchers who have used OKT may be requested from author. Researchers have used the OKT in both descriptive and experimental research. OKT = Osteoporosis Knowledge Test; KR20 = Kuder–Richardson-20.

New Zealand

Nutrition

von Hurst and Wham (2006)

Mandarin

72

United States Chinese immigrants

Nursing Psychology

Qi, Resnick, Smeltzer, and Bausell (2011)

Pilot: test–retest

English Spanish

Suggested need to modify for ethnicities

English

United States

Physical therapy

Mageea

Reliability

Cronbach’s alpha range: Total Added two questions: smoking scale = .83 to Calcium subscale and heavy drinking. Changed = .87; test−retest correlation cheese to soy bean, sardines to range: Total scale = .82 to shrimp, ice cream to bean curd Exercise subscale = .75. Item analysis differentiated higher and lower scoring groups

Modifications

English

Cantonese

Language

8 15-18 years

94 adult men and women with epilepsy

United States

Data management Public health Medicine

Elliott, Seals, and Jacobson (2008)

291 middle aged/ elderly 66 retest

Sample

444 men and women

Chinese

Ethnicity

United States

Nursing

Researchers’ Field

Nursing Doheny, Sedlak, Estok, and Zeller Statistics (2007)

Chen, Liu, and Cai (2005)

Researchers

Table 1. Researchers’ Use of Original Osteoporosis Knowledge Test.

5

Gendler et al.

Method Test Construction Procedures Methods for creation of criterion-referenced measures, as discussed by Waltz, Strickland, and Lenz (2010), were used to revise the OKT. The first step included item development based on the current literature. Second, there was testing of the items on the instrument with adult respondents in two phases. Finally, item analysis which included item difficulty, item discrimination, and effectiveness of distracters was completed. In addition, measures of internal consistency and stability were determined. Item generation for the Revised OKT (2012). The first step was to review current osteoporosis literature to determine relevancy and currency of questions on the OKT (Kim, Horan, & Gendler, 1991) and the need for additional questions. The literature review is organized based on the domains of the instrument: risk factors, exercise, nutrition, and general. Table 2 compares the focus of the questions on the original OKT with related questions on the Revised OKT (2012), organized by domains. Note that the general domain is not included in Table 2 as it was not included in the original OKT. It is a new three-item addition to the Revised OKT (2012) and includes questions addressing diagnosis, treatment, and bone development. Other questions added to the Revised OKT (2012), not listed in Table 2, include factors increasing risk, such as being an elderly man, alcohol use, smoking, overweight, and eating disorders. In the nutrition domain, four questions on vitamin D were added. Finally, as a result of the review, it was determined that most of the questions from the original OKT were still relevant. The modifications made to the original OKT by multiple researchers, who have added a variety of questions or revised particular items, were also considered (see Table 1). Overall, changes were made to update it to include current recommendations for calcium and vitamin D, exercise requirements, diagnosis and treatment, and a more comprehensive understanding of bone development and osteoporosis risk factors. Risk factors. Although genetics is reported to be responsible for between 60% (Rizzoli, Bianchi, Garabedian, McKay, & Moreno, 2010) and 80% (Boreham & McKay, 2011) of the variance in peak bone mass (PBM), modifiable risk factors including adequate intake of vitamin D and dietary calcium and weight-bearing physical activity are important to maximize potential PBM. While some risk factors for osteoporosis are non-modifiable, according to the HBM, the perceived threat of osteoporosis should serve as motivation to decrease as many modifiable risks as possible.


6


Table 2. Comparisons of Original Osteoporosis Knowledge Test (OKT) Items with the Revised OKT (2012). Original OKT Question Eating a diet LOW in MILK products Being menopausal; “change of life” Having big bones Eating a diet high in dark green leafy vegetables MOTHER/GRANDMOTHER osteoporosis Being a White woman WITH FAIR SKIN Having ovaries surgically removed Taking cortisone (steroids, for example, Prednisone) for long time Exercising on a regular basis Which EXERCISE is best way to reduce osteoporosis? Which EXERCISE is best way to reduce osteoporosis? Days a week exercise to strengthen bones? LEAST TIME exercise each occasion? Exercise makes bones strong, but it must be hard enough to make breathing: EXERCISE best way to reduce osteoporosis? EXERCISE best way to reduce osteoporosis? Which of these is A GOOD source of calcium? Which of these is A GOOD source of calcium?

Revised OKT (ROKT; 2012) Question

Domain

Eating a diet LOW in DAIRY products No change

Risks

Omitted after final psychometric results Omitted

PARENT/GRANDPARENT osteoporosis Being a White OR ASIAN woman No change

Risks

No change

Risks

Omitted Which ACTIVITY is best way to reduce osteoporosis? Which ACTIVITY is best way to reduce osteoporosis? To strengthen bones, exercise at a moderately intense level for 30 minutes a day at least Omitted

Exercise

Exercise makes bones strong, but it must be hard enough to make breathing: ACTIVITY best way to reduce osteoporosis? ACTIVITY best way to reduce osteoporosis? Which of these is THE BEST source of calcium? Which of these is THE BEST source of calcium? (response choices updated)

Risks

Risks Risks

Exercise Exercise Exercise Exercise Exercise Nutrition Nutrition

(continued)


7

Gendler et al. Table 2. (continued) Original OKT Question Which of these is A GOOD source of calcium? Which of these is A GOOD source of calcium? Which of these is A GOOD source of calcium? Recommended calcium intake for an adult? Milk adult drink for calcium? Best reason for taking a calcium supplement?

Revised OKT (ROKT; 2012) Question Which of these is THE BEST source of calcium? Which of these is THE BEST source of calcium? Which of these is THE BEST source of calcium? No change to question (response choices updated) No change to question (response choices updated) No change

Domain Nutrition Nutrition Nutrition Nutrition Nutrition Nutrition

Note. Responses for risk items for osteoporosis: more likely, less likely, neutral, don’t know. Word changes in items are capitalized. OKT = Osteoporosis Knowledge Test.

The risk factors supported by current, original research and position statements from professional organizations (North American Menopause Society [NAMS], 2010) were included in the Revised OKT (2012). Two new questions about risks in White or Asian women and elderly men are substantiated in the literature as our population ages (Looker, Borrud, Dawson-Hughes, Shepherd, & Wright, 2012; Rutecki, 2008). Unfortunately, calcium intake deficiency has continued to be cited as the leading modifiable risk factor for bone health and osteoporosis prevention (Chevalley, Bonjour, Ferrari, Hans, & Rizzoli, 2005; Prince, Devine, Dhaliwal, & Dick, 2006). A question on smoking was added to the Revised OKT (2012) because of significant evidence that smoking has harmful effects on bone metabolism (U.S. Department of Health and Human Services [USDHHS], 2004). Excessive alcohol is also a risk factor for osteoporosis (NAMS, 2010), so a question was added to the Revised OKT (2012). Thinness, which could be associated with eating disorders, is a risk (NAMS, 2010). Thus, questions addressing weight (overweight) and eating disorders were added. Exercise. When the original OKT was developed, there were limited recommendations for exercises specific for strengthening bone. General weightbearing exercises such as walking were considered beneficial for bone health as well as for cardiac health. Exercise research has continued to demonstrate benefits to bone health at every age (USDHHS, 2004; World Health


8


Organization [WHO], 2010). The current recommendations for exercise to build and maintain bones include 150 min of moderate exercise each week, as well as muscle strengthening activities (American College of Sports Medicine, 2010; WHO, 2010). Thus, the exercise questions were modified on the Revised OKT (2012) to reflect these guidelines. Nutrition. The nutrition domain includes the two most crucial nutrients for bone health: calcium and vitamin D. Updated research has influenced both national and international groups (International Osteoporosis Foundation [IOF], 2012; NOF, 2010) to increase calcium intake recommendations to 1,300 mg for adolescents (9 or 10-13 years of age), 1,000 mg daily for adults (19-50 years), and 1,200 mg daily for older adults (>50 years) with the exception of the IOF, which recommends 1,300 mg daily for postmenopausal women (despite age) and men over 65 years of age. Therefore, the Revised OKT (2012) has modified calcium questions to reflect these recommendations for increased daily calcium intake. The original OKT (Kim, Horan, & Gendler, 1991) was developed prior to the expansion of vitamin D research and knowledge of its relationship to osteoporosis prevention. Vitamin D deficiency has been well established as a widespread condition that influences a spectrum of bone-related diseases. Vitamin D research-based recommendations have subsequently been reviewed and endorsed by national and international organizations. Vitamin D insufficiency is not only related to intake but also to exposure to sun. The season, latitude, time of day of exposure, skin pigmentation, age-related skin changes, sunscreen use, and the amount of ultraviolet (UVB) light filtered through windows are factors affecting vitamin D absorption from the sun (Holick, 2004). The majority of vitamin D is obtained from exposure to UVB rays in sunlight (Holick, 2004). To a much smaller degree, vitamin D is supplied through dietary consumption of foods, including oily fish, eggs, and various fortified foods. Both the IOF (2012) and the NOF (2010) have age-specific recommendations for vitamin D intake of 800 to 1,000 IU daily for older adults. There is some controversy about what the requirements should be as the field is changing constantly, and all the authorities do not agree. However, because the IOF (2012) and NOF (2010) are reputable organizations and their information is accessible to the public, their recommendations were used. Therefore, the Revised OKT (2012) provides questions related to the role of vitamin D in bone health, the recommended daily intake of 800 to 1,000 IU, and questions regarding knowledge of both dietary and non-dietary sources of vitamin D. General. The general domain was not a part of the original OKT (Kim, Horan, & Gendler, 1991). The general domain in the Revised OKT (2012) includes a question each on diagnosis and treatment of osteoporosis, and Downloaded from wjn.sagepub.com at MOUNT ALLISON UNIV on June 15, 2015

9

Gendler et al.

bone development. When the OKT was first developed, there were no sophisticated measures of bone density available. Current practice is to perform dual X-ray absorptiometry (DXA) testing when there are risk factors for fracture (NAMS, 2010). Thus, a question about diagnosing osteoporosis was included in the Revised OKT (2012). A question about treatment was included in the Revised OKT (2012) because there are now many pharmacological options for treating osteoporosis (NAMS, 2010). Developing healthy bones early in life is critical to maximizing PBM and bone density, which can reduce the risk of osteoporosis in later life (Jackowski et al., 2011). Beginning prenatally and continuing through adolescence, bone continues to grow in length, width, and density (Boreham & McKay, 2011; Rizzoli et al., 2010). The most dramatic changes occur with the onset of puberty, primarily during Tanner stages 2 and 3 (Welch & Weaver, 2005), or ages 11 to 14 in girls and 12 to 17 in boys (Jackowski et al., 2011). During this time period, up to 34% (Jackowski et al., 2011) of adult bone mass is attained. Because of the critical timing of bone development during adolescence, a question about it was included in the Revised OKT (2012). In summary, the reviews of literature and recent feedback from various researchers, who had used the OKT in their research, were utilized to revise items from the original OKT and create new items for the Revised OKT (2012). Instrument testing. Instrument testing was accomplished in two phases. Each phase had a different group of participants. The following procedures were the same in each phase. Data collection. The study procedures were reviewed by the Human Research Review Committee of Grand Valley State University and determined by that body to be an investigation that was exempt from the Code of Federal Regulations Title 45, part 46, Protection of Human Subjects (Office for Human Research Protections, 2010). The participants were then recruited by faculty, research assistants, and graduate students, who were trained in research procedures. Recruitment included contact with the appropriate point person of the facility and flyers explaining the research. Standard scripts were used to explain the study, its purposes, and the handling of the surveys following completion by the participants. After the research was explained, faculty, research assistants, and graduate students distributed the instruments and collected them when completed. Informed consent was implied by the completion of the self-administered questionnaires on-site. Initial data management. Initial analyses were performed using SPSS 17.1 statistical software and were verified by re-analyzing with SAS 9.2 statistical software. Scantron PARSCORE software (R. Johnson, personal Downloaded from wjn.sagepub.com at MOUNT ALLISON UNIV on June 15, 2015

10


communication, January 13, 2014) was used for item analyses. The initial examination of the responses to items included a determination of the count and percentage of the participants choosing each of the four potential answers to the questions. Although the responses offered to participants were multiple choice, there was only one correct answer. Thus, the items were constructed to create a criterion-referenced measure of knowledge, rather than attitudes about osteoporosis. Analyses were designed to correspond to those described by Waltz and colleagues (2010) and in educational literature (Twigg, 2009; Varma, n.d.) for these types of measures. Consistent with the procedures used for designing criterion-referenced measures and the original OKT, participants’ responses to the items were recoded as correct (coded as 1) or incorrect (coded as 0). With this coding, each item on the scale was dichotomous. The correct answers were counted to create a raw score. From the raw scores, the percentage of correct answers was then computed. To assess validity and reliability of the revised OKT, both item and scale analyses were completed. For items, the effectiveness of distracters (i.e., the wrong, but plausible responses), item difficulty and discrimination, and point-biserial correlations were examined. For the scale, the KR20 statistic, a test that parallels Cronbach’s alpha coefficient but is suitable for dichotomous items was used to determine an estimate of internal consistency. Stability was assessed through correlation of the percentage of correct items from two administrations of the test, occurring approximately 2 weeks apart. When examining items, recommendations from Twigg (2009) and Varma (n.d.) regarding desirable parameters for each characteristic were used. Response options that were incorrect distracters were considered acceptable if they were chosen by at least one of the respondents. To accurately test knowledge and avoid guessing, questions should have distracters that are not obviously wrong. The proportion (P) of respondents answering correctly represented the item difficulty, targeted to be between .85 (easy) and .20 (difficult). However, all elements of the item analysis were considered in conjunction with the others. Items were acceptable discriminators if the discrimination (D) exceeded .20, although D = .30 was consistent with the magnitude identified by Twigg (2009) as “good.” Point-biserial correlations between the scores respondents received on an item and the total scores received are correlations between a dichotomous variable (one correct answer) and a continuous variable (the total score on the test). Varma’s (n.d.) recommendation for a minimum point-biserial correlation of .15 was used. Utilizing Twigg’s (2009) recommendation for internal consistency of the scale, a KR20 coefficient of .70 was considered sufficient. In determining stability, desired power of the Pearson correlation coefficient of at least .70 was determined a priori. For an alpha of .05, with r = .70, power of .90 could


11

Gendler et al.

be obtained with a sample size greater than or equal to 13 (Cohen, 1988). With n = 16, power for r = .70 would reach .95, and power would reach at least .99 for n ≥ 23 (Cohen, 1988). All power estimates would represent a large effect size. Therefore, a minimum sample size of 20 was established for the desired coefficient of stability. Phase 1. This first phase involved a pilot test of a draft revised OKT containing 32 items, to verify ease of interpretation of the new items, determine the appropriateness of the response distracters, and to assess the clarity of instructions added for self-administration. This draft and a demographic questionnaire were administered over a 2-week period at a recreation center of a university. Forty-seven adult participants, 18 to 46 years old, completed the self-administered questionnaires on-site. The sample included 30 females and 16 males. Education ranged from 12 to 20 years of school. Most subjects identified themselves as White with the remaining respondents identifying themselves as Asian (n = 1), Black (n = 1), Latino (n = 1), Pacific Islander (n = 1), other (n = 1), and multiracial (n = 5). Phase 2. Based on the pilot test (Phase 1) and further review of the literature, a 33-item draft scale was developed to test in Phase 2. To develop this scale, two exercise questions from the draft revised OKT containing 32 items were combined into one to clarify the amount of exercise recommended each week (the number of days to exercise and the total time of exercising). This reduced the 32-item scale administered in Phase 1 to 31 items. Then two additional questions were added about vitamin D. These additional two questions resulted in the 33-item draft revision of the OKT, which was then administered in Phase 2. The 33-item draft revision of the OKT is the version subjected to extensive analysis of reliability and validity. Sample and setting. A convenience sample of 105 adults above the age of 18 was recruited from diverse sites in western and northern Michigan over the period of a year. All subjects were able to read English. The data collection sites included private homes, senior centers, and churches. Instruments. Two questionnaires were used in this investigation. The first was a demographic questionnaire. The demographic questionnaire asked the gender, age, race, and educational level of study participants. The second was the 33-item draft revision of the OKT. Procedure. This investigation was carried out in a variety of communities in western and northern Michigan. The 33-item draft revision of the OKT was


12


administered to a sample of adults of various ages. A subsample completed the instrument on two occasions to allow determination of the scale’s stability.

Results Sample The sample (N = 105) for Phase 2 of the study was comprised of 32 male (30.5%) and 73 female (69.5%) adult participants above the age of 18 (24-91, M = 51.7, SD = 25.7). The participants identified themselves as White (88%, n = 93), Black (3.8%, n = 4), and Native American (3.8%, n = 4); the remainder had missing data. Education ranged from 8 to 23 years with 43 (41.0%) participants having a bachelor’s degree or higher.

Evaluation of Instrument Items: Reliability and Validity Score distributions, item analysis, point-biserial correlations, internal consistency, and stability reliability were examined to determine which questions from the 33-item draft revision of the OKT would be retained for the Revised OKT (2012). If problematic, item characteristics can reduce a test’s reliability and validity (Varma, n.d.). Criteria discussed in an earlier section were used for evaluating items and the overall scale (see Table 3). Item analysis. To consider validity of items, the distribution of participant scores was determined. The 33-item draft revision of the OKT exhibited a wide distribution of raw scores, ranging from 0 (0%) to 30 (90.9%) items answered correctly by participants (M = 52.9%, Mdn = 55, Mo = 58, SD = 6.2%). Item analysis was done to determine each individual item’s difficulty, discrimination, and the effectiveness of distracters. Table 3 illustrates the analysis of item difficulty indicated by the proportion (P) of participants choosing the correct responses to individual items. These ranged from 94 to 14. Item discrimination indices ranged from 14 to 75 for the 13 questions that were most difficult for the participants (answered correctly by fewer than 50% of the sample). In addition to evaluating item difficulty and item discrimination, the importance of the question in terms of an individual’s understanding of bone health was considered. Finally, the effectiveness of distracters was considered. Except for one response option in one question, all of the possible responses were chosen by at least one person. In summary, the items demonstrated acceptable difficulty, discrimination, and distracters. None were eliminated as a result of item analysis.


13

Gendler et al. Table 3. Item Analysis, Reliability, and Validity of 33-Item Draft Revision of the OKT. Possible Answers

KR20 Without Item

1

2

3

4

1. Diet low in dairy

.84

76a

2. Menopausal 3. Big bones 4. P arent/grandparent with osteoporosis 5. White or Asian woman

.84 .85 .84

72a 6 79a

11 5 28a 3

6 5 39 8

8 10 26 12

61 61 25 46

.75 .73 .28 .77

.52 .45 .24 .47

.85

48a

6. Elderly man 7. Ovaries removed 8. Cortisone long time 9. Overweight 10. Eating disorder 11. More than two alcohols daily 12. Smoking daily 13. Days per week exercise 14. Exercise hard enough 15. Walking is best activity 16. Lifting weight is best activity 17. Jogging is best activity 18. Dance is best activity 19. Cheese as best calcium 20. Sardines is best calcium 21. Broccoli is best calcium 22. Yogurt is best calcium

.85 .84 .84 .84 .84 .84 .84 .85 .85 .84 .84 .85 .84 .84 .84 .85 .85

37a 42a 52a 41 71a 37a 55a 64 28 27 15 64a 8 2 28 15 89a

10 17 8 10 16a 7 8 5 15 53a 51a 15 4 1 96a 3 61a

12 18 16 8 20 9 28 16 19a 3 10 49a 11 71a 1 52a 3

30 25 33 31 26 13 29 23 5 17 12 21 21 21 3 17 21

43 21 71 68 36 64 68 75 14 54 61 75 29 46 25 61 36

.48 .38 .42 .51 .16 .71 .36 .56 .18 .52 .51 .49 .64 .70 .94 .52 .61

.36 .17 .57 .48 .43 .53 .55 .61 .24 .35 .47 .58 .28 .44 .47 .51 .32

23. Ice cream is best calcium 24. Calcium intake 25. Glasses of milk 26. Calcium supplement 27. Vitamin calcium absorption 28. Source vitamin D 29. Food to absorb calcium 30. Amount vitamin to absorb calcium 31. Time to build strong bones

.84 .85 .84 .85 .84 .84 .85 .85

79a 15 20 0 4 5 33 27a

5 4 47a 27 79a 6 9 21 39

1 7 14 41a 21 79a 71a 22a 9

7 12 26 14 3 14 17 27 28

21 46 32 64 21 50 57 25 29

.87 .77 .46 .40 .77 .77 .70 .21 .26

.39 .44 .31 .54 .32 .32 .49 .21 .23

.85

71

14a

32. Diagnose osteoporosis 33. Treat osteoporosis

.84 .84

6 1

58a 81a

11 8 7

6 30 11

21 61 46

.14 .57 .81

.26 .48 .44

Item

Discrimination Difficulty Index Index

PointBiserial

Note. Possible answer 4 is “do not know.”aCorrect answer.

Point-biserial. Point-biserial strategies were used to evaluate validity. All of the questions met the recommended point-biserial correlation of at least .15


14


and most reached what is considered a good level of at least .25 (Varma, n.d.; see Table 3). Items with less than a .25 point-biserial correlation were examined and considered for inclusion based on their importance and relevance to knowledge about osteoporosis and if they had been part of the original OKT. Following point-biserial analysis, all the questions from the 33-item draft revision of the OKT were maintained. Reliability. Reliability was evaluated by determining both internal consistency and stability of the 33-item draft revision of the OKT. Internal consistency was evaluated by calculating KR20 for the 33-item draft revision of the OKT. The KR20 was .85. Then, to determine whether questions could be eliminated from the 33-item draft revision of the OKT, to reduce the length and minimize respondent fatigue, a KR20 was calculated with each item removed separately (see Table 3). After deliberation, one item, the “big bones” risk question, was omitted. This question was difficult and was considered of lesser importance and relevance to knowledge about osteoporosis. The KR20 for the resulting 32-item Revised OKT (2012) was .85. The original OKT was designed to be used either as a Calcium subscale, an Exercise subscale, or a Total scale. In the Revised OKT (2012), these subscales are Nutrition and Exercise, which share 14 common items including risk factors and general knowledge (bone development, diagnosis, and treatment). KR20 coefficients for the 32-item Revised OKT (2012) were as follows: Total scale = .85; Nutrition subscale = .83; Exercise subscale = .81. To evaluate stability, test–retest analysis was done. The Pearson correlation coefficient was .87 (n = 27; p < .0001). This suggests very good stability of the Revised OKT (2012). Content validity. As previously discussed, the Revised OKT (2012) is based on the original OKT. Content validity of the original OKT was established through extensive review of the literature on osteoporosis as well as review by an expert panel. Fourteen of the 32 questions in the Revised OKT (2012) have been retained without modification from the original OKT (see Table 2). Use of the OKT by multiple researchers with expertise representing different disciplines (see Table 1) also supports its content validity. As the foundation of the Revised OKT (2012) is the valid original OKT, and changes to the original OKT were justified based on recent professional guidelines and suggestions from the variety of researchers who have used the original OKT, content validity of the Revised OKT (2012) is supported.


15

Gendler et al.

Discussion The focus of the Revised OKT (2012) is bone health promotion knowledge. The instrument has been established as valid and reliable when selfadministered. It provides researchers the opportunity to use an up-to-date instrument to assess osteoporosis knowledge with adults across multiple settings. There are various researcher-developed instruments for measuring osteoporosis knowledge (Ailinger, Harper, & Lasus, 1998: Facts On Osteoporosis Quiz [FOOQ]; Ailinger, Lasus, & Braun, 2003: Revised FOOQ; Berarducci, Lengacher, & Keller, 2002: Osteoporosis Knowledge Questionnaire [OKQ]; Pande et al., 2000: Osteoporosis Questionnaire [OPQ]; Winzenberg, Oldenburg, Frendin, & Jones, 2003: Osteoporosis Knowledge Assessment Tool [OKAT]). Table 4 compares the Revised OKT (2012) with these cited osteoporosis knowledge instruments that are designed to be used with adults, include a similar number of items, and address common issues related to osteoporosis knowledge. These instruments had sufficient details reported in the publication to make a comparison. As can be seen from Table 4, the Revised OKT (2012) has a number of strengths. It is comprehensive and reflects current research on osteoporosis risk factors, nutrition, exercise recommendations, and general issues (i.e., bone development, diagnosis, and treatment). There are clusters of questions to measure the domains of knowledge, which add to the validity and reliability of the instrument. It avoids true/false questions and offers a “don’t know” response, which minimizes guessing. It has been tested on both women and men with a wide age range. There was extensive psychometric analysis before the final version was accepted. The Exercise subscale and the Nutrition subscale each have good internal consistency, and can be used alone when the focus of the intervention is solely on one or the other. The Revised OKT (2012) incorporated suggestions of researchers from a wide variety of cultures. When administering the Revised OKT (2012) with any other instrument, it should be administered first to ensure that responses have not been influenced by information from the other instruments. As the Revised OKT (2012) is currently used in a variety of cultures, it will be necessary to evaluate the effect of the content changes that are made for these cultures and their respective language translations. It will also be necessary to evaluate the effectiveness of distracters for the population of interest. In addition, food questions may need to be modified for a specific region or culture.


16


Item analysis Difficulty Discrimination Reliability KR202

Age

Gender

Nutrition Physical activity Risk factors Diagnosis Treatment Bone development Item format Sample

Number of items

Instrument Characteristics

.12-.66 D-value = 44%1 α = .69

.85

25-44 years M = 37.8

.83a α = .76b

.20-.80a >.20a,b

M = 33

TF, DK N = 104a N = 256b Women

TF, DK N = 467 Healthy women

X X X

25a, 20b

FOOQ (Ailinger, Harper, & Lasus, 1998)a (Ailinger, Lasus, & Braun, 2003)b

X X X

20

.14-.94 14-75

73 female 32 male 24-91 M = 51.7

32 two subscales X X X X X X MC, DK N = 105

Revised OKT (2012)

OKAT (Winzenberg, Oldenburg, Frendin, & Jones, 2003)

Table 4. Comparison of Tests of Osteoporosis Knowledge.

M = 50.7

RNs

MC N = 81

X X X X X

22

OKQ (Berarducci, Lengacher, & Keller, 2002)

(continued)

α = .84

.2-.7 41.6-66.6

above 50

Women

X X X MC, DK N = 50

20

OPQ (Pande et al., 2000)


17

.3-.7

t = 3.21 p = .003

CVI = .96

CVI = .92a CVI = .87b

X

X

r = .77

OKQ (Berarducci, Lengacher, & Keller, 2002)

r = .87

Revised OKT (2012)

OKAT (Winzenberg, Oldenburg, Frendin, & Jones, 2003)

FOOQ (Ailinger, Harper, & Lasus, 1998)a (Ailinger, Lasus, & Braun, 2003)b

p = .0034

OPQ (Pande et al., 2000)

Note. MC = multiple choice; DK = do not know; TF = true/false; OKAT = Osteoporosis Knowledge Assessment Tool; OKT = Osteoporosis Knowledge Test; FOOQ = Facts on Osteoporosis Quiz; OPQ = Osteoporosis Questionnaire; OKQ = Osteoporosis Knowledge Questionnaire. 1.D value = subtracting proportion of respondents in lowest quartile answering correctly from those in highest quartile. 2.KR20 (Kuder–Richardson-20) is equivalent to a Cronbach’s alpha (α) for dichotomous variables. 3.CVI = content validity index. 4.Mann-Whitney U-test using contrasted groups. aFOOQ (Ailinger, Harper, & Lasus, 1998) bFOOQ (Ailinger, Lasus, & Braun, 2003)

Criterion Point-biserial

Construct

Test–retest Validity Content3

Instrument Characteristics

Table 4. (continued)

18


Current technology provides opportunities to expand the use of the Revised OKT (2012) through the Internet and social media. For example, one researcher has administered the Revised OKT (2012) electronically (Vu Nguyen, personal communication, May 15, 2012). As more researchers administer the Revised OKT (2012) using new technologies, it will be important to test the reliability and validity of these adaptations. Even though the Revised OKT (2012) has reflected questions from other researchers who have used the original OKT in various cultures, a limitation of this study was using a convenience sample and recruiting from a narrow geographic area. This resulted in a sample that had limited ethnic diversity. Further use of the Revised OKT (2012) in other geographic areas is needed to establish its reliability and validity with diverse populations. Another limitation may have been the cognitive function of some participants. Some subjects scored very low on the scale, with one person getting all the questions incorrect, which would be unlikely in a person with normal cognitive function. Finally, prior to administration, the instrument was not assessed for reading level. Evidence supporting recommendations for calcium and vitamin D intake, as well as types of exercises that are effective for strengthening bones and preventing fractures, is accumulating rapidly. Therefore, it is important to keep abreast of the latest guidelines, research, and research syntheses that consider the body of research and its application to a particular population. It will be necessary to continue to evaluate the currency of the Revised OKT (2012) and update it as appropriate in light of new knowledge. In summary, the purpose of this research was to revise the original OKT to incorporate new knowledge and to determine the reliability and validity of the Revised OKT (2012). Standard test construction methods were used which included an extensive review of literature to generate items, piloting a revision, and then administering the 33-item draft revision of the OKT to a large sample. Item analysis, as well as point-biserial correlations and reliability, was completed resulting in the 32-item Revised OKT (2012). The items have appropriate difficulty, discrimination, and distracters. The Revised OKT (2012) is reliable and valid in adult populations and can be used to assess osteoporosis knowledge and evaluate the effectiveness of educational programs. Acknowledgment We appreciate the contributions of Cassidy Boeskool, MSN, RN; Joanne M. Finazzi, DNP, RN; Amy Axline-Hillard, MSN, RN; Prabha Narayanaswamy, MS; Elizabeth Recker, MSN, RN; and Ellison Klaasen Timmer, MSN, RN, who were graduate students assisting with data collection, analysis, and manuscript preparation.


19

Gendler et al. Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The author(s) received no financial support for the research, authorship, and/or publication of this article.

References Ailinger, R. L., Harper, D. C., & Lasus, H. A. (1998). Bone up on osteoporosis: Development of the facts on osteoporosis quiz. Orthopaedic Nursing, 17(5), 66-73. Ailinger, R. L., Lasus, H., & Braun, M. A. (2003). Revision of the facts on osteoporosis quiz. Nursing Research, 52, 198-201. American College of Sports Medicine. (2010). ACSM’s guidelines for exercise testing and prescription (8th ed.). Baltimore, MD: Lippincott Williams & Wilkins. Becker, M. H., & Janz, N. K. (1985). The Health Belief Model applied to understanding diabetes regimen compliance. Diabetes Educator, 11(1), 41-47. doi:10.1177/014572178501100108 Berarducci, A., Lengacher, C. A., & Keller, R. (2002). The impact of osteoporosis continuing education on nurses’ knowledge and attitudes. The Journal of Continuing Education in Nursing, 33, 210-216; quiz 238-239. Boreham, C. A., & McKay, H. A. (2011). Physical activity in childhood and bone health. British Journal of Sports Medicine, 45, 877-879. doi:10.1136/bjsports-2011-090188 Chang, S. (2008). Knowledge, health beliefs and health-related behaviours of first-degree relatives of women suffering from osteoporosis in Taiwan: A questionnaire survey. Journal of Clinical Nursing, 17, 1280-1286. doi:10.1111/ j.1365-2702.2006.01859.x Chen, Y. P., Liu, X. Q., & Cai, D. (2005). Reliability and validity of the revised Chinese version of the Osteoporosis Knowledge Test. Chinese Journal of Osteoporosis, 11, 339-341. Chevalley, T., Bonjour, J. P., Ferrari, S., Hans, D., & Rizzoli, R. (2005). Skeletal site selectivity in the effects of calcium supplementation on areal bone mineral density gain: A randomized, double-blind, placebo-controlled trial in prepubertal boys. The Journal of Clinical Endocrinology & Metabolism, 90, 33423349. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Doheny, M. O., Sedlak, C. A., Estok, P. J., & Zeller, R. (2007). Osteoporosis knowledge, health beliefs, and DXA T-scores in men and women 50 years of age and older. Orthopaedic Nursing, 26, 243-250. doi:10.1097/01.NOR.0000284654.68215.de


20


Elliott, J. O., Seals, B. F., & Jacobson, M. P. (2008). Osteoprotective knowledge in a multiethnic epilepsy population. Journal of Neuroscience Nursing, 40, 14-24. Holick, M. F. (2004). Sunlight and vitamin D for bone health and prevention of autoimmune diseases, cancers, and cardiovascular disease. American Journal of Clinical Nutrition, 80(Suppl.), 1678S-1688S. Horan, M., Kim, K, Gendler, P., Froman, R., & Patel, M. (1998). Development and evaluation of the Osteoporosis Self-Efficacy Scale. Research in Nursing & Health, 21, 395-403. International Osteoporosis Foundation. (2012). Calcium. Retrieved from http://www. iofbonehealth.org/calcium Jackowski, S. A., Erlandson, M. C., Mirwald, R. L., Faulkner, R. A., Bailey, D. A., Kontulainen, S. A., . . . Baxter-Jones, A. D. G. (2011). Effect of maturational timing on bone mineral content accrual from childhood to adulthood: Evidence from 15 years of longitudinal data. Bone, 48, 1178-1185. doi:10.1016/j. bone.2011.02.010 Kim, K., Horan, M. L., & Gendler, P. (1991). The Osteoporosis Knowledge Test. Allendale, MI: Grand Valley State University. Kim, K. K., Horan, M. L., Gendler, P., & Patel, M. K. (1991). Development and evaluation of osteoporosis health belief scale. Research in Nursing & Health, 14, 155-163. doi:10.1002/nur.4770140210 Looker, A. C., Borrud, L. G., Dawson-Hughes, B., Shepherd, J. A., & Wright, N. C. (2012). Osteoporosis or low bone mass at the femur neck or lumbar spine in older adults: United States, 2005-2008 (NCHS Data Brief, No. 93). Atlanta, GA: National Center for Health Statistics. National Osteoporosis Foundation. (2010). Clinician’s guide to prevention and treatment of osteoporosis. Washington, DC: Author. North American Menopause Society. (2010). Management of osteoporosis in postmenopausal women: 2010 position statement of The North American Menopause Society. Menopause, 17, 25-54. doi:10.1097/gme.0b013e318c617e6 Office for Human Research Protections. (2010). Code of federal regulations. Retrieved from http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html Pande, K. C., de Takats, D., Kanis, J. A., Edwards, V., Slade, P., & McCloskey, E. V. (2000). Development of a questionnaire (OPQ) to assess patient’s knowledge about osteoporosis. Maturitas, 37, 75-81. doi:10.1016/S0378-5122(00)00165-1 Prince, R. L., Devine, A., Dhaliwal, S. S., & Dick, I. M. (2006). Effects of calcium supplementation on clinical fracture and bone structure: Results of a 5-year, double-blind, placebo-controlled trial in elderly women [Abstract]. Archives of Internal Medicine, 166, 869-875. doi:10.1001/arcinte.166.8.869 Qi, B., Resnick, B., Smeltzer, S. C., & Bausell, B. (2011). Self-efficacy program to prevent osteoporosis among Chinese immigrants. Nursing Research, 60, 393404. doi:10.1097/NNR.0b013e3182337dc3 Rizzoli, R., Bianchi, M. L., Garabedian, M., McKay, H. A., & Moreno, L. A. (2010). Maximizing bone mineral mass gain during growth for the prevention


21

Gendler et al.

of fractures in the adolescents and the elderly. Bone, 46, 294-305. doi:10.1016/j. bone.2009.10.005 Rosenstock, I. M., Strecher, V. J., & Becker, M. H. (1988). Social learning theory and the health belief model. Health Education Quarterly, 15, 175-183. doi:10.1177/109019818801500203 Rutecki, G. W. (2008). Osteoporosis- -It’s not just for women anymore: When is screening for osteoporosis warranted in older men? Consultant, 48, 920. Sedlak, C. A., Doheny, M. O., & Jones, S. L. (2000). Osteoporosis education programs: Changing knowledge and behavior. Public Health Nursing, 17, 398-402. Twigg, P. (2009). Developing and using classroom tests. In D. M. Billings & J. A. Halstead (Eds.), Teaching in nursing: A guide for faculty (3rd ed., pp. 429-448). St. Louis, MO: Saunders Elsevier. U.S. Department of Health and Human Services. (2004). Bone health and osteoporosis: A report of the surgeon general. Rockville, MD: Author. Varma, S. (n.d.). Preliminary item statistics using point-biserial correlation and p-values. Morgan Hill, CA: Educational Data Systems. Retrieved from http:// www.eddata.com/resources/publications/EDS_Point_Biserial.pdf von Hurst, P., & Wham, C. (2006). Attitudes and knowledge about osteoporosis risk prevention: A survey of New Zealand women. Public Health Nutrition, 10, 747753. Waltz, C. F., Strickland, O. L., & Lenz, E. R. (Eds.). (2010). Measurement in nursing and health research (4th ed.). New York, NY: Springer. Welch, J. M., & Weaver, C. M. (2005). Calcium and exercise affect the growing skeleton. Nutrition Reviews, 63, 361-373. Winzenberg, T. M., Oldenburg, B., Frendin, S., & Jones, G. (2003). The design of a valid and reliable questionnaire to measure osteoporosis knowledge in women: The osteoporosis knowledge assessment tool (OKAT). BioMed Central Musculoskeletal Disorders, 4(17), 1-7. doi:10.1186/1471-2474-4-17 World Health Organization. (2010). Global recommendations on physical activity for health. Geneva: Author. Retrieved from http://whqlibdoc.who.int/publications/2010/9789241599979_eng.pdf?ua=1


Reliability and Validity of the Inline Skating Skill Test.

Test re-test reliability and construct validity of the star-track test of manual dexterity.

Reliability and validity of nutrition knowledge questionnaire for adults.

Validity and reliability of Initial Letter Word Association Test.

Determining and interpreting test validity, reliability and practicality.

Developing Azeri aphasia screening test and preliminary validity and reliability.

Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score.

Maximal oxygen consumption test during arm exercise--reliability and validity.

Extended version of the "Sniffin' Sticks" identification test: test-retest reliability and validity.

Test-retest reliability and validity of the Sniffin' TOM odor memory test.

Pacific acuity test: testability, validity, and interobserver reliability.

Validity and test-retest reliability of a novel simple back extensor muscle strength test.

Reliability and validity of a revised version of the General Nutrition Knowledge Questionnaire.

Validity, reliability and interpretability of the Thai version of the urticaria control test (UCT).

Validity and test-retest reliability of the Persian version of the Montgomery-Asberg Depression Rating Scale.

The reliability and validity of the persian version of sinonasal outcome test 22 (snot 22) questionnaires.

Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults; statistical issues in validity and reliability analysis.

Shuttle walking test in obese women: test-retest reliability and concurrent validity with peak oxygen uptake.

Stroke specific quality of life questionnaire: Test of reliability and validity of the Persian version.

Reliability, validity and evaluation of the haemophilia coping and perception test.

Reliability and validity test of the Korean version of Noe's evaluation.

Measures of concurrent validity and alternate-form reliability of the Test of Nonverbal Intelligence.

Validity and reliability of the Turkish version of CRAFFT Substance Abuse Screening Test among adolescents.

WONCA: Reliability and validity of the test administered by telephone].