Research Quarterly for Exercise and Sport
ISSN: 0270-1367 (Print) 2168-3824 (Online) Journal homepage: http://www.tandfonline.com/loi/urqe20
Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults Roberta E. Rikli To cite this article: Roberta E. Rikli (2000) Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults, Research Quarterly for Exercise and Sport, 71:sup2, 89-96, DOI: 10.1080/02701367.2000.11082791 To link to this article: http://dx.doi.org/10.1080/02701367.2000.11082791
Published online: 13 Feb 2015.
Submit your article to this journal
Article views: 27
View related articles
Citing articles: 34 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=urqe20 Download by: [University of Nebraska, Lincoln]
Date: 14 September 2015, At: 04:52
Rikli
Research Quarterly for Exercise and Sport ©2000 bythe American Alliance for Health, Physical Education, Recreation and Dance Vol. 71, No.2, pp. 89-96
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults Roberta E. Rikli
Keywords: reliability, validity, physical activity, older adult
T
h is paper is written in response to the previous article in this journal issue on "Assessment of Physical Activity in Older Adults" (Washburn, 2000). Reactions to the Washburn paper will be organized into the following categories: 1) background comments, 2) methodology, 3) summary statements, 4) additional reliability and validity issues, and 5) conclusions.
Background Comments As pointed out in the Washburn (2000) article, there is increasing evidence that physical activity participation by older adults is associated with improved quality and quantity oflife. Specifically, the recent Surgeon General's Report on PhysicalActivity and Health describes the following benefits of physical activity for older adults (U.S. Dept. of Health and Human Services, 1996): • Reduces risk of dying from coronary heart disease and of developing high blood pressure, colon cancer, and diabetes. • Helps people with chronic disabling conditions improve their stamina and strength. • Reduces anxiety and depression; improves mood and feelings of well-being. • Helps maintain healthy bones, muscles, and joints. • Helps control joint swelling and pain associated with arthritis. • Helps maintain ability to live independently; reduces risk of falling and fracturing bones Evidence also suggests that as average life expectancy
Roberta E. Rikliis with the Division of Kinesiology and Health Promotion at California State University, Fullerton.
ROES: June 2000
increases, so does the importance of physical activity in later years. By the time people reach their late seventies or early eighties, age-related declines in physical strength and endurance associated with inactive or low active lifestyles often progress below that needed for performing normal everyday activities (Evans & Rosenberg, 1991; Shephard, 1997). Activities such as climbing stairs or getting out of a chair or bathtub, for example, require that adequate levels of strength and stamina be maintained. The role of physical activity during aging is especially critical considering that the 85+ population is the fastest growing of all age groups, and in fact, is expected to increase by 500% over the next 40-50 years (U.S. Bureau of the Census, 1996). Although there is considerable evidence supporting the value of physical activity in later years, few studies have been completed concerning the optimum exercise prescriptions for different age cohorts or for people with various health or mobility conditions. Also, little is known concerning the demographic correlates of physical activity in later years or concerning effective strategies for increasing exercise in older adults. As Washburn (2000) clearly points out, studies addressing these and other physical activity concerns will require reliable and valid means of assessing physical activity level. Assessments are needed which can be used effectively in large epidemiologic studies, as well as in smaller prospective studies such as those designed to test specific intervention effects. Although reducing disability and increasing the number of active years during the life span has been a major objective of the Healthy People 2000 project (Dept. of Health and Human Services, 1990), recent reports indicate that the nation may be losing ground on this important goal (Dept. of Health and Human Services, 1996). As life expectancy continues to increase, so too, apparently, does the possibility of living more years with major physical limitations. Planning effective strategies to reduce physical de-
89
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
cline in later years requires an understanding the various stages of cha nge leadi ng to frailty. Traditionally, it has been suggested (Nagi. 1965; Nagi, 1991) that the progression to di sability includes four stages: 1) dise ase/pathology, 2) ph ysical impairm ent. 3) functional limitation. and 4) disability. Specifically, the Nagi model (Figure l a) suggests that disease or pathology leads to impairment (decline in body systems-e.g., muscular, card iovascular) . whic h in tum leads to fun ctional limitation (restrictions in ph ysical behaviors such aswalking and stair climbing) . and even tually to disability (loss of abili ty to carry o ut normal activities such as bathing oneself, housework, or shopping). Recent evidenc e, however, indicates that as people are living longer, lifestyle factors such as physical inactivity o r disuse can also lead to frailty in later years and. in fact, it sho uld be co nsidered equally impo rtant to d isease / pa thol ogy in the model (Buchner & Wagner. 1992; Cha ndler & Hadl ey. 1996; Dll'ietro, 1996; Morey. Pieper, & Comoni-Huntley. 1998; Rikli &jon es. 1997). Such a mo di ficatio n. as indicated in Figure lb. not only has important implicati ons for planning interventions to prevent or delay the onset o f frailty. but also has implications for th e types ofassessment tools needed for research in this area. To understand th e changes th at occu r during the disability process, tool s are needed that can assess behavior at eac h stage in the model. in clu d ing pe opl e's ph ysical activity be havior at stage one. Although assessment tools are needed for each stage of the model. the development of measu re s to assess initial changes has lagged be hind that of measures to assess pe rformance in the latter stages. Various reviews, for example. ha ve identified mo re that 50 published mea-
sures that co uld be used to assess behavior in the lat er two stages. stages that involve functional limitation and loss of ability (Fein stein.josephy, & Wells. 1986; Spirduso, 1995). Most of these measures were developed by rehabilitation specialists to assess the effects o f treat ment or to d etermine the amount of assistan ce needed by individuals after they have expe rienced serio us losses in fun ctional ability. However. far fewer measures have been developed to assess behavior at th e first two stages of th e d isabili ty model. that is, measures that ca n assess level of ph ysical activityparticipa tion or level of ph ysiological impairment (declines in str ength . e nd u rance. e tc.}. The abili ty to assess performance at these initial stages is espe cially imponant since deficiencies here are ofte n the forerunners of more serious losses in fun ctional ability. Certainly, studies design ed to test strategi es for preventing o r d elaying loss of function will req uire tools that can assess performance early on in th e abili ty/disability model. The Fullerton Fun ctional Fitn ess Test for Seniors (Rikli &j ones. 1999a ; Rikli &jones. 1m b) is an example of a test that was recen tly d eveloped to provide a me ans of assessing physical im pa irmen t at stage two of th e model. The measures in the Fullerton test can detect ab normal declines in performance so that they possibly can be treated before leading to more serious fun ctional limitation. Of potentiallyeven greater importance in determining ways to p revent physical decline is the availabili ty of measu res to assess physical act ivity be havior at stage o ne of th e model. With statistics indi cating that less than 15% of o lder adults engage in the recommended am ount of physical exercise (U.S. Dept. o f Heahh and Human Services. 1996) , research is needed on ways to increase ph~i-
1.1
iC::"~ "1 ~J'."' 1
Disability Model (Nagi, I9911
.", ;.t\
1'1
Disability Modal lamendad)
-_.t;:-~ >t: , " -' - ,
Figure 1.la) Nagi's (19911 model describingthe progressionI"ding to disability, and (blan amended version 01 the model suggesting that aninactive lifestyle, indapendent of thedis"sa process, canhalla comparabla effects on thedisabling process. From -Assessing Physical Parformance in Indepandant OIdar Adults: Issues and Guidelines ~ by A. Aikli & C. J. Jones, 1991, Journal ofAgingandPhysical Activiry. 5. p. 247. Copyright 1997 by Human Kinatics. Adapted with parmission.
e.
911
RQES: Jun. 2I1l1O
Rikli
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
cal activity participation within this age group. Naturally, such studies will require reliable and valid tools for assessing physical activity level. The purpose of the Washburn (2000) article and of this response paper is to review the status of measurement tools currently available to assess physical activity level in older adults, a variable which for many may be the lead factor in either contributing to or preventing physical decline in later years.
Methodology In his introduction, Washburn (2000) does a nicejob of reviewing the importance of physical activity during aging, citing evidence supporting the effect of physical activity on decreased risk of mortality, improved health, reduced risk of falling, decreased loss of bone, and decreased disability in later years. Following his discussion of the value of physical activity in later years, he offers compelling reasons why it is important to be able to measure physical activity in older people, reasons which include learning more about the correlates, causes, and consequences of physical inactivity and learning more about the optimum amounts and types of physical activity at different ages and for people with various health and mobility conditions. In describing the status of the assessment techniques available for measuring physical activity level in older adults, Washburn provides a brief overview of the various physical activity assessment options including motion sensors (accelerometers), heart rate monitoring, activity diaries, doubly labeled water, and physical activity (PA) questionnaires. He further explains that because of the cost and various logistical considerations associated with most techniques, the only method feasible for use in large epidemiological studies is the questionnaire. Consequently, he has limited his review of physical activity assessment techniques to questionnaires only. Prior to reviewing the PA questionnaires developed specifically for older adults, Washburn addressed the inappropriateness of using age-neutral questionnaires with older people. Age-neutral questionnaires, he explains, focus primarily on youth-oriented types of activities (e.g., sports and recreational activities) and, therefore, tend to result in an underestimate of total activity level and/or energy expenditure in older people whose more common forms of exercise include walking, gardening, and household activities. Washburn also describes the role of both direct and indirect techniques in validating PA questionnaires. Comparing questionnaire results with activity diaries, motion sensors, heart rate monitoring, and doubly labeled water are examples of direct validation evidence. Indirect validation evidence, on the other hand, is provided through the association ofPA question-
ROES: June 2000
naire results with variables that are influenced by physical activity, such as maximal oxygen uptake, resting heart rate, muscular strength, and body composition. In the main body of his paper, Washburn describes the four questionnaires that have been specifically developed for older adults-the Modified Baecke Questionnaire for Older Adults (Voorrips, Ravelli, Dongelmans, Deurenberg, & van staveren, 1993), the Zutphen Physical Activity Questionnaire (Caspersen, Bloemberg, Saris, Merritt, & Kromhout, 1991), the Yale Physical Activity Survey (DiPietro, Caspersen, Ostfeld, & Nadel, 1993), and the Physical Activity Scale for the Elderly (Washburn, Smith,Jette, &Janney, 1993). For each questionnaire he describes its general characteristics and comments on the evidence, or lack of evidence, supporting its reliability and validity. Following his review of the specific questionnaires, Washburn then makes several general statements summarizing his findings with respect to the documented quality of PA questionnaires.
Summary Statements Presented below are five summary statements made by Washburn based on his review of the four PA questionnaires developed specifically for older adults. Following each statement are my reactions and comments. The statements are not necessarily in the order discussed in the original paper.
Summary Statement #1: There is limited information supporting the reliability and validity of the questionnaires developed for older adults. The data that are available indicate little improvement over the age-neutral questionnaires as a means of assessing PA in older adults.
Response: It is not surprising to find limited or inconsistent information with respect to the reliability and validity of these instruments. The changing characteristics and increased variability in the health status and physical and mental ability levels of older adults create special challenges in developing reliable and valid measurement tools for this population. Examples of conditions commonly observed in older adults that might affect performance on PA questionnaire items, as well as performance on the criterion measures that are used to validate the questionnaires include: fluctuations in fatigue or pain level, temporary distractions/lack of concentration, impaired cognitive ability, reduced information processing 91
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
speed, changing medical conditions or medications, and fluctuations in mood, anxiety, and level depression. Increased individual inconsistency or intraindividual variability during aging has been well-documented on a number of characteristics and behaviors (Rikli &Jones, 1997; Spirduso, 1995; Spirduso, Francis, Light, & Reilly, 199Y) Although test unreliability can be due to either instrumentation inconsistency or to individual inconsistency, it is the latter which typically presents the greatest challenge in assessing older adults. To increase scoring precision and accuracy in older people, special adjustments may be needed-adjustments which might involve providing clearer instructions, repeated cueing, enhanced motivation, additional practice, and/or an increased number of trials. Although little data are available demonstrating the effects of increased intraindividual variability on questionnaire results, data on other kinds of measures clearly show the potential consequences of increased subject variability on test reliability. For example, in most of our research at Fullerton involving older adult assessments we have found that to achieve reliable measures we had to take extra care in clarifying test instructions and, on performance tests, had to increase both the amount of pretest practice time and the number of test trials compared to what typically is required for younger people. In a study involving simple reaction time, for example, nearly four times as many trials were required to obtain reliable measures for older people than generally are needed for younger people. Whereas only eight trials were necessary to achieve reliable test-retest scores for those ages 18 to 40 (R = .87) (Haywood & Teeple, 1976), more than 30 trials were required to achieve similar consistency in older adults (Lupinacci, Rikli,Jones, & Ross, 1993). Also, in developing a modified step test to assess aerobic endurance in older adults, we found test-retest reliabilities of only .42 to .58, using the same single-trial protocol that had been developed for younger subjects with reliability values over .90. However, by requiring a pretest practice trial on a prior day, step test reliability values for older adults increased to .90 and above (Dugas, 1996). Similarly, we have found that achieving stable 1 RM (repetition maximum) strength measures for older adults requires a three-day protocol (one day of practice and two test days) instead of the typical single day protocol commonly used with younger people (Rikli, Jones, Beam, Duncan, & Lamar, 1996). Although few studies have been completed on protocols for assessing physical activity level in older people, similar findings might be expected with respect to the need for an increased number of trials. Considering that a 7-day average of motion sensor readings was required to achieve a test-retest reliability of .80 or better in young people (Trost, Pate, Freedson, Sallis, & Taylor, 1999), it is presumed that considerably more than seven days might be required to achieve stable scores with older,
92
typically more variable individuals. With respect to PA questionnaire assessment specifically, it would be logical to assume that measures may also need to be averaged over more than one time period in order to obtain stable (reliable) indicators of an older person's typical activity patterns. Even with younger people, the reliability of questionnaire data was increased when participants were asked to report their activities over two different time periods (over the past 7 days and over a 'typical' 7-day period), with scores then averaged over the two periods (Prochaska, Sallis, & Long, 1999).
Summary Statement #2: When evaluating reliability, it is difficult to determine if poor agreement between repeated test administrations is due to poor recall or to actual changes in physical activity behavior.
Response: Although the question of error in recall versus actual changes in behavior is an important concern in evaluating test-retest reliability, the issue is much more complex than this. For example, in most questionnaire development to date, little attention has been given to the cognitive strategies that might best enhance memory in older adults and would solicit the most complete, accurate, and reliable types of information. In a recent review of the cognitive psychology literature (Durante & Ainsworth, 1996), it is suggested that one's recall ability can be significantly influenced by the types of probing questions or retrieval cues that are given. A general question asking about a person's engagement in moderate level activity is surely not as likely to provide the same degree of recall as would more probing questions asking about specific activity behaviors. For consistency of measurement, it also is important that all terms used in the questionnaire are clearly defined. Determining what constitutes "moderate" or "vigorous" types of activity, for example, is difficult enough for professionals in the field. Certainly, people unfamiliar with these terms will need clear explanations and examples of what they mean. Similarly, test administrators who are interpreting participant responses need clear guidelines as to what constitutes various levels of activity. Although most questionnaires involve translating activities into some type of "energy" or "MET" equivalents, there currently are no guidelines for making such interpretations for older adults. Energy cost classification systems, such as the Compendium of Physical Activities (Ainsworth et aI., 1993), have been developed based on normal activity patterns of young people, but may be far from accurate for use with older adults. The energy costs
ROES: June 2000
Rikli
of walking one mile for an older adult with a knee replacement or with a restricted gait pattern surely will be different than that for young people with normal walking patterns.
Summary Statement #3:
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
The evaluation of questionnaire validity is problematic due to the lack of an acceptable criterion measure of physical activity for older adults.
Response: The lack of suitable criterion measures for validating activity questionnaires for older adults is indeed a serious problem. Difficulties are encountered in virtually all of the common methods of assessment when they are used to measure physical activity level in older adultsactivity diaries, motion sensors, heart rate monitoring, or doubly labeled water. The use of activity diaries can be problematic with some older adults because of their inability to understand the directions and provide accurate records over extended periods of time or because of their lack of motivation in carrying out this rather tedious procedure which commonly involves recording one's type ofphysical activity every 15 minutes over several 24-hour periods. Of even greater concern, as discussed earlier, is the lack of a MET conversion system that can accurately estimate the energy expenditure associated with older adult activity patterns. Because the information on activity diaries typically is translated into energy expenditure or intensity MET values, the lack of such guidelines seriously limits the accuracy of this technique. The use of motion sensors to analyze activity level are also problematic with older people, with a major concern being that of their appropriate anatomical placement, a concern that is especially relevant in older adults who have some type of gait abnormality. For a person with arthritis in one hip, for example, very different readings would be obtained depending on whether the sensor was placed on the affected hip or the non-affected hip. Even in younger people with normal movement patterns, significantly different readings have been found for a motion sensor on one hip compared to one on the other hip (Nichols, Morgan, Sarkin, Sallis, & Calfas, 1999). Obviously, such discrepancies would be much greater when there are abnormal gait patterns of any kind. In fact, walking efficiency as a percent of normal has been estimated to be approximately 70% for people with arthritic knees or hips and only about 40% for people using a cane (Mulcare, Ayres, & Allen, 1999). Similarly, as would be expected, significantly different accelerometer (motion sensor) readings have been found for people walking
ROES: June 2000
under normal conditions compared to walking under simulated orthopedic conditions such as while wearing a knee brace or an ankle or foot orthodic (Mulcare et aI., 1999). Although the doubly labeled water method of assessing activity level is considered the 'gold standard' with respect to assessing total energy expenditure in young people, the appropriateness of this technique with older people is also questionable, particularly for people taking any kind of medication that could affect water retention. Additionally, the accuracy of the doubly labeled water technique might be questionable in people with any medical condition that affects the circulatory system-conditions involving peripheral vascular or cardiovascular circulation, renal function, or respiration, as examples. Heart rate monitoring is also less effective as a measure of physical activity level in older adults than it is in younger people. A number of common medications, particularly those affecting cardiovascular function, can influence both normal and exercise heart rates. Also, in some individuals, certain anatomical characteristics (e.g., excessive adipose tissue or redistributed body fat) can make it difficult to correctly position the heart rate monitor and obtain accurate readings from the monitors.
Summary Statement #4: It is suggested that using indirect methods in the validation of activity questionnaires may be inappropriate for older adults given that they typically do not participate in the kinds of high intensity conditioning that affects indirect measures, measures such as maximum aerobic capacity, body composition, muscular strength, and blood pressure and blood lipids, as examples. Response: Rather than eliminate the use of indirect methods in questionnaire validation, it might be more appropriate to reconsider the types of indirect measures that are most relevant for assessing the effects of physical activity for older adults. In older people, where maintaining functional ability is of increasing concern and, in fact, is a major reason why physical activity participation is important in later years, it would seem to make sense to use "functional" measures as indirect indicators of PA questionnaire validity instead of, or in addition to, some of the more traditional measures. In later years, for example, a person's ability to climb stairs, walk moderate distances, and do one's own shopping and household activities may be more important consequences (indicators) of regular physical activity than is maximal aerobic capacity or blood cholesterol level.
93
Rikli
Summary Statement #5: To improve response accuracy, it is suggested that PA questionnaires for older adults be interviewer-administered either in person or over the phone.
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Response: Although interviewer-administered questionnaires probably generate more accurate and reliable information than self-administered ones, the associated time demands may prohibit their use in many situations. It is extremely difficult, for example, to limit the conversation and 'story-telling' of some older people when asking them to describe their past activity levels. Such questions naturally remind them of past experiences, many of which they enjoy discussing in some detail. In using PA questionnaires in our studies at Fullerton, we have found that a combination of self-administered and intervieweradministered procedures works best. We typically mail the questionnaires out to people so that they can be completed in advance and then brought in to be reviewed by an interviewer. Completing a questionnaire at home gives people more time to respond to questions regarding their medical histories and types of medications they are taking, and gives them time to think through their past activities. The interviewer then meets with each person to go over the questionnaire to check for completeness, to clarify information, and to check for any potential errors due to misunderstood terms which often can happen, we have learned. In one of our studies, for example, an 80-year old woman had evaluated her own activity level as "vigorous," based on the fact that she swam nearly every day for an hour. However, through followup probing concerning the details of her swimming activity-like how many laps she averaged per day and the types of strokes she generally used, it soon was determined that she really did not 'swim' at all, but just went to the pool for an hour every day. Although the accuracy of self-administered questionnaires may be acceptable in large-scale epidemiologic studies, increased precision generally is needed in order to detect expected differences or relationships in smaller-scale studies.
Additional Reliability and Validity Issues For any test to be of value, it must have acceptable reliability and validity for the population of interest (American Psychological Association, 1985). A reliable test is one that produces consistent and dependable scores that are relatively free of measurement error. A valid test is one that accurately measures what it is intended to measure. To increase the reliability and validity of physical activity measures (or of any other mea-
94
sure), it is important to identify as many sources of error as possible, then plan strategies for reducing the error. It also is important to determine whether the measurement error is "random" or "systematic." Random or chance error generally affects a test's reliability and is treated in a different way than is systematic error which generally affects test validity. For example, if one were to measure another person's height using only a 12-inch ruler, there certainly would be more error involved (and lower test-retest reliability) than if a commercial height meter were used. However, the type of error involved in using a 12-inch ruler to measure height would be random error and could be greatly reduced by taking a large number of measures and averaging the results to obtain one overall measure. An example of systematic error, on the other hand, would be that which typically occurs when height is measured through self-report. Most people, it has been observed, tend to overestimate their height and underestimate their weight (Palta, Prineas, Berman, & Hannan, 1982), thus, resulting in biased and invalid measures of height and weight. Reducing this type of systematic error would require a different procedure than that involved in reducing random error. Simply repeating the measures (or questions, in this case) and averaging the responses would not improve the accuracy of the height measures at all. However, if it were possible to determine people's typical amounts of height overestimations at different ages, a conversion formula could be developed to correct for the error and thus improve the accuracy (validity) of self-reported height measures. Also, to improve reliability and validity it is important to consider not only the type of error (random vs systematic), but also the source of error-that is, whether the measurement error is associated with test instrumentation/administration procedures or if it is due to inconsistencies within the individual. For example, if the testretest reliability of a certain 'motion sensor' method of assessing physical activity data is low, the sources of error could be numerous and would need to be identified before strategies could be planned for reducing error. Possible sources of error in motion sensor measurement include improper calibration of instruments, inconsistent placement of instruments on the body, improper placement of the motion sensors, incorrect translations or coding of movement data, and/ or inconsistent behavior on the part of the individual, a condition which is especially characteristic of older people due to their fluctuations in pain level, energy level, or in their medical conditions, for example. Again, it is important to identify both the sources of error (instrumentation vs. individual) and the types of error (random vs. systematic) in order to plan effective ways of reducing error. Depending on the source and type of error, the strategies for reducing it might require
ROES: June 2000
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
more frequent calibration of the instruments, better training of testing technicians, increased knowledge with respect to the optimum placement of motion sensors, and/or an increased number of days/trials over which measures are taken and averaged, thus, increasing the stability of measurements. In one recent study, for example, the reliability of the data from a 7-day activity record was increased by having participants record their activities over two 7-day periods and taking the average of the two (Baranowski et a!., 1999). Also, as mentioned earlier, Prochaska et a!. (1999) found that the reliability of questionnaire data could be increased by asking people to report their activities in two different waysfor a typical week and for the past seven days, then using an average of the two reports. Although it is an over simplification of the many kinds of potential measurement error, a grid similar to that presented in Figure 2 can be helpful in sorting out the basic types and sources of error involved in assessing physical activity level in older adults. After the possible sources of error have been identified, then studies can be designed to confirm the errors and/or to quantify the amount of error so that appropriate corrections or adaptations can be made. The work of Nichols and colleagues at San Diego State University (Nichols et a!., 1999) and the work cited earlier by Mulcare provide good examples of ways to begin sorting out the sources and amounts of error involved in measuring physical activity level. For example, by attaching different accelerometers to a mechanical shake table with a prescribed amount of motion, Nichols et a!. (1999) were able to determine the amount ofvariation in accelerometer data that might be attributable to instrument variability versus individual variability. Mulcare et a!. (1999), through simulating various orthopedic conditions in their laboratory, were able to estimate the amount of extra energy requirements associated with walking under specific abnormal conditions, such as with a knee brace or with an ankle/foot orthodic. Certainly,
Assessment Method
findings from studies such as these should improve the accuracy and interpretation of the data obtained in assessing physical activity level in older adults.
Conclusion In conclusion, Dr. Washburn is commended for providing a good overview of the availability and quality of PA questionnaires for older adults. In addition, it is clear that the issues related to improving the reliability and validity of physical activity assessment in older adults, including questionnaire assessment, are quite complex. Studies so far have barely scratched the surface with respect to identifying and responding to the potential kinds of error involved in assessing physical activity in older adults. .
References Ainsworth, B. E., Haskell, W. L., Leon, A. S., Jacobs, D. R., Montoye, H.]., Sallis,]. F., & Paffenbarger, R. S. (1993). Compendium of physical activities: Classification of energy costs of human physical activities. Medicine and Science in Sports and Exercise, 25,71-80. American Psychological Association. (1985). Joint standards for education and psychological tests. Washington, DC: American Psychological Association. Baranowski, T., Smith, M., Thompson, W.O., Baranowski,]., Hebert, D., & de Moor, C. (1999). Intraindividual variability and reliability in a 7-day exercise record. Medicine and Science in Sports and Exercise, 31,1619-1622. Buchner, D. M., & Wagner, E. H. (1992). Preventing frail health. Health Promotion and Disease Prevention, 8, 1-17. Caspersen, G.]., Bloemberg, B. P., Saris, W. H. M., Merritt, R. K., & Kromhout, D. (1991). The prevalence of selected
Random Error Instrument
Individual
Systematic Error Instrument
Individual
Figure 2. A grid for identifying potentialmeasurement errors accordingto types and sources.
RaES: June 2000
95
Rikli
physical activities and their relation with coronary heart disease risk factors in elderly men: The Zutphen study, 1985. American journal ofEpidemiology, 133, 1078-1092. Chandler,]. M., & Hadley, E. e. (1996). Exercise to improve physiologic and functional performance in old age. Clinics in Geriatric Medicine, 12, 761-784. Dept. of Health and Human Services. (1990). Healthy People
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
2000: National health promotion and disease prevention objectives (PHS 91-50213). Washington, DC: U.S. Government
Printing Office. Dept. of Health and Human Services. (1996). Healthy People 2000: Midcourse review and 1995 revisions. Washington, DC: U.S. Government Printing Office. DiPietro, L. (1996). The epidemiology of physical activity and physical function in older people. Medicine and Science in Sports and Exercise, 28,596-600. DiPietro, L., Caspersen, C.]., Ostfeld, A. M., & Nadel, E. R. (1993). A survey for assessing physical activity among older adults. Medicine and Sciencein Sports and Exercise, 25,628-642. Dugas, E. W. (1996). The development and validation ofa 2-minute step test to estimate aerobic endurance in older adults. Unpublished Master's Thesis, California State University, Fullerton, Fullerton, CA. Durante, R., & Ainsworth, B. E. (1996). The recall of physical activity: Using a cognitive model of the question-answering process. Medicine and Science in Sports and Exercise, 28, 1282-1291. Evans, w., & Rosenberg, I. H. (1991). Biomarkers: The lOdeterminants ofaging you can control. New York: Simon & Schuster. Feinstein, A. R.,Josephy, B. R., & Wells, c. K. (1986). Scientific and clinical problems in indexes of functional disabiliity. Annals oflntemalMedicine, 105,413-420.
Haywood, K. M., & Teeple,]. B. (1976). Representative simple reaction and movement time scores. Research Qy.arterly, 47,855-856. Lupinacci, N. S., Rikli, R. E., Jones, C.]., & Ross, D. (1993). Age and physical activity effects on reaction time and digit symbol substitution performance in cognitively active adults. Research Quarterly for Exercise and Sport, 64, 144-150. Morey, M. e., Pieper, e. E, & Comoni-Huntley,]. (1998). Physical fitness and functional limitations in community-dwelling older adults. Medicineand Science in Sports and Exercise, 30, 715-723. Mulcare.j. A., Ayres, S., & Allen,]. (1999). Measuring Physical Activity in Persons with Chronic Physical, Psychological, or Cognitive Disabilities. Paper presented at the American College of Sports Medicine Annual Meeting, Seattle, WA. Nagi, S. Z. (1965). Some conceptual issues in disability and rehabilitation. In M. B. Sussman (Ed.), Sociologyand Rehabilitation (pp. 100-113). Washington, D.C.: American Sociological Association. Nagi, S. Z. (1991). Disability concepts revisited: Implication for prevention. In A. M. Pope & A. R. Tarlov (Eds.), Disability in America: Toward a national agenda for prevention (pp. 309327). Washington, DC: National Academy Press. Nichols,]. E, Morgan, C. G., Sarkin,]. A., Sallis,]. E, & Calfas, K.]. (1999). Validity, reliability, and calibration of the Tritrac accererometer as a measure of physical activity. Medicine and Science in Sports and Exercise, 31, 908-912.
96
Palta, M., Prineas, R., Berman, R., & Hannan, P. (1982). Comparison of self-reported and measured height and weight. AmericanJoumalofEpidemiology, 115,223-230. Prochaska.j.j., Sallis,]. E, & Long, B. (1999, October). Strengths and limitations of youth self-reported physical activity. Paper presented at the Measurement of Physical Activity Symposium, Dallas. Rikli, R. E., & Jones, C.]. (1997). Assessing physical performance in independent older adults: Issues and guidelines. [ournal ofAging and Physical Activity, 5, 244-261. Rikli, R. E., &Jones, C.]. (1999a). Development and validation of a functional fitness test for community-residing older adults. journal of Aging and Physical Activity, 6, 127-159. Rikli, R. E., & Jones, C.]. (1999b). Functional fitness normative scores for community-residing adults, ages 60-94. Journal ofAging and Physical Activity, 6, 160-179. Rikli, R. E.,Jones, C.]., Beam, W. B., Duncan, S.]., & Lamar, B. (1996). Testing versus training effects on lRM strength measures in older adults. Medicine and Science in Sports and Exercise, 28(Suppl.), S153 (Abstract). Shephard, R.]. (1997). Aging, physical activity, and health. Champaign, IL: Human Kinetics. Spirduso, W.W. (1995). Physical dimensions ofaging. Champaign, IL: Human Kinetics. Spirduso, W. w., Francis, K. L., Light, K. E., & Reilly, M. A. (1999). Age differences in life-span reaction time: Withinand between-subject variability. [ournal ofAging and Physical Activity, 7, 296 (abstract). Trost, S. G., Pate, R. R., Freedson, P. S., Sallis,]. F., & Taylor, W. e. (1999, October). Using objectivephysical activity measures with youth: How many days of monitoring are needed? Paper presented at the Measurement of Physical Activity Symposium, Dallas. U.S. Bureau of the Census. (1996). Sixtyjiveplus in the United States: Current population reports (P23-190). Washington, DC: U. S. Department of Commerce. U.S. Dept. of Health and Human Services. (1996). Physical activity and health: A report of the surgeon general . Atlan ta: U. S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion. Voorrips, L. E., Ravelli, A. C.]., Dongelmans, P. C. A., Deurenberg, P., & van staveren, W. A. (1993). A physical activity questionnaire for the elderly. Medicine and Science in Sports and Exercise, 23,974-979. Washburn, R. A. (2000). Assessment of physical activity in older adults. Research Quarterly for Exercise and Sport, 71 (Supp!.) 79-89. Washburn, R. A., Smith, K. v«, Jette, A. M., & Janney, C. A. (1993). The physical activity scale for the elderly (PASE): Development and validation. journal of Clinical Epidemiology, 46, 153-162.
ROES: June 2000
ISSN: 0270-1367 (Print) 2168-3824 (Online) Journal homepage: http://www.tandfonline.com/loi/urqe20
Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults Roberta E. Rikli To cite this article: Roberta E. Rikli (2000) Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults, Research Quarterly for Exercise and Sport, 71:sup2, 89-96, DOI: 10.1080/02701367.2000.11082791 To link to this article: http://dx.doi.org/10.1080/02701367.2000.11082791
Published online: 13 Feb 2015.
Submit your article to this journal
Article views: 27
View related articles
Citing articles: 34 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=urqe20 Download by: [University of Nebraska, Lincoln]
Date: 14 September 2015, At: 04:52
Rikli
Research Quarterly for Exercise and Sport ©2000 bythe American Alliance for Health, Physical Education, Recreation and Dance Vol. 71, No.2, pp. 89-96
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Reliability, Validity, and Methodological Issues in Assessing Physical Activity in Older Adults Roberta E. Rikli
Keywords: reliability, validity, physical activity, older adult
T
h is paper is written in response to the previous article in this journal issue on "Assessment of Physical Activity in Older Adults" (Washburn, 2000). Reactions to the Washburn paper will be organized into the following categories: 1) background comments, 2) methodology, 3) summary statements, 4) additional reliability and validity issues, and 5) conclusions.
Background Comments As pointed out in the Washburn (2000) article, there is increasing evidence that physical activity participation by older adults is associated with improved quality and quantity oflife. Specifically, the recent Surgeon General's Report on PhysicalActivity and Health describes the following benefits of physical activity for older adults (U.S. Dept. of Health and Human Services, 1996): • Reduces risk of dying from coronary heart disease and of developing high blood pressure, colon cancer, and diabetes. • Helps people with chronic disabling conditions improve their stamina and strength. • Reduces anxiety and depression; improves mood and feelings of well-being. • Helps maintain healthy bones, muscles, and joints. • Helps control joint swelling and pain associated with arthritis. • Helps maintain ability to live independently; reduces risk of falling and fracturing bones Evidence also suggests that as average life expectancy
Roberta E. Rikliis with the Division of Kinesiology and Health Promotion at California State University, Fullerton.
ROES: June 2000
increases, so does the importance of physical activity in later years. By the time people reach their late seventies or early eighties, age-related declines in physical strength and endurance associated with inactive or low active lifestyles often progress below that needed for performing normal everyday activities (Evans & Rosenberg, 1991; Shephard, 1997). Activities such as climbing stairs or getting out of a chair or bathtub, for example, require that adequate levels of strength and stamina be maintained. The role of physical activity during aging is especially critical considering that the 85+ population is the fastest growing of all age groups, and in fact, is expected to increase by 500% over the next 40-50 years (U.S. Bureau of the Census, 1996). Although there is considerable evidence supporting the value of physical activity in later years, few studies have been completed concerning the optimum exercise prescriptions for different age cohorts or for people with various health or mobility conditions. Also, little is known concerning the demographic correlates of physical activity in later years or concerning effective strategies for increasing exercise in older adults. As Washburn (2000) clearly points out, studies addressing these and other physical activity concerns will require reliable and valid means of assessing physical activity level. Assessments are needed which can be used effectively in large epidemiologic studies, as well as in smaller prospective studies such as those designed to test specific intervention effects. Although reducing disability and increasing the number of active years during the life span has been a major objective of the Healthy People 2000 project (Dept. of Health and Human Services, 1990), recent reports indicate that the nation may be losing ground on this important goal (Dept. of Health and Human Services, 1996). As life expectancy continues to increase, so too, apparently, does the possibility of living more years with major physical limitations. Planning effective strategies to reduce physical de-
89
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
cline in later years requires an understanding the various stages of cha nge leadi ng to frailty. Traditionally, it has been suggested (Nagi. 1965; Nagi, 1991) that the progression to di sability includes four stages: 1) dise ase/pathology, 2) ph ysical impairm ent. 3) functional limitation. and 4) disability. Specifically, the Nagi model (Figure l a) suggests that disease or pathology leads to impairment (decline in body systems-e.g., muscular, card iovascular) . whic h in tum leads to fun ctional limitation (restrictions in ph ysical behaviors such aswalking and stair climbing) . and even tually to disability (loss of abili ty to carry o ut normal activities such as bathing oneself, housework, or shopping). Recent evidenc e, however, indicates that as people are living longer, lifestyle factors such as physical inactivity o r disuse can also lead to frailty in later years and. in fact, it sho uld be co nsidered equally impo rtant to d isease / pa thol ogy in the model (Buchner & Wagner. 1992; Cha ndler & Hadl ey. 1996; Dll'ietro, 1996; Morey. Pieper, & Comoni-Huntley. 1998; Rikli &jon es. 1997). Such a mo di ficatio n. as indicated in Figure lb. not only has important implicati ons for planning interventions to prevent or delay the onset o f frailty. but also has implications for th e types ofassessment tools needed for research in this area. To understand th e changes th at occu r during the disability process, tool s are needed that can assess behavior at eac h stage in the model. in clu d ing pe opl e's ph ysical activity be havior at stage one. Although assessment tools are needed for each stage of the model. the development of measu re s to assess initial changes has lagged be hind that of measures to assess pe rformance in the latter stages. Various reviews, for example. ha ve identified mo re that 50 published mea-
sures that co uld be used to assess behavior in the lat er two stages. stages that involve functional limitation and loss of ability (Fein stein.josephy, & Wells. 1986; Spirduso, 1995). Most of these measures were developed by rehabilitation specialists to assess the effects o f treat ment or to d etermine the amount of assistan ce needed by individuals after they have expe rienced serio us losses in fun ctional ability. However. far fewer measures have been developed to assess behavior at th e first two stages of th e d isabili ty model. that is, measures that ca n assess level of ph ysical activityparticipa tion or level of ph ysiological impairment (declines in str ength . e nd u rance. e tc.}. The abili ty to assess performance at these initial stages is espe cially imponant since deficiencies here are ofte n the forerunners of more serious losses in fun ctional ability. Certainly, studies design ed to test strategi es for preventing o r d elaying loss of function will req uire tools that can assess performance early on in th e abili ty/disability model. The Fullerton Fun ctional Fitn ess Test for Seniors (Rikli &j ones. 1999a ; Rikli &jones. 1m b) is an example of a test that was recen tly d eveloped to provide a me ans of assessing physical im pa irmen t at stage two of th e model. The measures in the Fullerton test can detect ab normal declines in performance so that they possibly can be treated before leading to more serious fun ctional limitation. Of potentiallyeven greater importance in determining ways to p revent physical decline is the availabili ty of measu res to assess physical act ivity be havior at stage o ne of th e model. With statistics indi cating that less than 15% of o lder adults engage in the recommended am ount of physical exercise (U.S. Dept. o f Heahh and Human Services. 1996) , research is needed on ways to increase ph~i-
1.1
iC::"~ "1 ~J'."' 1
Disability Model (Nagi, I9911
.", ;.t\
1'1
Disability Modal lamendad)
-_.t;:-~ >t: , " -' - ,
Figure 1.la) Nagi's (19911 model describingthe progressionI"ding to disability, and (blan amended version 01 the model suggesting that aninactive lifestyle, indapendent of thedis"sa process, canhalla comparabla effects on thedisabling process. From -Assessing Physical Parformance in Indepandant OIdar Adults: Issues and Guidelines ~ by A. Aikli & C. J. Jones, 1991, Journal ofAgingandPhysical Activiry. 5. p. 247. Copyright 1997 by Human Kinatics. Adapted with parmission.
e.
911
RQES: Jun. 2I1l1O
Rikli
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
cal activity participation within this age group. Naturally, such studies will require reliable and valid tools for assessing physical activity level. The purpose of the Washburn (2000) article and of this response paper is to review the status of measurement tools currently available to assess physical activity level in older adults, a variable which for many may be the lead factor in either contributing to or preventing physical decline in later years.
Methodology In his introduction, Washburn (2000) does a nicejob of reviewing the importance of physical activity during aging, citing evidence supporting the effect of physical activity on decreased risk of mortality, improved health, reduced risk of falling, decreased loss of bone, and decreased disability in later years. Following his discussion of the value of physical activity in later years, he offers compelling reasons why it is important to be able to measure physical activity in older people, reasons which include learning more about the correlates, causes, and consequences of physical inactivity and learning more about the optimum amounts and types of physical activity at different ages and for people with various health and mobility conditions. In describing the status of the assessment techniques available for measuring physical activity level in older adults, Washburn provides a brief overview of the various physical activity assessment options including motion sensors (accelerometers), heart rate monitoring, activity diaries, doubly labeled water, and physical activity (PA) questionnaires. He further explains that because of the cost and various logistical considerations associated with most techniques, the only method feasible for use in large epidemiological studies is the questionnaire. Consequently, he has limited his review of physical activity assessment techniques to questionnaires only. Prior to reviewing the PA questionnaires developed specifically for older adults, Washburn addressed the inappropriateness of using age-neutral questionnaires with older people. Age-neutral questionnaires, he explains, focus primarily on youth-oriented types of activities (e.g., sports and recreational activities) and, therefore, tend to result in an underestimate of total activity level and/or energy expenditure in older people whose more common forms of exercise include walking, gardening, and household activities. Washburn also describes the role of both direct and indirect techniques in validating PA questionnaires. Comparing questionnaire results with activity diaries, motion sensors, heart rate monitoring, and doubly labeled water are examples of direct validation evidence. Indirect validation evidence, on the other hand, is provided through the association ofPA question-
ROES: June 2000
naire results with variables that are influenced by physical activity, such as maximal oxygen uptake, resting heart rate, muscular strength, and body composition. In the main body of his paper, Washburn describes the four questionnaires that have been specifically developed for older adults-the Modified Baecke Questionnaire for Older Adults (Voorrips, Ravelli, Dongelmans, Deurenberg, & van staveren, 1993), the Zutphen Physical Activity Questionnaire (Caspersen, Bloemberg, Saris, Merritt, & Kromhout, 1991), the Yale Physical Activity Survey (DiPietro, Caspersen, Ostfeld, & Nadel, 1993), and the Physical Activity Scale for the Elderly (Washburn, Smith,Jette, &Janney, 1993). For each questionnaire he describes its general characteristics and comments on the evidence, or lack of evidence, supporting its reliability and validity. Following his review of the specific questionnaires, Washburn then makes several general statements summarizing his findings with respect to the documented quality of PA questionnaires.
Summary Statements Presented below are five summary statements made by Washburn based on his review of the four PA questionnaires developed specifically for older adults. Following each statement are my reactions and comments. The statements are not necessarily in the order discussed in the original paper.
Summary Statement #1: There is limited information supporting the reliability and validity of the questionnaires developed for older adults. The data that are available indicate little improvement over the age-neutral questionnaires as a means of assessing PA in older adults.
Response: It is not surprising to find limited or inconsistent information with respect to the reliability and validity of these instruments. The changing characteristics and increased variability in the health status and physical and mental ability levels of older adults create special challenges in developing reliable and valid measurement tools for this population. Examples of conditions commonly observed in older adults that might affect performance on PA questionnaire items, as well as performance on the criterion measures that are used to validate the questionnaires include: fluctuations in fatigue or pain level, temporary distractions/lack of concentration, impaired cognitive ability, reduced information processing 91
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
speed, changing medical conditions or medications, and fluctuations in mood, anxiety, and level depression. Increased individual inconsistency or intraindividual variability during aging has been well-documented on a number of characteristics and behaviors (Rikli &Jones, 1997; Spirduso, 1995; Spirduso, Francis, Light, & Reilly, 199Y) Although test unreliability can be due to either instrumentation inconsistency or to individual inconsistency, it is the latter which typically presents the greatest challenge in assessing older adults. To increase scoring precision and accuracy in older people, special adjustments may be needed-adjustments which might involve providing clearer instructions, repeated cueing, enhanced motivation, additional practice, and/or an increased number of trials. Although little data are available demonstrating the effects of increased intraindividual variability on questionnaire results, data on other kinds of measures clearly show the potential consequences of increased subject variability on test reliability. For example, in most of our research at Fullerton involving older adult assessments we have found that to achieve reliable measures we had to take extra care in clarifying test instructions and, on performance tests, had to increase both the amount of pretest practice time and the number of test trials compared to what typically is required for younger people. In a study involving simple reaction time, for example, nearly four times as many trials were required to obtain reliable measures for older people than generally are needed for younger people. Whereas only eight trials were necessary to achieve reliable test-retest scores for those ages 18 to 40 (R = .87) (Haywood & Teeple, 1976), more than 30 trials were required to achieve similar consistency in older adults (Lupinacci, Rikli,Jones, & Ross, 1993). Also, in developing a modified step test to assess aerobic endurance in older adults, we found test-retest reliabilities of only .42 to .58, using the same single-trial protocol that had been developed for younger subjects with reliability values over .90. However, by requiring a pretest practice trial on a prior day, step test reliability values for older adults increased to .90 and above (Dugas, 1996). Similarly, we have found that achieving stable 1 RM (repetition maximum) strength measures for older adults requires a three-day protocol (one day of practice and two test days) instead of the typical single day protocol commonly used with younger people (Rikli, Jones, Beam, Duncan, & Lamar, 1996). Although few studies have been completed on protocols for assessing physical activity level in older people, similar findings might be expected with respect to the need for an increased number of trials. Considering that a 7-day average of motion sensor readings was required to achieve a test-retest reliability of .80 or better in young people (Trost, Pate, Freedson, Sallis, & Taylor, 1999), it is presumed that considerably more than seven days might be required to achieve stable scores with older,
92
typically more variable individuals. With respect to PA questionnaire assessment specifically, it would be logical to assume that measures may also need to be averaged over more than one time period in order to obtain stable (reliable) indicators of an older person's typical activity patterns. Even with younger people, the reliability of questionnaire data was increased when participants were asked to report their activities over two different time periods (over the past 7 days and over a 'typical' 7-day period), with scores then averaged over the two periods (Prochaska, Sallis, & Long, 1999).
Summary Statement #2: When evaluating reliability, it is difficult to determine if poor agreement between repeated test administrations is due to poor recall or to actual changes in physical activity behavior.
Response: Although the question of error in recall versus actual changes in behavior is an important concern in evaluating test-retest reliability, the issue is much more complex than this. For example, in most questionnaire development to date, little attention has been given to the cognitive strategies that might best enhance memory in older adults and would solicit the most complete, accurate, and reliable types of information. In a recent review of the cognitive psychology literature (Durante & Ainsworth, 1996), it is suggested that one's recall ability can be significantly influenced by the types of probing questions or retrieval cues that are given. A general question asking about a person's engagement in moderate level activity is surely not as likely to provide the same degree of recall as would more probing questions asking about specific activity behaviors. For consistency of measurement, it also is important that all terms used in the questionnaire are clearly defined. Determining what constitutes "moderate" or "vigorous" types of activity, for example, is difficult enough for professionals in the field. Certainly, people unfamiliar with these terms will need clear explanations and examples of what they mean. Similarly, test administrators who are interpreting participant responses need clear guidelines as to what constitutes various levels of activity. Although most questionnaires involve translating activities into some type of "energy" or "MET" equivalents, there currently are no guidelines for making such interpretations for older adults. Energy cost classification systems, such as the Compendium of Physical Activities (Ainsworth et aI., 1993), have been developed based on normal activity patterns of young people, but may be far from accurate for use with older adults. The energy costs
ROES: June 2000
Rikli
of walking one mile for an older adult with a knee replacement or with a restricted gait pattern surely will be different than that for young people with normal walking patterns.
Summary Statement #3:
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
The evaluation of questionnaire validity is problematic due to the lack of an acceptable criterion measure of physical activity for older adults.
Response: The lack of suitable criterion measures for validating activity questionnaires for older adults is indeed a serious problem. Difficulties are encountered in virtually all of the common methods of assessment when they are used to measure physical activity level in older adultsactivity diaries, motion sensors, heart rate monitoring, or doubly labeled water. The use of activity diaries can be problematic with some older adults because of their inability to understand the directions and provide accurate records over extended periods of time or because of their lack of motivation in carrying out this rather tedious procedure which commonly involves recording one's type ofphysical activity every 15 minutes over several 24-hour periods. Of even greater concern, as discussed earlier, is the lack of a MET conversion system that can accurately estimate the energy expenditure associated with older adult activity patterns. Because the information on activity diaries typically is translated into energy expenditure or intensity MET values, the lack of such guidelines seriously limits the accuracy of this technique. The use of motion sensors to analyze activity level are also problematic with older people, with a major concern being that of their appropriate anatomical placement, a concern that is especially relevant in older adults who have some type of gait abnormality. For a person with arthritis in one hip, for example, very different readings would be obtained depending on whether the sensor was placed on the affected hip or the non-affected hip. Even in younger people with normal movement patterns, significantly different readings have been found for a motion sensor on one hip compared to one on the other hip (Nichols, Morgan, Sarkin, Sallis, & Calfas, 1999). Obviously, such discrepancies would be much greater when there are abnormal gait patterns of any kind. In fact, walking efficiency as a percent of normal has been estimated to be approximately 70% for people with arthritic knees or hips and only about 40% for people using a cane (Mulcare, Ayres, & Allen, 1999). Similarly, as would be expected, significantly different accelerometer (motion sensor) readings have been found for people walking
ROES: June 2000
under normal conditions compared to walking under simulated orthopedic conditions such as while wearing a knee brace or an ankle or foot orthodic (Mulcare et aI., 1999). Although the doubly labeled water method of assessing activity level is considered the 'gold standard' with respect to assessing total energy expenditure in young people, the appropriateness of this technique with older people is also questionable, particularly for people taking any kind of medication that could affect water retention. Additionally, the accuracy of the doubly labeled water technique might be questionable in people with any medical condition that affects the circulatory system-conditions involving peripheral vascular or cardiovascular circulation, renal function, or respiration, as examples. Heart rate monitoring is also less effective as a measure of physical activity level in older adults than it is in younger people. A number of common medications, particularly those affecting cardiovascular function, can influence both normal and exercise heart rates. Also, in some individuals, certain anatomical characteristics (e.g., excessive adipose tissue or redistributed body fat) can make it difficult to correctly position the heart rate monitor and obtain accurate readings from the monitors.
Summary Statement #4: It is suggested that using indirect methods in the validation of activity questionnaires may be inappropriate for older adults given that they typically do not participate in the kinds of high intensity conditioning that affects indirect measures, measures such as maximum aerobic capacity, body composition, muscular strength, and blood pressure and blood lipids, as examples. Response: Rather than eliminate the use of indirect methods in questionnaire validation, it might be more appropriate to reconsider the types of indirect measures that are most relevant for assessing the effects of physical activity for older adults. In older people, where maintaining functional ability is of increasing concern and, in fact, is a major reason why physical activity participation is important in later years, it would seem to make sense to use "functional" measures as indirect indicators of PA questionnaire validity instead of, or in addition to, some of the more traditional measures. In later years, for example, a person's ability to climb stairs, walk moderate distances, and do one's own shopping and household activities may be more important consequences (indicators) of regular physical activity than is maximal aerobic capacity or blood cholesterol level.
93
Rikli
Summary Statement #5: To improve response accuracy, it is suggested that PA questionnaires for older adults be interviewer-administered either in person or over the phone.
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Response: Although interviewer-administered questionnaires probably generate more accurate and reliable information than self-administered ones, the associated time demands may prohibit their use in many situations. It is extremely difficult, for example, to limit the conversation and 'story-telling' of some older people when asking them to describe their past activity levels. Such questions naturally remind them of past experiences, many of which they enjoy discussing in some detail. In using PA questionnaires in our studies at Fullerton, we have found that a combination of self-administered and intervieweradministered procedures works best. We typically mail the questionnaires out to people so that they can be completed in advance and then brought in to be reviewed by an interviewer. Completing a questionnaire at home gives people more time to respond to questions regarding their medical histories and types of medications they are taking, and gives them time to think through their past activities. The interviewer then meets with each person to go over the questionnaire to check for completeness, to clarify information, and to check for any potential errors due to misunderstood terms which often can happen, we have learned. In one of our studies, for example, an 80-year old woman had evaluated her own activity level as "vigorous," based on the fact that she swam nearly every day for an hour. However, through followup probing concerning the details of her swimming activity-like how many laps she averaged per day and the types of strokes she generally used, it soon was determined that she really did not 'swim' at all, but just went to the pool for an hour every day. Although the accuracy of self-administered questionnaires may be acceptable in large-scale epidemiologic studies, increased precision generally is needed in order to detect expected differences or relationships in smaller-scale studies.
Additional Reliability and Validity Issues For any test to be of value, it must have acceptable reliability and validity for the population of interest (American Psychological Association, 1985). A reliable test is one that produces consistent and dependable scores that are relatively free of measurement error. A valid test is one that accurately measures what it is intended to measure. To increase the reliability and validity of physical activity measures (or of any other mea-
94
sure), it is important to identify as many sources of error as possible, then plan strategies for reducing the error. It also is important to determine whether the measurement error is "random" or "systematic." Random or chance error generally affects a test's reliability and is treated in a different way than is systematic error which generally affects test validity. For example, if one were to measure another person's height using only a 12-inch ruler, there certainly would be more error involved (and lower test-retest reliability) than if a commercial height meter were used. However, the type of error involved in using a 12-inch ruler to measure height would be random error and could be greatly reduced by taking a large number of measures and averaging the results to obtain one overall measure. An example of systematic error, on the other hand, would be that which typically occurs when height is measured through self-report. Most people, it has been observed, tend to overestimate their height and underestimate their weight (Palta, Prineas, Berman, & Hannan, 1982), thus, resulting in biased and invalid measures of height and weight. Reducing this type of systematic error would require a different procedure than that involved in reducing random error. Simply repeating the measures (or questions, in this case) and averaging the responses would not improve the accuracy of the height measures at all. However, if it were possible to determine people's typical amounts of height overestimations at different ages, a conversion formula could be developed to correct for the error and thus improve the accuracy (validity) of self-reported height measures. Also, to improve reliability and validity it is important to consider not only the type of error (random vs systematic), but also the source of error-that is, whether the measurement error is associated with test instrumentation/administration procedures or if it is due to inconsistencies within the individual. For example, if the testretest reliability of a certain 'motion sensor' method of assessing physical activity data is low, the sources of error could be numerous and would need to be identified before strategies could be planned for reducing error. Possible sources of error in motion sensor measurement include improper calibration of instruments, inconsistent placement of instruments on the body, improper placement of the motion sensors, incorrect translations or coding of movement data, and/ or inconsistent behavior on the part of the individual, a condition which is especially characteristic of older people due to their fluctuations in pain level, energy level, or in their medical conditions, for example. Again, it is important to identify both the sources of error (instrumentation vs. individual) and the types of error (random vs. systematic) in order to plan effective ways of reducing error. Depending on the source and type of error, the strategies for reducing it might require
ROES: June 2000
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
Rikli
more frequent calibration of the instruments, better training of testing technicians, increased knowledge with respect to the optimum placement of motion sensors, and/or an increased number of days/trials over which measures are taken and averaged, thus, increasing the stability of measurements. In one recent study, for example, the reliability of the data from a 7-day activity record was increased by having participants record their activities over two 7-day periods and taking the average of the two (Baranowski et a!., 1999). Also, as mentioned earlier, Prochaska et a!. (1999) found that the reliability of questionnaire data could be increased by asking people to report their activities in two different waysfor a typical week and for the past seven days, then using an average of the two reports. Although it is an over simplification of the many kinds of potential measurement error, a grid similar to that presented in Figure 2 can be helpful in sorting out the basic types and sources of error involved in assessing physical activity level in older adults. After the possible sources of error have been identified, then studies can be designed to confirm the errors and/or to quantify the amount of error so that appropriate corrections or adaptations can be made. The work of Nichols and colleagues at San Diego State University (Nichols et a!., 1999) and the work cited earlier by Mulcare provide good examples of ways to begin sorting out the sources and amounts of error involved in measuring physical activity level. For example, by attaching different accelerometers to a mechanical shake table with a prescribed amount of motion, Nichols et a!. (1999) were able to determine the amount ofvariation in accelerometer data that might be attributable to instrument variability versus individual variability. Mulcare et a!. (1999), through simulating various orthopedic conditions in their laboratory, were able to estimate the amount of extra energy requirements associated with walking under specific abnormal conditions, such as with a knee brace or with an ankle/foot orthodic. Certainly,
Assessment Method
findings from studies such as these should improve the accuracy and interpretation of the data obtained in assessing physical activity level in older adults.
Conclusion In conclusion, Dr. Washburn is commended for providing a good overview of the availability and quality of PA questionnaires for older adults. In addition, it is clear that the issues related to improving the reliability and validity of physical activity assessment in older adults, including questionnaire assessment, are quite complex. Studies so far have barely scratched the surface with respect to identifying and responding to the potential kinds of error involved in assessing physical activity in older adults. .
References Ainsworth, B. E., Haskell, W. L., Leon, A. S., Jacobs, D. R., Montoye, H.]., Sallis,]. F., & Paffenbarger, R. S. (1993). Compendium of physical activities: Classification of energy costs of human physical activities. Medicine and Science in Sports and Exercise, 25,71-80. American Psychological Association. (1985). Joint standards for education and psychological tests. Washington, DC: American Psychological Association. Baranowski, T., Smith, M., Thompson, W.O., Baranowski,]., Hebert, D., & de Moor, C. (1999). Intraindividual variability and reliability in a 7-day exercise record. Medicine and Science in Sports and Exercise, 31,1619-1622. Buchner, D. M., & Wagner, E. H. (1992). Preventing frail health. Health Promotion and Disease Prevention, 8, 1-17. Caspersen, G.]., Bloemberg, B. P., Saris, W. H. M., Merritt, R. K., & Kromhout, D. (1991). The prevalence of selected
Random Error Instrument
Individual
Systematic Error Instrument
Individual
Figure 2. A grid for identifying potentialmeasurement errors accordingto types and sources.
RaES: June 2000
95
Rikli
physical activities and their relation with coronary heart disease risk factors in elderly men: The Zutphen study, 1985. American journal ofEpidemiology, 133, 1078-1092. Chandler,]. M., & Hadley, E. e. (1996). Exercise to improve physiologic and functional performance in old age. Clinics in Geriatric Medicine, 12, 761-784. Dept. of Health and Human Services. (1990). Healthy People
Downloaded by [University of Nebraska, Lincoln] at 04:52 14 September 2015
2000: National health promotion and disease prevention objectives (PHS 91-50213). Washington, DC: U.S. Government
Printing Office. Dept. of Health and Human Services. (1996). Healthy People 2000: Midcourse review and 1995 revisions. Washington, DC: U.S. Government Printing Office. DiPietro, L. (1996). The epidemiology of physical activity and physical function in older people. Medicine and Science in Sports and Exercise, 28,596-600. DiPietro, L., Caspersen, C.]., Ostfeld, A. M., & Nadel, E. R. (1993). A survey for assessing physical activity among older adults. Medicine and Sciencein Sports and Exercise, 25,628-642. Dugas, E. W. (1996). The development and validation ofa 2-minute step test to estimate aerobic endurance in older adults. Unpublished Master's Thesis, California State University, Fullerton, Fullerton, CA. Durante, R., & Ainsworth, B. E. (1996). The recall of physical activity: Using a cognitive model of the question-answering process. Medicine and Science in Sports and Exercise, 28, 1282-1291. Evans, w., & Rosenberg, I. H. (1991). Biomarkers: The lOdeterminants ofaging you can control. New York: Simon & Schuster. Feinstein, A. R.,Josephy, B. R., & Wells, c. K. (1986). Scientific and clinical problems in indexes of functional disabiliity. Annals oflntemalMedicine, 105,413-420.
Haywood, K. M., & Teeple,]. B. (1976). Representative simple reaction and movement time scores. Research Qy.arterly, 47,855-856. Lupinacci, N. S., Rikli, R. E., Jones, C.]., & Ross, D. (1993). Age and physical activity effects on reaction time and digit symbol substitution performance in cognitively active adults. Research Quarterly for Exercise and Sport, 64, 144-150. Morey, M. e., Pieper, e. E, & Comoni-Huntley,]. (1998). Physical fitness and functional limitations in community-dwelling older adults. Medicineand Science in Sports and Exercise, 30, 715-723. Mulcare.j. A., Ayres, S., & Allen,]. (1999). Measuring Physical Activity in Persons with Chronic Physical, Psychological, or Cognitive Disabilities. Paper presented at the American College of Sports Medicine Annual Meeting, Seattle, WA. Nagi, S. Z. (1965). Some conceptual issues in disability and rehabilitation. In M. B. Sussman (Ed.), Sociologyand Rehabilitation (pp. 100-113). Washington, D.C.: American Sociological Association. Nagi, S. Z. (1991). Disability concepts revisited: Implication for prevention. In A. M. Pope & A. R. Tarlov (Eds.), Disability in America: Toward a national agenda for prevention (pp. 309327). Washington, DC: National Academy Press. Nichols,]. E, Morgan, C. G., Sarkin,]. A., Sallis,]. E, & Calfas, K.]. (1999). Validity, reliability, and calibration of the Tritrac accererometer as a measure of physical activity. Medicine and Science in Sports and Exercise, 31, 908-912.
96
Palta, M., Prineas, R., Berman, R., & Hannan, P. (1982). Comparison of self-reported and measured height and weight. AmericanJoumalofEpidemiology, 115,223-230. Prochaska.j.j., Sallis,]. E, & Long, B. (1999, October). Strengths and limitations of youth self-reported physical activity. Paper presented at the Measurement of Physical Activity Symposium, Dallas. Rikli, R. E., & Jones, C.]. (1997). Assessing physical performance in independent older adults: Issues and guidelines. [ournal ofAging and Physical Activity, 5, 244-261. Rikli, R. E., &Jones, C.]. (1999a). Development and validation of a functional fitness test for community-residing older adults. journal of Aging and Physical Activity, 6, 127-159. Rikli, R. E., & Jones, C.]. (1999b). Functional fitness normative scores for community-residing adults, ages 60-94. Journal ofAging and Physical Activity, 6, 160-179. Rikli, R. E.,Jones, C.]., Beam, W. B., Duncan, S.]., & Lamar, B. (1996). Testing versus training effects on lRM strength measures in older adults. Medicine and Science in Sports and Exercise, 28(Suppl.), S153 (Abstract). Shephard, R.]. (1997). Aging, physical activity, and health. Champaign, IL: Human Kinetics. Spirduso, W.W. (1995). Physical dimensions ofaging. Champaign, IL: Human Kinetics. Spirduso, W. w., Francis, K. L., Light, K. E., & Reilly, M. A. (1999). Age differences in life-span reaction time: Withinand between-subject variability. [ournal ofAging and Physical Activity, 7, 296 (abstract). Trost, S. G., Pate, R. R., Freedson, P. S., Sallis,]. F., & Taylor, W. e. (1999, October). Using objectivephysical activity measures with youth: How many days of monitoring are needed? Paper presented at the Measurement of Physical Activity Symposium, Dallas. U.S. Bureau of the Census. (1996). Sixtyjiveplus in the United States: Current population reports (P23-190). Washington, DC: U. S. Department of Commerce. U.S. Dept. of Health and Human Services. (1996). Physical activity and health: A report of the surgeon general . Atlan ta: U. S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion. Voorrips, L. E., Ravelli, A. C.]., Dongelmans, P. C. A., Deurenberg, P., & van staveren, W. A. (1993). A physical activity questionnaire for the elderly. Medicine and Science in Sports and Exercise, 23,974-979. Washburn, R. A. (2000). Assessment of physical activity in older adults. Research Quarterly for Exercise and Sport, 71 (Supp!.) 79-89. Washburn, R. A., Smith, K. v«, Jette, A. M., & Janney, C. A. (1993). The physical activity scale for the elderly (PASE): Development and validation. journal of Clinical Epidemiology, 46, 153-162.
ROES: June 2000
