This article was downloaded by: [130.132.123.28] On: 01 February 2015, At: 08:57 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Nutrition and Cancer Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hnuc20
Reproducibility and comparability of a computerized, self‐administered food frequency questionnaire a
a
b
Althea Engle , Lois L. Lynn , Kenneth Koury & Andrea P. Boyar
c
a
American Health Foundation , 320 E. 43rd St., New York, NY, 10017 b
Lederle Laboratories , Pearl River, NY
c
Lehman College , City University of New York , Bronx, NY Published online: 04 Aug 2009.
To cite this article: Althea Engle , Lois L. Lynn , Kenneth Koury & Andrea P. Boyar (1990) Reproducibility and comparability of a computerized, self‐administered food frequency questionnaire, Nutrition and Cancer, 13:4, 281-292, DOI: 10.1080/01635589009514070 To link to this article: http://dx.doi.org/10.1080/01635589009514070
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Reproducibility and Comparability of a Computerized, Self-Administered Food Frequency Questionnaire
Downloaded by [] at 08:57 01 February 2015
Althea Engle, Lois L. Lynn, Kenneth Koury, and Andrea P. Boyar
Abstract A self-administered dietary assessment questionnaire was developedfor the microcomputer to identify individuals whose dietary patterns may put them at risk for cancer. It was tested among 50 adult volunteers in a New York school district. The quantitative food frequency portion of the questionnaire (FFQ), administered twice one month apart, was reproducible for calories, fat, percentage of calories from fat, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber (Spearman r = 0.56-0.87). To test for relative validity, individual nutrient intake calculatedfrom each administration of the FFQ was compared with the nutrient intake calculated from seven-day food records collected one month after the second FFQ administration. Nutrient intake from the first and second FFQ compared with food record nutrient intake yielded a Spearman's correlation coefficient of 0.58 and 0.62, respectively, for percentage of kilocalories from fat. No significant difference in mean intake of percentage of calories from fat was found between the FFQ 1 and FFQ 2 or between the FFQs and the food record. However, there were significant differences between mean food record and FFQ estimates of kilocalories, fat, vitamin A, vitamin C, calcium, and dietary fiber. We concluded that computerized nutrient assessment, which utilizes the subject in data entry, may be suitable for some clinical and educational uses and research studies of intake of fat as a percentage of calories among healthy adults. (Nutr Cancer 13, 281-292, 1990)
Introduction Increased interest in nutrition related to cancer and other chronic diseases has created an awareness of the need for improved dietary assessment methods (1). The dietary history method developed by Burke (2) was an attempt to determine the individual subjects' usual diet. It has been modified and used in epidemiological studies to determine subjects' past diet and thus exposure to nutritional factors that might influence their cancer risk (1). However, A. Engle and L.L. Lynn are affiliated with the American Health Foundation, New York, NY 10017. K. Koury is affiliated with Lederle Laboratories, Pearl River, NY. A.P. Boyar is affiliated with Lehman College of the City University of New York, Bronx, NY.
Copyright © 1990, Lawrence Erlbaum Associates, Inc.
Downloaded by [] at 08:57 01 February 2015
its reliance on the subject's ability to recall dietary intake in the distant past is a disadvantage. Furthermore, it is time consuming and costly to administer, because an extensive interview by a nutritionist is needed for data collection (3). Written food records are used for determination of actual food intake of subjects (3). To represent an individual's actual diet as usual over a long time would require few or many days of record keeping, depending on the nutrient of interest and the population group studied (4,5). Because subjects must record in detail the foods eaten and the amounts eaten at the time of eating (3), records require considerable time and cooperation from motivated subjects. Review and coding of the records before analysis is a time-consuming process performed by highly trained individuals. As a short-cut approach to obtaining usual intake, food frequency questionnaires (FFQ) that elicit information about how often subjects eat selected foods or types of foods per day, week, month, or year have been tested (6-10). When portion size is also obtained, the information can be converted to average estimated nutrient intake during the time period considered (6-10). Computerization has greatly aided efforts in dietary research and education. Review articles have described computer software used for diet assessment (11) as well as nutrition education (12). Computerized programs for the layperson are used to collect dietary practice information and analyze the food intake for a typical day or days but do not analyze the frequency of intake and usual portion size over a longer period of time as do FFQs (11-14). Computerized FFQ programs designed for professional use (15-18) require paper and pencil responses by the subject (16,18) or responses by a dietitian in an interview with the subject (15) or are entered directly by an interviewer into the computer (17,18). The purpose of this study was to develop and evaluate a computerized, self-administered questionnaire that would facilitate entry of responses by individual participants and would allow for rapid, cost-effective analysis of individual nutrient intake, without requiring extensive professional time. The ultimate goal was to identify current dietary and life style patterns that may predispose a healthy adult to greater risk of cancer. To our knowledge, this program is the first interactive computer questionnaire program intended to allow food frequency and portion size information to be entered directly by laypersons for determination of current usual nutrient intake of individuals. In this paper we report the nutrient intake from diet, excluding vitamin/mineral supplements. Materials and Methods
Development of Questionnaire An interactive microcomputer-based program was developed to allow participants to enter their own responses to a 120-item questionnaire. The major part of the questionnaire requested subjects to estimate how frequently they ate the foods listed, per day, week, or month, over the last three months. If the respondent reported eating a food, the portion size was automatically requested. Additional questions on demographic and anthropometric characteristics, smoking and exercise patterns, as well as the use of vitamin/mineral and /J-carotene supplements were included. The intent of this instrument was to cover a long enough time period to reflect eating patterns in the current season. If foods were eaten less than once per month, they were omitted from the frequency. Foods eaten every day for one week were reported as an average per month over the three months. The FFQ was designed to elicit information about individual consumption of dietary components associated with cancer risk in previous studies (19-30), namely kilocalories, fat, cholesterol, dietary fiber, vitamins A and C, and calcium. The 85 foods and food groupings included in the FFQ contributed at least 85% of the total intake of the nutrients of interest
282
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
in the diet of the general adult United States population, as determined by Block and co-workers (31,32). Fewer foods were listed individually on this questionnaire than the 100-item questionnaire developed by Block and others (33). The Block questionnaire was semiquantitative and classified portions as small, medium, and large, whereas the questionnaire in this study requested that respondents specify their exact portion size, that is, in cups, ounces, or tablespoons, whichever unit of measure the FFQ designated when asking for portion size. A small data file consisting of nutrient composition values for each food frequency item was developed using a representative food or a calculated, unweighted average of foods grouped together in a question. These values were obtained from the American Health Foundation's Diet Analysis System (DIAN), derived primarily from the USDA standard reference tape containing Agriculture Handbooks 8-1 through 8-12 (34), plus other published sources (35-37) and food manufacturers. In the FFQ data base all food items were given a value for each nutrient; the few missing values received a zero. Average portion size was requested in a specified unit of measure for each FFQ food or food grouping, that is, ounces for meat and candy and household measures for other foods (e.g., teaspoons for butter, cups for pasta). The individual's nutrient consumption was calculated by multiplying each portioi size estimation by the number of times consumed per day, week, or month, adjusted to a daily total, and by the nutrient value in the previously described data file for the unit of measure of food specified in each question. All nutrients supplied by each food or food groupi were then summed for total daily nutrient intake. Subjects Healthy adult volunteers were recruited from the faculty and staff of a Long Island, New York school district. Although the program was offered to all personnel, a total of 54 persons completed two administrations of the computerized questionnaire and one seven-day food record. Four persons who reported that they went on a diet after the first questionnaire administration were excluded, yielding 50 persons (34 women and 16 men) for the analysis. The age (mean ± SD) of the subjects was 49.3 ± 9.6 years, ranging from 26 to 69 years. Of the subjects, 84% were married, and 98% were white. The volunteers were not compensated financially but were given the nutrient analysis of their diet at the end of the study. Collection of Questionnaire Data Subjects responded to the questionnaire two times, one month apart, by typing their responses directly into IBM PC computers located in the school district's computer department. A set of Nasco food models, weighed food samples, empty food containers, and measuring cups and spoons were set up within reach of the participants for easy reference. Nutritionists supervised the questionnaire administration to a maximum of six people simultaneously per nutritionist and assisted subjects who had difficulty operating the computer, understanding the questions, or using the food models. Subjects' time needed for completion of questionnaire ranged from 20 to 70 minutes (average, 45 min). To assure completeness of the data, the computer program did not permit termination and storing of responses until all unanswered questions had been repeated and answered. Food Record Administration One month after the administration of FFQ 2, subjects completed a seven-day food record. Written and verbal instructions and standardized recording forms were provided by the nutritionists to the subjects prior to their completion of the seven-day food record. Subjects were asked to use the scale provided (HOAN 1-lb factory-calibrated dietary scale style no. 21011) or household measuring utensils or dimensions to quantify the food portions
Vol. 13, No. 4
283
consumed. At the end of the seven-day period, the food records were collected by the program administrator and sent to the American Health Foundation (New York) where they were reviewed by the nutritionists for completeness and adequacy of descriptive information. If necessary, participants were telephoned for clarification of the information. Food records were analyzed by the American Health Foundation's DIAN nutrient analysis system, operated by a nutritionist trained in the use of the system. Nutrient values were virtually complete in the data base, except for dietary fiber in a few foods eaten in small quantities and in other foods with minute amounts.
Downloaded by [] at 08:57 01 February 2015
Statistical Methods
Correlations among the values obtained from the food record and the two administrations of the FFQ were calculated using Spearman's correlation coefficient, a nonparametric measure, which is based on ranking of the data and is not overly influenced by extreme values. Pearson correlation coefficients were also calculated on both the raw and logtransformed nutrient intake values. All methods produced similar correlations, and Spearman coefficients were reported here. In addition, intraclass correlation coefficients (38) were computed as measures of reproducibility. A high intraclass correlation implies low within-person variability, in comparison to between-person variability. Adjustment for calories was carried out by dividing the dietary intake value by the number of calories consumed and multiplying the results by 1,000, yielding units of nutrients/1,000 calories. After nutrient estimates were transformed to their natural log to improve normality, a paired t test was performed to assess the statistical significance of the difference between mean nutrient estimates from the two FFQ administrations and the food record. Extremely high intake values of certain nutrients on the FFQ from four women were flagged as likely caused by errors in typing of answers. These women were reasked the particular questions by telephone interview after the second questionnaire, and those responses were used in the final analysis of the data. Spearman correlations computed with the unedited data and edited data did not differ by more than 0.05 for any of the nutrients. Program Adequacy Testing
One of the reasons for nutrient intake differences when comparing nutrient intake derived from an FFQ with intake calculated from a seven-day food record is the need to group foods on the questionnaire rather than to analyze specific foods. To condense the multitude of foods on the market into a short list of 75-100 food items in an FFQ requires categorizing foods in the FFQ into groups based on their similarity. Yet, the nutritional contribution of the foods categorized together is not identical. As a test of the ability of the 85 food items and the nutrient values for each food item in the FFQ to accurately estimate individual nutrient intakes, the seven-day food records from 10 of the subjects (a 20% sample) were used as the reference food consumption pattern for responding to the FFQ. A nutritionist entered responses to the FFQ based on the foods and amounts reported on the food record and the nutrient intake calculated from these responses was compared with the nutrient intake from the food record. This approach was used by Block and colleagues (33) in testing a data-based FFQ. However, Block used 24-hour recalls as the reference food consumption pattern. Spearman correlation coefficients between the two instruments ranged from 0.77 to 0.97 (Pearson log transformed r = 0.77-0.99). The lowest Spearman correlations were for fiber (0.85), cholesterol (0.85), and percentage of calories from fat (0.77). The Spearman correlations for kilocalories, fat, calcium, vitamin A, and vitamin C were above 0.90.
284
Nutrition and Cancer 1990
Results Descriptive statistics for each of the nutrient estimates and percentage of calories from fat are presented in Table 1. Mean and median values of all nutrients and cholesterol were lower on each successive nutrient intake assessment. Statistically significant differences were observed between mean nutrient intakes determined by each administration of the FFQ compared with the food record. However, mean and median percentage of calories from fat were within a narrow range of 32%-3i>%, and the means were not significantly different. Questionnaire 1 Versus Questionnaire 2
Downloaded by [] at 08:57 01 February 2015
Spearman correlation coefficients b;tween individual nutrient intake estimates derived from the two administrations of the FFQ are presented in Table 2. The values obtained by the two FFQs, when unadjusted for calories, correlated well (range r = 0.56-0.87) for all dietary components. When adjusted for calories, the correlations for nutrients changed inconsistently. Only the correlations for vitamin A and dietary fiber were slightly higher (r = 0.75 and r = 0.62) among the total group. For a sample size of 50, all correlation Table 1. Intake of Dietary Components From Computerized Questionnaire and Food Record" Nutrient/Instrument Kilocalories, kcal FFQ 1 FFQ 2 Food record Fat, g FFQ1 FFQ 2 Food record Kilocalories from fat, % FFQ1 FFQ 2 Food record Vitamin A, IU FFQ1 FFQ 2 Food record Vitamin C, mg FFQ1 FFQ 2 Food record Calcium, mg FFQ 1 FFQ 2 Food record Dietary fiber, g FFQ 1 FFQ 2 Food record Cholesterol, mg FFQ 1 FFQ 2 Food record
Mean ± SD* 2,679 ±111* 2,24!! ± 798* 1,717 ± 518* 104 ± 58"
Median
Range
2,498 2,188 1,678
1,086-5,172 956-5,129 819-3,070
84 ± 38* 62 ± 24*
90 79 58
28-208 28-200 26-121
34 ± 10" 33 ± 8* 3:> ± 7*
35 34 33
13-55 17-48 18-46
18,887 ± 15,263* 16,691 ± 10,869* 7,87:! ± 5,279*
15,724 13,243 6,722
2,652-85,308 3,358-46,943 1,435-24,405
24."! ± 152* 207 ± 1 1 8 * 13:> ± 63*
213 186 126
52-1,547 61-592 31-327
1,209 ± 668* 930 ± 358* 693 ± 233*
1,028 906 687
441-3,300 406-1,795 322-1,239
22 ± 15* 1!) ± 9* l(i ± 7*
18 16
14
7-85 5-53 6-34
412 ± 174* 367 ± 150* 30IS ± 121*
384 320 272
166-3,097 157-1,572 106-608
a: Data were provided by healthy volunteers in a New York school district in 1986,;i = 50. b: Means with same symbol are not significantly different, p < 0.05.
Vol. 13, No. 4
285
Downloaded by [] at 08:57 01 February 2015
Table 2. Spearman Correlation Coefficients Between Dietary Assessment Instruments" FFQ 1* and FFQ2C
FFQ 1 and Food Record
FFQ 2 and Food Record
Kilocalories Fat % Kilocalories from fat Cholesterol Vitamin A Vitamin C Calcium Dietary fiber
0.79 0.87 0.68 0.66 0.70 0.56 0.72 0.61
0.33 0.33 0.58 0.32 0.37 0.16 0.20 0.25
0.16 0.35 0.62 0.46 0.36 0.11 0.08 0.17
Adjusted for kilocalories Cholesterol Vitamin A Vitamin C Calcium Dietary fiber
0.60 0.75 0.50 0.53 0.62
0.32 0.40 0.19 0.53 0.36
0.43 0.46 0.37 0.44 0.51
a: Data were provided by healthy volunteers in a New York school district in 1986, n = 50. b: First FFQ administration. c: Second FFQ administration.
coefficients over 0.28 are statistically significant, that is, significantly different from zero (p < 0.05). Because the objective is to show the magnitude of association, the statistical significance of the correlations is not shown. Intraclass correlation coefficients ranged from 0.34 for calcium to 0.67 for fat and vitamin A among the total group of subjects (data not shown). For the percentage of calories from fat, the intraclass correlation was 0.63. Questionnaires Versus Seven-Day Food Record Spearman correlation coefficients for individual nutrient intake compared between FFQl and FFQ 2 and the food record, adjusted and unadjusted for calories, are also reported in Table 2. For percentage of calories from fat, the values obtained from FFQ 1 and the food record correlated well (r = 0.58). The adjustment for calories increased the correlation between FFQ 1 and the food record for all nutrients except cholesterol and resulted in a moderate correlation for calcium (r = 0.53). The correlations between FFQ 2 and the food record were similar to those obtained for FFQ 1. Except for calcium, all calorie-adjusted nutrient correlations between FFQ 2 and food record were higher than those between FFQ 1 and the food record. Individual nutrient intake from FFQ 1 and FFQ 2 and the food record was divided into three categories at the 25th and 75th percentile, thus combining the middle quartiles. The joint classification of individuals for percentage of calories from fat, dietary fiber, and vitamin A intake are listed in Table 3. Sixty-six percent fell into the same classification for percentage of calories from fat from FFQ 1 and the food record; only 4% were classified into opposite categories, that is, grossly misclassified. Similar results were obtained between FFQ 2 and the food record. Discussion The purpose of this paper has been to evaluate the performance of an interactive, computerized FFQ in assessing dietary intake of individuals. We observed that daily nutrient intake of individuals correlated well between two administrations of the FFQ. We also
286
Nutrition and Cancer 1990
Table 3. Joint Classification of Questionnaire and Food Record Estimates of Nutrient Intake Into Categories" Classification Same category, % Nutrient Percentage of calories from fat Dietary fiber Vitamin A
Adjacent category, %
Opposite category, %
FFQ 1
FFQ 2
FFQ 1
FFQ 2
FFQ I
FFQ 2
66 52 48
50 40 50
30 38 44
48 54 44
4 10 8
2 6 6
a: Data were provided by healthy volunteers in a New York school district in 1986, n = 50.
Downloaded by [] at 08:57 01 February 2015
observed that daily intake of fat as a percentage of calories measured by FFQ 1 and FFQ 2 correlated well (r = 0.58,0.62) with the daily percentage of calories from fat calculated from seven-day food records. There is no accepted "gold standard" for assessing individual dietary intake by which to judge the validity of other methods. To determine true validity would require that the actual food intake be known. However, when dietary intake over a long period of time is investigated (e.g., 3 mo), knowledge of the actual food intake is virtually impossible to obtain in a free-living population. Practical considerations preclude observation of such individuals 24 hours a day for a long period of time. Although the seven-day food record has been considered to be a reasonably accurate measure for assessing nutrient intake of individuals, the record may underestimate the actual consumption of food because of the respondent burden in recording (39). Because underestimation in food record measurement should tend to lower correlations between the two methods, a conservative estimate of relative FFQ validity is obtained in this study. The correlation of dietary intake with the FFQ could actually be higher if the true food intake had been used for the comparison. Other FFQ validation studies (9,10,40-42) have used a reference method of food records, with a time period ranging from one week (10,42) to one year (41). The Spearman correlation coefficient in our study for percentage of calories from fat measured by FFQ 1 and the seven-day food record is similar or high;r than correlations in studies with longer periods of food records (9,40,41). It is similar to the correlation (r = 0.51) between calorie-adjusted fat intake derived from a semiquantitative questionnaire and a one-year food record in a study reported by Willett and co-workers (41). Calorie-adjusted calcium intake, determined from FFQ 1, correlated moderately well (r = 0.53) with the calorie-adjusted daily calcium intake calculated from seven-day food records. Thus, expressed on a nutrient density basis, calcium intake from the two instruments was comparable. Our correlation for calorie-adjusted calcium intake (r = 0.53) is similar to the correlation of calorie-adjusted calcium intake from a self-administered FFQ (r = 0.55) reported by Willett and colleagues (41) in comparison to a one-year food record. It is higher than the correlation (r = 0.21, 0.26) reported by Willett and others (9) among female nurses and higher than the correlation reported by Stuff and co-workers (42) among lactating women. However, it is not as high as the unadjusted correlation (r = 0.76) reported by Cummings and colleagues (10) in a comparison of seven-day food records with a dietitianadministered semiquantitative FFQ among elderly women. The comparability of vitamin A intake has been generally poor in studies utilizing food records or a dietary history as a reference method (9,41-43). Correlation coefficients for vitamin A intakes measured by the FFQ and the reference methods, excluding vitamin supplements, ranged from r = 0.03 (43), r = 0.21 (9), and r = 0.40 (40) to r - 0.63 (41). The correlation coefficient of 0.37 and 0.36 observed by us between unadjusted vitamin A
Vol. 13, No. 4
287
Downloaded by [] at 08:57 01 February 2015
intake derived from FFQ 1 and FFQ 2 and vitamin A intake from the food record was similar to that observed by Pietinen and others (40) and higher than those in previous studies (9,43). Our correlation coefficients for vitamin C intake were lower than those in other studies, which ranged from 0.29 (43) to 0.46 (9), unadjusted for caloric intake, and from 0.46 (41) to 0.52 (9), when adjusted for caloric intake. It is possible that seasonal variation in market availability of food could account for part of the difference between instruments, particularly for assessment of vitamins A and C. In a comparison of dietary instruments, observed differences in diet could be attributed either to actual changes in the diet or to differences in the assessment technique. In this study, all assessments were carried out within a 10-week period in the winter to early spring so that seasonal variation in the diet is unlikely. However, if fewer oranges were eaten during the food record in May than during the winter months covered by the FFQ, the vitamin C and fiber intake could have been lower as a consequence of changed diet, not of faulty assessment. Thus, the average intake over three months as requested by the FFQ may have been closer to the real intake than the estimation of a particular week. Block (44) has suggested that questionnaires used to determine longer-term intake may be closer to the truth than a one-week record. For estimating and reporting food portions eaten, we provided subjects with a variety of food portion visuals, which they could use at their discretion. We thought it advisable not to limit the use of models for each question. Appropriateness of model is determined by the form in which food is eaten; for example, one-half cup mound used for mashed potato could be less accurate than one-half measuring cup in estimating boiled new potatoes. We do not know what bias may have been introduced by the participant's choice of a model for estimating portion size. Generally high reproducibility (mean r = 0.70) was obtained from administering the same FFQ twice, spaced one month apart. Thus, the subjects were able to answer the questions fairly consistently, yet the range of nutrient intake estimates narrowed on the second FFQ. It is conceivable that the subjects' greater familiarity with both the FFQ and the computer, plus the lag time over which respondents may have thought further about their answers could lead to fewer data entry errors and wild guesses on the second FFQ. Results from our study agreed with other studies in which nutrient intake estimates calculated from questionnaires tended to yield higher mean estimates than those calculated from food records (44,45). In our study this happened consistently with all dietary variables except percentage of calories from fat. Sorenson and co-workers (45) also found that vitamins A and C, calcium, and crude fiber gave the least consistent nutrient estimates among four dietary intake instruments, while fat was most consistently estimated when adjusted for calories. The correlation of dietary fiber between the FFQs and the seven-day record in our study was not as high as in other studies of fiber that looked at dietary or crude fiber (9,40,41). In the only study with dietary fiber, Pietinen and colleagues (40) reported correlation coefficients of 0.61 to 0.67 for unadjusted dietary fiber intake determined from a FFQ and 24 days of food records. However, the subjects were elderly men in Finland, where the daily intake of dietary fiber may be more stable than in the United States. The test of FFQ and nutrient data file adequacy undertaken as part of this study indicated that calculation of nutrient intake from responses of subjects to the FFQ did not correlate as well with nutrient intake from the seven-day food record as did FFQ responses using the individual's one-week record of food consumption entered by a nutritionist. It appears that the questions and the nutrient values in the FFQ are adequate to calculate estimated nutrient intake from individual diets. On the other hand, when individuals report their own intake, from recall, there may be over- or underestimation that results in lowered correlations. Whether the low correlation for intake of specific nutrients can be attributed to differences
288
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
in portion size estimation or to differences in estimation of consumption frequency was not ascertained. By allowing participants to specify their own portion sizes in common units of measure, we reduced the potential for systematic bias in the definition of portion sizes. However, the possibility of error, when individuals attempted to estimate portion sizes to a precise number or fraction of ounces, tablespoons, cups, and so on, was greater as the number of choices increased. Cummings and others (10) noted that estimates of calcium intake correlated more highly with seven-day records when portion sizes were reported as small, medium, or large rather than as specific numbers of ounces, cups, and so on. The variation in correlation coefficients of the different nutrients raised the question of whether the individuals are able to estimate consumption of some foods more successfully than other foods. Mullen and co-workers (46) suggested that the frequency of consumption of some foods was more accurately estimated than that of other foods and that some individuals were less successful in estimating their frequency of food consumption than others. These findings should be interpreted very carefully because the time period covered by the FFQ and food record was considerably different. Better correlation between nutrient intake of the food record with the FFQ might have resulted if the FFQ had covered the period of time that included the food record. However, we thought that keeping a food record would cause a training effect for portion size estimation and frequency of eating. By collecting food record data after the FFQ, we more closely duplicated the conditions under which a person answers the questions in a screening situation, without the advantage of previously recording their intake. For certain applications of the computerized questionnaire, this approach may have produced a more realistic evaluation of its performance as an assessment tool.
Limitations and Use
Participants in this study were administrative, professional, and clerical staff in a public school district. They were accustomed to paperwork and had good reading skills, although many had not previously used a computer. Thus, they may be better able to answer a questionnaire of this type than some other population groups in the United States. Further research would be needed to establish the reproducibility and relative validity in other groups. As the general United State; population becomes more knowledgeable about computers, and as computers become more readily available, a computerized questionnaire such as this may become even more feasible to implement. For small-scale epidemiological and dietary intervention studies, the FFQ could be used to evaluate estimated mean intake of percentage of calories from fat in a group and the distribution of individual intake of calories from fat. Nutrient data analysis time can be cut to one minute and respondent burden can be minimized by asking respondents to complete a 45-minute computerized questionnaire instead of a multiday food record. However, the amount of time required to complete the questionnaire may be a limitation in achieving a high cooperation rate among subjects. The computerized questionnaire is designed so that revision and addition of questions and nutrient data file values can be done easily with a word processing program. A revised version of the questionnaire has been tested and used for determination of type of fat consumed, polyunsaturated fat-to-saturated fat ratio, and percentage of calories from fat in a dietary intervention study (47). Sex differences in nutrient intake estimates by different assessment methods have been reported in the literature (45,48). In the present evaluation, we were not able to look at sex differences because of the small sample size. Further testing should be done before using the FFQ in large-scale nutrition research programs.
Vol. 13, No. 4
289
Use of the FFQ for determination of the other nutrients studied—kilocalories, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber—in place of a seven-day food record needs further testing, with subjects entering their own data at the computer. The program adequacy test demonstrated that the potential exists for excellent correlation between nutrient intake calculated from food records and nutrient intake computed by the FFQ. The challenge lies in assisting the respondents to complete the FFQ accurately. Orientation that would enable subjects to become aware of their eating without causing them to change their normal pattern could be useful. Perhaps the reference period of the FFQ should be shortened to a more easily recalled time period, such as the previous two weeks or one month.
Downloaded by [] at 08:57 01 February 2015
Summary and Conclusions
We found that when individuals entered their own responses to an interactive, computerized FFQ, estimates of individual nutrient intake, and percentage of calories from fat, correlated well between two administrations of the FFQ. Estimates of fat as a percentage of calories were comparable between the FFQ administrations and the food record. However, the estimates of total kilocalories, fat, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber were less in agreement with the seven-day records. These findings should be interpreted with caution because sex differences in response, which might affect reliability of the nutrient estimates, were not analyzed. We conclude that the FFQ could potentially be used for educational, clinical, and research purposes to produce reasonably accurate estimates of percentage of calories from fat in current usual diets of healthy adults. Particularly for clinical evaluation and education of individuals, the advantage of respondent-entered data and rapid nutrient analysis may be helpful. Acknowledgments and Notes The authors acknowledge the invaluable contributions of participants in this study and the advice of Dr. James Hebert and Dr. Clare Mahan. The authors thank Rita Sheets and Denis Tarpey of East Meadow School District (East Meadow, LI, New York). They also thank Edwigh Beauvoir for the preparation of the manuscript and Elanah Toporoff and Emma Engle for helpful comments. Work on this project was supported by Grant No. CA-40839 and CA-41144 from the National Cancer Institute, National Institutes of Health (Bethesda, MD). Address reprint requests to A. Engle, American Health Foundation, 320 E. 43rd St., New York, NY 10017. Submitted 30 March 1989; accepted in final form 29 November 1989.
References 1. Hebert, JR, and Miller, DR: "Methodologic Considerations for Investigating the Diet-Cancer Link." Am J Clin Nutr 47, 1068-1077, 1988. 2. Burke, BS: "The Dietary History as a Tool in Research." J Am Diet Assoc 23, 1041-1046, 1947. 3. Sorenson, AW: "Assessment of Nutrition in Epidemiologic Studies." In Cancer Epidemiology and Prevention, D Schottenfeld and JF Fraumeni (eds). Philadelphia, PA: Saunders, 1982, pp 434-474. 4. Marr, JW, and Heady, JA: "Within- and Between-Person Variation in Dietary Surveys: Number of Days Needed to Classify Individuals." Hum Nutr Appl Nutr 40A, 347-364, 1986. 5. Basiotis, PP, Welsh, SO, Cronin, FJ, Kelsay, JL, and Mertz, W: "Number of Days of Food Intake Records Required to Estimate Individual and Group Nutrient Intakes With Defined Confidence." J Nutr 117, 1638-1641, 1987. 6. Abramson, JH, Slome, C, and Kosovsky, C: "Food Frequency Interview as an Epidemiological Tool." Am J Public Health S3, 1093-1101, 1963. 7. Balogh, M, Medalie, JH, Smith, H, and Groen, JJ: "The Development of a Dietary Questionnaire for an Ischemic Heart Disease Survey." Isr J Med Sci 4, 195-203, 1968. 8. Hankin, JH, Rhoads, GG, and Glober, GA: "A Dietary Method for an Epidemiologic Study of Gastrointestinal Cancer." Am J Clin Nutr 28, 1055-1060, 1975. 9. Willett, WC, Sampson, L, Stampfer, MJ, Rosner, B, Bain, C, et al.: "Reproducibility and Validity of a Semiquantitative Food Frequency Questionnaire." Am J Epidemiol 122, 51-65, 1987. 10. Cummings, SR, Block, G, McHenry, K, and Baron, RB: "Evaluation of Two Food Frequency Methods of Measuring Dietary Calcium Intake." Am J Epidemiol 126, 796-802, 1987.
290
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
11. Ohlhausen, AA: "Facilitation of Nutritional Assessment by Computer Dietary Analysis." J Environ Pathol Toxicol Oncol 5, 249-263, 1985. 12. Slack, W, Porter, D, Witschi, J, Sullivan, M, Buxbaum, R, et al.: "Dietary Interviewing by Computer." J Am Diet Assoc 69, 514-517, 1976. 13. Levine, JA, Madden, AM, and Morgan, MY: "Validation of a Computer-Based System for Assessing Dietary Intake." Br Med J 295, 369-372, 1987. 14. Byrd-Bredbenner, C, and Pelican, S: "Software: How Do You Choose?" J Nutr Educ 16, 77-117, 1984. 15. Vailas, LI, Blankenhorn, DH, Selzer, RH, and Johnson, RL: "A Computerized Quantitative Food Frequency Analysis for the Clinical Setting: Use in Documentation and Counseling." J Am Diet Assoc 87, 1539-1543. 16. Hunt, IF, Luje, LS, Murphy, NJ, Clark, VA, and Coulson, AH: "Nutrient Estimates From Computerized Questionnaires vs 24-hr Recall Interviews." J Am Diet Assoc 74, 656-659, 1979. 17. Smucker, R, Block, G, Coyle, L, Harvin, A, and Kessler, L: "A Dietary and Risk Factor Questionnaire and Analysis System for Personal Computers." Am J Epidemiol 129, 445-449, 1989. 18. Marecic, M, and Bagby, R: "The Professional Dietitian." Nutr Today 24, 39-40, 1989. 19. Committee on Diet, Nutr & Cancer, Assembly of Life Sciences, Nationd Research Council: Diet, Nutrition, and Cancer. Washington, DC: National Academy Press, 1982. 20. Hinds, MW, Kolonel, LN, Lee, J, and Hankin, JH: "Dietary Cholesterol and Lung Cancer Risk Among Men in Hawaii." Am J Clin Nutr 37, 192-193, 1933. 21. Garland, C, Shekelle, RB, Barrett-Connor, E, Criqui, MH, Rosoff, AH, et al.: "Dietary Vitamin D and Calcium and Risk of Colorectal Cancer. A 19-year Prospective Study in Men." Lancet 1, 307-309, 1985. 22. Reddy, BS, Ekelund, G, Bohe, M, Engle, A, and Domellof, L: "Metabolic Epidemiology of Colon Cancer: Dietary Pattern and Fecal Sterol Concentrations of Three Populations." Nutr Cancer 5, 34-40, 1983. 23. Greenwald, P, Lanza, E, and Eddy, GH: "Dietary Fiber in the Reduction of Colon Cancer Risk." J Am Diet Assoc 87, 1178-1186, 1987. 24. Hinds, MW, Kolonel, LN, Hankin, JH, and Lee, J: "Dietary Vitamin A, Carotene, Vitamin C and Risk of Lung Cancer in Hawaii." Am J Epidemiol 119, 227-237, 1984. 25. Mettlin, C, Graham, S, and Swanson, M: "Vitamin A and Lung Cancer." JNCI 62, 1435-1438, 1979. 26. Byers, T, Vena, J, Mettlin, C, Swanson, M, and Graham, S: "Dietary Vitamin A and Lung Cancer Risk: An Analysis by Histologic Subtypes." Am J Epidemiol 120, 769-776, 1984. 27. Wald, NJ, Boreham, J, Hayward, JL, and Bulbrook, RD: "Plasma Retinol, β-Carotene and Vitamin E Levels in Relation to the Future Risk of Breast Cancer." Br J Cancer 49, 321-324, 1984. 28. Graham, S, Mettlin, C, Marshall, J, Priore, R, Rzepka, T, et al.: "Dietary Factors in the Epidemiology of Cancer of the Larynx." Am J Epidemiol 113, 675-680, 1981. 29. Marshall, J, Graham, S, Mettlin, A, Shedd, D, and Swanson, M: "Diet in the Epidemiology of Oral Cancer." Nutr Cancer 3, 145-149, 1982. 30. Wassertheil-Smoller, S, Rommey, SL, Wylie-Rosett, J, Slagle, S, Miller, G, et al.: "Dietary Vitamin C and Uterine Cervical Dysplasia." Am J Epidemiol 114, 714-724, 1981. 31. Block, G, Dresser, CM, Hartman, AM, and Carroll, MD: "Nutrient Sources in the American Diet: Quantitative Data From the NHANES II Survey. II. Macronutrients and Fats." Am J Epidemiol 122, 27-40, 1985. 32. Block, G, Dresser, CM, Hartman, AM, and Carroll, MD: "Nutrient Sources in the American Diet: Quantitative Data From the NHANES II Survey. I. Vitamins and Minerals." Am J Epidemiol 122, 13-26, 1985. 33. Block, G, Hartman, AM, Dresser, CM, Carroll, MD, Gannon, J, et al.: "A Data-Based Approach to Diet Questionnaire Design and Testing." Am J Epidemiol 124, 453-469, 1986. 34. USDA, Nutrition Monitoring Division, Survey Statistics Branch: Nutrient Data Base for Standard Reference. Hyattsville, MD: USDA, 1985. (Release no 5.) 35. Paul, AA, and Southgate, DAT: McCance and Widdowson's The Composition of Foods. London: Her Majesty's Stationery Office, 1978. 36. Lanza, E, and Butrum, RR: "A Critical Review of Food Fiber Analysis and Data." J Am Diet Assoc 86, 723-743, 1986. 37. Anderson, JW, Chen, WL, and Sieling, B: Plant Fiber in Food. Lexington, KY: HCF Diabetes Research Foundation, 1980. 38. Snedecor, GW, and Cochran, WG: Statistical Methods. Ames, IA: Iowa State Univ Press, 1980. 39. Gersovitz, M, Madden, JP, and Smiciklas-Wright, H: "Validity of the 24-Hour Dietary Recall and Seven-Day Record for Group Comparisons." J Am Die: Assoc 73, 48-55, 1978. 40. Pietinen, P, Hartman, AM, Haapa, E, Rasanen, L, Haapakoski, J, et al.: "Reproducibility and Validity of Dietary Assessment Instruments. II. A Qualitative Food Frequency Questionnaire." Am J Epidemiol 128, 667-676, 1988. 41. Willett, WC, Reynolds, RD, Cottrell-Hoehner, S, Sampson, L, and Browne, ML: "Validity of a SemiQuantitative Food Frequency Questionnaire: Comparison With a 1-Year Diet Record." J Am Diet Assoc 87, 43-47, 1987. 42. Stuff, JE, Garza, C, Smith, EO, Nichols, BL, and Montandon, CM: "A Comparison of Dietary Methods in Nutritional Studies." Am J Clin Nutr 37, 303-306, 1983.
Vol. 13, No. 4
291
Downloaded by [] at 08:57 01 February 2015
43. Gray, GE, Paganini-Hill, A, Ross, RK, and Henderson, BE: "Assessment of Three Brief Methods of Estimation of Vitamin A and C Intakes for a Prospective Study of Cancer: Comparison With Dietary History." Am J Epidemiol 119, 581-590, 1984. 44. Block, G: "A Review of Validations of Dietary Assessment Methods." Am J Epidemiol 115, 495-505, 1982. 45. Sorenson, A, Calkins, B, Connolly, MA, and Diamond, E: "Comparison of Nutrient Intake Determined by Four Dietary Intake Instruments." J Nutr Educ 17, 92-99, 1985. 46. Mullen, BJ, Krantzler, NJ, Grivetti, LE, Schutz, HG, and Meiselman, HL: "Validity of a Food Frequency Questionnaire for the Determination of Individual Food Intake." Am J Clin Nutr 39, 136-143, 1984. 47. Barone, J, Hebert, JR, and Reddy, M: "Dietary Fat and Natural Killer Cell Activity." Am J Clin Nutr 50, 861-867, 1989. 48. Bazzarre, TL, and Yuhas, JA: "Comparative Evaluation of Methods of Collecting Food Intake Data for Cancer Epidemiology Studies." Nutr Cancer 5, 201-214, 1983.
292
Nutrition and Cancer 1990-
Nutrition and Cancer Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hnuc20
Reproducibility and comparability of a computerized, self‐administered food frequency questionnaire a
a
b
Althea Engle , Lois L. Lynn , Kenneth Koury & Andrea P. Boyar
c
a
American Health Foundation , 320 E. 43rd St., New York, NY, 10017 b
Lederle Laboratories , Pearl River, NY
c
Lehman College , City University of New York , Bronx, NY Published online: 04 Aug 2009.
To cite this article: Althea Engle , Lois L. Lynn , Kenneth Koury & Andrea P. Boyar (1990) Reproducibility and comparability of a computerized, self‐administered food frequency questionnaire, Nutrition and Cancer, 13:4, 281-292, DOI: 10.1080/01635589009514070 To link to this article: http://dx.doi.org/10.1080/01635589009514070
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Reproducibility and Comparability of a Computerized, Self-Administered Food Frequency Questionnaire
Downloaded by [] at 08:57 01 February 2015
Althea Engle, Lois L. Lynn, Kenneth Koury, and Andrea P. Boyar
Abstract A self-administered dietary assessment questionnaire was developedfor the microcomputer to identify individuals whose dietary patterns may put them at risk for cancer. It was tested among 50 adult volunteers in a New York school district. The quantitative food frequency portion of the questionnaire (FFQ), administered twice one month apart, was reproducible for calories, fat, percentage of calories from fat, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber (Spearman r = 0.56-0.87). To test for relative validity, individual nutrient intake calculatedfrom each administration of the FFQ was compared with the nutrient intake calculated from seven-day food records collected one month after the second FFQ administration. Nutrient intake from the first and second FFQ compared with food record nutrient intake yielded a Spearman's correlation coefficient of 0.58 and 0.62, respectively, for percentage of kilocalories from fat. No significant difference in mean intake of percentage of calories from fat was found between the FFQ 1 and FFQ 2 or between the FFQs and the food record. However, there were significant differences between mean food record and FFQ estimates of kilocalories, fat, vitamin A, vitamin C, calcium, and dietary fiber. We concluded that computerized nutrient assessment, which utilizes the subject in data entry, may be suitable for some clinical and educational uses and research studies of intake of fat as a percentage of calories among healthy adults. (Nutr Cancer 13, 281-292, 1990)
Introduction Increased interest in nutrition related to cancer and other chronic diseases has created an awareness of the need for improved dietary assessment methods (1). The dietary history method developed by Burke (2) was an attempt to determine the individual subjects' usual diet. It has been modified and used in epidemiological studies to determine subjects' past diet and thus exposure to nutritional factors that might influence their cancer risk (1). However, A. Engle and L.L. Lynn are affiliated with the American Health Foundation, New York, NY 10017. K. Koury is affiliated with Lederle Laboratories, Pearl River, NY. A.P. Boyar is affiliated with Lehman College of the City University of New York, Bronx, NY.
Copyright © 1990, Lawrence Erlbaum Associates, Inc.
Downloaded by [] at 08:57 01 February 2015
its reliance on the subject's ability to recall dietary intake in the distant past is a disadvantage. Furthermore, it is time consuming and costly to administer, because an extensive interview by a nutritionist is needed for data collection (3). Written food records are used for determination of actual food intake of subjects (3). To represent an individual's actual diet as usual over a long time would require few or many days of record keeping, depending on the nutrient of interest and the population group studied (4,5). Because subjects must record in detail the foods eaten and the amounts eaten at the time of eating (3), records require considerable time and cooperation from motivated subjects. Review and coding of the records before analysis is a time-consuming process performed by highly trained individuals. As a short-cut approach to obtaining usual intake, food frequency questionnaires (FFQ) that elicit information about how often subjects eat selected foods or types of foods per day, week, month, or year have been tested (6-10). When portion size is also obtained, the information can be converted to average estimated nutrient intake during the time period considered (6-10). Computerization has greatly aided efforts in dietary research and education. Review articles have described computer software used for diet assessment (11) as well as nutrition education (12). Computerized programs for the layperson are used to collect dietary practice information and analyze the food intake for a typical day or days but do not analyze the frequency of intake and usual portion size over a longer period of time as do FFQs (11-14). Computerized FFQ programs designed for professional use (15-18) require paper and pencil responses by the subject (16,18) or responses by a dietitian in an interview with the subject (15) or are entered directly by an interviewer into the computer (17,18). The purpose of this study was to develop and evaluate a computerized, self-administered questionnaire that would facilitate entry of responses by individual participants and would allow for rapid, cost-effective analysis of individual nutrient intake, without requiring extensive professional time. The ultimate goal was to identify current dietary and life style patterns that may predispose a healthy adult to greater risk of cancer. To our knowledge, this program is the first interactive computer questionnaire program intended to allow food frequency and portion size information to be entered directly by laypersons for determination of current usual nutrient intake of individuals. In this paper we report the nutrient intake from diet, excluding vitamin/mineral supplements. Materials and Methods
Development of Questionnaire An interactive microcomputer-based program was developed to allow participants to enter their own responses to a 120-item questionnaire. The major part of the questionnaire requested subjects to estimate how frequently they ate the foods listed, per day, week, or month, over the last three months. If the respondent reported eating a food, the portion size was automatically requested. Additional questions on demographic and anthropometric characteristics, smoking and exercise patterns, as well as the use of vitamin/mineral and /J-carotene supplements were included. The intent of this instrument was to cover a long enough time period to reflect eating patterns in the current season. If foods were eaten less than once per month, they were omitted from the frequency. Foods eaten every day for one week were reported as an average per month over the three months. The FFQ was designed to elicit information about individual consumption of dietary components associated with cancer risk in previous studies (19-30), namely kilocalories, fat, cholesterol, dietary fiber, vitamins A and C, and calcium. The 85 foods and food groupings included in the FFQ contributed at least 85% of the total intake of the nutrients of interest
282
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
in the diet of the general adult United States population, as determined by Block and co-workers (31,32). Fewer foods were listed individually on this questionnaire than the 100-item questionnaire developed by Block and others (33). The Block questionnaire was semiquantitative and classified portions as small, medium, and large, whereas the questionnaire in this study requested that respondents specify their exact portion size, that is, in cups, ounces, or tablespoons, whichever unit of measure the FFQ designated when asking for portion size. A small data file consisting of nutrient composition values for each food frequency item was developed using a representative food or a calculated, unweighted average of foods grouped together in a question. These values were obtained from the American Health Foundation's Diet Analysis System (DIAN), derived primarily from the USDA standard reference tape containing Agriculture Handbooks 8-1 through 8-12 (34), plus other published sources (35-37) and food manufacturers. In the FFQ data base all food items were given a value for each nutrient; the few missing values received a zero. Average portion size was requested in a specified unit of measure for each FFQ food or food grouping, that is, ounces for meat and candy and household measures for other foods (e.g., teaspoons for butter, cups for pasta). The individual's nutrient consumption was calculated by multiplying each portioi size estimation by the number of times consumed per day, week, or month, adjusted to a daily total, and by the nutrient value in the previously described data file for the unit of measure of food specified in each question. All nutrients supplied by each food or food groupi were then summed for total daily nutrient intake. Subjects Healthy adult volunteers were recruited from the faculty and staff of a Long Island, New York school district. Although the program was offered to all personnel, a total of 54 persons completed two administrations of the computerized questionnaire and one seven-day food record. Four persons who reported that they went on a diet after the first questionnaire administration were excluded, yielding 50 persons (34 women and 16 men) for the analysis. The age (mean ± SD) of the subjects was 49.3 ± 9.6 years, ranging from 26 to 69 years. Of the subjects, 84% were married, and 98% were white. The volunteers were not compensated financially but were given the nutrient analysis of their diet at the end of the study. Collection of Questionnaire Data Subjects responded to the questionnaire two times, one month apart, by typing their responses directly into IBM PC computers located in the school district's computer department. A set of Nasco food models, weighed food samples, empty food containers, and measuring cups and spoons were set up within reach of the participants for easy reference. Nutritionists supervised the questionnaire administration to a maximum of six people simultaneously per nutritionist and assisted subjects who had difficulty operating the computer, understanding the questions, or using the food models. Subjects' time needed for completion of questionnaire ranged from 20 to 70 minutes (average, 45 min). To assure completeness of the data, the computer program did not permit termination and storing of responses until all unanswered questions had been repeated and answered. Food Record Administration One month after the administration of FFQ 2, subjects completed a seven-day food record. Written and verbal instructions and standardized recording forms were provided by the nutritionists to the subjects prior to their completion of the seven-day food record. Subjects were asked to use the scale provided (HOAN 1-lb factory-calibrated dietary scale style no. 21011) or household measuring utensils or dimensions to quantify the food portions
Vol. 13, No. 4
283
consumed. At the end of the seven-day period, the food records were collected by the program administrator and sent to the American Health Foundation (New York) where they were reviewed by the nutritionists for completeness and adequacy of descriptive information. If necessary, participants were telephoned for clarification of the information. Food records were analyzed by the American Health Foundation's DIAN nutrient analysis system, operated by a nutritionist trained in the use of the system. Nutrient values were virtually complete in the data base, except for dietary fiber in a few foods eaten in small quantities and in other foods with minute amounts.
Downloaded by [] at 08:57 01 February 2015
Statistical Methods
Correlations among the values obtained from the food record and the two administrations of the FFQ were calculated using Spearman's correlation coefficient, a nonparametric measure, which is based on ranking of the data and is not overly influenced by extreme values. Pearson correlation coefficients were also calculated on both the raw and logtransformed nutrient intake values. All methods produced similar correlations, and Spearman coefficients were reported here. In addition, intraclass correlation coefficients (38) were computed as measures of reproducibility. A high intraclass correlation implies low within-person variability, in comparison to between-person variability. Adjustment for calories was carried out by dividing the dietary intake value by the number of calories consumed and multiplying the results by 1,000, yielding units of nutrients/1,000 calories. After nutrient estimates were transformed to their natural log to improve normality, a paired t test was performed to assess the statistical significance of the difference between mean nutrient estimates from the two FFQ administrations and the food record. Extremely high intake values of certain nutrients on the FFQ from four women were flagged as likely caused by errors in typing of answers. These women were reasked the particular questions by telephone interview after the second questionnaire, and those responses were used in the final analysis of the data. Spearman correlations computed with the unedited data and edited data did not differ by more than 0.05 for any of the nutrients. Program Adequacy Testing
One of the reasons for nutrient intake differences when comparing nutrient intake derived from an FFQ with intake calculated from a seven-day food record is the need to group foods on the questionnaire rather than to analyze specific foods. To condense the multitude of foods on the market into a short list of 75-100 food items in an FFQ requires categorizing foods in the FFQ into groups based on their similarity. Yet, the nutritional contribution of the foods categorized together is not identical. As a test of the ability of the 85 food items and the nutrient values for each food item in the FFQ to accurately estimate individual nutrient intakes, the seven-day food records from 10 of the subjects (a 20% sample) were used as the reference food consumption pattern for responding to the FFQ. A nutritionist entered responses to the FFQ based on the foods and amounts reported on the food record and the nutrient intake calculated from these responses was compared with the nutrient intake from the food record. This approach was used by Block and colleagues (33) in testing a data-based FFQ. However, Block used 24-hour recalls as the reference food consumption pattern. Spearman correlation coefficients between the two instruments ranged from 0.77 to 0.97 (Pearson log transformed r = 0.77-0.99). The lowest Spearman correlations were for fiber (0.85), cholesterol (0.85), and percentage of calories from fat (0.77). The Spearman correlations for kilocalories, fat, calcium, vitamin A, and vitamin C were above 0.90.
284
Nutrition and Cancer 1990
Results Descriptive statistics for each of the nutrient estimates and percentage of calories from fat are presented in Table 1. Mean and median values of all nutrients and cholesterol were lower on each successive nutrient intake assessment. Statistically significant differences were observed between mean nutrient intakes determined by each administration of the FFQ compared with the food record. However, mean and median percentage of calories from fat were within a narrow range of 32%-3i>%, and the means were not significantly different. Questionnaire 1 Versus Questionnaire 2
Downloaded by [] at 08:57 01 February 2015
Spearman correlation coefficients b;tween individual nutrient intake estimates derived from the two administrations of the FFQ are presented in Table 2. The values obtained by the two FFQs, when unadjusted for calories, correlated well (range r = 0.56-0.87) for all dietary components. When adjusted for calories, the correlations for nutrients changed inconsistently. Only the correlations for vitamin A and dietary fiber were slightly higher (r = 0.75 and r = 0.62) among the total group. For a sample size of 50, all correlation Table 1. Intake of Dietary Components From Computerized Questionnaire and Food Record" Nutrient/Instrument Kilocalories, kcal FFQ 1 FFQ 2 Food record Fat, g FFQ1 FFQ 2 Food record Kilocalories from fat, % FFQ1 FFQ 2 Food record Vitamin A, IU FFQ1 FFQ 2 Food record Vitamin C, mg FFQ1 FFQ 2 Food record Calcium, mg FFQ 1 FFQ 2 Food record Dietary fiber, g FFQ 1 FFQ 2 Food record Cholesterol, mg FFQ 1 FFQ 2 Food record
Mean ± SD* 2,679 ±111* 2,24!! ± 798* 1,717 ± 518* 104 ± 58"
Median
Range
2,498 2,188 1,678
1,086-5,172 956-5,129 819-3,070
84 ± 38* 62 ± 24*
90 79 58
28-208 28-200 26-121
34 ± 10" 33 ± 8* 3:> ± 7*
35 34 33
13-55 17-48 18-46
18,887 ± 15,263* 16,691 ± 10,869* 7,87:! ± 5,279*
15,724 13,243 6,722
2,652-85,308 3,358-46,943 1,435-24,405
24."! ± 152* 207 ± 1 1 8 * 13:> ± 63*
213 186 126
52-1,547 61-592 31-327
1,209 ± 668* 930 ± 358* 693 ± 233*
1,028 906 687
441-3,300 406-1,795 322-1,239
22 ± 15* 1!) ± 9* l(i ± 7*
18 16
14
7-85 5-53 6-34
412 ± 174* 367 ± 150* 30IS ± 121*
384 320 272
166-3,097 157-1,572 106-608
a: Data were provided by healthy volunteers in a New York school district in 1986,;i = 50. b: Means with same symbol are not significantly different, p < 0.05.
Vol. 13, No. 4
285
Downloaded by [] at 08:57 01 February 2015
Table 2. Spearman Correlation Coefficients Between Dietary Assessment Instruments" FFQ 1* and FFQ2C
FFQ 1 and Food Record
FFQ 2 and Food Record
Kilocalories Fat % Kilocalories from fat Cholesterol Vitamin A Vitamin C Calcium Dietary fiber
0.79 0.87 0.68 0.66 0.70 0.56 0.72 0.61
0.33 0.33 0.58 0.32 0.37 0.16 0.20 0.25
0.16 0.35 0.62 0.46 0.36 0.11 0.08 0.17
Adjusted for kilocalories Cholesterol Vitamin A Vitamin C Calcium Dietary fiber
0.60 0.75 0.50 0.53 0.62
0.32 0.40 0.19 0.53 0.36
0.43 0.46 0.37 0.44 0.51
a: Data were provided by healthy volunteers in a New York school district in 1986, n = 50. b: First FFQ administration. c: Second FFQ administration.
coefficients over 0.28 are statistically significant, that is, significantly different from zero (p < 0.05). Because the objective is to show the magnitude of association, the statistical significance of the correlations is not shown. Intraclass correlation coefficients ranged from 0.34 for calcium to 0.67 for fat and vitamin A among the total group of subjects (data not shown). For the percentage of calories from fat, the intraclass correlation was 0.63. Questionnaires Versus Seven-Day Food Record Spearman correlation coefficients for individual nutrient intake compared between FFQl and FFQ 2 and the food record, adjusted and unadjusted for calories, are also reported in Table 2. For percentage of calories from fat, the values obtained from FFQ 1 and the food record correlated well (r = 0.58). The adjustment for calories increased the correlation between FFQ 1 and the food record for all nutrients except cholesterol and resulted in a moderate correlation for calcium (r = 0.53). The correlations between FFQ 2 and the food record were similar to those obtained for FFQ 1. Except for calcium, all calorie-adjusted nutrient correlations between FFQ 2 and food record were higher than those between FFQ 1 and the food record. Individual nutrient intake from FFQ 1 and FFQ 2 and the food record was divided into three categories at the 25th and 75th percentile, thus combining the middle quartiles. The joint classification of individuals for percentage of calories from fat, dietary fiber, and vitamin A intake are listed in Table 3. Sixty-six percent fell into the same classification for percentage of calories from fat from FFQ 1 and the food record; only 4% were classified into opposite categories, that is, grossly misclassified. Similar results were obtained between FFQ 2 and the food record. Discussion The purpose of this paper has been to evaluate the performance of an interactive, computerized FFQ in assessing dietary intake of individuals. We observed that daily nutrient intake of individuals correlated well between two administrations of the FFQ. We also
286
Nutrition and Cancer 1990
Table 3. Joint Classification of Questionnaire and Food Record Estimates of Nutrient Intake Into Categories" Classification Same category, % Nutrient Percentage of calories from fat Dietary fiber Vitamin A
Adjacent category, %
Opposite category, %
FFQ 1
FFQ 2
FFQ 1
FFQ 2
FFQ I
FFQ 2
66 52 48
50 40 50
30 38 44
48 54 44
4 10 8
2 6 6
a: Data were provided by healthy volunteers in a New York school district in 1986, n = 50.
Downloaded by [] at 08:57 01 February 2015
observed that daily intake of fat as a percentage of calories measured by FFQ 1 and FFQ 2 correlated well (r = 0.58,0.62) with the daily percentage of calories from fat calculated from seven-day food records. There is no accepted "gold standard" for assessing individual dietary intake by which to judge the validity of other methods. To determine true validity would require that the actual food intake be known. However, when dietary intake over a long period of time is investigated (e.g., 3 mo), knowledge of the actual food intake is virtually impossible to obtain in a free-living population. Practical considerations preclude observation of such individuals 24 hours a day for a long period of time. Although the seven-day food record has been considered to be a reasonably accurate measure for assessing nutrient intake of individuals, the record may underestimate the actual consumption of food because of the respondent burden in recording (39). Because underestimation in food record measurement should tend to lower correlations between the two methods, a conservative estimate of relative FFQ validity is obtained in this study. The correlation of dietary intake with the FFQ could actually be higher if the true food intake had been used for the comparison. Other FFQ validation studies (9,10,40-42) have used a reference method of food records, with a time period ranging from one week (10,42) to one year (41). The Spearman correlation coefficient in our study for percentage of calories from fat measured by FFQ 1 and the seven-day food record is similar or high;r than correlations in studies with longer periods of food records (9,40,41). It is similar to the correlation (r = 0.51) between calorie-adjusted fat intake derived from a semiquantitative questionnaire and a one-year food record in a study reported by Willett and co-workers (41). Calorie-adjusted calcium intake, determined from FFQ 1, correlated moderately well (r = 0.53) with the calorie-adjusted daily calcium intake calculated from seven-day food records. Thus, expressed on a nutrient density basis, calcium intake from the two instruments was comparable. Our correlation for calorie-adjusted calcium intake (r = 0.53) is similar to the correlation of calorie-adjusted calcium intake from a self-administered FFQ (r = 0.55) reported by Willett and colleagues (41) in comparison to a one-year food record. It is higher than the correlation (r = 0.21, 0.26) reported by Willett and others (9) among female nurses and higher than the correlation reported by Stuff and co-workers (42) among lactating women. However, it is not as high as the unadjusted correlation (r = 0.76) reported by Cummings and colleagues (10) in a comparison of seven-day food records with a dietitianadministered semiquantitative FFQ among elderly women. The comparability of vitamin A intake has been generally poor in studies utilizing food records or a dietary history as a reference method (9,41-43). Correlation coefficients for vitamin A intakes measured by the FFQ and the reference methods, excluding vitamin supplements, ranged from r = 0.03 (43), r = 0.21 (9), and r = 0.40 (40) to r - 0.63 (41). The correlation coefficient of 0.37 and 0.36 observed by us between unadjusted vitamin A
Vol. 13, No. 4
287
Downloaded by [] at 08:57 01 February 2015
intake derived from FFQ 1 and FFQ 2 and vitamin A intake from the food record was similar to that observed by Pietinen and others (40) and higher than those in previous studies (9,43). Our correlation coefficients for vitamin C intake were lower than those in other studies, which ranged from 0.29 (43) to 0.46 (9), unadjusted for caloric intake, and from 0.46 (41) to 0.52 (9), when adjusted for caloric intake. It is possible that seasonal variation in market availability of food could account for part of the difference between instruments, particularly for assessment of vitamins A and C. In a comparison of dietary instruments, observed differences in diet could be attributed either to actual changes in the diet or to differences in the assessment technique. In this study, all assessments were carried out within a 10-week period in the winter to early spring so that seasonal variation in the diet is unlikely. However, if fewer oranges were eaten during the food record in May than during the winter months covered by the FFQ, the vitamin C and fiber intake could have been lower as a consequence of changed diet, not of faulty assessment. Thus, the average intake over three months as requested by the FFQ may have been closer to the real intake than the estimation of a particular week. Block (44) has suggested that questionnaires used to determine longer-term intake may be closer to the truth than a one-week record. For estimating and reporting food portions eaten, we provided subjects with a variety of food portion visuals, which they could use at their discretion. We thought it advisable not to limit the use of models for each question. Appropriateness of model is determined by the form in which food is eaten; for example, one-half cup mound used for mashed potato could be less accurate than one-half measuring cup in estimating boiled new potatoes. We do not know what bias may have been introduced by the participant's choice of a model for estimating portion size. Generally high reproducibility (mean r = 0.70) was obtained from administering the same FFQ twice, spaced one month apart. Thus, the subjects were able to answer the questions fairly consistently, yet the range of nutrient intake estimates narrowed on the second FFQ. It is conceivable that the subjects' greater familiarity with both the FFQ and the computer, plus the lag time over which respondents may have thought further about their answers could lead to fewer data entry errors and wild guesses on the second FFQ. Results from our study agreed with other studies in which nutrient intake estimates calculated from questionnaires tended to yield higher mean estimates than those calculated from food records (44,45). In our study this happened consistently with all dietary variables except percentage of calories from fat. Sorenson and co-workers (45) also found that vitamins A and C, calcium, and crude fiber gave the least consistent nutrient estimates among four dietary intake instruments, while fat was most consistently estimated when adjusted for calories. The correlation of dietary fiber between the FFQs and the seven-day record in our study was not as high as in other studies of fiber that looked at dietary or crude fiber (9,40,41). In the only study with dietary fiber, Pietinen and colleagues (40) reported correlation coefficients of 0.61 to 0.67 for unadjusted dietary fiber intake determined from a FFQ and 24 days of food records. However, the subjects were elderly men in Finland, where the daily intake of dietary fiber may be more stable than in the United States. The test of FFQ and nutrient data file adequacy undertaken as part of this study indicated that calculation of nutrient intake from responses of subjects to the FFQ did not correlate as well with nutrient intake from the seven-day food record as did FFQ responses using the individual's one-week record of food consumption entered by a nutritionist. It appears that the questions and the nutrient values in the FFQ are adequate to calculate estimated nutrient intake from individual diets. On the other hand, when individuals report their own intake, from recall, there may be over- or underestimation that results in lowered correlations. Whether the low correlation for intake of specific nutrients can be attributed to differences
288
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
in portion size estimation or to differences in estimation of consumption frequency was not ascertained. By allowing participants to specify their own portion sizes in common units of measure, we reduced the potential for systematic bias in the definition of portion sizes. However, the possibility of error, when individuals attempted to estimate portion sizes to a precise number or fraction of ounces, tablespoons, cups, and so on, was greater as the number of choices increased. Cummings and others (10) noted that estimates of calcium intake correlated more highly with seven-day records when portion sizes were reported as small, medium, or large rather than as specific numbers of ounces, cups, and so on. The variation in correlation coefficients of the different nutrients raised the question of whether the individuals are able to estimate consumption of some foods more successfully than other foods. Mullen and co-workers (46) suggested that the frequency of consumption of some foods was more accurately estimated than that of other foods and that some individuals were less successful in estimating their frequency of food consumption than others. These findings should be interpreted very carefully because the time period covered by the FFQ and food record was considerably different. Better correlation between nutrient intake of the food record with the FFQ might have resulted if the FFQ had covered the period of time that included the food record. However, we thought that keeping a food record would cause a training effect for portion size estimation and frequency of eating. By collecting food record data after the FFQ, we more closely duplicated the conditions under which a person answers the questions in a screening situation, without the advantage of previously recording their intake. For certain applications of the computerized questionnaire, this approach may have produced a more realistic evaluation of its performance as an assessment tool.
Limitations and Use
Participants in this study were administrative, professional, and clerical staff in a public school district. They were accustomed to paperwork and had good reading skills, although many had not previously used a computer. Thus, they may be better able to answer a questionnaire of this type than some other population groups in the United States. Further research would be needed to establish the reproducibility and relative validity in other groups. As the general United State; population becomes more knowledgeable about computers, and as computers become more readily available, a computerized questionnaire such as this may become even more feasible to implement. For small-scale epidemiological and dietary intervention studies, the FFQ could be used to evaluate estimated mean intake of percentage of calories from fat in a group and the distribution of individual intake of calories from fat. Nutrient data analysis time can be cut to one minute and respondent burden can be minimized by asking respondents to complete a 45-minute computerized questionnaire instead of a multiday food record. However, the amount of time required to complete the questionnaire may be a limitation in achieving a high cooperation rate among subjects. The computerized questionnaire is designed so that revision and addition of questions and nutrient data file values can be done easily with a word processing program. A revised version of the questionnaire has been tested and used for determination of type of fat consumed, polyunsaturated fat-to-saturated fat ratio, and percentage of calories from fat in a dietary intervention study (47). Sex differences in nutrient intake estimates by different assessment methods have been reported in the literature (45,48). In the present evaluation, we were not able to look at sex differences because of the small sample size. Further testing should be done before using the FFQ in large-scale nutrition research programs.
Vol. 13, No. 4
289
Use of the FFQ for determination of the other nutrients studied—kilocalories, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber—in place of a seven-day food record needs further testing, with subjects entering their own data at the computer. The program adequacy test demonstrated that the potential exists for excellent correlation between nutrient intake calculated from food records and nutrient intake computed by the FFQ. The challenge lies in assisting the respondents to complete the FFQ accurately. Orientation that would enable subjects to become aware of their eating without causing them to change their normal pattern could be useful. Perhaps the reference period of the FFQ should be shortened to a more easily recalled time period, such as the previous two weeks or one month.
Downloaded by [] at 08:57 01 February 2015
Summary and Conclusions
We found that when individuals entered their own responses to an interactive, computerized FFQ, estimates of individual nutrient intake, and percentage of calories from fat, correlated well between two administrations of the FFQ. Estimates of fat as a percentage of calories were comparable between the FFQ administrations and the food record. However, the estimates of total kilocalories, fat, cholesterol, vitamin A, vitamin C, calcium, and dietary fiber were less in agreement with the seven-day records. These findings should be interpreted with caution because sex differences in response, which might affect reliability of the nutrient estimates, were not analyzed. We conclude that the FFQ could potentially be used for educational, clinical, and research purposes to produce reasonably accurate estimates of percentage of calories from fat in current usual diets of healthy adults. Particularly for clinical evaluation and education of individuals, the advantage of respondent-entered data and rapid nutrient analysis may be helpful. Acknowledgments and Notes The authors acknowledge the invaluable contributions of participants in this study and the advice of Dr. James Hebert and Dr. Clare Mahan. The authors thank Rita Sheets and Denis Tarpey of East Meadow School District (East Meadow, LI, New York). They also thank Edwigh Beauvoir for the preparation of the manuscript and Elanah Toporoff and Emma Engle for helpful comments. Work on this project was supported by Grant No. CA-40839 and CA-41144 from the National Cancer Institute, National Institutes of Health (Bethesda, MD). Address reprint requests to A. Engle, American Health Foundation, 320 E. 43rd St., New York, NY 10017. Submitted 30 March 1989; accepted in final form 29 November 1989.
References 1. Hebert, JR, and Miller, DR: "Methodologic Considerations for Investigating the Diet-Cancer Link." Am J Clin Nutr 47, 1068-1077, 1988. 2. Burke, BS: "The Dietary History as a Tool in Research." J Am Diet Assoc 23, 1041-1046, 1947. 3. Sorenson, AW: "Assessment of Nutrition in Epidemiologic Studies." In Cancer Epidemiology and Prevention, D Schottenfeld and JF Fraumeni (eds). Philadelphia, PA: Saunders, 1982, pp 434-474. 4. Marr, JW, and Heady, JA: "Within- and Between-Person Variation in Dietary Surveys: Number of Days Needed to Classify Individuals." Hum Nutr Appl Nutr 40A, 347-364, 1986. 5. Basiotis, PP, Welsh, SO, Cronin, FJ, Kelsay, JL, and Mertz, W: "Number of Days of Food Intake Records Required to Estimate Individual and Group Nutrient Intakes With Defined Confidence." J Nutr 117, 1638-1641, 1987. 6. Abramson, JH, Slome, C, and Kosovsky, C: "Food Frequency Interview as an Epidemiological Tool." Am J Public Health S3, 1093-1101, 1963. 7. Balogh, M, Medalie, JH, Smith, H, and Groen, JJ: "The Development of a Dietary Questionnaire for an Ischemic Heart Disease Survey." Isr J Med Sci 4, 195-203, 1968. 8. Hankin, JH, Rhoads, GG, and Glober, GA: "A Dietary Method for an Epidemiologic Study of Gastrointestinal Cancer." Am J Clin Nutr 28, 1055-1060, 1975. 9. Willett, WC, Sampson, L, Stampfer, MJ, Rosner, B, Bain, C, et al.: "Reproducibility and Validity of a Semiquantitative Food Frequency Questionnaire." Am J Epidemiol 122, 51-65, 1987. 10. Cummings, SR, Block, G, McHenry, K, and Baron, RB: "Evaluation of Two Food Frequency Methods of Measuring Dietary Calcium Intake." Am J Epidemiol 126, 796-802, 1987.
290
Nutrition and Cancer 1990
Downloaded by [] at 08:57 01 February 2015
11. Ohlhausen, AA: "Facilitation of Nutritional Assessment by Computer Dietary Analysis." J Environ Pathol Toxicol Oncol 5, 249-263, 1985. 12. Slack, W, Porter, D, Witschi, J, Sullivan, M, Buxbaum, R, et al.: "Dietary Interviewing by Computer." J Am Diet Assoc 69, 514-517, 1976. 13. Levine, JA, Madden, AM, and Morgan, MY: "Validation of a Computer-Based System for Assessing Dietary Intake." Br Med J 295, 369-372, 1987. 14. Byrd-Bredbenner, C, and Pelican, S: "Software: How Do You Choose?" J Nutr Educ 16, 77-117, 1984. 15. Vailas, LI, Blankenhorn, DH, Selzer, RH, and Johnson, RL: "A Computerized Quantitative Food Frequency Analysis for the Clinical Setting: Use in Documentation and Counseling." J Am Diet Assoc 87, 1539-1543. 16. Hunt, IF, Luje, LS, Murphy, NJ, Clark, VA, and Coulson, AH: "Nutrient Estimates From Computerized Questionnaires vs 24-hr Recall Interviews." J Am Diet Assoc 74, 656-659, 1979. 17. Smucker, R, Block, G, Coyle, L, Harvin, A, and Kessler, L: "A Dietary and Risk Factor Questionnaire and Analysis System for Personal Computers." Am J Epidemiol 129, 445-449, 1989. 18. Marecic, M, and Bagby, R: "The Professional Dietitian." Nutr Today 24, 39-40, 1989. 19. Committee on Diet, Nutr & Cancer, Assembly of Life Sciences, Nationd Research Council: Diet, Nutrition, and Cancer. Washington, DC: National Academy Press, 1982. 20. Hinds, MW, Kolonel, LN, Lee, J, and Hankin, JH: "Dietary Cholesterol and Lung Cancer Risk Among Men in Hawaii." Am J Clin Nutr 37, 192-193, 1933. 21. Garland, C, Shekelle, RB, Barrett-Connor, E, Criqui, MH, Rosoff, AH, et al.: "Dietary Vitamin D and Calcium and Risk of Colorectal Cancer. A 19-year Prospective Study in Men." Lancet 1, 307-309, 1985. 22. Reddy, BS, Ekelund, G, Bohe, M, Engle, A, and Domellof, L: "Metabolic Epidemiology of Colon Cancer: Dietary Pattern and Fecal Sterol Concentrations of Three Populations." Nutr Cancer 5, 34-40, 1983. 23. Greenwald, P, Lanza, E, and Eddy, GH: "Dietary Fiber in the Reduction of Colon Cancer Risk." J Am Diet Assoc 87, 1178-1186, 1987. 24. Hinds, MW, Kolonel, LN, Hankin, JH, and Lee, J: "Dietary Vitamin A, Carotene, Vitamin C and Risk of Lung Cancer in Hawaii." Am J Epidemiol 119, 227-237, 1984. 25. Mettlin, C, Graham, S, and Swanson, M: "Vitamin A and Lung Cancer." JNCI 62, 1435-1438, 1979. 26. Byers, T, Vena, J, Mettlin, C, Swanson, M, and Graham, S: "Dietary Vitamin A and Lung Cancer Risk: An Analysis by Histologic Subtypes." Am J Epidemiol 120, 769-776, 1984. 27. Wald, NJ, Boreham, J, Hayward, JL, and Bulbrook, RD: "Plasma Retinol, β-Carotene and Vitamin E Levels in Relation to the Future Risk of Breast Cancer." Br J Cancer 49, 321-324, 1984. 28. Graham, S, Mettlin, C, Marshall, J, Priore, R, Rzepka, T, et al.: "Dietary Factors in the Epidemiology of Cancer of the Larynx." Am J Epidemiol 113, 675-680, 1981. 29. Marshall, J, Graham, S, Mettlin, A, Shedd, D, and Swanson, M: "Diet in the Epidemiology of Oral Cancer." Nutr Cancer 3, 145-149, 1982. 30. Wassertheil-Smoller, S, Rommey, SL, Wylie-Rosett, J, Slagle, S, Miller, G, et al.: "Dietary Vitamin C and Uterine Cervical Dysplasia." Am J Epidemiol 114, 714-724, 1981. 31. Block, G, Dresser, CM, Hartman, AM, and Carroll, MD: "Nutrient Sources in the American Diet: Quantitative Data From the NHANES II Survey. II. Macronutrients and Fats." Am J Epidemiol 122, 27-40, 1985. 32. Block, G, Dresser, CM, Hartman, AM, and Carroll, MD: "Nutrient Sources in the American Diet: Quantitative Data From the NHANES II Survey. I. Vitamins and Minerals." Am J Epidemiol 122, 13-26, 1985. 33. Block, G, Hartman, AM, Dresser, CM, Carroll, MD, Gannon, J, et al.: "A Data-Based Approach to Diet Questionnaire Design and Testing." Am J Epidemiol 124, 453-469, 1986. 34. USDA, Nutrition Monitoring Division, Survey Statistics Branch: Nutrient Data Base for Standard Reference. Hyattsville, MD: USDA, 1985. (Release no 5.) 35. Paul, AA, and Southgate, DAT: McCance and Widdowson's The Composition of Foods. London: Her Majesty's Stationery Office, 1978. 36. Lanza, E, and Butrum, RR: "A Critical Review of Food Fiber Analysis and Data." J Am Diet Assoc 86, 723-743, 1986. 37. Anderson, JW, Chen, WL, and Sieling, B: Plant Fiber in Food. Lexington, KY: HCF Diabetes Research Foundation, 1980. 38. Snedecor, GW, and Cochran, WG: Statistical Methods. Ames, IA: Iowa State Univ Press, 1980. 39. Gersovitz, M, Madden, JP, and Smiciklas-Wright, H: "Validity of the 24-Hour Dietary Recall and Seven-Day Record for Group Comparisons." J Am Die: Assoc 73, 48-55, 1978. 40. Pietinen, P, Hartman, AM, Haapa, E, Rasanen, L, Haapakoski, J, et al.: "Reproducibility and Validity of Dietary Assessment Instruments. II. A Qualitative Food Frequency Questionnaire." Am J Epidemiol 128, 667-676, 1988. 41. Willett, WC, Reynolds, RD, Cottrell-Hoehner, S, Sampson, L, and Browne, ML: "Validity of a SemiQuantitative Food Frequency Questionnaire: Comparison With a 1-Year Diet Record." J Am Diet Assoc 87, 43-47, 1987. 42. Stuff, JE, Garza, C, Smith, EO, Nichols, BL, and Montandon, CM: "A Comparison of Dietary Methods in Nutritional Studies." Am J Clin Nutr 37, 303-306, 1983.
Vol. 13, No. 4
291
Downloaded by [] at 08:57 01 February 2015
43. Gray, GE, Paganini-Hill, A, Ross, RK, and Henderson, BE: "Assessment of Three Brief Methods of Estimation of Vitamin A and C Intakes for a Prospective Study of Cancer: Comparison With Dietary History." Am J Epidemiol 119, 581-590, 1984. 44. Block, G: "A Review of Validations of Dietary Assessment Methods." Am J Epidemiol 115, 495-505, 1982. 45. Sorenson, A, Calkins, B, Connolly, MA, and Diamond, E: "Comparison of Nutrient Intake Determined by Four Dietary Intake Instruments." J Nutr Educ 17, 92-99, 1985. 46. Mullen, BJ, Krantzler, NJ, Grivetti, LE, Schutz, HG, and Meiselman, HL: "Validity of a Food Frequency Questionnaire for the Determination of Individual Food Intake." Am J Clin Nutr 39, 136-143, 1984. 47. Barone, J, Hebert, JR, and Reddy, M: "Dietary Fat and Natural Killer Cell Activity." Am J Clin Nutr 50, 861-867, 1989. 48. Bazzarre, TL, and Yuhas, JA: "Comparative Evaluation of Methods of Collecting Food Intake Data for Cancer Epidemiology Studies." Nutr Cancer 5, 201-214, 1983.
292
Nutrition and Cancer 1990-
