Vol. 21, No. 6 Printed in Greet Britain
International Journal of Epidemiology © International Epidemiological Association 1992
Effects of Different Designs and Extension of a Food Frequency Questionnaire on Response Rate, Completeness of Data and Food Frequency Responses Kuskowska-Wolk A (Cancer Epidemiology Unit, University Hospital, S-751 85 Uppsala, Sweden), Holte S, Ohlander E-M, Bruce A, Holmberg L, Adami H-0 and BergstromR. Effects of different designs and extension of a food frequency questionnaire on response rate, completeness of data and food frequency responses. International Journal of Epidemiology 1992; 21: 1144-1150. The authors studied how the introduction of several modifications to a basic food frequency questionnaire can influence the results of dietary surveys. Modifications covered eight combinations based on three levels: increasing versus decreasing order of frequency categories; questionnaires without versus with questions about portion sizes, and questionnaires without versus with extra non-dietary questions. The sample included 6783 women between the ages of 40 and 70 years who took part in mammography screening. The women were randomly assigned to one of the eight study groups. All of the women in each group received one of the eight differentiy modified questionnaires. The forms extended in length by extra non-dietary questions and portion size categories resulted in a 20% higher total nonresponse compared to the shorter basic form. Partial non-response was significantly lower for all four questionnaire types that included portion sizes. When portion sizes were included in the questionnaire, the reported mean frequency of consumption was significantly reduced for fat (-10 times per month), milk (-6), bread (-5), vegetables (-2) and fish (-0.4). The decreasing order of response to the frequency categories was associated with a statistically significant increase in the frequency responses for bread (2.6 times per month), vegetables (2) and fish (0.6). These data provide evidence that the design and extension of food frequency questionnaires influence the results of dietary studies.
In recent years, substantial development, modification, and evaluation of food frequency questionnaires have taken place, as well as some studies on the validity and reproducibility of such questionnaires.IJ However, other specific characteristics of selfadministered food frequency questionnaires do not seem to have been studied, although investigators suspect that a questionnaire design and extension can influence the collected information. Our aim was to determine to what degree the design and extension of a questionnaire can influence the response rate, completeness of information and food frequency responses in dietary surveys. By making different modifications to a food frequency questionnaire, i.e. using an increasing versus a decreasing order of frequency categories, and adding questions about portion sizes, we studied their impact on the food frequency results. The influence of an increased number
Food frequency questionnaires play a key role in nutritional epidemiology. Because data collection using this method, compared to others, is relatively simple and inexpensive, investigators who conduct large-scale epidemiological studies find such questionnaires particularly attractive. In contrast to other more expensive methods applied in dietary surveys, the respondents can complete the questionnaire without the assistance of a skilled professional. A better understanding of how the design of food frequency questionnaires can influence the results of nutritional studies may help in further developing dietary survey methods and in interpreting the results of such surveys. • Cancer Epidemiology Unit, University Hospital, S-751 85 Uppsala, Sweden. • • Department of Statistics, Uppsala University, Sweden. t The National Food Administration, Uppsala, Sweden. t Department of Surgery, University Hospital, Uppsala, Sweden.
1144
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
A KUSKOWSKA-WOLK,* S HOLTE." E M OHLANDER/A BRUCE.1 L HOLMBERG/t H-O ADAMI* AND R BERGSTROM*•••
1145
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE
MATERIALS AND METHODS Study Design and Subjects The target population, which was invited for mammographic screening, consisted of all 40-70 year old female residents of two different counties in Uppsala Health Care Region. The study cohort consisted of 6783 subjects from two medium-size towns, taken on a consecutive basis according to when they received their invitation for mammography over the period 13 October 1986 to 20 March 1987. The sample was randomly divided into eight groups. During November or December 1987, each woman received by mail one of eight differently designed food frequency questionnaires. This delay was unavoidable due to administrative reasons. A second copy of the questionnaire was sent as a reminder to those women who had not answered the first questionnaire within 5 weeks. Food Frequency Questionnaire (Table 1) The food frequency questionnaire included 60 foods and food groups, and was designed with the objective of categorizing individuals by intake of selected foods and nutrients over the previous 6-month period. For each food item, eight frequency categories were defined by intake: never/seldom; 1-3 times per month; once a week; 2-3 times a week; 4-6 times a week; once a day; 2-3 times a day; and 4 or more times a day. The first type of questionnaire (IF)—the basic six-page form—included questions on age, marital status, education, parity, occurrence of breast cancer in first-level relatives (mother, sister, daughter), body
Statistical Methods Before a statistical analysis was performed, the following values (frequency per month) were assigned to the eight increasing (I) response categories: from 0.5 (never/seldom) through 2, 4, 10, 20, 30, 75, up to 120 (4 times a day or more), respectively. Moreover, most foods covered in the questionnaire were divided into
TABLE 1 Description of differences in design and extension of eight types of food frequency questionnaire Extension
Design Increasing frequency categories (I) Decreasing frequency categories (D)
Frequency
Frequency + portion size
Frequency + extra questions
(F)
(F + P)
(F + Q)
Frequency + portion size + extra questions (F + P + Q)
IF (basic form) n = 671'
IFP
IFQ
IFPQ
n - 651
n = 653
n = 651
DF
DFP
DFQ
DFPQ
n - 687
n - 652
n = 616
n = 620
* Number of analysed questionnaires in each group
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
weight, height, and diet supplements. Thereafter, some qualitative questions were presented about the usual type of diet (e.g. omnivorous, vegetarian etc.), type of fat spread on sandwiches and type of fat used in food preparation. The 60-item food frequency section followed with the eight frequency categories in increasing (I) order as listed above. To the second type (IFP) of questionnaire, a column was added which included portion sizes (P)—predefined as small, medium or large—and in some cases open-ended response possibilities for quantities of, e.g. slices of bread, glasses of milk, numbers of apples, oranges, etc. The third type (IFQ), consisted of the basic form plus one page with four questions (Q) on cigarette smoking, use of oral contraceptives and hormone replacement therapy during menopause. The fourth type (IFPQ), included both the column with portion sizes (P) and the page with extra questions (Q). For each of these four versions of the questionnaire with increasing (I) frequencies of food consumption, a parallel form with decreasing (D) frequencies was produced. Thus, eight different types of questionnaire were used covering combinations on three levels (F—increasing versus decreasing frequencies, P—without portion sizes versus with portion sizes, Q—without extra questions versus with extra questions), i.e. a construction with 2 x 2 x 2 = 8 possibilities (Table 1).
of non-dietary questions on the response rate was also evaluated, and possible interactions among all these modifications were examined.
1146
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
RESULTS A comparison of the eight study groups with respect to characteristics of the subjects that might influence answers about food consumption (age, education, relative body weight) revealed a very similar distribution of these variables among the groups (Table 2). Mean age for the study population was 55.5 ± 9.1 years, and mean body mass index (BMI = weight/ height2 ± standard deviation) based on self-reported data was 24.4 ± 3.6 kg/m 2 . Response Rate Overall, 5.7% of the women directly expressed their unwillingness to participate in the study, whereas 17% did not return the questionnaire even after a reminder (Table 3). The most extended versions of the questionnaire (IFPQ, DFPQ) led to a response rate that was about 4-6% lower than for the simplest questionnaires. In a logistic regression model with the three main factors, the addition of extra questions (factor Q) entailed a significant 13% increased risk of non-response (odds ratio [OR] = 1.13, 95% confidence interval [CI]: 1.02-1.27), while the inclusion of portion size (factor P) was not significantly associated with the nonresponse rate (OR = 1.10, 95% CI : 0.99-1.24). A model with just an interaction term between extra questions and portion sizes (Q x P) gave an OR of 1.20(95% CI : 1.06-1.37), i.e. a 'relative risk' of nonresponse for questionnaires including both portion sizes and extra questions that was 20% higher than for shorter questionnaires without both of these features. Completeness of Responses Partial non-response within the questionnaire depended directly on both the kind of question and the food item. The lowest non-response was found for such qualitative questions as type of diet (0.4%), type of fat
TABLE 2 Sododemographic characteristics of study subjects by type of the questionnaire: mean (standard deviation [SD]), age and body mass index (BMI}, and percentage distribution of education level Education, Vt Questionnaire type
Age, years mean (SD)
IF
54.9 55.4 55.8 55.8 54.9 55.2 55.2 55.6
DF IFP DFP IFQ DFQ IFPQ DFPQ
(9.3) (8.8) (9.2) (9.3) (8.9) (9.2) (8.7) (8,9)
BMI, kg/m 2 mean (SD) 24.2 24.1 24.4 24.6 24.5 24.4 24.3 24.2
(3.7) (3.7) (3.8) (3.6) (3.6) (3.6) (3.8) (3.8)
< 9 years
9-12 years
>12 years
52.3 52.6 53.6 54.0 54.4 55.4 53.1 53.3
29.8 28.9 29.4 30.7 28.4 28.9 28.5 29.2
17.9 18.5 17.0 15.3 17.2 15.7 18.4 17.5
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
11 groups to give a clearer picture of the effect of the questionnaire type on the reported frequency of consumption. The effect of the questionnaire type on the response rate was tested by logistic regression analyses. Both simple models with just the basic factors (F, P, Q) and models which in addition included interaction terms between the factors were analysed. The partial nonresponse was analysed by a one-way analysis of variance where the eight different questionnaire versions were compared, as well as by three-way analysis of variance where the factorial design of the experiment was used. The effect of the questionnaire type on the reported frequency of consumption of different food groups (measured as frequencies per month) was analysed in two ways. First, by one-way analysis of variance and Kruskal-Wallis non-parametric tests in simple comparisons between the eight groups. Secondly, by threeway analysis of variance (which can also be formulated as a regression analysis based on dummy variables) that utilized the factorial design of the experiment. In the parametric analyses, variables were analysed both in original and logarithmic forms. The analysis of variance (regression analysis) formally requires normal distributions. For many food items the frequencies are skewed and/or kurtotic. To avoid the difficulties caused by this, we mainly rely on the results from the analyses based on log-transformed values in the case of parametric methods. The results of these parametric analyses were similar to those obtained by use of the Kruskall-Wallis test, which makes no distributional assumptions. In the simple presentation of results we use ordinary means and standard deviations for ease of interpretation, although a case could certainly be made for using medians or geometric means and some measure of dispersion based on quartiles.
1147
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE TABLE 3 Influence of the type of the food frequency questionnaire on response rates Questionnaire type
IF DF IFP DFP IFQ DFQ IFPQ DFPQ Total
Questionnaires sent out n 848 884* 842 843 848 805" 865" 848 6783
Refusal
Non-response
n
%
n
Exclusion6 n
%
Questionnaires analysed n
34 62 47 46 38 40 45 74
4.0 7.0 5.6 5.5 4.5 5.0 5.2 8.7
142 131 137 139 148 143 165 147
16.7 14.8 16.3 16.5 17.5 17.8 19.1 17.3
1 4 7 6 9 6 4 7
0.1 0.5 0.8 0.7 1.1 0.7 0.5 0.8
671 687 651 652 653 616 651 620
79.1 77.7 77.3 77.3 77.0 76.5 75.3 73.1
386
5.7
1152
17.0
44
0.6
5201
76.7
used in food preparation (0.6%), mother's breast cancer (0.8%) and education (0.9%). The highest partial non-response shown was for medium-fat milk (37.0%), low fat milk (35.9%), whole milk (35.6%), and for butter and margarine spread on sandwiches (27.6% and 21.7%, respectively). In order to find reasons for the high rate of nonresponse to questions about milk, butter and margarine, 28 women who represented a random sample were interviewed in person. All of these women explained that they thought it sufficient only to fill in the kind of milk or sandwich fat-spread they usually consumed and that their lack of response to frequency implied 'never/seldom'. Table 4 illustrates the total of partial non-responses for all food items. Compared with the reference form IF, there was a significantly lower number of partial non-responses (on average three responses more) on those four questionnaire types that included portion sizes.
TABLE 4 Partial non-response to questions about frequency of consumption of 60 food items for the eight questionnaire types
Food Frequency Responses A comparison of the frequency distribution of food consumption among the eight questionnaire versions revealed differences in frequency responses for some food groups. Table 5 shows the mean (±SE) of frequencies per month for 11 food groups. Maximum absolute differences in mean values among the eight groups of women vary from 0.9 times per month (potatoes), 1.1 (fish), 1.3 (alcohol), 2.2 (cereals/pasta), 3.8 (meat) to 6.1 (fruits), 6.7 (sugar/sweets/cakes), 7.1 (vegetables),
and 10.0 for milk, 10.8 for bread, and 13.2 for fat spread on sandwiches. Relative differences expressed as a percentage of the lowest mean values are from 3.9% (potatoes) to 11.7% (meat), 12.3% (fruits), 13.6% (bread), 13.9% (fish), 14.8% (vegetables), and 23.8% for fat spread on sandwiches and 24.8% for milk. A Kruskal-Wallis test comparing the eight questionnaire types indicated that these differences were significant for all but four (potatoes, cereals/pasta, sugar/sweets/cakes and alcohol) of the 11 food
Number of food iitems without response Questionnaire type IF DF IFP DFP IFQ DFQ IFPQ DFPQ
Questionnaires mean SD median mode with mode, V* F-value" 8.7 8.3 9.3 9.2 6.0 10.7 5.4 9.6 9.3 8.4 10.1 9.9 5.4 8.7 7.0 11.2
6 6 1 2 6 7 2 2
3 4 0 0 4 4 0 0
15 13 26 21 14 13 28 29
_b
1.69 25.74 37.82 1.54 7.91 38.64 10.11
° Results from a follow-up analysis in connection with the one-way analysis of variance b The tests compare each of the remaining seven questionnaire types with the basic questionntire (IF); critical F-value = 3.84, higher F-values statistically significant
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
" Inadvertently some individuals did not receive the correct type of the questionnaire; this caused a slight difference in the number of individuals per group. b The exclusion group consisted of people not belonging to the target population, i.e. younger or older than 40-70 years (n = 8) and people with unreliable responses (n = 36). Responses were considered as unrelialbe when reported frequency of consumption exceeded 2-3 times per day for some food items (e.g. potatoes, rice, pasta, salmon, shrimps etc.).
1148
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
TABLE 5 Mean (± standard error [SE]) frequency of consumption of food items per month for 11 food groups in each of eight questionnaire types as well as test for establishing significance of differences in the means. The lowest and the highest values in each food group underlined rype of questionnaire Food groups 1
DF
868
+
67.1 48.5 23.1 49.7 49.7 23.1 VI 7 8.6 57.7
± 2.1 ± 2.0 ± 0.4 ± 1.6 ± 1.4 ± 0.7 + 08 ± 0.4 ± 2.1
68.6 48.7 23.7 52.7 52.5 25.3 34.3 8.8 57.4
9.7
± 0.4
9.6
1 R 84.9
IFQ
IFP
DFQ
+
1 8 86.5
+ 70
90.3
±?
±
2.2
± 2.3 ± 2.1 ± 0.5 ± 1.9 ± 1.9 ± 0.8 ±1 1
± 1.8 ± 0.4 ± 1.5 ± 1.5 ± 0.7 + 09 ± 0.2 ± 2.0
65.5 50.4 23.1 49.6 53.1 24.1 33.9 8.7 55.6
± 2.0 ± 2.2 ± 0.4 ± 1.4 ± 1.6 ± 0.7 ± 0.2 ± 2.0
66.2 46.4 23.7 55.1 55.8 34.8 34.8 9.0 59.4
± 0.4
9.0
± 0.4
9.6
+
08
1 79.5
±0.3
± 2.4 ± 0.4
55.4 41.5 23.4 48.0 54.1 23.7 34.3 7.9 60.7
DFP
+1
8 85.7
± 1.7 ± 2.1 ± 0.4 ±
± ± +
± ±
55.4 44.0 24.0 1.2 49.9 1.5 54.6 0.7 24.7 0 8 36.3 0.2 9.0 2.3 62.3
8.8 ± 0.4
9.0
IFPQ 1 7 79.6
+
I 2 -value b
DFPQ 1 8 82.1
+1
8
30.7
± 2.0 ± 2.0 ± 0.4 ± 1.4 ± 1.5 ± 0.7 + 09 ± 0.3 ± 2.1
56.1 40.4 23.5 48.8 54.9 23.9 33.8 7.9 57.3
± 1.8 ± 1.9 ± 0.4 ± 1.4 ± 1.7 ± 1.0 +10 ± 0.2 ± 2.0
56.3 42.9 23.5 48.8 53.4 23.8 32.5 8.6 58.0
± 1.8 + 2.1 ± 0.4 ± 1.4 ± 1.7 ± 0.8 + 08 ± 0.3 ± 2.2
39.9 26.9 7.0 29.9 22.6 8.5 15.9 26.4 8.3
± 0.4
8.4
± 0.4
9.7
± 0.4
6.1
° Composition of food groups: 1. white, whole grain, hard bread; 2. butter, margarine; 3. milk 0.5% fat, 1.5ft fat, 3ft fat, sour milk and yoghurt 0.5% fat, 3ft fat; 4. boiled, fried and French fried potatoes; 5. carrots, beets, cabbage, tomatoes, iceberg/head lettuce, spinach and kale; 6. apples, pears, oranges, grapefruit, bananas, juice; 7. cooked oatmeal and other types of cooked breakfast cereal, com flakes and other types of cold breakfast cereal, pancakes and waffles, rice, spaghetti and noodles; 8. meat and sausage, main dishes, cut meats and sausage used on sandwiches, bacon, liver and kidneys, blood (black) pudding,liver pate, chicken; 9. salmon, mackerel, herring, sardines, tuna fish and other types of fish, shrimp, lobster; 10. cake, ice cream, sweet soup, jam, carbonated beverages, sweets, chocolate, sugar; 11. beer (1.8ft alcohol, 2.8ft and 4.5ft), wine, spirits. b Kruskal-Walhs test, values above the critical x 2 = 14.1 are significant.
groups. The three-way analysis of variance with F, P and Q as factors further clarified the effects of these factors for bread, fat, milk, vegetables and fish. Thus the addition of the column with portion sizes (P) reduced the mean monthly frequencies for these food groups by about 10 units for fat spread on sandwiches (which corresponded to -17% in relative terms), 6 units (-14%) for milk, 5 units (-6%) for bread, 2 units (-4%) for vegetables and 0.4 units (-5%) for fish. The decrease in the mean frequencies due to the factor P (portion size), measured as the regression coefficient of this factor, was very similar for different models (models that differed with respect to the inclusion/ exclusion of certain of the factors P, F and Q and interaction terms of these factors). The three-way analysis of variance also revealed that for bread, vegetables and fish—but none of the eight other food groups—the frequency responses were sensitive to whether the response categories were in increasing or decreasing order (the factor F). With the categories in decreasing order (from ^ 4 times/day to never/seldom) the average monthly frequencies were significantly increased by 2.6 units per month ( + 3%) for bread, 2 units ( + 4%) for vegetables and 0.6 units ( + 7%) for fish. Thus inclusion of portion sizes (the
factor P) and a decreasing order of response categories (the factor F) had opposite effects on the mean values of consumption frequencies for some foods. Inclusion of extra questions (the factor Q) did not have an effect for any food group. DISCUSSION The study included large groups of participants for each type of the questionnaire. Therefore, the statistical power of the analysis is great. This means that, although statistically significant, differences need not always be large enough to be of real importance. Response Rate It has been generally recognized that the response rate in a study depends on the complexity of the method used.1"4 Also, the strategy used in data collection, such as mailed questionnaire, telephone interview or inperson interview, influences the response rate. 36 To our knowledge, however, it has not been quantified just how much the design and extension of the questionnaire can influence the response rate when the same basic strategy and method are used. Our response rate for the basic questionnaire (79%) was comparable to that of a similar study (77%) done on women in
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
1. Bread 2. Fat used on sandwiches 3. Milk 4. Potatoes 5. Vegetables 6. Fruit 7. Cereal/pasta/rice 8. Meat 9. Fish 10. Sugar/sweets/ cake/ice cream, etc. 11. Alcohol (beer/wine/spirits)
IF
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE
Completeness of Responses The most striking partial non-response in our study was obtained for the three specific types of milk and two sorts of fat used on sandwiches. These questions clearly had deficiencies in their construction or in the information given about how to fill in the questionnaire. The subjects probably treated the separate questions about different types of milk as a multiple question, thus providing only one answer, similarly for the fat. In general omitted answers need not mean 'never/seldom' consumption. Missing values may also reflect an uncertainty on the part of subjects as to which response is the more appropriate. Participants might also inadvertently omit an answer. The study findings regarding partial non-response unexpectedly revealed significantly fewer blank answers for frequencies in all questionnaires that included portion sizes. It can be speculated that the questions about portion sizes for some food items make it easier for the respondent to choose answers for frequency categories or perhaps these forms are just more carefully studied. Food Frequency Responses We tested the consistency of frequency results from the eight differently designed and extended questionnaires. It should be noted that there is not necessarily a
relationship between consistency and validity. Thus, very highly consistent results from different questionnaires need not mean that these questionnaires have a high level of validity. It may also be true that there is not a relationship between rate of response to the individual questions and rate of return of the questionnaire. Findings in our study revealed that food frequency responses were sensitive to the portion sizes, i.e. if only frequencies were asked for, or if both frequencies and relative portion sizes had to be filled in. Although to a lesser extent, the frequency answers were also sensitive to whether the predefined categories were in increasing or decreasing order. However, not all food items were sensitive. Bread, vegetables and fish were the more sensitive both as regards portion sizes and increasing/decreasing frequencies. Fat and milk were sensitive only with respect to portion sizes. All other differences found among the frequency responses were no more than would be expected by chance. Our findings of statistically significant differences should have implications for the comparison between different dietary surveys and when quantitative metaanalyses of these or pooling analyses are attempted. It is noteworthy that food items identified as sensitive to the questionnaire design and extension in our study are those that are often considered as potential risk factors (fat, milk fat) or potential protectors (vegetables, fish, bread fibre) for some chronic diseases. Other recent studies have revealed that results from food frequency questionnaires can be sensitive to the cognitive context of the questions.7'8 Social desirability of response also affects different foods in different ways. Those foods that are considered as healthy, e.g. vegetables, have been found to be overestimated, while the opposite is true for 'unhealthy' foods, such as sweets.9 Food frequency questionnaires are subject to the sorts of biases that other dietary assessment methods, such as 24-hour recalls, are probably not subject to. This is true because of the long-term averaging that is involved in the subject having to recall exposures of very long periods of time and to average them in his/her head. This is qualitatively very different from asking individuals specifically what they ate the day before. There is a definite need for further studies on food frequency questionnaires in order to gain a better understanding of the type of misclassification they might cause and its implications for results of epidemiological studies of diet and disease. We plan to analyse these data further in order to determine how the design of food frequency questionnaires influences the calculation of the mean daily intake of different
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
nearly the same age groups, using a similar mailed frequency questionnaire.4 The present study revealed that addition of non-dietary questions and notably questions about relative portion sizes entailed a significantly lower response rate than the basic form. The sampling procedure used and the random allocation of study subjects to the type of questionnaire make it highly unlikely that any selection bias depending on sociodemographic variables could have occurred. Mean age, body mass index and distribution of type of education were similar among participants in the eight groups. A more difficult question, which has not been studied so far, is whether or not, with respect to the pattern of participation (or completeness of response), there is an interaction between the design and extension of the questionnaire and any type of disease. A questionnaire which, owing to its design, could introduce a selection bias (or differential misclassification) might invalidate a case-control study of diet as a risk factor. This also might invalidate even follow-up studies because people may believe in certain health-disease relationships irrespective of the presence of the disease. As believers, they may choose to respond in a way different from people who do not believe in these relationships.
1149
1150
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
foods and nutrients. In addition we plan to study relationships between background characteristics of subjects and reported nutritional values across the versions of the food frequency questionnaire. ACKNOWLEDGEMENTS This work was supported by grants from the Swedish Cancer Society. REFERENCES
5
6
7
8
9
Willett W C, Stampfer M J, Colditz G A el al. Dietary fat and the risk of breast cancer. N Engl J Med 1987; 316: 22-28. Rolmick S J, Gross C R, Garrard J, Gibson R W. A comparison of response rate, data quality, and cost in the collection of data on sexual history and personal behavior. Mail-survey approaches and in-person interview. Am J Epidemiol 1989; 129: 1052-61. Hochstim J R. A critical comparison of three strategies of collecting data from households. J Am Statist Assoc 1967; Sept: 976-89. Jobe J B, Mingay D J. Cognitive research improves questionnaires. AJPH 1989; 79: 1053-55. Smith A F, Jobe J B, Mingay D i. A cognitive Investigation of Responses to Dietary Surveys. American Statistical Association Proceedings of the Survey Research Methods. American Statistical Association, Alexandria, VA 407-12, 1989. Worsley A, Baghurst K J, Leitch D R. Social desirability response bias and dietary inventory responses. Human Nulr Appl Nutr 1984; 38A: 29-35.
(Revised version received June 1992)
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
' Willett W C, Sampson L, Stampfer M J « al. Reproducibility and validity of semiquantitative food frequency questionnaire. Am J Epidemiol 1985; 122: 51-65. 1 Pietinen P, Hartman A M, Haapa E et al. Reproducibitity and validity of dietary assessment instruments. II. A qualitative food frequency questionnaire. Am J Epidemiol 1988; 128: 667-76. 3 Farrow D C, Davis S. Diet and risk of pancreatic cancer in men. Am J Epidemiol 1990; 132: 423-31.
4
International Journal of Epidemiology © International Epidemiological Association 1992
Effects of Different Designs and Extension of a Food Frequency Questionnaire on Response Rate, Completeness of Data and Food Frequency Responses Kuskowska-Wolk A (Cancer Epidemiology Unit, University Hospital, S-751 85 Uppsala, Sweden), Holte S, Ohlander E-M, Bruce A, Holmberg L, Adami H-0 and BergstromR. Effects of different designs and extension of a food frequency questionnaire on response rate, completeness of data and food frequency responses. International Journal of Epidemiology 1992; 21: 1144-1150. The authors studied how the introduction of several modifications to a basic food frequency questionnaire can influence the results of dietary surveys. Modifications covered eight combinations based on three levels: increasing versus decreasing order of frequency categories; questionnaires without versus with questions about portion sizes, and questionnaires without versus with extra non-dietary questions. The sample included 6783 women between the ages of 40 and 70 years who took part in mammography screening. The women were randomly assigned to one of the eight study groups. All of the women in each group received one of the eight differentiy modified questionnaires. The forms extended in length by extra non-dietary questions and portion size categories resulted in a 20% higher total nonresponse compared to the shorter basic form. Partial non-response was significantly lower for all four questionnaire types that included portion sizes. When portion sizes were included in the questionnaire, the reported mean frequency of consumption was significantly reduced for fat (-10 times per month), milk (-6), bread (-5), vegetables (-2) and fish (-0.4). The decreasing order of response to the frequency categories was associated with a statistically significant increase in the frequency responses for bread (2.6 times per month), vegetables (2) and fish (0.6). These data provide evidence that the design and extension of food frequency questionnaires influence the results of dietary studies.
In recent years, substantial development, modification, and evaluation of food frequency questionnaires have taken place, as well as some studies on the validity and reproducibility of such questionnaires.IJ However, other specific characteristics of selfadministered food frequency questionnaires do not seem to have been studied, although investigators suspect that a questionnaire design and extension can influence the collected information. Our aim was to determine to what degree the design and extension of a questionnaire can influence the response rate, completeness of information and food frequency responses in dietary surveys. By making different modifications to a food frequency questionnaire, i.e. using an increasing versus a decreasing order of frequency categories, and adding questions about portion sizes, we studied their impact on the food frequency results. The influence of an increased number
Food frequency questionnaires play a key role in nutritional epidemiology. Because data collection using this method, compared to others, is relatively simple and inexpensive, investigators who conduct large-scale epidemiological studies find such questionnaires particularly attractive. In contrast to other more expensive methods applied in dietary surveys, the respondents can complete the questionnaire without the assistance of a skilled professional. A better understanding of how the design of food frequency questionnaires can influence the results of nutritional studies may help in further developing dietary survey methods and in interpreting the results of such surveys. • Cancer Epidemiology Unit, University Hospital, S-751 85 Uppsala, Sweden. • • Department of Statistics, Uppsala University, Sweden. t The National Food Administration, Uppsala, Sweden. t Department of Surgery, University Hospital, Uppsala, Sweden.
1144
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
A KUSKOWSKA-WOLK,* S HOLTE." E M OHLANDER/A BRUCE.1 L HOLMBERG/t H-O ADAMI* AND R BERGSTROM*•••
1145
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE
MATERIALS AND METHODS Study Design and Subjects The target population, which was invited for mammographic screening, consisted of all 40-70 year old female residents of two different counties in Uppsala Health Care Region. The study cohort consisted of 6783 subjects from two medium-size towns, taken on a consecutive basis according to when they received their invitation for mammography over the period 13 October 1986 to 20 March 1987. The sample was randomly divided into eight groups. During November or December 1987, each woman received by mail one of eight differently designed food frequency questionnaires. This delay was unavoidable due to administrative reasons. A second copy of the questionnaire was sent as a reminder to those women who had not answered the first questionnaire within 5 weeks. Food Frequency Questionnaire (Table 1) The food frequency questionnaire included 60 foods and food groups, and was designed with the objective of categorizing individuals by intake of selected foods and nutrients over the previous 6-month period. For each food item, eight frequency categories were defined by intake: never/seldom; 1-3 times per month; once a week; 2-3 times a week; 4-6 times a week; once a day; 2-3 times a day; and 4 or more times a day. The first type of questionnaire (IF)—the basic six-page form—included questions on age, marital status, education, parity, occurrence of breast cancer in first-level relatives (mother, sister, daughter), body
Statistical Methods Before a statistical analysis was performed, the following values (frequency per month) were assigned to the eight increasing (I) response categories: from 0.5 (never/seldom) through 2, 4, 10, 20, 30, 75, up to 120 (4 times a day or more), respectively. Moreover, most foods covered in the questionnaire were divided into
TABLE 1 Description of differences in design and extension of eight types of food frequency questionnaire Extension
Design Increasing frequency categories (I) Decreasing frequency categories (D)
Frequency
Frequency + portion size
Frequency + extra questions
(F)
(F + P)
(F + Q)
Frequency + portion size + extra questions (F + P + Q)
IF (basic form) n = 671'
IFP
IFQ
IFPQ
n - 651
n = 653
n = 651
DF
DFP
DFQ
DFPQ
n - 687
n - 652
n = 616
n = 620
* Number of analysed questionnaires in each group
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
weight, height, and diet supplements. Thereafter, some qualitative questions were presented about the usual type of diet (e.g. omnivorous, vegetarian etc.), type of fat spread on sandwiches and type of fat used in food preparation. The 60-item food frequency section followed with the eight frequency categories in increasing (I) order as listed above. To the second type (IFP) of questionnaire, a column was added which included portion sizes (P)—predefined as small, medium or large—and in some cases open-ended response possibilities for quantities of, e.g. slices of bread, glasses of milk, numbers of apples, oranges, etc. The third type (IFQ), consisted of the basic form plus one page with four questions (Q) on cigarette smoking, use of oral contraceptives and hormone replacement therapy during menopause. The fourth type (IFPQ), included both the column with portion sizes (P) and the page with extra questions (Q). For each of these four versions of the questionnaire with increasing (I) frequencies of food consumption, a parallel form with decreasing (D) frequencies was produced. Thus, eight different types of questionnaire were used covering combinations on three levels (F—increasing versus decreasing frequencies, P—without portion sizes versus with portion sizes, Q—without extra questions versus with extra questions), i.e. a construction with 2 x 2 x 2 = 8 possibilities (Table 1).
of non-dietary questions on the response rate was also evaluated, and possible interactions among all these modifications were examined.
1146
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
RESULTS A comparison of the eight study groups with respect to characteristics of the subjects that might influence answers about food consumption (age, education, relative body weight) revealed a very similar distribution of these variables among the groups (Table 2). Mean age for the study population was 55.5 ± 9.1 years, and mean body mass index (BMI = weight/ height2 ± standard deviation) based on self-reported data was 24.4 ± 3.6 kg/m 2 . Response Rate Overall, 5.7% of the women directly expressed their unwillingness to participate in the study, whereas 17% did not return the questionnaire even after a reminder (Table 3). The most extended versions of the questionnaire (IFPQ, DFPQ) led to a response rate that was about 4-6% lower than for the simplest questionnaires. In a logistic regression model with the three main factors, the addition of extra questions (factor Q) entailed a significant 13% increased risk of non-response (odds ratio [OR] = 1.13, 95% confidence interval [CI]: 1.02-1.27), while the inclusion of portion size (factor P) was not significantly associated with the nonresponse rate (OR = 1.10, 95% CI : 0.99-1.24). A model with just an interaction term between extra questions and portion sizes (Q x P) gave an OR of 1.20(95% CI : 1.06-1.37), i.e. a 'relative risk' of nonresponse for questionnaires including both portion sizes and extra questions that was 20% higher than for shorter questionnaires without both of these features. Completeness of Responses Partial non-response within the questionnaire depended directly on both the kind of question and the food item. The lowest non-response was found for such qualitative questions as type of diet (0.4%), type of fat
TABLE 2 Sododemographic characteristics of study subjects by type of the questionnaire: mean (standard deviation [SD]), age and body mass index (BMI}, and percentage distribution of education level Education, Vt Questionnaire type
Age, years mean (SD)
IF
54.9 55.4 55.8 55.8 54.9 55.2 55.2 55.6
DF IFP DFP IFQ DFQ IFPQ DFPQ
(9.3) (8.8) (9.2) (9.3) (8.9) (9.2) (8.7) (8,9)
BMI, kg/m 2 mean (SD) 24.2 24.1 24.4 24.6 24.5 24.4 24.3 24.2
(3.7) (3.7) (3.8) (3.6) (3.6) (3.6) (3.8) (3.8)
< 9 years
9-12 years
>12 years
52.3 52.6 53.6 54.0 54.4 55.4 53.1 53.3
29.8 28.9 29.4 30.7 28.4 28.9 28.5 29.2
17.9 18.5 17.0 15.3 17.2 15.7 18.4 17.5
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
11 groups to give a clearer picture of the effect of the questionnaire type on the reported frequency of consumption. The effect of the questionnaire type on the response rate was tested by logistic regression analyses. Both simple models with just the basic factors (F, P, Q) and models which in addition included interaction terms between the factors were analysed. The partial nonresponse was analysed by a one-way analysis of variance where the eight different questionnaire versions were compared, as well as by three-way analysis of variance where the factorial design of the experiment was used. The effect of the questionnaire type on the reported frequency of consumption of different food groups (measured as frequencies per month) was analysed in two ways. First, by one-way analysis of variance and Kruskal-Wallis non-parametric tests in simple comparisons between the eight groups. Secondly, by threeway analysis of variance (which can also be formulated as a regression analysis based on dummy variables) that utilized the factorial design of the experiment. In the parametric analyses, variables were analysed both in original and logarithmic forms. The analysis of variance (regression analysis) formally requires normal distributions. For many food items the frequencies are skewed and/or kurtotic. To avoid the difficulties caused by this, we mainly rely on the results from the analyses based on log-transformed values in the case of parametric methods. The results of these parametric analyses were similar to those obtained by use of the Kruskall-Wallis test, which makes no distributional assumptions. In the simple presentation of results we use ordinary means and standard deviations for ease of interpretation, although a case could certainly be made for using medians or geometric means and some measure of dispersion based on quartiles.
1147
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE TABLE 3 Influence of the type of the food frequency questionnaire on response rates Questionnaire type
IF DF IFP DFP IFQ DFQ IFPQ DFPQ Total
Questionnaires sent out n 848 884* 842 843 848 805" 865" 848 6783
Refusal
Non-response
n
%
n
Exclusion6 n
%
Questionnaires analysed n
34 62 47 46 38 40 45 74
4.0 7.0 5.6 5.5 4.5 5.0 5.2 8.7
142 131 137 139 148 143 165 147
16.7 14.8 16.3 16.5 17.5 17.8 19.1 17.3
1 4 7 6 9 6 4 7
0.1 0.5 0.8 0.7 1.1 0.7 0.5 0.8
671 687 651 652 653 616 651 620
79.1 77.7 77.3 77.3 77.0 76.5 75.3 73.1
386
5.7
1152
17.0
44
0.6
5201
76.7
used in food preparation (0.6%), mother's breast cancer (0.8%) and education (0.9%). The highest partial non-response shown was for medium-fat milk (37.0%), low fat milk (35.9%), whole milk (35.6%), and for butter and margarine spread on sandwiches (27.6% and 21.7%, respectively). In order to find reasons for the high rate of nonresponse to questions about milk, butter and margarine, 28 women who represented a random sample were interviewed in person. All of these women explained that they thought it sufficient only to fill in the kind of milk or sandwich fat-spread they usually consumed and that their lack of response to frequency implied 'never/seldom'. Table 4 illustrates the total of partial non-responses for all food items. Compared with the reference form IF, there was a significantly lower number of partial non-responses (on average three responses more) on those four questionnaire types that included portion sizes.
TABLE 4 Partial non-response to questions about frequency of consumption of 60 food items for the eight questionnaire types
Food Frequency Responses A comparison of the frequency distribution of food consumption among the eight questionnaire versions revealed differences in frequency responses for some food groups. Table 5 shows the mean (±SE) of frequencies per month for 11 food groups. Maximum absolute differences in mean values among the eight groups of women vary from 0.9 times per month (potatoes), 1.1 (fish), 1.3 (alcohol), 2.2 (cereals/pasta), 3.8 (meat) to 6.1 (fruits), 6.7 (sugar/sweets/cakes), 7.1 (vegetables),
and 10.0 for milk, 10.8 for bread, and 13.2 for fat spread on sandwiches. Relative differences expressed as a percentage of the lowest mean values are from 3.9% (potatoes) to 11.7% (meat), 12.3% (fruits), 13.6% (bread), 13.9% (fish), 14.8% (vegetables), and 23.8% for fat spread on sandwiches and 24.8% for milk. A Kruskal-Wallis test comparing the eight questionnaire types indicated that these differences were significant for all but four (potatoes, cereals/pasta, sugar/sweets/cakes and alcohol) of the 11 food
Number of food iitems without response Questionnaire type IF DF IFP DFP IFQ DFQ IFPQ DFPQ
Questionnaires mean SD median mode with mode, V* F-value" 8.7 8.3 9.3 9.2 6.0 10.7 5.4 9.6 9.3 8.4 10.1 9.9 5.4 8.7 7.0 11.2
6 6 1 2 6 7 2 2
3 4 0 0 4 4 0 0
15 13 26 21 14 13 28 29
_b
1.69 25.74 37.82 1.54 7.91 38.64 10.11
° Results from a follow-up analysis in connection with the one-way analysis of variance b The tests compare each of the remaining seven questionnaire types with the basic questionntire (IF); critical F-value = 3.84, higher F-values statistically significant
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
" Inadvertently some individuals did not receive the correct type of the questionnaire; this caused a slight difference in the number of individuals per group. b The exclusion group consisted of people not belonging to the target population, i.e. younger or older than 40-70 years (n = 8) and people with unreliable responses (n = 36). Responses were considered as unrelialbe when reported frequency of consumption exceeded 2-3 times per day for some food items (e.g. potatoes, rice, pasta, salmon, shrimps etc.).
1148
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
TABLE 5 Mean (± standard error [SE]) frequency of consumption of food items per month for 11 food groups in each of eight questionnaire types as well as test for establishing significance of differences in the means. The lowest and the highest values in each food group underlined rype of questionnaire Food groups 1
DF
868
+
67.1 48.5 23.1 49.7 49.7 23.1 VI 7 8.6 57.7
± 2.1 ± 2.0 ± 0.4 ± 1.6 ± 1.4 ± 0.7 + 08 ± 0.4 ± 2.1
68.6 48.7 23.7 52.7 52.5 25.3 34.3 8.8 57.4
9.7
± 0.4
9.6
1 R 84.9
IFQ
IFP
DFQ
+
1 8 86.5
+ 70
90.3
±?
±
2.2
± 2.3 ± 2.1 ± 0.5 ± 1.9 ± 1.9 ± 0.8 ±1 1
± 1.8 ± 0.4 ± 1.5 ± 1.5 ± 0.7 + 09 ± 0.2 ± 2.0
65.5 50.4 23.1 49.6 53.1 24.1 33.9 8.7 55.6
± 2.0 ± 2.2 ± 0.4 ± 1.4 ± 1.6 ± 0.7 ± 0.2 ± 2.0
66.2 46.4 23.7 55.1 55.8 34.8 34.8 9.0 59.4
± 0.4
9.0
± 0.4
9.6
+
08
1 79.5
±0.3
± 2.4 ± 0.4
55.4 41.5 23.4 48.0 54.1 23.7 34.3 7.9 60.7
DFP
+1
8 85.7
± 1.7 ± 2.1 ± 0.4 ±
± ± +
± ±
55.4 44.0 24.0 1.2 49.9 1.5 54.6 0.7 24.7 0 8 36.3 0.2 9.0 2.3 62.3
8.8 ± 0.4
9.0
IFPQ 1 7 79.6
+
I 2 -value b
DFPQ 1 8 82.1
+1
8
30.7
± 2.0 ± 2.0 ± 0.4 ± 1.4 ± 1.5 ± 0.7 + 09 ± 0.3 ± 2.1
56.1 40.4 23.5 48.8 54.9 23.9 33.8 7.9 57.3
± 1.8 ± 1.9 ± 0.4 ± 1.4 ± 1.7 ± 1.0 +10 ± 0.2 ± 2.0
56.3 42.9 23.5 48.8 53.4 23.8 32.5 8.6 58.0
± 1.8 + 2.1 ± 0.4 ± 1.4 ± 1.7 ± 0.8 + 08 ± 0.3 ± 2.2
39.9 26.9 7.0 29.9 22.6 8.5 15.9 26.4 8.3
± 0.4
8.4
± 0.4
9.7
± 0.4
6.1
° Composition of food groups: 1. white, whole grain, hard bread; 2. butter, margarine; 3. milk 0.5% fat, 1.5ft fat, 3ft fat, sour milk and yoghurt 0.5% fat, 3ft fat; 4. boiled, fried and French fried potatoes; 5. carrots, beets, cabbage, tomatoes, iceberg/head lettuce, spinach and kale; 6. apples, pears, oranges, grapefruit, bananas, juice; 7. cooked oatmeal and other types of cooked breakfast cereal, com flakes and other types of cold breakfast cereal, pancakes and waffles, rice, spaghetti and noodles; 8. meat and sausage, main dishes, cut meats and sausage used on sandwiches, bacon, liver and kidneys, blood (black) pudding,liver pate, chicken; 9. salmon, mackerel, herring, sardines, tuna fish and other types of fish, shrimp, lobster; 10. cake, ice cream, sweet soup, jam, carbonated beverages, sweets, chocolate, sugar; 11. beer (1.8ft alcohol, 2.8ft and 4.5ft), wine, spirits. b Kruskal-Walhs test, values above the critical x 2 = 14.1 are significant.
groups. The three-way analysis of variance with F, P and Q as factors further clarified the effects of these factors for bread, fat, milk, vegetables and fish. Thus the addition of the column with portion sizes (P) reduced the mean monthly frequencies for these food groups by about 10 units for fat spread on sandwiches (which corresponded to -17% in relative terms), 6 units (-14%) for milk, 5 units (-6%) for bread, 2 units (-4%) for vegetables and 0.4 units (-5%) for fish. The decrease in the mean frequencies due to the factor P (portion size), measured as the regression coefficient of this factor, was very similar for different models (models that differed with respect to the inclusion/ exclusion of certain of the factors P, F and Q and interaction terms of these factors). The three-way analysis of variance also revealed that for bread, vegetables and fish—but none of the eight other food groups—the frequency responses were sensitive to whether the response categories were in increasing or decreasing order (the factor F). With the categories in decreasing order (from ^ 4 times/day to never/seldom) the average monthly frequencies were significantly increased by 2.6 units per month ( + 3%) for bread, 2 units ( + 4%) for vegetables and 0.6 units ( + 7%) for fish. Thus inclusion of portion sizes (the
factor P) and a decreasing order of response categories (the factor F) had opposite effects on the mean values of consumption frequencies for some foods. Inclusion of extra questions (the factor Q) did not have an effect for any food group. DISCUSSION The study included large groups of participants for each type of the questionnaire. Therefore, the statistical power of the analysis is great. This means that, although statistically significant, differences need not always be large enough to be of real importance. Response Rate It has been generally recognized that the response rate in a study depends on the complexity of the method used.1"4 Also, the strategy used in data collection, such as mailed questionnaire, telephone interview or inperson interview, influences the response rate. 36 To our knowledge, however, it has not been quantified just how much the design and extension of the questionnaire can influence the response rate when the same basic strategy and method are used. Our response rate for the basic questionnaire (79%) was comparable to that of a similar study (77%) done on women in
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
1. Bread 2. Fat used on sandwiches 3. Milk 4. Potatoes 5. Vegetables 6. Fruit 7. Cereal/pasta/rice 8. Meat 9. Fish 10. Sugar/sweets/ cake/ice cream, etc. 11. Alcohol (beer/wine/spirits)
IF
DESIGN OF A FOOD FREQUENCY QUESTIONNAIRE
Completeness of Responses The most striking partial non-response in our study was obtained for the three specific types of milk and two sorts of fat used on sandwiches. These questions clearly had deficiencies in their construction or in the information given about how to fill in the questionnaire. The subjects probably treated the separate questions about different types of milk as a multiple question, thus providing only one answer, similarly for the fat. In general omitted answers need not mean 'never/seldom' consumption. Missing values may also reflect an uncertainty on the part of subjects as to which response is the more appropriate. Participants might also inadvertently omit an answer. The study findings regarding partial non-response unexpectedly revealed significantly fewer blank answers for frequencies in all questionnaires that included portion sizes. It can be speculated that the questions about portion sizes for some food items make it easier for the respondent to choose answers for frequency categories or perhaps these forms are just more carefully studied. Food Frequency Responses We tested the consistency of frequency results from the eight differently designed and extended questionnaires. It should be noted that there is not necessarily a
relationship between consistency and validity. Thus, very highly consistent results from different questionnaires need not mean that these questionnaires have a high level of validity. It may also be true that there is not a relationship between rate of response to the individual questions and rate of return of the questionnaire. Findings in our study revealed that food frequency responses were sensitive to the portion sizes, i.e. if only frequencies were asked for, or if both frequencies and relative portion sizes had to be filled in. Although to a lesser extent, the frequency answers were also sensitive to whether the predefined categories were in increasing or decreasing order. However, not all food items were sensitive. Bread, vegetables and fish were the more sensitive both as regards portion sizes and increasing/decreasing frequencies. Fat and milk were sensitive only with respect to portion sizes. All other differences found among the frequency responses were no more than would be expected by chance. Our findings of statistically significant differences should have implications for the comparison between different dietary surveys and when quantitative metaanalyses of these or pooling analyses are attempted. It is noteworthy that food items identified as sensitive to the questionnaire design and extension in our study are those that are often considered as potential risk factors (fat, milk fat) or potential protectors (vegetables, fish, bread fibre) for some chronic diseases. Other recent studies have revealed that results from food frequency questionnaires can be sensitive to the cognitive context of the questions.7'8 Social desirability of response also affects different foods in different ways. Those foods that are considered as healthy, e.g. vegetables, have been found to be overestimated, while the opposite is true for 'unhealthy' foods, such as sweets.9 Food frequency questionnaires are subject to the sorts of biases that other dietary assessment methods, such as 24-hour recalls, are probably not subject to. This is true because of the long-term averaging that is involved in the subject having to recall exposures of very long periods of time and to average them in his/her head. This is qualitatively very different from asking individuals specifically what they ate the day before. There is a definite need for further studies on food frequency questionnaires in order to gain a better understanding of the type of misclassification they might cause and its implications for results of epidemiological studies of diet and disease. We plan to analyse these data further in order to determine how the design of food frequency questionnaires influences the calculation of the mean daily intake of different
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
nearly the same age groups, using a similar mailed frequency questionnaire.4 The present study revealed that addition of non-dietary questions and notably questions about relative portion sizes entailed a significantly lower response rate than the basic form. The sampling procedure used and the random allocation of study subjects to the type of questionnaire make it highly unlikely that any selection bias depending on sociodemographic variables could have occurred. Mean age, body mass index and distribution of type of education were similar among participants in the eight groups. A more difficult question, which has not been studied so far, is whether or not, with respect to the pattern of participation (or completeness of response), there is an interaction between the design and extension of the questionnaire and any type of disease. A questionnaire which, owing to its design, could introduce a selection bias (or differential misclassification) might invalidate a case-control study of diet as a risk factor. This also might invalidate even follow-up studies because people may believe in certain health-disease relationships irrespective of the presence of the disease. As believers, they may choose to respond in a way different from people who do not believe in these relationships.
1149
1150
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
foods and nutrients. In addition we plan to study relationships between background characteristics of subjects and reported nutritional values across the versions of the food frequency questionnaire. ACKNOWLEDGEMENTS This work was supported by grants from the Swedish Cancer Society. REFERENCES
5
6
7
8
9
Willett W C, Stampfer M J, Colditz G A el al. Dietary fat and the risk of breast cancer. N Engl J Med 1987; 316: 22-28. Rolmick S J, Gross C R, Garrard J, Gibson R W. A comparison of response rate, data quality, and cost in the collection of data on sexual history and personal behavior. Mail-survey approaches and in-person interview. Am J Epidemiol 1989; 129: 1052-61. Hochstim J R. A critical comparison of three strategies of collecting data from households. J Am Statist Assoc 1967; Sept: 976-89. Jobe J B, Mingay D J. Cognitive research improves questionnaires. AJPH 1989; 79: 1053-55. Smith A F, Jobe J B, Mingay D i. A cognitive Investigation of Responses to Dietary Surveys. American Statistical Association Proceedings of the Survey Research Methods. American Statistical Association, Alexandria, VA 407-12, 1989. Worsley A, Baghurst K J, Leitch D R. Social desirability response bias and dietary inventory responses. Human Nulr Appl Nutr 1984; 38A: 29-35.
(Revised version received June 1992)
Downloaded from http://ije.oxfordjournals.org/ at Indiana University Library on July 13, 2015
' Willett W C, Sampson L, Stampfer M J « al. Reproducibility and validity of semiquantitative food frequency questionnaire. Am J Epidemiol 1985; 122: 51-65. 1 Pietinen P, Hartman A M, Haapa E et al. Reproducibitity and validity of dietary assessment instruments. II. A qualitative food frequency questionnaire. Am J Epidemiol 1988; 128: 667-76. 3 Farrow D C, Davis S. Diet and risk of pancreatic cancer in men. Am J Epidemiol 1990; 132: 423-31.
4
