Measurement of Attitudes and
Behaviors in Public Health Surveys Franqois Beland, PhD, Bngitte Maheux, MD, PhD, and Jean Lambert, PhD Introduction Attitudes and behaviors are salient factors in most of the current public health issues. Most studies measure attitudes and behaviors using a set of items included in a single closed-form questionnaire. Inasmuch as attitudes and behaviors toward the same act are examined, the wording of the items and the response formats should be nearly identical to each other.1-13 Feldman and Lynchl4 implied that a response to a question is likely to be retrieved as a basis for a subsequent response if they are presented consecutively in one questionnaire. Using similarly worded items to measure attitudes and behaviors compounds the problem. In the present paper we address the question of whether attitudes and behaviors can be measured accurately with a single questionnaire.
Methods We randomly divided a sample of 900 general practitioners into two groups. One group (A) received a questionnaire in which both their attitudes and behaviors were assessed. The other group (B) received a questionnaire measuring behaviors; on receipt of that completed questionnaire, we mailed them a second one which pertained to their attitudes. The response rate for group A was 64.3 percent. The response rate for group B was 73.2 percent. Ninety-two percent of the physicians in group B who answered the first questionnaire also completed the second questionnaire. Twenty-one attitudinal items based on a series of questions (see Appendix) and 21 similarly worded behavioral items were presented to physicians on a sixpoint rating scale. They measured practices that physicians considered appropriate to perform (attitudes) and those which they claimed to perform (behaviors) in three clinical situations: * the extent of information that a physician should provide to chronically ill patients about their illness; * the extent to which a physician should promote non-smoking behavior to patients that consult for reasons other than smoking problems;
* the extent to which a physician should discuss weight with an obese patient consulting for reasons other than obesity. The items measuring attitudes and behaviors were worded in the same manner.14 Systematic biases in measuring attitudes and behaviors were assessed by comparing the correlation coefficients between attitudes and behaviors in the two groups of physicians. The correlation coefficients were transformed into z-scores, using the Fisher Transformation, to test for the differencel1 between the association of attitudes and behaviors in the two groups of respondents. A total of 21 tests were performed for each of the comparisons between two groups. A P-plot procedure was used16 to estimate the number N of true null hypotheses. The estimate of N was then used to obtain an overall a-level which was less conservative than the usual Bonferonni level. The value of N was 6 in this study, thus the significant level was set at .05/6 = .008.
Resmlts Eight correlations between attitudes and behaviors were different in the two groups at the 0.008 level (Table 1). Pearson's correlation coefficients between attitudes and behaviors were larger by an average of .147 in the group who received one questionnaire than in the group who answered two separate questionnaires. Some differences in the item variances emerged (data not shown). Because a correlation coefficient is obtained from the division of the covariance with the variances, a small difference in variances can produce large differences in the estimated correlation coefficient. To test whether the differences in the correlations were due to differences in the variances in From the Group on Interdisciplinary Research (13land, Maheux), and the Faculty of Medicine, Universite de Montreal (all authors). Address reprint requests to Francois Beland, PhD, GRIS, Universite de Montreal, P.O. Box 6128, Station A, Montreal, Quebec H3C 3J7. This paper, submitted to the Journal June 27, 1989, was revised and accepted for publication May 18, 1990. on Health
American Journal of Public Health 103
Public Health Bniefs
£40
bm~~~~2
reported here was supported by a grant from the National Health Research and Development Program, Health and Welfare Canada. The first author was supported by a research fellowship from the National Health Research and Development Program, and the second author by the Fonds de la recherche en sante du Quebec. Editorial assistance was provided by Elena Joram.
*2~~.. .274
......75.. h It~........
ftp~~~~~~~~,4 Sun~~..........410 ~~~~~~~~~~
410
Semi! Swnt4 15 Sun Rem18 Ssmll ItuniS
-I-tem---------- 20 ...e.....
Sum2l
S.....
240~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~....... 470 4411...... £45 000'~~~~~~~~~~~~...... 84$ 404 19.~~~~~~~~~~~~~~~~~~~~~~~~~~~~......... WI US 882 .W5 117~~~~~~~~~~~~~~~~~~~~~~~~~~............. 842 .261.. ~ ~ ~ 472 ~ ~ ~ . . . . 041. .
the items, the 21 variance-covariance matrices for attitudinal and behavioral items of the two groups were compared using the Joreskog procedure.17 Only four covariances out of 21 were significantly different between the two groups, while 14 variances were significantly different. As a consequence, the differences in the estimated variances had an effect on the estimate of the correlations.
Discussion The results of our study showed that respondents to a questionnaire, wherein attitudinal items are followed by similarly worded behavioral items, retrieved their answer to the attitudinal items in order to answer questions on behavioral items. Pearsons correlations between attitudes and behaviors were larger by an average of .147 in the group who received one questionnaire than in the group who answered two separate questionnaires. Some of the differences between the correlations were accounted for by the variances. Thus, fewer covariances than correlations were affected by the data collecting procedures. However, because there were significant differences in the means on the behavioral items between the two groups (table not shown) and be-
104 American Journal of Public Health
864 AC?~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. ......
cause the computation of the covariances depends on the means, the estimates of the covariances were also biased. Psychometric methods are used in attitudinal and behavioral scale testing. More often than not, researchers use a correlation matrix rather than a covariance matrixwhen testing for reliability and studying the validity of their scales. Thus, using one or two questionnaires would yield very different reliability coefficients. The data from this study suggest that when similarly worded items are used to test for both attitudes and behaviors, two questionnaires are required. Nevertheless, the effect of features of questionnaire design should be tested before such a radical conclusion becomes conventional practice. For example, the set of behavioral items was listed immediately after the set of attitudinal items in this study. Had we introduced questions on other topics between the two sets of questions the response bias might have been lowered. Also, it may be possible to use different items to measure both or to use a set of similarly worded and a set of differently worded items. El
Acknowledgments This paper was presented at the 1987 APHA Annual Meeting in New Orleans. The research
References 1. Ajzen I, Fishbein M: Attitudinal and normative variables as predictors of specific behaviors. J Pers Soc Psychol 1973; 27:4157. 2. Ajzen I, Fishbein M: Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychol Bull 1977; 84:888-918. 3. Davidson AR, Jaccard JJ: Variables that moderate the attitude-behavior relation: Results of a longitudinal survey. J Pers Soc Psychol 1979; 37:1364-1376. 4. DeFleur ML, Westie FR: Attitude as a scientific concept. Soc Forces 1963; 42:17-31. 5. Fazco RH, Zanna MP: Attitudinal qualities relating to the strength of the attitude-behavior relationship. J Exper Soc Psychol 1978; 14:398-408. 6. Fishbein M: Factors influencing health behaviors: An analysis based on a theory of reasoned action. In Landy F (ed): Health Risk Estimation, Risk Reduction and Health Promotion: Proceedings of the 19th Annual Meeting, Society of Prospective Medicine, Quebec, 1984; 204-214. 7. Fishbein M, Jaccard JJ: Theoretical and methodological considerations in the prediction of family planning intentions and behavior. Represent Res Soc Psychol 1973; 4:37-51. 8. Heberlein TA, Black JS: Attitudinal specificity and the prediction of behavior in a field study. J Pers Soc Psychol 1976; 33:474-479. 9. Hill RJ: Attitudes and behaviors. In: Rosenberg M, Turner RH (eds): Social Psychology, Sociological Perspective. New York: Basic Books, 1981. 10. Liska AE: Emergent issues in the attitudebehavior consistency controversy. Am Sociol Rev 1974; 39:261-272. 11. Schuman H, Johnson HP: Attitudes and behaviors. Annu Rev Sociol 1976; 2:161207. 12. Weigel RH, Newman LS: Increasing attitude-behavior correspondence by broadening the scope of the behavioral research. J Pers Soc Psychol 1976; 33:793-802. 13. Weigel RH, Vernon DTA, Tognacci LN: Specificity of the attitude as a deterniinant of attitude-behavior congruence. J Pers Soc Psychol 1974; 30:724-728. 14. Feldman JM, LynchJG: Self-generatedvalidity and other effects of measurement on belief, attitudes, intention, and behavior. J Appl Psychol 1988; 73:421-435. 15. Snedecor GW, Cochran WG: Statistical Methods. Ames, Iowa: Iowa State University Press, 1967; 186. 16. SchewerT, SpjotvollE: Plotsof p-values to evaluate many tests simultaneously. Biometrika 1982; 69:493-502. 17. Joreskog KG: Analysis of covariance structures. Scand J Stat 1981; 8:65-92.
January 1991, Vol. 81, No. 1
Public Health Briefs
.........
.......................
........................ .......
....
........ %.......................
.....................
........................
...........
X.
.... ...... ..........
:::::
X
............
c................. ....
............................................ ................
X
.................
x .......... .......................
..................
.................
...........................
............................
.............................
-A.M, -A .......
... ......
...............
....
......................
................
...................
....... X
C. .. ............
...................
....................... ..
......................
............................... .... ................... ................
:.
..
:::N ...............
..........................O
.............
..........
........
........... ............. .........................
...............
..................... %'.'.'.'.'.'.'.........................
..........
......................
N. .........
..............................
.............................................
ON
January 1991, Vol. 81, No. 1
....................
...
American Journal of Public Health 105
Behaviors in Public Health Surveys Franqois Beland, PhD, Bngitte Maheux, MD, PhD, and Jean Lambert, PhD Introduction Attitudes and behaviors are salient factors in most of the current public health issues. Most studies measure attitudes and behaviors using a set of items included in a single closed-form questionnaire. Inasmuch as attitudes and behaviors toward the same act are examined, the wording of the items and the response formats should be nearly identical to each other.1-13 Feldman and Lynchl4 implied that a response to a question is likely to be retrieved as a basis for a subsequent response if they are presented consecutively in one questionnaire. Using similarly worded items to measure attitudes and behaviors compounds the problem. In the present paper we address the question of whether attitudes and behaviors can be measured accurately with a single questionnaire.
Methods We randomly divided a sample of 900 general practitioners into two groups. One group (A) received a questionnaire in which both their attitudes and behaviors were assessed. The other group (B) received a questionnaire measuring behaviors; on receipt of that completed questionnaire, we mailed them a second one which pertained to their attitudes. The response rate for group A was 64.3 percent. The response rate for group B was 73.2 percent. Ninety-two percent of the physicians in group B who answered the first questionnaire also completed the second questionnaire. Twenty-one attitudinal items based on a series of questions (see Appendix) and 21 similarly worded behavioral items were presented to physicians on a sixpoint rating scale. They measured practices that physicians considered appropriate to perform (attitudes) and those which they claimed to perform (behaviors) in three clinical situations: * the extent of information that a physician should provide to chronically ill patients about their illness; * the extent to which a physician should promote non-smoking behavior to patients that consult for reasons other than smoking problems;
* the extent to which a physician should discuss weight with an obese patient consulting for reasons other than obesity. The items measuring attitudes and behaviors were worded in the same manner.14 Systematic biases in measuring attitudes and behaviors were assessed by comparing the correlation coefficients between attitudes and behaviors in the two groups of physicians. The correlation coefficients were transformed into z-scores, using the Fisher Transformation, to test for the differencel1 between the association of attitudes and behaviors in the two groups of respondents. A total of 21 tests were performed for each of the comparisons between two groups. A P-plot procedure was used16 to estimate the number N of true null hypotheses. The estimate of N was then used to obtain an overall a-level which was less conservative than the usual Bonferonni level. The value of N was 6 in this study, thus the significant level was set at .05/6 = .008.
Resmlts Eight correlations between attitudes and behaviors were different in the two groups at the 0.008 level (Table 1). Pearson's correlation coefficients between attitudes and behaviors were larger by an average of .147 in the group who received one questionnaire than in the group who answered two separate questionnaires. Some differences in the item variances emerged (data not shown). Because a correlation coefficient is obtained from the division of the covariance with the variances, a small difference in variances can produce large differences in the estimated correlation coefficient. To test whether the differences in the correlations were due to differences in the variances in From the Group on Interdisciplinary Research (13land, Maheux), and the Faculty of Medicine, Universite de Montreal (all authors). Address reprint requests to Francois Beland, PhD, GRIS, Universite de Montreal, P.O. Box 6128, Station A, Montreal, Quebec H3C 3J7. This paper, submitted to the Journal June 27, 1989, was revised and accepted for publication May 18, 1990. on Health
American Journal of Public Health 103
Public Health Bniefs
£40
bm~~~~2
reported here was supported by a grant from the National Health Research and Development Program, Health and Welfare Canada. The first author was supported by a research fellowship from the National Health Research and Development Program, and the second author by the Fonds de la recherche en sante du Quebec. Editorial assistance was provided by Elena Joram.
*2~~.. .274
......75.. h It~........
ftp~~~~~~~~,4 Sun~~..........410 ~~~~~~~~~~
410
Semi! Swnt4 15 Sun Rem18 Ssmll ItuniS
-I-tem---------- 20 ...e.....
Sum2l
S.....
240~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~....... 470 4411...... £45 000'~~~~~~~~~~~~...... 84$ 404 19.~~~~~~~~~~~~~~~~~~~~~~~~~~~~......... WI US 882 .W5 117~~~~~~~~~~~~~~~~~~~~~~~~~~............. 842 .261.. ~ ~ ~ 472 ~ ~ ~ . . . . 041. .
the items, the 21 variance-covariance matrices for attitudinal and behavioral items of the two groups were compared using the Joreskog procedure.17 Only four covariances out of 21 were significantly different between the two groups, while 14 variances were significantly different. As a consequence, the differences in the estimated variances had an effect on the estimate of the correlations.
Discussion The results of our study showed that respondents to a questionnaire, wherein attitudinal items are followed by similarly worded behavioral items, retrieved their answer to the attitudinal items in order to answer questions on behavioral items. Pearsons correlations between attitudes and behaviors were larger by an average of .147 in the group who received one questionnaire than in the group who answered two separate questionnaires. Some of the differences between the correlations were accounted for by the variances. Thus, fewer covariances than correlations were affected by the data collecting procedures. However, because there were significant differences in the means on the behavioral items between the two groups (table not shown) and be-
104 American Journal of Public Health
864 AC?~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. ......
cause the computation of the covariances depends on the means, the estimates of the covariances were also biased. Psychometric methods are used in attitudinal and behavioral scale testing. More often than not, researchers use a correlation matrix rather than a covariance matrixwhen testing for reliability and studying the validity of their scales. Thus, using one or two questionnaires would yield very different reliability coefficients. The data from this study suggest that when similarly worded items are used to test for both attitudes and behaviors, two questionnaires are required. Nevertheless, the effect of features of questionnaire design should be tested before such a radical conclusion becomes conventional practice. For example, the set of behavioral items was listed immediately after the set of attitudinal items in this study. Had we introduced questions on other topics between the two sets of questions the response bias might have been lowered. Also, it may be possible to use different items to measure both or to use a set of similarly worded and a set of differently worded items. El
Acknowledgments This paper was presented at the 1987 APHA Annual Meeting in New Orleans. The research
References 1. Ajzen I, Fishbein M: Attitudinal and normative variables as predictors of specific behaviors. J Pers Soc Psychol 1973; 27:4157. 2. Ajzen I, Fishbein M: Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychol Bull 1977; 84:888-918. 3. Davidson AR, Jaccard JJ: Variables that moderate the attitude-behavior relation: Results of a longitudinal survey. J Pers Soc Psychol 1979; 37:1364-1376. 4. DeFleur ML, Westie FR: Attitude as a scientific concept. Soc Forces 1963; 42:17-31. 5. Fazco RH, Zanna MP: Attitudinal qualities relating to the strength of the attitude-behavior relationship. J Exper Soc Psychol 1978; 14:398-408. 6. Fishbein M: Factors influencing health behaviors: An analysis based on a theory of reasoned action. In Landy F (ed): Health Risk Estimation, Risk Reduction and Health Promotion: Proceedings of the 19th Annual Meeting, Society of Prospective Medicine, Quebec, 1984; 204-214. 7. Fishbein M, Jaccard JJ: Theoretical and methodological considerations in the prediction of family planning intentions and behavior. Represent Res Soc Psychol 1973; 4:37-51. 8. Heberlein TA, Black JS: Attitudinal specificity and the prediction of behavior in a field study. J Pers Soc Psychol 1976; 33:474-479. 9. Hill RJ: Attitudes and behaviors. In: Rosenberg M, Turner RH (eds): Social Psychology, Sociological Perspective. New York: Basic Books, 1981. 10. Liska AE: Emergent issues in the attitudebehavior consistency controversy. Am Sociol Rev 1974; 39:261-272. 11. Schuman H, Johnson HP: Attitudes and behaviors. Annu Rev Sociol 1976; 2:161207. 12. Weigel RH, Newman LS: Increasing attitude-behavior correspondence by broadening the scope of the behavioral research. J Pers Soc Psychol 1976; 33:793-802. 13. Weigel RH, Vernon DTA, Tognacci LN: Specificity of the attitude as a deterniinant of attitude-behavior congruence. J Pers Soc Psychol 1974; 30:724-728. 14. Feldman JM, LynchJG: Self-generatedvalidity and other effects of measurement on belief, attitudes, intention, and behavior. J Appl Psychol 1988; 73:421-435. 15. Snedecor GW, Cochran WG: Statistical Methods. Ames, Iowa: Iowa State University Press, 1967; 186. 16. SchewerT, SpjotvollE: Plotsof p-values to evaluate many tests simultaneously. Biometrika 1982; 69:493-502. 17. Joreskog KG: Analysis of covariance structures. Scand J Stat 1981; 8:65-92.
January 1991, Vol. 81, No. 1
Public Health Briefs
.........
.......................
........................ .......
....
........ %.......................
.....................
........................
...........
X.
.... ...... ..........
:::::
X
............
c................. ....
............................................ ................
X
.................
x .......... .......................
..................
.................
...........................
............................
.............................
-A.M, -A .......
... ......
...............
....
......................
................
...................
....... X
C. .. ............
...................
....................... ..
......................
............................... .... ................... ................
:.
..
:::N ...............
..........................O
.............
..........
........
........... ............. .........................
...............
..................... %'.'.'.'.'.'.'.........................
..........
......................
N. .........
..............................
.............................................
ON
January 1991, Vol. 81, No. 1
....................
...
American Journal of Public Health 105
