Telephone versus face-to-face interviewing for household drug use surveys.

Subst Use Misuse Downloaded from informahealthcare.com by University of Alberta on 12/29/14 For personal use only.

The International Journal of the Addictions, 27(1), 71-91, 1992

Telephone Versus Face-to-Face Interviewing for Household Drug Use Surveys William S. Aquiline,* PhD Center for Demography & Ecology University of Wisconsin-Madison Madison, Wisconsin 53706

Abstract This research investigated the use of telephone versus face-to-face interviewing to gather data on the use of tobacco, alcohol, and illicit drugs. Telephone and personal dnrg use surveys of the 18-34year-old household population were conducted in the state of New Jersey in 1986-1987. Survey modes were compared in terms of unit and item nonresponse rates, sample coverage, and levels of self-reported drug use. Results showed that the telephone survey achieved response rates lower than the personal survey, but comparable to telephone surveys of less threatening topics. Item nonresponse to sensitive drug questions was lower by phone than with the self-administered answer sheets in the personal mode. The exclusion of households without telephones in the telephone survey is a potential source of bias, and may lead to underestimation of alcohol and drug use for minority populations. After controlling for telephone status, the telephone survey furnished significantly lower drug use estimates on

*Address requests for reprints to the author at: Center for Demography & Ecology, 4412 Social Science Building, University of Wisconsin, Madison, Wisconsin 53706. 71 Copyright 0 1992 by Marcel Dekker, Inc.

AQUILINO

12

several indicators than the personal survey, with the largest mode differences found for Blacks.


Key words.Telephone interviewing; Survey methods; Self-reported drug use Drug use surveys, including the annual National Household Survey on Drug Abuse (NHSDA), are typically conducted with face-to-face interviewing. The increasingly high costs and lengthy field periods of the personal survey have intensified the desire for more cost-efficient methods of data collection. This research investigated the feasibility of conducting household drug abuse surveys by telephone. Advances in telephone survey methodology, such as computer-assisted methods (CATI) and more efficient sampling strategies, have made telephone surveys an attractive option in many fields of inquiry. Compared to face-to-face, computerassisted telephone surveys are less expensive, have shorter field periods, and allow closer monitoring of quality control during data collection and processing. Telephone costs per interview in most applications range from one-third to one-half the costs of face-to-face interviewing (Groves and Kahn, 1979;Weeks et al., 1983). Given data of equal quality, then, the Random Digit Dial (RDD) telephone survey would be the preferred mode of conducting a national or local probability survey of drug use in the general population. Very little is known, however, about the impact of survey mode on the quality of data from household drug use surveys, or about the practical difficulties of conducting sensitive interviews concerning illicit behavior by telephone. Purpose of the Research

This research compares face-to-face and telephone surveys of tobacco, alcohol, and drug use among 18-34year-old members of the household population. The surveys were conducted in 1986-1987 in the state of New Jersey. Comparisons focus on several indicators of a survey’s success: (1) unit response rates, the survey’s success in contacting eligible households (the contact rate), and in completing interviews with eligible respondents (the cooperation rate); (2) item nonresponse, success in obtaining answers to sensitive questions versus “refusal,” “don’t know,” or “no answer” outcomes to sensitive drug use items; (3)sample coverage, bias due to the different sampling frames of RDD surveys versus the multistage area probability samples used in personal surveys; of particular relevance here is the exclusion of households without telephones from the RDD sample, and the impact of this exclusion on drug use estimates for a household population; and (4)comparability of drug use estimates from the two survey modes, including evidence for differential underreporting of sensitive or illicit behavior by

TELEPHONE VERSUS FACE-TO-FACE INTERVIEWING

73

survey mode, and the extent to which mode differencesin drug use estimates vary by respondent demographic characteristicssuch as age, race, sex, and SES.

Literature


Unit Response Rates and Nonresponse Bias

Response rate differences between telephone and personal modes are well documented for surveys on topics other than drug use. Telephone rates typically are 5 to 15 points lower than personal rates @e Leeuw and Van Der Zouwen, 1988). Mode comparisons for the National Health Interview Survey yielded response rates of 80% by telephone, 94% in person (Groveset al., 1987). Groves and Kahn (1979) reported response rates of 75% in person, and from 59 to 70% in a national RDD survey. Mode differences in the demographic profile of nonrespondents are a known source of bias. Telephone nonresponse occurs disproportionately among minorities, the less affluent, less educated, and among older respondents (Freeman et al., 1982; Groves et al., 1987; O’Neil, 1979). Blacks tend to be disproportionatelyinaccessible in telephone surveys (Weaver et al., 1975), especially younger, lower income, and maIe Black respondents. The largest sourceof mode differencesin nonresponse is in screening,the survey’s successin contactingeligiblehouseholds.The number of persons who r e b to provide screeninginformation (needed for eligibilitydeterminationand respondent selection)tends to be higher by telephone than in-person (Grovesand Lyberg, 1988). Additionally, the number of sample units never contacted is higher by phone due to sampled numbers that result in busy signals, answering machines, or “no answer” during the entire data collection period (Sebold, 1988). No demographic or drug use informationis availableon these sample units, nor is it possible to gather such information. A central question for drug use research, then, is whether the response rate differential between the personal and telephone modes would be greater for surveys of illicit drug use than for surveys of less sensitive topics. Item Nonrespome There is conflicting evidence concerning survey mode differences in item nonresponse. Most evidence, however, on rates of missing data favors the in-person interview. Several comparisons of survey modes have reported slightly higher rates of missing data by telephone than in face-to-face interviewing (Grovesand Kahn, 1979; Aneshensel et al., 1982; b a n d e r and Burton, 1976). Others have shown that interviewer-administeredpersonal and telephone surveys are equiva-


14

AQUILINO

lent with regards to missing data,and that both modes are superior to mail or selfadministered surveys (Dillman, 1978; Hochstim, 1967). The survey literature strongly suggests that nonresponse rates vary with item sensitivity. Income is held to be the most sensitive or threatening question in telephone interviewing.Nonresponse rates for income, usually about 10% in personal surveys, often exceed 20% by telephone (Groves and Kahn, 1979; Jordan et al., 1980). The question investigated in this research was whether the perceived sensitivity of drug use items would produce mode effects similar to those for income, i..e., higher item nonresponse by telephone than face-to-face.

Sample Coverage Bias The RDD sampling frame excludes the approximately 7% of United Sates households without telephone service (Thornberry and Massey, 1988). The demographic correlates of nontelephone status make this a potentially serious bias for RDD drug use surveys. Respondents from nontelephone households are more likely than those with telephones to be non-White, of lower income and education, and to be never married, divorced, or separated (Thornberry and Massey, 1988; Tull and Albaum, 1977). Income is the strongest correlate of telephone status; nearly 30%of families with incomes below $5,000 in 1986were without telephone service (Thornberry and Massey, 1988). The impact of sample coverage differences on prevalence estimates of alcohol and drug use is not known.One goal of this paper is to describe drug use of persons living in nontelephone households, and the manner in which drug use in nontelephone households differs from the population as a whole.

Comparability of Estimates Without an external criterion, the validity of self-reported drug use is difficult to determine. Household surveys of the general population provide no external records or sources with which to validate respondents’ reports. In this situation, researchers typically assume that drinking and illicit drug use represent socially undesirable behaviors (Mensch and Kandel, 1988). If respondents feel threatened by interview questions on these topics, one would expect underreporting of drug use to be the largest threat tothe validity of drug use surveys (Bradburn et al., 1978). In mode comparisons, then, the direction of effects does provide a clue as to the relative validity of estimates from each mode. If underreporting is the largest threat to validity, social desirability bias is assumed to be stronger in the mode which furnishes the lower estimate of the undesirable or illegal behavior (Biemer, 1988). There have been relatively few empirical studies of mode effects on response tendencies. The question of which survey design most effectively reduces social desirability bias has not been resolved. Although the validity of telephone surveys


TELEPHONE VBRSUS FACE-TO-FACEINTERVIEWING

IS

for nonthreatening topics has been demonstrated (Rogers, 1976;Wiseman, 1972; Herzog et al., 1983; Henog and Rodgers, 1988; Groves et al., 1987; Siemiatycki, 1979), studies involving more sensitive topics have produced decidedly mixed results. Several studiessupport the validity of telephone methods for sensitivetopics. Hochstim (1967) found the proportion of women admitting alcohol use higher by telephonethan in-person. Mangione et al. (1982) reported significant mode differences on only one of 14 measures of drinking. In British surveys of alcohol use (Sykes and Collins, 1988), the telephone elicited less social desirability bias for sensitivequestionssuch as drunkenness; the self-reportedquantity of alcohol consumed was 10%higher by phone thanin-person. Kormendi (1988) reported higher nonresponseto income questionsby phone in Danish surveys, but no mode differences in amount of income reported. Other studies have cast doubt on the validity of telephone estimates. Refusal rates for sensitive questions tend to be higher in telephone than in personal interviewing (Groves and Kahn,1979). Telephone respondents were less likely to report psychiatric symptoms and depression than those inktviewed face-to-face (Henson et al., 1978), and had significantlyhigher social desirabilityscoreson the Crowne-Marlowe scales. Employeessurveyed by telephonewere less willing than those personally interviewed to reveal sensitive informationsuch as unlawful union campaignpracticesand how they intended to vote in union elections (Herman, 1977). College students reported less illicit drug use when interviewed by telephone than face-to-face (Johnson et al., 1989). In a meta-analysis of mode comparisons over the last four decades, De Leeuw and Van Der Zouwen (1988) concluded that the telephone survey was not as effective as personal interviewing in asking sensitive questions and evidenced a higher social desirability bias. Mode comparison studieshave failed to consider the possible differentialimpact of survey mode among population subgroups. The magnitude of mode effects may vary by respondents’ age, sex, race/ethnicity, urbanicity, or social class. This research examinesnot only mode effectsfor the total samples, but testsfor interactions between respondent characteristics(such as age, sex, race, and SES)and survey mode in estimating drug use for the population.

METHODS Data collection methods were basedon the 1985 National Household Survey on Drug Abuse. Severalstate governmentshave undertaken drug use surveysusing NHSDA methodology. One such survey, conducted in 1986-1987 for the state of New Jersey, provided the face-to-face interview data for this study.

76

AQUILINO

The Face-to-Face Survey


Questionnaire Design

The New Jersey face-to-face survey used the identical survey instruments developed for the 1985 NHSDA and replicated the data collection procedures of the NHSDA exactly. The 45-60 minute interview covered the use of tobacco, alcohol, marijuana, cocaine (crack was added to the state survey), opiates, hallucinogens, the nonmedical use of prescription drugs, and the health and behavioral consequences of drug use. Answers to drug use questions (except for tobacco) were recorded by respondents on self-administered answer sheets. These answer sheets were sealed in an envelope in the respondent's presence upon completion of the interview. No names were recorded on the questionnaires, answer sheets, or envelopes containing the answer sheets. Sampie

Respondents (N= 1,042 completed interviews)were selected through a multistage area probability sample of the household population aged 18-34 years. All screening and interviewing was done in-person. One respondent per household was randomly selected when more than one eligible person resided in the household. The field period extended from June 1986 through January 1987.

The Telephone Survey The 1985 NHSDA questionnaire was adapted for telephone administration (see Note 1). To minimize nonresponse, the telephone interview was shortened to an average of 25 minutes. A smaller number of drug categories were included: tobacco, alcohol, marijuana, and cocaine (including crack). To ensure the success of the telephone interview, it was necessary to reformat items that used many precoded responsecategories in the self-administeredformat. In the personal interview, items asking frequency of drug use presented respondents with up to eight response categories, while income had 12 categories presented in a show card. Remembering the distinctions among eight to 12 categories represents a greater burden for respondents by telephone than in person (Miller, 1984). With too many response categories to consider at once, telephone respondents tend to use the extreme categories more so than in face-to-face interviews (Jordan et al., 1980).To reduce respondent burden, items with six or more response categories were broken into a series of subitems which recreated the original NHSDA precoded scales. The series of two or more simpler queries, each with fewer response codes, yields less extreme responding (Miller, 1984) and less item


77

nonresponse (Locander and Burton, 1976), than the longer one-step numerical response scales.


Data Collection

The Waksberg procedure (Waksberg, 1978) of RDD was used to draw the telephone sample. Interviews were completed with 2,075 respondents 18 to 65 years old. This paper reports substantiveresults for 18-34 year olds only (N= 864), the same age group available in the face-to-face survey. One respondent was randomly chosen for the interview using selection tables developed by Kish (1965). Interviews were completed from October through December 1987.

RESULTS Unit Nonresponse Rates Nonresponserates for the pemnal and telephone surveys are shown in Table 1. Two forms of nonresponse are shown: (1) the noncontact rate, or the proportion

of households where the screening interview for eligibility and respondent selection was not completed; and (2) the noninterview rate, or the proportion of selected eligible respondents not interviewed by the end of the field period. The largest response rate difference between the two survey modes was in the noncontact rate, which was much higher by telephone than face-to-face (see Note 2). Nearly all of this differencewas due to screening refusals (14.5% by telephone, 3.1% in person). Household informants reached by telephone were much more reluctant to provide ages and first names of adults living in the household than those contacted face-to-face. The rate difference is consistent with other comparative studies (De Leeuw and Van Der Zouwen, 1988), where telephonesurveys typically yield response rates 5 to 15 points lower than comparable face-to-face surveys. The differential in screening refusals is not attributable to the survey’s subject matter-at the time of the initial screening contact, informants were told that the survey concerned health-related topics. Only selected respondents were told that the interview covered drug and alcohol use. The extent of bias introduced by the differential screening nonresponse in the two modes is difficult to assess. Higher nonresponse in one mode does not necessarily imply a more biased sample (Groves and Lyberg, 1988) if respondents and nonrespondents do not differ in the substantive areas of interest. If it is true, however, that telephone nonrespondents are more likely than personal survey nonrespondents to be minorities and of lower socioeconomic status (Freeman et al., 1982; Groves et al., 1987; Weaver et al., 1975), the higher nonresponse by telephone could have serious implicationsfor the accuracy of population drug use estimates based on a household survey.

AQUILINO

78

Table 1

Unit Nonresponse Rates by Survey Mode


Personal

Number of units sampled (addresses or telephone numbers) Noncontact rates: Number of sampled units eligible for screening. Refused screening Screening not completed (other than refusa1)b Total noncontact rate Noninterview tates: Number of eligible respondents Refused interview Interview not completed (other than refusal)c Total noninterview rate ~~~

Telephone

Number ofunits

96

Number ofunits

96

4,571

N.A.

6.932

N.A.

4,295

N.A.

4,631

N.A.

140 267

3.3 6.2

670 388

14.5 8.4

407

9.5

1,058

22.8

1,317 93 182

N.A. 7.1 13.8

2,583 258 250

N.A. 10.0

275

20.9

508

9.7 19.7

_____

asample units not in universe (not eligible for screening) include vacant housing units and those outside sampled areas in the personal mode; and nonresidential, nonworking, and disconnectednumbers in the telephone mode. a t h e r forms of screening nonresponse (noncontact)include no one ever home, language barrier, inaccessible locked apartment building, and phone numbers unanswered for duration of the field period. cother forms of interview nomponse include selected respondent never home, interview not completed by end of field period, language barrier. and respondent incapacitated.

The two surveys were very similar in their success in completing interviews with selected respondents; total interview nonresponse was about 20% in both modes. Eligible respondents in both modes knew that the survey topic was alcohol and drug use; this knowledge did not lead to differentialcooperation rates. Respondents refusals were slightly higher by telephone (10%)than in person (7.1%), but this was compensated by less nonresponse from other sources. In particular, the personal survey had a much higher rate than the telephone survey of “respondent not home” on all interview attempts during the duration of the field period (1 1% personal, less than 1% telephone). It is much cheaper to make repeated phone attempts to contact elusive respondents than to send interviewers to their homes in person. In sum, response rates for a telephone survey of drug use were lower than in the personal survey due to higher rates of screening refusals by phone (14.5% tele-


79

phone, 3.3% in person). The telephone response rates, however, were no lower than rates obtained for surveys of less threatening topics that have been reported in the survey literature (Groves and Kahn, 1979; Groves et al., 1987).

Item Nonresponse Subst Use Misuse Downloaded from informahealthcare.com by University of Alberta on 12/29/14 For personal use only.

Drug Use Items

Levels of nonresponse (refusals, no answer) to drug use items are displayed in Table 2. There were low levels of item nonre-sponsein both modes. All but one of the individual drug use items had less than 3% missing data in either mode. Item nonresponse was generally higher in the personal mode; 12% of personal survey respondents had missing data on at least one of the 2 1 drug use items analyzed, compared to just 3% by telephone. Mode differences in missing drug use data can be attributed to the method of administering the items. The self-administered format for asking alcohol, marijuana, and cocaine questions in the personal mode yielded significantly higher missing data levels than the interviewer-administered telephone items (significant differences on seven of 16 items). The nonresponse levels of the two modes are virtually identical for tobacco use, where items were interviewer-administeredin both surveys. It appears, then, that the use of self-administeredanswer sheets in the personal mode results in slightly but significantlyhigher levels of item nonresponse. Having trained interviewersadminister drug use questions minimizes item nonresponse in either mode. Logit models predicting the likelihoodof having one or more missing drug use items (versus no missing items are shown in Table 3). The self-administered format in the personal mode was associated with a significantly higher item nonresponse net of respondent demographiccharacteristics.There was but one significant interaction between survey mode and other predictors in the model. The significant sex-by-mode interaction indicated that the mode difference in drug item nonresponsewas larger for men (1.6% telephone, 14.3% personal) than for women (4.2% telephone, 10.3% personal). Men were either less careful than women in completingthe answer sheets or less willing to reveal their drug use in the personal mode. Income The personal income item was interviewer-administered in both modes. Personal survey respondents selected an income category (by letter) from a show card with 12 income levels ranging from none to over $50,000. Telephone respondents answered a series of three subitems that determined their income level by succes-

AQUILINO

80

Table 2

Percent of Cases with Missing Drug Use Data by Survey Mode 96 M i s s i d a t a


Item

Telephone

Personal

XZ

Percent with one or more missing items

3.1

12.0

50.65*

Tobacco (cigarette smoking): Age first tried a cigarette Smoked 5 or more packs lifetime Mmt recent cigarette Number of cigarettes smoked per day Number of years smoked daily

0.0 0.2 0.3 0.2 0.2

0.1 0.2 0.0 0.2

N.S. N.S. N.S. N.S. N.S.

Alcohol: Age at fmt drink Most recent drink Number of drinking days in past month Usual number of drinks per day Number of days had 5 or more drinks Frequency of getting drunk in past year

0.1 0.1 0.1 0.0 0.1 0.7

0.7

Marijuana use: Age fust tried marijuana Number of times used, lifetime Most recent use Number of days used, past month Frequency of use, past year

0.3 0.1 0.5 0.5 0.5

1 0.5 0.8 3.2 2.5

0.5 0.6

0.8 0.2

0.5

0.4 1.7 1.7

Cocaine use: Age fust tried cocaine Number of times used, lifetime Most recent use Number of days used, past month Frequency of use, past year

*p < .05. **p < .01. ***, < .nOl.

0.6 0.7

0.4

0.3

N.S. N.S.

2.6 1.4 2.3

20.00*** 12.54*** 17.46***

0.5

N.S.

.o

N.S.

N.S. N.S. 18.14*** 12.59***

N.S. N.S. N.S. 5.23* 4.05*

TELEPHONE VERSUS FACE-TO-FACEINTERVIEWING

81

Table 3 Logit Models for item Nonresponse to Drug Use and Income=

Missing 1 or more drug use items


Independent variable Sex (male)

Coefficient

.40

(SE)

Missing income Coefficient

(SE)

.13

(.27)

Age (in months)

-.01

.01

(.01)

Education (years completed)

-.I1

.a

(.M)

Respondent income

-.04

N.A.

Income missing

.72

N.A.

Currently enrolled in school

.02

Marital status (versus married): Separated/divorced Never married Employment status (versus employed): Unemployed Not in labor force Race (versus White): Black Other minority Survey mode (telephone) Sex/mode interaction (maleltelephone) Intercept

-.47

(.43)

-.05 -.05

-.81

(.62)

-.I5

(.31)

.52

-.51

.02 -.53

(.62) (.49)

-.01 -.15

.65 .44

(.35)+ (.37)

-.91

.66

-.17

(.27)*

N.A.

-1.31 -5.85

aFor drug use, the dependentvariable equals 1 if responses were missing to one or more. drug use items, zero otherwise (21drug use items); for income, the dependent variable equals 1 if income was missing, zero otherwise (single item). +p c .lo. *p c .05. **p c .01. ***p < .001.

sive approximation. The bracketing procedure (see Locander and Burton, 1976) reproduced the 12-point scale used in the personal mode. Although generally considered one of the most sensitive survey items, income nonresponse was low in both modes (2.2% in the face-to-face survey, 4.4% in the telephone survey). As shown in Table 3, nonresponse was significantly higher by telephonethan face-to-face,net of respondent demographic characteristics. In both modes, nonresponse was almost entirely due to respondents’ refusals to reveal any

82

AQUILINO


income information. Telephone nonrespondents typically refused to answer even the first question in the income sequence. This suggeststhat it is not the bracketing procedure or item format changes per se that lead to greater income nonresponse by telephone. Income appears to be a somewhat more sensitive question by phone than in face-to-faceinterviewing.A greater uneasiness to discuss income and other sensitive topics in telephone interviews has been noted often in survey literature (Groves and Kahn, 1979; Kormendi, 1988). Sample Coverage: The Exclusion of Households without Telephones Telephone surveys exclude from the sampling frame households without phone service. Bias from this exclusion can be evaluated in face-to-face surveys that determine respondents’telephone status. This was done in the New Jersey personal survey. About 5 % of the respondents had no phone service; most of those without phones were minorities, including 13% of Black and 15% of Hispanic respondents. High school dropouts, the unemployed, and low income respondents also were significantly less likely to have telephones in their homes (data not shown). The extent to which the nontelephone exclusion biases population estimated of drug use is unknown. Table 4 presents the weighted drug use profiles from the personal survey for respondents with and without telephones, both for the full sample and for Black respondents. Because of the small numbers of nontelephone households in the sample, the comparison of respondents with and without telephones is exploratory only. The comparison suggests that telephone status is a potential source of bias, but cannot definitively assess the magnitude or certainty of this bias. Nonparametric tests were used to compare the drug use distributions for respondents with and without telephone service. Binomial tests were performed for categorical variables; the distribution of the sample with telephones was taken as the expected values for the nontelephone sample. Kolmogorov-Smirnov tests comparing the distributions of telephone and nontelephone samples were computed for continuous variables. Because of the small number of nontelephone cases, only the largest differences reach significance. The first two columns of Table 4 show that respondents without telephones were significantly more likely than others to have smoked five packs lifetime and to be recent and current smokers. Nontelephone respondents drank alcohol on significantly more days and were more likely to have used cocaine in the past month than respondents with phones. Due to the relatively small proportion of nontelephone households, however, the exclusion of these households had very little impact on population estimates for the full sample (columns 2 and 3 in Table 4 are nearly identical).


83

Table 4

Weighted Drug Use Estimates by Telephone Status, Face-to-Face survep Full sample


No phone

N of cases (unweighted)

53

Tobacco: Smoked 5+ packs in life (I) 68** 65*** Smoked in last 12 months (96) Smoked in last 30 days (96) 61** If current smoker, smokes pack a day (96) 38 If smoked 5+ packs lifetime, number of years smoked daily 6.6

Black

With phone

Total

No phone

989

1,042

21

48 40 36

49 42 37

51

With phone

Total

139

160

72** 72* 72***

43 37 33

47 41 38

50

26

24

24

7.3

7.3

6.1

7.5

7.3

Alcohol:

Drank in past 30 days (%) Drunk 1+ times in past 12 months (96)

76

79

79

78

61

63

48

53

52

43

27

29

If drank in past 30 days: No. of drinking days No of drinks per day No of days had 5+ drinks

11.6** 3.8 3.5

6.5 3.0. 2.0

6.7 3.0 2.0

14.3+ 2.9 4.1

6.6 2.9 3.2

7.8 2.9 3.3

Marijuana: Used 1+ times in life (%) Used in last 12 months (96) Used in last 30 days (96) Used 10+ times in life (96)

49 31 19 32

63 26 15 36

63 26 16 36

65 43 33 47

66 25 21 32

66 27 22 34

Cocaine: Used 1+ times in life (96) Used in last 12 months (96) Used in last 30 days (%) Used 10+times in life (96)

31 21 14* 14

28 14 7 11

28 14 8 11

43+ 29* 24** 20

23 12 6 9

26 14 8 10

ONonparametric tests were used to compare the distributions for respondents with and without telephones. For categorical variables, binomial tests were performed; the distribution of the sample with telephoneswas taken as the expected values for the nontelephone sample. Kolmogorov-Shov tests comparing the distributions of telephone and nontelephone samples were done for continuous variables. +p < .lo. *p < .05. **p < .01. ***, < .001.


84

AQUILINO

The potential for bias appears stronger for black respondents (only Blacks are included in Table 4 since there were too few members of any other single minority group to warrant inclusion). Among Blacks, nontelephone respondents reported higher levels of use on 15 of the 18 drug use indicators in Table 4, including significantly more cigarette smoking, days of alcohol use, and past year and past month cocaine use. Excluding nontelephone respondents (comparing columns 5 and 6 in Table 4) produces slightly lower estimates of drug use for Blacks on 14 of the 18 items. Larger samples of nontelephone households are needed for a more complete evaluation of bias due to sample coverage. The results in Table 4 indicate only that the potential for bias, in the form of underestimates of drug use, exists when conducting a drug use survey by RDD telephone methods.

Comparability of Estimates-Mode Drug Use

Effects on Levels of Reported

Drug use estimates for the 18-34 year-old population of New Jersey are presented in Table 5. The effect of survey mode on estimates was assessed on logistic regression models for categorical dependent variables and OLS models for continuous dependent variables. In fitting these models, cases from both surveys were pooled, and mode of interview was entered as a categorical independent variable. Control variables in the models were age, sex, race, marital status, school enrollment, employment status, and income. Tests for two-way interactionsbetween survey mode and all other independent variables in the model were conducted. Only the race-mode interaction showed any consistent pattern of significant effects over the range of dependent variables. The regression effects presented in Table 5 are based on models including control variables, mode of interview, and a race/mode interaction term where significant. To control for sample coverage bias due to the exclusion of nontelephonehouseholds, the 53 households without telephones were dropped from the personal sample. The telephone and face-to-face surveys furnished nearly identical estimates on the five indicators of cigarette smoking. All mode effects for tobacco were nonsignificant. The two modes yield equally valid (or invalid) estimates of tobacco use for the state's 18-34 year-old population.. A number of significant mode and interaction effects were found for alcohol, marijuana, and cocaine. "he telephone survey provided significantly lower estimates than the personal survey for alcohol use in the past 30 days, getting drunk in the past year, lifetime prevalence of marijuana use, and the use of cocaine in the past 30 days. Significant race-by-mode interactions were found on five variables. Blacks interviewed by telephone reported significantly lower levels of recent drinking (number of drinking days, number of drinks per day, number of days with 5 or more drinks) and lower levels of recent and lifetime marijuana use than Blacks

TELEPHONE VERSUS FACE-TO-FACE INTERWEWINO

85

Table 5

Weighted Drug Use Estimates and Interview Mode Effects Regression effecw


Drug use estimates Personal

N of cases (unweighted)

Telephone

Mode (telephone)

Beta

(SE)

Racelmode inWCtiOll

Beta

(SE)

989

864

48 40 36

49 41 36

-.02

51

46

-.15

~17)

N.S.

7.3

7.1

-.08

(29)

N.S.

79 Drank in past 30 days (96) Drunk in past 12 months (I) 53 If drank in past 30 days:

70 45

-.48 -.30

(.ll)*** (.lo)**

N.S. N.S.

6.5 3.0 2.0

6.2 3.0 1.6

-.06 .14 -.18

(.37) -3.32 (1.19)** -.97 (.45)* (.14) (.23) -1.81 (.72)*

Marijuana: Used 1+ times in life (W) Used in last 12 months (96) Used in last 30 days (96) Used more than 10 times (96)

63 26 15 36

59 22 12 33

-.20

(.lo)*

-21 -.20 -.09

(.ll)+

Cocaine: Used 1+ times in life (96) Used in last 12 months (W) Used in last 30 days (96) Used more than 10 times (96)

28 14 7 11

29 11 5 15

.04 -.28 -.49 .33

t.11) (.15)+ (.21)*

N.S.

(.15)*

N.S.

Tobacco: Smoked 5+ packs in life (96) Smoked in last 12 months (I) Smoked in last 30 days (99) If current smoker, smokes pack a day (99) If smoked 5+ packs lifetime, number of years smoked daily

.09

.07

Alcohol:

No. of drinking days No of drinks per day No of days had 5+ drinks

(.15)

(-11)

N.S.

N.S. -1.01 (.46)* -.79 (.34)* N.S. N.S.

QRegressionestimates are based on logistic and OLS regression models controlling for age, sex, race, marital status,enrollment, employment, and income. Nontelephone households were excluded from the personal sample. A negative Beta for mode indicates that telephone estimates were lower than face-to-face estimates. A negative Beta for race/mode interaction indicatesthat the mode effect (lower telephone estimates) was larger for Blacks than for other respondents. +p < .lo. *p < .05. **p < .01. ***p < .001.

AQUILINO


86

interviewed in person. There were no mode differences on these indicators for Whites and other minorities. Telephone estimates were significantly higher than personal estimates on only one indicator: the proportion who used cocaine more than 10 times After controlling for household telephone status and for respondent demographic characteristics,the New Jersey telephone survey appears to have a greater susceptibility to drug use underreportingthan the face-to-face survey. It should be noted that all of the variables for which significant mode effects were found relied on self-administeredanswer sheets to record sensitive drug use information in the personal survey. The degree of anonymity offered by the self-administered format may be greater than that of the telephone interview, where admissions of illicit drug use must be made openly to the interviewer.

DISCUSSION From the standpoint of survey administration, the New Jersey telephone survey yielded quite positive results. Response rates for the telephone drug survey were comparable to telephone surveys of less threatening topics. The increased screening nonresponse (the noncontact rate) compared to the personal survey is typical of Random Digit Dial methodology. The higher rate of screening refusals by telephone cannot be attributed to the survey's sensitivity, since persons who answered the phone at first contact with the household were told that the survey was "health-related." Alcohol and drug use was mentioned only on first contact with selected respondents. Once households were screened and respondents selected, the telephone survey was as successful as the face-to-face survey in obtaining completed interviews. Selected telephone respondents, who h e w that the interview concerned alcohol and drug use, were no more reluctant to cooperate than those contacted in person. Item nonresponse was not a major problem in the telephone interview. The interviewer-administeredformat by telephone significantly reduced nonresponse to sensitive drug use items compared to the self-administered format used in the face-to-face survey. Refusal to reveal personal income was significantly more likely by telephonethan face-to-face;nevertheless, the income nonresponse rate by telephone (4.4%) would be considered quite acceptable in most surveys. In short, this research shows that drug surveys by telephone are "doable." Data collection is less expensive, with costs per interview about half of personal survey costs (Groves and Kahn, 1979); the data collection period is shorter, and unit and item response rates are likely to match those of less threatening telephone surveys. There are, however, sources of potential bias that must be considered in deciding the best survey mode for household drug use studies. Sample coverage is an important issue in telephone survey design. The RDD sample excludes households without phones. Although comprising only 7 % of the



87

US population (Thornberry and Massey, 1988), nontelephone households are much more likely to be found among minorities, the poor, and the less educated. Results from the New Jersey personal survey suggest that the nontelephone exclusion may lead to underestimation of current tobacco, slcohol, and cocaine use, especially among Blacks. So few White respondents were without phone service (less than 2%)that this exclusion would have no impact on drug use estimates for Whites in an RDD survey. Research with larger samples of nontelephone households is needed before the magnitude of sample coverage bias can be reliably assessed. The New Jersey results suggest that survey modes may differ in their ability to reduce response set bias due to social desirability. The telephone survey, compared to the self-administered format in the personal mode, yielded significantly lower estimates of several indicators of alcohol, marijuana, and cocaine use. Tests for interactions showed that mode differentials were especially high for Black respondents. It’s not possible to pinpoint the causes of the mode differences found in the New Jersey surveys. The origin of the observed mode differences may lie at least partially in several areas. First, there may be differential patterns of nonresponse (most likely at screening) between the survey modes. That is, nonrespondents in the telephone and personal modes may have different characteristics and different drug use patterns. Although demographic controls were introduced into the regression analyses of mode effects and the data were weighted, these controls cannot fully equate the samples or remove all sources of bias due to differential nonresponse. If survey respondents are not representative of the population (or population subgroups), no amount of weighting or statistical controls can completely eradicate the bias. Second, changes in question format (such as bracketing) required for telephone interviewing may affect responses. Although this source of bias cannot be ruled out, it should be noted that significant mode effects were obtained for items that required no change in format, including items on the frequency and quantity of alcohol use in the past month, and the lifetime prevalence of marijuana use. Third, the self-administeredanswer sheets may have maximized respondents’ perceptions of privacy and anonymity in the personal mode. Isolating the impact of the self-administered format on responses is a crucial question in evaluating the influence of interview mcde on the validity of drug use estimates. Data from the New Jersey surveys cannot answer the question of why Blacks might display more sensitivity than other respondents to variation in survey design. Without additional research, the origins of these effects are a matter for speculation. Studies of youthful populations have found Blacks and Hispanics more likely than Whites to underreport marijuana and cocaine use (Mensch and Kandel, 1988), and Black youth more likely to conceal their own criminal behavior (Hmdelang et al., 1981). It’s possible that minorities remain more suspicious than Whites of gov-


88

AQUILINO

ernment-sponsoredsurveys, especially those concerning illicit behavior. The traditional guarantees of confidentiality offered in survey research may inspire lessconfidence among minorities than Whites. If so, the (justifiably) heightened suspicions of minority respondents could foster increased reactivity to variations in survey methodology. The significant race-by-mode interactions point to an area of survey methodology that needs a great deal more research. The possibility that various subgroups of respondents, according to race, ethnicity, social class, or other criteria, may be differentially sensitive to alterations in survey design has not been adequately researched. This is true not only for drug studies but for research on survey methodology in general.

Implications Results of the New Jersey surveys have important implications for data collection policies and the use of household drug use data. In surveys that measure change in drug use over time, such as the annual NHSDA or longitudinal surveys, a change in survey mode may introduce systematic error into the estimation of trends, especially for minority populations. A switch from personal to telephone methods should not be made without additional research which will reveal the degree of bias associated with a change in mode. Additionally, when telephone surveys are used to estimate drug use for minority subpopulations, researchers may need to make special efforts to obtain interviews in a subsample of nontelephone households to correct for bias due to the nontelephone exclusion in RDD sampling. Policy-makers and other users of survey data on drug use need to be sensitive to the possibility that the data collection method itself has affected the results. Caution in the use of drug data for policymaking is needed, especially when population subgroups have been systematically excluded from the sample, when unit or item nonresponse rates are high, and when the interviewing mode has not afforded respondents the maximum degree of privacy in answering sensitive questions. Evaluating the efficacy of telephone interviewing for particular research applications is not clear-cut. Although the survey literature provides generally accepted standards for assessing response rates (a successful survey will interview 70%or more of eligible respondents), determination of overall data quality is complex. General population surveys provide no external criteria with which to validate self-reported substance use. A great deal more comparative research, involving the experimental manipulation of interviewing mode, is needed to evaluate method effects and their implications for drug use research.


89


ACKNOWLEDGMENTS The study was supported by Grant R01-DAM280 from the National Institute on Drug Abuse, Beth&, Maryland, and by PHS National Research Service Award AG00129-03 from the National Institute on Aging. Surveys were conducted by the Institute for Survey Research, Temple University. The personal interview data were collected by the New Jersey Household Survey on Drug Abuse, b d e d by the Department of Public Health, State of New Jersey.

NOTES Copies of the telephone version of the 1985 NHSDA used in thisresearch, and a report detailing sample design and survey procedures, are. available upon request from the author. 2. Telephone numbers that ring but remain unanswered for the duration of the data collection period create a problem for computing telephone nonresponse rates (Groves and Kahn, 1979). A minimum of 10 calls were ma& to these chronically unanswered numbers before designating "no answer" as final. The portion of these unanswered numbers included in response rate calculations greatly affects the achieved screening rates. Cummings (1979) obtained actual dispositions of unamwerednumbers in an RDD sample and found that only 20% were working household numbers; proportions of working household numbers as high as 40 and 60% have also been reported in the literature (Groves and Lyberg, 1988). To calculate the telephone noncontact rates reported in Table 1.40% of the unanswered numbers were assumed to represent working household numbers, and these were added to the number of nonscreened households in the telephone survey. 1.

REFERENCES ANESHENSEL,C. S., FRERICHS, R. R., CLARK, V. A., andYOKOPENIC, P. A. (1982). Telephone versus in-person surveys of community health status. Am. J. Public Health 72: 1017-1021. BIEMER, P. P. (1988). Measuring data quality. In R. M. Groves, P. BIemer, L. Lyberg, J. Massey, W. Nichols, and J. Waksberg (eds.), Telephone Survey Methodology.New York Wiley. BRADBURN, N. M., SUDMAN, S., BLAIR, E.. and STOCKING, C. (1978). Question threat and response bias Public Opin Q.42: 221-234. CUMMINGS,K. M. (1979). Random digit dialing: A sampling technique for telephone surveys Public Opin. Q. 43: 233-244. DE LEEUW, E., and VAN DER ZOUWEN, J. (1988). Data quality in telephone and face to face surveys: A comparative meta-analysis. In R. M. Groves et al. (eds.), Telephone Survey Methodology. New York: Wiley. DILLMAN, D. A. (1978). Mail and Telephone Surveys: The Total Design Method. New York Wiley. FREEMAN, H., KIECOLT, K., NICHOLLS, W., and SHANKS, J. M. (1982). Telephone sampling bias in surveying disability. Public Opin Q.46: 392-407. GROVES, R. M.,and KAHN, R. L. (1979). Surveys by Telephone:A National Comparisonwirh Personal Interviews. New York Academic Press. GROVES, R. M., and LYBERG, L. E. (1988). An overview of nonresponse issues in telephone surveys. In R. M. Groves et al. (eds.), Telephone Survey Methodology. New York Wiley. GROVES, R. M., MILLER, P. V., and CANNELL, C. P. (1978). Differences between the telephone and personal interview data. In 0.Thombeny (ed.), An Experimental Comparisonof Telephone and Health Interview Surveys (Vital Health Statistics. Series 2, No. 106. DHHS Pub. No. (PHS) 87-1380). Washington, D.C.: Public Health Service.

AQUILINO


90

HENSON. R., CANNELL, C. F., and ROW, A. (1978). Effects of interview mode on reporting of moods, symptoms, and need for social approval. J. SOC. Psychol. 105: 123-129. HERMAN, J. B. (1977). Mixed-mode data collection: Telephone and personal interviewing. J. Appl. Psychol. 62: 399-404. HERZOG,A. R., and RODGERS. W. L. (1988). Interviewing older adults: Mode comparison using data from a face to face survey and a telephone resurvey. Public Opin. Q. 52: 84-99. HERZOG, A. R., RODGERS. W. L., and KULKA, R. A. (1983). Interviewingolder adults: A comparison of telephone and face to face modalities Public Opin Q. 47: 405-418. HINDELANG, M. J., HIRSCHI. T., and WEIS. J. 0. (1981). Measuring Delinquency. Beverly Hills, California: Sage. HOCHSTIM, J. R. (1967). A critical comparison of three strategies of collecting data from households J. Am. Stat. Assoc. 62: 976-989. JOHNSON, T. P., HOUGLAND, J. G., and CLAYTON, R. R. (1989). Obtaining reports of sensitive behavior: A comparison of telephone and face to face interviews SOC. Sci. Q. 7 0 174-183. JORDAN, L., MARCUS, A., and REEDER, L. (1980). Response styles in telephone and household interviewing: A field experiment. Public Opin Q. 44:210-222. KISK, L. (1965). Survey Sampling. New York Wiley. KORMENDI, E. (1988). The quality of income information in telephone and face to face surveys. In R. M. Groves el al. (eds.), Telephone Survey Methodology. New York Wiley. LOCANDER,W., and BURTON, J. P.(1976). The effecl of question form on gathering income data by telephone. J. Mark Res. 13: 189-192. MANGIONE, T. W., HINGSON, R., and BARRETT, J. (1982). Collecting sensitive data: A comparison of three survcy strategies. Sociol. Metho& Res. 10: 337-346. MENSCH, B. S., and KANDEL, D. B. (1988). Underreporting substance use in a national longitudinal youth cohort: Individual and interviewer effects Public Opin. Q.52: 100-124. MILLER, P.(1984). Alternative question forms for attitude scale questions in telephone interviews. Public Opin. Q.48: 766-778. O'NEIL, M. (1979). Estimating the nonresponse bias due to refusals in telephone surveys Public Opin Q. 43: 218-232.

ROGERS, T. F. (1976). Interviewsby telephone and in person: Quality of responses and field performance. Public Opin. Q.40: 51-65. SEBOLD, J. (1988). Survey period length, unanswered numbers, and nonresponse in telephone surveys. In R. M. Groves et al. (eds.), Telephone Survey Methodology. New York Wiley. SIEMIATYCKI,J. (1979). A comparison of mail, telephone, and home interview strategies for household health surveys Am. J. Public Health 6 9 238-244. SYKES, W., and COLLINS,M. (1988). Effects of mode of interview: Experiments in the UK.In R. M. Groves et al. (eds.). Telephone Survey Methodology. New York Wiley. THORNBERRY, 0.T., and MASSEY, J. T. (1988). Trends in United States telephone coverage across time and subgroups. In R. M. Groves et al. (eds.), Telephone Survey Methodology. New York Wiley. TULL, D. S., and ALBAUM, G. (1977). Bias in random digit dialed surveys. Public Opin. Q. 41: 389-395.

WAKSBERG, J. (1978). Sampling methods for random digit dialing. 1.Am. Stat. Assoc. 73: 40-46. WEAVER, C. HOLMES, S., and GLENN, N. (1975). Some characteristicsof inaccessiblerespondents in a telephone survey. 1. Appl. Psychol. 60: 260-262. WEEKS, M.F., KULKA, R. A.. LESSLER, J. T., and WHITMORE, R. A. (1983). Personal versus telephone surveys for collectinghousehold health data at the local level. Am. J. Public Health 73: 1389- 1394.

WISEMAN, F.(1972). Methodological bias in public opinion surveys Public Opin Q. 3 6 105-108.


91


THE AUTHOR William S. Aquilino is a Research Scientist with the Center for Demography and Ecology (CDE) in the Sociology Department, University of WisconsinMadison. Prior to joining the CDE staff in 1988,he was Senior Study Directorat the institute for Survey Research, Temple University, Philadelphia, Pennsylvania. As Study Director, he was highly involved in epidemiological drug use surveys. He conducted the 1987 New Jersey Telephone Drug Survey (funded by NIDA), and assisted in survey operations for the 1985 National Household Survey on Drug Abuse and the 1986 Follow-up Survey of Drug Use among Young Men. He continues his research on survey mode effects in drug use surveys with a NIDA-funded grant that began in summer 1990. He holds a PhD in Human Development and Family Studies from Penn State University (1979).

Collecting daily self-reports of injection drug use via automated telephone interviewing.

A comparison of mail, telephone, and home interview strategies for household health surveys.

Use of telephone interviewing in health care research.

Potential use of telephone surveys for non-communicable disease surveillance in developing countries: evidence from a national household survey in Lebanon.

Correcting for the underreporting of drug use in opinion surveys.

Surveys of drug use among young people.

Issues and considerations in the use of serologic biomarkers for classifying vaccination history in household surveys.

Interviewing adolescents by telephone: is it a useful methodological strategy?

Mail versus telephone surveys on mammography utilization among women 50-75 years old.

Drug Use and Spatial Dynamics of Household Allocation.

[Potential selection bias in telephone surveys: landline and mobile phones].

Telephone Surveys Underestimate Cigarette Smoking among African-Americans.

Estimates of average energy requirements in Bangladesh: Adult Male Equivalent values for use in analyzing household consumption and expenditure surveys.

Monitoring Change in Child Mortality through Household Surveys.

A Multi-country Study of the Household Willingness-to-Pay for Dengue Vaccines: Household Surveys in Vietnam, Thailand, and Colombia.

Motivational Interviewing Fidelity in a Community Corrections Setting: Treatment Initiation and Subsequent Drug Use.

Factors associated with the use of mosquito bed nets: results from two cross-sectional household surveys in Zambézia Province, Mozambique.

The effectiveness of telephone counselling in the treatment of illicit drug and alcohol use concerns.

What Do Our Respondents Think We're Asking? Using Cognitive Interviewing to Improve Medical Education Surveys.

HIV seroprevalence surveys in drug treatment centers.

Use of the telephone in consultant practice.

Letter: The defensive use of the telephone.

Drug Use during Acute Illness in Tigray Region, Northern Ethiopia: A Household Study.

[Microbiological evaluation of chemical disinfectants for household use].