Journal of Clinical Epidemiology
-
(2013)
-
ORIGINAL ARTICLE
Prenotification but not envelope teaser increased response rates in a bulimia nervosa mental health literacy survey Si^an A. McLeana,*, Susan J. Paxtona, Robin Masseya, Jonathan M. Mondb,c, Bryan Rodgersd, Phillipa J. Haye,f a School of Psychological Science, La Trobe University, Kingsbury Drive, Melbourne, VIC 3086, Australia Research School of Psychology, The Australian National University, Barry Drive, Canberra, ACT 0200, Australia c Centre for Rural and Remote Mental Health, University of Newcastle, University Drive, Orange, NSW 2308, Australia d Australian Demographic & Social Research Institute, The Australian National University, Barry Drive, Canberra, ACT 0200, Australia e Centre for Health Research, School of Medicine, University of Western Sydney, Narellan Road, Sydney, NSW 2751, Australia f School of Medicine, James Cook University, 1 James Cook Drive, Townsville, QLD 4811, Australia b
Accepted 25 October 2013; Published online xxxx
Abstract Objectives: Understanding attitudes to mental health issues can inform public health interventions. However, low response rates may contribute to nonresponse bias. We examined the effect of a prenotification postcard and a short message (teaser) on response rates to a mailed questionnaire about bulimia nervosa ‘‘mental health literacy.’’ Study Design and Setting: Questionnaires were mailed to 3,010 adults (50.6% female and 49.4% male) aged 18e65 years. Procedures were varied according to sending prenotification before the questionnaire and the placement of a teaser on the survey envelope. Outcomes considered were response rate, response time, and cost. Results: The overall response rate was 22.0%. Significant main effects showed higher response rates for the use of prenotification, among female participants, and older participants. A significant interaction of teaser by gender indicated lower response rates for men who received the teaser but not for women. Older participants returned the questionnaire more promptly than younger participants. Femalesdbut not malesdwho received the teaser were slower to return the questionnaire. Higher response rates for participants receiving the postcard compensated for increased costs, particularly for males and older participants. Conclusion: Response rates to a mental health postal survey can be increased through the use of prenotification. Ó 2013 Elsevier Inc. All rights reserved. Keywords: Questionnaire response rates; Nonresponse bias; Prenotification; Teaser; Bulimia nervosa; Demographic groups
1. Introduction Population-based surveys are important for providing understanding of community knowledge, beliefs, and attitudes toward mental health problems, collectively referred to as mental health literacy (MHL) [1], which contributes to increased recognition of [2,3] and treatment seeking for [2] mental health problems. Furthermore, populationbased surveys provide valuable information about prevalence Conflict of interest: None. Funding: This research was funded by a grant (DP1095656) from the Australian Research Council to S.J.P., J.M.M., P.J.H., and B.R. B.R. is supported by National Health & Medical Research Council Fellowship No. 471429. * Corresponding author. Tel.: þ61-3-9749-2949; fax: þ61-3-94791956. E-mail address: [email protected] (S.A. McLean). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2013.10.013
rates for mental illnesses (eg, [4e6]) and are an efficient and cost-effective method to gain information from all sections of the community. However, low response rates to mailed questionnaires can affect the generalizability of findings because of nonresponse bias. This study investigates ways to optimize response rates to surveys of this kind. Little research is available regarding effective strategies for enhancing response rates to mental health and specifically eating disorder surveys. Responses to these surveys may be impacted by commonly held stigmatizing attitudes toward mental illness [7] and by the potentially sensitive nature of the survey questions [8]. Response rates to mailed questionnaires on mental health vary, but they are typically low, ranging from 32% to 47% [9,10]. Similarly, low response rates have been found for surveys of eating behaviors and eating disorders. The Food, Drink, Lifestyle and Wellbeing mail survey in Australia reported a response rate
2
S.A. McLean et al. / Journal of Clinical Epidemiology
What is new? Key findings Receipt of a prenotification postcard enhanced response rates compared with nonreceipt of the postcard. The influence of prenotification was particularly strong for males and older participants. The use of a teaser did not positively influence response rates and was associated with lower responding in males. What this adds to what was known? The results show that the prenotification strategy for a mental health survey is particularly effective for males and older participants. What is the implication and what should change now? The use of prenotification should be considered for mental health questionnaires for low responding groups, such as males.
of 15.9% [11], and Mond et al. [12] reported an initial response rate (before reminder) of 29% for an eating disorder questionnaire mailed to adult females using a comparable approach. A similar response rate of 24% was achieved for a study of the prevalence of eating disorder symptoms in males and females [13]. A number of strategies have been used to increase response rates in postal surveys. A recent Cochrane review [14] concluded that the use of monetary incentives, prenotification of the arrival of a questionnaire, indicating that the questionnaire originated from a university rather than a government department or commercial organization, short relative to long questionnaires, and follow-up of nonresponders with a second questionnaire substantially increased response rates. Providing prior notification of a survey increased response rates by almost half, relative to no preliminary contact [14]. Prenotification can occur in different ways. A notice may be sent in the mail, either in a letter or postcard format, or a presurvey telephone call may be made. Varying the form of prenotification has not been found to affect response rates [14,15]. Another strategy that might increase response in postal surveys is the use of an envelope ‘‘teaser,’’ namely, a short message on the survey material to entice participants to open it. Increased return rates for a mailed questionnaire were obtained when an envelope teaser indicated that money could be gained from reading the contents [16]. Alternatively, the teaser could indicate that the survey emanates from a university or similarly respected institution,
-
(2013)
-
reflect an altruistic appeal, and/or allude to the topic of the survey. Each of these approaches has the potential to highlight aspects of the survey that are differentially salient to different target samples, thus increasing response rate as posited by the leverage-salience theory [17]. Consequently, for the present study, strategies that have not been trialed previously for mental health surveys about eating disorders (prenotification postcards and an envelope teaser) were added to standard procedures for questionnaire mail-outs (the use of monetary incentives and reminder follow-up of nonresponders) to test their effect on rates of questionnaire return. The effects on response rates of two key demographic variables, age and gender, along with topic salience were also considered. Finally, we examined the number of days taken to return the questionnaire, and costs were estimated for survey completion. It was expected that the two strategies would enhance response rates, shorten the time to return questionnaires, and reduce the unit cost of obtaining completed questionnaires.
2. Method 2.1. Study design and participants The sample was 3,010 adults from the state of Victoria, Australia, randomly selected from 11 metropolitan and 6 regional electorates of the Australian Electoral Roll, a near complete database of Australian adults from which to draw a sample. An a priori sample size calculation using the G*Power program [18] with power 0.80, alpha level 0.05, and estimated odds ratio of 1.3 indicated that a total sample size of 3,120 would be required for logistic regression. The sample was stratified by gender (male and female) and age group (18e34, 35e45, and 46e65 years) to reflect the demographic composition of adults in Victoria. Using a 2 2 factorial design, individuals were randomly allocated to the experimental strategies. The postcard was sent to 1,513 individuals, and 1,497 individuals were not sent the postcard. Within each of these groups, approximately half were randomly allocated to either receive the teaser (n 5 747) or not receive the teaser (n 5 750; Fig. 1). 2.2. Materials 2.2.1. Questionnaire package The standard questionnaire packet contained a letter of invitation printed on a university letterhead paper, the questionnaire, and a university branded reply paid envelope for questionnaire return. The materials for the survey were mailed in a large envelope with university branding and sender details. 2.2.2. Prenotification postcard Printed on the front side of the postcard was the study title and a brief statement about the project. The text was imposed on a blue picture background showing silhouettes
S.A. McLean et al. / Journal of Clinical Epidemiology
-
(2013)
-
3
Fig. 1. Response rates according to experimental condition. AEC, Australian Electoral Commission; Ret. Qn, returned questionnaire; RTS, return to sender; Other, questionnaire completed and returned by a person other than the intended recipient. RTS and other cases were considered to be nonreturned questionnaires in analyses.
2.2.4. Questionnaire measures The questionnaire contained 260 items including demographics, measures of mental and physical well-being [19], self-esteem [20], general psychological distress [21], eating disorder symptomatology [22], familiarity with bulimia nervosa (BN; adapted from Ref. [23]), and other aspects of BN-MHL developed for the present study.
allocation to the experimental condition was conducted using a random number generator in Microsoft Excel and was stratified by age and gender such that equal proportions of individuals within gender and age groups were allocated to each condition. The survey mail-out was conducted on three occasions, staggered for logistical reasons. Approximately 1,000 questionnaires were mailed on each occasion. Dates of questionnaire mail-out and return were recorded on a database. The standard protocol was for individuals to receive a questionnaire package with a personalized letter of invitation to participate that fully explained the procedures of the study. Written consent was not required, and informed consent was implied by questionnaire return, as indicated in the invitation to participate and in accordance with ethics approval. Participants were offered a $10 shopping voucher, which was contingent on questionnaire return. Reminder questionnaires were mailed to nonresponders 1 month after the original questionnaire had been posted. Fig. 1 shows the study design and participant flow. Prenotification postcards were mailed 1 week before the initial mailing of the questionnaire. The envelope teaser was placed on the original and reminder envelopes.
2.3. Procedure
2.4. Data analysis
Study approval was granted from La Trobe University’s Human Ethics Committee, and procedures were in accordance with the 2008 Declaration of Helsinki. Random
Effects on response rates (number of completed returned questionnaires divided by the number of sent questionnaires) of experimental condition, gender, and age group were
of people. The reverse side of the postcard had a white background with the university logo and a brief statement on the left side concerning the method by which individuals were selected to participate in the study. This was signed by the chief investigator of the study. The reverse side of the postcard contained postage information. 2.2.3. Envelope teaser The envelope teaser consisted of a white sticker (9.0 cm wide 5.2 cm high) placed on the center top of the outgoing envelope. The sticker contained the text ‘‘You can share valuable information to help people with eating disorders’’ printed in blue, size 16 font. Further text printed in red, size 11 font, below the initial message read ‘‘Australian Research Council Funded Survey Inside.’’
4
S.A. McLean et al. / Journal of Clinical Epidemiology
examined with separate logistic regression analyses for the initial and reminder mail-outs. Four-way independent analysis of variance (ANOVA) examined differences in time taken to return questionnaires (difference between mail-out and return date) as a function of prenotification (present or absent), teaser (present or absent), gender, and age group. Cost per completion was determined by multiplying cost per unit production for questionnaires (consisting of the sum of materials, printing, salary for administrative tasks, and postage) by the number of questionnaires required to be mailed to receive one returned questionnaire.
3. Results At the first mail-out, completed questionnaires were received from 487 (16.2%) participants, and after the reminder, 174 questionnaires were returned, producing a total of 661 questionnaires, a response rate of 22.0%. Of the participants, 61.4% were female and 38.6% were male. A little over a quarter (27.5%) of the returned questionnaires was from the 18- to 34-year age group, 30.1% were from the 35- to 45-year age group, and 42.4% were from the 46- to 65-year age group. Response rates according to experimental condition and demographic group for the initial mail-out are presented in Table 1 (reminder questionnaire response rates are available from the authors). Response rates are conservative as they do not account for return to sender (N 5 195) or questionnaires completed by someone other than the intended individual (N 5 17). Table 2 shows summary statistics for main effects for the logistic regression analyses, which revealed significant effects for prenotification, gender, and age on response rates for the initial mail-out. Significant interaction terms for prenotification by gender (P ! 0.05), prenotification by age group (P ! 0.01), and teaser by gender (P ! 0.05) were found. Higher response rates were achieved with prenotification, particularly for males and the oldest age group. The interaction between teaser and gender reflected lower response rates for males and slightly higher response rates for females who got the teaser. Logistic regression analysis of response rates for the reminder questionnaire showed significant effects related to prenotification, gender, and age group (Table 2). Higher response rates to the reminder questionnaire were achieved for older participants if a prenotification postcard had been sent (P ! 0.05), but lower rates were achieved for participants in the middle age group who received prenotification (P ! 0.05). A gender by age group interaction reflected higher response rates for females in the middle age group (P ! 0.05). 3.1. Effect of topic salience A two-way ANOVA examined whether individuals with higher scores on familiarity with BN, indicating more
-
(2013)
-
experience of the survey topic, were attracted by prenotification or the teaser. Neither main effects of condition nor interaction effects were significant, indicating no difference between groups on familiarity with BN. 3.2. Response time The mean number of days to return the questionnaire for the total sample was 13.0 [standard deviation (SD) 5 7.43]. Four-way ANOVA revealed significant main effects for age group (F (2, 463) 5 3.47; P 5 0.032), with small effect size (partial h2 5 0.02). Post hoc comparisons with Tukey honestly significant difference test indicated that younger participants (M 5 14.1; SD 5 8.5) took significantly longer time to return the questionnaire than older participants (M 5 12.0, SD 5 7.4). A significant interaction between gender and teaser was found (F (1, 463) 5 4.32; P 5 0.038), with small effect size (partial h2 5 0.01). Females who received the teaser (M 5 13.9; SD 5 11.42) were slower to return the questionnaire than females who did not receive the teaser (M 5 11.4; SD 5 8.5). There was no effect of teaser on response time for males (teaser: M 5 13.0, SD 5 6.9; no teaser: M 5 13.7, SD 5 6.8). 3.3. Comparison of costs Cost per completion, taking into account response rates by gender, for prenotification (with teaser absent) was compared with cost per completion when neither prenotification nor teaser was used. Despite the higher cost to implement the prenotification method ($4.50 per questionnaire) compared with the no strategy method ($3.57), cost per completion was lower for the total sample for prenotification compared with no strategy ($23.68 vs. $26.25) and markedly lower for older and younger males (Fig. 2). Cost estimates are not shown for the teaser condition as this did not improve response rates.
4. Discussion This study examined the effect of two strategies, prenotification and an envelope teaser, on response rates to a BN-MHL questionnaire. As expected, the presence of a prenotification postcard resulted in significantly higher response rates to the initial mail-out. However, the effect was observed only for males and older participants and was not found for females or younger age groups. In addition, the positive effect of prenotification flowed through to the reminder mail-out. Contrary to expectations, the presence of an envelope teaser did not improve response rates to either mail-out. Furthermore, for the initial mail-out, already low response rates for males were exacerbated by the presence of a teaser. Females had higher response rates than males to the initial, but not reminder, mail-out, whereas older participants had higher response rates than younger participants for both mail-outs.
S.A. McLean et al. / Journal of Clinical Epidemiology
-
(2013)
-
5
Table 1. Response rates for the initial mail-out as a function of prenotification, envelope teaser, gender, and age group
Variable Overall Prenotification present Teaser present Teaser absent Prenotification absent Teaser present Teaser absent Main effects Prenotification present Prenotification absent Teaser present Teaser absent
Gender
Age group (yr)
Overall, N (%)
Males, N (%)
Females, N (%)
18e34, N (%)
35e45, N (%)
46e65, N (%)
487 (16.2)
174 (11.7)
313 (20.6)
139 (12.4)
142 (15.3)
206 (21.4)
125 (16.6) 145 (19.0)
41 (11.0) 68 (18.0)
84 (22.2) 77 (20.1)
29 (10.4) 42 (14.7)
34 (14.7) 33 (14.1)
70 (28.8) 62 (25.6)
115 (15.4) 102 (13.6)
32 (8.8) 33 (8.9)
83 (21.7) 69 (18.3)
37 (13.5) 31 (11.1)
42 (17.9) 33 (14.3)
36 (15.2) 38 (15.8)
71 68 66 73
67 75 76 66
270 217 240 247
(17.8) (14.5) (16.0) (16.3)
109 65 73 101
(14.5) (8.8) (9.9) (13.5)
161 152 167 146
(21.1) (20.0) (21.9) (19.2)
(12.6) (12.3) (11.9) (12.9)
(14.4) (16.1) (16.3) (14.2)
132 74 98 108
(27.2) (15.5) (20.5) (22.3)
%dReturn rate for the questionnaire (N returned/N sent 100).
Higher response rates for prenotification is consistent with the study by Edwards et al. [14], who found that participants were approximately 1.5 times more likely to return the questionnaire following prenotification. A similar ratio was observed in the present study. The reason for the effectiveness of the prenotification was not explored. However, it may enhance the credibility and seriousness of the survey or act as a ‘‘polite’’ way of informing a person of a future request. A novel finding was that males and older participants were particularly susceptible to an effect of prenotification on response rates. A trend for enhanced responding for males and older participants following prenotification was found for a previous health-related survey [24], although the effect was stronger in the present study. This is the first demonstration of such an effect for a survey about eating disorders. In contrast to the study by Dommeyer et al. [16] who demonstrated increased response rates with the use of a teaser, response rates in the present study were not improved by an envelope teaser. Moreover, the teaser appeared to dampen response for males. We are not aware of any previous reports of this latter finding. Data bearing on reasons for participation or nonparticipation were not obtained. However, one possibility is that the reference to ‘‘eating disorders’’
within the teaser may have been off-putting to male respondents, given that BN is far more common in women than men [5]. Although the leverage-salience theory [17] might predict that individuals with higher interest in, or experience with the topic of the survey might be more likely to participate when the topic is highlighted, such as with a teaser, there were no differences between experimental conditions on levels of familiarity with BN. It is also possible that a teaser appealing to altruism rather than monetary gain does not provide a sufficiently compelling reason to complete the survey. A recent study found that the use of a teaser alluding to a ‘‘$25 incentive’’ did not significantly increase response rates of physicians [25], although physicians may not consider $25 adequate compensation for their time. Finally, it is possible that the presence of identifying information on the envelope, namely university branding, and on survey materials, was such that the teaser provided no additional incentive to participate. The higher response rates for females compared with males and for older participants compared with younger participants are consistent with findings from previous research (eg, [15,26e28]). However, the gender-by-age
Table 2. Summary statistics for the logistic regression analyses examining response rates Odds ratio (95% CI) Variable Prenotification Teaser Gender Age group (yr) 18e34 35e45 46e65
Initial mail-out
Reminder mail-out
0.58 (0.40, 0.85)** 0.84 (0.66, 1.08) 0.57 (0.38, 0.87)**
0.51 (0.27, 0.94)* 1.15 (0.84, 1.58) 0.87 (0.49, 1.53)
0.45 (0.31, 0.66)*** 0.51 (0.35, 0.75)** d
0.38 (0.19, 0.73)** 0.64 (0.35, 1.18) d
Abbreviation: CI, confidence interval. Prenotification: 0 5 yes, 1 5 no; teaser: 0 5 yes, 1 5 no; gender: 0 5 female, 1 5 male; age group: 46e65 yr reference group. *P ! 0.05; **P ! 0.01; ***P ! 0.001.
Fig. 2. Cost per questionnaire completion after the initial mail-out for gender and age groups comparing the presence or absence of prenotification.
6
S.A. McLean et al. / Journal of Clinical Epidemiology
interaction found for the reminder questionnaire in the present study is less commonly reported for mental health surveys. As noted, for a survey of eating disorders, the content may be more interesting to female participants, which could partially account for the higher response rates [29e31]. It is possible that the lower response rate for younger participants is also partially due to higher rates of incorrect contact details for this subgroup when using the Electoral Roll [12]. Response rate findings suggest that females were less affected by the strategies in this study. However, analysis of the time taken to return the questionnaire showed that females who received the teaser returned the questionnaire slower than those who did not. It is possible that some of the same factors that may have dampened response rates in males affected response times for females. Although this difference was only two and a half days, this time frame may be of importance in research in which a rapid response is relevant. Specific gender effects for differential response times for return of questionnaires with a teaser have not been previously reported. However, in contrast to the present finding, Dommeyer et al. [16] reported a faster response for the presence of a teaser compared with a control condition. The analysis of costs per completion indicated that sending prenotification was more cost-effective than not doing so, particularly for younger and older males. The increased response rates compensated for the added costs of preparation and postage. Although future research would need to examine the generalizability of the present findings to other mental health problems, a degree of generalizability seems likely given that negative attitudes toward, and behaviors in response to, mental illness in the community such as stigmatization [32] and reluctance to discuss psychiatric problems are shared across psychiatric disorders. Consequently, strategies that influence inclination to respond to questionnaires about different mental health problems are likely to be quite similar. A limitation of the present study was that factors other than prenotification and the use of a teaser may account for variability in response rates, and these were not considered in the present study. Although it was considered unnecessary to provide further evidence for beneficial effects of a monetary incentive and/or reminder questionnaire, the effects of one or both of these strategies might interact with prenotification and/or the use of the teaser. In addition, the study addressed a very specific content area, BN, which may limit generalizability of findings beyond the mental health field. This study provides the first evidence of higher response rates for the use of prenotification for a mental health survey about eating disorders and shows that the effect was strong for males and older participants. This finding is of particular relevance given the generally low response rates to postal surveys for males. The study also demonstrates
-
(2013)
-
that the benefits of prenotification may extend to an advantage in cost-effectiveness. References [1] Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B, Pollitt P. ‘‘Mental health literacy’’: a survey of the public’s ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med J Aust 1997;166:182e6. [2] Hepworth NS, Paxton SJ. Pathways to help-seeking in bulimia nervosa and binge eating problems: a concept mapping approach. Int J Eat Disord 2007;40:493e504. [3] Mond JM, Hay P, Rodgers B, Owen C. Mental health literacy and eating disorders: what do women with bulimic eating disorders think and know about bulimia nervosa and its treatment? J Ment Health 2008;17:565e75. [4] Zachrisson HD, Vedul-Kjelsas E, Gotestam KG, Mykletun A. Time trends in obesity and eating disorders. Int J Eat Disord 2008;41: 673e80. [5] Hay P, Mond J, Buttner P, Darby A. Eating disorder behaviors are increasing: findings from two sequential community surveys in South Australia. PLoS One 2008;3:e1541. [6] Creed FH, Davies I, Jackson J, Littlewood A, Chew-Graham C, Tomenson B, et al. The epidemiology of multiple somatic symptoms. J Psychosom Res 2012;72:311e7. [7] Griffiths KM, Nakane Y, Christensen H, Yoshioka K, Jorm AF, Nakane H. Stigma in response to mental disorders: a comparison of Australia and Japan. BMC Psychiatry 2006;6:21. [8] Tourangeau R, Yan T. Sensitive questions in surveys. Psychol Bull 2007;133:859e83. [9] Ravens-Sieberer U, Erhart M, Wetzel R, Kr€ugel A, Brambosch A. Phone respondents reported less mental health problems whereas mail interviewee gave higher physical health ratings. J Clin Epidemiol 2008;61:1056e60. [10] Shumway M, Unick GJ, McConnell WA, Catalano R, Forster P. Measuring community preferences for public mental health services: pilot test of a mail survey method. Community Ment Health J 2004; 40:281e95. [11] Lucas N, Windsor TD, Caldwell TM, Rodgers B. Psychological distress in non-drinkers: associations with previous heavy drinking and current social relationships. Alcohol Alcohol 2010;45:95e102. [12] Mond JM, Rodgers B, Hay PJ, Owen C, Beumont PJV. Mode of delivery, but not questionnaire length, affected response in an epidemiological study of eating-disordered behavior. J Clin Epidemiol 2004; 57:1167e71. [13] Striegel-Moore RH, Rosselli F, Perrin N, DeBar L, Wilson GT, May A, et al. Gender difference in the prevalence of eating disorder symptoms. Int J Eat Disord 2009;42:471e4. [14] Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009;(3):MR000008. [15] Beebe TJ, Rey E, Ziegenfuss JY, Jenkins S, Lackore K, Talley NJ, et al. Shortening a survey and using alternative forms of prenotification: impact on response rate and quality. BMC Med Res Methodol 2010;10:50. [16] Dommeyer CJ, Elganayan D, Umans C. Increasing mail survey response with an envelope teaser. J Market Res Soc 1991;33:137. [17] Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. Public Opin Q 2000;64: 299e308. [18] Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009;41:1149e60. [19] Ware J Jr, Kosinski M, Keller SD. A 12-item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220e33.
S.A. McLean et al. / Journal of Clinical Epidemiology [20] Rosenberg M. Measurement of self-esteem. In: Rosenberg M, editor. Society and the adolescent self-image. Princeton, NJ: Princeton University Press; 1965:16e36. [21] Kessler R, Andrews G, Colpe L, Hiripi E, Mroczek D, Normand S, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002;32:959e76. [22] Fairburn CG, Beglin SJ. Assessment of eating disorders: interview or self-report questionnaire? Int J Eat Disord 1994;16:363e70. [23] Mond JM, Robertson-Smith G, Vetere A. Stigma and eating disorders: is there evidence of negative attitudes towards anorexia nervosa among women in the community? J Ment Health 2006;15:519e32. [24] Eaker S, Bergstr€ om R, Bergstr€om A, Adami H-O, Nyren O. Response rate to mailed epidemiologic questionnaires: a population-based randomized trial of variations in design and mailing routines. Am J Epidemiol 1998;147:74e82. [25] Ziegenfuss JY, Burmeister K, James KM, Haas L, Tilburt JC, Beebe TJ. Getting physicians to open the survey: little evidence that an envelope teaser increases response rates. BMC Med Res Methodol 2012;12:41.
-
(2013)
-
7
[26] Picavet HSJ. National health surveys by mail or home interview: effects on response. J Epidemiol Community Health 2001;55: 408e13. [27] Lundberg I, Damstr€om Thakker K, H€allstr€om T, Forsell Y. Determinants of non-participation, and the effects of non-participation on potential cause-effect relationships, in the PART study on mental disorders. Soc Psychiatry Psychiatr Epidemiol 2005;40:475e83. [28] Tolonen H, Helakorpi S, Talala K, Helasoja V, Martelin T, Prattala R. 25-Year trends and socio-demographic differences in response rates: Finnish adult health behaviour survey. Eur J Epidemiol 2006;21: 409e15. [29] Goyden J. The silent minority: nonrespondents on sample surveys. Cambridge, UK: Polity Press; 1987. [30] Groves RM, Presser S, Dipko S. The role of topic interest in survey participation decisions. Public Opin Q 2004;68:2e31. [31] Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007;17:643e53. [32] Crisp AH, Gelder MG, Rix S, Meltzer HI, Rowlands OJ. Stigmatisation of people with mental illnesses. Br J Psychiatry 2000;177:4e7.
-
(2013)
-
ORIGINAL ARTICLE
Prenotification but not envelope teaser increased response rates in a bulimia nervosa mental health literacy survey Si^an A. McLeana,*, Susan J. Paxtona, Robin Masseya, Jonathan M. Mondb,c, Bryan Rodgersd, Phillipa J. Haye,f a School of Psychological Science, La Trobe University, Kingsbury Drive, Melbourne, VIC 3086, Australia Research School of Psychology, The Australian National University, Barry Drive, Canberra, ACT 0200, Australia c Centre for Rural and Remote Mental Health, University of Newcastle, University Drive, Orange, NSW 2308, Australia d Australian Demographic & Social Research Institute, The Australian National University, Barry Drive, Canberra, ACT 0200, Australia e Centre for Health Research, School of Medicine, University of Western Sydney, Narellan Road, Sydney, NSW 2751, Australia f School of Medicine, James Cook University, 1 James Cook Drive, Townsville, QLD 4811, Australia b
Accepted 25 October 2013; Published online xxxx
Abstract Objectives: Understanding attitudes to mental health issues can inform public health interventions. However, low response rates may contribute to nonresponse bias. We examined the effect of a prenotification postcard and a short message (teaser) on response rates to a mailed questionnaire about bulimia nervosa ‘‘mental health literacy.’’ Study Design and Setting: Questionnaires were mailed to 3,010 adults (50.6% female and 49.4% male) aged 18e65 years. Procedures were varied according to sending prenotification before the questionnaire and the placement of a teaser on the survey envelope. Outcomes considered were response rate, response time, and cost. Results: The overall response rate was 22.0%. Significant main effects showed higher response rates for the use of prenotification, among female participants, and older participants. A significant interaction of teaser by gender indicated lower response rates for men who received the teaser but not for women. Older participants returned the questionnaire more promptly than younger participants. Femalesdbut not malesdwho received the teaser were slower to return the questionnaire. Higher response rates for participants receiving the postcard compensated for increased costs, particularly for males and older participants. Conclusion: Response rates to a mental health postal survey can be increased through the use of prenotification. Ó 2013 Elsevier Inc. All rights reserved. Keywords: Questionnaire response rates; Nonresponse bias; Prenotification; Teaser; Bulimia nervosa; Demographic groups
1. Introduction Population-based surveys are important for providing understanding of community knowledge, beliefs, and attitudes toward mental health problems, collectively referred to as mental health literacy (MHL) [1], which contributes to increased recognition of [2,3] and treatment seeking for [2] mental health problems. Furthermore, populationbased surveys provide valuable information about prevalence Conflict of interest: None. Funding: This research was funded by a grant (DP1095656) from the Australian Research Council to S.J.P., J.M.M., P.J.H., and B.R. B.R. is supported by National Health & Medical Research Council Fellowship No. 471429. * Corresponding author. Tel.: þ61-3-9749-2949; fax: þ61-3-94791956. E-mail address: [email protected] (S.A. McLean). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2013.10.013
rates for mental illnesses (eg, [4e6]) and are an efficient and cost-effective method to gain information from all sections of the community. However, low response rates to mailed questionnaires can affect the generalizability of findings because of nonresponse bias. This study investigates ways to optimize response rates to surveys of this kind. Little research is available regarding effective strategies for enhancing response rates to mental health and specifically eating disorder surveys. Responses to these surveys may be impacted by commonly held stigmatizing attitudes toward mental illness [7] and by the potentially sensitive nature of the survey questions [8]. Response rates to mailed questionnaires on mental health vary, but they are typically low, ranging from 32% to 47% [9,10]. Similarly, low response rates have been found for surveys of eating behaviors and eating disorders. The Food, Drink, Lifestyle and Wellbeing mail survey in Australia reported a response rate
2
S.A. McLean et al. / Journal of Clinical Epidemiology
What is new? Key findings Receipt of a prenotification postcard enhanced response rates compared with nonreceipt of the postcard. The influence of prenotification was particularly strong for males and older participants. The use of a teaser did not positively influence response rates and was associated with lower responding in males. What this adds to what was known? The results show that the prenotification strategy for a mental health survey is particularly effective for males and older participants. What is the implication and what should change now? The use of prenotification should be considered for mental health questionnaires for low responding groups, such as males.
of 15.9% [11], and Mond et al. [12] reported an initial response rate (before reminder) of 29% for an eating disorder questionnaire mailed to adult females using a comparable approach. A similar response rate of 24% was achieved for a study of the prevalence of eating disorder symptoms in males and females [13]. A number of strategies have been used to increase response rates in postal surveys. A recent Cochrane review [14] concluded that the use of monetary incentives, prenotification of the arrival of a questionnaire, indicating that the questionnaire originated from a university rather than a government department or commercial organization, short relative to long questionnaires, and follow-up of nonresponders with a second questionnaire substantially increased response rates. Providing prior notification of a survey increased response rates by almost half, relative to no preliminary contact [14]. Prenotification can occur in different ways. A notice may be sent in the mail, either in a letter or postcard format, or a presurvey telephone call may be made. Varying the form of prenotification has not been found to affect response rates [14,15]. Another strategy that might increase response in postal surveys is the use of an envelope ‘‘teaser,’’ namely, a short message on the survey material to entice participants to open it. Increased return rates for a mailed questionnaire were obtained when an envelope teaser indicated that money could be gained from reading the contents [16]. Alternatively, the teaser could indicate that the survey emanates from a university or similarly respected institution,
-
(2013)
-
reflect an altruistic appeal, and/or allude to the topic of the survey. Each of these approaches has the potential to highlight aspects of the survey that are differentially salient to different target samples, thus increasing response rate as posited by the leverage-salience theory [17]. Consequently, for the present study, strategies that have not been trialed previously for mental health surveys about eating disorders (prenotification postcards and an envelope teaser) were added to standard procedures for questionnaire mail-outs (the use of monetary incentives and reminder follow-up of nonresponders) to test their effect on rates of questionnaire return. The effects on response rates of two key demographic variables, age and gender, along with topic salience were also considered. Finally, we examined the number of days taken to return the questionnaire, and costs were estimated for survey completion. It was expected that the two strategies would enhance response rates, shorten the time to return questionnaires, and reduce the unit cost of obtaining completed questionnaires.
2. Method 2.1. Study design and participants The sample was 3,010 adults from the state of Victoria, Australia, randomly selected from 11 metropolitan and 6 regional electorates of the Australian Electoral Roll, a near complete database of Australian adults from which to draw a sample. An a priori sample size calculation using the G*Power program [18] with power 0.80, alpha level 0.05, and estimated odds ratio of 1.3 indicated that a total sample size of 3,120 would be required for logistic regression. The sample was stratified by gender (male and female) and age group (18e34, 35e45, and 46e65 years) to reflect the demographic composition of adults in Victoria. Using a 2 2 factorial design, individuals were randomly allocated to the experimental strategies. The postcard was sent to 1,513 individuals, and 1,497 individuals were not sent the postcard. Within each of these groups, approximately half were randomly allocated to either receive the teaser (n 5 747) or not receive the teaser (n 5 750; Fig. 1). 2.2. Materials 2.2.1. Questionnaire package The standard questionnaire packet contained a letter of invitation printed on a university letterhead paper, the questionnaire, and a university branded reply paid envelope for questionnaire return. The materials for the survey were mailed in a large envelope with university branding and sender details. 2.2.2. Prenotification postcard Printed on the front side of the postcard was the study title and a brief statement about the project. The text was imposed on a blue picture background showing silhouettes
S.A. McLean et al. / Journal of Clinical Epidemiology
-
(2013)
-
3
Fig. 1. Response rates according to experimental condition. AEC, Australian Electoral Commission; Ret. Qn, returned questionnaire; RTS, return to sender; Other, questionnaire completed and returned by a person other than the intended recipient. RTS and other cases were considered to be nonreturned questionnaires in analyses.
2.2.4. Questionnaire measures The questionnaire contained 260 items including demographics, measures of mental and physical well-being [19], self-esteem [20], general psychological distress [21], eating disorder symptomatology [22], familiarity with bulimia nervosa (BN; adapted from Ref. [23]), and other aspects of BN-MHL developed for the present study.
allocation to the experimental condition was conducted using a random number generator in Microsoft Excel and was stratified by age and gender such that equal proportions of individuals within gender and age groups were allocated to each condition. The survey mail-out was conducted on three occasions, staggered for logistical reasons. Approximately 1,000 questionnaires were mailed on each occasion. Dates of questionnaire mail-out and return were recorded on a database. The standard protocol was for individuals to receive a questionnaire package with a personalized letter of invitation to participate that fully explained the procedures of the study. Written consent was not required, and informed consent was implied by questionnaire return, as indicated in the invitation to participate and in accordance with ethics approval. Participants were offered a $10 shopping voucher, which was contingent on questionnaire return. Reminder questionnaires were mailed to nonresponders 1 month after the original questionnaire had been posted. Fig. 1 shows the study design and participant flow. Prenotification postcards were mailed 1 week before the initial mailing of the questionnaire. The envelope teaser was placed on the original and reminder envelopes.
2.3. Procedure
2.4. Data analysis
Study approval was granted from La Trobe University’s Human Ethics Committee, and procedures were in accordance with the 2008 Declaration of Helsinki. Random
Effects on response rates (number of completed returned questionnaires divided by the number of sent questionnaires) of experimental condition, gender, and age group were
of people. The reverse side of the postcard had a white background with the university logo and a brief statement on the left side concerning the method by which individuals were selected to participate in the study. This was signed by the chief investigator of the study. The reverse side of the postcard contained postage information. 2.2.3. Envelope teaser The envelope teaser consisted of a white sticker (9.0 cm wide 5.2 cm high) placed on the center top of the outgoing envelope. The sticker contained the text ‘‘You can share valuable information to help people with eating disorders’’ printed in blue, size 16 font. Further text printed in red, size 11 font, below the initial message read ‘‘Australian Research Council Funded Survey Inside.’’
4
S.A. McLean et al. / Journal of Clinical Epidemiology
examined with separate logistic regression analyses for the initial and reminder mail-outs. Four-way independent analysis of variance (ANOVA) examined differences in time taken to return questionnaires (difference between mail-out and return date) as a function of prenotification (present or absent), teaser (present or absent), gender, and age group. Cost per completion was determined by multiplying cost per unit production for questionnaires (consisting of the sum of materials, printing, salary for administrative tasks, and postage) by the number of questionnaires required to be mailed to receive one returned questionnaire.
3. Results At the first mail-out, completed questionnaires were received from 487 (16.2%) participants, and after the reminder, 174 questionnaires were returned, producing a total of 661 questionnaires, a response rate of 22.0%. Of the participants, 61.4% were female and 38.6% were male. A little over a quarter (27.5%) of the returned questionnaires was from the 18- to 34-year age group, 30.1% were from the 35- to 45-year age group, and 42.4% were from the 46- to 65-year age group. Response rates according to experimental condition and demographic group for the initial mail-out are presented in Table 1 (reminder questionnaire response rates are available from the authors). Response rates are conservative as they do not account for return to sender (N 5 195) or questionnaires completed by someone other than the intended individual (N 5 17). Table 2 shows summary statistics for main effects for the logistic regression analyses, which revealed significant effects for prenotification, gender, and age on response rates for the initial mail-out. Significant interaction terms for prenotification by gender (P ! 0.05), prenotification by age group (P ! 0.01), and teaser by gender (P ! 0.05) were found. Higher response rates were achieved with prenotification, particularly for males and the oldest age group. The interaction between teaser and gender reflected lower response rates for males and slightly higher response rates for females who got the teaser. Logistic regression analysis of response rates for the reminder questionnaire showed significant effects related to prenotification, gender, and age group (Table 2). Higher response rates to the reminder questionnaire were achieved for older participants if a prenotification postcard had been sent (P ! 0.05), but lower rates were achieved for participants in the middle age group who received prenotification (P ! 0.05). A gender by age group interaction reflected higher response rates for females in the middle age group (P ! 0.05). 3.1. Effect of topic salience A two-way ANOVA examined whether individuals with higher scores on familiarity with BN, indicating more
-
(2013)
-
experience of the survey topic, were attracted by prenotification or the teaser. Neither main effects of condition nor interaction effects were significant, indicating no difference between groups on familiarity with BN. 3.2. Response time The mean number of days to return the questionnaire for the total sample was 13.0 [standard deviation (SD) 5 7.43]. Four-way ANOVA revealed significant main effects for age group (F (2, 463) 5 3.47; P 5 0.032), with small effect size (partial h2 5 0.02). Post hoc comparisons with Tukey honestly significant difference test indicated that younger participants (M 5 14.1; SD 5 8.5) took significantly longer time to return the questionnaire than older participants (M 5 12.0, SD 5 7.4). A significant interaction between gender and teaser was found (F (1, 463) 5 4.32; P 5 0.038), with small effect size (partial h2 5 0.01). Females who received the teaser (M 5 13.9; SD 5 11.42) were slower to return the questionnaire than females who did not receive the teaser (M 5 11.4; SD 5 8.5). There was no effect of teaser on response time for males (teaser: M 5 13.0, SD 5 6.9; no teaser: M 5 13.7, SD 5 6.8). 3.3. Comparison of costs Cost per completion, taking into account response rates by gender, for prenotification (with teaser absent) was compared with cost per completion when neither prenotification nor teaser was used. Despite the higher cost to implement the prenotification method ($4.50 per questionnaire) compared with the no strategy method ($3.57), cost per completion was lower for the total sample for prenotification compared with no strategy ($23.68 vs. $26.25) and markedly lower for older and younger males (Fig. 2). Cost estimates are not shown for the teaser condition as this did not improve response rates.
4. Discussion This study examined the effect of two strategies, prenotification and an envelope teaser, on response rates to a BN-MHL questionnaire. As expected, the presence of a prenotification postcard resulted in significantly higher response rates to the initial mail-out. However, the effect was observed only for males and older participants and was not found for females or younger age groups. In addition, the positive effect of prenotification flowed through to the reminder mail-out. Contrary to expectations, the presence of an envelope teaser did not improve response rates to either mail-out. Furthermore, for the initial mail-out, already low response rates for males were exacerbated by the presence of a teaser. Females had higher response rates than males to the initial, but not reminder, mail-out, whereas older participants had higher response rates than younger participants for both mail-outs.
S.A. McLean et al. / Journal of Clinical Epidemiology
-
(2013)
-
5
Table 1. Response rates for the initial mail-out as a function of prenotification, envelope teaser, gender, and age group
Variable Overall Prenotification present Teaser present Teaser absent Prenotification absent Teaser present Teaser absent Main effects Prenotification present Prenotification absent Teaser present Teaser absent
Gender
Age group (yr)
Overall, N (%)
Males, N (%)
Females, N (%)
18e34, N (%)
35e45, N (%)
46e65, N (%)
487 (16.2)
174 (11.7)
313 (20.6)
139 (12.4)
142 (15.3)
206 (21.4)
125 (16.6) 145 (19.0)
41 (11.0) 68 (18.0)
84 (22.2) 77 (20.1)
29 (10.4) 42 (14.7)
34 (14.7) 33 (14.1)
70 (28.8) 62 (25.6)
115 (15.4) 102 (13.6)
32 (8.8) 33 (8.9)
83 (21.7) 69 (18.3)
37 (13.5) 31 (11.1)
42 (17.9) 33 (14.3)
36 (15.2) 38 (15.8)
71 68 66 73
67 75 76 66
270 217 240 247
(17.8) (14.5) (16.0) (16.3)
109 65 73 101
(14.5) (8.8) (9.9) (13.5)
161 152 167 146
(21.1) (20.0) (21.9) (19.2)
(12.6) (12.3) (11.9) (12.9)
(14.4) (16.1) (16.3) (14.2)
132 74 98 108
(27.2) (15.5) (20.5) (22.3)
%dReturn rate for the questionnaire (N returned/N sent 100).
Higher response rates for prenotification is consistent with the study by Edwards et al. [14], who found that participants were approximately 1.5 times more likely to return the questionnaire following prenotification. A similar ratio was observed in the present study. The reason for the effectiveness of the prenotification was not explored. However, it may enhance the credibility and seriousness of the survey or act as a ‘‘polite’’ way of informing a person of a future request. A novel finding was that males and older participants were particularly susceptible to an effect of prenotification on response rates. A trend for enhanced responding for males and older participants following prenotification was found for a previous health-related survey [24], although the effect was stronger in the present study. This is the first demonstration of such an effect for a survey about eating disorders. In contrast to the study by Dommeyer et al. [16] who demonstrated increased response rates with the use of a teaser, response rates in the present study were not improved by an envelope teaser. Moreover, the teaser appeared to dampen response for males. We are not aware of any previous reports of this latter finding. Data bearing on reasons for participation or nonparticipation were not obtained. However, one possibility is that the reference to ‘‘eating disorders’’
within the teaser may have been off-putting to male respondents, given that BN is far more common in women than men [5]. Although the leverage-salience theory [17] might predict that individuals with higher interest in, or experience with the topic of the survey might be more likely to participate when the topic is highlighted, such as with a teaser, there were no differences between experimental conditions on levels of familiarity with BN. It is also possible that a teaser appealing to altruism rather than monetary gain does not provide a sufficiently compelling reason to complete the survey. A recent study found that the use of a teaser alluding to a ‘‘$25 incentive’’ did not significantly increase response rates of physicians [25], although physicians may not consider $25 adequate compensation for their time. Finally, it is possible that the presence of identifying information on the envelope, namely university branding, and on survey materials, was such that the teaser provided no additional incentive to participate. The higher response rates for females compared with males and for older participants compared with younger participants are consistent with findings from previous research (eg, [15,26e28]). However, the gender-by-age
Table 2. Summary statistics for the logistic regression analyses examining response rates Odds ratio (95% CI) Variable Prenotification Teaser Gender Age group (yr) 18e34 35e45 46e65
Initial mail-out
Reminder mail-out
0.58 (0.40, 0.85)** 0.84 (0.66, 1.08) 0.57 (0.38, 0.87)**
0.51 (0.27, 0.94)* 1.15 (0.84, 1.58) 0.87 (0.49, 1.53)
0.45 (0.31, 0.66)*** 0.51 (0.35, 0.75)** d
0.38 (0.19, 0.73)** 0.64 (0.35, 1.18) d
Abbreviation: CI, confidence interval. Prenotification: 0 5 yes, 1 5 no; teaser: 0 5 yes, 1 5 no; gender: 0 5 female, 1 5 male; age group: 46e65 yr reference group. *P ! 0.05; **P ! 0.01; ***P ! 0.001.
Fig. 2. Cost per questionnaire completion after the initial mail-out for gender and age groups comparing the presence or absence of prenotification.
6
S.A. McLean et al. / Journal of Clinical Epidemiology
interaction found for the reminder questionnaire in the present study is less commonly reported for mental health surveys. As noted, for a survey of eating disorders, the content may be more interesting to female participants, which could partially account for the higher response rates [29e31]. It is possible that the lower response rate for younger participants is also partially due to higher rates of incorrect contact details for this subgroup when using the Electoral Roll [12]. Response rate findings suggest that females were less affected by the strategies in this study. However, analysis of the time taken to return the questionnaire showed that females who received the teaser returned the questionnaire slower than those who did not. It is possible that some of the same factors that may have dampened response rates in males affected response times for females. Although this difference was only two and a half days, this time frame may be of importance in research in which a rapid response is relevant. Specific gender effects for differential response times for return of questionnaires with a teaser have not been previously reported. However, in contrast to the present finding, Dommeyer et al. [16] reported a faster response for the presence of a teaser compared with a control condition. The analysis of costs per completion indicated that sending prenotification was more cost-effective than not doing so, particularly for younger and older males. The increased response rates compensated for the added costs of preparation and postage. Although future research would need to examine the generalizability of the present findings to other mental health problems, a degree of generalizability seems likely given that negative attitudes toward, and behaviors in response to, mental illness in the community such as stigmatization [32] and reluctance to discuss psychiatric problems are shared across psychiatric disorders. Consequently, strategies that influence inclination to respond to questionnaires about different mental health problems are likely to be quite similar. A limitation of the present study was that factors other than prenotification and the use of a teaser may account for variability in response rates, and these were not considered in the present study. Although it was considered unnecessary to provide further evidence for beneficial effects of a monetary incentive and/or reminder questionnaire, the effects of one or both of these strategies might interact with prenotification and/or the use of the teaser. In addition, the study addressed a very specific content area, BN, which may limit generalizability of findings beyond the mental health field. This study provides the first evidence of higher response rates for the use of prenotification for a mental health survey about eating disorders and shows that the effect was strong for males and older participants. This finding is of particular relevance given the generally low response rates to postal surveys for males. The study also demonstrates
-
(2013)
-
that the benefits of prenotification may extend to an advantage in cost-effectiveness. References [1] Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B, Pollitt P. ‘‘Mental health literacy’’: a survey of the public’s ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med J Aust 1997;166:182e6. [2] Hepworth NS, Paxton SJ. Pathways to help-seeking in bulimia nervosa and binge eating problems: a concept mapping approach. Int J Eat Disord 2007;40:493e504. [3] Mond JM, Hay P, Rodgers B, Owen C. Mental health literacy and eating disorders: what do women with bulimic eating disorders think and know about bulimia nervosa and its treatment? J Ment Health 2008;17:565e75. [4] Zachrisson HD, Vedul-Kjelsas E, Gotestam KG, Mykletun A. Time trends in obesity and eating disorders. Int J Eat Disord 2008;41: 673e80. [5] Hay P, Mond J, Buttner P, Darby A. Eating disorder behaviors are increasing: findings from two sequential community surveys in South Australia. PLoS One 2008;3:e1541. [6] Creed FH, Davies I, Jackson J, Littlewood A, Chew-Graham C, Tomenson B, et al. The epidemiology of multiple somatic symptoms. J Psychosom Res 2012;72:311e7. [7] Griffiths KM, Nakane Y, Christensen H, Yoshioka K, Jorm AF, Nakane H. Stigma in response to mental disorders: a comparison of Australia and Japan. BMC Psychiatry 2006;6:21. [8] Tourangeau R, Yan T. Sensitive questions in surveys. Psychol Bull 2007;133:859e83. [9] Ravens-Sieberer U, Erhart M, Wetzel R, Kr€ugel A, Brambosch A. Phone respondents reported less mental health problems whereas mail interviewee gave higher physical health ratings. J Clin Epidemiol 2008;61:1056e60. [10] Shumway M, Unick GJ, McConnell WA, Catalano R, Forster P. Measuring community preferences for public mental health services: pilot test of a mail survey method. Community Ment Health J 2004; 40:281e95. [11] Lucas N, Windsor TD, Caldwell TM, Rodgers B. Psychological distress in non-drinkers: associations with previous heavy drinking and current social relationships. Alcohol Alcohol 2010;45:95e102. [12] Mond JM, Rodgers B, Hay PJ, Owen C, Beumont PJV. Mode of delivery, but not questionnaire length, affected response in an epidemiological study of eating-disordered behavior. J Clin Epidemiol 2004; 57:1167e71. [13] Striegel-Moore RH, Rosselli F, Perrin N, DeBar L, Wilson GT, May A, et al. Gender difference in the prevalence of eating disorder symptoms. Int J Eat Disord 2009;42:471e4. [14] Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009;(3):MR000008. [15] Beebe TJ, Rey E, Ziegenfuss JY, Jenkins S, Lackore K, Talley NJ, et al. Shortening a survey and using alternative forms of prenotification: impact on response rate and quality. BMC Med Res Methodol 2010;10:50. [16] Dommeyer CJ, Elganayan D, Umans C. Increasing mail survey response with an envelope teaser. J Market Res Soc 1991;33:137. [17] Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. Public Opin Q 2000;64: 299e308. [18] Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009;41:1149e60. [19] Ware J Jr, Kosinski M, Keller SD. A 12-item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220e33.
S.A. McLean et al. / Journal of Clinical Epidemiology [20] Rosenberg M. Measurement of self-esteem. In: Rosenberg M, editor. Society and the adolescent self-image. Princeton, NJ: Princeton University Press; 1965:16e36. [21] Kessler R, Andrews G, Colpe L, Hiripi E, Mroczek D, Normand S, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002;32:959e76. [22] Fairburn CG, Beglin SJ. Assessment of eating disorders: interview or self-report questionnaire? Int J Eat Disord 1994;16:363e70. [23] Mond JM, Robertson-Smith G, Vetere A. Stigma and eating disorders: is there evidence of negative attitudes towards anorexia nervosa among women in the community? J Ment Health 2006;15:519e32. [24] Eaker S, Bergstr€ om R, Bergstr€om A, Adami H-O, Nyren O. Response rate to mailed epidemiologic questionnaires: a population-based randomized trial of variations in design and mailing routines. Am J Epidemiol 1998;147:74e82. [25] Ziegenfuss JY, Burmeister K, James KM, Haas L, Tilburt JC, Beebe TJ. Getting physicians to open the survey: little evidence that an envelope teaser increases response rates. BMC Med Res Methodol 2012;12:41.
-
(2013)
-
7
[26] Picavet HSJ. National health surveys by mail or home interview: effects on response. J Epidemiol Community Health 2001;55: 408e13. [27] Lundberg I, Damstr€om Thakker K, H€allstr€om T, Forsell Y. Determinants of non-participation, and the effects of non-participation on potential cause-effect relationships, in the PART study on mental disorders. Soc Psychiatry Psychiatr Epidemiol 2005;40:475e83. [28] Tolonen H, Helakorpi S, Talala K, Helasoja V, Martelin T, Prattala R. 25-Year trends and socio-demographic differences in response rates: Finnish adult health behaviour survey. Eur J Epidemiol 2006;21: 409e15. [29] Goyden J. The silent minority: nonrespondents on sample surveys. Cambridge, UK: Polity Press; 1987. [30] Groves RM, Presser S, Dipko S. The role of topic interest in survey participation decisions. Public Opin Q 2004;68:2e31. [31] Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007;17:643e53. [32] Crisp AH, Gelder MG, Rix S, Meltzer HI, Rowlands OJ. Stigmatisation of people with mental illnesses. Br J Psychiatry 2000;177:4e7.
