Original Article
Development and Preliminary Validation of the Adult Asthma Adherence QuestionnaireTM Michael Schatz, MD, MSa,b, Robert S. Zeiger, MD, PhDa,b, Su-Jau Yang, PhDb, Andrew G. Weinstein, MDc, Wansu Chen, MSb, Renee N. Saris-Baglama, PhDd, and Diane M. Turner-Bowker, PhDe San Diego and Los Angeles, Calif; Wilmington, Del; Smithfield, RI; and Pittsburgh, Pa What is already known about this topic? Adherence is an important determinant of asthma outcomes, and identifying nonadherence in clinical practice can be challenging. What does this article add to our knowledge? This study identified five questions that can be used clinically to identify patients at risk of nonadherence and the specific adherence barriers involved to allow targeted intervention to improve adherence. How does this study impact current management guidelines? This questionnaire is recommended for use in patients with uncontrolled asthma to evaluate the possible contribution of nonadherence and the specific barriers that need to be addressed in patients at risk of nonadherence. BACKGROUND: Asthma medication adherence is related to better asthma outcomes, but identification of suboptimal patient adherence behavior is not standardized in clinical settings. OBJECTVE: The purpose of this study was to develop a practical questionnaire that reflects nonadherence risk and identifies potential adherence barriers. METHODS: A questionnaire that included 20 potential adherence questions was completed by 420 adult patients with asthma who filled a prescription for an inhaled corticosteroid (ICS) and a short-acting beta agonist (SABA) in the previous 6 months. Questions without substantial floor or ceiling effects that were significantly related to self-reported low adherence or previous ICS canister dispensings were identified. Internal consistency reliability was tested by Cronbach a. Relationships of these questions to Asthma Control Test scores, future percent of days covered for ICS dispensings, and future asthma exacerbations and SABA dispensings were determined.
RESULTS: Five final questions were identified: following “my medication plan,” forgetting, not “needing” the medications, side effects, and cost. Low internal consistency reliability (1 ICS dispensed in 18-56 year olds and >1 SABA dispensed
3 months
6 months
PDC
PDC
FIGURE 1. Adherence study design. ICS, Inhaled corticosteroid canisters; PDC, percent of days covered; SABA, short-acting beta agonist canisters.
a prescription for an ICS in the previous 6 months, and filled a prescription for a short-acting beta agonist (SABA) in the previous 6 months were mailed the study questionnaire in fall 2010 (Figure 1). Only patients who reported on the questionnaire that they had been told by a doctor that they had asthma were included in the study. The study was approved by the KPSC Institutional Review Board.
Questionnaire Potential adherence questions were identified from the Webbased questionnaire of one of the authors (A.G.W.)18 and from the literature.8-15 Twenty questions that covered the spectrum of important desired content were chosen by consensus of the investigators. Responses were provided as a six-point Likert scale as follows: I agree completely, I agree mostly, I agree somewhat, I disagree somewhat, I disagree mostly, I disagree completely. The questionnaire also contained the following: (1) Two questions that allowed the determination of Questionnaire Low Adherence (QLA): “How often are you actually taking your inhaled steroid medication now?” and “Based on your doctor’s most recent instructions, how often were you advised to be taking your inhaled steroid medication now?” QLA was defined as the response to the first question indicating that the patient was actually taking the inhaled steroid medication less frequently than advised according to the response to the second question. (2) Asthma Control Test (ACT)19 is a five-question validated control tool, with scores that ranged from 5 to 25, with higher scores indicating better control. (3) Current smoking (in past 30 days); (4) sex; (5) ethnicity (Hispanic or non-Hispanic) and race (white, African American, Asian/Pacific Islander, other); and (6) education, with low education defined as high school or technical school graduate or less (no college). Administrative data ICS canister dispensings and asthma utilization (SABA canister dispensings, oral corticosteroid [OCS] dispensings linked within 7 days to an asthma encounter, emergency department visit for asthma, and hospitalizations for asthma) were captured for 2010 and 2011 from the KPSC administrative data. Data analyses Descriptive statistics were used to characterize the participant sample and questionnaire responses. To determine floor and ceiling effects (skewed responses at the bottom or top of the response scale), frequency statistics were examined. An item floor or ceiling effect was considered to be present when more than 60% of respondents reported a response option of 1 or 6, respectively, on any item.
282
SCHATZ ET AL
An exploratory factor analysis, using the maximum likelihood factor method followed by varimax rotation, was conducted to investigate the underlying factor structure of the 20 adherence questions. Eigenvalues represent the variance explained by each factor, and the proportion of variance explained by each factor is the eigenvalue for this factor divided by the sum of all eigenvalues. The number of factors chosen for the varimax rotation was based on the number of factors with an eigenvalue greater than 1, because factors with eigenvalues greater than 1 explain more variance than that of any single standardized original question.20 Relationships of responses to each question to demographic factors (sex, low education, and non-white race) and smoking were determined. Regression methods were used to select a subset of questions without floor or ceiling effects that showed the greatest discriminant validity in relation to QLA or ICS canister dispensings in the previous year. Internal consistency reliability (Cronbach a) was computed. Optimal cutoffs for individual question scores in relationship to QLA or ICS were determined with receiver operating characteristic (ROC) curve analyses. Construct validity was tested by examining the relationships of the final adherence questions to ACT scores. Predictive validity was determined by examining the relationships of the final adherence questions to (1) percent of days covered (PDC) of ICS prescriptions (based on days’ supply of dispensed canisters) over the follow-up at 3, 6, and 12 months; and (2) health care utilization during the follow-up 12 months, defined as SABA canister dispensings, OCS dispensings occurring within 7 days of an asthma encounter, and emergency department visits or hospitalizations for asthma (Figure 1). Performance properties (sensitivity, specificity, positive predictive value, and negative predictive value) of the final questionnaire were calculated for 6-month PDC < 0.50 and < 0.75. Univariate relationships were evaluated by means of c2 analyses for two dichotomous variables, Spearman correlations for two continuous variables, and by Wilcoxon rank sum test for relationships between continuous and dichotomous variables. Multivariable analyses to determine independent predictors of QLA or ICS use were performed by logistic (QLA) or linear (ICS use) stepwise regression analyses. ROC analyses for optimal response item cutoffs were performed by identifying the cutoff that produced the highest c-statistic area under the ROC curve value in logistic regression models with QLA or ICS use (cut at the median) as the outcomes. Nominal two-tailed statistical significance was set at P < .05.
RESULTS Patients The survey was mailed to 4055 patients, of whom 420 (10.4%) completed the survey (Table I). More than 60% of participants were white, with 20% Hispanic and 16% African American. Participants were well educated, and more than 90% were nonsmokers. QLA was reported by approximately 20% of patients and uncontrolled asthma by approximately one-half of patients. Patients were dispensed a mean of more than seven canisters of ICSs in the previous year. All adherence questions The exploratory factor analysis of the 20 potential adherence questions identified four factors (Table II). On the basis of the
J ALLERGY CLIN IMMUNOL: IN PRACTICE MAY/JUNE 2013
TABLE I. Characteristics of the study sample (n ¼ 420) Characteristic
Age (y), mean SD Female, no. (%) Ethnicity, no. (%) Hispanic or Latino Not Hispanic or Latino Unknown Race, no. (%) White African American Asian/Pacific Islander Other Unknown Low education (high school/technical school), no. (%) Current smoker, no. (%) QLA, no. (%) ACT, mean SD ACT < 20, no. (%) ICS canisters dispensed in previous year, mean SD SABA canisters dispensed in previous year, mean SD
Result
41.6 9.1 280 (66.7) 85 (20.2) 316 (75.2) 19 (4.5) 266 (63.3) 69 (16.4) 20 (4.8) 20 (4.8) 45 (10.7) 68 (16) 35 (8.3) 90 (21.4) 18.2 (4.8) 212 (52) 7.3 4.9 5.7 4.3
ACT, Asthma Control Test; ICS, inhaled corticosteroid; QLA, Questionnaire Low Adherence; SABA, inhaled short-acting beta agonist.
content of the questions loading on each factor, the factors were named self-reported adherence, need/effectiveness, side effects, and cost. Relationships of each question to QLA, number of ICS canisters dispensed in the previous year, smoking, and demographic factors were evaluated (Table III). Some questions from the self-reported adherence and need/effectiveness domains were related to both QLA and ICS use. Questions from the side effects factor were related to QLA, and questions from the cost domain were related to ICS use. Females were more likely to report that they start treatment without delay. Smokers were more likely to respond that ICS use does not help and that they forget at least one dose a day and were less likely to report that they follow their medication instructions for an attack. Lower education and nonwhite status were related to several need/effectiveness and side effects questions, and lower education was related to concerns about cost.
Final adherence questions Criteria for selecting the final questions included (1) relationship to QLA and/or ICS canisters in the previous year and (2) absence of greater than 60% response on one response option. Questions meeting these criteria included the following: self-reported adherence Q1, Q4, Q5; need/effectiveness Q10 and Q15; side effects Q16 and Q17; and cost Q19 and Q20. Ten (50%) of the questions (Q2, Q3, Q6-Q9, Q11-Q14) showed greater than 60% of responses on one option and were thus excluded. If more than one item from a single factor met the final question inclusion criteria, only items from that factor that were independently related to QLA or ICS use in stepwise regressions were included. These analyses eliminated Q4, Q15, Q17, and Q19. Five final questions (Q1, Q5, Q10, Q16, and Q20), which represented all four factors, were chosen according to these criteria.
SCHATZ ET AL
J ALLERGY CLIN IMMUNOL: IN PRACTICE VOLUME 1, NUMBER 3
283
TABLE II. Results of factor analysis using the 20 adherence questions from the questionnaire Question
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.
Self-reported adherence
Need/effectiveness
Side effects
Cost
0.85* 0.58* 0.52* 0.42* 0.41* 0.12 0.16 0.01 0.18 0.11 0.13 0.15 0.02 0.10 0.12 0.04 0.05 0.01 0.08 0.11
0.13 0.15 0.18 0.01 0.21 0.73* 0.59* 0.57* 0.55* 0.51* 0.51* 0.45* 0.40* 0.40* 0.37* 0.14 0.18 0.17 0.20 0.10
0.02 0.10 0.01 0.01 0.07 0.12 0.16 0.16 0.12 0.02 0.16 0.15 0.01 0.07 0.19 0.91* 0.58* 0.53* 0.17 0.12
0.02 0.08 0.02 0.03 0.12 0.01 0.13 0.14 0.09 0.07 0.04 0.14 0.07 0.02 0.13 0.04 0.10 0.12 0.89* 0.77*
I follow my medication plan I follow medication instructions for attack I start treatment without delay Avoid known allergens/irritants Forget at least one dose per day Don’t need the ICS ICS doesn’t help Don’t understand why taking Doctor starts strong medications too soon Does not need preventive treatment Spouse disagrees with treatment plan Doctor doesn’t understand my asthma Uncertain how to use medication Uncertain when to use medication Prefer not to take regular medications My ICS causes side effects I am prone to medication side effects Concerned about future side effects I can’t afford my ICS My ICS costs too much
ICS, Inhaled corticosteroid. Negative values indicate inverse relationship of question to factor. *Highest loading of each question.
TABLE III. Relationship of adherence questions to QLA, ICS, and demographic factors Factor/question
Self-reported adherence 1. I follow my medication plan 2. I follow medication instructions for attack 3. I start treatment without delay 4. Avoid known allergens/irritants 5. Forget at least one dose per day Need/effectiveness 6. Don’t need the ICS 7. ICS doesn’t help 8. Don’t understand why taking 9. Doctor starts strong medications too soon 10. Does not need preventive treatment 11. Spouse disagrees with treatment plan 12. Doctor doesn’t understand my asthma 13. Uncertain how to use medication 14. Uncertain when to use medication 15. Prefer not to take regular medications Side effects 16. My ICS causes side effects 17. I am prone to medication side effects 18. Concerned about future side effects Cost 19. My ICS costs too much 20. I can’t afford my ICS
QLA (P)
ICS (r)
Smoker (P)
Female (P)
Low education (P)
Non-white (P)
Development and Preliminary Validation of the Adult Asthma Adherence QuestionnaireTM Michael Schatz, MD, MSa,b, Robert S. Zeiger, MD, PhDa,b, Su-Jau Yang, PhDb, Andrew G. Weinstein, MDc, Wansu Chen, MSb, Renee N. Saris-Baglama, PhDd, and Diane M. Turner-Bowker, PhDe San Diego and Los Angeles, Calif; Wilmington, Del; Smithfield, RI; and Pittsburgh, Pa What is already known about this topic? Adherence is an important determinant of asthma outcomes, and identifying nonadherence in clinical practice can be challenging. What does this article add to our knowledge? This study identified five questions that can be used clinically to identify patients at risk of nonadherence and the specific adherence barriers involved to allow targeted intervention to improve adherence. How does this study impact current management guidelines? This questionnaire is recommended for use in patients with uncontrolled asthma to evaluate the possible contribution of nonadherence and the specific barriers that need to be addressed in patients at risk of nonadherence. BACKGROUND: Asthma medication adherence is related to better asthma outcomes, but identification of suboptimal patient adherence behavior is not standardized in clinical settings. OBJECTVE: The purpose of this study was to develop a practical questionnaire that reflects nonadherence risk and identifies potential adherence barriers. METHODS: A questionnaire that included 20 potential adherence questions was completed by 420 adult patients with asthma who filled a prescription for an inhaled corticosteroid (ICS) and a short-acting beta agonist (SABA) in the previous 6 months. Questions without substantial floor or ceiling effects that were significantly related to self-reported low adherence or previous ICS canister dispensings were identified. Internal consistency reliability was tested by Cronbach a. Relationships of these questions to Asthma Control Test scores, future percent of days covered for ICS dispensings, and future asthma exacerbations and SABA dispensings were determined.
RESULTS: Five final questions were identified: following “my medication plan,” forgetting, not “needing” the medications, side effects, and cost. Low internal consistency reliability (1 ICS dispensed in 18-56 year olds and >1 SABA dispensed
3 months
6 months
PDC
PDC
FIGURE 1. Adherence study design. ICS, Inhaled corticosteroid canisters; PDC, percent of days covered; SABA, short-acting beta agonist canisters.
a prescription for an ICS in the previous 6 months, and filled a prescription for a short-acting beta agonist (SABA) in the previous 6 months were mailed the study questionnaire in fall 2010 (Figure 1). Only patients who reported on the questionnaire that they had been told by a doctor that they had asthma were included in the study. The study was approved by the KPSC Institutional Review Board.
Questionnaire Potential adherence questions were identified from the Webbased questionnaire of one of the authors (A.G.W.)18 and from the literature.8-15 Twenty questions that covered the spectrum of important desired content were chosen by consensus of the investigators. Responses were provided as a six-point Likert scale as follows: I agree completely, I agree mostly, I agree somewhat, I disagree somewhat, I disagree mostly, I disagree completely. The questionnaire also contained the following: (1) Two questions that allowed the determination of Questionnaire Low Adherence (QLA): “How often are you actually taking your inhaled steroid medication now?” and “Based on your doctor’s most recent instructions, how often were you advised to be taking your inhaled steroid medication now?” QLA was defined as the response to the first question indicating that the patient was actually taking the inhaled steroid medication less frequently than advised according to the response to the second question. (2) Asthma Control Test (ACT)19 is a five-question validated control tool, with scores that ranged from 5 to 25, with higher scores indicating better control. (3) Current smoking (in past 30 days); (4) sex; (5) ethnicity (Hispanic or non-Hispanic) and race (white, African American, Asian/Pacific Islander, other); and (6) education, with low education defined as high school or technical school graduate or less (no college). Administrative data ICS canister dispensings and asthma utilization (SABA canister dispensings, oral corticosteroid [OCS] dispensings linked within 7 days to an asthma encounter, emergency department visit for asthma, and hospitalizations for asthma) were captured for 2010 and 2011 from the KPSC administrative data. Data analyses Descriptive statistics were used to characterize the participant sample and questionnaire responses. To determine floor and ceiling effects (skewed responses at the bottom or top of the response scale), frequency statistics were examined. An item floor or ceiling effect was considered to be present when more than 60% of respondents reported a response option of 1 or 6, respectively, on any item.
282
SCHATZ ET AL
An exploratory factor analysis, using the maximum likelihood factor method followed by varimax rotation, was conducted to investigate the underlying factor structure of the 20 adherence questions. Eigenvalues represent the variance explained by each factor, and the proportion of variance explained by each factor is the eigenvalue for this factor divided by the sum of all eigenvalues. The number of factors chosen for the varimax rotation was based on the number of factors with an eigenvalue greater than 1, because factors with eigenvalues greater than 1 explain more variance than that of any single standardized original question.20 Relationships of responses to each question to demographic factors (sex, low education, and non-white race) and smoking were determined. Regression methods were used to select a subset of questions without floor or ceiling effects that showed the greatest discriminant validity in relation to QLA or ICS canister dispensings in the previous year. Internal consistency reliability (Cronbach a) was computed. Optimal cutoffs for individual question scores in relationship to QLA or ICS were determined with receiver operating characteristic (ROC) curve analyses. Construct validity was tested by examining the relationships of the final adherence questions to ACT scores. Predictive validity was determined by examining the relationships of the final adherence questions to (1) percent of days covered (PDC) of ICS prescriptions (based on days’ supply of dispensed canisters) over the follow-up at 3, 6, and 12 months; and (2) health care utilization during the follow-up 12 months, defined as SABA canister dispensings, OCS dispensings occurring within 7 days of an asthma encounter, and emergency department visits or hospitalizations for asthma (Figure 1). Performance properties (sensitivity, specificity, positive predictive value, and negative predictive value) of the final questionnaire were calculated for 6-month PDC < 0.50 and < 0.75. Univariate relationships were evaluated by means of c2 analyses for two dichotomous variables, Spearman correlations for two continuous variables, and by Wilcoxon rank sum test for relationships between continuous and dichotomous variables. Multivariable analyses to determine independent predictors of QLA or ICS use were performed by logistic (QLA) or linear (ICS use) stepwise regression analyses. ROC analyses for optimal response item cutoffs were performed by identifying the cutoff that produced the highest c-statistic area under the ROC curve value in logistic regression models with QLA or ICS use (cut at the median) as the outcomes. Nominal two-tailed statistical significance was set at P < .05.
RESULTS Patients The survey was mailed to 4055 patients, of whom 420 (10.4%) completed the survey (Table I). More than 60% of participants were white, with 20% Hispanic and 16% African American. Participants were well educated, and more than 90% were nonsmokers. QLA was reported by approximately 20% of patients and uncontrolled asthma by approximately one-half of patients. Patients were dispensed a mean of more than seven canisters of ICSs in the previous year. All adherence questions The exploratory factor analysis of the 20 potential adherence questions identified four factors (Table II). On the basis of the
J ALLERGY CLIN IMMUNOL: IN PRACTICE MAY/JUNE 2013
TABLE I. Characteristics of the study sample (n ¼ 420) Characteristic
Age (y), mean SD Female, no. (%) Ethnicity, no. (%) Hispanic or Latino Not Hispanic or Latino Unknown Race, no. (%) White African American Asian/Pacific Islander Other Unknown Low education (high school/technical school), no. (%) Current smoker, no. (%) QLA, no. (%) ACT, mean SD ACT < 20, no. (%) ICS canisters dispensed in previous year, mean SD SABA canisters dispensed in previous year, mean SD
Result
41.6 9.1 280 (66.7) 85 (20.2) 316 (75.2) 19 (4.5) 266 (63.3) 69 (16.4) 20 (4.8) 20 (4.8) 45 (10.7) 68 (16) 35 (8.3) 90 (21.4) 18.2 (4.8) 212 (52) 7.3 4.9 5.7 4.3
ACT, Asthma Control Test; ICS, inhaled corticosteroid; QLA, Questionnaire Low Adherence; SABA, inhaled short-acting beta agonist.
content of the questions loading on each factor, the factors were named self-reported adherence, need/effectiveness, side effects, and cost. Relationships of each question to QLA, number of ICS canisters dispensed in the previous year, smoking, and demographic factors were evaluated (Table III). Some questions from the self-reported adherence and need/effectiveness domains were related to both QLA and ICS use. Questions from the side effects factor were related to QLA, and questions from the cost domain were related to ICS use. Females were more likely to report that they start treatment without delay. Smokers were more likely to respond that ICS use does not help and that they forget at least one dose a day and were less likely to report that they follow their medication instructions for an attack. Lower education and nonwhite status were related to several need/effectiveness and side effects questions, and lower education was related to concerns about cost.
Final adherence questions Criteria for selecting the final questions included (1) relationship to QLA and/or ICS canisters in the previous year and (2) absence of greater than 60% response on one response option. Questions meeting these criteria included the following: self-reported adherence Q1, Q4, Q5; need/effectiveness Q10 and Q15; side effects Q16 and Q17; and cost Q19 and Q20. Ten (50%) of the questions (Q2, Q3, Q6-Q9, Q11-Q14) showed greater than 60% of responses on one option and were thus excluded. If more than one item from a single factor met the final question inclusion criteria, only items from that factor that were independently related to QLA or ICS use in stepwise regressions were included. These analyses eliminated Q4, Q15, Q17, and Q19. Five final questions (Q1, Q5, Q10, Q16, and Q20), which represented all four factors, were chosen according to these criteria.
SCHATZ ET AL
J ALLERGY CLIN IMMUNOL: IN PRACTICE VOLUME 1, NUMBER 3
283
TABLE II. Results of factor analysis using the 20 adherence questions from the questionnaire Question
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.
Self-reported adherence
Need/effectiveness
Side effects
Cost
0.85* 0.58* 0.52* 0.42* 0.41* 0.12 0.16 0.01 0.18 0.11 0.13 0.15 0.02 0.10 0.12 0.04 0.05 0.01 0.08 0.11
0.13 0.15 0.18 0.01 0.21 0.73* 0.59* 0.57* 0.55* 0.51* 0.51* 0.45* 0.40* 0.40* 0.37* 0.14 0.18 0.17 0.20 0.10
0.02 0.10 0.01 0.01 0.07 0.12 0.16 0.16 0.12 0.02 0.16 0.15 0.01 0.07 0.19 0.91* 0.58* 0.53* 0.17 0.12
0.02 0.08 0.02 0.03 0.12 0.01 0.13 0.14 0.09 0.07 0.04 0.14 0.07 0.02 0.13 0.04 0.10 0.12 0.89* 0.77*
I follow my medication plan I follow medication instructions for attack I start treatment without delay Avoid known allergens/irritants Forget at least one dose per day Don’t need the ICS ICS doesn’t help Don’t understand why taking Doctor starts strong medications too soon Does not need preventive treatment Spouse disagrees with treatment plan Doctor doesn’t understand my asthma Uncertain how to use medication Uncertain when to use medication Prefer not to take regular medications My ICS causes side effects I am prone to medication side effects Concerned about future side effects I can’t afford my ICS My ICS costs too much
ICS, Inhaled corticosteroid. Negative values indicate inverse relationship of question to factor. *Highest loading of each question.
TABLE III. Relationship of adherence questions to QLA, ICS, and demographic factors Factor/question
Self-reported adherence 1. I follow my medication plan 2. I follow medication instructions for attack 3. I start treatment without delay 4. Avoid known allergens/irritants 5. Forget at least one dose per day Need/effectiveness 6. Don’t need the ICS 7. ICS doesn’t help 8. Don’t understand why taking 9. Doctor starts strong medications too soon 10. Does not need preventive treatment 11. Spouse disagrees with treatment plan 12. Doctor doesn’t understand my asthma 13. Uncertain how to use medication 14. Uncertain when to use medication 15. Prefer not to take regular medications Side effects 16. My ICS causes side effects 17. I am prone to medication side effects 18. Concerned about future side effects Cost 19. My ICS costs too much 20. I can’t afford my ICS
QLA (P)
ICS (r)
Smoker (P)
Female (P)
Low education (P)
Non-white (P)
