Journal of Clinical Epidemiology 67 (2014) 1076e1082
Adherence measurement and patient recruitment methods are poor in intervention trials to improve patient adherence Rebecca A. Jefferya,b, Tamara Navarroa, Nancy L. Wilczynskia, Emma C. Isermana, Arun Keepanasserila, Bhairavi Sivaramalingama, Thomas Agoritsasa, R. Brian Haynesa,c,* a
Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada, L8S 4K1 b Faculty of Medicine, Dalhousie University, 1459 Oxford Street, Halifax, Nova Scotia, Canada, B3H 4R2 c Department of Medicine, McMaster University, Health Sciences Centre, 1200 Main Street West, Hamilton, Ontario, Canada, L8N 3Z5 Accepted 1 June 2014; Published online 30 July 2014
Abstract Objectives: To develop a scale and survey the measurement of patient adherence and patient recruitment, and to explore how these methods impact the results in randomized controlled trials of interventions to improve patient adherence to medications. Study Design: Analytic survey of a purposively selected sample of patient adherence intervention trials from a systematic review, assessing the quality of adherence measurement and patient recruitment methods. Results: We identified 44 different measures of adherence, with qualities ranging from valid and objective to unreliable and subjective. The median overall quality of measures of adherence was 5 (interquartile range [IQR], 3; range, 0e9, 9 is high quality). The quality of the measures was associated with variation in the estimate of adherence (Spearman r 5 0.66; 95% confidence interval: 0.39, 0.83). The median overall quality of patient recruitment methods was 2 (IQR, 1; maximum score 6, higher is better). There was no significant correlation between the power of the trial to detect an effect and the quality of the patient recruitment methods. Conclusion: Measurement and recruitment methods in adherence trials varied considerably, and most methods were of low quality. Adherence research could be advanced by using higher quality measures of adherence and better selection and baseline assessment of study participants. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Adherence; Measurement; Quality; Methodology; Patient recruitment; Systematic review
1. Introduction Patient adherence may be defined as the extent to which a patient’s behavior corresponds with the prescription they are given for a self-administered treatment [1e3]. Medication adherence is usually quantified as the amount of medication consumed by the patient divided by the amount of medication the patient should have consumed, had they adhered to their prescribed treatment. Poor adherence is common, with typical adherence rates averaging 50%, although adherence
Conflict of interest: R.A.J. was financially supported by Ontario Graduate and Canadian Institutes of Health Research Banting and Best Graduate Scholarships (Award application number 273458). T.A. was financially supported by a Fellowship for Prospective Researchers Grant number PBGEP3-142251 from the Swiss National Science Foundation. Funding: Funding for this study comes from Canadian Institutes of Health Research Knowledge Synthesis Grant (KRS 262115). * Corresponding author. Tel.: 905-525-9140x20152; fax: 905-526-8447. E-mail address: [email protected] (R.B. Haynes). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2014.06.008
can range from 0% to over 100% for different regimens [2,4]. This is of importance to clinicians and patients as multiple studies have indicated the role of good adherence in improving clinical outcomes, such as controlling blood pressure [1,3,5,6]. Furthermore, a recent report has estimated the annual cost of the consequences of poor adherence for the United States to be $290 billion [7]. Interventions to improve medication adherence have been tested in numerous randomized controlled trials [3]. Unfortunately, studies have found singular interventions often fail [3,8]. Although no single intervention is likely to prove a cure-all for nonadherence because of its complexity, methodologic issues in the design and conduct of clinical trials may explain some of the failures and inconsistencies in the evaluations of these interventions [3]. Major methodologic issues include the choice of measurement tools to assess adherence, the recruitment of patients into trials, and selective reporting of outcomes [1,6,8e12]. Trials included in a previous systematic review [3] showed considerable variation in their methods, which
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
What is new? This study demonstrated that the quality of medication adherence measurement and participant selection methods varied greatly within a purposive sample of randomized controlled trials that tested interventions to improve patient adherence to medication. This study found an association between the quality of the adherence measurement methods used and the precision of adherence results found in trials, although other associations between methods quality and adherence results were not supported. We developed a scale for evaluating the quality of measurement of adherence methods and patient recruitment methods, which can be used by future adherence researchers.
could be the source of heterogeneity in their findings and disparity in conclusions. This hypothesis is supported by Dimatteo et al. in their meta-analysis of the correlation between patient outcomes and adherence rates, which included an analysis of some methodologic factors and their impact on the effect size [6]. The review by Dimatteo et al., however, focused on adherence outcome reporting, regardless of how reliably adherence was measured, and did not assess the impact of individual measures of adherence on the relationship between adherence and clinical outcomes [6]. Furthermore, this review did not explore the impact that patient recruitment methods (ie, how subjects were identified, declared eligible, and entered into trials) might have on adherence intervention trials. Several authors have stressed the importance of patient recruitment and retention in ensuring reliable results [13,14]. This study investigated the qualities of methods of measuring medication adherence and of the recruitment and baseline assessment of participants in adherence intervention trials, in a sample of randomized controlled trials from a systematic review that is currently in progress. It also investigated the impact that this quality had on the adherence results in these trials.
2. Methods 2.1. Study design and sample This analytic survey is based on data from a subset of randomized controlled trials included in a Cochrane systematic review update on interventions to improve patient adherence to self-administered medications, which was a separate study. Methods for this review are detailed elsewhere [3]. In brief, the systematic review searched MEDLINE,
1077
Embase, CINAHL, PsycINFO, Sociological abstracts, and the Cochrane Library, using key terms such as ‘‘compliance,’’ ‘‘medication,’’ and ‘‘clinical trial.’’ Articles retrieved from these searches were screened by title and abstract, then by full text in duplicate by independent reviewers, and disagreements were resolved through adjudication by a third reviewer. Data were then extracted in duplicate, and agreement between extractors was fair, with a k value of 0.342 (95% confidence interval [CI]: 0.114, 0.678) for 50 extractions. Disagreements between extractors were resolved by a third extractor. Authors were contacted to provide their revisions on extractions of their articles, which resulted in some alterations of the data set, based on authors’ comments. The response rate for authors was 31 of 50 (62%) after a maximum of three e-mail reminders. For this survey, a sample of 50 randomized trials of adherence interventions was chosen purposively to represent a broad range of adherence measurement techniques. Variables of interest for this study included methods of measuring medication adherence, binary and continuous adherence results, and patient selection methods, applying quality scales developed in this study for the two methodologic areas of interest. 2.2. Research questions and hypotheses Our overall objectives were to explore the quality of measurement of adherence methods and patient recruitment methods in randomized controlled trials that test interventions to improve adherence and the impact of that quality on the adherence results. Specific questions related to adherence measurement included 1. Based on an overview of the literature, what are the advantages, disadvantages, and quality aspects, including the validity, reliability, objectivity, unobtrusiveness, and longitudinality, of different measures of patient adherence to prescribed medications? 2. What are the types of adherence measures used in a sample of randomized trials of interventions to increase patient adherence to prescribed medications, and what are their measurement qualities? (An analytic survey) 3. How does the quality of measures of adherence affect adherence results in randomized trials of interventions to increase patient adherence to prescribed medications? We had two hypotheses for this question. First, in trials with continuous adherence rates, higher quality measurement of adherence would be correlated with a greater value of a measure of precision of the adherence results, defined by the coefficient of variation. Second, in trials with binary adherence data, the proportion of control group patients who are found adherent would decrease with higher quality measures, as bias for lower quality measures tends to overestimate adherence.
1078
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
Our research questions related to patient recruitment methods were as follows: 1. What is the quality of patient recruitment methodology in a sample of adherence intervention trials, based on important methodologic features of patient recruitment? (An analytic survey) 2. How do patient recruitment methods affect the statistical power of adherence results in adherence intervention trials? We hypothesized that studies with higher quality patient recruitment methods are more likely to be powered to detect a clinical effect. 2.3. Measures of Adherence Quality Scale To provide an overview of measures of medication adherence and to assist in the development of scales to assess the quality of these two methodologic areas, a separate literature review was conducted by gathering articles in two ways. The first involved a key word search in Ovid MEDLINE and PubMed Clinical Queries of the terms ‘‘weights and measures’’ [MeSH Terms] OR measure [Text Word] and ‘‘patient compliance’’ [MeSH Terms] OR ‘‘compliance’’ [MeSH Terms] OR compliance [Text Word], created in collaboration with a research librarian. The other method of identifying articles for inclusion involved screeners for the full systematic review update flagging articles of potential interest concerning measuring adherence or patient recruitment, which were then assessed separately for this literature review. Many aspects of measures of adherence were identified in the literature to be of methodologic importance, including reliability and validity, objectivity, unobtrusiveness, and the degree to which the measure provides longitudinal data (‘‘longitudinality’’) [6,8,15]. Reliability refers to the method’s reproducibility, and validity assesses whether it accurately measures what it purports to [16]. The objectivity of a measure of adherence refers to how
independent it is of the perception of the observer. An unobtrusive measure is one of which the patient is unaware, minimizing the risk of reactivity, deceit or evasion. Longitudinality refers to a measure that provides adherence findings over a period of follow-up. This period should be longer than 6 months for chronic medication adherence, and timing is important even when measuring adherence to antibiotics over a shorter period, as early adherence typically is higher than adherence toward the end of the prescription period [3]. The application of this scale (Table 1) to each study used author statements about whether a measure was reliable or valid. Assessments were made by two reviewers working independently, with differences in ratings being resolved by an adjudicator. Kappa values for agreement for the five components of this quality scale, before adjudication, were as follows: validity k 5 0.244, reliability k 5 0.257, objectivity k 5 0.506, obtrusiveness k 5 0.651, and longitudinality k 5 0.175. 2.4. Patient recruitment methods quality scale Patient recruitment methods that were identified from this literature search to be of importance included recruiting patients based on their baseline adherence levels, the representativeness of the sample, and whether the results were reported by baseline adherence [13e15,17,18]. Recruiting only nonadherent participants will result in more powerful detection of intervention effects, as adherent patients may blunt the effect of the intervention to improve adherence through a ceiling effect [17]. The representativeness of the study sample was measured by assessing the number of patients who were asked to participate in the study before the investigators reaching their sample size. This was assessed by extractors based on what authors stated regarding patient recruitment and follow-up. This aspect is based on the assumption that a sample is less representative (ie, suffers from selection bias) if a large proportion of
Table 1. Categorization of features of adherence measures (reliability, validity, objectivity, unobtrusiveness, and longitudinality) into scale scores based on whether the feature is absent (no), uncertainly absent or present (uncertain), or present (yes) Feature Reliability Validity Objectivity
Score 0 (no) Documented not to be reliable Documented not to be valid
Score 1 (uncertain) Reliability not assessed Validity not assessed
Subjective measure without appropriate Subjective measure with uncertain blinding to patients’ treatment group blinding (method or blinded group not explicitly stated) (appropriate blinding 5 method of blinding stated and blinding of patient and assessor or blinding of assessor when impossible to blind patients) Unobtrusiveness Obtrusive to patient leading to potential Unclear whether the patient is aware Hawthorne effect (eg, electronic adherence is being measured or the monitoring) extent to which the measure would interfere with their usual medication consumption Longitudinality Data provided by measure covers the past Data by measure covers a longer period 1e7 days of adherence for a chronic (O7 days) for a chronic medication (long-term) regimen regimen
Score 2 (yes) Measure documented be reliable Measure documented to be valid in comparison with a criterion standard Objective measure (medication event monitoring system, pharmacy refill data, biologic measure of drug) or subjective measure with method of blinding explained and includes appropriate blinding Patient is unaware the measure is being taken and the measure does not interrupt the normal pattern of medication consumption (pharmacy refill record)
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
eligible patients decline before the study population is recruited, especially as declining may be related to low adherence. Finally, reporting results by initial adherence status (eg, among patient with low vs. high baseline adherence) may clarify whether the intervention had an effect in those who needed it most. If only nonadherent patients were recruited, the study received the highest rating in this category as results would be reported based on all patients being initially nonadherent (Table 2). For each for these dimensions of quality of adherence measurement and quality of patient recruitment, we generated ordinal rating items using the features of an ideal measure (valid, reliable, unobtrusive, objective, and provides longitudinal data) and patient recruitment (recruited based on, and reported by, baseline adherence levels and representativeness). Each item was scored as ‘‘No,’’ ‘‘Uncertain’’ or ‘‘Yes,’’ corresponding to ‘‘dummy’’ values of 0 to 2 to translate the rating of the trials’ methodology into a value to correlate with adherence results statistically. The phrasing of each category in the data extraction form was revised repeatedly through pretesting with data extractors to improve reliability. The overall quality of these methods in a study was computed by summing the quality score across all scale items. 2.5. Statistical analysis We computed frequencies for categorical variable and medians, range, and interquartile range for the quality of the studies’ methodologies. We assessed the association between the quality of the methods and the adherence results using a Spearman rank correlation coefficient. The adherence results varied across studies according to the type of data (continuous or binary) and the methodologic area being investigated (measure of adherence or patient recruitment), as specified in the hypotheses mentioned previously. Therefore, we used a measure of precision, the coefficient of variation to represent adherence results reported as continuous
1079
variables and the proportion adherent in the control group, when adherence was reported as a binary variable. For the patient recruitment methods, the power of the trial to detect an effect was used for both binary and continuous data, which was calculated by a proportion recruited, equal to the actual sample size in a trial divided by a theoretical sample size that was calculated using a 25% minimally important absolute difference. Analyses were completed in SPSS version 20, and P ! 0.05 was considered statistically significant.
3. Results 3.1. Methods of measuring adherence: literature review For our first question for the measures of adherence, our initial literature review of articles about measuring adherence retrieved 36 different types of measures of adherence. Twenty-four of these measures were selfreport questionnaires that often contained similar questions and generally had the same advantages and disadvantages of being easy to use and economical but obtrusive to patients and subjective. Twenty measures were considered to be ‘‘valid’’ in contexts in which validity had been tested. Three measures, including physician estimates of patient adherence, were considered to be invalid based on empirical assessments by the study author or by another study of that measure. Twenty-six measures of adherence were considered to be reliable. Twelve measures were considered objective, although only four of those are direct measures of adherence. Nine measures were considered unobtrusive. Most measures of adherence were capable of measuring adherence over a long period (longitudinality). However, self-reports were often only trustworthy when measuring adherence during the previous week, as longer periods might result in recall bias. Blood or urine drug concentration
Table 2. Categorization of features of patient recruitment methods into scale scores based on whether that feature is absent (no), uncertainly absent or present (uncertain), and present (yes) Feature
Score 0 (no)
Score 1 (uncertain)
Score 2 (yes)
Nonadherent patients selected
No (no mention of past adherence in any capacity, assume both adherent and nonadherent patients recruited)
Yes (adherence was measured before study inclusion, and only those with low adherence were included in the study, or those with low adherence to begin with are separately reported)
Representativeness of sample (number recruited/number screened who met eligibility criteria) Results reported based on baseline adherence
Number of patients asked to join is much greater than sample size (O2:1 nonenrolled to enrolled)
Indeterminate (eg, only included patients with high blood pressure or patients who had that physiologic state possibly due to a lack of adherence in the past but not explicitly stated that adherence was measured before selection) Does not report number of patients asked before reaching sample size Baseline adherence measured, but results were not reported according to initial adherence level
Yes, results were reported based on baseline adherence level or if only nonadherent patients are recruited, if intention-to-treat analysis is followed
Baseline adherence was not measured
The number of patients asked to join is similar to sample size (2:1 nonenrolled to enrolled)
1080
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
assessments were only capable of measuring short periods when the drug in question has a short half-life. 3.1.1. Methods of measuring adherence: analytic survey results Our analytic survey of 50 randomized controlled trials of interventions to improve adherence included 92 measures of adherence, as 54% of studies used more than one measure of adherence. We categorized these measures into 11 generic types of measures of adherence (classified in Table 3). The frequency of different measures, the quality per measure, and the median quality for each scale item for each measure in the 50 selected trials are summarized in Table 3 (detailed table of measures per study available in Supplementary Table S1/Appendix at www.jclinepi.com). The median overall quality of measures of adherence was 5 (interquartile range [IQR], 3) on a score ranging from 0 (low quality) to 9 (high quality), maximum possible score 9. Few studies included measures such as ‘‘attendance,’’ ‘‘clinician judgment,’’ ‘‘direct observation,’’ and ‘‘therapeutic response.’’ These infrequently used measures of adherence generally had low quality scores, although therapeutic response had a median quality score of 6 (of maximum 9). Other measures of adherence, such as medication event monitoring system (MEMS), pill counts, and pharmacy records, were used more frequently and were of higher quality than other measures. Self-reports were an anomaly in this regard, as they were used quite frequently but were of lower quality (median score of 3). Higher quality scores were associated with a larger variation in the estimate of adherence. Indeed, when adherence was reported as continuous data, the overall Spearman rank correlation coefficient between the quality of the measures of adherence and precision, assessed by the coefficient of
variation of adherence, was 0.66 (95% CI: 0.39, 0.83; P ! 0.001). This indicates that the quality of the measure may impact the adherence results that are found in a trial. Furthermore, as we predicted, higher quality measures of adherence are associated with larger variance because they more fully assess the actual range of adherence rates among patients, whereas lower quality subjective measures are biased toward overestimating adherence. When adherence was reported as binary data, Spearman correlation between the quality of the measures of adherence and the proportion adherent was 0.21 (95% CI: 0.44, 0.04, P 5 0.10), indicating a higher quality measure of adherence may be associated with a lower proportion adherent in a trial, although this relationship was not statistically significant. 3.2. Patient recruitment methods The overall median quality of patient recruitment methods was 2 (IQR, 1; maximum possible score was 6) (Table 4). Few studies (3 of 50) recruited only nonadherent patients, which is a methodologic step that is thought to be very important in determining the true effect of the intervention. Most studies in our sample recruited ‘‘representative groups’’ of patients. Only three studies reported their results based on baseline adherence. However, the overall Spearman correlation between the quality score and the percentage of sample size recruited in a study (power) was not statistically significant for either continuous or binary data (0.04 [95% CI: 0.38, 0.41], P 5 0.85; and 0.14 [95% CI: 0.13, 0.39], P 5 0.30, respectively).
4. Discussion We investigated the types and quality of methodology used in trials of interventions to improve patient
Table 3. Measurement qualities of all generic types of adherence measures from a sample of randomized trials of interventions to increase patient adherence
Measurea Attendance Clinician judgment Direct observation Drug concentration in body MEMS Pharmacy refill record Pill count Self-report questionnaire Self-report diary Self-report interview Therapeutic response
Frequency of use in sample of RCTs (n)
Median validity score (max 2) and rangeb
Median reliability score (max 2) and rangeb
2 1 2 6 16 14 20 17 3 8 3
0.5 (0e1) 0 1 1 2 (1e2) 1.5 (1e2) 1 (1e2) 1 (1e2) 1 1.5 (0e2) 1
0.5 (0e1) 0 1 1 (1e2) 2 (1e2) 1.5 (1e2) 1 (1e2) 1 (1e2) 1 1.5 (0e2) 1
Median objectivity score (max 2) and rangeb
2 2 0 0
0 0 2 2 (0e2) 2 (0e2) (0e1) 0 (0e1) 2
Median unobtrusive score (max 2) and rangeb
1.5 0 0
0 1
1 0 0 (1e2) (0e2) 2 (0e2) 0 0 (0e1) (1e2)
Median longitudinality score (max 1) and rangeb 1 1 0.5 (0e1) 1 1 1 1 0 (0e1) 1 0 (0e1) 1
Median total score (max 9) and IQR 3 1 4.5 7 6.5 8 5 3 3 4.5 6
(1) (0) (0.5) (0.75) (2.25) (2) (1.25) (2) (0) (3.25) (0.5)
Abbreviations: IQR, interquartile range; MEMS, medication event monitoring system; RCT, randomized controlled trials. a Measures are not meant to be compared against each other as the nature of each quality category (ie, validity, and so forth) is not comparable between measure types as criterion standards can differ. Some measures are from the same study as some studies used multiple measures of adherence. b Range is only included in brackets if a range is possible to include based on the data.
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
1081
Table 4. Quality of patient recruitment methods in a sample of randomized trials of interventions to increase patient adherence
Dimension of patient recruitment
Frequency of studies that did not fulfill quality dimension, n (%)
Frequency of studies that are unclear if quality dimension fulfilled, n (%)
Frequency of studies that fulfill that quality dimension, n (%)
Median score in this dimension and range
Nonadherent patients selected Representativeness of sample Reporting of results
36 (72) 8 (16) 30 (60)
11 (22) 22 (44) 17 (34)
3 (6) 20 (40) 3 (6)
0 (0e2) 1 (0e2) 0 (0e2)
adherence to medication. The types of measures of adherence used in trials of interventions to improve adherence ranged from poor quality, such as attendance, to higher quality, such as pharmacy refill records. Lower quality measures of adherence were used less frequently in the literature than those with overall higher median quality scores, such as pharmacy refill records and MEMS. For the analyses of the relationship between the quality of the methods and the results of the trials, there was a significant association between the quality of methods of measurement and precision of adherence rates, suggesting the poorer methods of measurement are biased. This bias is likely to be overestimating adherence, when subjective obtrusive methods such as patient self-report are used [19]. Multiple types of measures were used in the sample of 50 randomized controlled trials of interventions to improve adherence. These range from poor quality, such as clinician judgments, to higher quality, such as pharmacy refill records. Given the low quality of adherence measurement in some studies, it would appear that not all researchers are currently aware of the importance of this aspect of methodology in adherence trials. On the other hand, some trials used three, four, or five measures of adherence, as they noted the importance of composite measures in establishing a fuller, more detailed picture of adherence. The quality of measures of adherence appears to relate to the adherence results in trials that test interventions to improve adherence. This finding, although exploratory, further supports the importance of this methodologic aspect in trials. The patient recruitment methodologic quality also varied considerably. Few studies used baseline adherence rates as an inclusion criterion for participants, despite the potential importance of this step in establishing the true effect of an intervention. Our exploratory analysis was underpowered, however, and did not establish whether this methodologic step correlated with the power of the study or not. Although questions remain as to whether patient recruitment methods might impact adherence results or the power of the study to detect an effect, it is clear researchers do not often consider these aspects of methodology in their trials, as demonstrated by the low overall median quality score. This study is limited by several factors. The sample size for this analytic survey was chosen for feasibility reasons and did not allow for a highly powered analysis for the
exploratory research questions. However, these analyses were secondary to the main aim of the study, which was to describe and assess the quality of adherence measures in randomized controlled trials. The scale that was developed in this study to assess the quality of the methodology was not assessed for validity or reliability before applying it to these studies. However, the online data extraction form was extensively tested, and adjudicated forms were sent to authors for confirmation of our interpretation of their study, adding to the accuracy of the data. Trials were included in this study based on the type of measure of adherence used in the trial, selecting them purposively. This approach may have been biased, but the full set of articles included in the separate systematic review had not been identified and extracted at that point. Although a random sample, stratifying studies by the type of measure used in them, may have been the best approach with the least selection bias, this approach was not possible given the stage of the full review. Given these limitations, there are several future directions this area of research could take. Further testing of the scales that were used in this study to assess the quality of the methods used to determine the reliability and validity of the scale would be worthwhile. Although establishing a gold standard for measuring adherence would also be a useful next step, this may not be feasible given the diversity of diseases and medications that need to be measured. Instead, developing a quality threshold using the scale applied to measures in this study might be better suited to comparatively evaluate the quality of given measures of adherence. The methodologic steps outlined in this study could be applied to future adherence trials to ensure that the most accurate results are found. This process may reduce the confounding across trials and reduce the variability in efficacy of a given intervention. It may then reduce the heterogeneity across trials to allow for future meta-analyses of these studies. 5. Conclusion The state of the measurement of adherence and patient recruitment methods in randomized controlled trials to improve patient adherence to medication leaves considerable room for improvement. Using stronger methods of measurement currently available is likely to result in more effective and efficient testing of adherence interventions, based on the correlation we found between the quality of
1082
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
the methods of measuring adherence and the coefficient of variation in randomized controlled trials. The lack of a gold standard for measuring adherence is a major issue that needs to be addressed in future research. Acknowledgments The members of the Patient Adherence Systematic Review Team included the Principal Investigator, Co-Investigators, Co-Applicants/Senior Management Decision-makers, CoApplicants/Clinical Service Decision-Makers, and Research Staff. The following were involved in collection and/or organization of data: Tamara Navarro, McMaster University; Emma Iserman, McMaster University; Dawn Jedras, McMaster University; Nancy Wilczynski, McMaster University; R Brian Haynes, McMaster University; Alfonso Iorio, McMaster University; Robby Nieuwlaat, McMaster University; Reem Mustafa, McMaster University; Susan Jack, McMaster University; Norma Brown, McMaster University; Fadi Bdair, State University of New York at Buffalo; and Nicholas Hobson, DiplT at McMaster University provided programming and information technology support.
Appendix Supplementary data Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jclinepi.2014.06.008. References [1] Krousel-Wood M, Thomas S, Muntner P, Morisky D. Medication adherence: a key factor in achieving blood pressure control and good clinical outcomes in hypertensive patients. Curr Opin Cardiol 2004; 19:357e62. [2] World Health Report 2003- Adherence to long-term therapies: evidence for action. Geneva, World Health Organization, 2003. ISBN 92 4 154599 2, 110 pages.
[3] Haynes RB, Ackloo E, Sahota N, Hp M, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2008;(2):CD000011. [4] Quittner AL, Modi AC, Lemanek KL, Ievers-Landis CE, Rapoff M. Evidence-based assessment of adherence to medical treatments in pediatric psychology. J Pediatr Psychol 2008;33(9):916e36. [5] Dunbar-Jacob J, Mortimer-Stephens MK. Treatment adherence in chronic disease. J Clin Epidemiol 2001;54(Suppl 1 (July)):S57e60. [6] DiMatteo MR, Giordani PJ, Lepper HS, Croghan TW. Patient adherence and medical treatment outcomes: a meta-analysis. Med Care 2002;40:794e811. [7] CVS Caremark. State of the States: adherence report. 2013: 1e108. Available at http://info.cvscaremark.com/cvs-insights/state-states2013. Accessed July 22, 2014. [8] Farmer KC. Methods for measuring and monitoring adherence in clinical trial and clinical practice. Clin Ther 1999;21:1074e90. [9] La Greca AM. Issues in adherence with pediatric regimens. J Pediatr Psychol 1990;15(4):423e36. [10] J€uni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ 2001;323:42e6. [11] Vermeire E, Wens J, Van Royen P, Biot Y, Hearnshaw H, Lindenmeyer A. Interventions for improving adherence to treatment recommendations in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2005;(2):CD003638. [12] Becker MH. Patient adherence to prescribed therapies. Med Care 1985;23:539e55. [13] Gross CP, Mallory R, Heiat A, Krumholz HM. Reporting the recruitment process in clinical trials: who are these patients and how did they get there? Ann Intern Med 2002;137:10e6. [14] Haynes RB, Dantes R. Patient compliance and the conduct and interpretation of therapeutic trials. Control Clin Trials 1987;8:12e9. [15] Gordis L. Conceptual and methodologic problems in measuring patient compliance. In: Haynes R, Sackett D, editors. Compliance in health care. Baltimore, MD: John Hopkins University Press; 1979: 23e45. [16] Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York, NY: Oxford University Press; 2003:4e226. [17] Norman GR, Streiner DL. Biostatistics: the bare essentials. Shelton, CT: People’s Medical Publishing House; 2008:55. [18] Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv 2008;1:211e7. [19] Haynes R, Taylor D, Sackett D, Gibson E, Bernholz C, Mukherjee J. Can simple clinical measurements detect patient noncompliance? Hypertension 1980;2:757e64.
Adherence measurement and patient recruitment methods are poor in intervention trials to improve patient adherence Rebecca A. Jefferya,b, Tamara Navarroa, Nancy L. Wilczynskia, Emma C. Isermana, Arun Keepanasserila, Bhairavi Sivaramalingama, Thomas Agoritsasa, R. Brian Haynesa,c,* a
Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada, L8S 4K1 b Faculty of Medicine, Dalhousie University, 1459 Oxford Street, Halifax, Nova Scotia, Canada, B3H 4R2 c Department of Medicine, McMaster University, Health Sciences Centre, 1200 Main Street West, Hamilton, Ontario, Canada, L8N 3Z5 Accepted 1 June 2014; Published online 30 July 2014
Abstract Objectives: To develop a scale and survey the measurement of patient adherence and patient recruitment, and to explore how these methods impact the results in randomized controlled trials of interventions to improve patient adherence to medications. Study Design: Analytic survey of a purposively selected sample of patient adherence intervention trials from a systematic review, assessing the quality of adherence measurement and patient recruitment methods. Results: We identified 44 different measures of adherence, with qualities ranging from valid and objective to unreliable and subjective. The median overall quality of measures of adherence was 5 (interquartile range [IQR], 3; range, 0e9, 9 is high quality). The quality of the measures was associated with variation in the estimate of adherence (Spearman r 5 0.66; 95% confidence interval: 0.39, 0.83). The median overall quality of patient recruitment methods was 2 (IQR, 1; maximum score 6, higher is better). There was no significant correlation between the power of the trial to detect an effect and the quality of the patient recruitment methods. Conclusion: Measurement and recruitment methods in adherence trials varied considerably, and most methods were of low quality. Adherence research could be advanced by using higher quality measures of adherence and better selection and baseline assessment of study participants. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Adherence; Measurement; Quality; Methodology; Patient recruitment; Systematic review
1. Introduction Patient adherence may be defined as the extent to which a patient’s behavior corresponds with the prescription they are given for a self-administered treatment [1e3]. Medication adherence is usually quantified as the amount of medication consumed by the patient divided by the amount of medication the patient should have consumed, had they adhered to their prescribed treatment. Poor adherence is common, with typical adherence rates averaging 50%, although adherence
Conflict of interest: R.A.J. was financially supported by Ontario Graduate and Canadian Institutes of Health Research Banting and Best Graduate Scholarships (Award application number 273458). T.A. was financially supported by a Fellowship for Prospective Researchers Grant number PBGEP3-142251 from the Swiss National Science Foundation. Funding: Funding for this study comes from Canadian Institutes of Health Research Knowledge Synthesis Grant (KRS 262115). * Corresponding author. Tel.: 905-525-9140x20152; fax: 905-526-8447. E-mail address: [email protected] (R.B. Haynes). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2014.06.008
can range from 0% to over 100% for different regimens [2,4]. This is of importance to clinicians and patients as multiple studies have indicated the role of good adherence in improving clinical outcomes, such as controlling blood pressure [1,3,5,6]. Furthermore, a recent report has estimated the annual cost of the consequences of poor adherence for the United States to be $290 billion [7]. Interventions to improve medication adherence have been tested in numerous randomized controlled trials [3]. Unfortunately, studies have found singular interventions often fail [3,8]. Although no single intervention is likely to prove a cure-all for nonadherence because of its complexity, methodologic issues in the design and conduct of clinical trials may explain some of the failures and inconsistencies in the evaluations of these interventions [3]. Major methodologic issues include the choice of measurement tools to assess adherence, the recruitment of patients into trials, and selective reporting of outcomes [1,6,8e12]. Trials included in a previous systematic review [3] showed considerable variation in their methods, which
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
What is new? This study demonstrated that the quality of medication adherence measurement and participant selection methods varied greatly within a purposive sample of randomized controlled trials that tested interventions to improve patient adherence to medication. This study found an association between the quality of the adherence measurement methods used and the precision of adherence results found in trials, although other associations between methods quality and adherence results were not supported. We developed a scale for evaluating the quality of measurement of adherence methods and patient recruitment methods, which can be used by future adherence researchers.
could be the source of heterogeneity in their findings and disparity in conclusions. This hypothesis is supported by Dimatteo et al. in their meta-analysis of the correlation between patient outcomes and adherence rates, which included an analysis of some methodologic factors and their impact on the effect size [6]. The review by Dimatteo et al., however, focused on adherence outcome reporting, regardless of how reliably adherence was measured, and did not assess the impact of individual measures of adherence on the relationship between adherence and clinical outcomes [6]. Furthermore, this review did not explore the impact that patient recruitment methods (ie, how subjects were identified, declared eligible, and entered into trials) might have on adherence intervention trials. Several authors have stressed the importance of patient recruitment and retention in ensuring reliable results [13,14]. This study investigated the qualities of methods of measuring medication adherence and of the recruitment and baseline assessment of participants in adherence intervention trials, in a sample of randomized controlled trials from a systematic review that is currently in progress. It also investigated the impact that this quality had on the adherence results in these trials.
2. Methods 2.1. Study design and sample This analytic survey is based on data from a subset of randomized controlled trials included in a Cochrane systematic review update on interventions to improve patient adherence to self-administered medications, which was a separate study. Methods for this review are detailed elsewhere [3]. In brief, the systematic review searched MEDLINE,
1077
Embase, CINAHL, PsycINFO, Sociological abstracts, and the Cochrane Library, using key terms such as ‘‘compliance,’’ ‘‘medication,’’ and ‘‘clinical trial.’’ Articles retrieved from these searches were screened by title and abstract, then by full text in duplicate by independent reviewers, and disagreements were resolved through adjudication by a third reviewer. Data were then extracted in duplicate, and agreement between extractors was fair, with a k value of 0.342 (95% confidence interval [CI]: 0.114, 0.678) for 50 extractions. Disagreements between extractors were resolved by a third extractor. Authors were contacted to provide their revisions on extractions of their articles, which resulted in some alterations of the data set, based on authors’ comments. The response rate for authors was 31 of 50 (62%) after a maximum of three e-mail reminders. For this survey, a sample of 50 randomized trials of adherence interventions was chosen purposively to represent a broad range of adherence measurement techniques. Variables of interest for this study included methods of measuring medication adherence, binary and continuous adherence results, and patient selection methods, applying quality scales developed in this study for the two methodologic areas of interest. 2.2. Research questions and hypotheses Our overall objectives were to explore the quality of measurement of adherence methods and patient recruitment methods in randomized controlled trials that test interventions to improve adherence and the impact of that quality on the adherence results. Specific questions related to adherence measurement included 1. Based on an overview of the literature, what are the advantages, disadvantages, and quality aspects, including the validity, reliability, objectivity, unobtrusiveness, and longitudinality, of different measures of patient adherence to prescribed medications? 2. What are the types of adherence measures used in a sample of randomized trials of interventions to increase patient adherence to prescribed medications, and what are their measurement qualities? (An analytic survey) 3. How does the quality of measures of adherence affect adherence results in randomized trials of interventions to increase patient adherence to prescribed medications? We had two hypotheses for this question. First, in trials with continuous adherence rates, higher quality measurement of adherence would be correlated with a greater value of a measure of precision of the adherence results, defined by the coefficient of variation. Second, in trials with binary adherence data, the proportion of control group patients who are found adherent would decrease with higher quality measures, as bias for lower quality measures tends to overestimate adherence.
1078
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
Our research questions related to patient recruitment methods were as follows: 1. What is the quality of patient recruitment methodology in a sample of adherence intervention trials, based on important methodologic features of patient recruitment? (An analytic survey) 2. How do patient recruitment methods affect the statistical power of adherence results in adherence intervention trials? We hypothesized that studies with higher quality patient recruitment methods are more likely to be powered to detect a clinical effect. 2.3. Measures of Adherence Quality Scale To provide an overview of measures of medication adherence and to assist in the development of scales to assess the quality of these two methodologic areas, a separate literature review was conducted by gathering articles in two ways. The first involved a key word search in Ovid MEDLINE and PubMed Clinical Queries of the terms ‘‘weights and measures’’ [MeSH Terms] OR measure [Text Word] and ‘‘patient compliance’’ [MeSH Terms] OR ‘‘compliance’’ [MeSH Terms] OR compliance [Text Word], created in collaboration with a research librarian. The other method of identifying articles for inclusion involved screeners for the full systematic review update flagging articles of potential interest concerning measuring adherence or patient recruitment, which were then assessed separately for this literature review. Many aspects of measures of adherence were identified in the literature to be of methodologic importance, including reliability and validity, objectivity, unobtrusiveness, and the degree to which the measure provides longitudinal data (‘‘longitudinality’’) [6,8,15]. Reliability refers to the method’s reproducibility, and validity assesses whether it accurately measures what it purports to [16]. The objectivity of a measure of adherence refers to how
independent it is of the perception of the observer. An unobtrusive measure is one of which the patient is unaware, minimizing the risk of reactivity, deceit or evasion. Longitudinality refers to a measure that provides adherence findings over a period of follow-up. This period should be longer than 6 months for chronic medication adherence, and timing is important even when measuring adherence to antibiotics over a shorter period, as early adherence typically is higher than adherence toward the end of the prescription period [3]. The application of this scale (Table 1) to each study used author statements about whether a measure was reliable or valid. Assessments were made by two reviewers working independently, with differences in ratings being resolved by an adjudicator. Kappa values for agreement for the five components of this quality scale, before adjudication, were as follows: validity k 5 0.244, reliability k 5 0.257, objectivity k 5 0.506, obtrusiveness k 5 0.651, and longitudinality k 5 0.175. 2.4. Patient recruitment methods quality scale Patient recruitment methods that were identified from this literature search to be of importance included recruiting patients based on their baseline adherence levels, the representativeness of the sample, and whether the results were reported by baseline adherence [13e15,17,18]. Recruiting only nonadherent participants will result in more powerful detection of intervention effects, as adherent patients may blunt the effect of the intervention to improve adherence through a ceiling effect [17]. The representativeness of the study sample was measured by assessing the number of patients who were asked to participate in the study before the investigators reaching their sample size. This was assessed by extractors based on what authors stated regarding patient recruitment and follow-up. This aspect is based on the assumption that a sample is less representative (ie, suffers from selection bias) if a large proportion of
Table 1. Categorization of features of adherence measures (reliability, validity, objectivity, unobtrusiveness, and longitudinality) into scale scores based on whether the feature is absent (no), uncertainly absent or present (uncertain), or present (yes) Feature Reliability Validity Objectivity
Score 0 (no) Documented not to be reliable Documented not to be valid
Score 1 (uncertain) Reliability not assessed Validity not assessed
Subjective measure without appropriate Subjective measure with uncertain blinding to patients’ treatment group blinding (method or blinded group not explicitly stated) (appropriate blinding 5 method of blinding stated and blinding of patient and assessor or blinding of assessor when impossible to blind patients) Unobtrusiveness Obtrusive to patient leading to potential Unclear whether the patient is aware Hawthorne effect (eg, electronic adherence is being measured or the monitoring) extent to which the measure would interfere with their usual medication consumption Longitudinality Data provided by measure covers the past Data by measure covers a longer period 1e7 days of adherence for a chronic (O7 days) for a chronic medication (long-term) regimen regimen
Score 2 (yes) Measure documented be reliable Measure documented to be valid in comparison with a criterion standard Objective measure (medication event monitoring system, pharmacy refill data, biologic measure of drug) or subjective measure with method of blinding explained and includes appropriate blinding Patient is unaware the measure is being taken and the measure does not interrupt the normal pattern of medication consumption (pharmacy refill record)
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
eligible patients decline before the study population is recruited, especially as declining may be related to low adherence. Finally, reporting results by initial adherence status (eg, among patient with low vs. high baseline adherence) may clarify whether the intervention had an effect in those who needed it most. If only nonadherent patients were recruited, the study received the highest rating in this category as results would be reported based on all patients being initially nonadherent (Table 2). For each for these dimensions of quality of adherence measurement and quality of patient recruitment, we generated ordinal rating items using the features of an ideal measure (valid, reliable, unobtrusive, objective, and provides longitudinal data) and patient recruitment (recruited based on, and reported by, baseline adherence levels and representativeness). Each item was scored as ‘‘No,’’ ‘‘Uncertain’’ or ‘‘Yes,’’ corresponding to ‘‘dummy’’ values of 0 to 2 to translate the rating of the trials’ methodology into a value to correlate with adherence results statistically. The phrasing of each category in the data extraction form was revised repeatedly through pretesting with data extractors to improve reliability. The overall quality of these methods in a study was computed by summing the quality score across all scale items. 2.5. Statistical analysis We computed frequencies for categorical variable and medians, range, and interquartile range for the quality of the studies’ methodologies. We assessed the association between the quality of the methods and the adherence results using a Spearman rank correlation coefficient. The adherence results varied across studies according to the type of data (continuous or binary) and the methodologic area being investigated (measure of adherence or patient recruitment), as specified in the hypotheses mentioned previously. Therefore, we used a measure of precision, the coefficient of variation to represent adherence results reported as continuous
1079
variables and the proportion adherent in the control group, when adherence was reported as a binary variable. For the patient recruitment methods, the power of the trial to detect an effect was used for both binary and continuous data, which was calculated by a proportion recruited, equal to the actual sample size in a trial divided by a theoretical sample size that was calculated using a 25% minimally important absolute difference. Analyses were completed in SPSS version 20, and P ! 0.05 was considered statistically significant.
3. Results 3.1. Methods of measuring adherence: literature review For our first question for the measures of adherence, our initial literature review of articles about measuring adherence retrieved 36 different types of measures of adherence. Twenty-four of these measures were selfreport questionnaires that often contained similar questions and generally had the same advantages and disadvantages of being easy to use and economical but obtrusive to patients and subjective. Twenty measures were considered to be ‘‘valid’’ in contexts in which validity had been tested. Three measures, including physician estimates of patient adherence, were considered to be invalid based on empirical assessments by the study author or by another study of that measure. Twenty-six measures of adherence were considered to be reliable. Twelve measures were considered objective, although only four of those are direct measures of adherence. Nine measures were considered unobtrusive. Most measures of adherence were capable of measuring adherence over a long period (longitudinality). However, self-reports were often only trustworthy when measuring adherence during the previous week, as longer periods might result in recall bias. Blood or urine drug concentration
Table 2. Categorization of features of patient recruitment methods into scale scores based on whether that feature is absent (no), uncertainly absent or present (uncertain), and present (yes) Feature
Score 0 (no)
Score 1 (uncertain)
Score 2 (yes)
Nonadherent patients selected
No (no mention of past adherence in any capacity, assume both adherent and nonadherent patients recruited)
Yes (adherence was measured before study inclusion, and only those with low adherence were included in the study, or those with low adherence to begin with are separately reported)
Representativeness of sample (number recruited/number screened who met eligibility criteria) Results reported based on baseline adherence
Number of patients asked to join is much greater than sample size (O2:1 nonenrolled to enrolled)
Indeterminate (eg, only included patients with high blood pressure or patients who had that physiologic state possibly due to a lack of adherence in the past but not explicitly stated that adherence was measured before selection) Does not report number of patients asked before reaching sample size Baseline adherence measured, but results were not reported according to initial adherence level
Yes, results were reported based on baseline adherence level or if only nonadherent patients are recruited, if intention-to-treat analysis is followed
Baseline adherence was not measured
The number of patients asked to join is similar to sample size (2:1 nonenrolled to enrolled)
1080
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
assessments were only capable of measuring short periods when the drug in question has a short half-life. 3.1.1. Methods of measuring adherence: analytic survey results Our analytic survey of 50 randomized controlled trials of interventions to improve adherence included 92 measures of adherence, as 54% of studies used more than one measure of adherence. We categorized these measures into 11 generic types of measures of adherence (classified in Table 3). The frequency of different measures, the quality per measure, and the median quality for each scale item for each measure in the 50 selected trials are summarized in Table 3 (detailed table of measures per study available in Supplementary Table S1/Appendix at www.jclinepi.com). The median overall quality of measures of adherence was 5 (interquartile range [IQR], 3) on a score ranging from 0 (low quality) to 9 (high quality), maximum possible score 9. Few studies included measures such as ‘‘attendance,’’ ‘‘clinician judgment,’’ ‘‘direct observation,’’ and ‘‘therapeutic response.’’ These infrequently used measures of adherence generally had low quality scores, although therapeutic response had a median quality score of 6 (of maximum 9). Other measures of adherence, such as medication event monitoring system (MEMS), pill counts, and pharmacy records, were used more frequently and were of higher quality than other measures. Self-reports were an anomaly in this regard, as they were used quite frequently but were of lower quality (median score of 3). Higher quality scores were associated with a larger variation in the estimate of adherence. Indeed, when adherence was reported as continuous data, the overall Spearman rank correlation coefficient between the quality of the measures of adherence and precision, assessed by the coefficient of
variation of adherence, was 0.66 (95% CI: 0.39, 0.83; P ! 0.001). This indicates that the quality of the measure may impact the adherence results that are found in a trial. Furthermore, as we predicted, higher quality measures of adherence are associated with larger variance because they more fully assess the actual range of adherence rates among patients, whereas lower quality subjective measures are biased toward overestimating adherence. When adherence was reported as binary data, Spearman correlation between the quality of the measures of adherence and the proportion adherent was 0.21 (95% CI: 0.44, 0.04, P 5 0.10), indicating a higher quality measure of adherence may be associated with a lower proportion adherent in a trial, although this relationship was not statistically significant. 3.2. Patient recruitment methods The overall median quality of patient recruitment methods was 2 (IQR, 1; maximum possible score was 6) (Table 4). Few studies (3 of 50) recruited only nonadherent patients, which is a methodologic step that is thought to be very important in determining the true effect of the intervention. Most studies in our sample recruited ‘‘representative groups’’ of patients. Only three studies reported their results based on baseline adherence. However, the overall Spearman correlation between the quality score and the percentage of sample size recruited in a study (power) was not statistically significant for either continuous or binary data (0.04 [95% CI: 0.38, 0.41], P 5 0.85; and 0.14 [95% CI: 0.13, 0.39], P 5 0.30, respectively).
4. Discussion We investigated the types and quality of methodology used in trials of interventions to improve patient
Table 3. Measurement qualities of all generic types of adherence measures from a sample of randomized trials of interventions to increase patient adherence
Measurea Attendance Clinician judgment Direct observation Drug concentration in body MEMS Pharmacy refill record Pill count Self-report questionnaire Self-report diary Self-report interview Therapeutic response
Frequency of use in sample of RCTs (n)
Median validity score (max 2) and rangeb
Median reliability score (max 2) and rangeb
2 1 2 6 16 14 20 17 3 8 3
0.5 (0e1) 0 1 1 2 (1e2) 1.5 (1e2) 1 (1e2) 1 (1e2) 1 1.5 (0e2) 1
0.5 (0e1) 0 1 1 (1e2) 2 (1e2) 1.5 (1e2) 1 (1e2) 1 (1e2) 1 1.5 (0e2) 1
Median objectivity score (max 2) and rangeb
2 2 0 0
0 0 2 2 (0e2) 2 (0e2) (0e1) 0 (0e1) 2
Median unobtrusive score (max 2) and rangeb
1.5 0 0
0 1
1 0 0 (1e2) (0e2) 2 (0e2) 0 0 (0e1) (1e2)
Median longitudinality score (max 1) and rangeb 1 1 0.5 (0e1) 1 1 1 1 0 (0e1) 1 0 (0e1) 1
Median total score (max 9) and IQR 3 1 4.5 7 6.5 8 5 3 3 4.5 6
(1) (0) (0.5) (0.75) (2.25) (2) (1.25) (2) (0) (3.25) (0.5)
Abbreviations: IQR, interquartile range; MEMS, medication event monitoring system; RCT, randomized controlled trials. a Measures are not meant to be compared against each other as the nature of each quality category (ie, validity, and so forth) is not comparable between measure types as criterion standards can differ. Some measures are from the same study as some studies used multiple measures of adherence. b Range is only included in brackets if a range is possible to include based on the data.
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
1081
Table 4. Quality of patient recruitment methods in a sample of randomized trials of interventions to increase patient adherence
Dimension of patient recruitment
Frequency of studies that did not fulfill quality dimension, n (%)
Frequency of studies that are unclear if quality dimension fulfilled, n (%)
Frequency of studies that fulfill that quality dimension, n (%)
Median score in this dimension and range
Nonadherent patients selected Representativeness of sample Reporting of results
36 (72) 8 (16) 30 (60)
11 (22) 22 (44) 17 (34)
3 (6) 20 (40) 3 (6)
0 (0e2) 1 (0e2) 0 (0e2)
adherence to medication. The types of measures of adherence used in trials of interventions to improve adherence ranged from poor quality, such as attendance, to higher quality, such as pharmacy refill records. Lower quality measures of adherence were used less frequently in the literature than those with overall higher median quality scores, such as pharmacy refill records and MEMS. For the analyses of the relationship between the quality of the methods and the results of the trials, there was a significant association between the quality of methods of measurement and precision of adherence rates, suggesting the poorer methods of measurement are biased. This bias is likely to be overestimating adherence, when subjective obtrusive methods such as patient self-report are used [19]. Multiple types of measures were used in the sample of 50 randomized controlled trials of interventions to improve adherence. These range from poor quality, such as clinician judgments, to higher quality, such as pharmacy refill records. Given the low quality of adherence measurement in some studies, it would appear that not all researchers are currently aware of the importance of this aspect of methodology in adherence trials. On the other hand, some trials used three, four, or five measures of adherence, as they noted the importance of composite measures in establishing a fuller, more detailed picture of adherence. The quality of measures of adherence appears to relate to the adherence results in trials that test interventions to improve adherence. This finding, although exploratory, further supports the importance of this methodologic aspect in trials. The patient recruitment methodologic quality also varied considerably. Few studies used baseline adherence rates as an inclusion criterion for participants, despite the potential importance of this step in establishing the true effect of an intervention. Our exploratory analysis was underpowered, however, and did not establish whether this methodologic step correlated with the power of the study or not. Although questions remain as to whether patient recruitment methods might impact adherence results or the power of the study to detect an effect, it is clear researchers do not often consider these aspects of methodology in their trials, as demonstrated by the low overall median quality score. This study is limited by several factors. The sample size for this analytic survey was chosen for feasibility reasons and did not allow for a highly powered analysis for the
exploratory research questions. However, these analyses were secondary to the main aim of the study, which was to describe and assess the quality of adherence measures in randomized controlled trials. The scale that was developed in this study to assess the quality of the methodology was not assessed for validity or reliability before applying it to these studies. However, the online data extraction form was extensively tested, and adjudicated forms were sent to authors for confirmation of our interpretation of their study, adding to the accuracy of the data. Trials were included in this study based on the type of measure of adherence used in the trial, selecting them purposively. This approach may have been biased, but the full set of articles included in the separate systematic review had not been identified and extracted at that point. Although a random sample, stratifying studies by the type of measure used in them, may have been the best approach with the least selection bias, this approach was not possible given the stage of the full review. Given these limitations, there are several future directions this area of research could take. Further testing of the scales that were used in this study to assess the quality of the methods used to determine the reliability and validity of the scale would be worthwhile. Although establishing a gold standard for measuring adherence would also be a useful next step, this may not be feasible given the diversity of diseases and medications that need to be measured. Instead, developing a quality threshold using the scale applied to measures in this study might be better suited to comparatively evaluate the quality of given measures of adherence. The methodologic steps outlined in this study could be applied to future adherence trials to ensure that the most accurate results are found. This process may reduce the confounding across trials and reduce the variability in efficacy of a given intervention. It may then reduce the heterogeneity across trials to allow for future meta-analyses of these studies. 5. Conclusion The state of the measurement of adherence and patient recruitment methods in randomized controlled trials to improve patient adherence to medication leaves considerable room for improvement. Using stronger methods of measurement currently available is likely to result in more effective and efficient testing of adherence interventions, based on the correlation we found between the quality of
1082
R.A. Jeffery et al. / Journal of Clinical Epidemiology 67 (2014) 1076e1082
the methods of measuring adherence and the coefficient of variation in randomized controlled trials. The lack of a gold standard for measuring adherence is a major issue that needs to be addressed in future research. Acknowledgments The members of the Patient Adherence Systematic Review Team included the Principal Investigator, Co-Investigators, Co-Applicants/Senior Management Decision-makers, CoApplicants/Clinical Service Decision-Makers, and Research Staff. The following were involved in collection and/or organization of data: Tamara Navarro, McMaster University; Emma Iserman, McMaster University; Dawn Jedras, McMaster University; Nancy Wilczynski, McMaster University; R Brian Haynes, McMaster University; Alfonso Iorio, McMaster University; Robby Nieuwlaat, McMaster University; Reem Mustafa, McMaster University; Susan Jack, McMaster University; Norma Brown, McMaster University; Fadi Bdair, State University of New York at Buffalo; and Nicholas Hobson, DiplT at McMaster University provided programming and information technology support.
Appendix Supplementary data Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jclinepi.2014.06.008. References [1] Krousel-Wood M, Thomas S, Muntner P, Morisky D. Medication adherence: a key factor in achieving blood pressure control and good clinical outcomes in hypertensive patients. Curr Opin Cardiol 2004; 19:357e62. [2] World Health Report 2003- Adherence to long-term therapies: evidence for action. Geneva, World Health Organization, 2003. ISBN 92 4 154599 2, 110 pages.
[3] Haynes RB, Ackloo E, Sahota N, Hp M, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2008;(2):CD000011. [4] Quittner AL, Modi AC, Lemanek KL, Ievers-Landis CE, Rapoff M. Evidence-based assessment of adherence to medical treatments in pediatric psychology. J Pediatr Psychol 2008;33(9):916e36. [5] Dunbar-Jacob J, Mortimer-Stephens MK. Treatment adherence in chronic disease. J Clin Epidemiol 2001;54(Suppl 1 (July)):S57e60. [6] DiMatteo MR, Giordani PJ, Lepper HS, Croghan TW. Patient adherence and medical treatment outcomes: a meta-analysis. Med Care 2002;40:794e811. [7] CVS Caremark. State of the States: adherence report. 2013: 1e108. Available at http://info.cvscaremark.com/cvs-insights/state-states2013. Accessed July 22, 2014. [8] Farmer KC. Methods for measuring and monitoring adherence in clinical trial and clinical practice. Clin Ther 1999;21:1074e90. [9] La Greca AM. Issues in adherence with pediatric regimens. J Pediatr Psychol 1990;15(4):423e36. [10] J€uni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ 2001;323:42e6. [11] Vermeire E, Wens J, Van Royen P, Biot Y, Hearnshaw H, Lindenmeyer A. Interventions for improving adherence to treatment recommendations in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2005;(2):CD003638. [12] Becker MH. Patient adherence to prescribed therapies. Med Care 1985;23:539e55. [13] Gross CP, Mallory R, Heiat A, Krumholz HM. Reporting the recruitment process in clinical trials: who are these patients and how did they get there? Ann Intern Med 2002;137:10e6. [14] Haynes RB, Dantes R. Patient compliance and the conduct and interpretation of therapeutic trials. Control Clin Trials 1987;8:12e9. [15] Gordis L. Conceptual and methodologic problems in measuring patient compliance. In: Haynes R, Sackett D, editors. Compliance in health care. Baltimore, MD: John Hopkins University Press; 1979: 23e45. [16] Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York, NY: Oxford University Press; 2003:4e226. [17] Norman GR, Streiner DL. Biostatistics: the bare essentials. Shelton, CT: People’s Medical Publishing House; 2008:55. [18] Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv 2008;1:211e7. [19] Haynes R, Taylor D, Sackett D, Gibson E, Bernholz C, Mukherjee J. Can simple clinical measurements detect patient noncompliance? Hypertension 1980;2:757e64.
