Research in Social and Administrative Pharmacy 12 (2016) 29–40
Original Research
An evaluation of three statistical estimation methods for assessing health policy effects on prescription drug claims Manish Mittal, Ph.D.a,*,1,2, Donald L. Harrison, Ph.D., F.A.Ph.A.a, David M. Thompson, Ph.D.b, Michael J. Miller, R.Ph., Dr.P.H., F.A.Ph.A.c, Kevin C. Farmer, Ph.D., F.A.Ph.A.a, Yu-Tze Ng, M.D., F.R.A.C.P.d,3 a
Department of Pharmacy, Clinical and Administrative Sciences, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA b Department of Biostatistics and Epidemiology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA c Department of Pharmacy, Clinical and Administrative Sciences, The University of Oklahoma Health Sciences Center, Tulsa, OK, USA d Department of Neurology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA
Abstract Background: While the choice of analytical approach affects study results and their interpretation, there is no consensus to guide the choice of statistical approaches to evaluate public health policy change. Objectives: This study compared and contrasted three statistical estimation procedures in the assessment of a U.S. Food and Drug Administration (FDA) suicidality warning, communicated in January 2008 and implemented in May 2009, on antiepileptic drug (AED) prescription claims. Methods: Longitudinal designs were utilized to evaluate Oklahoma (U.S. State) Medicaid claim data from January 2006 through December 2009. The study included 9289 continuously eligible individuals with prevalent diagnoses of epilepsy and/or psychiatric disorder. Segmented regression models using three estimation procedures [i.e., generalized linear models (GLM), generalized estimation equations (GEE), and generalized linear mixed models (GLMM)] were used to estimate trends of AED prescription claims across three time periods: before (January 2006–January 2008); during (February 2008–May 2009); and after (June 2009–December 2009) the FDA warning. Results: All three statistical procedures estimated an increasing trend (P ! 0.0001) in AED prescription claims before the FDA warning period. No procedures detected a significant change in trend during (GLM: 30.0%, 99% CI: 60.0% to 10.0%; GEE: 20.0%, 99% CI: 70.0% to 30.0%; GLMM: 23.5%, 99% CI: 58.8% to 1.2%) and after (GLM: 50.0%, 99% CI: 70.0% to 160.0%; GEE:
1 I ‘Manish Mittal’ am currently an employee of the Abbvie. However, during the conduct of this research I was a doctoral student at the OUHSC. Abbvie is in no manner associated with this research. 2 Present address: Health Economics and Outcomes Research, Abbvie, North Chicago, IL, USA. 3 Present address: Department of Pediatrics, Baylor College of Medicine, San Antonio, TX, USA. * Corresponding author. Dept. GMH1, AP31-1 NE, 1 North Waukegan Road, North Chicago, IL 60064, USA. Tel.: þ1 847 935 9190, þ1 917 379 3390(mobile); fax: þ1 847 937 1992. E-mail address: [email protected] (M. Mittal).
1551-7411/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.sapharm.2015.03.004
30
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
80.0%, 99% CI: 20.0% to 200.0%; GLMM: 47.1%, 99% CI: 41.2% to 135.3%) the FDA warning when compared to pre-warning period. Conclusions: Although the three procedures provided consistent inferences, the GEE and GLMM approaches accounted appropriately for correlation. Further, marginal models estimated using GEE produced more robust and valid population-level estimations. Ó 2016 Elsevier Inc. All rights reserved. Keywords: Generalized linear model; Generalized estimation equations; Generalized linear mixed models; FDA alert; Antiepileptic drugs
Introduction Studies evaluating health care regulatory actions are common. However, certain types of data, in particular longitudinal data, require health service researchers to use statistical procedures that obtain robust and valid estimates to provide accurate assessments of the outcomes of these actions.1,2 There is no consensus to guide the choice of statistical approaches to formally evaluate a public health policy change.3,4 For example, some studies have used relatively simple generalized linear models (GLM), which use maximum likelihood estimation (MLE), to estimate policy effect without accounting for correlation induced by repeatedly measuring observations from the same individual over time.5–11 Analyzing longitudinal data without taking into account the correlation between the outcomes12 causes standard errors to be underestimated or overestimated, which may lead to making either a type I or II error.13 Ultimately, ignoring correlation produces inaccurate inferences about the effect of policies evaluated. Generalized estimation equations (GEE) calculate standard errors for their parameter estimates by incorporating a “sandwich estimator”.14 Because the parameter estimates are robust, GEE estimation is gaining popularity among health service researchers15–17 although it has not been widely used to evaluate FDA policy changes.4 While GEE estimates a population-level policy effect, it does not attempt to quantify heterogeneity of responses in the effect across individual subjects. In contrast, the generalized linear mixed model (GLMM), which uses a pseudo-likelihood estimation technique, accounts for autocorrelation via the introduction of random effect and allows for subject-specific inferences.18 Because of limited accessibility to the software, GLMM has not been commonly employed by health service researchers.19
The three aforementioned analytical approaches may yield different results and subsequent policy interpretations when applied to the same data to answer a common research question. It is known that the absolute magnitude of the parameter estimates derived from GLMM are generally larger than those derived from GEE estimation.20 In January 2008, the U.S. FDA issued an alert, followed later (May 2009) by a warning of an increased risk of suicidality, defined as suicidal ideation and behavior, among users of antiepileptic drugs (AEDs).21 Using this policy change as an example case, the objective of this study was to compare and contrast the results and conclusions of three estimation procedures (i.e., GLM, GEE and GLMM) used to assess the association between the FDA suicidality warning and AED prescription claims among Oklahoma Medicaid individuals diagnosed with epilepsy and/or psychiatric disorder(s), from 2006 through 2009. Oklahoma is a state located in south-central U.S., and Medicaid is a federal program providing medical insurance primarily to indigent, and to other populations. Methods Study design A longitudinal segmented regression analysis of Oklahoma Medicaid claims data from January 2006 through December 2009 was used to evaluate the change in AED prescription claims before and after the FDA suicidality alert and warning. A time-series of 48 consecutive months was created using person level data as a unit of analysis. For each month, the proportion of individuals with an AED prescription claim was calculated. This study was reviewed and approved by the Institutional Review Board at the University of Oklahoma Health Sciences Center.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
31
Study population
Study variables
Fig. 1 depicts the individual inclusion and exclusion process. There were 1,183,668 individuals enrolled in the Oklahoma Medicaid program between January 2006 and December 2009. A group of 14,881 continuously eligible individuals was selected who were less than 65 years of age, not dually eligible for Medicaid and Medicare, had a diagnosis of epilepsy and/or psychiatric disorder(s) in an inpatient or outpatient setting, and had an AED prescription claim in at least one of the 48 months between January 2006 and December 2009 (Table 1). The date of the first AED prescription claim during the study period was considered the AED prescription index date. Individuals were excluded if they had received a diagnosis of attempted suicide (ICD-9 CM: E950-E959) or received polytherapy (2 or more AED drugs, indicating more severely ill group) within six months prior to the AED prescription index date. To ensure standardized length of AED exposure time among study participants and to reduce bias related to the time-varying nature of an AED prescription fill, 9289 individuals were selected for the final analytical cohort who had prevalent (existing) diagnoses of epilepsy and/or psychiatric disorder on or prior to January 2006 (Fig. 1).
Dependent variable For each individual, a supply diary of AEDs was created by adding together consecutive prescription fills of AEDs based on dispensing dates and the reported days’ supply.22 An individual was considered to have an AED prescription claim for any month during which the diary showed at least 15 days of AED therapy. Because of the binomial distribution of the outcome, the log odds of filling an AED prescription claim for at least 15 days at each monthly time point was calculated in the analytical cohort. Independent variables Regression models included parameters that defined two interruptions introduced during the study period. The first interruption was the month when the FDA alert was issued (January 2008). The second interruption was the month when the FDA warning was issued (May 2009). While this study focused primarily on the periods immediately before and after the introduction of the FDA warning, these two interruptions created 3 segments in a time-series of 48 months: (1) before (January 2006–January 2008); (2) during (February 2008–May 2009); and (3) after (June 2009–December 2009) the FDA warning to
Fig. 1. Inclusion and exclusion criteria for the Study Population. E ¼ Patients with epilepsy alone; E þ P ¼ Patients with epilepsy and comorbid psychiatric disorder; P ¼ Patients with psychiatric disorder alone.
32
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Table 1 Diagnostic groups per International Classification of Diseases, 9th Revision, Clinical Modification (ICD9-CM) Codes and list of antiepileptic drugs (AEDs) included in the study Factors I. Diagnoses (a) Epilepsy alone16 (b) Psychiatric disorder alone
(c) Epilepsy and comorbid psychiatric disorder II. Antiepileptic drugs (AEDs)
Categories (ICD9) Epilepsy (345.xx) Convulsion (780.3x) Psychiatry disorder (295–319) - Depression (296.2x, 296.3x, 300.4, 311) - Bipolar disorder (296.0x, 296.1x, 296.4x-296.8x) - Schizophrenia (295.xx) - Anxiety/Phobia disorder (300.0x, 300.2x, 300.3) - Adjustment disorder (309.xx) - ADHD (314.0x) - Other psychiatric disorder Patients diagnosed with both medical conditions Carbamazepine, Clonazepam, Clorazepam, Divalproex Sodium, Valproic acid, Ethosuximide, Ethotoin, Mephenytoin, Methsuximide, Phenytoin, Primidone, Lamotrigine, Gabapentin, Pregabalin, Oxcarbazepine, Levetiracetam, Tiagabine, Topiramate, Vigabatrin, Felbamate, Zonisamide
more explicitly account for a potential lag period between implementation of the FDA warning and physicians’ prescribing response to it. Covariates The selection of covariates, available in the Medicaid dataset, was guided by the Andersen Behavioral Model for Access to Medical Care and included baseline predisposing (i.e., individuals’ propensity to use services), need (i.e., illness level), and enabling (i.e., individuals’ ability to access services) factors that have been identified to potentially influence the use of health services.23 Predisposing characteristics included age group, sex, and race. Need characteristics included diagnostic groups,24 neurological comorbidities,25–28 other comorbid conditions,29,30 and use of psychotropic drugs. Enabling characteristics included physician’s specialty, geographical region, and metropolitan statistical area (MSA) status (Table 2). Statistical analysis Univariate and descriptive statistics were used to profile all study covariates. Because of the outcome’s binomial distribution, a linear regression line was fit on a log odds (logit) scale. Segmented logistic regression models31 were employed to estimate change in level (intercept) and trend (slope) in log odds of AED prescription claims during and after the FDA warning with
the pre-warning period as the referent. Each model was adjusted for each of the predisposing, enabling and need characteristics. Three statistical estimation procedures i.e., (a) GLM, (b) GEE and, (c) GLMM, were used to account for autocorrelation between repeated observations as well as heterogeneity across individuals in different manners. Autocorrelation between repeated observations was detected using correlograms (residuals versus time)32 and the Durbin–Watson (DW) test statistics.33 Data management and analysis was accomplished via PCSAS (v9.2), with an a-priori alpha set at P ! 0.01. The longitudinal results (i.e., point estimates, width of 99% CI and P-values of change in trend of log odds of AED prescription claim), attained by all three estimation procedures were compared. Because no standard estimation or hypothesis test exists to quantitatively evaluate the difference in results attained by the three estimation procedures, the direction, magnitude and statistical significance of the results (trend estimates) were reviewed and compared for consistency. Moreover, the predicted segmented trajectories produced by the three models were graphed.
Results Baseline predisposing, enabling and need characteristics of the study population are presented in
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40 Table 2 Descriptive statistics for predisposing, enabling, and need characteristics of the study population from January 2006 through December 2009 (N ¼ 9289) Characteristics
Study population, N ¼ 9289 (%)
Predisposing characteristics Age !18 3671 (39.5%) 18–44 2936 (31.6%) 45–64 2682 (28.9%) Sex Female 5212 (56.1%) Male 4077 (43.9%) Race White 6991 (75.3%) Black 1209 (13.0%) Othera 1089 (11.7%) Need characteristics Diagnostic groups Epilepsy alone 345 (3.7%) Epilepsy & comorbid 3162 (34.0%) psychiatric disorder Psychiatric disorder alone 5782 (62.3%) Neurological comorbiditiesb (Neuropathic pain, migraine, Movement disorder, or chronic Pain) Yes 3799 (40.9%) No 5490 (59.1%) Other comorbiditiesc (Charlson comorbiditity score) 0 3325 (35.8%) R1 5964 (64.2%) Use of psychotropic therapy (Antidepressant, antipsychotic, stimulants, anxiolytic/hypnotics) Yes 7125 (76.7%) No 2164 (23.3%) Enabling characteristics CNS health care providerd Yes 3026 (32.6%) No 6263 (67.4%) G.Region NW 598 (6.4%) SW 905 (9.7%) OKC 2621 (28.2%) Tulsa 1779 (19.2%) NE 1574 (16.9%) SE 1742 (18.8%) MSA Urban 5156 (55.5%) Rural 4004 (43.1%) Age, sex, race, use of psychotropic therapy, physician specialty, geographical region, and MSA were measured at the AED prescription index date. Diagnostic groups, neurological comorbidities and other comorbidities were measured during the entire study period. Percentage of geographic region and MSA did not add up to 100% because of missing data. a Other races included asian, native hawaiian/other pacific islander, american indian/alaska native, multiple races.
33
Table 2. Overall, the percentage of individuals with an AED prescription claim consistently increased, from 38.36% to 46.90%, between January 2006 and December 2009. For the group of individuals with diagnosis of epilepsy alone, the percentage of individuals with an AED prescription claim actually declined from 72.17% to 58.55%. However, the pattern steadily increased in individuals with diagnoses of epilepsy and comorbid psychiatric disorder(s) trending upward from 53.42% to 63.47% and in individuals with psychiatric disorder(s) alone trending from 28.10% to 37.15%. Distinct, non-random regions of negative and positive residuals for the correlogram and a significant DW statistic (1.4672, P ¼ 0.0203) indicated a positive autocorrelation between repeated measures obtained from the same individual in the Medicaid claims data. Fig. 2 illustrates the segmented trajectories for AED prescription claims, predicted by the three statistical estimation procedures, for a subgroup of white males, under the age of 18 years, who lived in OKC and urban areas, and who were without neurological comorbidity, other comor-
b Neurological comorbidities included a list of four chronic conditions commonly prescribed with AEDs i.e., Neuropathic pain [diabetes with neurological manifestations (250.6x), trigeminal nerve disorders (350.xx), glossopharyngeal neuralgia (352.1x), neuropathy (356.0x, 356.8x), postherpatic trigeminal neuralgia (053.12), and unspecified neuralgia, neuritis and radiculitis (729.2x)]; Migraine (346.xx); Movement disorder [essential tremor (331.1x, 781.0x), restless legs syndrome (333.94)]; and Chronic pain (338.2x, 729.1x).2,16–18 c Other comorbid conditions were adjusted by measuring the Charlson comorbid score indicating the severity of the disease.19,20 d Physicians’ specialty indicated whether or not a physician was a central nervous system (CNS) health care provider who had prescribed the first AED prescription to individual. A list of CNS health care providers included psychiatric hospital, intermediate care facility/mental health (ICF/MH) O 6 beds, intermediate care facility/mental health (ICF/MH) ! 6 beds, outpatient mental health clinic, community mental health center (CMHC), psychologist, pre-admission screening and resident review (PASRR) CMHC, health service provider in psychology (HSPP), mental health – department of mental health and substance abuse services (DMHSAS), mental health case management all ages (public and private), neurological surgeon, neurologist, psychiatrist, psychiatry child.
34
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Fig. 2. Predicted mean monthly percentage of study individuals with AED prescription claims produced by three estimation procedures over the 48 months from January 2006 through December 2009. (a). Total study population (N ¼ 9289). (b). Individuals with epilepsy alone (n ¼ 345). (c). Individual with epilepsy and comorbid psychiatric disorder (n ¼ 3162). (d). Individual with psychiatric disorder alone (n ¼ 5782). Predicted mean monthly percentage of study individuals with AED prescription claims are displayed for a single subgroup: individual with age !18 years, male, white, epilepsy diagnosis alone, no neurological comorbidity, no other comorbid condition, no use of psychotropic drugs, prescribed by non-CNS health care provider, lives in OKC and urban area.
bid condition, or use of psychotropic drugs, and whose AEDs were prescribed by non-CNS health care providers. The figure’s three panels distinguish among diagnostic groups. Table 3 details the parameter estimates that underlie the predicted trajectories illustrated in Fig. 2. 1. Direction of point estimates: The directional sign of point estimates from the three estimation procedures were consistent for each warning period. 2. Magnitude of point estimates: For all three warning periods, GLM and GEE estimation procedures provided point estimates of similar magnitude observed by visual inspection. In contrast, the magnitudes of point estimates attained by the GLMM were larger than those calculated in the other estimation procedures. 3. Precision of point estimates: Table 3 illustrates that all three statistical procedures estimated an increasing trend (P ! 0.0001) in log odds of AED prescription claims before the FDA warning period. None of the procedures detected a significant change in estimated trend both during (GLM: 30.0%, 99%
CI: 60.0% to 10.0%; GEE: 20.0%, 99% CI: 70.0% to 30.0%; GLMM: 23.5%, 99% CI: 58.8% to 1.2%) and after (GLM: 50.0%, 99% CI: 70.0% to 160.0%; GEE: 80.0%, 99% CI: 20.0% to 200.0%; GLMM: 47.1%, 99% CI: 41.2% to 135.3%) the FDA warning period when compared to pre-warning period. Although none of the three estimation techniques detected associations between changing trends in AED prescription claims and the FDA suicidality warning, the GLMM produced the most precise confidence intervals and the GEE produced the least precise confidence interval among all three estimation procedures. 4. Predicted segmented trajectories: Fig. 2(a) illustrates that predicted mean monthly percentages of overall study individuals with AED prescription claims attained by the three estimation procedures follow similar trajectory, indicating a consistency in the direction of point estimates. However, while using the same data, a discrepancy in the magnitudes of point estimates predicted from three estimation procedures is evident. Magnitude produced from GLM (dotted line) and GEE
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
(squared line) were very much overlapping each other; and the magnitude produced from GLMM (triangle line) were generally larger compared to other estimation procedures. Predicted segmented trajectories were graphed (Fig. 2) for each diagnostic group. For individuals with epilepsy alone and individuals with epilepsy and comorbid psychiatric disorder(s), GLMM (triangle line) predicted higher percentage estimates of AED prescription claims compared to other estimation procedures (Fig. 2(b and c)). In contrast, in individuals with psychiatric disorder(s) alone, the GLMM (triangle line) predicted lower percentage estimates of AED prescription claims compared to the other two procedures, as presented in Fig. 2(d).
Discussion Although all three procedures estimated trajectories whose changes in level and trend were in the same directions, and were of similar significance, as expected, the magnitude of point estimates and precision of confidence interval varies. In general, it appears that the GLM produced a smaller magnitude of point estimates and more precise confidence intervals and the GEE estimation produced similar magnitude of point estimates but relatively less precise confidence intervals. Finally, the GLMM produced a relatively larger magnitude of point estimates and relatively more precise confidence intervals. The inconsistency in the magnitude of trend estimates and confidence interval width may be due to different approaches by the three statistical estimation procedures. GLM completely ignores the correlation between repeated observations obtained from the same individual in longitudinal data and assumes single observation for each individual.34 For example, in this study, the observations of AED prescription claims were collected at each month over a period of 48 months from each (total of 9289) individual. Because of the independence assumption, the GLM estimates the odds of AED prescription claims for a total of 9289 48 ¼ 445,872 observations. A large sample size may potentially lead to discernible underestimations of the variability of the point estimate. This, in turn, results in confidence intervals that are too narrow which may further lead to making a type I error (rejecting the true null hypothesis).13
35
As an alternative to the ML approach that is customarily used to estimate linear models, GEE accounts for autocorrelation among longitudinal data14 and produces more robust and valid estimates at the population-level.35 A notable feature of the GEE is that it separately models the regression parameters and covariance structure among repeated observations.36 This separation ensures that the interpretation of the regression coefficients does not rely on the assumed structure for the covariance among the responses. In our example, accounting for autocorrelation in the Medicaid data, is potentially why the GEE produced point estimates similar in magnitude to the GLM, but relatively less precise, unbiased confidence intervals. The GEE and GLMM are two alternative approaches that account for the autocorrelation among longitudinal data. However, for the same Medicaid data, these two estimation procedures also provided different results in terms of magnitude and precision of the estimates. This difference may be due to the fact that these two estimation procedures have different objectives for making inferences. Ultimately these two approaches may subtly address different scientific questions.36 When applied to a dichotomous outcome, the GEE estimation makes inferences at the population (mean) level and does not account for heterogeneity across individuals.36 The GEE approach yields an estimate for any individual randomly selected from the population without accounting for subject-specific propensity to respond, in this case, to file a claim for an AED, this is also known as “population averaged” or marginal model.37 The GEE indicates that the model for the AED prescription claims depends only on the covariates of interest and not on any random effect across individuals. In conclusion, this procedure should be of interest to health service researchers who consider evaluating the potential impact of an intervention at the population level. However, the basic premise underlying the GLMM is the assumption of heterogeneity across individuals in the study population. The GLMM model considers the odds of filling an AED prescription claim as conditioned on subjectspecific random intercepts. These intercepts reflect a natural heterogeneity, due to unmeasured factors, in individuals’ propensity to fill an AED prescription claim. For example, the model conditioned on individuals with a high propensity to fill AED prescription claims, would estimate a
Diagnostic groups
Epilepsy alone (n ¼ 345)
Baseline level (b0) Baseline trend (b1) Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0) Baseline trend (b1)
Epilepsy þ comorbid psychiatric disorder(s) (n ¼ 3162)
Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0) Baseline trend (b1) Level change at FDA alert (b2)
GLM
GEE
GLMM
Point estimates (99% CI)
a
Change in log odds of AED Rx claims % (99% CI)
Point estimates (99% CI)
a
Change in log odds of AED Rx claims % (99% CI)
Point estimates (99% CI)
Change in log odds of AED Rx claims % (99% CI)
0.63** (0.57–0.68) 0.01** (0.008–0.011) 0.02 (0.06 to 0.02) 0.003 (0.006 to 0.001) 0.05 (0.11 to 0.01) 0.005 (0.007–0.016) 1.09** (0.92–1.26) 0.01** (0.02 to 0.005) 0.07 (0.27 to 0.13) 0.002 (0.02 to 0.02) 0.03 (0.32 to 0.25)
Referent
0.62** (0.39–0.84) 0.01** (0.008–0.013) 0.04 (0.07 to 0.002) 0.002 (0.007 to 0.003) 0.09** (0.14 to 0.05) 0.008 (0.002–0.02) 1.08** (0.52–1.65) 0.01 (0.03 to 0.001) 0.06 (0.26 to 0.14) 0.002 (0.02–0.03) 0.09 (0.31 to 0.13)
Referent
1.27** (0.93–1.62) 0.017** (0.015–0.019) 0.04 (0.09 to 0.01) 0.004 (0.01 to 0.0002) 0.09* (0.16 to 0.01) 0.008 (0.007–0.023) 2.21** (1.20–3.22) 0.02** (0.04 to 0.01) 0.13 (0.40 to 0.14) 0.002 (0.02–0.03) 0.07 (0.47 to 0.32)
Referent
0.002 (0.06 to 0.06) 0.62** (0.57–0.68) 0.013** (0.01–0.015) 0.04 (0.10 to 0.02)
Referent 3.2% (9.5–3.2) 30.0% (60.0–10.0) 7.9% (17.5–1.6) 50.0% (70.0–160.0) Referent Referent 6.4% (24.8–11.9) 20.0% (200.0–200.0) 2.7% (29.4–22.9) 20.0% (600.0–600.0) Referent Referent 6.5% (16.1–3.2)
0.009 (0.03–0.05) 0.61** (0.42–0.81) 0.013** (0.01–0.02) 0.06 (0.13 to 0.002)
Referent 6.5% (11.3 to 0.3) 20.0% (70.0–30.0) 14.5% (22.6 to 8.1) 80.0% (20.0–200.0) Referent Referent 5.5% (24.1–13.0) 20.0% (200.0–300.0) 8.3% (28.7–12.0) 90.0% (300.0–500.0) Referent Referent 9.8% (21.3 to 0.3)
0.008 (0.07–0.09) 1.11** (0.77–1.45) 0.023** (0.019–0.027) 0.07 (0.15 to 0.02)
a
Referent 3.2% (7.1–0.8) 23.5% (58.8–1.2) 7.1% (12.6 to 0.8) 47.1% (41.2–135.3) Referent Referent 5.9% (18.1–6.3) 10.0% (100.0–150.0) 3.2% (21.3–14.5) 40.0% (350.0–450.0) Referent Referent 6.3% (13.5–1.8)
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Epilepsy and/or psychiatric disorder(s) (N ¼ 9289)
Level/trend
36
Table 3 Segmented regression analysis to evaluate the change in AED Prescription claims among the study population in the Oklahoma Medicaid setting from January 2006 through December 2009
Baseline trend (b1) Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5)
0.005 (0.011 to 0.001) 0.05 (0.14 to 0.05) 0.0008 (0.02 to 0.02) 0.67** (0.72 to 0.62) 0.009** (0.007–0.012) 0.01 (0.06 to 0.04) 0.002 (0.01 to 0.003) 0.05 (0.12 to 0.02) 0.007 (0.01–0.02)
38.5% (84.6–7.7) 8.1% (22.6–8.0) 6.2% (153.8–153.8) Referent Referent 1.5% (9.0–6.0) 22.2% (111.1–33.3) 7.5% (17.9–3.0) 77.8% (111.1–222.2)
0.005 (0.013 to 0.003) 0.09* (0.17 to 0.01) 0.002 (0.02 to 0.02) 0.66** (0.86 to 0.47) 0.01** (0.006–0.013) 0.02 (0.07 to 0.03) 0.0003 (0.01 to 0.01) 0.1** (0.16 to 0.04) 0.01 (0.001–0.025)
38.5% (100.0–23.0) 14.8% (27.9 to 1.6) 15.4% (153.8–153.8) Referent Referent 3.0% (10.6–4.5) 3.0% (100.0–100.0) 15.2% (24.2 to 6.0) 100.0% (10.0–250.0)
0.011* (0.019 to 0.003) 0.03 (0.16 to 0.1) 0.006 (0.03 to 0.02) 0.95** (1.23 to 0.67) 0.016** (0.013–0.018) 0.01 (0.07 to 0.05) 0.002 (0.008 to 0.004) 0.09 (0.18 to 0.003) 0.01 (0.007–0.03)
47.8% (82.6 to 13.0) 2.7% (14.4–9.0) 26.1% (130.4–87.0) Referent Referent 1.1% (7.4–5.3) 12.5% (50.0–25.0) 9.5% (18.9 to 0.3) 62.5% (43.8–187.5)
*P ! 0.01, **P ! 0.0001. Bold indicates an area where there appears to be some inconsistencies between the various estimation techniques. The models were estimated using PROC LOGISTIC/GENMOD/GLIMMIX (SAS 9.2) with a binomial distribution logit function. The correlation structure was AR(1). The overall model was adjusted for a number of covariates included age, sex, race, diagnostic groups, neurological comorbidities, other comorbidities, use of psychotropic therapy, physician specialty, geographical region, and MSA. All three stratified diagnosis-specific models were adjusted for a number of covariates excluding diagnostic groups. Calculation of 99% CI: (a) Lower bound ¼ 0.007 100/0.01 ¼ 70.0% (b) Upper bound ¼ 0.003 100/0.01 ¼ 30.0%. % Change in log odds of AED Rx claims during the FDA warning period was 20.0% (99% CI: 70.0 to 30.0). Other calculations were performed in a similar manner. Negative sign indicates the decline in log odds of AED Rx claims and was not included in the mathematical calculation. a Example of calculating change in log odds of AED Rx claims %: % Change in trend of log odds of AED Rx claims during the FDA warning (b3) in patients with epilepsy and/or psychiatric disorder(s) (N ¼ 9289) using GEE ¼ Trend change during the FDA warning period/Baseline trend in pre-warning period ¼ b3 100/ b1 ¼ 0.002 100/0.01 ¼ 20.0%.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Psychiatric disorder(s) alone (n ¼ 5782)
Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0)
37
38
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
different trajectory for claims than the model conditioned on individuals with low propensity to fill AED prescription claims. Therefore, the GLMM permits the research to make subjectspecific inferences. Consequently, there was, as expected, a large difference in the magnitude of point estimates between the GLMM and the marginal models as presented in Fig. 2(a). Because the main objective of the GLMM analytical approach is to make inferences at the subject-level, the magnitude of point estimates may potentially be influenced by subjects included in the analytical sample. This rationale was strengthened by examining the graphs predicting segmented trajectories by individual diagnostic groups. Individuals with epilepsy alone and individuals with epilepsy and comorbid psychiatric disorder(s), represent a more severely affected group of patients in terms of medical illness who have no alternative pharmacotherapy available for treating their epilepsy and may be more dependent or committed to the use of AEDs. For these two diagnostic groups, GLMM predicted higher percentage estimates of AED prescription claims compared to other estimation procedures. In contrast, individuals with psychiatric disorder(s) alone may be less dependent or committed to AED therapy, than individuals in the other two diagnostic groups, because there are alternative treatments beyond AEDs for psychiatric disorders. Hence, Fig. 2(d) depicts the lower predicted percentage estimates of AED prescription claims produced by the GLMM when compared to other procedures. Therefore, we may conclude that the GLMM yields point estimates that address different scientific questions that will be of most interest to the individual and his/her physician within the physicianpatient context. Health service researchers might appropriately use any of the three approaches depending on the general scientific questions they wish to address. It is known that the GLM is the most powerful estimation model in a cross-sectional study where the response is measured at a single occasion (no repeated observations). The GEE and GLMM extend the GLM approach to longitudinal studies by accounting for autocorrelation among repeated observations. With time-stationary covariates (e.g., gender) or with time-varying covariates that are fixed by study design (e.g., age), GEE is useful in a type of setting where all covariates are between-subject variables and time-invariant and, as in this study, inferences
can be made at the population-level.14,20 The GLMM, which produces subject-specific interpretation, is far more natural for covariates that vary within an individual.36 As a result, the GLMM is most useful when the main scientific objective is to make inferences about individuals with specific characteristics, rather than overall population averages. This study has important limitations to consider. The AED related suicidality issue was first raised in 2004; but reliable administrative claims data were not available before January 1, 2006 in the Oklahoma Medicaid dataset. The time period included after the FDA warning was very short; potentially too short to measure a potential long-term change in outcome measures. There may be several other factors, including advertising by pharmaceutical manufacturers, communications by professional organizations, publication of new safety and effectiveness information, and media reports that could affect the utilization pattern of AEDs during that period. These external factors could not be accounted for in this study. Missing data is always a potential problem, especially in research employing claims data.
Conclusion Analytical approaches follow certain assumptions about the response variable, and failing to account for these assumptions may lead to inefficient regression estimates, biased standard errors and/or invalid inferences.38 Among the three statistical estimation procedures in this study, generalized estimation equations (GEE) appears to produce more robust and valid population-level estimates in assessing the association between AED prescription claims and FDA suicidality warning in an Oklahoma Medicaid population. The GEE yields estimates that will be of most interest to health service researchers and policy makers who evaluate drug policy change at the population level.
Acknowledgment No financial or material support was received by any of the authors in the conduction of this research or the preparation of this manuscript. The principal author takes full responsibility for the data, the analyses and interpretation, and the conduct of the research; full access to all of the data; and the right to publish any and all data.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
We thank Nancy Nesser, PharmD, JD, Pharmacy Director, Oklahoma Health Care Authority, for assisting in accessing the Oklahoma Medicaid administrative data and Shellie Keast, PhD, Clinical Assistant Professor, Pharmacy Management Consultants, for guidance in data analysis. References 1. Fitzmaurice GM, Ravichandran C. A primer in longitudinal data analysis. Circulation 2008;118:2005– 2010. 2. Lagarde M. How to do (or not to do) . Assessing the impact of a policy change with routine longitudinal data. Health Policy Plan 2012;27:76–83. 3. Dusetzina SB, Higashi AS, Dorsey ER, et al. Impact of FDA drug risk communications on health care utilization and health behaviors: a systematic review. Med Care 2012;50:466–478. 4. Briesacher BA, Soumerai S, Zhang F, et al. In: MiniSentinel White Paper: Methods to Evaluate the Impact of FDA Regulatory Actions March 12, 2012;2012. 5. Busch SH, Frank RG, Leslie DL, et al. Antidepressants and suicide risk: how did specific information in FDA safety warnings affect treatment patterns? Psychiatr Serv 2010;61:11–16. 6. Hassanin H, Harbi A, Saif A, Davis J, Easa D, Harrigan R. Changes in antidepressant medications prescribing trends in children and adolescents in Hawai’i following the FDA black box warning. Hawaii Med J 2010;69:17–19. 7. Weatherby LB, Nordstrom BL, Fife D, Walker AM. The impact of wording in “Dear doctor” letters and in black box labels. Clin Pharmacol Ther 2002;72: 735–742. 8. Weatherby LB, Walker AM, Fife D, Vervaet P, Klausner MA. Contraindicated medications dispensed with cisapride: temporal trends in relation to the sending of ‘Dear Doctor’ letters. Pharmacoepidemiol Drug Saf 2001;10:211–218. 9. Burkhart GA, Sevka MJ, Temple R, Honig PK. Temporal decline in filling prescriptions for terfenadine closely in time with those for either ketoconazole or erythromycin. Clin Pharmacol Ther 1997;61: 93–96. 10. Shatin D, Gardner JS, Stergachis A, Blough D, Graham D. Impact of mailed warning to prescribers on the co-prescription of tramadol and antidepressants. Pharmacoepidemiol Drug Saf 2005;14:149–154. 11. Willy ME, Manda B, Shatin D, Drinkard CR, Graham DJ. A study of compliance with FDA recommendations for pemoline (Cylert). J Am Acad Child Adolesc Psychiatry 2002;41:785–790. 12. Agresti A. An Introduction to Categorical Data Analysis. 2nd ed. John Wiley & Sons, Inc; 2007. 13. Velicer WF, Colby SM. A comparison of missingdata procedures for arima time-series analysis. Educ Psychol Meas 2005;65:595–615.
39
14. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics 1988;44:1049–1060. 15. Mittal M, Harrison DL, Miller MJ, Farmer KC, Thompson DM, Ng YT. Have antiepileptic drug prescription claims changed following the FDA suicidality warning? An evaluation in a state Medicaid program. Epilepsy Behav 2014;34:109–115. 16. Dusetzina SB, Busch AB, Conti RM, Donohue JM, Alexander GC, Huskamp HA. Changes in antipsychotic use among patients with severe mental illness after a Food and Drug Administration advisory. Pharmacoepidemiol Drug Saf 2012 Dec;21(12): 1251–1260, http://dx.doi.org/10.1002/pds.3272. Epub 2012 May 3. 17. Valluri S, Zito JM, Safer DJ, Zuckerman IH, Mullins CD, Korelitz JJ. Impact of the 2004 Food and Drug Administration pediatric suicidality warning on antidepressant and psychotherapy treatment for new-onset depression. Med Care 2010;48:947–954. 18. The GLIMMIX Procedure: PROC GLIMMIX Contrasted with Other SAS Procedures. Cary, NC, USA: SAS Institute Inc; 2013. 19. Morrato EH, Druss B, Hartung DM, et al. Metabolic testing rates in 3 state Medicaid programs after FDA warnings and ADA/APA recommendations for second-generation antipsychotic drugs. Arch Gen Psychiatry 2010;67:17–24. 20. Neuhaus JM, Kalbfleisch JD, Hauck WW. A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data. Int Stat Rev 1991;59:25–35. 21. Machado P, Landewe R. Spondyloarthritis: Is it time to replace BASDAI with ASDAS? Nat Rev Rheumatol 2013;9:388–390. 22. Leslie SR. Using Arrays to Calculate Medication Utilization. San Diego, CA: SAS Global Forum 2007. MedImpact Healthcare Systems, Inc.; 2007. 23. Andersen RM. Revisiting the behavioral model and access to medical care: does it matter? J Health Soc Behav 1995;36:1–10. 24. Kee VR, Gilchrist B, Granner MA, Sarrazin NR, Carnahan RM. A systematic review of validated methods for identifying seizures, convulsions, or epilepsy using administrative and claims data. Pharmacoepidemiol Drug Saf 2012;21(suppl 1):183–193. 25. Mentari EHA, Feeney J, Stone MUS Department of Health and Human Services, Food and Drug Administration. In: . Statistical Review and Evaluation: Antiepileptic Drugs and Suicidality May 23, 2008;2013. 26. Ettinger AB, Argoff CE. Use of antiepileptic drugs for nonepileptic conditions: psychiatric disorders and chronic pain. Neurotherapeutics 2007;4:75–83. 27. Johannessen Landmark C. Antiepileptic drugs in non-epilepsy disorders: relations between mechanisms of action and clinical efficacy. CNS Drugs 2008;22:27–47. 28. Spina E, Perugi G. Antiepileptic drugs: indications other than epilepsy. Epileptic Disord 2004;6:57–75.
40
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
29. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–383. 30. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45:613–619. 31. Wagner AK, Soumerai SB, Zhang F, RossDegnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27:299–309. 32. Time series analysis. In: Ostrom CW, ed. Sage University Papers Series on Quantitative Applications in the Social Sciences. Thousand Oaks, CA: Sage Publications Inc.; 1990. 33. Watson DB. Testing for serial correlation in least square regression. Biometrika 1951;37:409–428.
34. McCullagh P, Nelder JA. Generalized Linear Models. London: Chapman and Hall; 1983. 35. Hubbard AE, Ahern J, Fleischer NL, et al. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology 2010;21:467–474. 36. Fitzmaurice Garrett M, Laird NM, Ware James H. Applied Longitudinal Analysis. John Wiley & Sons Inc; 2004. 37. Hanley JA, Negassa A, Edwardes MD, Forrester JE. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 2003;157:364–375. 38. French B, Heagerty PJ. Analysis of longitudinal data to evaluate a policy change. Stat Med 2008;27: 5005–5025.
Original Research
An evaluation of three statistical estimation methods for assessing health policy effects on prescription drug claims Manish Mittal, Ph.D.a,*,1,2, Donald L. Harrison, Ph.D., F.A.Ph.A.a, David M. Thompson, Ph.D.b, Michael J. Miller, R.Ph., Dr.P.H., F.A.Ph.A.c, Kevin C. Farmer, Ph.D., F.A.Ph.A.a, Yu-Tze Ng, M.D., F.R.A.C.P.d,3 a
Department of Pharmacy, Clinical and Administrative Sciences, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA b Department of Biostatistics and Epidemiology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA c Department of Pharmacy, Clinical and Administrative Sciences, The University of Oklahoma Health Sciences Center, Tulsa, OK, USA d Department of Neurology, The University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA
Abstract Background: While the choice of analytical approach affects study results and their interpretation, there is no consensus to guide the choice of statistical approaches to evaluate public health policy change. Objectives: This study compared and contrasted three statistical estimation procedures in the assessment of a U.S. Food and Drug Administration (FDA) suicidality warning, communicated in January 2008 and implemented in May 2009, on antiepileptic drug (AED) prescription claims. Methods: Longitudinal designs were utilized to evaluate Oklahoma (U.S. State) Medicaid claim data from January 2006 through December 2009. The study included 9289 continuously eligible individuals with prevalent diagnoses of epilepsy and/or psychiatric disorder. Segmented regression models using three estimation procedures [i.e., generalized linear models (GLM), generalized estimation equations (GEE), and generalized linear mixed models (GLMM)] were used to estimate trends of AED prescription claims across three time periods: before (January 2006–January 2008); during (February 2008–May 2009); and after (June 2009–December 2009) the FDA warning. Results: All three statistical procedures estimated an increasing trend (P ! 0.0001) in AED prescription claims before the FDA warning period. No procedures detected a significant change in trend during (GLM: 30.0%, 99% CI: 60.0% to 10.0%; GEE: 20.0%, 99% CI: 70.0% to 30.0%; GLMM: 23.5%, 99% CI: 58.8% to 1.2%) and after (GLM: 50.0%, 99% CI: 70.0% to 160.0%; GEE:
1 I ‘Manish Mittal’ am currently an employee of the Abbvie. However, during the conduct of this research I was a doctoral student at the OUHSC. Abbvie is in no manner associated with this research. 2 Present address: Health Economics and Outcomes Research, Abbvie, North Chicago, IL, USA. 3 Present address: Department of Pediatrics, Baylor College of Medicine, San Antonio, TX, USA. * Corresponding author. Dept. GMH1, AP31-1 NE, 1 North Waukegan Road, North Chicago, IL 60064, USA. Tel.: þ1 847 935 9190, þ1 917 379 3390(mobile); fax: þ1 847 937 1992. E-mail address: [email protected] (M. Mittal).
1551-7411/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.sapharm.2015.03.004
30
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
80.0%, 99% CI: 20.0% to 200.0%; GLMM: 47.1%, 99% CI: 41.2% to 135.3%) the FDA warning when compared to pre-warning period. Conclusions: Although the three procedures provided consistent inferences, the GEE and GLMM approaches accounted appropriately for correlation. Further, marginal models estimated using GEE produced more robust and valid population-level estimations. Ó 2016 Elsevier Inc. All rights reserved. Keywords: Generalized linear model; Generalized estimation equations; Generalized linear mixed models; FDA alert; Antiepileptic drugs
Introduction Studies evaluating health care regulatory actions are common. However, certain types of data, in particular longitudinal data, require health service researchers to use statistical procedures that obtain robust and valid estimates to provide accurate assessments of the outcomes of these actions.1,2 There is no consensus to guide the choice of statistical approaches to formally evaluate a public health policy change.3,4 For example, some studies have used relatively simple generalized linear models (GLM), which use maximum likelihood estimation (MLE), to estimate policy effect without accounting for correlation induced by repeatedly measuring observations from the same individual over time.5–11 Analyzing longitudinal data without taking into account the correlation between the outcomes12 causes standard errors to be underestimated or overestimated, which may lead to making either a type I or II error.13 Ultimately, ignoring correlation produces inaccurate inferences about the effect of policies evaluated. Generalized estimation equations (GEE) calculate standard errors for their parameter estimates by incorporating a “sandwich estimator”.14 Because the parameter estimates are robust, GEE estimation is gaining popularity among health service researchers15–17 although it has not been widely used to evaluate FDA policy changes.4 While GEE estimates a population-level policy effect, it does not attempt to quantify heterogeneity of responses in the effect across individual subjects. In contrast, the generalized linear mixed model (GLMM), which uses a pseudo-likelihood estimation technique, accounts for autocorrelation via the introduction of random effect and allows for subject-specific inferences.18 Because of limited accessibility to the software, GLMM has not been commonly employed by health service researchers.19
The three aforementioned analytical approaches may yield different results and subsequent policy interpretations when applied to the same data to answer a common research question. It is known that the absolute magnitude of the parameter estimates derived from GLMM are generally larger than those derived from GEE estimation.20 In January 2008, the U.S. FDA issued an alert, followed later (May 2009) by a warning of an increased risk of suicidality, defined as suicidal ideation and behavior, among users of antiepileptic drugs (AEDs).21 Using this policy change as an example case, the objective of this study was to compare and contrast the results and conclusions of three estimation procedures (i.e., GLM, GEE and GLMM) used to assess the association between the FDA suicidality warning and AED prescription claims among Oklahoma Medicaid individuals diagnosed with epilepsy and/or psychiatric disorder(s), from 2006 through 2009. Oklahoma is a state located in south-central U.S., and Medicaid is a federal program providing medical insurance primarily to indigent, and to other populations. Methods Study design A longitudinal segmented regression analysis of Oklahoma Medicaid claims data from January 2006 through December 2009 was used to evaluate the change in AED prescription claims before and after the FDA suicidality alert and warning. A time-series of 48 consecutive months was created using person level data as a unit of analysis. For each month, the proportion of individuals with an AED prescription claim was calculated. This study was reviewed and approved by the Institutional Review Board at the University of Oklahoma Health Sciences Center.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
31
Study population
Study variables
Fig. 1 depicts the individual inclusion and exclusion process. There were 1,183,668 individuals enrolled in the Oklahoma Medicaid program between January 2006 and December 2009. A group of 14,881 continuously eligible individuals was selected who were less than 65 years of age, not dually eligible for Medicaid and Medicare, had a diagnosis of epilepsy and/or psychiatric disorder(s) in an inpatient or outpatient setting, and had an AED prescription claim in at least one of the 48 months between January 2006 and December 2009 (Table 1). The date of the first AED prescription claim during the study period was considered the AED prescription index date. Individuals were excluded if they had received a diagnosis of attempted suicide (ICD-9 CM: E950-E959) or received polytherapy (2 or more AED drugs, indicating more severely ill group) within six months prior to the AED prescription index date. To ensure standardized length of AED exposure time among study participants and to reduce bias related to the time-varying nature of an AED prescription fill, 9289 individuals were selected for the final analytical cohort who had prevalent (existing) diagnoses of epilepsy and/or psychiatric disorder on or prior to January 2006 (Fig. 1).
Dependent variable For each individual, a supply diary of AEDs was created by adding together consecutive prescription fills of AEDs based on dispensing dates and the reported days’ supply.22 An individual was considered to have an AED prescription claim for any month during which the diary showed at least 15 days of AED therapy. Because of the binomial distribution of the outcome, the log odds of filling an AED prescription claim for at least 15 days at each monthly time point was calculated in the analytical cohort. Independent variables Regression models included parameters that defined two interruptions introduced during the study period. The first interruption was the month when the FDA alert was issued (January 2008). The second interruption was the month when the FDA warning was issued (May 2009). While this study focused primarily on the periods immediately before and after the introduction of the FDA warning, these two interruptions created 3 segments in a time-series of 48 months: (1) before (January 2006–January 2008); (2) during (February 2008–May 2009); and (3) after (June 2009–December 2009) the FDA warning to
Fig. 1. Inclusion and exclusion criteria for the Study Population. E ¼ Patients with epilepsy alone; E þ P ¼ Patients with epilepsy and comorbid psychiatric disorder; P ¼ Patients with psychiatric disorder alone.
32
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Table 1 Diagnostic groups per International Classification of Diseases, 9th Revision, Clinical Modification (ICD9-CM) Codes and list of antiepileptic drugs (AEDs) included in the study Factors I. Diagnoses (a) Epilepsy alone16 (b) Psychiatric disorder alone
(c) Epilepsy and comorbid psychiatric disorder II. Antiepileptic drugs (AEDs)
Categories (ICD9) Epilepsy (345.xx) Convulsion (780.3x) Psychiatry disorder (295–319) - Depression (296.2x, 296.3x, 300.4, 311) - Bipolar disorder (296.0x, 296.1x, 296.4x-296.8x) - Schizophrenia (295.xx) - Anxiety/Phobia disorder (300.0x, 300.2x, 300.3) - Adjustment disorder (309.xx) - ADHD (314.0x) - Other psychiatric disorder Patients diagnosed with both medical conditions Carbamazepine, Clonazepam, Clorazepam, Divalproex Sodium, Valproic acid, Ethosuximide, Ethotoin, Mephenytoin, Methsuximide, Phenytoin, Primidone, Lamotrigine, Gabapentin, Pregabalin, Oxcarbazepine, Levetiracetam, Tiagabine, Topiramate, Vigabatrin, Felbamate, Zonisamide
more explicitly account for a potential lag period between implementation of the FDA warning and physicians’ prescribing response to it. Covariates The selection of covariates, available in the Medicaid dataset, was guided by the Andersen Behavioral Model for Access to Medical Care and included baseline predisposing (i.e., individuals’ propensity to use services), need (i.e., illness level), and enabling (i.e., individuals’ ability to access services) factors that have been identified to potentially influence the use of health services.23 Predisposing characteristics included age group, sex, and race. Need characteristics included diagnostic groups,24 neurological comorbidities,25–28 other comorbid conditions,29,30 and use of psychotropic drugs. Enabling characteristics included physician’s specialty, geographical region, and metropolitan statistical area (MSA) status (Table 2). Statistical analysis Univariate and descriptive statistics were used to profile all study covariates. Because of the outcome’s binomial distribution, a linear regression line was fit on a log odds (logit) scale. Segmented logistic regression models31 were employed to estimate change in level (intercept) and trend (slope) in log odds of AED prescription claims during and after the FDA warning with
the pre-warning period as the referent. Each model was adjusted for each of the predisposing, enabling and need characteristics. Three statistical estimation procedures i.e., (a) GLM, (b) GEE and, (c) GLMM, were used to account for autocorrelation between repeated observations as well as heterogeneity across individuals in different manners. Autocorrelation between repeated observations was detected using correlograms (residuals versus time)32 and the Durbin–Watson (DW) test statistics.33 Data management and analysis was accomplished via PCSAS (v9.2), with an a-priori alpha set at P ! 0.01. The longitudinal results (i.e., point estimates, width of 99% CI and P-values of change in trend of log odds of AED prescription claim), attained by all three estimation procedures were compared. Because no standard estimation or hypothesis test exists to quantitatively evaluate the difference in results attained by the three estimation procedures, the direction, magnitude and statistical significance of the results (trend estimates) were reviewed and compared for consistency. Moreover, the predicted segmented trajectories produced by the three models were graphed.
Results Baseline predisposing, enabling and need characteristics of the study population are presented in
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40 Table 2 Descriptive statistics for predisposing, enabling, and need characteristics of the study population from January 2006 through December 2009 (N ¼ 9289) Characteristics
Study population, N ¼ 9289 (%)
Predisposing characteristics Age !18 3671 (39.5%) 18–44 2936 (31.6%) 45–64 2682 (28.9%) Sex Female 5212 (56.1%) Male 4077 (43.9%) Race White 6991 (75.3%) Black 1209 (13.0%) Othera 1089 (11.7%) Need characteristics Diagnostic groups Epilepsy alone 345 (3.7%) Epilepsy & comorbid 3162 (34.0%) psychiatric disorder Psychiatric disorder alone 5782 (62.3%) Neurological comorbiditiesb (Neuropathic pain, migraine, Movement disorder, or chronic Pain) Yes 3799 (40.9%) No 5490 (59.1%) Other comorbiditiesc (Charlson comorbiditity score) 0 3325 (35.8%) R1 5964 (64.2%) Use of psychotropic therapy (Antidepressant, antipsychotic, stimulants, anxiolytic/hypnotics) Yes 7125 (76.7%) No 2164 (23.3%) Enabling characteristics CNS health care providerd Yes 3026 (32.6%) No 6263 (67.4%) G.Region NW 598 (6.4%) SW 905 (9.7%) OKC 2621 (28.2%) Tulsa 1779 (19.2%) NE 1574 (16.9%) SE 1742 (18.8%) MSA Urban 5156 (55.5%) Rural 4004 (43.1%) Age, sex, race, use of psychotropic therapy, physician specialty, geographical region, and MSA were measured at the AED prescription index date. Diagnostic groups, neurological comorbidities and other comorbidities were measured during the entire study period. Percentage of geographic region and MSA did not add up to 100% because of missing data. a Other races included asian, native hawaiian/other pacific islander, american indian/alaska native, multiple races.
33
Table 2. Overall, the percentage of individuals with an AED prescription claim consistently increased, from 38.36% to 46.90%, between January 2006 and December 2009. For the group of individuals with diagnosis of epilepsy alone, the percentage of individuals with an AED prescription claim actually declined from 72.17% to 58.55%. However, the pattern steadily increased in individuals with diagnoses of epilepsy and comorbid psychiatric disorder(s) trending upward from 53.42% to 63.47% and in individuals with psychiatric disorder(s) alone trending from 28.10% to 37.15%. Distinct, non-random regions of negative and positive residuals for the correlogram and a significant DW statistic (1.4672, P ¼ 0.0203) indicated a positive autocorrelation between repeated measures obtained from the same individual in the Medicaid claims data. Fig. 2 illustrates the segmented trajectories for AED prescription claims, predicted by the three statistical estimation procedures, for a subgroup of white males, under the age of 18 years, who lived in OKC and urban areas, and who were without neurological comorbidity, other comor-
b Neurological comorbidities included a list of four chronic conditions commonly prescribed with AEDs i.e., Neuropathic pain [diabetes with neurological manifestations (250.6x), trigeminal nerve disorders (350.xx), glossopharyngeal neuralgia (352.1x), neuropathy (356.0x, 356.8x), postherpatic trigeminal neuralgia (053.12), and unspecified neuralgia, neuritis and radiculitis (729.2x)]; Migraine (346.xx); Movement disorder [essential tremor (331.1x, 781.0x), restless legs syndrome (333.94)]; and Chronic pain (338.2x, 729.1x).2,16–18 c Other comorbid conditions were adjusted by measuring the Charlson comorbid score indicating the severity of the disease.19,20 d Physicians’ specialty indicated whether or not a physician was a central nervous system (CNS) health care provider who had prescribed the first AED prescription to individual. A list of CNS health care providers included psychiatric hospital, intermediate care facility/mental health (ICF/MH) O 6 beds, intermediate care facility/mental health (ICF/MH) ! 6 beds, outpatient mental health clinic, community mental health center (CMHC), psychologist, pre-admission screening and resident review (PASRR) CMHC, health service provider in psychology (HSPP), mental health – department of mental health and substance abuse services (DMHSAS), mental health case management all ages (public and private), neurological surgeon, neurologist, psychiatrist, psychiatry child.
34
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Fig. 2. Predicted mean monthly percentage of study individuals with AED prescription claims produced by three estimation procedures over the 48 months from January 2006 through December 2009. (a). Total study population (N ¼ 9289). (b). Individuals with epilepsy alone (n ¼ 345). (c). Individual with epilepsy and comorbid psychiatric disorder (n ¼ 3162). (d). Individual with psychiatric disorder alone (n ¼ 5782). Predicted mean monthly percentage of study individuals with AED prescription claims are displayed for a single subgroup: individual with age !18 years, male, white, epilepsy diagnosis alone, no neurological comorbidity, no other comorbid condition, no use of psychotropic drugs, prescribed by non-CNS health care provider, lives in OKC and urban area.
bid condition, or use of psychotropic drugs, and whose AEDs were prescribed by non-CNS health care providers. The figure’s three panels distinguish among diagnostic groups. Table 3 details the parameter estimates that underlie the predicted trajectories illustrated in Fig. 2. 1. Direction of point estimates: The directional sign of point estimates from the three estimation procedures were consistent for each warning period. 2. Magnitude of point estimates: For all three warning periods, GLM and GEE estimation procedures provided point estimates of similar magnitude observed by visual inspection. In contrast, the magnitudes of point estimates attained by the GLMM were larger than those calculated in the other estimation procedures. 3. Precision of point estimates: Table 3 illustrates that all three statistical procedures estimated an increasing trend (P ! 0.0001) in log odds of AED prescription claims before the FDA warning period. None of the procedures detected a significant change in estimated trend both during (GLM: 30.0%, 99%
CI: 60.0% to 10.0%; GEE: 20.0%, 99% CI: 70.0% to 30.0%; GLMM: 23.5%, 99% CI: 58.8% to 1.2%) and after (GLM: 50.0%, 99% CI: 70.0% to 160.0%; GEE: 80.0%, 99% CI: 20.0% to 200.0%; GLMM: 47.1%, 99% CI: 41.2% to 135.3%) the FDA warning period when compared to pre-warning period. Although none of the three estimation techniques detected associations between changing trends in AED prescription claims and the FDA suicidality warning, the GLMM produced the most precise confidence intervals and the GEE produced the least precise confidence interval among all three estimation procedures. 4. Predicted segmented trajectories: Fig. 2(a) illustrates that predicted mean monthly percentages of overall study individuals with AED prescription claims attained by the three estimation procedures follow similar trajectory, indicating a consistency in the direction of point estimates. However, while using the same data, a discrepancy in the magnitudes of point estimates predicted from three estimation procedures is evident. Magnitude produced from GLM (dotted line) and GEE
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
(squared line) were very much overlapping each other; and the magnitude produced from GLMM (triangle line) were generally larger compared to other estimation procedures. Predicted segmented trajectories were graphed (Fig. 2) for each diagnostic group. For individuals with epilepsy alone and individuals with epilepsy and comorbid psychiatric disorder(s), GLMM (triangle line) predicted higher percentage estimates of AED prescription claims compared to other estimation procedures (Fig. 2(b and c)). In contrast, in individuals with psychiatric disorder(s) alone, the GLMM (triangle line) predicted lower percentage estimates of AED prescription claims compared to the other two procedures, as presented in Fig. 2(d).
Discussion Although all three procedures estimated trajectories whose changes in level and trend were in the same directions, and were of similar significance, as expected, the magnitude of point estimates and precision of confidence interval varies. In general, it appears that the GLM produced a smaller magnitude of point estimates and more precise confidence intervals and the GEE estimation produced similar magnitude of point estimates but relatively less precise confidence intervals. Finally, the GLMM produced a relatively larger magnitude of point estimates and relatively more precise confidence intervals. The inconsistency in the magnitude of trend estimates and confidence interval width may be due to different approaches by the three statistical estimation procedures. GLM completely ignores the correlation between repeated observations obtained from the same individual in longitudinal data and assumes single observation for each individual.34 For example, in this study, the observations of AED prescription claims were collected at each month over a period of 48 months from each (total of 9289) individual. Because of the independence assumption, the GLM estimates the odds of AED prescription claims for a total of 9289 48 ¼ 445,872 observations. A large sample size may potentially lead to discernible underestimations of the variability of the point estimate. This, in turn, results in confidence intervals that are too narrow which may further lead to making a type I error (rejecting the true null hypothesis).13
35
As an alternative to the ML approach that is customarily used to estimate linear models, GEE accounts for autocorrelation among longitudinal data14 and produces more robust and valid estimates at the population-level.35 A notable feature of the GEE is that it separately models the regression parameters and covariance structure among repeated observations.36 This separation ensures that the interpretation of the regression coefficients does not rely on the assumed structure for the covariance among the responses. In our example, accounting for autocorrelation in the Medicaid data, is potentially why the GEE produced point estimates similar in magnitude to the GLM, but relatively less precise, unbiased confidence intervals. The GEE and GLMM are two alternative approaches that account for the autocorrelation among longitudinal data. However, for the same Medicaid data, these two estimation procedures also provided different results in terms of magnitude and precision of the estimates. This difference may be due to the fact that these two estimation procedures have different objectives for making inferences. Ultimately these two approaches may subtly address different scientific questions.36 When applied to a dichotomous outcome, the GEE estimation makes inferences at the population (mean) level and does not account for heterogeneity across individuals.36 The GEE approach yields an estimate for any individual randomly selected from the population without accounting for subject-specific propensity to respond, in this case, to file a claim for an AED, this is also known as “population averaged” or marginal model.37 The GEE indicates that the model for the AED prescription claims depends only on the covariates of interest and not on any random effect across individuals. In conclusion, this procedure should be of interest to health service researchers who consider evaluating the potential impact of an intervention at the population level. However, the basic premise underlying the GLMM is the assumption of heterogeneity across individuals in the study population. The GLMM model considers the odds of filling an AED prescription claim as conditioned on subjectspecific random intercepts. These intercepts reflect a natural heterogeneity, due to unmeasured factors, in individuals’ propensity to fill an AED prescription claim. For example, the model conditioned on individuals with a high propensity to fill AED prescription claims, would estimate a
Diagnostic groups
Epilepsy alone (n ¼ 345)
Baseline level (b0) Baseline trend (b1) Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0) Baseline trend (b1)
Epilepsy þ comorbid psychiatric disorder(s) (n ¼ 3162)
Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0) Baseline trend (b1) Level change at FDA alert (b2)
GLM
GEE
GLMM
Point estimates (99% CI)
a
Change in log odds of AED Rx claims % (99% CI)
Point estimates (99% CI)
a
Change in log odds of AED Rx claims % (99% CI)
Point estimates (99% CI)
Change in log odds of AED Rx claims % (99% CI)
0.63** (0.57–0.68) 0.01** (0.008–0.011) 0.02 (0.06 to 0.02) 0.003 (0.006 to 0.001) 0.05 (0.11 to 0.01) 0.005 (0.007–0.016) 1.09** (0.92–1.26) 0.01** (0.02 to 0.005) 0.07 (0.27 to 0.13) 0.002 (0.02 to 0.02) 0.03 (0.32 to 0.25)
Referent
0.62** (0.39–0.84) 0.01** (0.008–0.013) 0.04 (0.07 to 0.002) 0.002 (0.007 to 0.003) 0.09** (0.14 to 0.05) 0.008 (0.002–0.02) 1.08** (0.52–1.65) 0.01 (0.03 to 0.001) 0.06 (0.26 to 0.14) 0.002 (0.02–0.03) 0.09 (0.31 to 0.13)
Referent
1.27** (0.93–1.62) 0.017** (0.015–0.019) 0.04 (0.09 to 0.01) 0.004 (0.01 to 0.0002) 0.09* (0.16 to 0.01) 0.008 (0.007–0.023) 2.21** (1.20–3.22) 0.02** (0.04 to 0.01) 0.13 (0.40 to 0.14) 0.002 (0.02–0.03) 0.07 (0.47 to 0.32)
Referent
0.002 (0.06 to 0.06) 0.62** (0.57–0.68) 0.013** (0.01–0.015) 0.04 (0.10 to 0.02)
Referent 3.2% (9.5–3.2) 30.0% (60.0–10.0) 7.9% (17.5–1.6) 50.0% (70.0–160.0) Referent Referent 6.4% (24.8–11.9) 20.0% (200.0–200.0) 2.7% (29.4–22.9) 20.0% (600.0–600.0) Referent Referent 6.5% (16.1–3.2)
0.009 (0.03–0.05) 0.61** (0.42–0.81) 0.013** (0.01–0.02) 0.06 (0.13 to 0.002)
Referent 6.5% (11.3 to 0.3) 20.0% (70.0–30.0) 14.5% (22.6 to 8.1) 80.0% (20.0–200.0) Referent Referent 5.5% (24.1–13.0) 20.0% (200.0–300.0) 8.3% (28.7–12.0) 90.0% (300.0–500.0) Referent Referent 9.8% (21.3 to 0.3)
0.008 (0.07–0.09) 1.11** (0.77–1.45) 0.023** (0.019–0.027) 0.07 (0.15 to 0.02)
a
Referent 3.2% (7.1–0.8) 23.5% (58.8–1.2) 7.1% (12.6 to 0.8) 47.1% (41.2–135.3) Referent Referent 5.9% (18.1–6.3) 10.0% (100.0–150.0) 3.2% (21.3–14.5) 40.0% (350.0–450.0) Referent Referent 6.3% (13.5–1.8)
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Epilepsy and/or psychiatric disorder(s) (N ¼ 9289)
Level/trend
36
Table 3 Segmented regression analysis to evaluate the change in AED Prescription claims among the study population in the Oklahoma Medicaid setting from January 2006 through December 2009
Baseline trend (b1) Level change at FDA alert (b2) Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5)
0.005 (0.011 to 0.001) 0.05 (0.14 to 0.05) 0.0008 (0.02 to 0.02) 0.67** (0.72 to 0.62) 0.009** (0.007–0.012) 0.01 (0.06 to 0.04) 0.002 (0.01 to 0.003) 0.05 (0.12 to 0.02) 0.007 (0.01–0.02)
38.5% (84.6–7.7) 8.1% (22.6–8.0) 6.2% (153.8–153.8) Referent Referent 1.5% (9.0–6.0) 22.2% (111.1–33.3) 7.5% (17.9–3.0) 77.8% (111.1–222.2)
0.005 (0.013 to 0.003) 0.09* (0.17 to 0.01) 0.002 (0.02 to 0.02) 0.66** (0.86 to 0.47) 0.01** (0.006–0.013) 0.02 (0.07 to 0.03) 0.0003 (0.01 to 0.01) 0.1** (0.16 to 0.04) 0.01 (0.001–0.025)
38.5% (100.0–23.0) 14.8% (27.9 to 1.6) 15.4% (153.8–153.8) Referent Referent 3.0% (10.6–4.5) 3.0% (100.0–100.0) 15.2% (24.2 to 6.0) 100.0% (10.0–250.0)
0.011* (0.019 to 0.003) 0.03 (0.16 to 0.1) 0.006 (0.03 to 0.02) 0.95** (1.23 to 0.67) 0.016** (0.013–0.018) 0.01 (0.07 to 0.05) 0.002 (0.008 to 0.004) 0.09 (0.18 to 0.003) 0.01 (0.007–0.03)
47.8% (82.6 to 13.0) 2.7% (14.4–9.0) 26.1% (130.4–87.0) Referent Referent 1.1% (7.4–5.3) 12.5% (50.0–25.0) 9.5% (18.9 to 0.3) 62.5% (43.8–187.5)
*P ! 0.01, **P ! 0.0001. Bold indicates an area where there appears to be some inconsistencies between the various estimation techniques. The models were estimated using PROC LOGISTIC/GENMOD/GLIMMIX (SAS 9.2) with a binomial distribution logit function. The correlation structure was AR(1). The overall model was adjusted for a number of covariates included age, sex, race, diagnostic groups, neurological comorbidities, other comorbidities, use of psychotropic therapy, physician specialty, geographical region, and MSA. All three stratified diagnosis-specific models were adjusted for a number of covariates excluding diagnostic groups. Calculation of 99% CI: (a) Lower bound ¼ 0.007 100/0.01 ¼ 70.0% (b) Upper bound ¼ 0.003 100/0.01 ¼ 30.0%. % Change in log odds of AED Rx claims during the FDA warning period was 20.0% (99% CI: 70.0 to 30.0). Other calculations were performed in a similar manner. Negative sign indicates the decline in log odds of AED Rx claims and was not included in the mathematical calculation. a Example of calculating change in log odds of AED Rx claims %: % Change in trend of log odds of AED Rx claims during the FDA warning (b3) in patients with epilepsy and/or psychiatric disorder(s) (N ¼ 9289) using GEE ¼ Trend change during the FDA warning period/Baseline trend in pre-warning period ¼ b3 100/ b1 ¼ 0.002 100/0.01 ¼ 20.0%.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
Psychiatric disorder(s) alone (n ¼ 5782)
Trend change during FDA warning (b3) Level change at FDA warning (b4) Trend change after FDA warning (b5) Baseline level (b0)
37
38
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
different trajectory for claims than the model conditioned on individuals with low propensity to fill AED prescription claims. Therefore, the GLMM permits the research to make subjectspecific inferences. Consequently, there was, as expected, a large difference in the magnitude of point estimates between the GLMM and the marginal models as presented in Fig. 2(a). Because the main objective of the GLMM analytical approach is to make inferences at the subject-level, the magnitude of point estimates may potentially be influenced by subjects included in the analytical sample. This rationale was strengthened by examining the graphs predicting segmented trajectories by individual diagnostic groups. Individuals with epilepsy alone and individuals with epilepsy and comorbid psychiatric disorder(s), represent a more severely affected group of patients in terms of medical illness who have no alternative pharmacotherapy available for treating their epilepsy and may be more dependent or committed to the use of AEDs. For these two diagnostic groups, GLMM predicted higher percentage estimates of AED prescription claims compared to other estimation procedures. In contrast, individuals with psychiatric disorder(s) alone may be less dependent or committed to AED therapy, than individuals in the other two diagnostic groups, because there are alternative treatments beyond AEDs for psychiatric disorders. Hence, Fig. 2(d) depicts the lower predicted percentage estimates of AED prescription claims produced by the GLMM when compared to other procedures. Therefore, we may conclude that the GLMM yields point estimates that address different scientific questions that will be of most interest to the individual and his/her physician within the physicianpatient context. Health service researchers might appropriately use any of the three approaches depending on the general scientific questions they wish to address. It is known that the GLM is the most powerful estimation model in a cross-sectional study where the response is measured at a single occasion (no repeated observations). The GEE and GLMM extend the GLM approach to longitudinal studies by accounting for autocorrelation among repeated observations. With time-stationary covariates (e.g., gender) or with time-varying covariates that are fixed by study design (e.g., age), GEE is useful in a type of setting where all covariates are between-subject variables and time-invariant and, as in this study, inferences
can be made at the population-level.14,20 The GLMM, which produces subject-specific interpretation, is far more natural for covariates that vary within an individual.36 As a result, the GLMM is most useful when the main scientific objective is to make inferences about individuals with specific characteristics, rather than overall population averages. This study has important limitations to consider. The AED related suicidality issue was first raised in 2004; but reliable administrative claims data were not available before January 1, 2006 in the Oklahoma Medicaid dataset. The time period included after the FDA warning was very short; potentially too short to measure a potential long-term change in outcome measures. There may be several other factors, including advertising by pharmaceutical manufacturers, communications by professional organizations, publication of new safety and effectiveness information, and media reports that could affect the utilization pattern of AEDs during that period. These external factors could not be accounted for in this study. Missing data is always a potential problem, especially in research employing claims data.
Conclusion Analytical approaches follow certain assumptions about the response variable, and failing to account for these assumptions may lead to inefficient regression estimates, biased standard errors and/or invalid inferences.38 Among the three statistical estimation procedures in this study, generalized estimation equations (GEE) appears to produce more robust and valid population-level estimates in assessing the association between AED prescription claims and FDA suicidality warning in an Oklahoma Medicaid population. The GEE yields estimates that will be of most interest to health service researchers and policy makers who evaluate drug policy change at the population level.
Acknowledgment No financial or material support was received by any of the authors in the conduction of this research or the preparation of this manuscript. The principal author takes full responsibility for the data, the analyses and interpretation, and the conduct of the research; full access to all of the data; and the right to publish any and all data.
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
We thank Nancy Nesser, PharmD, JD, Pharmacy Director, Oklahoma Health Care Authority, for assisting in accessing the Oklahoma Medicaid administrative data and Shellie Keast, PhD, Clinical Assistant Professor, Pharmacy Management Consultants, for guidance in data analysis. References 1. Fitzmaurice GM, Ravichandran C. A primer in longitudinal data analysis. Circulation 2008;118:2005– 2010. 2. Lagarde M. How to do (or not to do) . Assessing the impact of a policy change with routine longitudinal data. Health Policy Plan 2012;27:76–83. 3. Dusetzina SB, Higashi AS, Dorsey ER, et al. Impact of FDA drug risk communications on health care utilization and health behaviors: a systematic review. Med Care 2012;50:466–478. 4. Briesacher BA, Soumerai S, Zhang F, et al. In: MiniSentinel White Paper: Methods to Evaluate the Impact of FDA Regulatory Actions March 12, 2012;2012. 5. Busch SH, Frank RG, Leslie DL, et al. Antidepressants and suicide risk: how did specific information in FDA safety warnings affect treatment patterns? Psychiatr Serv 2010;61:11–16. 6. Hassanin H, Harbi A, Saif A, Davis J, Easa D, Harrigan R. Changes in antidepressant medications prescribing trends in children and adolescents in Hawai’i following the FDA black box warning. Hawaii Med J 2010;69:17–19. 7. Weatherby LB, Nordstrom BL, Fife D, Walker AM. The impact of wording in “Dear doctor” letters and in black box labels. Clin Pharmacol Ther 2002;72: 735–742. 8. Weatherby LB, Walker AM, Fife D, Vervaet P, Klausner MA. Contraindicated medications dispensed with cisapride: temporal trends in relation to the sending of ‘Dear Doctor’ letters. Pharmacoepidemiol Drug Saf 2001;10:211–218. 9. Burkhart GA, Sevka MJ, Temple R, Honig PK. Temporal decline in filling prescriptions for terfenadine closely in time with those for either ketoconazole or erythromycin. Clin Pharmacol Ther 1997;61: 93–96. 10. Shatin D, Gardner JS, Stergachis A, Blough D, Graham D. Impact of mailed warning to prescribers on the co-prescription of tramadol and antidepressants. Pharmacoepidemiol Drug Saf 2005;14:149–154. 11. Willy ME, Manda B, Shatin D, Drinkard CR, Graham DJ. A study of compliance with FDA recommendations for pemoline (Cylert). J Am Acad Child Adolesc Psychiatry 2002;41:785–790. 12. Agresti A. An Introduction to Categorical Data Analysis. 2nd ed. John Wiley & Sons, Inc; 2007. 13. Velicer WF, Colby SM. A comparison of missingdata procedures for arima time-series analysis. Educ Psychol Meas 2005;65:595–615.
39
14. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics 1988;44:1049–1060. 15. Mittal M, Harrison DL, Miller MJ, Farmer KC, Thompson DM, Ng YT. Have antiepileptic drug prescription claims changed following the FDA suicidality warning? An evaluation in a state Medicaid program. Epilepsy Behav 2014;34:109–115. 16. Dusetzina SB, Busch AB, Conti RM, Donohue JM, Alexander GC, Huskamp HA. Changes in antipsychotic use among patients with severe mental illness after a Food and Drug Administration advisory. Pharmacoepidemiol Drug Saf 2012 Dec;21(12): 1251–1260, http://dx.doi.org/10.1002/pds.3272. Epub 2012 May 3. 17. Valluri S, Zito JM, Safer DJ, Zuckerman IH, Mullins CD, Korelitz JJ. Impact of the 2004 Food and Drug Administration pediatric suicidality warning on antidepressant and psychotherapy treatment for new-onset depression. Med Care 2010;48:947–954. 18. The GLIMMIX Procedure: PROC GLIMMIX Contrasted with Other SAS Procedures. Cary, NC, USA: SAS Institute Inc; 2013. 19. Morrato EH, Druss B, Hartung DM, et al. Metabolic testing rates in 3 state Medicaid programs after FDA warnings and ADA/APA recommendations for second-generation antipsychotic drugs. Arch Gen Psychiatry 2010;67:17–24. 20. Neuhaus JM, Kalbfleisch JD, Hauck WW. A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data. Int Stat Rev 1991;59:25–35. 21. Machado P, Landewe R. Spondyloarthritis: Is it time to replace BASDAI with ASDAS? Nat Rev Rheumatol 2013;9:388–390. 22. Leslie SR. Using Arrays to Calculate Medication Utilization. San Diego, CA: SAS Global Forum 2007. MedImpact Healthcare Systems, Inc.; 2007. 23. Andersen RM. Revisiting the behavioral model and access to medical care: does it matter? J Health Soc Behav 1995;36:1–10. 24. Kee VR, Gilchrist B, Granner MA, Sarrazin NR, Carnahan RM. A systematic review of validated methods for identifying seizures, convulsions, or epilepsy using administrative and claims data. Pharmacoepidemiol Drug Saf 2012;21(suppl 1):183–193. 25. Mentari EHA, Feeney J, Stone MUS Department of Health and Human Services, Food and Drug Administration. In: . Statistical Review and Evaluation: Antiepileptic Drugs and Suicidality May 23, 2008;2013. 26. Ettinger AB, Argoff CE. Use of antiepileptic drugs for nonepileptic conditions: psychiatric disorders and chronic pain. Neurotherapeutics 2007;4:75–83. 27. Johannessen Landmark C. Antiepileptic drugs in non-epilepsy disorders: relations between mechanisms of action and clinical efficacy. CNS Drugs 2008;22:27–47. 28. Spina E, Perugi G. Antiepileptic drugs: indications other than epilepsy. Epileptic Disord 2004;6:57–75.
40
Mittal et al. / Research in Social and Administrative Pharmacy 12 (2016) 29–40
29. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–383. 30. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45:613–619. 31. Wagner AK, Soumerai SB, Zhang F, RossDegnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27:299–309. 32. Time series analysis. In: Ostrom CW, ed. Sage University Papers Series on Quantitative Applications in the Social Sciences. Thousand Oaks, CA: Sage Publications Inc.; 1990. 33. Watson DB. Testing for serial correlation in least square regression. Biometrika 1951;37:409–428.
34. McCullagh P, Nelder JA. Generalized Linear Models. London: Chapman and Hall; 1983. 35. Hubbard AE, Ahern J, Fleischer NL, et al. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology 2010;21:467–474. 36. Fitzmaurice Garrett M, Laird NM, Ware James H. Applied Longitudinal Analysis. John Wiley & Sons Inc; 2004. 37. Hanley JA, Negassa A, Edwardes MD, Forrester JE. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 2003;157:364–375. 38. French B, Heagerty PJ. Analysis of longitudinal data to evaluate a policy change. Stat Med 2008;27: 5005–5025.
