New Practitioners Forum
New Practitioners Forum Using administrative data for your research project: 10 considerations before you begin
A
residency research project is a requirement for most ASHP-accredited residency programs. As many pharmacy graduates have not had the opportunity to conduct or even explore research before their residency, the research project can be daunting. Recent literature has provided guidance on how to develop a
including administrative data resources, for time and feasibility reasons.1,2 Increasingly, administrative claims data are a preferred data source for research projects as they are perceived to be less expensive, are easily obtainable, and provide a substantial number of subjects to garner sufficient power to address most
research project from a well-formulated clinically relevant question, as well as pearls for the timely completion of the proposed project.1-3 One recommendation provided is the use of existing data,
relevant study questions, including those related to comparative effectiveness and cost of pharmaceutical treatments.4-6 A very good description of administrative claims data has been previously
The New Practitioners Forum column features articles that address the special professional needs of pharmacists early in their careers as they transition from students to practitioners. Authors include new practitioners or others with expertise in a topic of interest to new practitioners. AJHP readers are invited to submit topics or articles for this column to the New Practitioners Forum, c/o Jill Haug, 7272 Wisconsin Avenue, Bethesda, MD 20814 (301-664-8821 or [email protected]).
184
Am J Health-Syst Pharm—Vol 72 Feb 1, 2015
published.7 In general, administrative claims data are medical insurance data collected for billing purposes; therefore, they contain only the information needed for this purpose, including drug information usually summarized with one national drug code (drug name, strength, dose, quantity, date of dispensing), outcome and confounder information (diagnoses in the form of International Classification of Diseases, 9th Revision codes, reason for visit, date of service, date of death [if applicable]), and other pertinent information (age, sex, comorbidities, provider specialty). These data can be obtained from large commercial insurance companies or federal sources (i.e., Centers for Medicare and Medicaid Services and the Department of Veterans Affairs). Often administrative claims data contain records for millions of individuals. While this is often thought of as advantageous, there are important concepts that must be considered when conducting studies using these data sources. Ten of these considerations are described below (Figure). Study question. Successful development of an appropriate research question has been outlined in the literature.8-10 Defining a research question to be answered using administrative data should use techniques outlined previously including the use of the FINER (feasible, interesting, novel, ethical, and relevant) and PICOT (population, intervention/exposure, comparison, outcome, and time) criteria.8 However, when considering the use of administrative data, special attention should be placed on project feasibility. Will it be possible to answer the question using the available data? For example, administrative claims data do not contain laboratory test results; therefore, “Does anticholesterol medication x control cholesterol levels better than anticholesterol medication y?” would not be a feasible question to be answered using an administrative claims data source. The following question may be more appropriate: “Does anticholes-
Feasible Interesting Novel Ethical Relevant
Generalizability
Valid interpretation
Appropriate design
Good research
Good research question
Research gap
Background knowledge
Timeline
Data issues
Data storage
Data-use restrictions
Availability and cost
Software availability
Analytical skills
Population Intervention/exposure Comparison Outcome Time
Data analysis
Individual factors
Figure. Concepts that must be considered when conducting studies using administrative claims data sources.
New Practitioners Forum
Am J Health-Syst Pharm—Vol 72 Feb 1, 2015
185
New Practitioners Forum
terol medication x result in less myocardial infarction events than anticholesterol medication y?” Another common constraint of using administrative data is the length of follow-up. For example, if the research question is “Does treatment of diabetes patients with medication x decrease mortality?”, it is necessary to evaluate whether the length of follow-up and data available are adequate. Study design. Your study question will guide your study design. It is important that care is taken when designing the study and study protocol to ensure that the results of the research are valid and generalizable to the population of interest. Furthermore, investigators should limit “looking” at the data to a basic assessment of power and sample size requirements for the research question. The study design should include a description of all variables (e.g., outcome, exposure, confounders, covariates), reasons for data inclusion, and data availability. One must also consider variances in data availability over time. Most studies that analyze administrative data are retrospective in nature and fall into the cross-sectional, cohort, or case–control category of epidemiologic study design. Drug-utilization studies or studies that describe patterns of medication use are generally cross-sectional in nature. Studies that aim to explore a cause-and-effect relationship are likely to be case–control or cohort studies. When examining associations, there are very important study design aspects that should be considered, including enrollment of incident users11 (if the study pertains to a medication exposure), use of an active comparator design (if possible),12 and consideration of important confounding variables. These topics can be complicated and are beyond the scope of this article. However, these topics are thoroughly discussed elsewhere.7,13 Other important study design considerations include the validity and potential misclassification of study variables (i.e., exposure, outcome, and confounders). For instance, depending on the variables being measured, claims data will vary in the degree of accuracy of the measurement depending on the type of claims and the time window being 186
used. For example, if a resident wants to identify patients with diabetes, a definition requiring either a diagnosis code or a pharmacy claim for any oral antidiabetes medication will probably identify the majority of patients with this diagnosis. However, this approach might falsely label some patients as having diabetes. To the contrary, requiring both a specific diagnosis code and a pharmacy claim will minimize false-positive results but increase false-negative results. Given these complicated factors, including the operational definitions for study variables with optimum tradeoff between sensitivity and specificity, it is important that a pharmacoepidemiologist or biostatistician familiar with the conduct of administrative claims analysis be consulted early and often throughout the duration of the residency project. Without an appropriate study design tailored to the research question, the results will be biased and have limited clinical impact. Spurious results and interpretation. As was previously stated, conducting a study using administrative claims data often involves the inclusion of a very large study population. As the numbers are quite large, the probability of random errors is reduced. With fewer chances of random errors, the chance of a low p value (i.e.,
New Practitioners Forum Using administrative data for your research project: 10 considerations before you begin
A
residency research project is a requirement for most ASHP-accredited residency programs. As many pharmacy graduates have not had the opportunity to conduct or even explore research before their residency, the research project can be daunting. Recent literature has provided guidance on how to develop a
including administrative data resources, for time and feasibility reasons.1,2 Increasingly, administrative claims data are a preferred data source for research projects as they are perceived to be less expensive, are easily obtainable, and provide a substantial number of subjects to garner sufficient power to address most
research project from a well-formulated clinically relevant question, as well as pearls for the timely completion of the proposed project.1-3 One recommendation provided is the use of existing data,
relevant study questions, including those related to comparative effectiveness and cost of pharmaceutical treatments.4-6 A very good description of administrative claims data has been previously
The New Practitioners Forum column features articles that address the special professional needs of pharmacists early in their careers as they transition from students to practitioners. Authors include new practitioners or others with expertise in a topic of interest to new practitioners. AJHP readers are invited to submit topics or articles for this column to the New Practitioners Forum, c/o Jill Haug, 7272 Wisconsin Avenue, Bethesda, MD 20814 (301-664-8821 or [email protected]).
184
Am J Health-Syst Pharm—Vol 72 Feb 1, 2015
published.7 In general, administrative claims data are medical insurance data collected for billing purposes; therefore, they contain only the information needed for this purpose, including drug information usually summarized with one national drug code (drug name, strength, dose, quantity, date of dispensing), outcome and confounder information (diagnoses in the form of International Classification of Diseases, 9th Revision codes, reason for visit, date of service, date of death [if applicable]), and other pertinent information (age, sex, comorbidities, provider specialty). These data can be obtained from large commercial insurance companies or federal sources (i.e., Centers for Medicare and Medicaid Services and the Department of Veterans Affairs). Often administrative claims data contain records for millions of individuals. While this is often thought of as advantageous, there are important concepts that must be considered when conducting studies using these data sources. Ten of these considerations are described below (Figure). Study question. Successful development of an appropriate research question has been outlined in the literature.8-10 Defining a research question to be answered using administrative data should use techniques outlined previously including the use of the FINER (feasible, interesting, novel, ethical, and relevant) and PICOT (population, intervention/exposure, comparison, outcome, and time) criteria.8 However, when considering the use of administrative data, special attention should be placed on project feasibility. Will it be possible to answer the question using the available data? For example, administrative claims data do not contain laboratory test results; therefore, “Does anticholesterol medication x control cholesterol levels better than anticholesterol medication y?” would not be a feasible question to be answered using an administrative claims data source. The following question may be more appropriate: “Does anticholes-
Feasible Interesting Novel Ethical Relevant
Generalizability
Valid interpretation
Appropriate design
Good research
Good research question
Research gap
Background knowledge
Timeline
Data issues
Data storage
Data-use restrictions
Availability and cost
Software availability
Analytical skills
Population Intervention/exposure Comparison Outcome Time
Data analysis
Individual factors
Figure. Concepts that must be considered when conducting studies using administrative claims data sources.
New Practitioners Forum
Am J Health-Syst Pharm—Vol 72 Feb 1, 2015
185
New Practitioners Forum
terol medication x result in less myocardial infarction events than anticholesterol medication y?” Another common constraint of using administrative data is the length of follow-up. For example, if the research question is “Does treatment of diabetes patients with medication x decrease mortality?”, it is necessary to evaluate whether the length of follow-up and data available are adequate. Study design. Your study question will guide your study design. It is important that care is taken when designing the study and study protocol to ensure that the results of the research are valid and generalizable to the population of interest. Furthermore, investigators should limit “looking” at the data to a basic assessment of power and sample size requirements for the research question. The study design should include a description of all variables (e.g., outcome, exposure, confounders, covariates), reasons for data inclusion, and data availability. One must also consider variances in data availability over time. Most studies that analyze administrative data are retrospective in nature and fall into the cross-sectional, cohort, or case–control category of epidemiologic study design. Drug-utilization studies or studies that describe patterns of medication use are generally cross-sectional in nature. Studies that aim to explore a cause-and-effect relationship are likely to be case–control or cohort studies. When examining associations, there are very important study design aspects that should be considered, including enrollment of incident users11 (if the study pertains to a medication exposure), use of an active comparator design (if possible),12 and consideration of important confounding variables. These topics can be complicated and are beyond the scope of this article. However, these topics are thoroughly discussed elsewhere.7,13 Other important study design considerations include the validity and potential misclassification of study variables (i.e., exposure, outcome, and confounders). For instance, depending on the variables being measured, claims data will vary in the degree of accuracy of the measurement depending on the type of claims and the time window being 186
used. For example, if a resident wants to identify patients with diabetes, a definition requiring either a diagnosis code or a pharmacy claim for any oral antidiabetes medication will probably identify the majority of patients with this diagnosis. However, this approach might falsely label some patients as having diabetes. To the contrary, requiring both a specific diagnosis code and a pharmacy claim will minimize false-positive results but increase false-negative results. Given these complicated factors, including the operational definitions for study variables with optimum tradeoff between sensitivity and specificity, it is important that a pharmacoepidemiologist or biostatistician familiar with the conduct of administrative claims analysis be consulted early and often throughout the duration of the residency project. Without an appropriate study design tailored to the research question, the results will be biased and have limited clinical impact. Spurious results and interpretation. As was previously stated, conducting a study using administrative claims data often involves the inclusion of a very large study population. As the numbers are quite large, the probability of random errors is reduced. With fewer chances of random errors, the chance of a low p value (i.e.,
