COMMENTARIES
Editorials represent the opinions of the authors and not necessarily those of the American Dental Association.
GUEST EDITORIAL
The limitations of using insurance data for research
Jeffrey Hyman, DDS, PhD
I
n the March issue of The Journal, there was a reanalysis1 of an earlier study by Giannobile and colleagues2 and responses by Braun and colleagues3 and Ioannidis.4 These studies use data from Delta Dental of Michigan, a dental insurance claims database that includes data on 60 million people.5 Insurance data have become very popular for researchers and have been used in a number of oral health–related studies, often looking at oral health treatment and possible reductions in the incidence of systemic diseases.6,7 Insurance data have many advantages for researchers. The data are already collected and the sample sizes are very large, making it easy to achieve statistically significant results. This makes it more likely that the study will be published, due to the well-known issue of publication bias that favors publication of significant results.8 Given high research costs, it is extremely unlikely that studies involving collection of new data with the very large sample sizes available in insurance databases will be ever be done. Moreover, the large number of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes combined with large sample sizes of insurance databases allows researchers to address a wide range of questions. However, studies that are based on insurance claims or other third-party data are often misinterpreted or flawed because the information contained in insurance data is often limited, making it unsuitable for exploring important questions about disease and risk factors, causation, and treatment. Prominent issues that must be considered in evaluating studies based on third-party data include external validity, selection bias, confounding, misclassification bias, and causality. External validity—or the ability to generalize the results of a study to a wider population—is a major consideration when interpreting studies that are based on data from third-party payers. In 2012, approximately 60% of the US population had dental insurance, and these people were healthier than those who did not have insurance.9 They also had higher incomes and probably more education.10 Results based on these studies likely only apply to those with insurance. Most importantly, studies that are based on insurance data may be flawed or misinterpreted because they are often subject to selection bias or confounding. Insurance data are primarily collected for financial purposes and not to answer important questions about disease such as prevalence, etiology, risk, and treatment. Selection bias occurs when a sample of study participants is not representative of the population that is of interest. Confounding is similar and occurs when other variables associated with a disease are unevenly distributed
JADA 146(5)
http://jada.ada.org
May 2015 283
COMMENTARIES
between the study groups. This distorts (or confounds) the apparent association between exposure to something that may be a cause of a disease (or a risk factor) and the disease or outcome. Moreover, selection bias and confounding can sometimes overlap.11 Selection bias and confounding can be confusing and difficult to understand, but the bottom line is that selection bias prevents us from applying our results to the wider population and produces results that may be distorted or wrong, whereas confounding can result in finding associations between the exposure under study and the disease that are misleading or wrong.12 Welldesigned studies must consider the important issues of selection bias and confounding and measure exposures and outcomes correctly. They need to include a number of factors or variables beyond the exposure or variable of interest and the outcome (disease) to prevent selection bias and control confounding. Bias and confounding have caused many prominent studies to be totally wrong. For example, in 1981 MacMahon and colleagues13 published the results of a case control study of pancreatic cancer. Case control studies involve a comparison of 2 groups of people, one with the disease (cases) and one without disease (controls). The groups are compared to determine how many people in each group have an exposure that is being studied. This study13 suggested that coffee “might account for a substantial proportion of the cases of this disease in the United States.” This was incorrect due to selection bias. The hospitalized control patients were selected because they had the same physicians as patients who were being treated for pancreatic cancer (case patients). Because the control patients had gastrointestinal disease, few of them drank coffee, and so they were not representative of the wider population. This selection bias incorrectly made coffee look like a major risk factor for pancreatic
284 JADA 146(5) http://jada.ada.org
cancer.14 Dr. MacMahon chaired the Epidemiology Department at the Harvard School of Public Health for 30 years. This demonstrates how even a highly competent researcher can make a serious mistake. A second example demonstrates both confounding and selection bias. Until the 1990s, hormone replacement therapy (HRT) was widely prescribed for women to relieve the symptoms of menopause. It was also believed to reduce the risk of cardiovascular disease, especially after several large observational studies found it to be protective.15,16 However, by 2002, randomized controlled clinical trials17,18 had shown that HRT actually increased the chance of a heart attack.19 In the earlier studies, women and their physicians “selfselected” for HRT therapy. People who choose a treatment tend to be more educated, wealthier, nonsmokers, have healthier diets, health insurance, and follow their doctor’s orders more carefully (that is, they are very different than people who do not select treatment). This caused a healthy-user bias,20 a form of selection bias. This is similar to the problem of external validity among those who have insurance. The earlier studies also were confounded by indication,21 a type of confounding that occurs when treatments are not assigned randomly to patients, but because the treatment is expected to have some future benefit. In summary, the 2 observational studies15,16 were biased, and they did not look at all of the confounding variables. Therefore, HRT appeared to be protective for heart disease. The 2 later studies were randomized clinical trials, which avoided these problems.17,18 Randomization is the standard method used to prevent selection bias in cohort studies and to obtain treatment, alternative treatment, or nontreatment groups that have roughly equal initial risk for the outcome under study. However, it is difficult or impossible to randomize insurance data, because the participants have already self-selected for
May 2015
their treatments. Overall, studies that are based on insurance data are at high risk for selection bias and confounding. If, for example, one wanted to study the possible association of dental treatment and cardiovascular disease risk, the dental treatment and nontreatment groups would need to have equal baseline risk for cardiovascular disease. The United Kingdom National Health Service22 lists these risk factors for cardiovascular disease: - hypertension; - smoking; - high blood cholesterol; - diabetes; - lack of exercise; - being overweight or obese; - family history of heart disease; - ethnic background. If these variables are not included in the study (along with the healthy user variables), it is likely that the same mistake would be made as in the early HRT observational studies, in which a reduction in risk was attributed to a treatment when it was really an artifact of selection bias and the resulting unequal baseline risk. Misclassification (or errors of measurement) of exposures and outcomes are also a frequent problem, which can badly bias results. For example, studies using prescription claims data are susceptible to misclassification bias due to prescriptions being filled but not used, prescriptions only being taken as needed, health conditions not reported, and prescriptions being filled outside of the insurance plan and being absent from the claims data.23 Dental claims data are particularly susceptible to misclassification because they are primarily based on CDT procedure codes24 and not on diagnoses. ICD-9-CM diagnosis codes are often only submitted for oral surgical procedures, and periodontal probing data are frequently only included to support periodontal surgery. Some studies of oral health treatments and systemic diseases have used treatment as an exposure.7 These types of studies are at high
COMMENTARIES
risk of misclassifications and other biases because the researchers do not know how much disease there is in the study groups, or how those who self selected for the treatment differ in their initial risk for the systemic disease compared with those who did not have the treatment. When an oral health ICD-9-CM diagnosis is available in the data, it is often incomplete. For example, there are only 4 codes for gingivitis, 3 for chronic periodontitis, and so forth. When a complex diagnosis is forced into 1 of a few broad categories, most of the diagnostic detail is lost and misclassification can easily result. Studies that find an association or correlation frequently make a claim of causality. Causality is actually extremely difficult to establish from observational studies. There are a number of causality criteria that must be met. The most problematic is often the need to show temporality, or as Hill stated, “which is the cart and which is the horse?”25 With insurance data, we know when a condition was first treated and possibly when it was first diagnosed, at least under the current policy. However, we generally have no idea when the condition or exposure actually started. In other words, we can sometimes see the horse and the cart, but we rarely know which came first. Ioannidis, who wrote a commentary in the March issue of JADA,4 has famously estimated that most published medical research is wrong.26 His estimates are not universally accepted,27 but he has highlighted a number of serious and pervasive problems in research. It is always difficult to do high-quality research, but it is imperative that research that has implications for patient care have broad generalizability, minimize biases, carefully consider causality, and control confounding. Insurance claims databases can be a valuable resource for research if they are used correctly and if their limitations are understood. However, if studies are done when important information is lacking or unavailable,
they may be seriously flawed. It is very possible that the results of such a study will be misleading and that the conclusions will be wrong. Because studies based on insurance data are so susceptible to biases and confounding, we should keep these potential problems in mind when we read them. If we think the study has any of the problems that we have discussed here, then we should be aware of the very real possibility that the study is wrong. n http://dx.doi.org/10.1016/j.adaj.2015.02.010 Copyright ª 2015 American Dental Association. All rights reserved.
Dr. Hyman is a dental epidemiologist in Vienna, VA, e-mail [email protected]. Address correspondence to Dr. Hyman. Disclosure. Dr. Hyman did not report any disclosures. 1. Diehl SR, Kuo F, Hart TC. Interleukin 1 genetic tests provide no support for reduction of preventive dental care. JADA. 2015;146(3):164-173.e4. 2. Giannobile WV, Braun TM, Caplis AK, Doucette-Stamm L, Duff GW, Kornman KS. Patient stratification for preventive care in dentistry. J Dent Res. 2013;92(8):694-701. 3. Braun TM, Doucette-Stamm L, Duff GW, Kornman KS, Giannobile WV. Counterpoint: risk factors, including genetic information, add value in stratifying patients for optimal preventive dental care. JADA. 2015;146(3):174-178. 4. Ioannidis JPA. Preventing tooth loss with biannual dental visits and genetic testing: does it work? JADA. 2015;146(3):141-143. 5. Delta Dental Plans Association. Delta Dental by the numbers. Available at: www.deltadental. com/Public/Company/stats2.jsp?DView¼About DeltaDentalStats. Accessed January 30, 2015. 6. Lee YL, Hu HY, Chou P, Chu D. Dental prophylaxis decreases the risk of acute myocardial infarction: a nationwide population-based study in Taiwan. Clin Interv Aging. 2015;10:175-182. 7. Jeffcoat MK, Jeffcoat RL, Gladowski PA, Bramson JB, Blum JJ. Impact of periodontal therapy on general health: evidence from insurance data for five systemic conditions. Am J Prev Med. 2014;47(2):166-174. 8. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337(8746):867-872. 9. National Association of Dental Plans. Who has dental benefits? http://www.nadp.org/ Dental_Benefits_Basics/Dental_BB_1.aspx. Accessed January 29, 2015. 10. Haley J, Kenney G, Pelletier J. Access to affordable dental care: gaps for low-income adults. The Henry J. Kaiser Family Foundation. Available at: www.allhealth.org/briefingmaterials/ kaiserlowincomeaccesssurvey-1276.pdf. Accessed January 30, 2015. 11. Rothman KJ, Greenland S, Lash TL. Validity in epidemiologic studies. In: Modern
Epidemiology. 3rd ed. Philadelphia, PA: Lippincott, Williams, & Wilkins; 2012:136. 12. Paneth N. Supercourse lecture: bias and confounding—part 1. Available at: www.pitt. edu/wsuper1/lecture/lec18951/index.htm. Accessed January 28, 2015. 13. MacMahon B, Yen S, Trichopoulos D, Warren K, Nardi G. Coffee and cancer of the pancreas. N Engl J Med. 1981;304(11):630-633. 14. Pai M, Kaufman JS. Bias file 2: should we stop drinking coffee? The story of coffee and pancreatic cancer. Available at: http://pages.stat. wisc.edu/wifischer/Intro_Stat/Lecture_ Notes/6_-_Statistical_Inference/BIAS.pdf. Accessed January 29, 2015. 15. Stampfer MJ, Colditz GA, Willett WC, et al. Postmenopausal estrogen therapy and cardiovascular disease. Ten-year follow-up from the nurses’ health study. N Engl J Med. 1991; 325(11):756-762. 16. Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med. 1992;117(12):1016-1037. 17. Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/Progestin Replacement Study (HERS) Research Group. JAMA. 1998;280(7):605-613. 18. Rossouw JE, Anderson JL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA. 2002;288(3):321-333. 19. Madhukar P, Kaufman JS. Bias file 1: the rise and fall of hormone replacement therapy. Available at: www.teachepi.org/documents/ courses/bfiles/The%20B%20Files_File1_HRT_ Final_Complete.pdf. Accessed January 29, 2015. 20. Shrank WH, Patrick AR, Alan Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J Gen Intern Med. 2011;26(5):546-550. 21. Signorello LB, McLaughlin JK, Lipworth L, Friis S, Sørensen HT, Blot WJ. Confounding by indication in epidemiologic studies of commonly used analgesics. Am J Ther. 2002;9(3):199-205. 22. NHS Choices. Cardiovascular disease— risk factors. Available at: www.nhs.uk/ Conditions/cardiovascular-disease/Pages/Riskfactors.aspx. Accessed January 27, 2015. 23. Funk MJ, Landi S. Misclassification in administrative claims data: quantifying the impact on treatment effect estimates. Curr Epidemiol Rep. 2014;1(4):175-185. 24. American Dental Association. For accurate coding, discover CDT 2015. Available at: www. ada.org/en/publications/ada-news/2014-archive/ september/for-accurate-coding-discover-cdt-2015. Accessed February 16, 2016. 25. Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965; 58(5):295-300. 26. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. 27. MIT Technology Review. The statistical puzzle over how much biomedical research is wrong; 2013. Available at: www.technologyreview. com/view/510126/the-statistical-puzzle-over-howmuch-biomedical-research-is-wrong. Accessed January 28, 2015.
JADA 146(5)
http://jada.ada.org
May 2015 285
Editorials represent the opinions of the authors and not necessarily those of the American Dental Association.
GUEST EDITORIAL
The limitations of using insurance data for research
Jeffrey Hyman, DDS, PhD
I
n the March issue of The Journal, there was a reanalysis1 of an earlier study by Giannobile and colleagues2 and responses by Braun and colleagues3 and Ioannidis.4 These studies use data from Delta Dental of Michigan, a dental insurance claims database that includes data on 60 million people.5 Insurance data have become very popular for researchers and have been used in a number of oral health–related studies, often looking at oral health treatment and possible reductions in the incidence of systemic diseases.6,7 Insurance data have many advantages for researchers. The data are already collected and the sample sizes are very large, making it easy to achieve statistically significant results. This makes it more likely that the study will be published, due to the well-known issue of publication bias that favors publication of significant results.8 Given high research costs, it is extremely unlikely that studies involving collection of new data with the very large sample sizes available in insurance databases will be ever be done. Moreover, the large number of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes combined with large sample sizes of insurance databases allows researchers to address a wide range of questions. However, studies that are based on insurance claims or other third-party data are often misinterpreted or flawed because the information contained in insurance data is often limited, making it unsuitable for exploring important questions about disease and risk factors, causation, and treatment. Prominent issues that must be considered in evaluating studies based on third-party data include external validity, selection bias, confounding, misclassification bias, and causality. External validity—or the ability to generalize the results of a study to a wider population—is a major consideration when interpreting studies that are based on data from third-party payers. In 2012, approximately 60% of the US population had dental insurance, and these people were healthier than those who did not have insurance.9 They also had higher incomes and probably more education.10 Results based on these studies likely only apply to those with insurance. Most importantly, studies that are based on insurance data may be flawed or misinterpreted because they are often subject to selection bias or confounding. Insurance data are primarily collected for financial purposes and not to answer important questions about disease such as prevalence, etiology, risk, and treatment. Selection bias occurs when a sample of study participants is not representative of the population that is of interest. Confounding is similar and occurs when other variables associated with a disease are unevenly distributed
JADA 146(5)
http://jada.ada.org
May 2015 283
COMMENTARIES
between the study groups. This distorts (or confounds) the apparent association between exposure to something that may be a cause of a disease (or a risk factor) and the disease or outcome. Moreover, selection bias and confounding can sometimes overlap.11 Selection bias and confounding can be confusing and difficult to understand, but the bottom line is that selection bias prevents us from applying our results to the wider population and produces results that may be distorted or wrong, whereas confounding can result in finding associations between the exposure under study and the disease that are misleading or wrong.12 Welldesigned studies must consider the important issues of selection bias and confounding and measure exposures and outcomes correctly. They need to include a number of factors or variables beyond the exposure or variable of interest and the outcome (disease) to prevent selection bias and control confounding. Bias and confounding have caused many prominent studies to be totally wrong. For example, in 1981 MacMahon and colleagues13 published the results of a case control study of pancreatic cancer. Case control studies involve a comparison of 2 groups of people, one with the disease (cases) and one without disease (controls). The groups are compared to determine how many people in each group have an exposure that is being studied. This study13 suggested that coffee “might account for a substantial proportion of the cases of this disease in the United States.” This was incorrect due to selection bias. The hospitalized control patients were selected because they had the same physicians as patients who were being treated for pancreatic cancer (case patients). Because the control patients had gastrointestinal disease, few of them drank coffee, and so they were not representative of the wider population. This selection bias incorrectly made coffee look like a major risk factor for pancreatic
284 JADA 146(5) http://jada.ada.org
cancer.14 Dr. MacMahon chaired the Epidemiology Department at the Harvard School of Public Health for 30 years. This demonstrates how even a highly competent researcher can make a serious mistake. A second example demonstrates both confounding and selection bias. Until the 1990s, hormone replacement therapy (HRT) was widely prescribed for women to relieve the symptoms of menopause. It was also believed to reduce the risk of cardiovascular disease, especially after several large observational studies found it to be protective.15,16 However, by 2002, randomized controlled clinical trials17,18 had shown that HRT actually increased the chance of a heart attack.19 In the earlier studies, women and their physicians “selfselected” for HRT therapy. People who choose a treatment tend to be more educated, wealthier, nonsmokers, have healthier diets, health insurance, and follow their doctor’s orders more carefully (that is, they are very different than people who do not select treatment). This caused a healthy-user bias,20 a form of selection bias. This is similar to the problem of external validity among those who have insurance. The earlier studies also were confounded by indication,21 a type of confounding that occurs when treatments are not assigned randomly to patients, but because the treatment is expected to have some future benefit. In summary, the 2 observational studies15,16 were biased, and they did not look at all of the confounding variables. Therefore, HRT appeared to be protective for heart disease. The 2 later studies were randomized clinical trials, which avoided these problems.17,18 Randomization is the standard method used to prevent selection bias in cohort studies and to obtain treatment, alternative treatment, or nontreatment groups that have roughly equal initial risk for the outcome under study. However, it is difficult or impossible to randomize insurance data, because the participants have already self-selected for
May 2015
their treatments. Overall, studies that are based on insurance data are at high risk for selection bias and confounding. If, for example, one wanted to study the possible association of dental treatment and cardiovascular disease risk, the dental treatment and nontreatment groups would need to have equal baseline risk for cardiovascular disease. The United Kingdom National Health Service22 lists these risk factors for cardiovascular disease: - hypertension; - smoking; - high blood cholesterol; - diabetes; - lack of exercise; - being overweight or obese; - family history of heart disease; - ethnic background. If these variables are not included in the study (along with the healthy user variables), it is likely that the same mistake would be made as in the early HRT observational studies, in which a reduction in risk was attributed to a treatment when it was really an artifact of selection bias and the resulting unequal baseline risk. Misclassification (or errors of measurement) of exposures and outcomes are also a frequent problem, which can badly bias results. For example, studies using prescription claims data are susceptible to misclassification bias due to prescriptions being filled but not used, prescriptions only being taken as needed, health conditions not reported, and prescriptions being filled outside of the insurance plan and being absent from the claims data.23 Dental claims data are particularly susceptible to misclassification because they are primarily based on CDT procedure codes24 and not on diagnoses. ICD-9-CM diagnosis codes are often only submitted for oral surgical procedures, and periodontal probing data are frequently only included to support periodontal surgery. Some studies of oral health treatments and systemic diseases have used treatment as an exposure.7 These types of studies are at high
COMMENTARIES
risk of misclassifications and other biases because the researchers do not know how much disease there is in the study groups, or how those who self selected for the treatment differ in their initial risk for the systemic disease compared with those who did not have the treatment. When an oral health ICD-9-CM diagnosis is available in the data, it is often incomplete. For example, there are only 4 codes for gingivitis, 3 for chronic periodontitis, and so forth. When a complex diagnosis is forced into 1 of a few broad categories, most of the diagnostic detail is lost and misclassification can easily result. Studies that find an association or correlation frequently make a claim of causality. Causality is actually extremely difficult to establish from observational studies. There are a number of causality criteria that must be met. The most problematic is often the need to show temporality, or as Hill stated, “which is the cart and which is the horse?”25 With insurance data, we know when a condition was first treated and possibly when it was first diagnosed, at least under the current policy. However, we generally have no idea when the condition or exposure actually started. In other words, we can sometimes see the horse and the cart, but we rarely know which came first. Ioannidis, who wrote a commentary in the March issue of JADA,4 has famously estimated that most published medical research is wrong.26 His estimates are not universally accepted,27 but he has highlighted a number of serious and pervasive problems in research. It is always difficult to do high-quality research, but it is imperative that research that has implications for patient care have broad generalizability, minimize biases, carefully consider causality, and control confounding. Insurance claims databases can be a valuable resource for research if they are used correctly and if their limitations are understood. However, if studies are done when important information is lacking or unavailable,
they may be seriously flawed. It is very possible that the results of such a study will be misleading and that the conclusions will be wrong. Because studies based on insurance data are so susceptible to biases and confounding, we should keep these potential problems in mind when we read them. If we think the study has any of the problems that we have discussed here, then we should be aware of the very real possibility that the study is wrong. n http://dx.doi.org/10.1016/j.adaj.2015.02.010 Copyright ª 2015 American Dental Association. All rights reserved.
Dr. Hyman is a dental epidemiologist in Vienna, VA, e-mail [email protected]. Address correspondence to Dr. Hyman. Disclosure. Dr. Hyman did not report any disclosures. 1. Diehl SR, Kuo F, Hart TC. Interleukin 1 genetic tests provide no support for reduction of preventive dental care. JADA. 2015;146(3):164-173.e4. 2. Giannobile WV, Braun TM, Caplis AK, Doucette-Stamm L, Duff GW, Kornman KS. Patient stratification for preventive care in dentistry. J Dent Res. 2013;92(8):694-701. 3. Braun TM, Doucette-Stamm L, Duff GW, Kornman KS, Giannobile WV. Counterpoint: risk factors, including genetic information, add value in stratifying patients for optimal preventive dental care. JADA. 2015;146(3):174-178. 4. Ioannidis JPA. Preventing tooth loss with biannual dental visits and genetic testing: does it work? JADA. 2015;146(3):141-143. 5. Delta Dental Plans Association. Delta Dental by the numbers. Available at: www.deltadental. com/Public/Company/stats2.jsp?DView¼About DeltaDentalStats. Accessed January 30, 2015. 6. Lee YL, Hu HY, Chou P, Chu D. Dental prophylaxis decreases the risk of acute myocardial infarction: a nationwide population-based study in Taiwan. Clin Interv Aging. 2015;10:175-182. 7. Jeffcoat MK, Jeffcoat RL, Gladowski PA, Bramson JB, Blum JJ. Impact of periodontal therapy on general health: evidence from insurance data for five systemic conditions. Am J Prev Med. 2014;47(2):166-174. 8. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337(8746):867-872. 9. National Association of Dental Plans. Who has dental benefits? http://www.nadp.org/ Dental_Benefits_Basics/Dental_BB_1.aspx. Accessed January 29, 2015. 10. Haley J, Kenney G, Pelletier J. Access to affordable dental care: gaps for low-income adults. The Henry J. Kaiser Family Foundation. Available at: www.allhealth.org/briefingmaterials/ kaiserlowincomeaccesssurvey-1276.pdf. Accessed January 30, 2015. 11. Rothman KJ, Greenland S, Lash TL. Validity in epidemiologic studies. In: Modern
Epidemiology. 3rd ed. Philadelphia, PA: Lippincott, Williams, & Wilkins; 2012:136. 12. Paneth N. Supercourse lecture: bias and confounding—part 1. Available at: www.pitt. edu/wsuper1/lecture/lec18951/index.htm. Accessed January 28, 2015. 13. MacMahon B, Yen S, Trichopoulos D, Warren K, Nardi G. Coffee and cancer of the pancreas. N Engl J Med. 1981;304(11):630-633. 14. Pai M, Kaufman JS. Bias file 2: should we stop drinking coffee? The story of coffee and pancreatic cancer. Available at: http://pages.stat. wisc.edu/wifischer/Intro_Stat/Lecture_ Notes/6_-_Statistical_Inference/BIAS.pdf. Accessed January 29, 2015. 15. Stampfer MJ, Colditz GA, Willett WC, et al. Postmenopausal estrogen therapy and cardiovascular disease. Ten-year follow-up from the nurses’ health study. N Engl J Med. 1991; 325(11):756-762. 16. Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med. 1992;117(12):1016-1037. 17. Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/Progestin Replacement Study (HERS) Research Group. JAMA. 1998;280(7):605-613. 18. Rossouw JE, Anderson JL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA. 2002;288(3):321-333. 19. Madhukar P, Kaufman JS. Bias file 1: the rise and fall of hormone replacement therapy. Available at: www.teachepi.org/documents/ courses/bfiles/The%20B%20Files_File1_HRT_ Final_Complete.pdf. Accessed January 29, 2015. 20. Shrank WH, Patrick AR, Alan Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J Gen Intern Med. 2011;26(5):546-550. 21. Signorello LB, McLaughlin JK, Lipworth L, Friis S, Sørensen HT, Blot WJ. Confounding by indication in epidemiologic studies of commonly used analgesics. Am J Ther. 2002;9(3):199-205. 22. NHS Choices. Cardiovascular disease— risk factors. Available at: www.nhs.uk/ Conditions/cardiovascular-disease/Pages/Riskfactors.aspx. Accessed January 27, 2015. 23. Funk MJ, Landi S. Misclassification in administrative claims data: quantifying the impact on treatment effect estimates. Curr Epidemiol Rep. 2014;1(4):175-185. 24. American Dental Association. For accurate coding, discover CDT 2015. Available at: www. ada.org/en/publications/ada-news/2014-archive/ september/for-accurate-coding-discover-cdt-2015. Accessed February 16, 2016. 25. Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965; 58(5):295-300. 26. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. 27. MIT Technology Review. The statistical puzzle over how much biomedical research is wrong; 2013. Available at: www.technologyreview. com/view/510126/the-statistical-puzzle-over-howmuch-biomedical-research-is-wrong. Accessed January 28, 2015.
JADA 146(5)
http://jada.ada.org
May 2015 285
