LETTERS TO THE EDITOR
nature publishing group
ADR Improvement: The Result of the Intervention or the Hawthorne Effect Maida J. Sewitch, PhD1, Stephanie Carpentier, MD2 and Talat Bessissow, MD3 doi:10.1038/ajg.2013.309
To the Editor: The adenoma detection rate (ADR) is an independent predictor of interval colorectal cancer that suggests screening failure, and is considered an important colonoscopy quality indicator (1). We read with great interest the study by Coe et al., (2) who are to be congratulated for their experimental study that showed that ADRs can be improved. However, we have concerns about the study methodology and, therefore, the conclusions drawn. Among the strengths of an experimental study are randomization of the intervention and utilization of similar study procedures (i.e., number of contacts and assessments) in the two study groups to avoid introducing bias (3). Randomization and similar number of contacts and assessments ensure that the intervention and control groups are similar in all respects, except for the intervention under study. Randomization is performed to produce balanced characteristics (potential confounding variables) between the study groups, but does not work well with a small sample size, such as with 15 endoscopists. Moreover, the use of very different study procedures resulted in intervention physicians receiving much more attention compared to control physicians. Intervention group physicians were given two 1-h educational sessions, preand post-testing, access to an intranet site, and repeated monthly feedback on their ADRs. In comparison, control group physicians received no education, no testing, no intranet access, and no contact from the study personnel; they had some awareness © 2013 by the American College of Gastroenterology
that they were being monitored and had received ADR feedback at the start of the study. Because the intervention group was consistently reminded throughout the study of their ongoing participation, the Hawthorne effect was more pronounced in this group compared with the control group. The differential effect of the Hawthorne effect may itself have been the study intervention responsible for the improved ADRs. While we applaud the investigators for getting the cooperation of all endoscopists in their unit and for being among the first to implement an intervention aimed at improving ADRs, methodological issues need to be addressed before conclusive results can be drawn about the value of an educational intervention to increase ADRs. CONFLICT OF INTEREST
The authors declare no conflict of interest. REFERENCES 1. Kaminski MF, Regula J, Kraszewska E et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362: 1795–803. 2. Coe SG, Crook JE, Diehl NN et al. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol 2013;108:219–26. 3. Kramer MS. Epidemiologic research design: an overview. Clinical Epidemiology and Biostatistics. A Primer for Clinical Investigators and Decision-Makers. Springer-Verlag: Heidelberg, Germany, 1988, pp 37–46. 1
Department of Medicine, McGill University, Montreal, Quebec, Canada; 2Division of Gastroenterology, McGill University, Montreal, Quebec, Canada; 3Division of Gastroenterology, McGill University Health Center, Montreal, Quebec, Canada. Correspondence: Maida J. Sewitch, PhD, Department of Medicine, McGill University, 687 Pine Avenue West, V Building V2.15, Montreal, Quebec, Canada H3A 1A1. E-mail: [email protected]
To the Editor: We would like to thank Dr Sewitch et al. (1) for their interest in our paper. They raised two concerns about our study. The first is whether the randomization was appropriate for small sample size. We certainly agree that the smaller the sample size the more likely there will be differences in the groups. Despite this inherent limitation, we still feel that randomization was still the most effective strategy for dividing physicians into two groups. We carefully stratified randomization based on the baseline adenoma detection rate (ADR). We also attempted to control for overall volume by selecting only physicians who had relatively high volume practices. The second concern raised was whether the Hawthorne effect may have contributed to the observation of improved adenomatous detection rate. We certainly agree that it could have done so. In fact it was our intention that awareness of being monitored may contribute to improved ADR. The goal of our study was to show that a relatively simple, low-cost educational intervention combined with regular feedback would improve the quality of colonoscopy as measured by ADR. A Hawthorne effect, in this case, was an intentional therapeutic intervention to achieve the desired outcome. We cannot discern whether the simple knowledge of being monitored, as compared to the other active intervention such as the education program and regular feedback, was associated with the outcome. The specific effect of Hawthorne was also diminished between groups, as both groups were aware of the ongoing study and that ADR was being monitored. We discussed this in detail in the discussion section of the paper. We chose a multifactorial intervention because other studies mentioned in discussion using single interventions did not achieve the desired outcome.
Response to Sewitch et al.
CONFLICT OF INTEREST
Michael B. Wallace, MD, MPH1
REFERENCES
doi:10.1038/ajg.2013.312
1. Sewitch MJF, Carpentier S, Bessissow T. ADR improvement: the result of the intervention
The author declares no conflict of interest.
The American Journal of GASTROENTEROLOGY
1929
1930
Letters to the Editor
or the Hawthorne effect. Am J Gastroenterol 2013;108:1929 (this issue). 1 Mayo Clinic, Jacksonville, Florida, USA. Correspondence: Michael B. Wallace, MD, MPH, Professor of Medicine, Mayo Clinic, Jacksonville, Florida, USA. E-mail: [email protected]
The Role of Diagnostic Colonoscopy in Constipation: A Quality Improvement Project Brittny Neis, BA1, Doris Nguyen, MD2, Amindra Arora, MBBS1 and Sunanda Kane, MD, MSPH, FACG1 doi:10.1038/ajg.2013.337
bility and reduce the number ordered, an educational intervention involved sharing these results with colleagues. Over the next 6 months, only eight, or 0.14% of all procedures ordered over the subsequent time period, had the sole indication of constipation. None of the eight had any clinically relevant pathology. The results from our busy endoscopy practice affirm that constipation alone should not be an indication for colonoscopy (2,3) and that simple educational intervention can increase the availability for procedures more likely to yield significant findings, ultimately providing more cost-effective care. Maximizing health-care resources to benefit the most number of people by minimizing utilization of expensive tests for inappropriate indications is an important goal as we move into the next generation of health-care delivery. CONFLICT OF INTEREST
To the Editor: We read with great interest the recently published systematic review and meta-analysis by Power et al. (1) on the association between constipation and colorectal cancer (CRC). They found that there is no increase in prevalence of CRC in patients or individuals with constipation. We thank Dr Power and his group for their work as we took this assumption for ourselves, and implemented a quality improvement project to quantitate: (i) the diagnostic yield of colonoscopy when the sole indication was constipation, and (ii) the success of an educational intervention to subsequently reduce the number of unnecessary tests with low pretest probability for an abnormality. The anticipated benefit of a successful program would be to eliminate unnecessary procedures to allow for improved access for those procedures clearly indicated. Utilizing Define, Measure, Analyze, Improve and Control (DMAIC) methodology, we first measured over a 2-year time period how many procedures were ordered for the sole indication of constipation. Two hundred and ninety one procedures were ordered, of which 39% were completely without any diagnostic abnormality. The remainder yielded benign findings, including polyps < 5 mm, diverticulosis, hemorrhoids, and melanosis coli. No cases of CRC were found. To improve our accessiThe American Journal of GASTROENTEROLOGY
reducing the number of colonoscopies requested with constipation as the sole indication in their center. Constipation is a common complaint in the community (3), and may be associated with a reduction in survival (4), but our study demonstrated that there was no increase in odds of colorectal cancer in patients with constipation, compared with those without, at colonoscopy, suggesting that constipation alone does not warrant colonoscopic investigation, unless other alarm symptoms or signs are present, although these also perform poorly in predicting a diagnosis of CRC (5). Physicians continue to refer constipated patients for lower gastrointestinal examination in order to rule out colorectal cancer. Hopefully our data, as well as the quality improvement program reported by Neis et al., will act as a deterrent to such an approach.
The authors declare no conflict of interest. REFERENCES
CONFLICT OF INTEREST
1. Power AM, Talley NJ, Ford A. Association between constipation and colorectal cancer: systematic review and meta-analysis of observational studies. Am J Gastroenterol 2013;108:894–903. 2. Shah ND, Chitkara DK, Locke GR et al. Ambulatory care for constipation in the United States, 1993–2004. Am J Gastroenterol 2008;103:1746–53. 3. On On Chan A, Mo Hui W, Leung G et al. Patients with functional constipation do not have increased prevalence of colorectal cancer precursors. Gut 2007;56:451–2.
The authors declare no conflict of interest.
1
Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA; 2 Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA. Correspondence: Sunanda Kane, MD, MSPH, FACG, Division of Gastroenterology and Hepatology, 200 First Street SW, Rochester, Minnesota 55905, USA. E-mail: [email protected]
Response to Neis et al. Alexander C. Ford, MBChB, MD, FRCP1,2 and Nicholas J. Talley, MD, PhD3 doi:10.1038/ajg.2013.338
To the Editor: We thank Dr Neis et al. (1) for their letter about our study (2). We congratulate them on their work in
REFERENCES 1. Neis B, Nguyen D, Arora A et al. The role of diagnostic colonoscopy in constipation: a quality improvement project. Am J Gastroenterol 2013;108:1930 (this issue). 2. Power AM, Talley NJ, Ford AC. Association between constipation and colorectal cancer: Systematic review and meta-analysis of observational studies. Am J Gastroenterol 2013;108:894–903. 3. Suares NC, Ford AC. Prevalence of, and risk factors for, chronic idiopathic constipation in the community: Systematic review and meta-analysis. Am J Gastroenterol 2011;106: 1582–91. 4. Chang JY, Locke GR III, McNally MA et al. Impact of functional gastrointestinal disorders on survival in the community. Am J Gastroenterol 2010;105:822–32. 5. Ford AC, Veldhuyzen Van Zanten SJO, Rodgers CC et al. Diagnostic utility of alarm features for colorectal cancer: Systematic review and meta-analysis. Gut 2008;57: 1545–53.
1
Leeds Gastroenterology Institute, St. James’s University Hospital, Leeds, UK; 2Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK; 3Faculty of Health, University of Newcastle, New South Wales, Australia. Correspondence: Alexander C. Ford, MBChB, MD, FRCP, Leeds Gastroenterology Institute, St. James’s University Hospital, 4th Floor, Bexley Wing, Beckett Street, Leeds LS9 7TF, UK. E-mail: [email protected]
VOLUME 108 | DECEMBER 2013 www.amjgastro.com
nature publishing group
ADR Improvement: The Result of the Intervention or the Hawthorne Effect Maida J. Sewitch, PhD1, Stephanie Carpentier, MD2 and Talat Bessissow, MD3 doi:10.1038/ajg.2013.309
To the Editor: The adenoma detection rate (ADR) is an independent predictor of interval colorectal cancer that suggests screening failure, and is considered an important colonoscopy quality indicator (1). We read with great interest the study by Coe et al., (2) who are to be congratulated for their experimental study that showed that ADRs can be improved. However, we have concerns about the study methodology and, therefore, the conclusions drawn. Among the strengths of an experimental study are randomization of the intervention and utilization of similar study procedures (i.e., number of contacts and assessments) in the two study groups to avoid introducing bias (3). Randomization and similar number of contacts and assessments ensure that the intervention and control groups are similar in all respects, except for the intervention under study. Randomization is performed to produce balanced characteristics (potential confounding variables) between the study groups, but does not work well with a small sample size, such as with 15 endoscopists. Moreover, the use of very different study procedures resulted in intervention physicians receiving much more attention compared to control physicians. Intervention group physicians were given two 1-h educational sessions, preand post-testing, access to an intranet site, and repeated monthly feedback on their ADRs. In comparison, control group physicians received no education, no testing, no intranet access, and no contact from the study personnel; they had some awareness © 2013 by the American College of Gastroenterology
that they were being monitored and had received ADR feedback at the start of the study. Because the intervention group was consistently reminded throughout the study of their ongoing participation, the Hawthorne effect was more pronounced in this group compared with the control group. The differential effect of the Hawthorne effect may itself have been the study intervention responsible for the improved ADRs. While we applaud the investigators for getting the cooperation of all endoscopists in their unit and for being among the first to implement an intervention aimed at improving ADRs, methodological issues need to be addressed before conclusive results can be drawn about the value of an educational intervention to increase ADRs. CONFLICT OF INTEREST
The authors declare no conflict of interest. REFERENCES 1. Kaminski MF, Regula J, Kraszewska E et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362: 1795–803. 2. Coe SG, Crook JE, Diehl NN et al. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol 2013;108:219–26. 3. Kramer MS. Epidemiologic research design: an overview. Clinical Epidemiology and Biostatistics. A Primer for Clinical Investigators and Decision-Makers. Springer-Verlag: Heidelberg, Germany, 1988, pp 37–46. 1
Department of Medicine, McGill University, Montreal, Quebec, Canada; 2Division of Gastroenterology, McGill University, Montreal, Quebec, Canada; 3Division of Gastroenterology, McGill University Health Center, Montreal, Quebec, Canada. Correspondence: Maida J. Sewitch, PhD, Department of Medicine, McGill University, 687 Pine Avenue West, V Building V2.15, Montreal, Quebec, Canada H3A 1A1. E-mail: [email protected]
To the Editor: We would like to thank Dr Sewitch et al. (1) for their interest in our paper. They raised two concerns about our study. The first is whether the randomization was appropriate for small sample size. We certainly agree that the smaller the sample size the more likely there will be differences in the groups. Despite this inherent limitation, we still feel that randomization was still the most effective strategy for dividing physicians into two groups. We carefully stratified randomization based on the baseline adenoma detection rate (ADR). We also attempted to control for overall volume by selecting only physicians who had relatively high volume practices. The second concern raised was whether the Hawthorne effect may have contributed to the observation of improved adenomatous detection rate. We certainly agree that it could have done so. In fact it was our intention that awareness of being monitored may contribute to improved ADR. The goal of our study was to show that a relatively simple, low-cost educational intervention combined with regular feedback would improve the quality of colonoscopy as measured by ADR. A Hawthorne effect, in this case, was an intentional therapeutic intervention to achieve the desired outcome. We cannot discern whether the simple knowledge of being monitored, as compared to the other active intervention such as the education program and regular feedback, was associated with the outcome. The specific effect of Hawthorne was also diminished between groups, as both groups were aware of the ongoing study and that ADR was being monitored. We discussed this in detail in the discussion section of the paper. We chose a multifactorial intervention because other studies mentioned in discussion using single interventions did not achieve the desired outcome.
Response to Sewitch et al.
CONFLICT OF INTEREST
Michael B. Wallace, MD, MPH1
REFERENCES
doi:10.1038/ajg.2013.312
1. Sewitch MJF, Carpentier S, Bessissow T. ADR improvement: the result of the intervention
The author declares no conflict of interest.
The American Journal of GASTROENTEROLOGY
1929
1930
Letters to the Editor
or the Hawthorne effect. Am J Gastroenterol 2013;108:1929 (this issue). 1 Mayo Clinic, Jacksonville, Florida, USA. Correspondence: Michael B. Wallace, MD, MPH, Professor of Medicine, Mayo Clinic, Jacksonville, Florida, USA. E-mail: [email protected]
The Role of Diagnostic Colonoscopy in Constipation: A Quality Improvement Project Brittny Neis, BA1, Doris Nguyen, MD2, Amindra Arora, MBBS1 and Sunanda Kane, MD, MSPH, FACG1 doi:10.1038/ajg.2013.337
bility and reduce the number ordered, an educational intervention involved sharing these results with colleagues. Over the next 6 months, only eight, or 0.14% of all procedures ordered over the subsequent time period, had the sole indication of constipation. None of the eight had any clinically relevant pathology. The results from our busy endoscopy practice affirm that constipation alone should not be an indication for colonoscopy (2,3) and that simple educational intervention can increase the availability for procedures more likely to yield significant findings, ultimately providing more cost-effective care. Maximizing health-care resources to benefit the most number of people by minimizing utilization of expensive tests for inappropriate indications is an important goal as we move into the next generation of health-care delivery. CONFLICT OF INTEREST
To the Editor: We read with great interest the recently published systematic review and meta-analysis by Power et al. (1) on the association between constipation and colorectal cancer (CRC). They found that there is no increase in prevalence of CRC in patients or individuals with constipation. We thank Dr Power and his group for their work as we took this assumption for ourselves, and implemented a quality improvement project to quantitate: (i) the diagnostic yield of colonoscopy when the sole indication was constipation, and (ii) the success of an educational intervention to subsequently reduce the number of unnecessary tests with low pretest probability for an abnormality. The anticipated benefit of a successful program would be to eliminate unnecessary procedures to allow for improved access for those procedures clearly indicated. Utilizing Define, Measure, Analyze, Improve and Control (DMAIC) methodology, we first measured over a 2-year time period how many procedures were ordered for the sole indication of constipation. Two hundred and ninety one procedures were ordered, of which 39% were completely without any diagnostic abnormality. The remainder yielded benign findings, including polyps < 5 mm, diverticulosis, hemorrhoids, and melanosis coli. No cases of CRC were found. To improve our accessiThe American Journal of GASTROENTEROLOGY
reducing the number of colonoscopies requested with constipation as the sole indication in their center. Constipation is a common complaint in the community (3), and may be associated with a reduction in survival (4), but our study demonstrated that there was no increase in odds of colorectal cancer in patients with constipation, compared with those without, at colonoscopy, suggesting that constipation alone does not warrant colonoscopic investigation, unless other alarm symptoms or signs are present, although these also perform poorly in predicting a diagnosis of CRC (5). Physicians continue to refer constipated patients for lower gastrointestinal examination in order to rule out colorectal cancer. Hopefully our data, as well as the quality improvement program reported by Neis et al., will act as a deterrent to such an approach.
The authors declare no conflict of interest. REFERENCES
CONFLICT OF INTEREST
1. Power AM, Talley NJ, Ford A. Association between constipation and colorectal cancer: systematic review and meta-analysis of observational studies. Am J Gastroenterol 2013;108:894–903. 2. Shah ND, Chitkara DK, Locke GR et al. Ambulatory care for constipation in the United States, 1993–2004. Am J Gastroenterol 2008;103:1746–53. 3. On On Chan A, Mo Hui W, Leung G et al. Patients with functional constipation do not have increased prevalence of colorectal cancer precursors. Gut 2007;56:451–2.
The authors declare no conflict of interest.
1
Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA; 2 Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA. Correspondence: Sunanda Kane, MD, MSPH, FACG, Division of Gastroenterology and Hepatology, 200 First Street SW, Rochester, Minnesota 55905, USA. E-mail: [email protected]
Response to Neis et al. Alexander C. Ford, MBChB, MD, FRCP1,2 and Nicholas J. Talley, MD, PhD3 doi:10.1038/ajg.2013.338
To the Editor: We thank Dr Neis et al. (1) for their letter about our study (2). We congratulate them on their work in
REFERENCES 1. Neis B, Nguyen D, Arora A et al. The role of diagnostic colonoscopy in constipation: a quality improvement project. Am J Gastroenterol 2013;108:1930 (this issue). 2. Power AM, Talley NJ, Ford AC. Association between constipation and colorectal cancer: Systematic review and meta-analysis of observational studies. Am J Gastroenterol 2013;108:894–903. 3. Suares NC, Ford AC. Prevalence of, and risk factors for, chronic idiopathic constipation in the community: Systematic review and meta-analysis. Am J Gastroenterol 2011;106: 1582–91. 4. Chang JY, Locke GR III, McNally MA et al. Impact of functional gastrointestinal disorders on survival in the community. Am J Gastroenterol 2010;105:822–32. 5. Ford AC, Veldhuyzen Van Zanten SJO, Rodgers CC et al. Diagnostic utility of alarm features for colorectal cancer: Systematic review and meta-analysis. Gut 2008;57: 1545–53.
1
Leeds Gastroenterology Institute, St. James’s University Hospital, Leeds, UK; 2Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK; 3Faculty of Health, University of Newcastle, New South Wales, Australia. Correspondence: Alexander C. Ford, MBChB, MD, FRCP, Leeds Gastroenterology Institute, St. James’s University Hospital, 4th Floor, Bexley Wing, Beckett Street, Leeds LS9 7TF, UK. E-mail: [email protected]
VOLUME 108 | DECEMBER 2013 www.amjgastro.com
