419
Aust. N.Z. J . Surg. 1991,61,419-422
COMPETENCY AND THE COLONOSCOPIST A LEARNING CURVE BRYANR. PARRY*AND SHEILA M. WILLIAMS+ Departments of *Surgery. *Gastroenterology,and 'Preventive and Social Medicine, University of Otago, Dunedin, New Zealand The first 334 consecutive unassisted studies of a trainee colonoscopist were audited to analyse the relationship between experience and a target 90% completion rate of colonoscopy. The cumulative sum (cusum) score was applied to examine the time trend for the attaining of the target 90% completion rate. This technique described a learning curve which showed, in this instance, that approximately 100 studies were necessary before a 90% completion rate was approached. A further 100 studies were required before this target completion rate was achieved consistently. A trend for continued improvement above the 90% completion rate level was also seen after 200 studies. Polypectomy was completed in 117 of the 123 studies where indicated. Cusum analysis may be a useful method for monitoring the progress of the trainee colonoscopist and the attainment of satisfactory competence. Training requirements, therefore, might better emphasize the attainment of an acceptable completion rate rather than an arbitrary quota per se. Key words: clinical competence, donoscopy, learning.
Introduction
For the trainee colonoscopist, Hippocrates' oftquoted aphorism, 'the life so short; the technique so long to learn', is most appropriate. Long, too, is the colon - even experienced endoscopists remark wryly on the length and tortuosity of the human colon.' Indeed, it is these features of the colonic anatomy that demand of the colonoscopist considerable dexterity in the control of the instrument as well as mastery of manoeuvres to reach the caecum. It is axiomatic, in the interests of diagnostic accuracy and therapeutic opportunity, that total colonoscopy be regarded as standard practice with the proviso that any therapeutic procedures found necessary, such as polypectomy. be carried out successfully as well. The completion rate for total colonoscopy is variously reported as between 55 and 83%,*-* although 90% is regarded as normative.' It is generally accepted that training in colonic endoscopy is necessary and protocols have been established accordingly. "-" The duration of such training is. however, arbitrary and although the term 'learning curve' is common parlance, few if any data are available to support this intuititive assumption and thereby permit the setting of rational guidelines. In industry, concern for quality control has given rise to various mathematical and statistical techCorrespandence: Bryan R. Parry. Senior Lecturer. Department of Surgery. University of Otago, Dunedin. New Zealand. Accepted for publication 19 December 1990.
niques to monitor performance of humans and machines. l3-I5 One such technique is the cumulative sum score (cusum) method which, from its name, generates a cumulative sum of pre-defined scores (which depend on outcome) from consecutive events. This sequence of arbitrary scores describes a curve, the slope of which enables trends in performance to be analysed. Using this technique, the unassisted, but supervised, early experience of one trainee colonoscopist with considerable previous upper gastrointestinal endoscopy experience was audited and analysed to determine the nature of the learning curve and the number of procedures required to achieve a nominal completion rate of 90%. Methods
From August 1987 to October 1990, the detailed, standardized records and dictated reports of consecutive total colonoscopy studies carried out by one of the authors (BRP) were analysed. This period extends from the commencement of the trainee's colonoscopic experience up to the current level of experience. The indications, findings and completeness or otherwise of the study were recorded. Completion of the study was defined as either reaching the caecum (or ileo-colic anastomosis where appropriate) or encountering an impassable obstructive lesion. Performance was assessed using the cusum score technique because of its usefulness as a device for showing time trends. I3-I5 In general. the cusum (Si) for a series of observations Xi . . . X, is defined as si = ' Zj= I(X" - Xj)
PARRY AND WILLIAMS
420
where & is a reference or target value representing some sort of expected value for the data.'' For this study the target value of 0.9 was chosen (representing a completion rate of 90%). X,was 1 if the study was completed and 0 otherwise. Si was plotted against i.
Results Of the 334 consecutive colonoscopy studies, 29 studies (8.7%) were excluded from the analysis because of inadequate bowel preparation leading to abandonment of the examination. Overall, complete studies were possible in 250 (82%) of the remaining 305 evaluable examinations, and polypectomy was carried out in 117 of 123 patients on whom it was indicated (95%). Details of the indications for colonoscopy and the primary findings are set out in Tables I and 2 respectively. The relatively low number of patients with inflammatory bowel disease studied was due to selection bias reflecting the special interests of other endoscopists in the Gastroenterological Unit. Table 3 sets out details of the incomplete studies by consecutive sets of 100 studies.
The cusum plot for the target value of 90% is shown in Fig. I . There is a steep positive slope for the first 100 studies, indicating a poor completion rate in relation to the 90% rate target. Between the 100th and 200th studies, however, the slope tends towards horizontal, indicating a completion rate approaching the target of 90%. After the 200th study, the slope becomes slightly negative, suggesting that the target completion rate of 90% was being exceeded. Table 2. Primary findings
No. patients
Finding Polyps No abnormality Inflammatory bowel disease Diverticular disease Carcinoma Angiodysplasia Other
I23 I I9 25
Total
305
21 14
I 2
Table 1. Indications for colonoscopy Indication
No. patients
Cancer surveillance Previous colorectal cancer Previous colorectal polyps Inflammatory bowel disease Family history colorectal cancer Polyps Bowel symptoms Bleeding Change of bowel habit Diarrhoea Barium enema abnormality Inflammatory bowel disease mapping Anaemia Other Total
81 50
7 3 56
33 9 6 25 17 12 6 305
0
100
200
300
Colonoicopy study no.
Fig. 1. Cusurn for 305 consecutive colonoscopy studies for completion rate target value of 90%. Performance in relation to the target value is indicated by the slope (positive slope: below target. zero slope: achieving and maintaining target. negative slope: exceeding target).
Table 3. Incomplete colonoscopy studies data Study no. 001-100
No.cases No. completed ( % ) Reason incomplete Sigmoid looping R. colon looping Instrument failure Patient distress Not recorded No. studies excluded for poor bowel preparation ( O h )
100
67 (67.0)
101-200
20 I -305
100 88 (88.0)
I05 95 (90.5)
15 4
4 7
I I 12
I
I I (9.9)
I
-
12 (10.7)
7 -
I I 6 (5.4)
42 I
COLONOSCOPY LEARNING CURVE
Discussion The cusum plot indicates that the completion rate increased with experience, thereby confirming a learning curve phenomenon in this instance. A completion rate of 90% was reached and maintained after approximately 200 studies, and exceeded subsequently. As it is almost certain that a wide variation exists among trainees in the rapidity or otherwise of their acquiring an acceptable completion rate, it may be that this trainee's quantum of studies represents an under- or over-estimation of the ideal. In this instance, the quota of study numbers usually recommended by various authorities to attain competence may have proved optimistic. Stratifying the studies into consecutive centuries (Table 3) proved useful in revealing the changing pattern for incomplete studies. Such a retrospective analysis, despite being useful from an audit per-spective, cannot depict an instananeous trend to which the operator might usefully react. Ongoing cusum analysis can, by contrast, point up failure to improve in the learning period, and subsequent deterioration from an attained standard. Although polypectomy presented no undue recognized difficulties, not all (5%) of intended polypectomies were successful. It is conceded too that polyps may have been missed, especially at the beginning of the trainee's experience. Nevertheless, the distinction between diagnostic and therapeutic colonoscopies for training purposes" is artificial and opportunity to acquire experience of polypectomy need not be deferred. The method of assessing competence used was completed, or total, colonoscopy. It might be argued that other methods were appropriate. Speed, as in surgery, is useful but is not necessarily associated with proficiency per se. Diagnostic sensitivity and specificity are valid criteria for comparing colonoscopy with other investigations such as double contrast barium enema, but are not suitable when colonoscopy alone is assessed, because of the lack of a reference standard. Complication rates are low in colonoscopy," so that the monitoring of complications alone would be unlikely to identify the incompetent trainee in other than the most extreme case. The overall high rate (8.7%) of unsatisfactory bowel preparation was of concern and this Unit turned increasingly during the course of this study to a colonic lavage electrolyte solution as standard preparation, particularly in the elderly. It is of some consolation, therefore, that the rate dropped to 5.4% (Table 3) for the last 105 studies. It is possible that 100-200 studies may be necessary to attain a 90% completion rate in colonoscopy. Because of individual differences in aptitude, however, training guidelines should be linked to the attainment and maintenance of an agreed comple-
tion rate of colonoscopy rather than fulfilling an arbitrary quota of studies per se. This highlights the need for a properly constructed supervised audit system for all trainee colonoscopists. The cusum analysis outlined here is suggested as an important adjunct in the monitoring of such training. Concern has been expressed about colonoscopy training and there have been calls for a concerted effort to equip and train more endoscopists, especially surgeon endoscopists. 1 1 * ' 7 ~ 1 8These must be matched by an equal concern to provide clear standards of competency, such as an acceptable colonoscopy completion rate, and the establishment of an audit mechanism to monitor trainees' performance - something already accepted as de rigueur for operative surgical training in theory, if not in practice.
Achnowledgements We thank our colleagues in the Gastroenterology Department, Prof. G. Barbezat, Drs M.Schlup, S. Parry and G.Langstreth, for their help and encouragement and Prof. A. M. Van Rij for his helpful critical advice.
References 1. COTTON P. B. & WILLIAMS C. B. (1982) Practical GastrointestinalEndoscopy, 2nd edn. Blackwell Scientific Publications. Oxford. 2. ALDRlDGE M.C. & SIM A. J. w. (1986) Colonoscopy findings in symptomatic patients without X-ray evidence of colonic neoplasms. Lancer ii,833-4. 3. ABERGT., LINGL., BRELAND U. & NORLUND A. (1985) Does flexible sigmoidoscopy have any justification as compared with complete colonoscopy? An analysis of cost-effectiveness and medical risk. Endoscopy 13, 133-6. 4. ABRAHMS J. S. (1982) A second look at colonoscopy: indications, failures, and costs. Arch. Surg. 117, 913-7. 5 . BATESON M. C. & BOUCHIER 1. A. (1988) Clinical Investigations in Gastroenterology. Kluwer Academic Publishers, Dordrecht. N. S. 6. DURDLEY P., WESTON P. M. T. & WILLIAMS (1987) Colonoscopy or Barium enema as initial investigation of colonic disease. Lancet ii, 549-5 I. 7. ISBISTERW. H. (1987) Colonoscopy: a ten year Wellington experience. N.Z.Med. J . 100, 74-7. D. W. & 017. 8. OBRECHT W. R., Wu W. C., GELFAND D. J. (1984) The extent of successful colonoscopy: a second assessment using modern equipment. Gastrointest. Radiol. 9, 161-2. 9. WILLIAMS C. B. (1987) Colonoscopy. Curr. Opin. Gastroenterol. 3, 36-42. 10. BENSON J. A. & COHENS. (1987) Evaluation of procedural skills in gastroenterologists. Gastroenterology 92, 254. 1 I. BRITISH SWlETY OF GASTROENTEROLOGY (1987) Future requirements for colonoscopy in Britain. Report of the Endoscopy Section Committee of the British Society of Gastroenterology. Gut 28, 772-5.
422 12. CONJOINT COMMITTEE ON ENCCISCOPY (1989) Endoscopy Training. Report of the Conjoint Committee on Endoscopy of the Gastroenterological Society of Australia, the Royal Australasian College of Physicians and the Royal Australasian College of Surgeons. RACS Bulletin 7. 19-20, 13. EWEN W. D. & KEMPK. W. (1960) Sampling inspection of continuous processes with no autocorrelation between successive results. Biornetriku 47, 363-80. 14. PAWE. S. (1954) Continuous inspection schemes. Biornetrika 41, 100- 15. 15. ALTMAND. G. & ROYSTONI . P. (1988) The hidden
PARRY AND WILLIAMS
effect of time. Srurisrirs in Medicine 7, 629-37. 16. REIERTSEN 0.. SKJ0T0 J., JACOBSENC. D. & ROSSELAND A. R. (1987) Complicationsof fibreoptic gastrointestinal endoscopy: five years experience in a central hospital. Endosropy 19, 1-6. 17. ISBISTER W.H. (1986) Quality control in endoscopy: Workshop report and personal comments. Aust. N.Z. J. Surg. 56. 205-8. 18. FUMIO NAKAYAMA (1988) Editorial. Impact of recent advances in imaging techniques on surgical practice and its implications. Ausr. N.Z.J. Surg. 58. 87-8.
Aust. N.Z. J . Surg. 1991,61,419-422
COMPETENCY AND THE COLONOSCOPIST A LEARNING CURVE BRYANR. PARRY*AND SHEILA M. WILLIAMS+ Departments of *Surgery. *Gastroenterology,and 'Preventive and Social Medicine, University of Otago, Dunedin, New Zealand The first 334 consecutive unassisted studies of a trainee colonoscopist were audited to analyse the relationship between experience and a target 90% completion rate of colonoscopy. The cumulative sum (cusum) score was applied to examine the time trend for the attaining of the target 90% completion rate. This technique described a learning curve which showed, in this instance, that approximately 100 studies were necessary before a 90% completion rate was approached. A further 100 studies were required before this target completion rate was achieved consistently. A trend for continued improvement above the 90% completion rate level was also seen after 200 studies. Polypectomy was completed in 117 of the 123 studies where indicated. Cusum analysis may be a useful method for monitoring the progress of the trainee colonoscopist and the attainment of satisfactory competence. Training requirements, therefore, might better emphasize the attainment of an acceptable completion rate rather than an arbitrary quota per se. Key words: clinical competence, donoscopy, learning.
Introduction
For the trainee colonoscopist, Hippocrates' oftquoted aphorism, 'the life so short; the technique so long to learn', is most appropriate. Long, too, is the colon - even experienced endoscopists remark wryly on the length and tortuosity of the human colon.' Indeed, it is these features of the colonic anatomy that demand of the colonoscopist considerable dexterity in the control of the instrument as well as mastery of manoeuvres to reach the caecum. It is axiomatic, in the interests of diagnostic accuracy and therapeutic opportunity, that total colonoscopy be regarded as standard practice with the proviso that any therapeutic procedures found necessary, such as polypectomy. be carried out successfully as well. The completion rate for total colonoscopy is variously reported as between 55 and 83%,*-* although 90% is regarded as normative.' It is generally accepted that training in colonic endoscopy is necessary and protocols have been established accordingly. "-" The duration of such training is. however, arbitrary and although the term 'learning curve' is common parlance, few if any data are available to support this intuititive assumption and thereby permit the setting of rational guidelines. In industry, concern for quality control has given rise to various mathematical and statistical techCorrespandence: Bryan R. Parry. Senior Lecturer. Department of Surgery. University of Otago, Dunedin. New Zealand. Accepted for publication 19 December 1990.
niques to monitor performance of humans and machines. l3-I5 One such technique is the cumulative sum score (cusum) method which, from its name, generates a cumulative sum of pre-defined scores (which depend on outcome) from consecutive events. This sequence of arbitrary scores describes a curve, the slope of which enables trends in performance to be analysed. Using this technique, the unassisted, but supervised, early experience of one trainee colonoscopist with considerable previous upper gastrointestinal endoscopy experience was audited and analysed to determine the nature of the learning curve and the number of procedures required to achieve a nominal completion rate of 90%. Methods
From August 1987 to October 1990, the detailed, standardized records and dictated reports of consecutive total colonoscopy studies carried out by one of the authors (BRP) were analysed. This period extends from the commencement of the trainee's colonoscopic experience up to the current level of experience. The indications, findings and completeness or otherwise of the study were recorded. Completion of the study was defined as either reaching the caecum (or ileo-colic anastomosis where appropriate) or encountering an impassable obstructive lesion. Performance was assessed using the cusum score technique because of its usefulness as a device for showing time trends. I3-I5 In general. the cusum (Si) for a series of observations Xi . . . X, is defined as si = ' Zj= I(X" - Xj)
PARRY AND WILLIAMS
420
where & is a reference or target value representing some sort of expected value for the data.'' For this study the target value of 0.9 was chosen (representing a completion rate of 90%). X,was 1 if the study was completed and 0 otherwise. Si was plotted against i.
Results Of the 334 consecutive colonoscopy studies, 29 studies (8.7%) were excluded from the analysis because of inadequate bowel preparation leading to abandonment of the examination. Overall, complete studies were possible in 250 (82%) of the remaining 305 evaluable examinations, and polypectomy was carried out in 117 of 123 patients on whom it was indicated (95%). Details of the indications for colonoscopy and the primary findings are set out in Tables I and 2 respectively. The relatively low number of patients with inflammatory bowel disease studied was due to selection bias reflecting the special interests of other endoscopists in the Gastroenterological Unit. Table 3 sets out details of the incomplete studies by consecutive sets of 100 studies.
The cusum plot for the target value of 90% is shown in Fig. I . There is a steep positive slope for the first 100 studies, indicating a poor completion rate in relation to the 90% rate target. Between the 100th and 200th studies, however, the slope tends towards horizontal, indicating a completion rate approaching the target of 90%. After the 200th study, the slope becomes slightly negative, suggesting that the target completion rate of 90% was being exceeded. Table 2. Primary findings
No. patients
Finding Polyps No abnormality Inflammatory bowel disease Diverticular disease Carcinoma Angiodysplasia Other
I23 I I9 25
Total
305
21 14
I 2
Table 1. Indications for colonoscopy Indication
No. patients
Cancer surveillance Previous colorectal cancer Previous colorectal polyps Inflammatory bowel disease Family history colorectal cancer Polyps Bowel symptoms Bleeding Change of bowel habit Diarrhoea Barium enema abnormality Inflammatory bowel disease mapping Anaemia Other Total
81 50
7 3 56
33 9 6 25 17 12 6 305
0
100
200
300
Colonoicopy study no.
Fig. 1. Cusurn for 305 consecutive colonoscopy studies for completion rate target value of 90%. Performance in relation to the target value is indicated by the slope (positive slope: below target. zero slope: achieving and maintaining target. negative slope: exceeding target).
Table 3. Incomplete colonoscopy studies data Study no. 001-100
No.cases No. completed ( % ) Reason incomplete Sigmoid looping R. colon looping Instrument failure Patient distress Not recorded No. studies excluded for poor bowel preparation ( O h )
100
67 (67.0)
101-200
20 I -305
100 88 (88.0)
I05 95 (90.5)
15 4
4 7
I I 12
I
I I (9.9)
I
-
12 (10.7)
7 -
I I 6 (5.4)
42 I
COLONOSCOPY LEARNING CURVE
Discussion The cusum plot indicates that the completion rate increased with experience, thereby confirming a learning curve phenomenon in this instance. A completion rate of 90% was reached and maintained after approximately 200 studies, and exceeded subsequently. As it is almost certain that a wide variation exists among trainees in the rapidity or otherwise of their acquiring an acceptable completion rate, it may be that this trainee's quantum of studies represents an under- or over-estimation of the ideal. In this instance, the quota of study numbers usually recommended by various authorities to attain competence may have proved optimistic. Stratifying the studies into consecutive centuries (Table 3) proved useful in revealing the changing pattern for incomplete studies. Such a retrospective analysis, despite being useful from an audit per-spective, cannot depict an instananeous trend to which the operator might usefully react. Ongoing cusum analysis can, by contrast, point up failure to improve in the learning period, and subsequent deterioration from an attained standard. Although polypectomy presented no undue recognized difficulties, not all (5%) of intended polypectomies were successful. It is conceded too that polyps may have been missed, especially at the beginning of the trainee's experience. Nevertheless, the distinction between diagnostic and therapeutic colonoscopies for training purposes" is artificial and opportunity to acquire experience of polypectomy need not be deferred. The method of assessing competence used was completed, or total, colonoscopy. It might be argued that other methods were appropriate. Speed, as in surgery, is useful but is not necessarily associated with proficiency per se. Diagnostic sensitivity and specificity are valid criteria for comparing colonoscopy with other investigations such as double contrast barium enema, but are not suitable when colonoscopy alone is assessed, because of the lack of a reference standard. Complication rates are low in colonoscopy," so that the monitoring of complications alone would be unlikely to identify the incompetent trainee in other than the most extreme case. The overall high rate (8.7%) of unsatisfactory bowel preparation was of concern and this Unit turned increasingly during the course of this study to a colonic lavage electrolyte solution as standard preparation, particularly in the elderly. It is of some consolation, therefore, that the rate dropped to 5.4% (Table 3) for the last 105 studies. It is possible that 100-200 studies may be necessary to attain a 90% completion rate in colonoscopy. Because of individual differences in aptitude, however, training guidelines should be linked to the attainment and maintenance of an agreed comple-
tion rate of colonoscopy rather than fulfilling an arbitrary quota of studies per se. This highlights the need for a properly constructed supervised audit system for all trainee colonoscopists. The cusum analysis outlined here is suggested as an important adjunct in the monitoring of such training. Concern has been expressed about colonoscopy training and there have been calls for a concerted effort to equip and train more endoscopists, especially surgeon endoscopists. 1 1 * ' 7 ~ 1 8These must be matched by an equal concern to provide clear standards of competency, such as an acceptable colonoscopy completion rate, and the establishment of an audit mechanism to monitor trainees' performance - something already accepted as de rigueur for operative surgical training in theory, if not in practice.
Achnowledgements We thank our colleagues in the Gastroenterology Department, Prof. G. Barbezat, Drs M.Schlup, S. Parry and G.Langstreth, for their help and encouragement and Prof. A. M. Van Rij for his helpful critical advice.
References 1. COTTON P. B. & WILLIAMS C. B. (1982) Practical GastrointestinalEndoscopy, 2nd edn. Blackwell Scientific Publications. Oxford. 2. ALDRlDGE M.C. & SIM A. J. w. (1986) Colonoscopy findings in symptomatic patients without X-ray evidence of colonic neoplasms. Lancer ii,833-4. 3. ABERGT., LINGL., BRELAND U. & NORLUND A. (1985) Does flexible sigmoidoscopy have any justification as compared with complete colonoscopy? An analysis of cost-effectiveness and medical risk. Endoscopy 13, 133-6. 4. ABRAHMS J. S. (1982) A second look at colonoscopy: indications, failures, and costs. Arch. Surg. 117, 913-7. 5 . BATESON M. C. & BOUCHIER 1. A. (1988) Clinical Investigations in Gastroenterology. Kluwer Academic Publishers, Dordrecht. N. S. 6. DURDLEY P., WESTON P. M. T. & WILLIAMS (1987) Colonoscopy or Barium enema as initial investigation of colonic disease. Lancet ii, 549-5 I. 7. ISBISTERW. H. (1987) Colonoscopy: a ten year Wellington experience. N.Z.Med. J . 100, 74-7. D. W. & 017. 8. OBRECHT W. R., Wu W. C., GELFAND D. J. (1984) The extent of successful colonoscopy: a second assessment using modern equipment. Gastrointest. Radiol. 9, 161-2. 9. WILLIAMS C. B. (1987) Colonoscopy. Curr. Opin. Gastroenterol. 3, 36-42. 10. BENSON J. A. & COHENS. (1987) Evaluation of procedural skills in gastroenterologists. Gastroenterology 92, 254. 1 I. BRITISH SWlETY OF GASTROENTEROLOGY (1987) Future requirements for colonoscopy in Britain. Report of the Endoscopy Section Committee of the British Society of Gastroenterology. Gut 28, 772-5.
422 12. CONJOINT COMMITTEE ON ENCCISCOPY (1989) Endoscopy Training. Report of the Conjoint Committee on Endoscopy of the Gastroenterological Society of Australia, the Royal Australasian College of Physicians and the Royal Australasian College of Surgeons. RACS Bulletin 7. 19-20, 13. EWEN W. D. & KEMPK. W. (1960) Sampling inspection of continuous processes with no autocorrelation between successive results. Biornetriku 47, 363-80. 14. PAWE. S. (1954) Continuous inspection schemes. Biornetrika 41, 100- 15. 15. ALTMAND. G. & ROYSTONI . P. (1988) The hidden
PARRY AND WILLIAMS
effect of time. Srurisrirs in Medicine 7, 629-37. 16. REIERTSEN 0.. SKJ0T0 J., JACOBSENC. D. & ROSSELAND A. R. (1987) Complicationsof fibreoptic gastrointestinal endoscopy: five years experience in a central hospital. Endosropy 19, 1-6. 17. ISBISTER W.H. (1986) Quality control in endoscopy: Workshop report and personal comments. Aust. N.Z. J. Surg. 56. 205-8. 18. FUMIO NAKAYAMA (1988) Editorial. Impact of recent advances in imaging techniques on surgical practice and its implications. Ausr. N.Z.J. Surg. 58. 87-8.
