523412 research-article2014
SJS0010.1177/1457496914523412J. Rystedt, A. Montgomery, G. PerssonScandinavian Journal of Surgery X(X)
Original Article
Scandinavian Journal of Surgery 103: 237–244, 2014
Completeness and Correctness of Cholecystectomy Data in a National Register—Gallriks J. Rystedt1, A. Montgomery1, G. Persson2 1 Department 2
of Surgery, Skåne University Hospital, Lund and Malmö, Sweden Department of Surgery, Ryhov Hospital, Jönköping, Sweden
Abstract
Background and Aims: To validate the Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography (GallRiks) concerning completeness and correctness of entered data for cholecystectomies and evaluating the effect of repeated audits. It is crucial for any register to obtain a high accuracy in order to be a credible and reliable source for quality evaluation, research, and development. Materials and Methods: Completeness was determined by cross-matching the register with the Swedish National Patient Register. Completeness and overall correctness were assessed by comparing registered data to medical records. Correctness for rare occasions, such as bile duct injuries, was evaluated by comparing with claims sent to the Swedish Patient Insurance. Results: Of 64,538 cholecystectomies, 82.9% of the operations were registered in GallRiks and the coverage increased over time. By random sample, 94,919 sets of data were available for comparison at the first and second audit and the entries were found to be correct in 97.2% and 98.2% cases, respectively. A 100% correctness for bile duct injuries (n = 40) was seen when the index-operation was registered in GallRiks. Conclusion: GallRiks demonstrates high completeness, high correctness of entered data, and no indications of failure to report serious adverse events. Repeated audits increased the quality of registered data. GallRiks may be used for clinical evaluation on local and national level and the database enables scientific studies to be performed. Key words: Population-based register; validation; cholecystectomy; bile duct injury
Correspondence: Jenny Rystedt MD Lund University Department of Surgery Skåne University Hospital SE-221 85 Lund Sweden Email: [email protected]
238
J. Rystedt, et al.
Introduction Quality assessment and monitoring in healthcare is becoming increasingly important, and in Sweden, more than 70 national quality registries have been established during the last decades (1). Swedish quality registers receive public funding from the National Board of Health and Welfare (NBHW; Swedish: Socialstyrelsen) and are the main source of information for the report on the quality of healthcare in Sweden: Quality and Efficiency in Swedish Health Care—“Open Comparisons,” which has been published annually since 2006 (2). GallRiks—The Swedish Register of Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography (ERCP)—was launched in May 2005. Both open and laparoscopic cholecystectomies as well as all endoscopic interventions of the bile ducts are included. The register aims to achieve complete national coverage for the treatment of gallstones and its complications. It is approved by the Swedish Surgical Society, who also appoints the members of the GallRiks Board. As most of the Swedish registries for quality assurance, GallRiks was developed by the profession for the purpose of quality assurance as well as for patient safety and research. No goals or objectives have been defined by the NBHW or other healthcare authorities. Incomplete registration of adverse events in voluntary registers has detrimental effects on the credibility of databases and registers. This also makes adequate research impossible and might mislead authorities in healthcare, politicians, and policy makers in decisions as well as in giving invalid information to the public (3, 4). The aim of this study was to present a validation process, pursued over time, with regard to completeness, overall correctness, and correctness of rare adverse events for GallRiks concerning cholecystectomies. Material and Methods Gallriks
The perioperative data are registered into a secure website by the operating surgeon. The postoperative data (30 days and 6 months) are collected from the medical records or at a telephone interview by a specially trained nurse. A complete list of variables is available on the Web page of the register (5). Participating surgeons can by individual login get instant access to Web-based reports on personal as well as hospital levels, with nation-based results as reference. National and regional data are analyzed and the results are e-published once a year (6). The Swedish National Patient Register
The National Patient Register (NPR; Swedish: Patientregistret) is run by the NBHW and contains administrative and medical data such as diagnose and procedure codes on all inpatient care and outpatient visits including day surgery. It is mandatory for all healthcare providers to register in NPR and the
register has been validated (7, 8). The NPR reached complete coverage (100% of Swedish hospitals) for inpatient care in 1987 and reports an overall positive predictive value of diagnoses of about 85%–95%. From 2001 onward, the register also includes outpatient visits and day surgery from both private and public caregivers. NPR is the gold standard for validation of patient registers in Sweden in terms of coverage and completeness of events. The Patient Insurance
In Sweden, one common national Patient Insurance (LÖF; Swedish: Landstingens Ömsesidiga Försäkringsbolag) deals with all economical claims concerning medical care. Litigation in a court of law is only used when crime is suspected. Bile duct injuries and other severe adverse events during cholecystectomy are generally entitled to a financial compensation according to the Patient Injury Act. All claims are stored in a database. Completeness of Events—National, Regional, and Local Coverage
The unique personal identification number assigned to all Swedish residents enables databases to be crossmatched patient by patient. At the NBHW the quality registers can be cross-matched to the NPR annually for estimation of the coverage on national, regional, and local levels. To estimate the register agreement of GallRiks for cholecystectomies, that is, the number of registrations divided by the number of operations performed, a cross-match with the NPR on the procedure codes for open and laparoscopic cholecystectomy (JKA20, JKA21, and JKA96) was performed for the years 2007 to 2011. Cholecystectomies performed as part of surgery for malignancy were excluded (ICD codes C22 and C25). Analyses were made on national, regional, and local level. The total number of cholecystectomies performed in Sweden is defined as the total number of patients registered in NPR and/or in GallRiks. Overall Correctness of Data—Validation by Repeated Audits
Hospitals taking part in GallRiks are being audited, on-site, every 3 years, by a specially appointed senior surgeon, familiar with the functionality and definitions used in the register. In addition to validating data in the register, the visit intends to give support to the local coordinator and surgeons, gather information on difficulties encountered in managing entries, and ensure that adequate resources are allocated to the register work. To evaluate the correctness of data a random sample of 25 patients per audit was retrieved from the GallRiks database and compared to the corresponding information in the medical records, being regarded as the true values. The sample included both cholecystectomies and ERCPs, in proportion to the number of each procedure performed at the specific hospital. The medical records were scrutinized for information
Population-based national register
239
Statistical Analysis
2011 n=13290 2010 n=13199
81%
4%
2009 n=13360
80%
8%
2008 n=12386
79%
2007 n=12303
3% 3%
70% In GallRiks and in NPR
Only in GallRiks
14%
4%
83%
15% 12% 18%
27%
Only in NPR
Fig. 1. The number of cholecystectomies registered both in GallRiks and in NPR, only in GallRiks, and only in NPR is shown as percentages for each year 2007–2011. GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography; NPR: National Patient Register.
on 43 predefined variables including preoperative, operative, and 30 days follow-up data. An entry was made in an Excel chart for each of the variables using three different alternatives: correct, incorrect, and inconclusive. The latter being used when the correctness could not be determined or the corresponding data were missing in the medical record. In this study, only cholecystectomy data were analyzed. The audits were conducted 2007 through 2012 reviewing data for 2007–2011. Correctness of Rare Adverse Events
All claims concerning complications of cholecystectomy sent to LÖF during the period 1 June 2005 to 31 May 2010 were retrieved and the identification number was used to search for the corresponding registrations in GallRiks. A claim is regularly accompanied by copies of the medical records where detailed information about the incident is available. Complications were grouped in major and minor bile duct injuries, early surgical and late and/or nonsurgical. In GallRiks, a major bile duct injury is defined as damage to more than one-third (>1/3) of the circumference of the bile duct, equaling Strasberg grade B-E or Hanover grade C2-D (9, 10). Early procedurerelated surgical complications are defined and have to be either negated or confirmed in the follow-up chart. Late or nonsurgical complications like incisional hernia, cardiovascular incidents, skin problems, injuries from inadequate patient positioning at operation and dental injuries from intubation are examples of complications that may be reason for medical claim but are not predefined in the register chart. For patients identified in LÖF claims, medical records and data in GallRiks were compared and discrepancies noted. The minimum follow-up time was 6 months. Ethical Approval
Ethical approval for this study was obtained from the Regional Ethical Committee of Linköping, Sweden.
For comparing the proportions of a complete match on individual level between the first and second audit, Mann–Whitney nonparametric test was used. Proportions for correctness and inconclusiveness were compared using the Chi-square test. Due to multiple comparisons, the significance level was set at 0.01. A total sum of >10 registrations including first and second audit for incorrect and inconclusive registrations was required to be subjected for statistical evaluation. Results Completeness of Events—National, Regional, and Local Coverage
In total, 64,538 (100%) cholecystectomies were performed during the study period, 2007–2011. A total of 53,504 (82.9%) of the operations were registered in GallRiks and 61,687 (95.6%) in NPR. Overall, 50,653 (78.5%) of the patients were found in both registers (see Fig. 1). As the inclusion of hospitals progressed, the national coverage gradually increased and after 2009 was above 85%. Missed registrations in NPR that were identified in GallRiks varied between 3% and 8% and missed registrations in GallRiks that were identified in NPR varied between 12% and 27%. In 2011, the coverage in the 21 different counties ranged between 55% and 98%, an example of regional differences. In eight counties, the coverage was ≥90% in total including 6907 (52%) of the 13,290 cholecystectomies performed. At the hospital level, the corresponding figures for 2011 were median 91%, ranging between 0% (representing two hospitals not enrolled) and 100% (three hospitals). A total of 37 hospitals out of 70 reported more than 90% of their cholecystectomies, together including 8200 (62%) out of the 13,290 cholecystectomies performed. Overall Correctness of Data—Validation by Repeated Audits
As of December 2011, a total number of 63,851 cholecystectomies were reported to GallRiks from 76 different hospitals. Seven hospitals recently either joined the register (four) or stopped performing cholecystectomies (three) and were accordingly excluded. Thus, 69 hospitals were eligible and audited twice during the study period. At the first audit, medical records for 1257 cholecystectomies and at the second visit 1252 records were scrutinized. A complete match between GallRiks and the medical record was found in 780 (62%) of the patients during the first audit, but increased to 914 (73%) during the second (p < 0.001) (Table 1). Altogether 94,919 sets of data were available for comparison at the first and second audit. The entries were found to be correct in 46,213(97.2%) occasions at the first audit and in 46,522 (98.2%) at the second. In 782 (1.7%) and 543 (1.2%) occasions, respectively, the entries were incorrect. The amount of data where the correctness could not be determined or the corresponding data were missing in the medical record were reduced from 599 (1.2%) to 291
240
J. Rystedt, et al.
Table 1 The correctness of the registrations during the first audit compared to the second with regard to the number of records with a complete match, that is, all 43 variables correct, 1, 2, or >2 errors respectively (p < 0.001). First audit
Second audit
n
%
n
%
Complete match 1 error 2 errors > 2 errors Number of patients
780 306 101 70 1257
62 24 8 6 100
914 213 87 38 1252
73 17 7 3 100
Table 2 The number of cholecystectomies with a claim to LÖF (n = 253) registered and not registered in GallRiks in different groups of complications (p = ns). (LÖF: Swedish Patient Insurance; GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography). Registered
Not registered
Claim to LÖF (n = 253)
Number
%
Number
%
Major bile duct (n = 15) Minor bile duct (n = 30) Early surgical (n = 129) Late or none surgical (n = 79)
14 26 114 73 227
93 87 88 92 90
1 4 15 6 26
7 13 12 8 10
LÖF: Swedish Patient Insurance. 354 claims to LÖF
82 from hospitals not par cipa ng in GallRiks
Major bile duct (n=14)
14
0
Minor bile duct (n=26)
26
0
Early surgical (n=114)
103
Late or non surgical (n=73) 272 from hospitals par cipa ng in GallRiks
19 claims without complica on
253 complica ons from hospitals par cipa ng in GallRiks
26 complica ons opera on not registered in GallRiks
24
0% Opera on registered
11 49
50%
100%
Complica on missing
Fig. 3. The number of cholecystectomies with a claim to LÖF registered in GallRiks (n = 227) with and without a correct registration of the complication (p < 0.001). LÖF: Swedish Patient Insurance; GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography.
and 6.9% respectively. Registration of “biliary leakage” showed good agreement, 99.2% and 99.1% (Appendix). Correctness of Rare Adverse Events
227 with complica ons and registered in GallRiks
Fig. 2. Claims received by LÖF. GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography; LÖF: Swedish Patient Insurance.
(0.6%) between audits. Of the possible 43 variables controlled for each patient, operating time had the highest proportion of incorrect registrations in the first audit; 10.4% of entries were noted incorrect, which was reduced to less than half in the second audit. In the complication registration, “cause of biliary leakage” had the least agreement with the medical records in the first and second audits, 6.8%
During the study period, 354 claims concerning injuries related to cholecystectomy were received by LÖF (Fig. 2). In 272 cases, the operation was performed at a hospital participating in GallRiks at the time of the injury. All these patients were assessed in detail. In 19 cases, the claim was not caused by a complication and was excluded, resulting in 253 patients remaining for evaluation. Of the 253 claims, a total of 227 (90%) were registered in GallRiks. The 26 claims where the cholecystectomy was not registered were evenly distributed between the different groups of complications (Table 2). For the 227 cases registered in GallRiks, the claims, medical records, and data in the register were compared and discrepancies noted. Of 114 early surgical complications, 103 (90%) were registered, whereas nonsurgical/late complications were recorded in 33% of the cases (Fig. 3). When the cholecystectomy was registered in GallRiks, all bile duct injuries, major (n = 14) and minor (n = 26), were reported correctly.
Population-based national register
Discussion This study estimates the validity of GallRiks database with regard to completeness and correctness of the registered data. The overall coverage is 83% and above 85% in later years, all in line with increasing inclusion of hospitals in the register. The study demonstrates that the overall correctness in GallRiks is high and increasing over time, possibly attributed to the periodical audits. The high proportion of incorrect registrations (7%) for the complication “cause of biliary leakage” is probably due to the fact that it is often difficult to decide what the most likely cause is. Since the correctness for “biliary leakage” itself was above 99% at both audits, we estimate the risk of bile duct injuries not being noted in the register to be minor. There are no signs of systematical bias regarding registration of operations resulting in procedure-related early surgical complications or serious adverse events such as bile duct injuries. Regarding nonsurgical or late complications such as incisional hernias the register is not reliable since the follow-up is not designed to include this information. Although the healthcare in Sweden is publically funded, it is not compulsory to register in databases like GallRiks. But it is expected that all units performing cholecystectomies on a regular basis will participate since the results are published annually, on national and regional level, in “Open Comparisons,” information available to the public (2). In Denmark, on the other hand, the Danish Cholecystectomy Database (DCD) started in 2006 and was made mandatory. It had specified objectives, which were to increase the proportion of cholecystectomies completed laparoscopically, to decrease the postoperative length of stay, and to reduce the rate of surgical complications (11). This study also enlightens the fact that no register is flawless. The NPR is regarded the most complete patient register in Sweden and is often used as the “gold standard” in epidemiological studies. However, 3% up to 8% of the cholecystectomies were missing in the NPR during the study period. Neither are the medical records complete nor totally accurate. Wauben et al. (12) reported that the operative notes of laparoscopic cholecystectomy lacked information on several important procedural steps compared to video recordings. Incomplete medical records will obviously limit the possibility to make an adequate validation of registers. In our study, information was lacking in the medical records in some, but few, cases. Cholecystectomy is one of the most common surgical procedures in Sweden and iatrogenic bile duct injuries a much feared complication. The incidence of cholecystectomies in Sweden has been estimated to be approximately 122 per 100,000 inhabitants. Thus, among the 9.5 million Swedish inhabitants, approximately 11,600 cholecystectomies were expected to be performed annually (13). However, this study shows that the annual number of cholecystectomies in Sweden is 13,200, that is, 139 per 100,000 inhabitants. A bile duct injury can be fatal or render impaired quality of life for the patient (14–16). It represents a
241
considerable cost for society and is a natural focus for patient safety work (17, 18). A higher incidence of bile duct injuries has been suggested after the introduction of laparoscopic surgery (19, 20). A Swedish epidemiological study from 2006 on 150,000 patients demonstrated only a small to moderate increase, from 0.40% to 0.47%, in the incidence of bile duct injuries after the introduction of laparoscopy (21). In the early laparoscopic era, there was a register for laparoscopic cholecystectomy in Sweden (LR), no longer active, which included over 11,000 procedures. In the audit in 1997, comparing LR to LÖF, it was found that in 11 out of 24 cases the bile duct injury was not noted in LR (22). In this study, we noted a 100% registration of bile duct injuries in GallRiks compared to LÖF. When studying important but rare complications such as bile duct injuries, two different types of bias might be considered. One is the neglect to enter an operation with a certain type or severity of complication in the register. Another potential risk is an incorrect registration of the operation as being uncomplicated when a complication did occur. To study the correctness of entries of rare events like bile duct injuries, the sample size has to be very large, and the resources required for such a study will be out of proportion. An alternative approach, used in this investigation, is to study patients suffering from the complication and assess whether the operation and the associated complication actually were registered. In such an analysis, another source for identifying these patients is needed. We used a national patient insurance database (LÖF), and neither neglect to register the cholecystectomy nor incorrect registration of the complication was found in GallRiks. This is contrary to what was found in a previous study of a Swedish register of laparoscopic cholecystectomy (22). The national quality registries enable studies to be performed on large unselected populations. This may be a feasible alternative to address scientific issues difficult to study in prospective randomized trials. This unique possibility has to be cared for by keeping data input valid and as complete as possible. In voluntary registers, compliance is essential and is enhanced by regular feedback of results and by a thorough validation process. This is met by GallRiks annual reports, individual online feedback, and by audit by independent investigators. Auditing on-site may promote local hospital management to allocate adequate resources to register work as well as increase thoroughness in the local work of entering data. Furthermore, at the on-site visits, suggestions for improvement of the register have been gathered and implemented in the register. Fine et al. (23) showed that it is possible to improve the quality of a database by validating, monitoring, and giving feedback. In this study, the validation process of a large population-based register on cholecystectomy is described and its strengths and limitations pointed out for the use in clinical development and for research.
Overall (43 variables) Preoperative (9) Operation (12) Perioperative complications (4) 30-day follow-up (18) Preoperative Date of surgery Inpatient or day care (intention) Date of admission Acute or elective surgery Indication for surgery Cholecystitis (ongoing) Pancreatitis (ongoing) Cholecystitis (past) Pancreatitis (past) Operation Surgical method Mode of gallbladder mobilization Antibiotics Surgeon Cholangiography Method of treatment of duct stones Method of cystic duct closure Method of cystic artery closure Perioperative diagnosis Gallbladder sent for PAD Perforation of gallbladder (accidental) Operating time Perioperative complications Perioperative complication Perforated bowel
47356 10235 13727 4997 18397 1249 1238 247 1248 1251 1251 1251 1250 1250 1245 1150 1246 1247 1243 149 1246 1244 1242 1243 1234 1238 1247 1251
47554 10445 13750 4982 18377
1251 1247 430 1252 1254 1252 1253 1254 1252
1253 1114 1251 1252 1247 153 1246 1250 1250 1251 1242
1241
1248 1230
2nd audit
1233 1228
1063
1245 1078 1195 1242 1206 147 1136 1164 1180 1224 1149
1237 1221 409 1240 1188 1212 1243 1209 1229
46213 10188 13029 4956 18040
8 1
129
7 30 31 8 32 6 48 23 19 9 56
13 24 21 10 51 34 5 32 17
782 207 398 17 160
0.6 0.1
10.8
0.6 2.7 2.5 0.6 2.6 3.9 4.1 1.9 1.6 0.7 4.6
1.0 1.9 4.9 0.8 4.1 2.7 0.4 2.6 1.4
1.7 2.0 3.0 0.3 0.9
1234 1250
1073
1242 1135 1213 1232 1219 139 1224 1187 1204 1222 1201
1243 1215 240 1240 1188 1231 1249 1202 1223
46522 10031 13291 4974 18226
10 1
50
2 11 24 11 16 8 7 24 16 10 28
6 21 6 7 52 16 2 45 24
543 179 207 17 140
0.8 0.1
4.5
0.2 1.0 1.9 0.9 1.3 5.4 0.6 2.0 1.3 0.8 2.3
0.5 1.7 2.4 0.6 4.2 1.3 0.2 3.6 1.9
1.2 1.8 1.5 0.3 0.8
Correct Incorrect Incorrect Correct Incorrect Incorrect (n) (n) (%) (n) (n) (%)
1st audit Total 1st Total 2nd audit audit (n) (n)
Correctness
0.107 0.670 0.118 0.469 0.923 0.010 0.256 0.138 0.273
0.640 0.990
SJS0010.1177/1457496914523412J. Rystedt, A. Montgomery, G. PerssonScandinavian Journal of Surgery X(X)
Original Article
Scandinavian Journal of Surgery 103: 237–244, 2014
Completeness and Correctness of Cholecystectomy Data in a National Register—Gallriks J. Rystedt1, A. Montgomery1, G. Persson2 1 Department 2
of Surgery, Skåne University Hospital, Lund and Malmö, Sweden Department of Surgery, Ryhov Hospital, Jönköping, Sweden
Abstract
Background and Aims: To validate the Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography (GallRiks) concerning completeness and correctness of entered data for cholecystectomies and evaluating the effect of repeated audits. It is crucial for any register to obtain a high accuracy in order to be a credible and reliable source for quality evaluation, research, and development. Materials and Methods: Completeness was determined by cross-matching the register with the Swedish National Patient Register. Completeness and overall correctness were assessed by comparing registered data to medical records. Correctness for rare occasions, such as bile duct injuries, was evaluated by comparing with claims sent to the Swedish Patient Insurance. Results: Of 64,538 cholecystectomies, 82.9% of the operations were registered in GallRiks and the coverage increased over time. By random sample, 94,919 sets of data were available for comparison at the first and second audit and the entries were found to be correct in 97.2% and 98.2% cases, respectively. A 100% correctness for bile duct injuries (n = 40) was seen when the index-operation was registered in GallRiks. Conclusion: GallRiks demonstrates high completeness, high correctness of entered data, and no indications of failure to report serious adverse events. Repeated audits increased the quality of registered data. GallRiks may be used for clinical evaluation on local and national level and the database enables scientific studies to be performed. Key words: Population-based register; validation; cholecystectomy; bile duct injury
Correspondence: Jenny Rystedt MD Lund University Department of Surgery Skåne University Hospital SE-221 85 Lund Sweden Email: [email protected]
238
J. Rystedt, et al.
Introduction Quality assessment and monitoring in healthcare is becoming increasingly important, and in Sweden, more than 70 national quality registries have been established during the last decades (1). Swedish quality registers receive public funding from the National Board of Health and Welfare (NBHW; Swedish: Socialstyrelsen) and are the main source of information for the report on the quality of healthcare in Sweden: Quality and Efficiency in Swedish Health Care—“Open Comparisons,” which has been published annually since 2006 (2). GallRiks—The Swedish Register of Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography (ERCP)—was launched in May 2005. Both open and laparoscopic cholecystectomies as well as all endoscopic interventions of the bile ducts are included. The register aims to achieve complete national coverage for the treatment of gallstones and its complications. It is approved by the Swedish Surgical Society, who also appoints the members of the GallRiks Board. As most of the Swedish registries for quality assurance, GallRiks was developed by the profession for the purpose of quality assurance as well as for patient safety and research. No goals or objectives have been defined by the NBHW or other healthcare authorities. Incomplete registration of adverse events in voluntary registers has detrimental effects on the credibility of databases and registers. This also makes adequate research impossible and might mislead authorities in healthcare, politicians, and policy makers in decisions as well as in giving invalid information to the public (3, 4). The aim of this study was to present a validation process, pursued over time, with regard to completeness, overall correctness, and correctness of rare adverse events for GallRiks concerning cholecystectomies. Material and Methods Gallriks
The perioperative data are registered into a secure website by the operating surgeon. The postoperative data (30 days and 6 months) are collected from the medical records or at a telephone interview by a specially trained nurse. A complete list of variables is available on the Web page of the register (5). Participating surgeons can by individual login get instant access to Web-based reports on personal as well as hospital levels, with nation-based results as reference. National and regional data are analyzed and the results are e-published once a year (6). The Swedish National Patient Register
The National Patient Register (NPR; Swedish: Patientregistret) is run by the NBHW and contains administrative and medical data such as diagnose and procedure codes on all inpatient care and outpatient visits including day surgery. It is mandatory for all healthcare providers to register in NPR and the
register has been validated (7, 8). The NPR reached complete coverage (100% of Swedish hospitals) for inpatient care in 1987 and reports an overall positive predictive value of diagnoses of about 85%–95%. From 2001 onward, the register also includes outpatient visits and day surgery from both private and public caregivers. NPR is the gold standard for validation of patient registers in Sweden in terms of coverage and completeness of events. The Patient Insurance
In Sweden, one common national Patient Insurance (LÖF; Swedish: Landstingens Ömsesidiga Försäkringsbolag) deals with all economical claims concerning medical care. Litigation in a court of law is only used when crime is suspected. Bile duct injuries and other severe adverse events during cholecystectomy are generally entitled to a financial compensation according to the Patient Injury Act. All claims are stored in a database. Completeness of Events—National, Regional, and Local Coverage
The unique personal identification number assigned to all Swedish residents enables databases to be crossmatched patient by patient. At the NBHW the quality registers can be cross-matched to the NPR annually for estimation of the coverage on national, regional, and local levels. To estimate the register agreement of GallRiks for cholecystectomies, that is, the number of registrations divided by the number of operations performed, a cross-match with the NPR on the procedure codes for open and laparoscopic cholecystectomy (JKA20, JKA21, and JKA96) was performed for the years 2007 to 2011. Cholecystectomies performed as part of surgery for malignancy were excluded (ICD codes C22 and C25). Analyses were made on national, regional, and local level. The total number of cholecystectomies performed in Sweden is defined as the total number of patients registered in NPR and/or in GallRiks. Overall Correctness of Data—Validation by Repeated Audits
Hospitals taking part in GallRiks are being audited, on-site, every 3 years, by a specially appointed senior surgeon, familiar with the functionality and definitions used in the register. In addition to validating data in the register, the visit intends to give support to the local coordinator and surgeons, gather information on difficulties encountered in managing entries, and ensure that adequate resources are allocated to the register work. To evaluate the correctness of data a random sample of 25 patients per audit was retrieved from the GallRiks database and compared to the corresponding information in the medical records, being regarded as the true values. The sample included both cholecystectomies and ERCPs, in proportion to the number of each procedure performed at the specific hospital. The medical records were scrutinized for information
Population-based national register
239
Statistical Analysis
2011 n=13290 2010 n=13199
81%
4%
2009 n=13360
80%
8%
2008 n=12386
79%
2007 n=12303
3% 3%
70% In GallRiks and in NPR
Only in GallRiks
14%
4%
83%
15% 12% 18%
27%
Only in NPR
Fig. 1. The number of cholecystectomies registered both in GallRiks and in NPR, only in GallRiks, and only in NPR is shown as percentages for each year 2007–2011. GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography; NPR: National Patient Register.
on 43 predefined variables including preoperative, operative, and 30 days follow-up data. An entry was made in an Excel chart for each of the variables using three different alternatives: correct, incorrect, and inconclusive. The latter being used when the correctness could not be determined or the corresponding data were missing in the medical record. In this study, only cholecystectomy data were analyzed. The audits were conducted 2007 through 2012 reviewing data for 2007–2011. Correctness of Rare Adverse Events
All claims concerning complications of cholecystectomy sent to LÖF during the period 1 June 2005 to 31 May 2010 were retrieved and the identification number was used to search for the corresponding registrations in GallRiks. A claim is regularly accompanied by copies of the medical records where detailed information about the incident is available. Complications were grouped in major and minor bile duct injuries, early surgical and late and/or nonsurgical. In GallRiks, a major bile duct injury is defined as damage to more than one-third (>1/3) of the circumference of the bile duct, equaling Strasberg grade B-E or Hanover grade C2-D (9, 10). Early procedurerelated surgical complications are defined and have to be either negated or confirmed in the follow-up chart. Late or nonsurgical complications like incisional hernia, cardiovascular incidents, skin problems, injuries from inadequate patient positioning at operation and dental injuries from intubation are examples of complications that may be reason for medical claim but are not predefined in the register chart. For patients identified in LÖF claims, medical records and data in GallRiks were compared and discrepancies noted. The minimum follow-up time was 6 months. Ethical Approval
Ethical approval for this study was obtained from the Regional Ethical Committee of Linköping, Sweden.
For comparing the proportions of a complete match on individual level between the first and second audit, Mann–Whitney nonparametric test was used. Proportions for correctness and inconclusiveness were compared using the Chi-square test. Due to multiple comparisons, the significance level was set at 0.01. A total sum of >10 registrations including first and second audit for incorrect and inconclusive registrations was required to be subjected for statistical evaluation. Results Completeness of Events—National, Regional, and Local Coverage
In total, 64,538 (100%) cholecystectomies were performed during the study period, 2007–2011. A total of 53,504 (82.9%) of the operations were registered in GallRiks and 61,687 (95.6%) in NPR. Overall, 50,653 (78.5%) of the patients were found in both registers (see Fig. 1). As the inclusion of hospitals progressed, the national coverage gradually increased and after 2009 was above 85%. Missed registrations in NPR that were identified in GallRiks varied between 3% and 8% and missed registrations in GallRiks that were identified in NPR varied between 12% and 27%. In 2011, the coverage in the 21 different counties ranged between 55% and 98%, an example of regional differences. In eight counties, the coverage was ≥90% in total including 6907 (52%) of the 13,290 cholecystectomies performed. At the hospital level, the corresponding figures for 2011 were median 91%, ranging between 0% (representing two hospitals not enrolled) and 100% (three hospitals). A total of 37 hospitals out of 70 reported more than 90% of their cholecystectomies, together including 8200 (62%) out of the 13,290 cholecystectomies performed. Overall Correctness of Data—Validation by Repeated Audits
As of December 2011, a total number of 63,851 cholecystectomies were reported to GallRiks from 76 different hospitals. Seven hospitals recently either joined the register (four) or stopped performing cholecystectomies (three) and were accordingly excluded. Thus, 69 hospitals were eligible and audited twice during the study period. At the first audit, medical records for 1257 cholecystectomies and at the second visit 1252 records were scrutinized. A complete match between GallRiks and the medical record was found in 780 (62%) of the patients during the first audit, but increased to 914 (73%) during the second (p < 0.001) (Table 1). Altogether 94,919 sets of data were available for comparison at the first and second audit. The entries were found to be correct in 46,213(97.2%) occasions at the first audit and in 46,522 (98.2%) at the second. In 782 (1.7%) and 543 (1.2%) occasions, respectively, the entries were incorrect. The amount of data where the correctness could not be determined or the corresponding data were missing in the medical record were reduced from 599 (1.2%) to 291
240
J. Rystedt, et al.
Table 1 The correctness of the registrations during the first audit compared to the second with regard to the number of records with a complete match, that is, all 43 variables correct, 1, 2, or >2 errors respectively (p < 0.001). First audit
Second audit
n
%
n
%
Complete match 1 error 2 errors > 2 errors Number of patients
780 306 101 70 1257
62 24 8 6 100
914 213 87 38 1252
73 17 7 3 100
Table 2 The number of cholecystectomies with a claim to LÖF (n = 253) registered and not registered in GallRiks in different groups of complications (p = ns). (LÖF: Swedish Patient Insurance; GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography). Registered
Not registered
Claim to LÖF (n = 253)
Number
%
Number
%
Major bile duct (n = 15) Minor bile duct (n = 30) Early surgical (n = 129) Late or none surgical (n = 79)
14 26 114 73 227
93 87 88 92 90
1 4 15 6 26
7 13 12 8 10
LÖF: Swedish Patient Insurance. 354 claims to LÖF
82 from hospitals not par cipa ng in GallRiks
Major bile duct (n=14)
14
0
Minor bile duct (n=26)
26
0
Early surgical (n=114)
103
Late or non surgical (n=73) 272 from hospitals par cipa ng in GallRiks
19 claims without complica on
253 complica ons from hospitals par cipa ng in GallRiks
26 complica ons opera on not registered in GallRiks
24
0% Opera on registered
11 49
50%
100%
Complica on missing
Fig. 3. The number of cholecystectomies with a claim to LÖF registered in GallRiks (n = 227) with and without a correct registration of the complication (p < 0.001). LÖF: Swedish Patient Insurance; GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography.
and 6.9% respectively. Registration of “biliary leakage” showed good agreement, 99.2% and 99.1% (Appendix). Correctness of Rare Adverse Events
227 with complica ons and registered in GallRiks
Fig. 2. Claims received by LÖF. GallRiks: The Swedish Register for Gallstone Surgery and Endoscopic Retrograde Cholangiopancreatography; LÖF: Swedish Patient Insurance.
(0.6%) between audits. Of the possible 43 variables controlled for each patient, operating time had the highest proportion of incorrect registrations in the first audit; 10.4% of entries were noted incorrect, which was reduced to less than half in the second audit. In the complication registration, “cause of biliary leakage” had the least agreement with the medical records in the first and second audits, 6.8%
During the study period, 354 claims concerning injuries related to cholecystectomy were received by LÖF (Fig. 2). In 272 cases, the operation was performed at a hospital participating in GallRiks at the time of the injury. All these patients were assessed in detail. In 19 cases, the claim was not caused by a complication and was excluded, resulting in 253 patients remaining for evaluation. Of the 253 claims, a total of 227 (90%) were registered in GallRiks. The 26 claims where the cholecystectomy was not registered were evenly distributed between the different groups of complications (Table 2). For the 227 cases registered in GallRiks, the claims, medical records, and data in the register were compared and discrepancies noted. Of 114 early surgical complications, 103 (90%) were registered, whereas nonsurgical/late complications were recorded in 33% of the cases (Fig. 3). When the cholecystectomy was registered in GallRiks, all bile duct injuries, major (n = 14) and minor (n = 26), were reported correctly.
Population-based national register
Discussion This study estimates the validity of GallRiks database with regard to completeness and correctness of the registered data. The overall coverage is 83% and above 85% in later years, all in line with increasing inclusion of hospitals in the register. The study demonstrates that the overall correctness in GallRiks is high and increasing over time, possibly attributed to the periodical audits. The high proportion of incorrect registrations (7%) for the complication “cause of biliary leakage” is probably due to the fact that it is often difficult to decide what the most likely cause is. Since the correctness for “biliary leakage” itself was above 99% at both audits, we estimate the risk of bile duct injuries not being noted in the register to be minor. There are no signs of systematical bias regarding registration of operations resulting in procedure-related early surgical complications or serious adverse events such as bile duct injuries. Regarding nonsurgical or late complications such as incisional hernias the register is not reliable since the follow-up is not designed to include this information. Although the healthcare in Sweden is publically funded, it is not compulsory to register in databases like GallRiks. But it is expected that all units performing cholecystectomies on a regular basis will participate since the results are published annually, on national and regional level, in “Open Comparisons,” information available to the public (2). In Denmark, on the other hand, the Danish Cholecystectomy Database (DCD) started in 2006 and was made mandatory. It had specified objectives, which were to increase the proportion of cholecystectomies completed laparoscopically, to decrease the postoperative length of stay, and to reduce the rate of surgical complications (11). This study also enlightens the fact that no register is flawless. The NPR is regarded the most complete patient register in Sweden and is often used as the “gold standard” in epidemiological studies. However, 3% up to 8% of the cholecystectomies were missing in the NPR during the study period. Neither are the medical records complete nor totally accurate. Wauben et al. (12) reported that the operative notes of laparoscopic cholecystectomy lacked information on several important procedural steps compared to video recordings. Incomplete medical records will obviously limit the possibility to make an adequate validation of registers. In our study, information was lacking in the medical records in some, but few, cases. Cholecystectomy is one of the most common surgical procedures in Sweden and iatrogenic bile duct injuries a much feared complication. The incidence of cholecystectomies in Sweden has been estimated to be approximately 122 per 100,000 inhabitants. Thus, among the 9.5 million Swedish inhabitants, approximately 11,600 cholecystectomies were expected to be performed annually (13). However, this study shows that the annual number of cholecystectomies in Sweden is 13,200, that is, 139 per 100,000 inhabitants. A bile duct injury can be fatal or render impaired quality of life for the patient (14–16). It represents a
241
considerable cost for society and is a natural focus for patient safety work (17, 18). A higher incidence of bile duct injuries has been suggested after the introduction of laparoscopic surgery (19, 20). A Swedish epidemiological study from 2006 on 150,000 patients demonstrated only a small to moderate increase, from 0.40% to 0.47%, in the incidence of bile duct injuries after the introduction of laparoscopy (21). In the early laparoscopic era, there was a register for laparoscopic cholecystectomy in Sweden (LR), no longer active, which included over 11,000 procedures. In the audit in 1997, comparing LR to LÖF, it was found that in 11 out of 24 cases the bile duct injury was not noted in LR (22). In this study, we noted a 100% registration of bile duct injuries in GallRiks compared to LÖF. When studying important but rare complications such as bile duct injuries, two different types of bias might be considered. One is the neglect to enter an operation with a certain type or severity of complication in the register. Another potential risk is an incorrect registration of the operation as being uncomplicated when a complication did occur. To study the correctness of entries of rare events like bile duct injuries, the sample size has to be very large, and the resources required for such a study will be out of proportion. An alternative approach, used in this investigation, is to study patients suffering from the complication and assess whether the operation and the associated complication actually were registered. In such an analysis, another source for identifying these patients is needed. We used a national patient insurance database (LÖF), and neither neglect to register the cholecystectomy nor incorrect registration of the complication was found in GallRiks. This is contrary to what was found in a previous study of a Swedish register of laparoscopic cholecystectomy (22). The national quality registries enable studies to be performed on large unselected populations. This may be a feasible alternative to address scientific issues difficult to study in prospective randomized trials. This unique possibility has to be cared for by keeping data input valid and as complete as possible. In voluntary registers, compliance is essential and is enhanced by regular feedback of results and by a thorough validation process. This is met by GallRiks annual reports, individual online feedback, and by audit by independent investigators. Auditing on-site may promote local hospital management to allocate adequate resources to register work as well as increase thoroughness in the local work of entering data. Furthermore, at the on-site visits, suggestions for improvement of the register have been gathered and implemented in the register. Fine et al. (23) showed that it is possible to improve the quality of a database by validating, monitoring, and giving feedback. In this study, the validation process of a large population-based register on cholecystectomy is described and its strengths and limitations pointed out for the use in clinical development and for research.
Overall (43 variables) Preoperative (9) Operation (12) Perioperative complications (4) 30-day follow-up (18) Preoperative Date of surgery Inpatient or day care (intention) Date of admission Acute or elective surgery Indication for surgery Cholecystitis (ongoing) Pancreatitis (ongoing) Cholecystitis (past) Pancreatitis (past) Operation Surgical method Mode of gallbladder mobilization Antibiotics Surgeon Cholangiography Method of treatment of duct stones Method of cystic duct closure Method of cystic artery closure Perioperative diagnosis Gallbladder sent for PAD Perforation of gallbladder (accidental) Operating time Perioperative complications Perioperative complication Perforated bowel
47356 10235 13727 4997 18397 1249 1238 247 1248 1251 1251 1251 1250 1250 1245 1150 1246 1247 1243 149 1246 1244 1242 1243 1234 1238 1247 1251
47554 10445 13750 4982 18377
1251 1247 430 1252 1254 1252 1253 1254 1252
1253 1114 1251 1252 1247 153 1246 1250 1250 1251 1242
1241
1248 1230
2nd audit
1233 1228
1063
1245 1078 1195 1242 1206 147 1136 1164 1180 1224 1149
1237 1221 409 1240 1188 1212 1243 1209 1229
46213 10188 13029 4956 18040
8 1
129
7 30 31 8 32 6 48 23 19 9 56
13 24 21 10 51 34 5 32 17
782 207 398 17 160
0.6 0.1
10.8
0.6 2.7 2.5 0.6 2.6 3.9 4.1 1.9 1.6 0.7 4.6
1.0 1.9 4.9 0.8 4.1 2.7 0.4 2.6 1.4
1.7 2.0 3.0 0.3 0.9
1234 1250
1073
1242 1135 1213 1232 1219 139 1224 1187 1204 1222 1201
1243 1215 240 1240 1188 1231 1249 1202 1223
46522 10031 13291 4974 18226
10 1
50
2 11 24 11 16 8 7 24 16 10 28
6 21 6 7 52 16 2 45 24
543 179 207 17 140
0.8 0.1
4.5
0.2 1.0 1.9 0.9 1.3 5.4 0.6 2.0 1.3 0.8 2.3
0.5 1.7 2.4 0.6 4.2 1.3 0.2 3.6 1.9
1.2 1.8 1.5 0.3 0.8
Correct Incorrect Incorrect Correct Incorrect Incorrect (n) (n) (%) (n) (n) (%)
1st audit Total 1st Total 2nd audit audit (n) (n)
Correctness
0.107 0.670 0.118 0.469 0.923 0.010 0.256 0.138 0.273
0.640 0.990
