Health Policy 119 (2015) 1017–1022
Contents lists available at ScienceDirect
Health Policy journal homepage: www.elsevier.com/locate/healthpol
Health reform monitor
Developing cross-sectoral quality assurance for cataract surgery in the statutory quality assurance program of the German health care system: Experiences and lessons learned Anke Bramesfeld a,b,∗ , Jürgen Pauletzki a , Lars Behrenz a , Joachim Szecsenyi a,c , Gerald Willms a , Björn Broge a a AQUA Institute for Applied Quality Improvement and Research in Health Care GmbH, Maschmühlenweg 8–10, 37073 Göttingen, Germany b Institute for Epidemiology, Social Medicine and Health System Research, Hanover Medical School, Carl-Neuberg-Straße 1, 30625 Hanover, Germany c Department of General Practice and Health Services Research, Universitätsklinikum Heidelberg, Voßstraße 2, 69115 Heidelberg, Germany
a r t i c l e
i n f o
Article history: Received 14 October 2014 Received in revised form 23 February 2015 Accepted 15 March 2015 Keywords: Quality in health care Quality assessment Quality improvement Health insurance data Health policy Health system
a b s t r a c t Since 2001, statutory external quality assurance (QA) for hospital care has been in place in the German health system. In 2009, the decision was taken to expand it to cross-sectoral procedures. This novel and unprecedented form of national QA aims at (1) making the quality procedures comparable that are provided both in inpatient and outpatient care, (2) following-up outcomes of hospital care after patients’ discharge and (3) measuring the quality of complex treatment chains across interfaces. As a pioneer procedure a QA procedure in cataract surgery QA was developed. Using this as an example, challenges of cross-sectoral QA are highlighted. These challenges relate, in particular, to three technical problems: triggering cases for documentation, following-up patients’ after hospital discharge, and the burden of documentation in outpatient care. These problems resulted finally in the haltering of the development of the QA procedure. However, the experiences gained with this first development of cross-sectoral QA inspired the reorientation and further development of the field in Germany. Future cross-sectoral QA will rigorously aim at keeping burden of documentation small. It will draw data for QA mainly at three sources: routine data, patient surveys and peer reviews using indicators. Policy implications of this reorientation are discussed. © 2015 Elsevier Ireland Ltd. All rights reserved.
1. Background Since 2001, all hospitals in Germany are legally required to provide data for external quality assurance (QA) [1].
∗ Corresponding author at: AQUA—Institut für angewandte Qualitätsförderung, und Forschung im Gesundheitswesen GmbH, Maschmühlenweg 8–10, 37073 Göttingen, Germany. Tel.: +49 551/789 52 273; fax: +49 551/789 52 10. E-mail addresses: [email protected], [email protected] (A. Bramesfeld). http://dx.doi.org/10.1016/j.healthpol.2015.03.008 0168-8510/© 2015 Elsevier Ireland Ltd. All rights reserved.
External QA aims at comparing similar services that are provided by different hospitals and thus, distinguishing hospitals by their quality of clinical performance. For the time being, this type of statutory QA that compares the performance of services is being implemented for inpatient services only. Therefore, at the moment, QA in hospitals can only provide reliable information on outcomes that are achieved during a hospital stay. The assessment of outcome indicators for long- or midterm outcomes after being discharged from hospital is only possible on a voluntary basis and if data is available.
1018
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
This inability of the present QA scheme to report midand long-term outcomes reliably was one of the reasons why the legislator decided in 2007 that external QA should in future be implemented preferably cross-sectorally (section 137a of Book V of the German Social Code (SGB V)). The need for reliable follow-up information after being discharged was reinforced by a steady drop in the number of days spent in hospital in recent decades [2]. It was also emphasized by an increase in surgical and other procedures that used to require hospital admission but which are nowadays performed in outpatient care [3]. In addition, it was acknowledged that the quality of inpatient care for certain diagnoses depends not only on processes in hospital, but also on processes in outpatient care before admission and after being discharged. Finally, it needs to be recognized that in a complex health system that is dominated by multi-morbidity, the quality of care depends significantly on the interplay and cooperation between the medical specialties and sectors involved. Thus, implementing QA from a cross-sectoral perspective seemed an obvious however novel way to proceed [4]. In 2009, the AQUA Institute was commissioned with the development and implementation of statutory QA in the German health care system. Since internationally there are no precedent models of cross-sectoral QA that cover complete health systems this required a pioneer approach. The first clinical area to be selected for development was cataract surgery. This first QA procedure, like also some that followed was halted on account of reservations mainly by the stakeholders about the effort-benefit ratio of these procedures. Although not developed right through to full implementation, this first QA procedure on cataract surgery is a valuable source for learning and highlighting central issues and pitfalls in cross-sectoral QA. By the example of cataract surgery this paper describes these central issues and pitfalls of cross-sectoral QA and reports on lessons learned. 2. Statutory Quo of statutory quality assurance in the German health system To better understand the challenges of implementing nationally cross-sectoral QA in the German health system, a few facts about this health system need to be highlighted: The German health care system is based on more than 90% of the population being health insured in one of Germany’s statutory health insurance companies. Which services in medical care are reimbursed by statutory health insurance companies and the framework and contents of QA for these services is specified centrally by the Federal Joint Committee (Gemeinsamer Bundesausschuss, http://www.english.g-ba.de/). It is the highest decisionmaking body of the joint self-government of physicians, dentists, hospitals and health insurance funds in Germany. It was the Federal Joint Committee that mandated the AQUA Institute with the development and implementation of QA in the health care system. At the moment statutory QA assesses clinical performance for 30 clinical areas in almost 2000 hospitals and approximately 4 million hospital inpatients by using indicators on structure, processes and also outcomes (see
Table 1 Clinical areas of the statutory external quality assurance scheme in the German health care system. 1. Cholecystectomy 2. Carotid artery reconstruction 3. Community-acquired pneumonia 4. Pacemaker: implantation 5. Pacemaker: replacement of generator/battery 6. Pacemaker: revision/system replacement/removal 7. Implantable cardioverter defibrillators: implantation 8. Implantable cardioverter defibrillators: replacement of generator/battery 9. Implantable cardioverter defibrillators: revision/system replacement/removal 10. Coronary angiography and percutaneous coronary intervention (PCI) 11. Coronary surgery, isolated 12. Aortic valve surgery, isolated 13. Combined coronary and aortic valve surgery 14. Heart transplantation 15. Lung and heart–lung transplantation 16. Liver transplantation 17. Living liver donation 18. Kidney transplantation 19. Living kidney donation 20. Pancreas and pancreas–kidney transplantation 21. Breast surgery 22. Obstetrics 23. Neonatology 24. Gynecological surgery 25. Femoral fracture near the hip joint 26. Hip replacement: primary implantation 27. Hip replacement: revision and component exchange 28. Total knee replacement: primary implantation 29. Knee replacement: revision and component exchange 30. Nursing: prevention of pressure ulcers
Table 1). For the indicators that are connected to the 30 clinical areas, hospitals collect data continuously throughout the year. Results of performance indicators are reported back to the hospitals once a year and hospitals need to justify their data once they perform outside the expected range [5]. At the beginning of the national QA scheme, data for indicators was collected solely by hand. Today, data is in some parts also drawn from routine data that is available in hospitals’ electronic reporting systems. 3. Developing cross-sectoral quality assurance When the political decision was taken to implement cross-sectoral QA, hopes were raised that this could be a solution for many problems arising from the strict segregation of inpatient and outpatient care in German health care [4]. Cataract surgery was chosen as one of the first clinical areas for which cross-sectoral QA was to be developed because – in part – it is considered a simple, clear cut intervention. Further it is performed similarly in inpatient and outpatient care and requires a simple one-stage follow-up consultation to check for complications and outcomes that is usually done in outpatient care. It is estimated that annually about 600,000 to 800,000 cases of cataract surgery are performed, of which 88% in outpatient care [6,7]. Thus, the established quality assurance system that focuses only on inpatient care would miss most cataract surgery. The development of new QA procedures by the AQUA Institute follows defined methodological standards that are
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
described in detail elsewhere [4,8]. In this developmental process, first indicators are defined with the help of multi-professional expert panels and following a modified RAND/UCLA methodology [9]. Then, triggers for documentation and instruments for assessing the necessary data are identified and applicability of the indicators is thereby checked. Finally, the QA procedure is trialled on a sample of service providers. Between each developmental step, agreement on the results and further progress is sought with the Federal Joint Committee. For cataract surgery, identifying indicators and agreeing on them with the Federal Joint Committee was not a problem. Difficulties started arising, however, when identifying triggers for documentation and when trialling the procedure. These difficulties were characteristic of the technical challenges of cross-sectoral QA and focused on three issues: (1) Triggering cases for documentation. (2) Follow-up of patients. (3) Documentation in outpatient care. 4. Triggering cases for documentation Statutory QA relies on reliable triggers for documentation that work with all service providers [10]. These triggers are best linked to processes that are coded for reimbursement reasons by service providers, such as diagnoses or procedures. Once coded, the service provider software automatically creates a pop up message that documentation for QA purposes is needed. Triggers need to be sensitive and selective at the same time. This means that they should catch all possible cases affected, however, as few false positive cases as possible. Reimbursement codes for diagnoses or procedures related to cataract surgery were revealed to be sensitive and selective triggers only in inpatient care. While inpatient care knows clear coding guidelines, this is lacking in outpatient care. Moreover, service providers in outpatient care are increasingly entering into specific contracts with health insurance companies, whereby specific services such as outpatient surgery are reimbursed under conditions that differ from the norm. In cases of specific contracts (known as selective contracts), coding can be totally different. The solution was to combine diagnostic codes indicating all possible forms of a cataract (ICD-10 H25 through H28) with the codes for the procedure of intra- and extracapsular lens extraction. This revealed to be the most sensitive and specific triggers for documentation both in inpatient and outpatient surgery, if outpatient surgery was performed under the regular reimbursement scheme. However, outpatient surgery performed under selective contracts cannot be monitored reliably by this method. It was expected that once the QA procedure had been implemented, health insurance companies would urge outpatient service providers to adapt to the coding required when using selective contracts for cataract surgery. 5. Follow-up of cases The follow-up of most interventions usually occurs in outpatient care. To trigger follow-up documentation,
1019
outpatient service providers need to be reliably informed that follow-up documentation on a specific procedure is needed. However, for administrative reasons and reasons regarding reimbursement, inpatient and outpatient care is strictly separated. There is no automatic and standardized data transfer from inpatient to outpatient care and vice versa. Hopes of attaching the information required for the quality assessment to an electronic health insurance card that each person with statutory health insurance would have to present when visiting a physician, had to be abandoned as technical problems and political controversy regarding data protection could not be resolved. The solution to the problem of follow-up information was to trigger follow-up documentation independently and then later to link the data from the initial procedure to data from the follow-up documentation for each individual patient. On account of this link, a patient’s identity must be kept confidential. Therefore, before the data of the procedure and follow-up appointment can be linked and analysed, it has to be pseudonymized by a centre handling confidential data. Only after that would the data be forwarded to the AQUA Institute for further processing and analysis. Finding a trigger for follow-up documentation poses similar demands such as finding a trigger for the initial documentation in outpatient care. For the follow-up of cataract surgery, no reimbursement code could be identified that would act as a trigger that is sensitive and specific at the same time. The follow-up to cataract-surgery is usually coded by a rather general code that is also used for other follow-up care of eye-related surgery. One possibility would have been to implement a filter question (“Has cataract surgery been performed recently?”) to increase the specificity of this trigger. However, this was felt by the Federal Joint Committee to require too much effort that was believed to be not reasonable for outpatient service providers. In addition, answering a filter question opens possibilities for manipulation towards under-reporting. Therefore, it was decided to trigger follow-up documentation by using only the unspecific reimbursement codes. Trialling the procedure, however, revealed that leaving the follow-up trigger as unspecific as it was, led to documentation being falsely triggered in 52% of cases. This was deemed unacceptable and in the end, plans for having follow-up documentation after cataract surgery were abandoned.
6. Documentation in outpatient care Implementation of QA documentation in outpatient care not only for follow up but already for the surgery itself proved to be a much bigger challenge than in inpatient care. There were several reasons for this: • Personnel: Most outpatient services are provided by single or relatively small provider practices. Resources for additional tasks, such as additional documentation are therefore limited. In comparison, hospitals have extra departments or at least trained personnel who are
1020
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
appointed for quality assessment and documentation management. • IT: Statutory documentation for quality purposes is a somewhat new topic in the outpatient sector. Where it already exists, it is limited to very specific services such as disease management programmes or dialysis. Therefore, often infrastructure is lacking, in particular when it concerns IT. For QA, software is needed that firstly triggers documentation and secondly supports data collection. Furthermore, this software needs to be capable of communicating with the software already implemented in the practices for reimbursement purposes. Implementing new software does not only mean financial investment for the practices, but also requires effort for developing such software. While in the inpatient sector there are about 30 large software companies that develop software solutions for hospitals in Germany, there are more than 150 software suppliers in the outpatient sector, some of them being very small companies. They all have to be considered when seeking implementation of QA documentation in outpatient care. • Experience and tradition: Finally, outpatient physicians are mostly inexperienced with having to provide extra documentation for the purposes of statutory QA. Not only that, there is no tradition of having to do this and therefore finding ophthalmologic practices that would participate in the trail phase for the QA-procedures revealed to be difficult. To overcome resistance towards providing documentation, it has proved helpful if medical associations are in support of the development and implementation of QA. This was the case with the QA procedure on Percutaneous Coronary Intervention (PCI) that was developed later. The National Association of Cardiologists actively supported the trial phase during the development of this QA procedure. Unlike the trial phase for cataract surgery which lacked such support, the trial phase for PCI went much better and the results were promising. Furthermore, many outpatient cardiologists were already participating in data collection for a voluntary registry. Thus, the IT infrastructure, as well as the experience in providing extra documentation, was already in place. 7. Lessons learned The implementation of a QA procedure for cataract surgery was halted mainly because in outpatient care the effort of providing documentation for the surgical procedure and also for the follow-up care was judged to be inacceptable. It was judged in particular that the effort required for providing follow-up documentation was too great. The idea of introducing a filter question to narrow down the amount of documentation being falsely triggered by an unspecific trigger was rejected; similarly, the QA procedure was also rejected on account of the fact that physicians were filling out about 50% of the documentation in vain following an unspecific trigger. Although this rejection of the QA procedure on account of the high rate of falsely triggered documentation is understandable, experience shows that acceptance does not inevitably mean a high level of
efficiency with regard to the documentation: Within the established QA procedure for pressure ulcers, hospitals documented in 2012 up to 1300,000 cases of which only about 10,000 were correct positive cases. Not all hospitals liked this QA procedure on pressure ulcers [11]. However, it worked. A lesson learned from the non-acceptance of the burden of documentation in the QA procedure for cataract surgery was the insight that QA needs to rigorously review its methods to keep the documentation burden as minimal as possible. This is in particular relevant for outpatient care where the documentation infrastructure is much less welldeveloped than in hospitals. Therefore, three additional data sources for statutory QA were introduced: routine data, patient surveys and peer reviews. • Routine data: Routine data usable for QA purposes includes data from the hospitals’ electronic accounting systems and claims data from the health insurance companies. The advantage of this routine data relates to it being available not only without the need for additional effort to obtain the extra documentation, but also the fact that it is available nationally in a similar form. In addition, health insurance claims data is available longitudinally and in outpatient as well as in inpatient care. The disadvantages of routine data relate to it being produced for claims purposes and not for quality measurement. It is often not adequate for mapping all the issues that are relevant for quality [12]. Routine data can often only assess proxies of outcomes and processes, and thereby only partially depict reality. In particular, patient organization representatives are concerned that important quality deficits/potential might be missed if QA is based more and more on routine data and oriented towards “effort-adequate procedures” [13]. Therefore, despite all available routine data and for have meaningful QA procedures in future that scrutinize the relevant processes, QA will not be able to do without additional manual documentation by service providers. In order to be able to use health insurance claims data for the purposes of statutory QA, an amendment of the current legislation was needed. This was undertaken in December 2012. Implementation directives are expected to be adopted in 2014. • Patient survey: Patients’ views are crucial for judging patient relevant outcomes such as pain or functioning in daily life, and they are often the only tie between different health care sectors and services [14,15]. Patients have been defined as clear definers of good quality, good evaluators of the heath care they receive and good rapporteurs of their experiences with health care [16]. The use of patient surveys does not only carry the advantage of providing valuable information from the patient perspective; it also offers the additional benefit of lowering the burden of documentation for service providers. Issues assessed by asking patients usually do not have to be additionally documented by service providers. Many QA procedures that have been commissioned for further development, such as a QA procedure on arthroscopy, now include a patient survey. It should, however, not be forgotten that implementing patient
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
1021
surveys has its own technical, legal and policy challenges that will be addressed elsewhere. • Peer review: Peer reviews are better able to value the performance of services in terms of their particularity and context than just indicators. E.g. the quality of indication might be better scrutinized by a peer review than by indicators. Unlike indicators, peer reviews are carried out only on a sample of service providers. As a side effect, peer-reviews can contribute to decrease the burden of documentation for service providers. The disadvantage of peer reviews is the organizational effort that they require. The development of a QA procedure using peer reviews has recently been commissioned for arthroscopy. This is work in progress; therefore the results and experiences are pending.
Federal Joint Committee, has just recently adopted a decision reinforcing the implementation of long- and mid-term statutory follow-up assessments [21]. The difficulties involved in data assessment in crosssectoral QA, as well as the sector diversity and the fact that many sectors are simply unprepared for the participation in QA, are more or less technical problems. However, the acceptance of QA and the question as to how much effort stakeholders are willing to make for QA is a political issue. It is related not only to the motivation for transparency, but also to the time and financial burden related to QA documentation; finally, it is also related to the expected results. Even if QA is as efficient as possible, it needs to be acknowledged that meaningful QA will never be available without a certain amount of time and effort being invested.
Despite the burden of documentation and issues of acceptability, the implementation of cross-sectoral QA has been challenged by technical problems arising at the interfaces of the service sectors as described in the paragraphs on triggering cases and on follow-up of cases. This includes problems with the transfer of data and information across sectors as well as problems with the different coding systems and traditions in inpatient and outpatient care These problems mirror the segregation of the German health care system, which cross-sectoral QA at the same time lines up to overcome [4]. In addition, problems arise on account of the diversity of the outpatient sector containing different services and different coding and reimbursement systems and on account of the outpatient sector not being prepared for participating in statutory external QA. Even Avedis Donabedian believed that QA in the outpatient sector was virtually impossible, for exactly these reasons [17]. The experiences of other modern health systems show, however, that some health systems succeed in implementing effective QA by indicators also in outpatient care: The Israeli Health System has implemented statutory QA for community health care. All data is collected from electronic patient records that all primary care physicians use that work for one of the four Health Plans covering the Israeli population [18,19]. Also Sweden reports indicators for outpatient care. These indicators are extracted from registries, in which data is inserted either electronically or by extra documentation [20].
Conflict of interest statement
8. Conclusion Cross-sectoral QA is a compelling concept that considers not only a single moment in the procedure, but the chain of health care services provided across sectors and over the course of time. It needs to be considered that most health care is provided in outpatient care and that in fact, the quality of chronic care is decided longitudinally and in the outpatient sector. For this reason, it is worth taking the challenge and extending the wellelaborated and implemented system of QA in hospital care to the outpatient sector and to cross-sectoral QA. More cross-sectoral QA, in particular, in what concerns followup care after being discharged from hospital is what will certainly be implemented in the near future in the German health care system. The responsible authority, namely the
All authors are employed by the AQUA-Institute and work within the development of cross-sectoral quality assurance within the German statutory quality assurance programme (www.sqg.de) that is financed by the Federal Joint Committee. Other than that no conflicts of interest are present. Acknowledgement We are grateful to Sharon Janicki for revising the English of this article. References [1] Willms G, Bramesfeld A, Pottkämper K, Broge B, Szecsenyi J. Aktuelle Herausforderungen der externen Qualitätssicherung im deutschen Gesundheitswesen. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen 2013;107(8):523–7 (Aktuelle Herausforderungen der externen Qualitatssicherung im deutschen Gesundheitswesen). [2] OECD. Health status Organisation for Economic Co-operation and Development. OECD; 2014 (cited 2014 04.02). Available from http://stats.oecd.org/index.aspx?DataSetCode=HEALTH STAT. [3] Bölt U, Graf T. 20 Jahre Krankenhausstatistik. Wirtschaft und Statistik 2012;2012(2):112–38. [4] Szecsenyi J, Broge B, Eckhardt J, Heller G, Kaufmann-Kolle P, Wensing M. Tearing down walls: opening the border between hospital and ambulatory care for quality improvement in Germany. International Journal for Quality in Health Care 2012;24(2):101–4 [Epub 2012/01/25], (journal of the International Society for Quality in Health Care/ISQua). [5] AQUA. Qualitätsreport 2012. Göttingen: AQUA—Institut für angewandte Qualitätsförderung und Forschung im Gesundheitswesen; 2013. [6] Wenzel M, Pham DT, Scharrer A, Schayan K, Klasen J. Derzeitiger Stand der ambulanten Ophthalmochirurgie: Ergebnisse der Umfrage 2006 des BDOC, BVA und der DGII. Ophtalmo—Chirurgie 2007;19:128–38. [7] Kohnen T, Baumeister M, Kook D, Klaproth OK, Ohrloff C. Kataraktchirurgie mit Implantation einer Kunstlinse. DtschArzteblInt 2009;106(43):695–702. [8] AQUA. Allgemeine Methoden im Rahmen der sektorenübergreifenden Qualitätssicherung im Gesundheitswesen nach §137 a SGB V. Version 3. 0 (Stand: 15. Februar 2013). Göttingen: AQUA—Institut für angewandte Qualitätsförderung und Forschung im Gesundheitswesen; 2013. [9] Fitch K, Bernstein SJ, Aguilar MD, Burnand B, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND; 2001. [10] Albrecht M, Loos S, Otten M. Sektorenübergreifende Qualitätssicherung in der ambulanten Versorgung. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen
1022
[11]
[12]
[13]
[14]
[15]
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022 2013;107(8):528–33 [Epub 2013/12/03]. (Sektorenubergreifende Qualitatssicherung in der ambulanten Versorgung). AQUA. German Hospital Quality Report 2012. Göttingen: AQUA—Institute for Applied Quality Improvement and Research in Health Care GmbH; 2014. Busse R, Nimptsch U, Mansky T. Measuring, monitoring, and managing quality in Germany’s hospitals. Health Affairs (Millwood) 2009;28(2):w294–304 [Epub 2009/01/29]. PatientenVertretung im Unterausschuss Qualitätssicherung der G-BA. Qualitätssicherung im Gesundheitswesen: Positionen der Patientenvertretung zum Eckpunktepapier der Träger des G-BA. Berlin: Gemeinsamer Bundesausschuss; 2013. Ludt S, Heiss F, Glassen K, Noest S, Klingenberg A, Ose D, et al. Die Patientenperspektive jenseits ambulant-stationärer Sektorengrenzen—Was ist Patientinnen und Patienten in der sektorenübergreifenden Versorgung wichtig? Gesundheitswesen 2014;76(6):359–65. Blum K, Satzinger W, Buck R. Patientenbefragungen und Qualitätsmanagement. Eine Einführung in die Thematik. In: Satzinger W, Trojan A, Kellermann-Mühlhoff P, editors. Patientenbefragungen in Krankenhäusern. Sankt Augustin: Asgard-Verlag; 2001. p. 25–40.
[16] Donabedian A. The Lichfield Lecture. Quality assurance in health care: consumers’ role. Quality in Health Care: QHC 1992;1(4):247–51. Epub 1992/11/04. [17] Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly 1966;44(3 Suppl.):166–206. [18] Jaffe DH, Shmueli A, Ben-Yehuda A, Paltiel O, Calderon R, Cohen AD, et al. Community healthcare in Israel: quality indicators 2007–2009. Israel Journal of Health Policy Research 2012;1(1):3. Epub 2012/08/24. [19] OECD. OECD reviews of health care quality: Israel. Executive summary, assessment and recommendations. Organisation for Economic Co-operation and Development; 2012. [20] Socialstyrelsen. Quality and efficiency in Swedish health care. Regional comparisons 2012. Stockholm: Swedish Association of Local Authorities and Regions, Swedish National Board of Health and Welfare Publikationsservice; 2013. [21] G-BA. Pressemitteilung Daten zur Behandlungsqualität in Krankenhäusern: G-BA treibt Follow-up-Verfahren voran (Nr. 27/2014 vom 19. Juni 2014). Berlin: Gemeinsamer Bundesausschuss; 2014 (cited 2014 27.06). Available from https://www.g-ba.de/downloads/34215-540/27-2014-06-19-QSKH.pdf.
Contents lists available at ScienceDirect
Health Policy journal homepage: www.elsevier.com/locate/healthpol
Health reform monitor
Developing cross-sectoral quality assurance for cataract surgery in the statutory quality assurance program of the German health care system: Experiences and lessons learned Anke Bramesfeld a,b,∗ , Jürgen Pauletzki a , Lars Behrenz a , Joachim Szecsenyi a,c , Gerald Willms a , Björn Broge a a AQUA Institute for Applied Quality Improvement and Research in Health Care GmbH, Maschmühlenweg 8–10, 37073 Göttingen, Germany b Institute for Epidemiology, Social Medicine and Health System Research, Hanover Medical School, Carl-Neuberg-Straße 1, 30625 Hanover, Germany c Department of General Practice and Health Services Research, Universitätsklinikum Heidelberg, Voßstraße 2, 69115 Heidelberg, Germany
a r t i c l e
i n f o
Article history: Received 14 October 2014 Received in revised form 23 February 2015 Accepted 15 March 2015 Keywords: Quality in health care Quality assessment Quality improvement Health insurance data Health policy Health system
a b s t r a c t Since 2001, statutory external quality assurance (QA) for hospital care has been in place in the German health system. In 2009, the decision was taken to expand it to cross-sectoral procedures. This novel and unprecedented form of national QA aims at (1) making the quality procedures comparable that are provided both in inpatient and outpatient care, (2) following-up outcomes of hospital care after patients’ discharge and (3) measuring the quality of complex treatment chains across interfaces. As a pioneer procedure a QA procedure in cataract surgery QA was developed. Using this as an example, challenges of cross-sectoral QA are highlighted. These challenges relate, in particular, to three technical problems: triggering cases for documentation, following-up patients’ after hospital discharge, and the burden of documentation in outpatient care. These problems resulted finally in the haltering of the development of the QA procedure. However, the experiences gained with this first development of cross-sectoral QA inspired the reorientation and further development of the field in Germany. Future cross-sectoral QA will rigorously aim at keeping burden of documentation small. It will draw data for QA mainly at three sources: routine data, patient surveys and peer reviews using indicators. Policy implications of this reorientation are discussed. © 2015 Elsevier Ireland Ltd. All rights reserved.
1. Background Since 2001, all hospitals in Germany are legally required to provide data for external quality assurance (QA) [1].
∗ Corresponding author at: AQUA—Institut für angewandte Qualitätsförderung, und Forschung im Gesundheitswesen GmbH, Maschmühlenweg 8–10, 37073 Göttingen, Germany. Tel.: +49 551/789 52 273; fax: +49 551/789 52 10. E-mail addresses: [email protected], [email protected] (A. Bramesfeld). http://dx.doi.org/10.1016/j.healthpol.2015.03.008 0168-8510/© 2015 Elsevier Ireland Ltd. All rights reserved.
External QA aims at comparing similar services that are provided by different hospitals and thus, distinguishing hospitals by their quality of clinical performance. For the time being, this type of statutory QA that compares the performance of services is being implemented for inpatient services only. Therefore, at the moment, QA in hospitals can only provide reliable information on outcomes that are achieved during a hospital stay. The assessment of outcome indicators for long- or midterm outcomes after being discharged from hospital is only possible on a voluntary basis and if data is available.
1018
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
This inability of the present QA scheme to report midand long-term outcomes reliably was one of the reasons why the legislator decided in 2007 that external QA should in future be implemented preferably cross-sectorally (section 137a of Book V of the German Social Code (SGB V)). The need for reliable follow-up information after being discharged was reinforced by a steady drop in the number of days spent in hospital in recent decades [2]. It was also emphasized by an increase in surgical and other procedures that used to require hospital admission but which are nowadays performed in outpatient care [3]. In addition, it was acknowledged that the quality of inpatient care for certain diagnoses depends not only on processes in hospital, but also on processes in outpatient care before admission and after being discharged. Finally, it needs to be recognized that in a complex health system that is dominated by multi-morbidity, the quality of care depends significantly on the interplay and cooperation between the medical specialties and sectors involved. Thus, implementing QA from a cross-sectoral perspective seemed an obvious however novel way to proceed [4]. In 2009, the AQUA Institute was commissioned with the development and implementation of statutory QA in the German health care system. Since internationally there are no precedent models of cross-sectoral QA that cover complete health systems this required a pioneer approach. The first clinical area to be selected for development was cataract surgery. This first QA procedure, like also some that followed was halted on account of reservations mainly by the stakeholders about the effort-benefit ratio of these procedures. Although not developed right through to full implementation, this first QA procedure on cataract surgery is a valuable source for learning and highlighting central issues and pitfalls in cross-sectoral QA. By the example of cataract surgery this paper describes these central issues and pitfalls of cross-sectoral QA and reports on lessons learned. 2. Statutory Quo of statutory quality assurance in the German health system To better understand the challenges of implementing nationally cross-sectoral QA in the German health system, a few facts about this health system need to be highlighted: The German health care system is based on more than 90% of the population being health insured in one of Germany’s statutory health insurance companies. Which services in medical care are reimbursed by statutory health insurance companies and the framework and contents of QA for these services is specified centrally by the Federal Joint Committee (Gemeinsamer Bundesausschuss, http://www.english.g-ba.de/). It is the highest decisionmaking body of the joint self-government of physicians, dentists, hospitals and health insurance funds in Germany. It was the Federal Joint Committee that mandated the AQUA Institute with the development and implementation of QA in the health care system. At the moment statutory QA assesses clinical performance for 30 clinical areas in almost 2000 hospitals and approximately 4 million hospital inpatients by using indicators on structure, processes and also outcomes (see
Table 1 Clinical areas of the statutory external quality assurance scheme in the German health care system. 1. Cholecystectomy 2. Carotid artery reconstruction 3. Community-acquired pneumonia 4. Pacemaker: implantation 5. Pacemaker: replacement of generator/battery 6. Pacemaker: revision/system replacement/removal 7. Implantable cardioverter defibrillators: implantation 8. Implantable cardioverter defibrillators: replacement of generator/battery 9. Implantable cardioverter defibrillators: revision/system replacement/removal 10. Coronary angiography and percutaneous coronary intervention (PCI) 11. Coronary surgery, isolated 12. Aortic valve surgery, isolated 13. Combined coronary and aortic valve surgery 14. Heart transplantation 15. Lung and heart–lung transplantation 16. Liver transplantation 17. Living liver donation 18. Kidney transplantation 19. Living kidney donation 20. Pancreas and pancreas–kidney transplantation 21. Breast surgery 22. Obstetrics 23. Neonatology 24. Gynecological surgery 25. Femoral fracture near the hip joint 26. Hip replacement: primary implantation 27. Hip replacement: revision and component exchange 28. Total knee replacement: primary implantation 29. Knee replacement: revision and component exchange 30. Nursing: prevention of pressure ulcers
Table 1). For the indicators that are connected to the 30 clinical areas, hospitals collect data continuously throughout the year. Results of performance indicators are reported back to the hospitals once a year and hospitals need to justify their data once they perform outside the expected range [5]. At the beginning of the national QA scheme, data for indicators was collected solely by hand. Today, data is in some parts also drawn from routine data that is available in hospitals’ electronic reporting systems. 3. Developing cross-sectoral quality assurance When the political decision was taken to implement cross-sectoral QA, hopes were raised that this could be a solution for many problems arising from the strict segregation of inpatient and outpatient care in German health care [4]. Cataract surgery was chosen as one of the first clinical areas for which cross-sectoral QA was to be developed because – in part – it is considered a simple, clear cut intervention. Further it is performed similarly in inpatient and outpatient care and requires a simple one-stage follow-up consultation to check for complications and outcomes that is usually done in outpatient care. It is estimated that annually about 600,000 to 800,000 cases of cataract surgery are performed, of which 88% in outpatient care [6,7]. Thus, the established quality assurance system that focuses only on inpatient care would miss most cataract surgery. The development of new QA procedures by the AQUA Institute follows defined methodological standards that are
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
described in detail elsewhere [4,8]. In this developmental process, first indicators are defined with the help of multi-professional expert panels and following a modified RAND/UCLA methodology [9]. Then, triggers for documentation and instruments for assessing the necessary data are identified and applicability of the indicators is thereby checked. Finally, the QA procedure is trialled on a sample of service providers. Between each developmental step, agreement on the results and further progress is sought with the Federal Joint Committee. For cataract surgery, identifying indicators and agreeing on them with the Federal Joint Committee was not a problem. Difficulties started arising, however, when identifying triggers for documentation and when trialling the procedure. These difficulties were characteristic of the technical challenges of cross-sectoral QA and focused on three issues: (1) Triggering cases for documentation. (2) Follow-up of patients. (3) Documentation in outpatient care. 4. Triggering cases for documentation Statutory QA relies on reliable triggers for documentation that work with all service providers [10]. These triggers are best linked to processes that are coded for reimbursement reasons by service providers, such as diagnoses or procedures. Once coded, the service provider software automatically creates a pop up message that documentation for QA purposes is needed. Triggers need to be sensitive and selective at the same time. This means that they should catch all possible cases affected, however, as few false positive cases as possible. Reimbursement codes for diagnoses or procedures related to cataract surgery were revealed to be sensitive and selective triggers only in inpatient care. While inpatient care knows clear coding guidelines, this is lacking in outpatient care. Moreover, service providers in outpatient care are increasingly entering into specific contracts with health insurance companies, whereby specific services such as outpatient surgery are reimbursed under conditions that differ from the norm. In cases of specific contracts (known as selective contracts), coding can be totally different. The solution was to combine diagnostic codes indicating all possible forms of a cataract (ICD-10 H25 through H28) with the codes for the procedure of intra- and extracapsular lens extraction. This revealed to be the most sensitive and specific triggers for documentation both in inpatient and outpatient surgery, if outpatient surgery was performed under the regular reimbursement scheme. However, outpatient surgery performed under selective contracts cannot be monitored reliably by this method. It was expected that once the QA procedure had been implemented, health insurance companies would urge outpatient service providers to adapt to the coding required when using selective contracts for cataract surgery. 5. Follow-up of cases The follow-up of most interventions usually occurs in outpatient care. To trigger follow-up documentation,
1019
outpatient service providers need to be reliably informed that follow-up documentation on a specific procedure is needed. However, for administrative reasons and reasons regarding reimbursement, inpatient and outpatient care is strictly separated. There is no automatic and standardized data transfer from inpatient to outpatient care and vice versa. Hopes of attaching the information required for the quality assessment to an electronic health insurance card that each person with statutory health insurance would have to present when visiting a physician, had to be abandoned as technical problems and political controversy regarding data protection could not be resolved. The solution to the problem of follow-up information was to trigger follow-up documentation independently and then later to link the data from the initial procedure to data from the follow-up documentation for each individual patient. On account of this link, a patient’s identity must be kept confidential. Therefore, before the data of the procedure and follow-up appointment can be linked and analysed, it has to be pseudonymized by a centre handling confidential data. Only after that would the data be forwarded to the AQUA Institute for further processing and analysis. Finding a trigger for follow-up documentation poses similar demands such as finding a trigger for the initial documentation in outpatient care. For the follow-up of cataract surgery, no reimbursement code could be identified that would act as a trigger that is sensitive and specific at the same time. The follow-up to cataract-surgery is usually coded by a rather general code that is also used for other follow-up care of eye-related surgery. One possibility would have been to implement a filter question (“Has cataract surgery been performed recently?”) to increase the specificity of this trigger. However, this was felt by the Federal Joint Committee to require too much effort that was believed to be not reasonable for outpatient service providers. In addition, answering a filter question opens possibilities for manipulation towards under-reporting. Therefore, it was decided to trigger follow-up documentation by using only the unspecific reimbursement codes. Trialling the procedure, however, revealed that leaving the follow-up trigger as unspecific as it was, led to documentation being falsely triggered in 52% of cases. This was deemed unacceptable and in the end, plans for having follow-up documentation after cataract surgery were abandoned.
6. Documentation in outpatient care Implementation of QA documentation in outpatient care not only for follow up but already for the surgery itself proved to be a much bigger challenge than in inpatient care. There were several reasons for this: • Personnel: Most outpatient services are provided by single or relatively small provider practices. Resources for additional tasks, such as additional documentation are therefore limited. In comparison, hospitals have extra departments or at least trained personnel who are
1020
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
appointed for quality assessment and documentation management. • IT: Statutory documentation for quality purposes is a somewhat new topic in the outpatient sector. Where it already exists, it is limited to very specific services such as disease management programmes or dialysis. Therefore, often infrastructure is lacking, in particular when it concerns IT. For QA, software is needed that firstly triggers documentation and secondly supports data collection. Furthermore, this software needs to be capable of communicating with the software already implemented in the practices for reimbursement purposes. Implementing new software does not only mean financial investment for the practices, but also requires effort for developing such software. While in the inpatient sector there are about 30 large software companies that develop software solutions for hospitals in Germany, there are more than 150 software suppliers in the outpatient sector, some of them being very small companies. They all have to be considered when seeking implementation of QA documentation in outpatient care. • Experience and tradition: Finally, outpatient physicians are mostly inexperienced with having to provide extra documentation for the purposes of statutory QA. Not only that, there is no tradition of having to do this and therefore finding ophthalmologic practices that would participate in the trail phase for the QA-procedures revealed to be difficult. To overcome resistance towards providing documentation, it has proved helpful if medical associations are in support of the development and implementation of QA. This was the case with the QA procedure on Percutaneous Coronary Intervention (PCI) that was developed later. The National Association of Cardiologists actively supported the trial phase during the development of this QA procedure. Unlike the trial phase for cataract surgery which lacked such support, the trial phase for PCI went much better and the results were promising. Furthermore, many outpatient cardiologists were already participating in data collection for a voluntary registry. Thus, the IT infrastructure, as well as the experience in providing extra documentation, was already in place. 7. Lessons learned The implementation of a QA procedure for cataract surgery was halted mainly because in outpatient care the effort of providing documentation for the surgical procedure and also for the follow-up care was judged to be inacceptable. It was judged in particular that the effort required for providing follow-up documentation was too great. The idea of introducing a filter question to narrow down the amount of documentation being falsely triggered by an unspecific trigger was rejected; similarly, the QA procedure was also rejected on account of the fact that physicians were filling out about 50% of the documentation in vain following an unspecific trigger. Although this rejection of the QA procedure on account of the high rate of falsely triggered documentation is understandable, experience shows that acceptance does not inevitably mean a high level of
efficiency with regard to the documentation: Within the established QA procedure for pressure ulcers, hospitals documented in 2012 up to 1300,000 cases of which only about 10,000 were correct positive cases. Not all hospitals liked this QA procedure on pressure ulcers [11]. However, it worked. A lesson learned from the non-acceptance of the burden of documentation in the QA procedure for cataract surgery was the insight that QA needs to rigorously review its methods to keep the documentation burden as minimal as possible. This is in particular relevant for outpatient care where the documentation infrastructure is much less welldeveloped than in hospitals. Therefore, three additional data sources for statutory QA were introduced: routine data, patient surveys and peer reviews. • Routine data: Routine data usable for QA purposes includes data from the hospitals’ electronic accounting systems and claims data from the health insurance companies. The advantage of this routine data relates to it being available not only without the need for additional effort to obtain the extra documentation, but also the fact that it is available nationally in a similar form. In addition, health insurance claims data is available longitudinally and in outpatient as well as in inpatient care. The disadvantages of routine data relate to it being produced for claims purposes and not for quality measurement. It is often not adequate for mapping all the issues that are relevant for quality [12]. Routine data can often only assess proxies of outcomes and processes, and thereby only partially depict reality. In particular, patient organization representatives are concerned that important quality deficits/potential might be missed if QA is based more and more on routine data and oriented towards “effort-adequate procedures” [13]. Therefore, despite all available routine data and for have meaningful QA procedures in future that scrutinize the relevant processes, QA will not be able to do without additional manual documentation by service providers. In order to be able to use health insurance claims data for the purposes of statutory QA, an amendment of the current legislation was needed. This was undertaken in December 2012. Implementation directives are expected to be adopted in 2014. • Patient survey: Patients’ views are crucial for judging patient relevant outcomes such as pain or functioning in daily life, and they are often the only tie between different health care sectors and services [14,15]. Patients have been defined as clear definers of good quality, good evaluators of the heath care they receive and good rapporteurs of their experiences with health care [16]. The use of patient surveys does not only carry the advantage of providing valuable information from the patient perspective; it also offers the additional benefit of lowering the burden of documentation for service providers. Issues assessed by asking patients usually do not have to be additionally documented by service providers. Many QA procedures that have been commissioned for further development, such as a QA procedure on arthroscopy, now include a patient survey. It should, however, not be forgotten that implementing patient
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022
1021
surveys has its own technical, legal and policy challenges that will be addressed elsewhere. • Peer review: Peer reviews are better able to value the performance of services in terms of their particularity and context than just indicators. E.g. the quality of indication might be better scrutinized by a peer review than by indicators. Unlike indicators, peer reviews are carried out only on a sample of service providers. As a side effect, peer-reviews can contribute to decrease the burden of documentation for service providers. The disadvantage of peer reviews is the organizational effort that they require. The development of a QA procedure using peer reviews has recently been commissioned for arthroscopy. This is work in progress; therefore the results and experiences are pending.
Federal Joint Committee, has just recently adopted a decision reinforcing the implementation of long- and mid-term statutory follow-up assessments [21]. The difficulties involved in data assessment in crosssectoral QA, as well as the sector diversity and the fact that many sectors are simply unprepared for the participation in QA, are more or less technical problems. However, the acceptance of QA and the question as to how much effort stakeholders are willing to make for QA is a political issue. It is related not only to the motivation for transparency, but also to the time and financial burden related to QA documentation; finally, it is also related to the expected results. Even if QA is as efficient as possible, it needs to be acknowledged that meaningful QA will never be available without a certain amount of time and effort being invested.
Despite the burden of documentation and issues of acceptability, the implementation of cross-sectoral QA has been challenged by technical problems arising at the interfaces of the service sectors as described in the paragraphs on triggering cases and on follow-up of cases. This includes problems with the transfer of data and information across sectors as well as problems with the different coding systems and traditions in inpatient and outpatient care These problems mirror the segregation of the German health care system, which cross-sectoral QA at the same time lines up to overcome [4]. In addition, problems arise on account of the diversity of the outpatient sector containing different services and different coding and reimbursement systems and on account of the outpatient sector not being prepared for participating in statutory external QA. Even Avedis Donabedian believed that QA in the outpatient sector was virtually impossible, for exactly these reasons [17]. The experiences of other modern health systems show, however, that some health systems succeed in implementing effective QA by indicators also in outpatient care: The Israeli Health System has implemented statutory QA for community health care. All data is collected from electronic patient records that all primary care physicians use that work for one of the four Health Plans covering the Israeli population [18,19]. Also Sweden reports indicators for outpatient care. These indicators are extracted from registries, in which data is inserted either electronically or by extra documentation [20].
Conflict of interest statement
8. Conclusion Cross-sectoral QA is a compelling concept that considers not only a single moment in the procedure, but the chain of health care services provided across sectors and over the course of time. It needs to be considered that most health care is provided in outpatient care and that in fact, the quality of chronic care is decided longitudinally and in the outpatient sector. For this reason, it is worth taking the challenge and extending the wellelaborated and implemented system of QA in hospital care to the outpatient sector and to cross-sectoral QA. More cross-sectoral QA, in particular, in what concerns followup care after being discharged from hospital is what will certainly be implemented in the near future in the German health care system. The responsible authority, namely the
All authors are employed by the AQUA-Institute and work within the development of cross-sectoral quality assurance within the German statutory quality assurance programme (www.sqg.de) that is financed by the Federal Joint Committee. Other than that no conflicts of interest are present. Acknowledgement We are grateful to Sharon Janicki for revising the English of this article. References [1] Willms G, Bramesfeld A, Pottkämper K, Broge B, Szecsenyi J. Aktuelle Herausforderungen der externen Qualitätssicherung im deutschen Gesundheitswesen. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen 2013;107(8):523–7 (Aktuelle Herausforderungen der externen Qualitatssicherung im deutschen Gesundheitswesen). [2] OECD. Health status Organisation for Economic Co-operation and Development. OECD; 2014 (cited 2014 04.02). Available from http://stats.oecd.org/index.aspx?DataSetCode=HEALTH STAT. [3] Bölt U, Graf T. 20 Jahre Krankenhausstatistik. Wirtschaft und Statistik 2012;2012(2):112–38. [4] Szecsenyi J, Broge B, Eckhardt J, Heller G, Kaufmann-Kolle P, Wensing M. Tearing down walls: opening the border between hospital and ambulatory care for quality improvement in Germany. International Journal for Quality in Health Care 2012;24(2):101–4 [Epub 2012/01/25], (journal of the International Society for Quality in Health Care/ISQua). [5] AQUA. Qualitätsreport 2012. Göttingen: AQUA—Institut für angewandte Qualitätsförderung und Forschung im Gesundheitswesen; 2013. [6] Wenzel M, Pham DT, Scharrer A, Schayan K, Klasen J. Derzeitiger Stand der ambulanten Ophthalmochirurgie: Ergebnisse der Umfrage 2006 des BDOC, BVA und der DGII. Ophtalmo—Chirurgie 2007;19:128–38. [7] Kohnen T, Baumeister M, Kook D, Klaproth OK, Ohrloff C. Kataraktchirurgie mit Implantation einer Kunstlinse. DtschArzteblInt 2009;106(43):695–702. [8] AQUA. Allgemeine Methoden im Rahmen der sektorenübergreifenden Qualitätssicherung im Gesundheitswesen nach §137 a SGB V. Version 3. 0 (Stand: 15. Februar 2013). Göttingen: AQUA—Institut für angewandte Qualitätsförderung und Forschung im Gesundheitswesen; 2013. [9] Fitch K, Bernstein SJ, Aguilar MD, Burnand B, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND; 2001. [10] Albrecht M, Loos S, Otten M. Sektorenübergreifende Qualitätssicherung in der ambulanten Versorgung. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen
1022
[11]
[12]
[13]
[14]
[15]
A. Bramesfeld et al. / Health Policy 119 (2015) 1017–1022 2013;107(8):528–33 [Epub 2013/12/03]. (Sektorenubergreifende Qualitatssicherung in der ambulanten Versorgung). AQUA. German Hospital Quality Report 2012. Göttingen: AQUA—Institute for Applied Quality Improvement and Research in Health Care GmbH; 2014. Busse R, Nimptsch U, Mansky T. Measuring, monitoring, and managing quality in Germany’s hospitals. Health Affairs (Millwood) 2009;28(2):w294–304 [Epub 2009/01/29]. PatientenVertretung im Unterausschuss Qualitätssicherung der G-BA. Qualitätssicherung im Gesundheitswesen: Positionen der Patientenvertretung zum Eckpunktepapier der Träger des G-BA. Berlin: Gemeinsamer Bundesausschuss; 2013. Ludt S, Heiss F, Glassen K, Noest S, Klingenberg A, Ose D, et al. Die Patientenperspektive jenseits ambulant-stationärer Sektorengrenzen—Was ist Patientinnen und Patienten in der sektorenübergreifenden Versorgung wichtig? Gesundheitswesen 2014;76(6):359–65. Blum K, Satzinger W, Buck R. Patientenbefragungen und Qualitätsmanagement. Eine Einführung in die Thematik. In: Satzinger W, Trojan A, Kellermann-Mühlhoff P, editors. Patientenbefragungen in Krankenhäusern. Sankt Augustin: Asgard-Verlag; 2001. p. 25–40.
[16] Donabedian A. The Lichfield Lecture. Quality assurance in health care: consumers’ role. Quality in Health Care: QHC 1992;1(4):247–51. Epub 1992/11/04. [17] Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly 1966;44(3 Suppl.):166–206. [18] Jaffe DH, Shmueli A, Ben-Yehuda A, Paltiel O, Calderon R, Cohen AD, et al. Community healthcare in Israel: quality indicators 2007–2009. Israel Journal of Health Policy Research 2012;1(1):3. Epub 2012/08/24. [19] OECD. OECD reviews of health care quality: Israel. Executive summary, assessment and recommendations. Organisation for Economic Co-operation and Development; 2012. [20] Socialstyrelsen. Quality and efficiency in Swedish health care. Regional comparisons 2012. Stockholm: Swedish Association of Local Authorities and Regions, Swedish National Board of Health and Welfare Publikationsservice; 2013. [21] G-BA. Pressemitteilung Daten zur Behandlungsqualität in Krankenhäusern: G-BA treibt Follow-up-Verfahren voran (Nr. 27/2014 vom 19. Juni 2014). Berlin: Gemeinsamer Bundesausschuss; 2014 (cited 2014 27.06). Available from https://www.g-ba.de/downloads/34215-540/27-2014-06-19-QSKH.pdf.
