Chin J Integr Med
•1•
THINKING AND METHODOLOGY Study on the Methodology of Developing Evidence-Based Clinical Practice Guidelines of Chinese Medicine CHEN Zheng-guang (陈争光)1, LUO Hui (罗 卉)2, XU Shan (徐 YANG Yan (杨 燕)3, and WANG Shou-chuan (汪受传)1
珊)1,
ABSTRACT At present, evidence-based clinical practice guideline (EBCPG) is the main mode of developing clinical practice guidelines (CPGs) in the world, but in China, most of CPGs of Chinese medicine (CM) are still guidelines based on expert consensus. The objective of this study is to construct initially the methodology of developing EBCPGs of CM and to promote the development of standardization of CM. Based on the development of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM", the methodology of developing EBCPG of CM was explored by analyzing the pertinent literature and considering the characteristics of CM. In this study, the key problem was to put forward the suggestion and strategies. However, due to the methodology study of developing EBCPG of CM is still in the initial stage, there are still some problems which need further study. KEYWORDS evidence-based clinical practice guideline, Chinese medicine, Delphi method, methodological study
With the development of social economy and the advances in medical technology, many countries around the world, especially the advanced countries, have been facing the growing complexity of medical environment since the 1980s. A series of measures adjusting and reforming the health care systems were obliged to be taken in many countries in order to control costs reasonably and strengthen the management of the quality of health services. Clinical practice guideline (CPG) is one of the effective measures. Therefore, the development and application of CPG have been on the agenda and developed rapidly in many countries. Especially in the recent 20 years, the evidence-based CPG (EBCPG) was promoted by the development of the evidence-based medicine, the epidemiology and the massive clinical research. In recent years, the medical service and health care system also faced the same problem in China. Increasing emphasis on the development and application of CPG was given by the department of health administration and the academic community. In the field of Chinese medicine (CM), the experts and scholars have also began to study the EBCPG of CM.(1,2) With the development of internationalization and modernization of CM, intensely urgent demand for EBCPG of CM has been shown.(3) CPG of CM not only embodies the characteristics of CM, but also follows the rules of EBCPG. The syndrome differentiation and treatment is one of
the basic characteristics of CM. The syndrome differentiation and treatment implies that the patient's symptoms and signs collected through the four diagnostic methods are analyzed and generalized so as to identify the etiology, nature and location of a disease, and the relation between healthy qi and pathogenic factors, thereby determining the nature of the syndrome. According to the result of syndrome differentiation, the corresponding therapy was selected. In this study, the methodology of integrating the characteristics into EBCPG of CM was explored by analyzing the pertinent literature and the Delphi method. Therefore, the key problems about the EBCPG of CM were discussed by analyzing the domestic and foreign related to achievement of study systematically and considering the characteristics of CM.
METHODS Workflow of Developing the EBCPG of CM The workflow of developing the EBCPG of CM
©The Chinese Journal of Integrated Traditional and Western Medicine Press and Springer-Verlag Berlin Heidelberg 2014 Supported by the Standardization of Chinese Medicine Projects of State Administration of Traditional Chinese Medicine (No. ZYYS-2009 [0004]-1) 1. Pediatric Institution of Nanjing University of Chinese Medicine, Nanjing (210029), China; 2. Department of Traditional Chinese Medicine, Children's Hospital of Zhengzhou, Zhengzhou (450000), China; 3. Department of Traditional Chinese Medicine, Beijing Children's Hospital, Beijing (100045), China Correspondence to: Prof. WANG Shou-chuan, Tel: 86-2586798182, E-mail: [email protected] DOI: 10.1007/s11655-014-1896-1
Chin J Integr Med
•2•
Selecting and identifying topic, determining the purpose and significance of the study Literature review
Designing the first round of the expert questionnaire Statistic analysis of the result of the first round of the expert questionnaire
Extracting relevant information Designing the second round of the expert questionnaire The Delphi method Searching and reviewing the medical evidence, forming the recommendation
Statistic analysis of the result of the second round of the expert questionnaire
Designing the third round of the expert questionnaire Forming the first draft of guideline Consulting the experts about the first draft of guideline
Statistic analysis of the result of the third round of the expert questionnaire
Forming the revised draft of guideline Evaluating guideline by using appraisal tool Forming the final draft of guideline by evaluating and experts' demonstration Publication of guideline
Clinical evaluation Literature review
Questionnaire survey
Updating and revising of guideline Updating the guideline on the basis of the literature review, the outcome of questionnaire survey and the expert demonstration meetings
Figure 1.
Workflow of Developing the EBCPG of CM
is proposed in Figure 1.
Selection Methods on Topic On the basis of the interview with or questionnaire to researchers of CM, clinicians and patients, and literature research, the initial topic selection of the EBCPG of CM was selected. Then the initial topic was discussed at the expert demonstration meetings which adopted the brainstorming.(4) Finally, the topic was formed. During the selection of topic, the advantages of CM and incidence of disease which the topic involved were also considered.
Application of Delphi Method The Delphi method asked the experts to answer questionnaires in several rounds in the anonymous way to consulting the broad experts. Ultimately, expert consensus was formed. In recent years, this method has been widely used in different fields of science.(5,6)
Role and Significance of the Delphi Method in Developing the EBCPG of CM CM is one of the traditional medicines which based on human social practical activities and
experience on protecting human health. The experience from ancient physicians particularly reflected the knowledge accumulation of CM in past dynasties. Therefore, CM is the integration of the physicians' experience of successive dynasties. The physicians' experience of successive dynasties is not only the essence of CM, but also the basis, the core, and the characteristic of the EBCPG of CM. But how the ancient and modern physicians' experience were abstracted and integrated into the EBCPG of CM is always the research hotspot and difficulty of developing the EBCPG of CM. At present, the Delphi method has been one of the most effective ways to abstract experts' experience. Based on the literature study, the Delphi method can not only abstract the modern experts' experience, but also the ancient's. And the method is also the effective way to integrate the ancient and modern doctors' experience into the EBCPG of CM.
Method of Abstracting the Ancient and Modern Doctors' Experience The methods of abstracting the ancient doctors'
Chin J Integr Med
experience were as follows: (1) identifying the topic of the study; (2) selecting the key and index word related to the topic; (3) according to the key and index word of the topic, the therapeutic methods (including CM formulae, Chinese herbal medicine, acupuncture and moxibustion, massage, external treatment, etc.) related to the topic were collected and compiled in the ancient and modern medical literature which can be gathered; and the therapeutic methods form the literature must be checked with the original; (4) the therapeutic methods which were recorded in ancient medical literature, inherited continuously by the doctors in the later dynasties, and were also used and studied extensively by the modern doctors were selected to lay the foundation for the further study, that is, drafting the first round of expert questionnaire. The methods of forming the expert consensus based on the ancient and modern literature were as follows. Firstly, according to the key and index word of the topic, the topic-related modern literatures of CM were also collected and sorted, and the common treatments at present in the literature were analyzed and selected, especially the treatments which were recorded in the ancient literature and continued to use up to now. Secondly, according to fundamental principles of the Delphi method, the questionnaire of the first round based on the study of the ancient and modern literature of CM was designed. By investigating the renowned experts in the field, the expert consensus based on the ancient and modern literature had the initial framework. This process needs three rounds of expert questionnaires. Finally, expert demonstration meetings (2–3) in the field were held to discuss the first draft of the expert consensus based on the ancient and modern literature. Based on the opinions and suggestions of experts, the consensus was revised. The expert consensus based on the ancient and modern literature was formed eventually.
Constructing the Basic Framework of Guideline The development of the EBCPG of CM needs in-depth and systematic literature study to collect and collate the relevant literature information and abstract the topic-related factors. Based on literature study, the questionnaire was made in light of the fundamental principles of the Delphi method. The basic framework of guideline including diagnosis,
•3•
differentiation of syndrome, treatment, care and so on were formed by asking experts to answer questionnaires in two or more rounds.
Forming the CM Characteristics of the Expert Consensus Evidence The EBCPG should be established on the basis of the clinical evidence. The clinical evidence is important guaranty for the scientificalness and practicability of CPG. Meanwhile, recommendations based on the clinical evidences are the major difference between the EBCPG and some other types. Therefore, clinical evidence is also the core of the EBCPG. According to expert consensus in the world, the clinical evidences were derived from the clinical reports, expert consensus and expertise. And the high quality of clinical evidences was often derived from the large sample multi-centre randomized controlled trials (RCTs). However, this kind of trial needs the immense cost in materials, money and manpower, and it is extremely limited. Therefore, this situation led to the fact that the high quality of clinical evidence in the EBCPG was low in number,(7) and the CM clinical study was also facing the similar problems. The expertise had particularly important role and significance in CM. The ancient doctors' experience was the summary of clinical practice in human which the ancient physicians have done in the long time. For historical reasons, the ancient doctors cannot provide the data of the large sample multi-centre RCTs. But the clinical presentations, pathogenesis, therapeutic principles and prescription, and other traditional therapy which the physicians of generations created and continued to use up to now over millennia were obviously the evidences based on mass clinical practice. Therefore, according to the fundamental principles of the Delphi method, the questionnaire was designed based on the study of the ancient and modern literature of CM. By investigating the experts in the field, the Delphi method can form expert consensus based on the ancient and modern literature. This consensus can provide high quality clinical evidence for the EBCPG of CM and enhance the scientificalness and practicability of guideline. So the Delphi method played an important role in the formation of the evidence of EBCPG of CM.
Chin J Integr Med
•4•
Application Mode of the Delphi Method in the EBCPG of CM During the development of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM", (8) we found that the classic mode of Delphi method cannot effectively abstract the expertise. Therefore, the Delphi method was improved. The improved design of the Delphi method has been focused on the developing of the expert questionnaire. The classical mode which was used in the first round expert open questionnaires is "a white paper method".(9) Expert needs more cost in energy and time, and the opinion is always so distributed on the topic that the statistic analysis of the outcome is very difficult, and easily to carry in the subjective factors of statistical analysis personnel. The improved design of the first round expert questionnaire was based on the literature study and used the rating scales mainly. The content of design was broader, but the scope of content was limited moderately to collect the expertise. Meanwhile, the experts were asked to submit the written views and suggestions.
Literature Study and Quality Evaluation System Significance of the Literature Study in EBCPG of CM The literature study is the basis of developing the EBCPG of CM. The research topic could be understood deeply the state of development at present by collecting and collating the topic-related literature systematically in order to identify the value of research topic. Meanwhile, by studying the related literature systematically, the ancient and modern theory, clinical medicine research and experimental research of the research topics can be summarized to collect the clinical evidence of guideline and design the questionnaire.
Content of the Literature Study Literature Retrieval The topic-related literature should be searched by manual search and computer-based retrieval.
Literature Review The literature review should be written after literature retrieval. The literature review includes systematic review and traditional narrative review.
Quality Evaluation of Literature The literature quality evaluation system is not
only the basis of the evidence source and grading, the formation of recommendations, but also the core of EBCPG. At present, the international quality evaluation system of literature is more complex. According to the situation of the topicrelated literature, the international organization of developing guideline developed or modified the common quality evaluation system of literature to suit the characteristic of the literature. In light of the characteristic of CM literature, the Jadad Scale was used in the evaluation of RCT (10,11) according to "Consolidated Standards for Reporting Trials (CONSORT) of Traditional Chinese Medicine".(12) The evaluation of non-RCT was used "the CONSORT statement".(13,14) The quality evaluation of the case report of CM was referenced "the suggested items case report of CM" (15) which was proposed by LIU Jian-ping according to "the international writing standard of the case report".(16) The difficulty of developing the EBCPG of CM is how to evaluate the ancient literature of CM. One viewpoint is that the authority of the ancient doctors should be graded. However, for thousands of ancient doctors with different culture, history and academic viewpoints, it is difficult to reach a consensus about the grading of ancient doctors. Someone deems that according to the citations of the therapeutic regimens which the ancient literature presented, the ancient literature can be graded. But there was not the same baseline between the citation of the literature of the Han Dynasty and the Qing Dynasty. So the ancient literature cannot be compared. Therefore, the evaluation of the ancient literature of CM should pay more attention to the clinical practice, especially whether the therapy is still used and recognized widely in the modern time. The citation rate of the ancient literature should be also considered as well.
Forming of Evidence Grading System The evidence grading system is the core part of the methodology of developing the EBCPG of CM and it is also an important guaranty for the scientificalness and practicability of CPG.(17) The evidence grading system of the EBCPG was formed according to the characteristic of the topic-related clinical evidences. The doctors' experience in successive dynasties is important in CM, and especially the experience which the ancient doctors proposed and continued to use up to now is the quintessence of CM. Therefore,
Chin J Integr Med
•5•
the importance of the ancient and modern doctors' experience should be reflected in the evidence grading system of the EBCPG of CM. By studying the evidence grading system popularizing in the world systemically, the "Delphi methodology for criterion of grading system of evidence"(18) which built by the International Sepsis Forum in 2000 was used according to the basic principles "not trivial, easy to grasp, suitable for guideline-developers who are clinical doctors as the main body to use". Considering the characteristic of CM and the literature of CM, CM edition of the "Delphi methodology for criterion of grading system of evidence" was revised and formed. The Delphi methodology for criterion of grading system of evidence (the revised edition of CM) strengthened the status of the expertise of CM in the EBCPG of CM and proposed specific description and different grade about "the expert consensus based on the ancient and modern literature", "the modern expert consensus" and "the expertise". The expert consensus based on the ancient and modern literature which was recorded in the ancient literature, continued to use up to now and reached the consensus through questionnaires of the contemporary experts can be the graded as Ⅲ. And the distinction is shown between "the expertise" in the grade Ⅳ and Ⅴ. The grading system of evidence which can reflect the characteristics of CM was constructed initially, specified as Table 1.
Evaluation and Revision of the EBCPG of CM Two following in the content of quality evaluation of the EBCPG of CM should be emphasized. One is whether the quality assessment of guideline accorded Table 1.
with academic features of CM and reflected the clinical diagnosis and treatment level of CM nowadays. The other is whether the quality assessment of guideline accorded with the standard of the international development of CPG and can provide clinical evidence and reasonable strength of recommendation for the plan of diagnosis and treatment of the guideline. The second aspect is the key problem for EBCPG of CM to go to the world. The method of evaluation of CPG should include the appraisal of guideline evaluation instrument and the clinical application. Applying the guidelines evaluation instrument to review systematically CPG is the first step of the guideline evaluation. The form and content of CPG is more standard and consistent. It is important to improve the quality, scientificalness and practicability of guideline, and it is also the precondition of the clinical application evaluation. The guidelines evaluation instrument considering the feature of CM should be developed according to the international guidelines evaluation instrument [such as Appraisal of Guidelines for Research and Evaluation (AGREE)(19) instrument]. Before the clinical application evaluation, the guideline should be evaluated by the instrument. The clinical application evaluation should be carried out in the representative pilot hospitals including the third class hospitals, the second class hospitals and the first class hospitals, even community in the urban areas and primary health institution in suburb. Only in this way can the general applicability of the guideline be analyzed roundly. The norm applied evaluation form was made up before the evaluation. The users of the guideline were trained in order to strengthen the knowledge of CPG to learn, help the users deeply understand the guideline and facilitate the
The Delphi Methodology for Criterion of Grading System of Evidence and Its Revised Edition of CM Grading criterion
Criterion of CM revised edition
Ⅰ Large, randomized trials with clear-cut results; low risk of false-positive (alpha) error of false-negative (beta) error
Ⅰ Large, randomized trials with clear-cut results; low risk of falsepositive (alpha) error of false-negative (beta) error
Ⅱ Small, randomized trials with uncertain results; moderate-to-high risk of false-positive (alpha) error and/or false-negative (beta) error
Ⅱ Small, randomized trials with uncertain results; moderate-to-high risk of false-positive (alpha) error and/or false-negative (beta) error
Ⅲ Nonrandomized, contemporaneous controls
Ⅲ Nonrandomized, contemporaneous controls, expert consensus based on the ancient and modern literaturea
Ⅳ Nonrandomized, historical controls and expert opinion
Ⅳ Nonrandomized, historical controls and modern CM expert consensusb
Ⅴ Case series, uncontrolled studies, and expert opinion
Ⅴ Case series, uncontrolled studies, and expert opinionc
Notes: a The expert consensus based on the ancient and modern literature in the grade Ⅲ is recorded in the ancient literature, continues to use up to now and reaches the consensus through questionnaires of the contemporary experts. b The modern CM expert consensus in the grade Ⅳ reaches the consensus through questionnaires of the contemporary experts. c The expert opinion in the grade Ⅴ refers to individual experts' opinion
Chin J Integr Med
•6•
clinical evaluation. The deadline for the contract is due in 6 months in the world. Otherwise, it will affect the promotion of guideline and the update cycle. According to the feedback of the appraisal of guideline evaluation instrument and the clinical application evaluation, the problems about the guideline were analyzed detailedly and demonstrated by experts to revise the guideline. Then, the guideline was published and implemented.
Update of the EBCPG of CM The EBCPG which based on the clinical evidence should be updated since the clinical evidence is constantly changing. The literature study, the questionnaire and the experts' demonstration are the main means of the update of CPG. The high quality evidences had been chosen by the literature study method which needed to search the topic-related literature in the last 2 to 3 years. The questionnaire aimed at the users of guideline, the clinical doctors and patients to research the questionnaires' practicability, intelligibility and problems in clinical application, and to summarize the problems finally. In the light of the outcome of the literature study and questionnaire, the problems of the guideline were discussed at the experts' demonstration meeting to update the guideline. At present, the update cycle of guideline is 2 to 3 years in the world, such as 2 years is the cycle of Global Initiative for Asthma report. Therefore, 2 to 3 years should also be the update cycle of the EBCPG of CM.
questions were applied in their first round and the questionnaire based on the "literature". And in the questionnaire of all the rounds, the blank was left for experts to add their suggestions. The suggestion was paid more attention and added to the questionnaire of the subsequent round. The consensus of opinion of the expert panel was gained in the end. Now, the methodology study of developing the EBCPG of CM is still in the initial stage. Based on the "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM",(8) the key problem in the methodology study of developing the EBCPG of CM was to put forward the suggestion and strategies. The technique of developing the EBCPG of CM was initially constructed by analyzing the pertinent literature and considering the characteristics of CM. However, there are still some problems which need further study. These problems should be discussed in academia to promote the development of the EBCPG of CM.
Conflict of Interest The authors declare no financial or commercial conflict of interests.
Author Contributions Chen ZG and Wang SC designed the research and wrote the manuscript. All authors participated in the performance of the research.
REFERENCES 1.
Xie YM, Yuwen Y, Dong FH, Sun SC, Wang HM, Liu QS, et al. Clinical practice guideline of traditional medicine for
Discussion The first round questionnaire of the classic Delphi survey is usually unstructured and seeks an open response. Open-ended questions are recognized to increase the richness of the data collected.(9) In the initial stage of the program of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM",(8) the classic Delphi method was applied and designing of the questionnaire was followed strictly the rules of the Delphi. Subsequently, we found that expert needs more cost in energy and time and feels fatigue, and the opinion is always so distributed on the topic that the statistic analysis of the outcome is very difficult, and easily to carry in the subjective factors of statistical analysis personnel. In the face of this situation, based on the analysis of the Delphi literature (20) and the discussion of our research group, semi-structured
primary osteoprosis. Chin J Integr Med 2011;17:52-63. 2.
Yuwen Y, Shi NN, Wang LY, Xie YM, Han XJ, Lu AP. Development of clinical practice guidelines in 11 commmon diseases with Chinese medicine interventions in China. Chin J Integr Med 2012;18:112-119.
3.
Liu JP, ed. The clinical research methods of the evidencebased Chinese medicine. Beijing: People's Medical Publishing House;2009:310-322.
4.
Zheng XL, Feng QS, Jia R, Fan XJ, Qiao WJ, Gou YL, et al. The initial exploration of the information collection scale of Chinese medicine syndrome of the chronic B-related hepatitis based on the brainstorming method and Delphi method. Asia Pacific Tradit Med (Chin) 2010;6:132-133.
5.
Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008-1015.
6.
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development
Chin J Integr Med
•7•
methods, and their use in clinical guideline development.
Statement: revised recommendations for improving the
Health Technol Assess 1998;2:1-88.
quality of reports of parallel group randomized trials. BMC
7. British Medical Journal Publishing Group. Clinical evidence.
Med Res Methodol 2001;1:2.
Beijing: Peking University Medical Press;2008:Introduction.
15. Yang H, Fei YT, Liu JP. The reporting methods of the
8. Wang SC, Yu S, Zhao X, Dai QG, Chen ZG, Han XM, et al.
clinical case record of Chinese medicine and the expertise—
Guideline for diagnosis and treatment of common pediatric
the design of the case reports. J Tradit Chin Med (Chin)
diseases in Chinese medicine. Beijing: China Press of
2008;49:215-217.
Traditional Chinese Medicine;2012:Ⅲ-Ⅳ. 9. Powell C. The Delphi technique: myths and realities. J Adv Nurs 2003;41:376-382.
16. Green BN, Johnson CD. Writing patient case reports for peer-reviewed journals: secrets of the trade. J Sports Chiropr Rehabilit 2000;14:51-59.
10. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds
17. Grades of Recommendation, Assessment, Development,
DJM, Gavaghan DJ, et al. Assessing the quality of reports
and Evaluation (GRADE) Working Group. Grading quality
of randomized clinical trials: is blinding necessary? Control
of evidence and strength of recommendations. BMJ
Clin Trials 1996;17:1-12.
2004;328:1490-1494.
11. Clark HD, Wells GA, Huet C, McAlister FA, Salmi LR,
18. Dellinger RP, Carlet JM, Masur H, Gerlach H, Calandra T,
Fergusson D, et al. Assessing the quality of randomized
Cohen J, et al. Surviving sepsis campaign guidelines for
trials: raliability of the Jadad Scale. Control Clin Trials
management of severe sepsis and septic shock. Crit Care
1999;20:448-452. 12. Wu TX, Li YP, Bian ZX, Li TQ, Li J, Dagenais S, et al. Consolidated standards for reporting trials of traditional Chinese medicine (CONSORT for TCM). Chin J Evid Based Med (Chin) 2007;7:601-605. 13. STROBE statement. Available at http://www.strobestatement.org/Checklist.html. 14. Moher D, Schulz KF, Altman DG. The CONSORT
Med 2004;32:858-873. 19. Appraisal of Guidelines for Research and Evaluation Ⅱ. Available at http://www.agreetrust.org/resource-centre/agree-ii/ 20. Duffield C. The Delphi technique: a comparison of results obtained using two expert panels. Int J Nurs Stud 1993;30:227-237. (Received February 20, 2013) Edited by YU Ming-zhu
•1•
THINKING AND METHODOLOGY Study on the Methodology of Developing Evidence-Based Clinical Practice Guidelines of Chinese Medicine CHEN Zheng-guang (陈争光)1, LUO Hui (罗 卉)2, XU Shan (徐 YANG Yan (杨 燕)3, and WANG Shou-chuan (汪受传)1
珊)1,
ABSTRACT At present, evidence-based clinical practice guideline (EBCPG) is the main mode of developing clinical practice guidelines (CPGs) in the world, but in China, most of CPGs of Chinese medicine (CM) are still guidelines based on expert consensus. The objective of this study is to construct initially the methodology of developing EBCPGs of CM and to promote the development of standardization of CM. Based on the development of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM", the methodology of developing EBCPG of CM was explored by analyzing the pertinent literature and considering the characteristics of CM. In this study, the key problem was to put forward the suggestion and strategies. However, due to the methodology study of developing EBCPG of CM is still in the initial stage, there are still some problems which need further study. KEYWORDS evidence-based clinical practice guideline, Chinese medicine, Delphi method, methodological study
With the development of social economy and the advances in medical technology, many countries around the world, especially the advanced countries, have been facing the growing complexity of medical environment since the 1980s. A series of measures adjusting and reforming the health care systems were obliged to be taken in many countries in order to control costs reasonably and strengthen the management of the quality of health services. Clinical practice guideline (CPG) is one of the effective measures. Therefore, the development and application of CPG have been on the agenda and developed rapidly in many countries. Especially in the recent 20 years, the evidence-based CPG (EBCPG) was promoted by the development of the evidence-based medicine, the epidemiology and the massive clinical research. In recent years, the medical service and health care system also faced the same problem in China. Increasing emphasis on the development and application of CPG was given by the department of health administration and the academic community. In the field of Chinese medicine (CM), the experts and scholars have also began to study the EBCPG of CM.(1,2) With the development of internationalization and modernization of CM, intensely urgent demand for EBCPG of CM has been shown.(3) CPG of CM not only embodies the characteristics of CM, but also follows the rules of EBCPG. The syndrome differentiation and treatment is one of
the basic characteristics of CM. The syndrome differentiation and treatment implies that the patient's symptoms and signs collected through the four diagnostic methods are analyzed and generalized so as to identify the etiology, nature and location of a disease, and the relation between healthy qi and pathogenic factors, thereby determining the nature of the syndrome. According to the result of syndrome differentiation, the corresponding therapy was selected. In this study, the methodology of integrating the characteristics into EBCPG of CM was explored by analyzing the pertinent literature and the Delphi method. Therefore, the key problems about the EBCPG of CM were discussed by analyzing the domestic and foreign related to achievement of study systematically and considering the characteristics of CM.
METHODS Workflow of Developing the EBCPG of CM The workflow of developing the EBCPG of CM
©The Chinese Journal of Integrated Traditional and Western Medicine Press and Springer-Verlag Berlin Heidelberg 2014 Supported by the Standardization of Chinese Medicine Projects of State Administration of Traditional Chinese Medicine (No. ZYYS-2009 [0004]-1) 1. Pediatric Institution of Nanjing University of Chinese Medicine, Nanjing (210029), China; 2. Department of Traditional Chinese Medicine, Children's Hospital of Zhengzhou, Zhengzhou (450000), China; 3. Department of Traditional Chinese Medicine, Beijing Children's Hospital, Beijing (100045), China Correspondence to: Prof. WANG Shou-chuan, Tel: 86-2586798182, E-mail: [email protected] DOI: 10.1007/s11655-014-1896-1
Chin J Integr Med
•2•
Selecting and identifying topic, determining the purpose and significance of the study Literature review
Designing the first round of the expert questionnaire Statistic analysis of the result of the first round of the expert questionnaire
Extracting relevant information Designing the second round of the expert questionnaire The Delphi method Searching and reviewing the medical evidence, forming the recommendation
Statistic analysis of the result of the second round of the expert questionnaire
Designing the third round of the expert questionnaire Forming the first draft of guideline Consulting the experts about the first draft of guideline
Statistic analysis of the result of the third round of the expert questionnaire
Forming the revised draft of guideline Evaluating guideline by using appraisal tool Forming the final draft of guideline by evaluating and experts' demonstration Publication of guideline
Clinical evaluation Literature review
Questionnaire survey
Updating and revising of guideline Updating the guideline on the basis of the literature review, the outcome of questionnaire survey and the expert demonstration meetings
Figure 1.
Workflow of Developing the EBCPG of CM
is proposed in Figure 1.
Selection Methods on Topic On the basis of the interview with or questionnaire to researchers of CM, clinicians and patients, and literature research, the initial topic selection of the EBCPG of CM was selected. Then the initial topic was discussed at the expert demonstration meetings which adopted the brainstorming.(4) Finally, the topic was formed. During the selection of topic, the advantages of CM and incidence of disease which the topic involved were also considered.
Application of Delphi Method The Delphi method asked the experts to answer questionnaires in several rounds in the anonymous way to consulting the broad experts. Ultimately, expert consensus was formed. In recent years, this method has been widely used in different fields of science.(5,6)
Role and Significance of the Delphi Method in Developing the EBCPG of CM CM is one of the traditional medicines which based on human social practical activities and
experience on protecting human health. The experience from ancient physicians particularly reflected the knowledge accumulation of CM in past dynasties. Therefore, CM is the integration of the physicians' experience of successive dynasties. The physicians' experience of successive dynasties is not only the essence of CM, but also the basis, the core, and the characteristic of the EBCPG of CM. But how the ancient and modern physicians' experience were abstracted and integrated into the EBCPG of CM is always the research hotspot and difficulty of developing the EBCPG of CM. At present, the Delphi method has been one of the most effective ways to abstract experts' experience. Based on the literature study, the Delphi method can not only abstract the modern experts' experience, but also the ancient's. And the method is also the effective way to integrate the ancient and modern doctors' experience into the EBCPG of CM.
Method of Abstracting the Ancient and Modern Doctors' Experience The methods of abstracting the ancient doctors'
Chin J Integr Med
experience were as follows: (1) identifying the topic of the study; (2) selecting the key and index word related to the topic; (3) according to the key and index word of the topic, the therapeutic methods (including CM formulae, Chinese herbal medicine, acupuncture and moxibustion, massage, external treatment, etc.) related to the topic were collected and compiled in the ancient and modern medical literature which can be gathered; and the therapeutic methods form the literature must be checked with the original; (4) the therapeutic methods which were recorded in ancient medical literature, inherited continuously by the doctors in the later dynasties, and were also used and studied extensively by the modern doctors were selected to lay the foundation for the further study, that is, drafting the first round of expert questionnaire. The methods of forming the expert consensus based on the ancient and modern literature were as follows. Firstly, according to the key and index word of the topic, the topic-related modern literatures of CM were also collected and sorted, and the common treatments at present in the literature were analyzed and selected, especially the treatments which were recorded in the ancient literature and continued to use up to now. Secondly, according to fundamental principles of the Delphi method, the questionnaire of the first round based on the study of the ancient and modern literature of CM was designed. By investigating the renowned experts in the field, the expert consensus based on the ancient and modern literature had the initial framework. This process needs three rounds of expert questionnaires. Finally, expert demonstration meetings (2–3) in the field were held to discuss the first draft of the expert consensus based on the ancient and modern literature. Based on the opinions and suggestions of experts, the consensus was revised. The expert consensus based on the ancient and modern literature was formed eventually.
Constructing the Basic Framework of Guideline The development of the EBCPG of CM needs in-depth and systematic literature study to collect and collate the relevant literature information and abstract the topic-related factors. Based on literature study, the questionnaire was made in light of the fundamental principles of the Delphi method. The basic framework of guideline including diagnosis,
•3•
differentiation of syndrome, treatment, care and so on were formed by asking experts to answer questionnaires in two or more rounds.
Forming the CM Characteristics of the Expert Consensus Evidence The EBCPG should be established on the basis of the clinical evidence. The clinical evidence is important guaranty for the scientificalness and practicability of CPG. Meanwhile, recommendations based on the clinical evidences are the major difference between the EBCPG and some other types. Therefore, clinical evidence is also the core of the EBCPG. According to expert consensus in the world, the clinical evidences were derived from the clinical reports, expert consensus and expertise. And the high quality of clinical evidences was often derived from the large sample multi-centre randomized controlled trials (RCTs). However, this kind of trial needs the immense cost in materials, money and manpower, and it is extremely limited. Therefore, this situation led to the fact that the high quality of clinical evidence in the EBCPG was low in number,(7) and the CM clinical study was also facing the similar problems. The expertise had particularly important role and significance in CM. The ancient doctors' experience was the summary of clinical practice in human which the ancient physicians have done in the long time. For historical reasons, the ancient doctors cannot provide the data of the large sample multi-centre RCTs. But the clinical presentations, pathogenesis, therapeutic principles and prescription, and other traditional therapy which the physicians of generations created and continued to use up to now over millennia were obviously the evidences based on mass clinical practice. Therefore, according to the fundamental principles of the Delphi method, the questionnaire was designed based on the study of the ancient and modern literature of CM. By investigating the experts in the field, the Delphi method can form expert consensus based on the ancient and modern literature. This consensus can provide high quality clinical evidence for the EBCPG of CM and enhance the scientificalness and practicability of guideline. So the Delphi method played an important role in the formation of the evidence of EBCPG of CM.
Chin J Integr Med
•4•
Application Mode of the Delphi Method in the EBCPG of CM During the development of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM", (8) we found that the classic mode of Delphi method cannot effectively abstract the expertise. Therefore, the Delphi method was improved. The improved design of the Delphi method has been focused on the developing of the expert questionnaire. The classical mode which was used in the first round expert open questionnaires is "a white paper method".(9) Expert needs more cost in energy and time, and the opinion is always so distributed on the topic that the statistic analysis of the outcome is very difficult, and easily to carry in the subjective factors of statistical analysis personnel. The improved design of the first round expert questionnaire was based on the literature study and used the rating scales mainly. The content of design was broader, but the scope of content was limited moderately to collect the expertise. Meanwhile, the experts were asked to submit the written views and suggestions.
Literature Study and Quality Evaluation System Significance of the Literature Study in EBCPG of CM The literature study is the basis of developing the EBCPG of CM. The research topic could be understood deeply the state of development at present by collecting and collating the topic-related literature systematically in order to identify the value of research topic. Meanwhile, by studying the related literature systematically, the ancient and modern theory, clinical medicine research and experimental research of the research topics can be summarized to collect the clinical evidence of guideline and design the questionnaire.
Content of the Literature Study Literature Retrieval The topic-related literature should be searched by manual search and computer-based retrieval.
Literature Review The literature review should be written after literature retrieval. The literature review includes systematic review and traditional narrative review.
Quality Evaluation of Literature The literature quality evaluation system is not
only the basis of the evidence source and grading, the formation of recommendations, but also the core of EBCPG. At present, the international quality evaluation system of literature is more complex. According to the situation of the topicrelated literature, the international organization of developing guideline developed or modified the common quality evaluation system of literature to suit the characteristic of the literature. In light of the characteristic of CM literature, the Jadad Scale was used in the evaluation of RCT (10,11) according to "Consolidated Standards for Reporting Trials (CONSORT) of Traditional Chinese Medicine".(12) The evaluation of non-RCT was used "the CONSORT statement".(13,14) The quality evaluation of the case report of CM was referenced "the suggested items case report of CM" (15) which was proposed by LIU Jian-ping according to "the international writing standard of the case report".(16) The difficulty of developing the EBCPG of CM is how to evaluate the ancient literature of CM. One viewpoint is that the authority of the ancient doctors should be graded. However, for thousands of ancient doctors with different culture, history and academic viewpoints, it is difficult to reach a consensus about the grading of ancient doctors. Someone deems that according to the citations of the therapeutic regimens which the ancient literature presented, the ancient literature can be graded. But there was not the same baseline between the citation of the literature of the Han Dynasty and the Qing Dynasty. So the ancient literature cannot be compared. Therefore, the evaluation of the ancient literature of CM should pay more attention to the clinical practice, especially whether the therapy is still used and recognized widely in the modern time. The citation rate of the ancient literature should be also considered as well.
Forming of Evidence Grading System The evidence grading system is the core part of the methodology of developing the EBCPG of CM and it is also an important guaranty for the scientificalness and practicability of CPG.(17) The evidence grading system of the EBCPG was formed according to the characteristic of the topic-related clinical evidences. The doctors' experience in successive dynasties is important in CM, and especially the experience which the ancient doctors proposed and continued to use up to now is the quintessence of CM. Therefore,
Chin J Integr Med
•5•
the importance of the ancient and modern doctors' experience should be reflected in the evidence grading system of the EBCPG of CM. By studying the evidence grading system popularizing in the world systemically, the "Delphi methodology for criterion of grading system of evidence"(18) which built by the International Sepsis Forum in 2000 was used according to the basic principles "not trivial, easy to grasp, suitable for guideline-developers who are clinical doctors as the main body to use". Considering the characteristic of CM and the literature of CM, CM edition of the "Delphi methodology for criterion of grading system of evidence" was revised and formed. The Delphi methodology for criterion of grading system of evidence (the revised edition of CM) strengthened the status of the expertise of CM in the EBCPG of CM and proposed specific description and different grade about "the expert consensus based on the ancient and modern literature", "the modern expert consensus" and "the expertise". The expert consensus based on the ancient and modern literature which was recorded in the ancient literature, continued to use up to now and reached the consensus through questionnaires of the contemporary experts can be the graded as Ⅲ. And the distinction is shown between "the expertise" in the grade Ⅳ and Ⅴ. The grading system of evidence which can reflect the characteristics of CM was constructed initially, specified as Table 1.
Evaluation and Revision of the EBCPG of CM Two following in the content of quality evaluation of the EBCPG of CM should be emphasized. One is whether the quality assessment of guideline accorded Table 1.
with academic features of CM and reflected the clinical diagnosis and treatment level of CM nowadays. The other is whether the quality assessment of guideline accorded with the standard of the international development of CPG and can provide clinical evidence and reasonable strength of recommendation for the plan of diagnosis and treatment of the guideline. The second aspect is the key problem for EBCPG of CM to go to the world. The method of evaluation of CPG should include the appraisal of guideline evaluation instrument and the clinical application. Applying the guidelines evaluation instrument to review systematically CPG is the first step of the guideline evaluation. The form and content of CPG is more standard and consistent. It is important to improve the quality, scientificalness and practicability of guideline, and it is also the precondition of the clinical application evaluation. The guidelines evaluation instrument considering the feature of CM should be developed according to the international guidelines evaluation instrument [such as Appraisal of Guidelines for Research and Evaluation (AGREE)(19) instrument]. Before the clinical application evaluation, the guideline should be evaluated by the instrument. The clinical application evaluation should be carried out in the representative pilot hospitals including the third class hospitals, the second class hospitals and the first class hospitals, even community in the urban areas and primary health institution in suburb. Only in this way can the general applicability of the guideline be analyzed roundly. The norm applied evaluation form was made up before the evaluation. The users of the guideline were trained in order to strengthen the knowledge of CPG to learn, help the users deeply understand the guideline and facilitate the
The Delphi Methodology for Criterion of Grading System of Evidence and Its Revised Edition of CM Grading criterion
Criterion of CM revised edition
Ⅰ Large, randomized trials with clear-cut results; low risk of false-positive (alpha) error of false-negative (beta) error
Ⅰ Large, randomized trials with clear-cut results; low risk of falsepositive (alpha) error of false-negative (beta) error
Ⅱ Small, randomized trials with uncertain results; moderate-to-high risk of false-positive (alpha) error and/or false-negative (beta) error
Ⅱ Small, randomized trials with uncertain results; moderate-to-high risk of false-positive (alpha) error and/or false-negative (beta) error
Ⅲ Nonrandomized, contemporaneous controls
Ⅲ Nonrandomized, contemporaneous controls, expert consensus based on the ancient and modern literaturea
Ⅳ Nonrandomized, historical controls and expert opinion
Ⅳ Nonrandomized, historical controls and modern CM expert consensusb
Ⅴ Case series, uncontrolled studies, and expert opinion
Ⅴ Case series, uncontrolled studies, and expert opinionc
Notes: a The expert consensus based on the ancient and modern literature in the grade Ⅲ is recorded in the ancient literature, continues to use up to now and reaches the consensus through questionnaires of the contemporary experts. b The modern CM expert consensus in the grade Ⅳ reaches the consensus through questionnaires of the contemporary experts. c The expert opinion in the grade Ⅴ refers to individual experts' opinion
Chin J Integr Med
•6•
clinical evaluation. The deadline for the contract is due in 6 months in the world. Otherwise, it will affect the promotion of guideline and the update cycle. According to the feedback of the appraisal of guideline evaluation instrument and the clinical application evaluation, the problems about the guideline were analyzed detailedly and demonstrated by experts to revise the guideline. Then, the guideline was published and implemented.
Update of the EBCPG of CM The EBCPG which based on the clinical evidence should be updated since the clinical evidence is constantly changing. The literature study, the questionnaire and the experts' demonstration are the main means of the update of CPG. The high quality evidences had been chosen by the literature study method which needed to search the topic-related literature in the last 2 to 3 years. The questionnaire aimed at the users of guideline, the clinical doctors and patients to research the questionnaires' practicability, intelligibility and problems in clinical application, and to summarize the problems finally. In the light of the outcome of the literature study and questionnaire, the problems of the guideline were discussed at the experts' demonstration meeting to update the guideline. At present, the update cycle of guideline is 2 to 3 years in the world, such as 2 years is the cycle of Global Initiative for Asthma report. Therefore, 2 to 3 years should also be the update cycle of the EBCPG of CM.
questions were applied in their first round and the questionnaire based on the "literature". And in the questionnaire of all the rounds, the blank was left for experts to add their suggestions. The suggestion was paid more attention and added to the questionnaire of the subsequent round. The consensus of opinion of the expert panel was gained in the end. Now, the methodology study of developing the EBCPG of CM is still in the initial stage. Based on the "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM",(8) the key problem in the methodology study of developing the EBCPG of CM was to put forward the suggestion and strategies. The technique of developing the EBCPG of CM was initially constructed by analyzing the pertinent literature and considering the characteristics of CM. However, there are still some problems which need further study. These problems should be discussed in academia to promote the development of the EBCPG of CM.
Conflict of Interest The authors declare no financial or commercial conflict of interests.
Author Contributions Chen ZG and Wang SC designed the research and wrote the manuscript. All authors participated in the performance of the research.
REFERENCES 1.
Xie YM, Yuwen Y, Dong FH, Sun SC, Wang HM, Liu QS, et al. Clinical practice guideline of traditional medicine for
Discussion The first round questionnaire of the classic Delphi survey is usually unstructured and seeks an open response. Open-ended questions are recognized to increase the richness of the data collected.(9) In the initial stage of the program of "Guideline for Diagnosis and Treatment of Common Pediatric Diseases in CM",(8) the classic Delphi method was applied and designing of the questionnaire was followed strictly the rules of the Delphi. Subsequently, we found that expert needs more cost in energy and time and feels fatigue, and the opinion is always so distributed on the topic that the statistic analysis of the outcome is very difficult, and easily to carry in the subjective factors of statistical analysis personnel. In the face of this situation, based on the analysis of the Delphi literature (20) and the discussion of our research group, semi-structured
primary osteoprosis. Chin J Integr Med 2011;17:52-63. 2.
Yuwen Y, Shi NN, Wang LY, Xie YM, Han XJ, Lu AP. Development of clinical practice guidelines in 11 commmon diseases with Chinese medicine interventions in China. Chin J Integr Med 2012;18:112-119.
3.
Liu JP, ed. The clinical research methods of the evidencebased Chinese medicine. Beijing: People's Medical Publishing House;2009:310-322.
4.
Zheng XL, Feng QS, Jia R, Fan XJ, Qiao WJ, Gou YL, et al. The initial exploration of the information collection scale of Chinese medicine syndrome of the chronic B-related hepatitis based on the brainstorming method and Delphi method. Asia Pacific Tradit Med (Chin) 2010;6:132-133.
5.
Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008-1015.
6.
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development
Chin J Integr Med
•7•
methods, and their use in clinical guideline development.
Statement: revised recommendations for improving the
Health Technol Assess 1998;2:1-88.
quality of reports of parallel group randomized trials. BMC
7. British Medical Journal Publishing Group. Clinical evidence.
Med Res Methodol 2001;1:2.
Beijing: Peking University Medical Press;2008:Introduction.
15. Yang H, Fei YT, Liu JP. The reporting methods of the
8. Wang SC, Yu S, Zhao X, Dai QG, Chen ZG, Han XM, et al.
clinical case record of Chinese medicine and the expertise—
Guideline for diagnosis and treatment of common pediatric
the design of the case reports. J Tradit Chin Med (Chin)
diseases in Chinese medicine. Beijing: China Press of
2008;49:215-217.
Traditional Chinese Medicine;2012:Ⅲ-Ⅳ. 9. Powell C. The Delphi technique: myths and realities. J Adv Nurs 2003;41:376-382.
16. Green BN, Johnson CD. Writing patient case reports for peer-reviewed journals: secrets of the trade. J Sports Chiropr Rehabilit 2000;14:51-59.
10. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds
17. Grades of Recommendation, Assessment, Development,
DJM, Gavaghan DJ, et al. Assessing the quality of reports
and Evaluation (GRADE) Working Group. Grading quality
of randomized clinical trials: is blinding necessary? Control
of evidence and strength of recommendations. BMJ
Clin Trials 1996;17:1-12.
2004;328:1490-1494.
11. Clark HD, Wells GA, Huet C, McAlister FA, Salmi LR,
18. Dellinger RP, Carlet JM, Masur H, Gerlach H, Calandra T,
Fergusson D, et al. Assessing the quality of randomized
Cohen J, et al. Surviving sepsis campaign guidelines for
trials: raliability of the Jadad Scale. Control Clin Trials
management of severe sepsis and septic shock. Crit Care
1999;20:448-452. 12. Wu TX, Li YP, Bian ZX, Li TQ, Li J, Dagenais S, et al. Consolidated standards for reporting trials of traditional Chinese medicine (CONSORT for TCM). Chin J Evid Based Med (Chin) 2007;7:601-605. 13. STROBE statement. Available at http://www.strobestatement.org/Checklist.html. 14. Moher D, Schulz KF, Altman DG. The CONSORT
Med 2004;32:858-873. 19. Appraisal of Guidelines for Research and Evaluation Ⅱ. Available at http://www.agreetrust.org/resource-centre/agree-ii/ 20. Duffield C. The Delphi technique: a comparison of results obtained using two expert panels. Int J Nurs Stud 1993;30:227-237. (Received February 20, 2013) Edited by YU Ming-zhu
