Opinion
EDITORIAL
Scoring Systems for Blistering Diseases in Practice Why Bother, and Which One Should You Use? Dédée F. Murrell, MA, BMBCh, MD, FACD; Masayuki Amagai, MD; Victoria P. Werth, MD
In most countries, pemphigus is one of those rare, potentially life-threatening skin and mucosal diseases that is referred to specialist centers for treatment. Because it is rare, there is a paucity of randomized clinical trials (RCTs) to inRelated article page 266 form dermatologists of the most effective and safe treatments.1 For rare diseases, metaanalysis, a statistical tool that allows pooling together the results of smaller, similar clinical trials, can be just as valuable as larger clinical trials, but in order to pool the data the studies need to use the same outcome measures for disease assessment.
History After the first pemphigus conference held at the National Institutes of Health in 2005, the international blistering disease community began to work together to establish outcome measures for each autoimmune blistering disease (AIBD) with championing of the concept by Lowell Goldsmith and JeanClaude Bystryn.2 Two of us (D.F.M.and V.P.W.) volunteered to lead this effort, and about 20 international dermatologists stated that they were interested in participating. The first meeting to develop an outcome measure took place in September 2005 in Paris, France, organized with the help of Pascal Joly. The international pemphigus group held several such meetings and adapted the process used to develop the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) for pemphigus, piloted different ways of scoring pemphigus in clinics, and eventually agreed on the Pemphigus Disease Area Index (PDAI). During the same period, the German blistering disease group, some of whom were also in the international group, published a proposed score, termed the Autoimmune Bullous Skin Disorder Intensity Score (ABSIS).3
Validated Scoring Systems Hence, because both of these scores needed to be validated, an initial study (designed by D.F.M. and V.P.W.) was conducted at the University of Pennsylvania, with 15 patients with pemphigus scored by 10 dermatologists who are pemphigus experts (mostly from the United States, for convenience) with the ABSIS, PDAI, and physician global assessment (PGA) scores. This study found that the PDAI had the highest interrater and intrarater intraclass correlation coefficients (ICCs).4 One limitation of the study was that most of the patients had relatively stable disease with lower percentages of skin extent.5 In most countries, it would be difficult to draw together enough patients with severe cases of this rare disease to rectify this limijamadermatology.com
tation. There has been some debate about which score would be best to use in international pemphigus trials, with some of the Europeans preferring the ABSIS and others preferring the international PDAI measure.6,7 Additional measures to improve validation of the PDAI are demonstration of sensitivity to change, accuracy, feasibility, and external validity and reproducibility in other centers.
External Validation From Greece and Japan Since then, just in the past few months, there have been 2 studies published that fill in missing pieces of the validation puzzle. Kamiya et al,8 from Japan, investigated whether pemphigus disease activity correlated with desmoglein (Dsg) 3 titers in 123 serum samples from 19 patients with pemphigus vulgaris (PV) over time and found that the titers of the conformational Dsg3 epitopes correlated with the PDAI score. This speaks to the issues of sensitivity, accuracy, and external validity. Another independent study, conducted in Greece by Patsatsi et al,9 studied 35 consecutive patients with PV and pemphigus foliaceus with PDAI and ABSIS scores at baseline, 6 months, and 12 months to assess disease correlation with the anti-Dsg1 and anti-Dsg3 enzyme-linked immunosorbent assay (ELISA) titers (MBL International Corp). They found statistically significant associations between the total PDAI and ABSIS scores and Dsg1 titers in the patients with cutaneous involvement, but for patients who had only mucosal involvement, the mucosal extent ABSIS and PDAI scores showed only moderate correlation with Dsg3 ELISA at baseline and 6 months. This study provides further validation of the scores in terms of sensitivity to change in clinical disease, accuracy, and external validity. Both these studies address reproducibility in additional centers.
Iranian Validation Study In this issue of JAMA Dermatology, Rahbar et al10 report a third independent study that provides additional validation of the PDAI and ABSIS. This group, who were not part of the International Pemphigus group, is from a dermatology-only teaching hospital (Razi Hospital, Tehran University of Medical Sciences, Tehran, Iran) and is led by Cheyda Chams Davatchi, who trained as a dermatologist at St Louis Hospital in Paris, and has an excellent track record in conducting 2 of the world’s 13 RCTs in pemphigus.1,11,12 Rahbar et al10 developed and validated another extent of disease score, termed the Pemphigus Vulgaris Activity Score (PVAS).13 Their study has the strength of being unbiased toward either the ABSIS or PDAI, of having large numbers of patients with PV and of a greater proportion of patients with severe pemphigus.14 Their prospective study evaluJAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
245
Opinion Editorial
Table. Time Course for Consensus Studies Related to Autoimmune Blistering Diseases Milestone
Location or Publication
Outcome
Formation of IPDG
NIH Pemphigus Meeting with International Pemphigus and Pemphigoid Foundation, May 2005
Lowell Goldsmith and Jean Claude Bystryn appointed D.F.M. and V.P.W. to lead this effort
First meeting of the IPDG
ESDR in Paris, France, September 2006, at Headquarters of Dermatology in Paris, courtesy of Pascal Joly
Began definitions for consensus on pemphigus and PDAI development; multiple meetings
3
Publication of ABSIS by German bullous group
Pfutze et al, 2007
Validation study started by IPDG in 2008
Publication of consensus definitions for pemphigus
Murrell et al,2 2008
Adopted in Cochrane Review 20091 and most pemphigus reports and studies since
Validation study of PDAI compared with ABSIS and PGA in pemphigus
Rosenbach et al,4 2009
In use in clinical trials and 3 further validation studies8-10 To be used for government registration in Japan
Bullous pemphigoid definitions and BPDAI
Murrell et al,16 2012
Validation studies of BPDAI in progress
ABQOL and TABQOL
Sebaratnam et al,17 2013 and Tjokrowidjaja et al,18 2013
Validation studies in foreign languages in progress
MMP definitions and MMPDAI
Meetings of the AAD, SID, and EADV, 2011-2012
Submitted for publication
EBA definitions and EBADAI
Meetings of the AAD, SID, EADV, and IID, 2012-2013
Submitted for publication
Abbreviations: AAD, American Academy of Dermatology; ABSIS, Autoimmune Bullous Skin Disorder Intensity Score; ABQOL, Autoimmune Bullous Disease Quality of Life; EADV, European Academy of Dermatology and Venereology; EBA, Epidermolysis Bullosa Acquisita; EBADAI, Epidermolysis Bullosa Acquisita Disease Area Index; ESDR, European Society for Dermatologic Research; IID, International Investigative Dermatology; IPDG, International Pemphigus
Definitions Group; NIH, National Institutes of Health; MMP, Mucous Membrane Pemphigoid; MMPDAI, Mucous Membrane Pemphigoid Disease Area Index; PDAI, Pemphigus Disease Area Index; PGA, physician global assessment; SID, Society of Investigative Dermatology; TABQOL, Treatment of Autoimmune Bullous Disease Quality of Life.
ated several validation parameters of these 3 outcome measures for PV and compared them against other activity measures, including Dsg1 and Dsg3 titers (Euroimmun kits) to see which outcome measure performed the best.
conducted this study during their normal pemphigus clinics, and the mean time for each score was less than 3 minutes. Rahbar et al10 are to be congratulated for conducting a rigorous study in a busy clinical setting with many cases of pemphigus, that informs us that the PDAI measure is the most reproducible for PV, which will assist the rest of the world in managing these complex patients. One advantage of the PDAI is its sensitivity to low numbers of lesions within defined anatomical areas, which results in increased interrater reliability. By contrast, the ABSIS and PVAS use some items that make the measures less reproducible. The ABSIS uses the low-agreement rule of nines to estimate the percentage of body surface area involvement, which becomes more difficult for limited disease activity. Both the ABSIS and PVAS require evaluation of lesion type and apply it as a weighting factor, which exaggerates small differences between raters and causes lower interrater reliability. Interestingly, the intrarater correlation coefficient between the PDAI and the PVAS was higher than the correlation coefficient between the PDAI and the ABSIS (ICC, 0.82 vs 0.66). Rahbar et al10 point out that the ABSIS may achieve much of its interrater reliability from the subjective component because the ICC for intrarater assessment was reduced when the patient’s subjective assessment was removed. Sponsored clinical trials in pemphigus require the use of robust and validated extent of disease measure tools. The PDAI has a much higher interrater ICC for PV than the PASI does for psoriasis. In practice, this means that in a clinical setting, if different dermatologists or registrars are using the PDAI properly, it can give an objective indication of the patient’s progress, even when the patient is seen by different physicians. Government and insurance funding for expensive treatments is likely to require validated measurements of skin disease extent, as is already happening in Australia for the screening of eligibility for expensive biological therapies for psoriasis,
Highest Interrater Reliability With PDAI In addition, they compared these severity scores with the ELISA scores for Dsg1 and Dsg3. The same 3 junior dermatologists examined each patient with PV independently (n = 100) and scored them on the same day with the 3 extent tools. As they included patients with more severe PV, their average disease extent scores were double those of the earlier study by Rosenbach et al.4 They found a higher interrater reliability for the PDAI of 0.95, compared with 0.88 for the ABSIS and 0.82 for the PVAS. The interrater reliability remained similar for the PDAI and the PVAS at both high and low Dsg titers, whereas for the ABSIS, it was less reliable at low Dsg titers (0.88 vs 0.84). The highest interrater reliability was for skin activity and mucosal activity of the PDAI, followed by skin activity of the PVAS and oral involvement of the ABSIS (interclass correlation coefficients [ICCs]: 0.97, 0.92, 0.90, and 0.89, respectively). Hence, the PDAI, developed by the international consensus group, proved to be the most useful measure. Interestingly, Chams-Davatchi and colleagues1,11,12 found good convergent validity between the PDAI and PVAS scores and Dsg1 titers but poor correlation with the ABSIS (0.65 for the PDAI, 0.32 for the ABSIS, and 0.52 for the PVAS) and poor convergent validity between all 3 scores and the Dsg3 titers (ICCs of 0.35, 0.38, and 0.35 for the PDAI, ABSIS, and PVAS, respectively). This extends our knowledge of the correlation between pemphigus severity previously reported in certain locations and Dsg titers.15
Sensitivity to Change Rahbar et al10 address the sensitivity to change aspect of validity testing, accuracy, feasibility, and external validity. They 246
JAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
jamadermatology.com
Editorial Opinion
and is likely to be used for pemphigus and other blistering diseases for the same reason. The time course for the international consensus that occurred took 3 years from the formation of the International Pemphigus Committee to the publication of the PDAI2 (Table). Subsequent validation studies that have included reliability, responsiveness, and now correlation of biomarkers with the ARTICLE INFORMATION Author Affiliations: Department of Dermatology, St George Hospital, Sydney, Australia (Murrell); Faculty of Medicine, University of New South Wales, Sydney, Australia (Murrell); Department of Dermatology, Keio University, Tokyo, Japan (Amagai); Department of Dermatology, Philadelphia Department of Veterans Affairs, Philadelphia, Pennsylvania (Werth); Department of Dermatology, University of Pennsylvania, Philadelphia (Werth). Corresponding Author: Dédée F. Murrell, MA, BMBCh, MD, FACD, Department of Dermatology, St George Hospital, Gray Street, Kogarah, Sydney, NSW 2217, Australia ([email protected]). Published Online: January 15, 2014. doi:10.1001/jamadermatol.2013.8179. Conflict of Interest Disclosures: None reported. REFERENCES 1. Martin LK, Werth V, Villanueva E, Segall J, Murrell DF, Murrell DF. Interventions for pemphigus vulgaris and pemphigus foliaceus. Cochrane Database Syst Rev. 2009;(1):CD006263. 2. Murrell DF, Dick S, Ahmed AR, et al. Consensus statement on definitions of disease, end points, and therapeutic response for pemphigus. J Am Acad Dermatol. 2008;58(6):1043-1046. 3. Pfütze M, Niedermeier A, Hertl M, Eming R. Introducing a novel Autoimmune Bullous Skin Disorder Intensity Score (ABSIS) in pemphigus. Eur J Dermatol. 2007;17(1):4-11. 4. Rosenbach M, Murrell DF, Bystryn J-C, et al. Reliability and convergent validity of two outcome instruments for pemphigus. J Invest Dermatol. 2009;129(10):2404-2410.
jamadermatology.com
PDAI, have been performed over the past 5 years. These studies have occurred in a relatively short time and build on the international consensus process that has facilitated the design and implementation of the PDAI. Consistency of implementation of outcome measures facilitates comparisons between studies, which is especially critical for a rare disease like pemphigus.
5. Bastuji-Garin S, Sbidian E. How to validate outcome instruments for pemphigus. J Invest Dermatol. 2009;129(10):2328-2330. 6. Daniel BS, Hertl M, Werth VP, Eming R, Murrell DF. Severity score indexes for blistering diseases. Clin Dermatol. 2012;30(1):108-113. 7. Behzad M, Möbs C, Kneisel A, et al. Combined treatment with immunoadsorption and rituximab leads to fast and prolonged clinical remission in difficult-to-treat pemphigus vulgaris. Br J Dermatol. 2012;166(4):844-852. 8. Kamiya K, Aoyama Y, Shirafuji Y, et al. A higher correlation of the antibody activities against the calcium-dependent epitopes of desmoglein 3 quantified by ethylenediaminetetraacetic acid-treated enzyme-linked immunosorbent assay with clinical disease activities of pemphigus vulgaris. J Dermatol Sci. 2013;70(3):190-195. 9. Patsatsi A, Kyriakou A, Giannakou A, Pavlitou-Tsiontsi A, Lambropoulos A, Sotiriadis D. Clinical significance of anti-desmoglein-1 and -3 circulating autoantibodies in pemphigus patients measured by Area Index and Intensity Score [published online August 27, 2013]. Acta Derm Venereol. doi:10.2340/00015555-1666. 10. Rahbar Z, Daneshpazhooh M, Mirshams-Shahshahani M, et al. Pemphigus disease activity measurements: Pemphigus Disease Area Index, Autoimmune Bullous Skin Disorder Intensity Score, and Pemphigus Vulgaris Activity Score [published online January 15, 2014]. JAMA Dermatol. doi:10.1001/jamadermatol.2013.8175. 11. Chams-Davatchi C, Esmaili N, Daneshpazhooh M, et al. Randomized controlled open-label trial of four treatment regimens for pemphigus vulgaris. J Am Acad Dermatol. 2007;57(4):622-628.
12. Chams-Davatchi C, Mortazavizadeh A, Daneshpazhooh M, et al. Randomized double blind trial of prednisolone and azathioprine, vs prednisolone and placebo, in the treatment of pemphigus vulgaris. J Eur Acad Dermatol Venereol. 2013;27(10):1285-1292. 13. Chams-Davatchi C, Rahbar Z, Daneshpazhooh M, et al. Pemphigus vulgaris activity score and assessment of convergent validity. Acta Med Iran. 2013;51(4):224-230. 14. Chams-Davatchi C, Valikhani M, Daneshpazhooh M, et al. Pemphigus: analysis of 1209 cases. Int J Dermatol. 2005;44(6):470-476. 15. Harman KE, Seed PT, Gratian MJ, Bhogal BS, Challacombe SJ, Black MM. The severity of cutaneous and oral pemphigus is related to desmoglein 1 and 3 antibody levels. Br J Dermatol. 2001;144(4):775-780. 16. Murrell DF, Daniel BS, Joly P, et al. Definitions and outcome measures for bullous pemphigoid: recommendations by an international panel of experts. J Am Acad Dermatol. 2012;66(3):479-485. 17. Sebaratnam DF, Hanna AM, Chee SN, et al. Development of a quality-of-life instrument for autoimmune bullous disease: the Autoimmune Bullous Disease Quality of Life Questionnaire [published online August 7, 2013]. JAMA Dermatol. doi:10.1001/jamadermatol.2013.4972. 18. Tjokrowidjaja A, Daniel BS, Frew JW, et al. The development and validation of the TABQOL, a tool to measure the quality of life impacts of treatments used in patients with autoimmune blistering disease. Br J Dermatol. In press.
JAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
247
EDITORIAL
Scoring Systems for Blistering Diseases in Practice Why Bother, and Which One Should You Use? Dédée F. Murrell, MA, BMBCh, MD, FACD; Masayuki Amagai, MD; Victoria P. Werth, MD
In most countries, pemphigus is one of those rare, potentially life-threatening skin and mucosal diseases that is referred to specialist centers for treatment. Because it is rare, there is a paucity of randomized clinical trials (RCTs) to inRelated article page 266 form dermatologists of the most effective and safe treatments.1 For rare diseases, metaanalysis, a statistical tool that allows pooling together the results of smaller, similar clinical trials, can be just as valuable as larger clinical trials, but in order to pool the data the studies need to use the same outcome measures for disease assessment.
History After the first pemphigus conference held at the National Institutes of Health in 2005, the international blistering disease community began to work together to establish outcome measures for each autoimmune blistering disease (AIBD) with championing of the concept by Lowell Goldsmith and JeanClaude Bystryn.2 Two of us (D.F.M.and V.P.W.) volunteered to lead this effort, and about 20 international dermatologists stated that they were interested in participating. The first meeting to develop an outcome measure took place in September 2005 in Paris, France, organized with the help of Pascal Joly. The international pemphigus group held several such meetings and adapted the process used to develop the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) for pemphigus, piloted different ways of scoring pemphigus in clinics, and eventually agreed on the Pemphigus Disease Area Index (PDAI). During the same period, the German blistering disease group, some of whom were also in the international group, published a proposed score, termed the Autoimmune Bullous Skin Disorder Intensity Score (ABSIS).3
Validated Scoring Systems Hence, because both of these scores needed to be validated, an initial study (designed by D.F.M. and V.P.W.) was conducted at the University of Pennsylvania, with 15 patients with pemphigus scored by 10 dermatologists who are pemphigus experts (mostly from the United States, for convenience) with the ABSIS, PDAI, and physician global assessment (PGA) scores. This study found that the PDAI had the highest interrater and intrarater intraclass correlation coefficients (ICCs).4 One limitation of the study was that most of the patients had relatively stable disease with lower percentages of skin extent.5 In most countries, it would be difficult to draw together enough patients with severe cases of this rare disease to rectify this limijamadermatology.com
tation. There has been some debate about which score would be best to use in international pemphigus trials, with some of the Europeans preferring the ABSIS and others preferring the international PDAI measure.6,7 Additional measures to improve validation of the PDAI are demonstration of sensitivity to change, accuracy, feasibility, and external validity and reproducibility in other centers.
External Validation From Greece and Japan Since then, just in the past few months, there have been 2 studies published that fill in missing pieces of the validation puzzle. Kamiya et al,8 from Japan, investigated whether pemphigus disease activity correlated with desmoglein (Dsg) 3 titers in 123 serum samples from 19 patients with pemphigus vulgaris (PV) over time and found that the titers of the conformational Dsg3 epitopes correlated with the PDAI score. This speaks to the issues of sensitivity, accuracy, and external validity. Another independent study, conducted in Greece by Patsatsi et al,9 studied 35 consecutive patients with PV and pemphigus foliaceus with PDAI and ABSIS scores at baseline, 6 months, and 12 months to assess disease correlation with the anti-Dsg1 and anti-Dsg3 enzyme-linked immunosorbent assay (ELISA) titers (MBL International Corp). They found statistically significant associations between the total PDAI and ABSIS scores and Dsg1 titers in the patients with cutaneous involvement, but for patients who had only mucosal involvement, the mucosal extent ABSIS and PDAI scores showed only moderate correlation with Dsg3 ELISA at baseline and 6 months. This study provides further validation of the scores in terms of sensitivity to change in clinical disease, accuracy, and external validity. Both these studies address reproducibility in additional centers.
Iranian Validation Study In this issue of JAMA Dermatology, Rahbar et al10 report a third independent study that provides additional validation of the PDAI and ABSIS. This group, who were not part of the International Pemphigus group, is from a dermatology-only teaching hospital (Razi Hospital, Tehran University of Medical Sciences, Tehran, Iran) and is led by Cheyda Chams Davatchi, who trained as a dermatologist at St Louis Hospital in Paris, and has an excellent track record in conducting 2 of the world’s 13 RCTs in pemphigus.1,11,12 Rahbar et al10 developed and validated another extent of disease score, termed the Pemphigus Vulgaris Activity Score (PVAS).13 Their study has the strength of being unbiased toward either the ABSIS or PDAI, of having large numbers of patients with PV and of a greater proportion of patients with severe pemphigus.14 Their prospective study evaluJAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
245
Opinion Editorial
Table. Time Course for Consensus Studies Related to Autoimmune Blistering Diseases Milestone
Location or Publication
Outcome
Formation of IPDG
NIH Pemphigus Meeting with International Pemphigus and Pemphigoid Foundation, May 2005
Lowell Goldsmith and Jean Claude Bystryn appointed D.F.M. and V.P.W. to lead this effort
First meeting of the IPDG
ESDR in Paris, France, September 2006, at Headquarters of Dermatology in Paris, courtesy of Pascal Joly
Began definitions for consensus on pemphigus and PDAI development; multiple meetings
3
Publication of ABSIS by German bullous group
Pfutze et al, 2007
Validation study started by IPDG in 2008
Publication of consensus definitions for pemphigus
Murrell et al,2 2008
Adopted in Cochrane Review 20091 and most pemphigus reports and studies since
Validation study of PDAI compared with ABSIS and PGA in pemphigus
Rosenbach et al,4 2009
In use in clinical trials and 3 further validation studies8-10 To be used for government registration in Japan
Bullous pemphigoid definitions and BPDAI
Murrell et al,16 2012
Validation studies of BPDAI in progress
ABQOL and TABQOL
Sebaratnam et al,17 2013 and Tjokrowidjaja et al,18 2013
Validation studies in foreign languages in progress
MMP definitions and MMPDAI
Meetings of the AAD, SID, and EADV, 2011-2012
Submitted for publication
EBA definitions and EBADAI
Meetings of the AAD, SID, EADV, and IID, 2012-2013
Submitted for publication
Abbreviations: AAD, American Academy of Dermatology; ABSIS, Autoimmune Bullous Skin Disorder Intensity Score; ABQOL, Autoimmune Bullous Disease Quality of Life; EADV, European Academy of Dermatology and Venereology; EBA, Epidermolysis Bullosa Acquisita; EBADAI, Epidermolysis Bullosa Acquisita Disease Area Index; ESDR, European Society for Dermatologic Research; IID, International Investigative Dermatology; IPDG, International Pemphigus
Definitions Group; NIH, National Institutes of Health; MMP, Mucous Membrane Pemphigoid; MMPDAI, Mucous Membrane Pemphigoid Disease Area Index; PDAI, Pemphigus Disease Area Index; PGA, physician global assessment; SID, Society of Investigative Dermatology; TABQOL, Treatment of Autoimmune Bullous Disease Quality of Life.
ated several validation parameters of these 3 outcome measures for PV and compared them against other activity measures, including Dsg1 and Dsg3 titers (Euroimmun kits) to see which outcome measure performed the best.
conducted this study during their normal pemphigus clinics, and the mean time for each score was less than 3 minutes. Rahbar et al10 are to be congratulated for conducting a rigorous study in a busy clinical setting with many cases of pemphigus, that informs us that the PDAI measure is the most reproducible for PV, which will assist the rest of the world in managing these complex patients. One advantage of the PDAI is its sensitivity to low numbers of lesions within defined anatomical areas, which results in increased interrater reliability. By contrast, the ABSIS and PVAS use some items that make the measures less reproducible. The ABSIS uses the low-agreement rule of nines to estimate the percentage of body surface area involvement, which becomes more difficult for limited disease activity. Both the ABSIS and PVAS require evaluation of lesion type and apply it as a weighting factor, which exaggerates small differences between raters and causes lower interrater reliability. Interestingly, the intrarater correlation coefficient between the PDAI and the PVAS was higher than the correlation coefficient between the PDAI and the ABSIS (ICC, 0.82 vs 0.66). Rahbar et al10 point out that the ABSIS may achieve much of its interrater reliability from the subjective component because the ICC for intrarater assessment was reduced when the patient’s subjective assessment was removed. Sponsored clinical trials in pemphigus require the use of robust and validated extent of disease measure tools. The PDAI has a much higher interrater ICC for PV than the PASI does for psoriasis. In practice, this means that in a clinical setting, if different dermatologists or registrars are using the PDAI properly, it can give an objective indication of the patient’s progress, even when the patient is seen by different physicians. Government and insurance funding for expensive treatments is likely to require validated measurements of skin disease extent, as is already happening in Australia for the screening of eligibility for expensive biological therapies for psoriasis,
Highest Interrater Reliability With PDAI In addition, they compared these severity scores with the ELISA scores for Dsg1 and Dsg3. The same 3 junior dermatologists examined each patient with PV independently (n = 100) and scored them on the same day with the 3 extent tools. As they included patients with more severe PV, their average disease extent scores were double those of the earlier study by Rosenbach et al.4 They found a higher interrater reliability for the PDAI of 0.95, compared with 0.88 for the ABSIS and 0.82 for the PVAS. The interrater reliability remained similar for the PDAI and the PVAS at both high and low Dsg titers, whereas for the ABSIS, it was less reliable at low Dsg titers (0.88 vs 0.84). The highest interrater reliability was for skin activity and mucosal activity of the PDAI, followed by skin activity of the PVAS and oral involvement of the ABSIS (interclass correlation coefficients [ICCs]: 0.97, 0.92, 0.90, and 0.89, respectively). Hence, the PDAI, developed by the international consensus group, proved to be the most useful measure. Interestingly, Chams-Davatchi and colleagues1,11,12 found good convergent validity between the PDAI and PVAS scores and Dsg1 titers but poor correlation with the ABSIS (0.65 for the PDAI, 0.32 for the ABSIS, and 0.52 for the PVAS) and poor convergent validity between all 3 scores and the Dsg3 titers (ICCs of 0.35, 0.38, and 0.35 for the PDAI, ABSIS, and PVAS, respectively). This extends our knowledge of the correlation between pemphigus severity previously reported in certain locations and Dsg titers.15
Sensitivity to Change Rahbar et al10 address the sensitivity to change aspect of validity testing, accuracy, feasibility, and external validity. They 246
JAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
jamadermatology.com
Editorial Opinion
and is likely to be used for pemphigus and other blistering diseases for the same reason. The time course for the international consensus that occurred took 3 years from the formation of the International Pemphigus Committee to the publication of the PDAI2 (Table). Subsequent validation studies that have included reliability, responsiveness, and now correlation of biomarkers with the ARTICLE INFORMATION Author Affiliations: Department of Dermatology, St George Hospital, Sydney, Australia (Murrell); Faculty of Medicine, University of New South Wales, Sydney, Australia (Murrell); Department of Dermatology, Keio University, Tokyo, Japan (Amagai); Department of Dermatology, Philadelphia Department of Veterans Affairs, Philadelphia, Pennsylvania (Werth); Department of Dermatology, University of Pennsylvania, Philadelphia (Werth). Corresponding Author: Dédée F. Murrell, MA, BMBCh, MD, FACD, Department of Dermatology, St George Hospital, Gray Street, Kogarah, Sydney, NSW 2217, Australia ([email protected]). Published Online: January 15, 2014. doi:10.1001/jamadermatol.2013.8179. Conflict of Interest Disclosures: None reported. REFERENCES 1. Martin LK, Werth V, Villanueva E, Segall J, Murrell DF, Murrell DF. Interventions for pemphigus vulgaris and pemphigus foliaceus. Cochrane Database Syst Rev. 2009;(1):CD006263. 2. Murrell DF, Dick S, Ahmed AR, et al. Consensus statement on definitions of disease, end points, and therapeutic response for pemphigus. J Am Acad Dermatol. 2008;58(6):1043-1046. 3. Pfütze M, Niedermeier A, Hertl M, Eming R. Introducing a novel Autoimmune Bullous Skin Disorder Intensity Score (ABSIS) in pemphigus. Eur J Dermatol. 2007;17(1):4-11. 4. Rosenbach M, Murrell DF, Bystryn J-C, et al. Reliability and convergent validity of two outcome instruments for pemphigus. J Invest Dermatol. 2009;129(10):2404-2410.
jamadermatology.com
PDAI, have been performed over the past 5 years. These studies have occurred in a relatively short time and build on the international consensus process that has facilitated the design and implementation of the PDAI. Consistency of implementation of outcome measures facilitates comparisons between studies, which is especially critical for a rare disease like pemphigus.
5. Bastuji-Garin S, Sbidian E. How to validate outcome instruments for pemphigus. J Invest Dermatol. 2009;129(10):2328-2330. 6. Daniel BS, Hertl M, Werth VP, Eming R, Murrell DF. Severity score indexes for blistering diseases. Clin Dermatol. 2012;30(1):108-113. 7. Behzad M, Möbs C, Kneisel A, et al. Combined treatment with immunoadsorption and rituximab leads to fast and prolonged clinical remission in difficult-to-treat pemphigus vulgaris. Br J Dermatol. 2012;166(4):844-852. 8. Kamiya K, Aoyama Y, Shirafuji Y, et al. A higher correlation of the antibody activities against the calcium-dependent epitopes of desmoglein 3 quantified by ethylenediaminetetraacetic acid-treated enzyme-linked immunosorbent assay with clinical disease activities of pemphigus vulgaris. J Dermatol Sci. 2013;70(3):190-195. 9. Patsatsi A, Kyriakou A, Giannakou A, Pavlitou-Tsiontsi A, Lambropoulos A, Sotiriadis D. Clinical significance of anti-desmoglein-1 and -3 circulating autoantibodies in pemphigus patients measured by Area Index and Intensity Score [published online August 27, 2013]. Acta Derm Venereol. doi:10.2340/00015555-1666. 10. Rahbar Z, Daneshpazhooh M, Mirshams-Shahshahani M, et al. Pemphigus disease activity measurements: Pemphigus Disease Area Index, Autoimmune Bullous Skin Disorder Intensity Score, and Pemphigus Vulgaris Activity Score [published online January 15, 2014]. JAMA Dermatol. doi:10.1001/jamadermatol.2013.8175. 11. Chams-Davatchi C, Esmaili N, Daneshpazhooh M, et al. Randomized controlled open-label trial of four treatment regimens for pemphigus vulgaris. J Am Acad Dermatol. 2007;57(4):622-628.
12. Chams-Davatchi C, Mortazavizadeh A, Daneshpazhooh M, et al. Randomized double blind trial of prednisolone and azathioprine, vs prednisolone and placebo, in the treatment of pemphigus vulgaris. J Eur Acad Dermatol Venereol. 2013;27(10):1285-1292. 13. Chams-Davatchi C, Rahbar Z, Daneshpazhooh M, et al. Pemphigus vulgaris activity score and assessment of convergent validity. Acta Med Iran. 2013;51(4):224-230. 14. Chams-Davatchi C, Valikhani M, Daneshpazhooh M, et al. Pemphigus: analysis of 1209 cases. Int J Dermatol. 2005;44(6):470-476. 15. Harman KE, Seed PT, Gratian MJ, Bhogal BS, Challacombe SJ, Black MM. The severity of cutaneous and oral pemphigus is related to desmoglein 1 and 3 antibody levels. Br J Dermatol. 2001;144(4):775-780. 16. Murrell DF, Daniel BS, Joly P, et al. Definitions and outcome measures for bullous pemphigoid: recommendations by an international panel of experts. J Am Acad Dermatol. 2012;66(3):479-485. 17. Sebaratnam DF, Hanna AM, Chee SN, et al. Development of a quality-of-life instrument for autoimmune bullous disease: the Autoimmune Bullous Disease Quality of Life Questionnaire [published online August 7, 2013]. JAMA Dermatol. doi:10.1001/jamadermatol.2013.4972. 18. Tjokrowidjaja A, Daniel BS, Frew JW, et al. The development and validation of the TABQOL, a tool to measure the quality of life impacts of treatments used in patients with autoimmune blistering disease. Br J Dermatol. In press.
JAMA Dermatology March 2014 Volume 150, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archderm.jamanetwork.com/ by a University Of Connecticut Health Center User on 05/12/2015
247
