Research
Diagnostic
Criteria
To the Editor\p=m-\Althoughwe welcome discussion of the relative merits of different research criteria for Psy chiatric diagnosis, the article by Drs Overall and Hollister in the Archives
(36:1198-1205,
1979)
"Comparative
Evaluation of Research Diagnostic Criteria for Schizophrenia" troubles us for several reasons. First of all, the authors do not address the issue of the different purposes of various sets of research diagnostic criteria. Whereas the purpose of the actuarial approach taken by Drs Overall and Hollister is to simulate competent or expert clinical practice, the purpose of the Wash-
ington University criteria, research diagnostic criteria (RDC), and DSMIII criteria is to improve usual clinical practice by incorporating into the
criteria distinctions that have been shown by research study to have some validity in terms of such variables as course, response to specific therapy, familial pattern, etc. Given this difference in purpose, it is hardly adequate to approach the evaluation of all of these sets of criteria by examining the agreement with clinical diagnosis, even with clinicians designated by Overall and Hollister as "expert." If the goal is to improve clinical practice, what is needed is evidence of the relative validity of these sets of criteria using external criteria, such as short- or long-term outcome. Kendell et al' have compared the prognostic validity of the RDC with the CATEGO program of Wing, Cooper, and Sartorius, the WHO crite¬ ria, the New Haven Schizophrenia Index, and with clinical diagnosis. When these criteria were compared on a mixed group of patients, including those with affective disorder and schizophrenia, the RDC diagnosis of schizophrenia was found to be among the most predictive of both incomplete symptomatic recovery and poor social outcome.
The authors state that "the relative merits of narrow versus broader defi-
nitions of the diagnostic populations should be evaluated before a set of criteria is chosen which in effect rede¬ fines the clinical concept of schizo¬ phrenia." Any review of the literature of the last decade would indicate that the predominance of evidence sug¬ gests that the broader definition of schizophrenia that has been used in this country, which includes nonpsychotic conditions and psychoses of brief duration, is extremely hetero¬ geneous regarding both outcome and familial pattern. It is the result of such research, and not merely the opinions of the "experts" who have been involved in the development of
the Washington University criteria, the RDC, and the DSM-III criteria, that has led to the incorporation into these criteria for schizophrenia of the requirement of psychotic features as well as some degree of chronicity. We wish that the Composite Diag¬
nostic Checklist for
Schizophrenia
would, in fact, facilitate the
compara¬
tive evaluation of research criteria for
schizophrenia, using appropriate ex¬ ternal criteria for validity. Unfortu¬ nately, examination of this form leads
to conclude that data collected with it will not be representative of data collected using the different sets of criteria as they are actually written. The most glaring example is the criti¬ cal item on their form of "Duration of us
present episode" categorized as one week, two to four weeks, four weeks to
six months, and six months or more. This item is used by the computer to score the duration criteria for the Washington University criteria and for the RDC. However, in both the Washington University criteria and RDC, the duration refers to the "ill¬ ness" and not to the "present episode." Hair-splitting? We think not. Despite any urging on the part of the authors to have raters consult the full-written criteria and definitions of terms, when the time comes to fill out the form, raters will undoubtedly conceptualize items the way they are phrased on the data form. The patient with a chronic illness who has recently
had an acute exacerbation of one month's duration is likely to be coded as having a "duration of present episode" of two to four weeks using the computerized form; yet, using the actual Washington University criteria would be noted as meeting the criteri¬ on of "a chronic illness with at least 6 months of symptoms." A heavy price to pay for the luxury of not having to refer to several sets of criteria that have overlapping items for each
patient!
We believe that this explains the seemingly hard to understand finding that only a quarter of the clinically diagnosed, and only a third of the RDC-diagnosed schizophrenic pa¬ tients met the Washington University criteria for schizophrenia. In our own experience in an ongoing study, approximately 70% of the patients who meet the RDC criteria for schizo¬
phrenia also meet the Washington University criteria. We believe that this large concordance is because we used the Washington University dura¬ tion criterion (six months) and the RDC duration criterion (two weeks) as are written (where the refer¬ ences are to an "illness" and not an
they
"episode").
It is of interest that the DSM-III criteria for schizophrenia now also require at least six months' duration of illness, including prodromal and residual phases of the illness. Despite what the reader of the article by Drs Overall and Hollister might antici¬ pate, that is, the defining away of schizophrenia by use of a "narrow definition" of the illness, data from the DSM-III Field Trials indicate that fully 30% of a general inpatient group (N 1,778) was given this diagnosis using the DSM-III criteria. Although this may represent a small drop in the frequency with which this diagnosis was given in similar facilities using the much broader DSM-II concept, it hardly suggests a drastic revision in diagnostic practice. That is un¬ doubtedly why a majority of the several hundred clinicians participat¬ ing in the DSM-III Field Trials indi=
Downloaded From: http://archpsyc.jamanetwork.com/pdfaccess.ashx?url=/data/journals/psych/12320/ by a University of California - San Diego User on 03/02/2017
cated in a questionnaire their support of the DSM-III concept of requiring six months' duration for schizophre¬ nia. The DSM-III criteria are based on an assessment by the American Psy¬ chiatric Association's Task Force on Nomenclature and Statistics and its advisory committees of the best avail¬ able evidence. Sometimes decisions are buttressed by research data and when these are lacking, by good clini¬ cal judgment. The comparative validi¬ ty of alternative sets of criteria needs to be studied so that future DSMs can benefit with modifications, however drastic, of the DSM-III criteria. How¬ ever, when studying different sets of criteria, meticulous attention must be given to ensure that they are used as intended. Otherwise, the results may be uninterpretable. Robert L. Spitzer, MD Jean Endicott, PhD Janet B. W. Williams, MSW New York State Psychiatric
Institute
722 W 168th St New York, NY 10032 1. Kendell RE, Brockington IF, Leff JP: Prognostic implications of six alternative definitions of schizophrenia. Arch Gen Psychiatry 36:25-31, 1979.
Reply.\p=m-\ If our article was "troubling", then it probably served its purpose. Unfortunately, the various diagnostic criteria are being widely and uncritically adopted without adequate validation. Our article was not intended as a criticism of any particular approach to operationalizing diagnostic criteria. It was intended merely to emphasize the need for empirical In
evaluation and validation of whatever approach is ultimately accepted. The rebuttal by Spitzer and colleagues states that the purpose of the Washington University criteria, the New York criteria, and the DSM-III criteria is to improve clinical practice by incorporating into diagnostic criteria distinctions that have been shown "by research study" to have some validity in terms of such variables as course, response to
specific therapy,
familial pattern, etc. Instead of repeating the oft-stated research claims, would it not be more convincing to document them? Dr Spitzer obviously recognizes the critical difference that specific wording of diagnostic criteria can make. While there may be a general empirical basis for some of the content incorporated into various diagnostic criteria, we ques¬ tion the research basis for the diag-
nostic criteria as they are specifically formulated. The recent study by Kendell et al,1 which is the only evidence cited by Spitzer and colleagues, is an excellent example of the kind of comparative evaluations that we believe to be crucial at this time. The Edinburgh (Kendell et al) results emphasize low concordance of the various research diagnostic criteria with clinical diag¬ nosis, as well as the fact that alterna¬ tive diagnostic criteria do not agree well among themselves. Also in agree¬ ment with our own results, numerous patients who received the clinical diagnosis of schizophrenia by the
Edinburgh psychiatrists
were
ex¬
cluded from that diagnostic category by the research criteria. Less than two thirds of the clinically diagnosed schizophrenics satisfied the Spitzer RDC, for example. We are especially encouraged that results consistent with our own have come from Edin¬ burgh because discrepancies between research and clinical diagnoses have been so often excused on the grounds that American psychiatrists overuse the diagnosis of schizophrenia. The Edinburgh study went on to examine relevance of the alternative diagnostic criteria for predicting in¬ complete symptomatic recovery and poor social adjustment. In compari¬ sons where the full sample of 134
patients was included, clinical diagno¬ sis was equal to any of the research criteria. Kendell et al generously concluded that the better of the objec¬ tive research diagnostic criteria were as good at predicting outcome as were the original clinical criteria. Rather
weak support we think for the conten¬ tion that the Washington University, New York, and DSM-III criteria improve, rather than merely operationalize, clinical diagnostic practices. If there are not real advantages to the more restrictive research diagnostic criteria, there are both practical and scientific disadvantages to defining
categories too narrowly. Dr Spitzer and colleagues singled out for special criticism our checklist item pertaining to "duration of pres¬ ent episode." In their critical com¬ ment, they state that the Washington University and New York criteria specify duration of "illness," not dura¬ tion of "episode," as the requirement for a diagnosis of schizophrenia. We did indeed interpret "episode" to be
the best succinct statement of the duration requirement as specifically elaborated in the Research Diagnostic Criteria (RDC) for a Selected Group of
Functional Disorders (ed 3).2 To quote exactly from that source document by Spitzer et al, with italics supplied by us, "Signs of the illness have lasted at least two weeks from the onset of a noticeable change in the subjects' usual condition...." We interpret that wording to imply an episode, as might be observed in a "patient with a chronic illness who has recently had an acute exacerbation of one month dura¬ tion." We leave it to the reader to decide how duration should be counted according to the quoted Spitzer defini¬ tion for the chronically ill patient described in the critical comment to which this reply is addressed. At best, a substantial ambiguity is present that would not be eliminated by providing the exact wording of the duration item from the New York RDC (1977) if that item does not refer
episode. wording of the duration re¬ quirement in the Washington Univer¬ sity research diagnostic criteria for schizophrenia is also subject to inter¬ pretation as an episode. To quote from Feighner et al with italics supplied by to
an
The
'
"A chronic illness with at least six months of symptoms prior to the index evaluation without return to the premorbid level of psychosocial adjust¬ ment." This definition recognizes that us:
patients
can
return to
level, and the timer is
premorbid
reset to start running at the onset of the next episode. In a data form to be used for recording information pertinent to
evaluating whether a patient does or does not satisfy alternative research diagnostic criteria, the term "episode"
come closer to captur¬ duration requirements than does "duration of illness." To the extent that we have misunderstood the written specifications of various research diagnostic criteria, we sus¬ pect that they may very well be misunderstood by others also. A final problem must be noted: Dr Spitzer refers to DSM-III and to a change in the duration requirements. Not only are diagnostic criteria being formulated and published without appropriate validation, but continued revision renders existing sets of crite¬ ria obsolete before they can be adequately evaluated even by others. That appears to be one sure way to keep ahead of the scientific communi¬ ty, but it fails to provide evidence that change constitutes improvement. To this end, we propose to continue to develop and validate scoring for the Composite Diagnostic Checklist for Schizophrenia which includes a broad seems
to
us
to
ing the quoted
Downloaded From: http://archpsyc.jamanetwork.com/pdfaccess.ashx?url=/data/journals/psych/12320/ by a University of California - San Diego User on 03/02/2017
Diagnostic
Criteria
To the Editor\p=m-\Althoughwe welcome discussion of the relative merits of different research criteria for Psy chiatric diagnosis, the article by Drs Overall and Hollister in the Archives
(36:1198-1205,
1979)
"Comparative
Evaluation of Research Diagnostic Criteria for Schizophrenia" troubles us for several reasons. First of all, the authors do not address the issue of the different purposes of various sets of research diagnostic criteria. Whereas the purpose of the actuarial approach taken by Drs Overall and Hollister is to simulate competent or expert clinical practice, the purpose of the Wash-
ington University criteria, research diagnostic criteria (RDC), and DSMIII criteria is to improve usual clinical practice by incorporating into the
criteria distinctions that have been shown by research study to have some validity in terms of such variables as course, response to specific therapy, familial pattern, etc. Given this difference in purpose, it is hardly adequate to approach the evaluation of all of these sets of criteria by examining the agreement with clinical diagnosis, even with clinicians designated by Overall and Hollister as "expert." If the goal is to improve clinical practice, what is needed is evidence of the relative validity of these sets of criteria using external criteria, such as short- or long-term outcome. Kendell et al' have compared the prognostic validity of the RDC with the CATEGO program of Wing, Cooper, and Sartorius, the WHO crite¬ ria, the New Haven Schizophrenia Index, and with clinical diagnosis. When these criteria were compared on a mixed group of patients, including those with affective disorder and schizophrenia, the RDC diagnosis of schizophrenia was found to be among the most predictive of both incomplete symptomatic recovery and poor social outcome.
The authors state that "the relative merits of narrow versus broader defi-
nitions of the diagnostic populations should be evaluated before a set of criteria is chosen which in effect rede¬ fines the clinical concept of schizo¬ phrenia." Any review of the literature of the last decade would indicate that the predominance of evidence sug¬ gests that the broader definition of schizophrenia that has been used in this country, which includes nonpsychotic conditions and psychoses of brief duration, is extremely hetero¬ geneous regarding both outcome and familial pattern. It is the result of such research, and not merely the opinions of the "experts" who have been involved in the development of
the Washington University criteria, the RDC, and the DSM-III criteria, that has led to the incorporation into these criteria for schizophrenia of the requirement of psychotic features as well as some degree of chronicity. We wish that the Composite Diag¬
nostic Checklist for
Schizophrenia
would, in fact, facilitate the
compara¬
tive evaluation of research criteria for
schizophrenia, using appropriate ex¬ ternal criteria for validity. Unfortu¬ nately, examination of this form leads
to conclude that data collected with it will not be representative of data collected using the different sets of criteria as they are actually written. The most glaring example is the criti¬ cal item on their form of "Duration of us
present episode" categorized as one week, two to four weeks, four weeks to
six months, and six months or more. This item is used by the computer to score the duration criteria for the Washington University criteria and for the RDC. However, in both the Washington University criteria and RDC, the duration refers to the "ill¬ ness" and not to the "present episode." Hair-splitting? We think not. Despite any urging on the part of the authors to have raters consult the full-written criteria and definitions of terms, when the time comes to fill out the form, raters will undoubtedly conceptualize items the way they are phrased on the data form. The patient with a chronic illness who has recently
had an acute exacerbation of one month's duration is likely to be coded as having a "duration of present episode" of two to four weeks using the computerized form; yet, using the actual Washington University criteria would be noted as meeting the criteri¬ on of "a chronic illness with at least 6 months of symptoms." A heavy price to pay for the luxury of not having to refer to several sets of criteria that have overlapping items for each
patient!
We believe that this explains the seemingly hard to understand finding that only a quarter of the clinically diagnosed, and only a third of the RDC-diagnosed schizophrenic pa¬ tients met the Washington University criteria for schizophrenia. In our own experience in an ongoing study, approximately 70% of the patients who meet the RDC criteria for schizo¬
phrenia also meet the Washington University criteria. We believe that this large concordance is because we used the Washington University dura¬ tion criterion (six months) and the RDC duration criterion (two weeks) as are written (where the refer¬ ences are to an "illness" and not an
they
"episode").
It is of interest that the DSM-III criteria for schizophrenia now also require at least six months' duration of illness, including prodromal and residual phases of the illness. Despite what the reader of the article by Drs Overall and Hollister might antici¬ pate, that is, the defining away of schizophrenia by use of a "narrow definition" of the illness, data from the DSM-III Field Trials indicate that fully 30% of a general inpatient group (N 1,778) was given this diagnosis using the DSM-III criteria. Although this may represent a small drop in the frequency with which this diagnosis was given in similar facilities using the much broader DSM-II concept, it hardly suggests a drastic revision in diagnostic practice. That is un¬ doubtedly why a majority of the several hundred clinicians participat¬ ing in the DSM-III Field Trials indi=
Downloaded From: http://archpsyc.jamanetwork.com/pdfaccess.ashx?url=/data/journals/psych/12320/ by a University of California - San Diego User on 03/02/2017
cated in a questionnaire their support of the DSM-III concept of requiring six months' duration for schizophre¬ nia. The DSM-III criteria are based on an assessment by the American Psy¬ chiatric Association's Task Force on Nomenclature and Statistics and its advisory committees of the best avail¬ able evidence. Sometimes decisions are buttressed by research data and when these are lacking, by good clini¬ cal judgment. The comparative validi¬ ty of alternative sets of criteria needs to be studied so that future DSMs can benefit with modifications, however drastic, of the DSM-III criteria. How¬ ever, when studying different sets of criteria, meticulous attention must be given to ensure that they are used as intended. Otherwise, the results may be uninterpretable. Robert L. Spitzer, MD Jean Endicott, PhD Janet B. W. Williams, MSW New York State Psychiatric
Institute
722 W 168th St New York, NY 10032 1. Kendell RE, Brockington IF, Leff JP: Prognostic implications of six alternative definitions of schizophrenia. Arch Gen Psychiatry 36:25-31, 1979.
Reply.\p=m-\ If our article was "troubling", then it probably served its purpose. Unfortunately, the various diagnostic criteria are being widely and uncritically adopted without adequate validation. Our article was not intended as a criticism of any particular approach to operationalizing diagnostic criteria. It was intended merely to emphasize the need for empirical In
evaluation and validation of whatever approach is ultimately accepted. The rebuttal by Spitzer and colleagues states that the purpose of the Washington University criteria, the New York criteria, and the DSM-III criteria is to improve clinical practice by incorporating into diagnostic criteria distinctions that have been shown "by research study" to have some validity in terms of such variables as course, response to
specific therapy,
familial pattern, etc. Instead of repeating the oft-stated research claims, would it not be more convincing to document them? Dr Spitzer obviously recognizes the critical difference that specific wording of diagnostic criteria can make. While there may be a general empirical basis for some of the content incorporated into various diagnostic criteria, we ques¬ tion the research basis for the diag-
nostic criteria as they are specifically formulated. The recent study by Kendell et al,1 which is the only evidence cited by Spitzer and colleagues, is an excellent example of the kind of comparative evaluations that we believe to be crucial at this time. The Edinburgh (Kendell et al) results emphasize low concordance of the various research diagnostic criteria with clinical diag¬ nosis, as well as the fact that alterna¬ tive diagnostic criteria do not agree well among themselves. Also in agree¬ ment with our own results, numerous patients who received the clinical diagnosis of schizophrenia by the
Edinburgh psychiatrists
were
ex¬
cluded from that diagnostic category by the research criteria. Less than two thirds of the clinically diagnosed schizophrenics satisfied the Spitzer RDC, for example. We are especially encouraged that results consistent with our own have come from Edin¬ burgh because discrepancies between research and clinical diagnoses have been so often excused on the grounds that American psychiatrists overuse the diagnosis of schizophrenia. The Edinburgh study went on to examine relevance of the alternative diagnostic criteria for predicting in¬ complete symptomatic recovery and poor social adjustment. In compari¬ sons where the full sample of 134
patients was included, clinical diagno¬ sis was equal to any of the research criteria. Kendell et al generously concluded that the better of the objec¬ tive research diagnostic criteria were as good at predicting outcome as were the original clinical criteria. Rather
weak support we think for the conten¬ tion that the Washington University, New York, and DSM-III criteria improve, rather than merely operationalize, clinical diagnostic practices. If there are not real advantages to the more restrictive research diagnostic criteria, there are both practical and scientific disadvantages to defining
categories too narrowly. Dr Spitzer and colleagues singled out for special criticism our checklist item pertaining to "duration of pres¬ ent episode." In their critical com¬ ment, they state that the Washington University and New York criteria specify duration of "illness," not dura¬ tion of "episode," as the requirement for a diagnosis of schizophrenia. We did indeed interpret "episode" to be
the best succinct statement of the duration requirement as specifically elaborated in the Research Diagnostic Criteria (RDC) for a Selected Group of
Functional Disorders (ed 3).2 To quote exactly from that source document by Spitzer et al, with italics supplied by us, "Signs of the illness have lasted at least two weeks from the onset of a noticeable change in the subjects' usual condition...." We interpret that wording to imply an episode, as might be observed in a "patient with a chronic illness who has recently had an acute exacerbation of one month dura¬ tion." We leave it to the reader to decide how duration should be counted according to the quoted Spitzer defini¬ tion for the chronically ill patient described in the critical comment to which this reply is addressed. At best, a substantial ambiguity is present that would not be eliminated by providing the exact wording of the duration item from the New York RDC (1977) if that item does not refer
episode. wording of the duration re¬ quirement in the Washington Univer¬ sity research diagnostic criteria for schizophrenia is also subject to inter¬ pretation as an episode. To quote from Feighner et al with italics supplied by to
an
The
'
"A chronic illness with at least six months of symptoms prior to the index evaluation without return to the premorbid level of psychosocial adjust¬ ment." This definition recognizes that us:
patients
can
return to
level, and the timer is
premorbid
reset to start running at the onset of the next episode. In a data form to be used for recording information pertinent to
evaluating whether a patient does or does not satisfy alternative research diagnostic criteria, the term "episode"
come closer to captur¬ duration requirements than does "duration of illness." To the extent that we have misunderstood the written specifications of various research diagnostic criteria, we sus¬ pect that they may very well be misunderstood by others also. A final problem must be noted: Dr Spitzer refers to DSM-III and to a change in the duration requirements. Not only are diagnostic criteria being formulated and published without appropriate validation, but continued revision renders existing sets of crite¬ ria obsolete before they can be adequately evaluated even by others. That appears to be one sure way to keep ahead of the scientific communi¬ ty, but it fails to provide evidence that change constitutes improvement. To this end, we propose to continue to develop and validate scoring for the Composite Diagnostic Checklist for Schizophrenia which includes a broad seems
to
us
to
ing the quoted
Downloaded From: http://archpsyc.jamanetwork.com/pdfaccess.ashx?url=/data/journals/psych/12320/ by a University of California - San Diego User on 03/02/2017
