Social Psychiatry a0~
Soc Psychiatry Psychiatr Epidemiol (1992)27:108-116
PsychiatricEpidemiology
9 Springer-Verlag 1992
A triaxial classification of health problems presenting in primary health care* A World Health Organization multi-centre study A. Clare 1, W. Gulbinat 2, and N. Sartorius 2
St. Patrick's Hospital, Dublin, Ireland 2Division of Mental Health, WHO, Geneva Accepted: December 30, 1991
Summary. Over the past two decades, research in a variety of countries and settings has revealed the extent to which psychological morbidity and social problems constitute an important proportion of primary health care contacts, either in their own right or in association with physical illhealth. The development of appropriate responses, medical and non-medical, to such problems is, in part, hindered by the relatively low level of awareness concerning the significance of such problems and by currently inappropriate methods of describing and classifying health care problems in the primary care setting. There is a pressing need for an appropriate, relatively simple and flexible classification or list of problems presenting in primary care. This study describes the development, under W H O auspices and in an international setting, of a triaxial classification of health problems and the testing and modification of the classification by means of an international case vignette rating exercise. A psychological problem list and a social problem list, together with detailed glossaries, suitably modified in the light of experience derived from the study, are described, and proposals enumerated of possible ways in which these lists might be further developed and used in research and clinical undertakings in primary care.
* This paper is based on the data and experience obtained during the participation of the authors in the WHO Project on Recording Health Problems Triaxially: the Physical, Psychological and Social Components of Primary Health Care Contacts, a project sponsored by the World Health Organization, the Rockefeller Foundation, the US National Institute of Mental Health (Grant ROIMH35753) and the participating field reseach centres.
Collaborating investigators: M. Bass, University of Western Ontario, London, Canada; E. Bushello, Vila Sao Jos6 do Murialdo Health Centre, Porto Alegre, Brazil; C. Climent, Universidad de Valle, Cali, Colombia; U. Laksanavicharn, Ministry of Health, Bangkok, Thailand; K.Selliah, Penang Hospital, Penang, Malaysia; R. Serpell, University of Zambia, Lusaka, Zambia. For a detailed discussion of the conceptual, methodological and statistical issues involved, readers are referred to Psychosocial Factors Affecting Health [1].
There is considerable evidence to indicate a close inter-relationship between psychiatric ill-health and social problems in the setting of primary health care [2, 3]. Studies have delineated the impact of life events [4], family and interpersonal stresses, and difficulties and deficiencies in social support [5] on the psychological and physical health of patients in this setting. Preliminary attempts indicate that modifying treatment to take account of the role of social problems may have important clinical implications [6]. The primary health care sector is in great need of a classification system of mental disorder diagnoses and psychosocial problems which is uncomplicated, appropriate and concurrently compatible with other more complex disease classification systems, such as the International Classification of Diseases [7]. The development and acceptance of such a system would increase the likelihood that problems for which patients seek medical care would be more accurately described. Implementation of such a classification system also holds out the possibility that clinicians would be benefitted in the management of their patients while policy makers would be assisted in planning services in response to medical and social need and in allocating resources. Educators of physicians and of professionals working alongside them in primary care might be encouraged to adapt current training to take account of primary health care realities. A more appropriate classification could well assist researchers in their endeavours properly to assess and understand the role of primary health care, establish the optimum size and nature of specialized mental health manpower, evaluate treatment outcome in the primary health care setting and determine the cost and cost effectiveness of such services. A number of proposals have already been put forward with the aim of developing a new classification system. The World Organization of National Colleges, Academies and Academic Associations of General Practitioners/Family Physicians (WONCA) have developed the International Classification of Health Problems in Primary Care [8]. This consists of detailed classificators and problem lists specially designed for use in primary health care. In the United States, the National Center for Health
109 Statistics is using a Reason for Visit Classification (RVC) to code reasons for visit data from the National Ambulatory Care Survey (NAMCS) [9]. However, neither of these developments has proved wholly satisfactory [10]. In November 1979, therefore, WHO organized an international conference, co-sponsored by the Rockefeller Foundation, on Psychosocial Factors Affecting Health Assessment, Classification and Utilization. The outcome was a provisional classification of social and psychological health care problems relating to primary health care contacts and recommendations concerning the development of a triaxial recording procedure of health care problems in this setting [11]. WHO then developed a detailed research proposal for field testing the triaxial classification and recording procedure internationally. Within this context a case vignette exercise was conducted to test the proposed classification. The specific aims of this project were: (a) to develop further the classification of the social and psychological components to the triaxial classification related to primary health care contacts, particularly by adapting them to different socio-cultural contacts; and (b)to test the feasibility of triaxial recording by primary health care workers of problems presented during primary health care encounters.
Methods and instruments of study
The classifications tested The classifications of psychological and social problems to be tested during the case vignette phase of the study are shown in Fig. 1 which is the actual form used for rating these in the subsequent exercise. The classifications emerged from a series of international meetings organized under WHO auspices and attended by primary health care physicians, psychiatrists, social researchers, statisticians and experts currently working in the area of disease classification and health problem recording. The classifications were accompanied by glossaries explaining the meaning of individual problem categories. These glossaries were prepared at WHO Headquarters following discussions and drawing on experience with glossaries developed in relation to other studies of mental health. While the research effort was mainly directed towards the question of recording psychological and social problems, their complex inter-relationship with physical problems presented in primary health care settings demand that a triaxial approach to problem recording be implemented. It was decided after discussions with primary health care workers to record physical problems using ICD or ICHPCC-2 classifications. Conditions pertaining to Section 5 (Mental Disorders) of ICD or ICHPCC-2 were not to be reported on the physical axis however but on the psychological axis or tbe proposed triaxial classification. Each of the seven participating research teams was requested to produce a set of 50 case vignettes and to have these rated by various categories of primary health care workers (the "local" rating exercise). The set of vignettes were then translated into English, where necessary, and
sent to WHO/HQ. The WHO research team then selected a representative sample of 50 case vignettes (the "international" set) from the 340 vignettes 1 received, and requested each local research group to have this set rated by the various categories of primary health care workers (the "international" rating exercise). For the purpose of the case vignette study, the following definitions of psychological and social problems presenting in the primary care setting were provided:
A psychological problem is: feelings, thoughts and/or behaviour, either recognized as morbid or out of proportion to any concurrent events or difficulties, that give distress to the individual concerned, or are a source of complaint by others, or are a real or potential hazard to his/her health. A socialproblems is: a feature or change in an individual's relationships with other persons, groups or organizations, or with his/her physical environment, which may be harmful to his/her general well-being, or to his/her physical or mental health. It is important to note that the case vignettes were constructed by primary health care workers in the participating countries. The guidelines provided explained that the ideal case vignette should provide a succinct encapsulation of a primary care consultation. Case vignettes were constructed to include, where possible, typical (i. e., commonly occurring) and less typical problems presenting at the participating centres. They included examples of problems that appeared to be primarily psychological, social or physical as well as problems that appeared to be a complex mixture of problems from two or three axes. They were constructed so as to represent a reasonable cross-section of primary health care attenders in terms of age, sex and social status. The international set of vignettes were selected to represent a reasonable crosssection of problems presenting at the 7 participating centres. Accordingly 7 examples were selected from vignettes prepared by Brazil, Canada, Colombia, Malaysia, Thailand and the US, and 8 from those prepared by Zambia. The case vignette was to provide a clear picture of the patient and the patient's current circumstances, a concise account of the problem or problems that led to the consultation and any findings that appeared relevant for subsequent action (or appropriate inaction). An account of the more detailed guidelines concerning the construction of the case vignettes can be obtained from WHO Headquarters. For an example of a typical case vignette, its rating and a discussion of the rating, see Appendix A.
The process of rating the case vignettes The national research centres were instructed to have the case vignettes (both the "local" and "international" sets) rated by at least four representatives of each professional group included in the study (i. e., physicians, nurses, medical assistants). In the national case vignette rating exercises, rating took place in two stages. First, the raters rated One of the centresconstructed40 Case Vignettes.
110 RATING FORM: INTERNATIONAL SET OF CASE VIGNETTES
08 09 RATER NAME:
11 12
10 VIGNETTE NO.
UC-1I I
PSYCHOLOGICAL PROBLEM(S)
(PLEASE CIRCLE APPROPRIATE NUMBER(S))
01
02 03 04 05
06 07 6
PO0
No psychological problem contributed to the visit Symptoms, Conditions or Diagnosis
P01 P02 P03 P04 P05 P06 P07 P08 P09 P10
feeling anxious, tense or nervous feeling depressed feeling irritable or angry feeling restless or agitated disturbance of sleep feeling tired, apathetic or listless feeling dissatisfied disturbances of memory or concentration suicidal ideas, impulses or acts delusions, hallucinations or other psychotic phenomena social withdrawal substance related problems specify disorders of personality
Pll P12 P13
SOCIAL PROBLEM(S)
No social problem contributing to the visit Social Problems
S01 S02 S03 S04 S05
conjugal/marital problems parent-child problems family disruption other problems of family relations problems of caring for sick, disabled or aged person housing problems change in residence other non-family interpersonal problems social isolation
S09
P17 P18 P19 P20 P21 P22 P23 P24 P25
sexual problems eating problems psychophysiologicalproblems (includes physical symptoms exacerbated by psychological factors) learning problems emotional problems in childhood conduct problems in childhood organic psychotic schizophrenic and paranoid affective psychoses other non-organic psychoses mental retardation other: (specify)
FT-
(PLEASE CIRCLE APPROPRIATE NUMBER(S))
SO0
S06 S07 S08
(leave blank) P14 P15 P16
(leave blank) $10 $11 $12 $13 $14 $15 $16 S17
financial problems problems with educational system occupational problems legal problems personal circumstances impeding access to health care environmental circumstances hazardous to health problems arising out of social or cultural practices or belief systems other: (specify)
PHYSICAL PROBLEM (S) Please circle ICHPPC2 (attached sheets) codes if available in the appropriate chapter. Any diagnosis not listed should be entered in writing under "Other" in the supplementary classification chapter. This will be coded later at MCV. If necessary, you may list or circle up to four diagnoses.
37 41 45
PLEASE UNDERLINE SINGLE MOST IMPORTANT PSYCHOLOGICAL, SOCIAL OR PHYSICAL PROBLEM
49
COMMENTS: Fig. 1. WHO collaborative projects on recording health problems triaxially: the physical, psychological and social components of primary health care contacts 10 of t h e 50 case v i g n e t t e s a n d t h e i r results w e r e discussed. W h i l e t h e ratings w e r e n o t a l t e r e d , t h e s e discussions ena b l e d i n f o r m a t i o n to b e e x c h a n g e d o n h o w the p r o b l e m c a t e g o r i e s w e r e u n d e r s t o o d a n d i n t e r p r e t e d b y t h e raters. T h e r a t e r s t h e n p r o c e e d e d to r a t e the r e m a i n i n g 40 case
vignettes. In the case of t h e i n t e r n a t i o n a l case v i g n e t t e rating exercise, t h e 50 v i g n e t t e s w e r e r a t e d w i t h o u t discussion. W i t h r e g a r d to the r a t i n g o f p h y s i c a l p r o b l e m s , t h e r a t e r s w e r e i n s t r u c t e d to r e c o r d s o m a t i c p r o b l e m s a c c o r d i n g to
111 Table L Number and professional category of health workers rating the national (N) and the international (I) case vignettes by centre
Doctors
Brazil Canada Columbia Malaysia Thailand USA Zambia
N 6 6 2 1 2 5 3
Nurses I 5 5 4 4 4 93 4
Auxiliaries
N
I 5
5 2
4 4
N 6
I
Medical assistant N I
4
4 2
7
7
Midwives N
I
4 4
Total no of raters N I 12 10 6 5 6 8 6 8 6 12 5 9 17 8
In USA, two groups of doctors were involved: 5 more and 4 less experienced
the established practice at the health centre. ICHPCC-2 codes were then entered centrally by a member of the research team (unless, as in the case of Canada, the raters were familiar with ICHPCC-2, when the coding was done by the individual raters). Table 1 shows the number and professional category of health workers involved in the nation al (N) and international (I) rating exercise respectively. While the rating exercise based on the sets of locally produced case vignettes allowed the assessment of the intracentre reliability of instruments and raters, the "international" case vignette exercise attempted to shed light on the potential cross-cultural applicability of the classification. In order to study this issue further, the senior investigators in all seven centres were requested to produce a "consensus" rating of each of the 50 "international" case vignettes. This was undertaken after the primary health care workers had completed their rating of the international case vignettes and the results had been dispatched to W H O / H Q . The senior investigators in each centre were requested to discuss each of the international case vignettes and reach an agreement on rating on all three axes. These "consensus" ratings were then compared and analyzed. A more detailed description of the training of the primary health care workers and the actual implementation of the local and international case vignette rating exercise is provided by Sartorius and Gulbinat [12].
Statistical methods of assessment The basis of any reliability assessment is the agreement among raters in their judgement. Three types of problem categories can be distinguished: (a) those on which there is complete agreement that the problem was present; (b) those on which there is complete agreement that the problem is absent, and (c) those on which there is disagreement, or, equivalently, partial agreement. While, theoretically, no statement is possible as to who is right or wrong as no truly objectives standard for comparison is available, in the international case vignette exercise the "consensus" rating provided by the senior investigators served such a purpose. In addition to this "consensus" rating comparison, a "majority rating" measure was used, and the individual rater's rating reliability was calculated by comparing it to the majority rating obtained in the centre. The majority rating is defined to be the rating of 50 % or more of the raters on a particular problem category.
Most raters, intuitively, feel that it is easier to assess the absence of a problem category than its presence - and, hence, that agreement on symptom absence is expected to be higher than on symptom presence. Accordingly, in the analyses, three types of reliability coefficients were calculated: (a) agreement rates based on all problem categories (regardless whether the majority ratings were positive or negative); (b) agreement rates based on problem categories only if these were, according to the majority judgement, or the consensus judgement, positive (or present); (c) agreement rates based on problem categories only if these were, according to the majority or consensus judgement, negative (or absent). The tables presented and the interpretation of the results refer to (b) i. e., the agreement based on problem categories only if these were, according to the majority judgement, or the consensus judgement, positive (or present). Agreement on negative/or absent categories was much higher but may be less telling. In order to compute the "agreement rate", an "agreement score" was defined to assume the numerical value 1, if the rater agrees (on a particular problem category pertaining to a particular vignette) with the majority rating, and the value 0, if he disagrees with the majority rating. The arithmetic mean of those scores of a group of raters is the agreement rate of this rater group in respect of the particular problem category. By definition, the lower limit of this rate is 0.5 for an even number of raters, and (n + 1)/2n for an odd number of raters. Apart from a factor 100, this average agreement rate is identical with the percent agreement with the majority rating. If the same rater assesses more than one vignette (e. g., each of the 50 vignettes of the national set), separately for each problem category, the average score of agreement with the respective majority ratings may be defined as the arithmetic mean of the agreement scores for the individual vignettes. In this situation the range of possible values is 0 in case of complete disagreement and 1 in case of complete agreement with the majority rating. Results
Social problems The reliability coefficients for each category in the social problems classification for the national and international case vignette rating exercises and for the consensus exer-
112 Table 2. Interrater agreement by social problems and country local vignette exercise
~D
Brazil Canada Columbia (A) Malaysia Thailand USA Zambia Total
0.84 0.70 0.81 0.81 0.67 0.84 0.63 0.76
0.74 0.69 0.84 0.78 0.86 0.78
0.75 0.75 0.65 0.63 0.79 0.70 0.61 0.70
0.83
0.68
0.83 0.67 0.70
0.83 0.50 0.72
0.76
0.68
0.81 1.00 0.75 0.90 0.78 0.67 0.67 0.58 0.50 0.73
0.83 0.80 0.85 0.78 0.81 0.85 0.66 0.80
0.80 0.67 0.64
0.73 0.67 0.63 0.67 0.73 0.60 0.79 0.67 0.73 0.67 0.67 0.72 0.63 0.65 0.73 0.67 0.73 0.64
0.75 0.73 0.72 0.80 0.69 0.71 0.81 0.73 0.80 0.73 0.65 0.74 0.72 0.72
Table 3. Interrater agreement by psychological problem category and country. International vignette exercies 9
9 9
. ~ . f i . ~
+..
8
~9
o~
=
~
-~
~
c~
v
'
~
O
0~
Soc Psychiatry Psychiatr Epidemiol (1992)27:108-116
PsychiatricEpidemiology
9 Springer-Verlag 1992
A triaxial classification of health problems presenting in primary health care* A World Health Organization multi-centre study A. Clare 1, W. Gulbinat 2, and N. Sartorius 2
St. Patrick's Hospital, Dublin, Ireland 2Division of Mental Health, WHO, Geneva Accepted: December 30, 1991
Summary. Over the past two decades, research in a variety of countries and settings has revealed the extent to which psychological morbidity and social problems constitute an important proportion of primary health care contacts, either in their own right or in association with physical illhealth. The development of appropriate responses, medical and non-medical, to such problems is, in part, hindered by the relatively low level of awareness concerning the significance of such problems and by currently inappropriate methods of describing and classifying health care problems in the primary care setting. There is a pressing need for an appropriate, relatively simple and flexible classification or list of problems presenting in primary care. This study describes the development, under W H O auspices and in an international setting, of a triaxial classification of health problems and the testing and modification of the classification by means of an international case vignette rating exercise. A psychological problem list and a social problem list, together with detailed glossaries, suitably modified in the light of experience derived from the study, are described, and proposals enumerated of possible ways in which these lists might be further developed and used in research and clinical undertakings in primary care.
* This paper is based on the data and experience obtained during the participation of the authors in the WHO Project on Recording Health Problems Triaxially: the Physical, Psychological and Social Components of Primary Health Care Contacts, a project sponsored by the World Health Organization, the Rockefeller Foundation, the US National Institute of Mental Health (Grant ROIMH35753) and the participating field reseach centres.
Collaborating investigators: M. Bass, University of Western Ontario, London, Canada; E. Bushello, Vila Sao Jos6 do Murialdo Health Centre, Porto Alegre, Brazil; C. Climent, Universidad de Valle, Cali, Colombia; U. Laksanavicharn, Ministry of Health, Bangkok, Thailand; K.Selliah, Penang Hospital, Penang, Malaysia; R. Serpell, University of Zambia, Lusaka, Zambia. For a detailed discussion of the conceptual, methodological and statistical issues involved, readers are referred to Psychosocial Factors Affecting Health [1].
There is considerable evidence to indicate a close inter-relationship between psychiatric ill-health and social problems in the setting of primary health care [2, 3]. Studies have delineated the impact of life events [4], family and interpersonal stresses, and difficulties and deficiencies in social support [5] on the psychological and physical health of patients in this setting. Preliminary attempts indicate that modifying treatment to take account of the role of social problems may have important clinical implications [6]. The primary health care sector is in great need of a classification system of mental disorder diagnoses and psychosocial problems which is uncomplicated, appropriate and concurrently compatible with other more complex disease classification systems, such as the International Classification of Diseases [7]. The development and acceptance of such a system would increase the likelihood that problems for which patients seek medical care would be more accurately described. Implementation of such a classification system also holds out the possibility that clinicians would be benefitted in the management of their patients while policy makers would be assisted in planning services in response to medical and social need and in allocating resources. Educators of physicians and of professionals working alongside them in primary care might be encouraged to adapt current training to take account of primary health care realities. A more appropriate classification could well assist researchers in their endeavours properly to assess and understand the role of primary health care, establish the optimum size and nature of specialized mental health manpower, evaluate treatment outcome in the primary health care setting and determine the cost and cost effectiveness of such services. A number of proposals have already been put forward with the aim of developing a new classification system. The World Organization of National Colleges, Academies and Academic Associations of General Practitioners/Family Physicians (WONCA) have developed the International Classification of Health Problems in Primary Care [8]. This consists of detailed classificators and problem lists specially designed for use in primary health care. In the United States, the National Center for Health
109 Statistics is using a Reason for Visit Classification (RVC) to code reasons for visit data from the National Ambulatory Care Survey (NAMCS) [9]. However, neither of these developments has proved wholly satisfactory [10]. In November 1979, therefore, WHO organized an international conference, co-sponsored by the Rockefeller Foundation, on Psychosocial Factors Affecting Health Assessment, Classification and Utilization. The outcome was a provisional classification of social and psychological health care problems relating to primary health care contacts and recommendations concerning the development of a triaxial recording procedure of health care problems in this setting [11]. WHO then developed a detailed research proposal for field testing the triaxial classification and recording procedure internationally. Within this context a case vignette exercise was conducted to test the proposed classification. The specific aims of this project were: (a) to develop further the classification of the social and psychological components to the triaxial classification related to primary health care contacts, particularly by adapting them to different socio-cultural contacts; and (b)to test the feasibility of triaxial recording by primary health care workers of problems presented during primary health care encounters.
Methods and instruments of study
The classifications tested The classifications of psychological and social problems to be tested during the case vignette phase of the study are shown in Fig. 1 which is the actual form used for rating these in the subsequent exercise. The classifications emerged from a series of international meetings organized under WHO auspices and attended by primary health care physicians, psychiatrists, social researchers, statisticians and experts currently working in the area of disease classification and health problem recording. The classifications were accompanied by glossaries explaining the meaning of individual problem categories. These glossaries were prepared at WHO Headquarters following discussions and drawing on experience with glossaries developed in relation to other studies of mental health. While the research effort was mainly directed towards the question of recording psychological and social problems, their complex inter-relationship with physical problems presented in primary health care settings demand that a triaxial approach to problem recording be implemented. It was decided after discussions with primary health care workers to record physical problems using ICD or ICHPCC-2 classifications. Conditions pertaining to Section 5 (Mental Disorders) of ICD or ICHPCC-2 were not to be reported on the physical axis however but on the psychological axis or tbe proposed triaxial classification. Each of the seven participating research teams was requested to produce a set of 50 case vignettes and to have these rated by various categories of primary health care workers (the "local" rating exercise). The set of vignettes were then translated into English, where necessary, and
sent to WHO/HQ. The WHO research team then selected a representative sample of 50 case vignettes (the "international" set) from the 340 vignettes 1 received, and requested each local research group to have this set rated by the various categories of primary health care workers (the "international" rating exercise). For the purpose of the case vignette study, the following definitions of psychological and social problems presenting in the primary care setting were provided:
A psychological problem is: feelings, thoughts and/or behaviour, either recognized as morbid or out of proportion to any concurrent events or difficulties, that give distress to the individual concerned, or are a source of complaint by others, or are a real or potential hazard to his/her health. A socialproblems is: a feature or change in an individual's relationships with other persons, groups or organizations, or with his/her physical environment, which may be harmful to his/her general well-being, or to his/her physical or mental health. It is important to note that the case vignettes were constructed by primary health care workers in the participating countries. The guidelines provided explained that the ideal case vignette should provide a succinct encapsulation of a primary care consultation. Case vignettes were constructed to include, where possible, typical (i. e., commonly occurring) and less typical problems presenting at the participating centres. They included examples of problems that appeared to be primarily psychological, social or physical as well as problems that appeared to be a complex mixture of problems from two or three axes. They were constructed so as to represent a reasonable cross-section of primary health care attenders in terms of age, sex and social status. The international set of vignettes were selected to represent a reasonable crosssection of problems presenting at the 7 participating centres. Accordingly 7 examples were selected from vignettes prepared by Brazil, Canada, Colombia, Malaysia, Thailand and the US, and 8 from those prepared by Zambia. The case vignette was to provide a clear picture of the patient and the patient's current circumstances, a concise account of the problem or problems that led to the consultation and any findings that appeared relevant for subsequent action (or appropriate inaction). An account of the more detailed guidelines concerning the construction of the case vignettes can be obtained from WHO Headquarters. For an example of a typical case vignette, its rating and a discussion of the rating, see Appendix A.
The process of rating the case vignettes The national research centres were instructed to have the case vignettes (both the "local" and "international" sets) rated by at least four representatives of each professional group included in the study (i. e., physicians, nurses, medical assistants). In the national case vignette rating exercises, rating took place in two stages. First, the raters rated One of the centresconstructed40 Case Vignettes.
110 RATING FORM: INTERNATIONAL SET OF CASE VIGNETTES
08 09 RATER NAME:
11 12
10 VIGNETTE NO.
UC-1I I
PSYCHOLOGICAL PROBLEM(S)
(PLEASE CIRCLE APPROPRIATE NUMBER(S))
01
02 03 04 05
06 07 6
PO0
No psychological problem contributed to the visit Symptoms, Conditions or Diagnosis
P01 P02 P03 P04 P05 P06 P07 P08 P09 P10
feeling anxious, tense or nervous feeling depressed feeling irritable or angry feeling restless or agitated disturbance of sleep feeling tired, apathetic or listless feeling dissatisfied disturbances of memory or concentration suicidal ideas, impulses or acts delusions, hallucinations or other psychotic phenomena social withdrawal substance related problems specify disorders of personality
Pll P12 P13
SOCIAL PROBLEM(S)
No social problem contributing to the visit Social Problems
S01 S02 S03 S04 S05
conjugal/marital problems parent-child problems family disruption other problems of family relations problems of caring for sick, disabled or aged person housing problems change in residence other non-family interpersonal problems social isolation
S09
P17 P18 P19 P20 P21 P22 P23 P24 P25
sexual problems eating problems psychophysiologicalproblems (includes physical symptoms exacerbated by psychological factors) learning problems emotional problems in childhood conduct problems in childhood organic psychotic schizophrenic and paranoid affective psychoses other non-organic psychoses mental retardation other: (specify)
FT-
(PLEASE CIRCLE APPROPRIATE NUMBER(S))
SO0
S06 S07 S08
(leave blank) P14 P15 P16
(leave blank) $10 $11 $12 $13 $14 $15 $16 S17
financial problems problems with educational system occupational problems legal problems personal circumstances impeding access to health care environmental circumstances hazardous to health problems arising out of social or cultural practices or belief systems other: (specify)
PHYSICAL PROBLEM (S) Please circle ICHPPC2 (attached sheets) codes if available in the appropriate chapter. Any diagnosis not listed should be entered in writing under "Other" in the supplementary classification chapter. This will be coded later at MCV. If necessary, you may list or circle up to four diagnoses.
37 41 45
PLEASE UNDERLINE SINGLE MOST IMPORTANT PSYCHOLOGICAL, SOCIAL OR PHYSICAL PROBLEM
49
COMMENTS: Fig. 1. WHO collaborative projects on recording health problems triaxially: the physical, psychological and social components of primary health care contacts 10 of t h e 50 case v i g n e t t e s a n d t h e i r results w e r e discussed. W h i l e t h e ratings w e r e n o t a l t e r e d , t h e s e discussions ena b l e d i n f o r m a t i o n to b e e x c h a n g e d o n h o w the p r o b l e m c a t e g o r i e s w e r e u n d e r s t o o d a n d i n t e r p r e t e d b y t h e raters. T h e r a t e r s t h e n p r o c e e d e d to r a t e the r e m a i n i n g 40 case
vignettes. In the case of t h e i n t e r n a t i o n a l case v i g n e t t e rating exercise, t h e 50 v i g n e t t e s w e r e r a t e d w i t h o u t discussion. W i t h r e g a r d to the r a t i n g o f p h y s i c a l p r o b l e m s , t h e r a t e r s w e r e i n s t r u c t e d to r e c o r d s o m a t i c p r o b l e m s a c c o r d i n g to
111 Table L Number and professional category of health workers rating the national (N) and the international (I) case vignettes by centre
Doctors
Brazil Canada Columbia Malaysia Thailand USA Zambia
N 6 6 2 1 2 5 3
Nurses I 5 5 4 4 4 93 4
Auxiliaries
N
I 5
5 2
4 4
N 6
I
Medical assistant N I
4
4 2
7
7
Midwives N
I
4 4
Total no of raters N I 12 10 6 5 6 8 6 8 6 12 5 9 17 8
In USA, two groups of doctors were involved: 5 more and 4 less experienced
the established practice at the health centre. ICHPCC-2 codes were then entered centrally by a member of the research team (unless, as in the case of Canada, the raters were familiar with ICHPCC-2, when the coding was done by the individual raters). Table 1 shows the number and professional category of health workers involved in the nation al (N) and international (I) rating exercise respectively. While the rating exercise based on the sets of locally produced case vignettes allowed the assessment of the intracentre reliability of instruments and raters, the "international" case vignette exercise attempted to shed light on the potential cross-cultural applicability of the classification. In order to study this issue further, the senior investigators in all seven centres were requested to produce a "consensus" rating of each of the 50 "international" case vignettes. This was undertaken after the primary health care workers had completed their rating of the international case vignettes and the results had been dispatched to W H O / H Q . The senior investigators in each centre were requested to discuss each of the international case vignettes and reach an agreement on rating on all three axes. These "consensus" ratings were then compared and analyzed. A more detailed description of the training of the primary health care workers and the actual implementation of the local and international case vignette rating exercise is provided by Sartorius and Gulbinat [12].
Statistical methods of assessment The basis of any reliability assessment is the agreement among raters in their judgement. Three types of problem categories can be distinguished: (a) those on which there is complete agreement that the problem was present; (b) those on which there is complete agreement that the problem is absent, and (c) those on which there is disagreement, or, equivalently, partial agreement. While, theoretically, no statement is possible as to who is right or wrong as no truly objectives standard for comparison is available, in the international case vignette exercise the "consensus" rating provided by the senior investigators served such a purpose. In addition to this "consensus" rating comparison, a "majority rating" measure was used, and the individual rater's rating reliability was calculated by comparing it to the majority rating obtained in the centre. The majority rating is defined to be the rating of 50 % or more of the raters on a particular problem category.
Most raters, intuitively, feel that it is easier to assess the absence of a problem category than its presence - and, hence, that agreement on symptom absence is expected to be higher than on symptom presence. Accordingly, in the analyses, three types of reliability coefficients were calculated: (a) agreement rates based on all problem categories (regardless whether the majority ratings were positive or negative); (b) agreement rates based on problem categories only if these were, according to the majority judgement, or the consensus judgement, positive (or present); (c) agreement rates based on problem categories only if these were, according to the majority or consensus judgement, negative (or absent). The tables presented and the interpretation of the results refer to (b) i. e., the agreement based on problem categories only if these were, according to the majority judgement, or the consensus judgement, positive (or present). Agreement on negative/or absent categories was much higher but may be less telling. In order to compute the "agreement rate", an "agreement score" was defined to assume the numerical value 1, if the rater agrees (on a particular problem category pertaining to a particular vignette) with the majority rating, and the value 0, if he disagrees with the majority rating. The arithmetic mean of those scores of a group of raters is the agreement rate of this rater group in respect of the particular problem category. By definition, the lower limit of this rate is 0.5 for an even number of raters, and (n + 1)/2n for an odd number of raters. Apart from a factor 100, this average agreement rate is identical with the percent agreement with the majority rating. If the same rater assesses more than one vignette (e. g., each of the 50 vignettes of the national set), separately for each problem category, the average score of agreement with the respective majority ratings may be defined as the arithmetic mean of the agreement scores for the individual vignettes. In this situation the range of possible values is 0 in case of complete disagreement and 1 in case of complete agreement with the majority rating. Results
Social problems The reliability coefficients for each category in the social problems classification for the national and international case vignette rating exercises and for the consensus exer-
112 Table 2. Interrater agreement by social problems and country local vignette exercise
~D
Brazil Canada Columbia (A) Malaysia Thailand USA Zambia Total
0.84 0.70 0.81 0.81 0.67 0.84 0.63 0.76
0.74 0.69 0.84 0.78 0.86 0.78
0.75 0.75 0.65 0.63 0.79 0.70 0.61 0.70
0.83
0.68
0.83 0.67 0.70
0.83 0.50 0.72
0.76
0.68
0.81 1.00 0.75 0.90 0.78 0.67 0.67 0.58 0.50 0.73
0.83 0.80 0.85 0.78 0.81 0.85 0.66 0.80
0.80 0.67 0.64
0.73 0.67 0.63 0.67 0.73 0.60 0.79 0.67 0.73 0.67 0.67 0.72 0.63 0.65 0.73 0.67 0.73 0.64
0.75 0.73 0.72 0.80 0.69 0.71 0.81 0.73 0.80 0.73 0.65 0.74 0.72 0.72
Table 3. Interrater agreement by psychological problem category and country. International vignette exercies 9
9 9
. ~ . f i . ~
+..
8
~9
o~
=
~
-~
~
c~
v
'
~
O
0~
