A Method for Analyzing Videotaped Genetic Counseling Sessions.

P1: FQJ Journal of Genetic Counseling [jgc]

HS071-48

March 7, 2000

13:46

Style file version Nov. 19th, 1999

Journal of Genetic Counseling, Vol. 9, No. 2, 2000

A Method for Analyzing Videotaped Genetic Counseling Sessions1 Alexander Liede,2,3 Lauren Kerzin-Storrar,2 and David Craufurd2

This study describes the development and evaluation of a multi-item scale for analyzing the genetic counseling process, the Manchester Observation Code (MOC) for genetic counseling. The instrument is specific to the field of genetic counseling and is designed for analysis of the communication between counselor and client. Coding is done directly from videotaped sessions. Because communication is the means by which genetic counseling is accomplished, the method measures four relevant components of communication: (1) grammatical form, (2) purpose, (3) subject, and (4) cue source. The instrument enables an observer to code the counselor’s statements into these four components. Three videotaped sessions were used to measure interrater reliability, or the consistency of rating for each of the four communication domains using this method. Three videotaped sessions were also used to measure test-retest reliability, or the consistency of the designed method from one time to another. A total of 21 videotaped sessions were tested using the method. A statistical measure of reliability established consistency of the designed method; Cohen’s kappa yielded 0.7 for interrater reliability and 0.79 for test-retest reliability. These findings suggest this instrument may be used to identify important elements of the genetic counseling process. KEY WORDS: Genetic counseling; process; audit; analysis; MOC; Manchester Observation Code.

INTRODUCTION Our understanding of the process of genetic counseling is somewhat limited. Studies which have examined the efficacy of genetic counseling in terms 1 Presented

at the American Society of Human Genetics Annual Meeting, Baltimore, October 29– November 1, 1997. Liede A, Kerzin-Storrar L, Craufurd D. A method designed for analysing the genetic counseling process. Am. J. Hum. Genet. 61S:A189; 1092. 2 Division of Clinical Genetics, University of Manchester, St. Mary’s Hospital, Manchester, UK. 3 Address correspondence to Alexander Liede, Ph.D. candidate, University of Toronto, 790 Bay Street, Suite 750A, Toronto, ON M5G 1N8, Canada; Fax: (416) 351-3767; e-mail: [email protected]. 117 C 2000 National Society of Genetic Counselors, Inc. 1059-7700/00/0400-0117$18.00/1 °


118

HS071-48

March 7, 2000

13:46


Liede, Kerzin-Storrar, and Craufurd

of patients’ recall and understanding of relevant medical and genetic information (Michie et al., 1997; Benjamin et al., 1993; Faulkner, 1995; Pearn, 1973; Seidenfeld and Antley, 1981) and reproductive decision making (Frets et al., 1990; Kessler, 1989) have extended our understanding of the process of genetic counseling. Marteau et al. (1994a, 1994b) surveyed those practicing genetic counseling to examine differences in approach to more complex situations. It may be that different counseling approaches are equally effective in achieving certain goals such as education (Frets et al., 1990; Kessler, 1989). Measures of the needs and satisfaction of the client undergoing genetic counseling are perhaps the most suitable measures for purposes of audit. However, there have been few attempts at measuring client satisfaction (Shiloh et al., 1990) or the information needs of clients attending specialized clinics (Hallowell et al., 1997). It is debatable whether the information alone or the psychotherapeutic aspects or both are helpful to clients. Similarly, research has not shown that clients benefit from a nondirective approach. Despite this lack of evidence, the intention of nondirectiveness remains in place mainly because of the grave concerns toward adversely influencing personal autonomy or reproductive decision making of clients. Nondirectiveness is certainly not the only goal of genetic counseling, but it continues to fuel debate in the field of genetics (Bartels et al., 1997). Armstrong et al. (1998) likened the genetic counseling process to the process of revealing the genetic identity of the client. Kessler (1979, 1982, 1989, 1990, 1992a, 1992b, 1997) and Clarke (1990, 1991, 1993; Clarke et al., 1996) have drawn attention to the need for research into the genetic counseling process in order to advance our understanding of these issues. It follows that as we further our understanding of the process, our understanding of directiveness, psychological intervention, client satisfaction, and decision making as well as other more subtle components of genetic counseling such as nonverbal interactions may be enhanced. Previous research has evaluated how communication during medical visits affects patient outcomes such as compliance, recall, and satisfaction. Clinical genetics and genetic counseling rely greatly on communication. Thus, an important component of audit in these fields is one which examines this communicative interaction. Clarke (1990) called for a better method of audit in clinical genetics to replace the quantitative approach, which used the count of abortions procured on genetic grounds. Communication may be measured by the exchange of words between counselor and client, such as a statement-by-statement approach. The instrument described in this study used precisely this approach to analyze the process of genetic counseling. The intent was to generate a scoring instrument specific for the analysis of communication in genetic counseling. A variety of techniques have been used to examine aspects of communication between medical professional and patient. These methods have been developed, modified, and validated in a variety of disciplines ranging from medical education (Maguire et al., 1978; Preven et al., 1986) and epidemiology (Edwards et al., 1994) to psychiatry and general practice (Inui et al., 1982; Carter et al., 1982; Wasserman and Inui, 1983; Bensing and Sluijs,


HS071-48

March 7, 2000

Manchester Observation Code for Genetic Counseling

13:46


119

1985; Bowman et al., 1992; Gask et al., 1987, 1988, 1989, 1991; Goldberg et al., 1993, 1980a, 1980b; Kaaya et al., 1992; Preven et al., 1986; Street, 1992; Roter et al., 1997). These methods were applicable, however, to a limited extent. Genetic counselors may exhibit psychiatric interviewing skills as they provide patients with genetic information, information that may have life-altering consequences. In the early stages of design, methods described in the literature for teaching psychiatric interviewing skills to general practitioners were modified, enhanced, and tested (Gask et al., 1987, 1988, 1989, 1991; Goldberg et al., 1993, 1980a, 1980b; Kaaya et al., 1992). The evolved product is presented here as the Manchester Observation Code for the analysis of videotaped sessions in genetic counseling.

METHODS The objectives of this study were (1) to develop a method for the analysis of communication during genetic counseling and (2) to assess the reliability of the resulting instrument. The instrument will be referred here as the Manchester Observation Code (MOC) for genetic counseling. The MOC codes each phrase or complete thought expressed by the counselor. Using the classification system depicted in Table I, the counselor’s “statements”, not those of the client, are classified into four domains of communication namely, (1) grammatical form, (2) function or purpose, (3) subject and (4) cue source. The MOC is depicted in Table I, and instructions for use are provided in the Appendix. Coding is done directly from videotaped counseling sessions. Videotapes were selected, rather than audiotapes, as they allow for the identification of nonverbal cues. The method may be used for various purposes. For example, videotapes might be analyzed for training genetic counseling students by identifying specific patterns of topics of discussion during a session. Alternatively, a specific research question may be studied such as the amount or proportion of time devoted to the discussion of psychological issues or the number of empathic statements uttered by the counselor during a counseling session after termination of a trisomy 21 pregnancy. In these instances, the domain “subject” may be of interest to the investigator. For example, Video 5 (Table II) is one of 21 videotaped genetic counseling sessions tested in the development of the instrument described here. Video 5a represents the initial 10 minutes of the 50-minute counseling session dealing with posttermination of a trisomy 21 fetus, detected by amniocentesis. The counseling session, particularly the first 10 minutes, was laden with psychological issues for the couple. On average, 12 (range 9–15) statements were labeled by four raters for statistical analysis. Of these main statements identified, 8 statements dealt with psychological issues; the remaining dealt with social and medical issues for the clients. Nine minutes of the 10-minute video, (90% of the time) were devoted to psychological issues. Most of these statements were counselor-initiated, and approximately 67% of these were labeled as psychological. Video 5 is discussed in further detail in “Results.”


HS071-48

March 7, 2000

120

13:46


Liede, Kerzin-Storrar, and Craufurd Table I. Classification of Counselor’s Statements Manchester Observation Code (MOC) for Genetic Counseling

Domains

Classification code

1. Form 1. 2. 3. 4.

Statement Open question Closed question Directive statement

1. 2. 3. 4. 5. 6.

Making conversation Eliciting information Giving information Giving advice Patient-centered behavior Exploring behavior

1. 2. 3. 4. 5. 6. 7.

Trivial About session Family history Medical Genetics Psychological Social/family factors

Examples “Your risk is . . .” “How are your children?” “Are you worried about this?” “Tell me what you know about . . .”

2. Function Checking-out questions “Some couples in your situation . . .” Empathy: “It must be a difficult time for you.” Attempt to instigate self-reflection in client: “Although . . . I sense that you are more anxious about . . .”

3. Subject Weather Here and now Medical history Inheritance Personal issues, coping

4. Cue source 1. Patient-initiated: verbal 2. Patient-initiated: nonverbal 3. Counselor-initiated

Client asks question Client looks distressed Counselor follows own agenda

The following considerations were given for the evaluation tool: 1. It should be specific to genetic counseling interviews. 2. For its application in research, it should be designed to fit the nature of the genetic counseling process (previously not attempted). 3. It should classify counselor’s statements or behaviors and determine their origin and function. 4. It should comprise primarily a multi-item rating of counselor’s statements. 5. It should identify infrequent but vital exchanges between counselor and client. 6. It should identify the sequence as well as the frequency of events. 7. It should be an objective method of assessment and exhibit adequate levels of interrater and test-retest reliability. 8. It should be relatively short and applicable to one or two viewings of a genetic counseling session. 9. It should be amenable for use across national and cultural boundaries (i.e., should exhibit cross-cultural validity).


HS071-48

March 7, 2000

13:46



121

Table II. Characteristics of Videotaped Sessions Total Scoring No. No. time time No. main No. clients raters Video (min) (min) statements counselors (+ children) (+ repeat) 1

31

50

9

1

1

1

2

24

50

21

1

1

5

3

46

60

23

1

2

1+1

4

14

30

11

2

1

6

5a

10

20

12

1

2

4+4

5b 6 7

50 11 34

70 30 60

13 15 24

1 1 1

2 2 2

1 1+1 1

8

36

60

9

3

1

1

9

36

60

13

3

1

1

10

65

90

25

2

1

1

11 12 13

30 16 38

40 29 50

8 20 13

2 2 2

2 1+3 2

1 1 1

14 15

34 33

45 45

15 12

2 2

2+2 2

1 1

16

22

30

8

2

1

1

17

34

70

11

2

2

1

18

25

35

6

2

2

1

19 20 21 Mean

63 27 51 33

72 30 75 50

4 6 20 14

2 2 2 2

2+2 2+2 2+1 2

1 1 1 2

Nature of referral DMD carrier screening initial DMD carrier screening result Huntington’s prepredictive test session 1 Huntington’s prepredictive test session 2 Posttermination trisomy 21 (initial 10 min.) Posttermination trisomy 21 FAP screening, home visit Cystic fibrosis carrier screening Colon cancer risk evaluation Ovarian cancer risk evaluation No diagnosis, obesity, depression Deafness Neurofibromatosis type 1 Colon cancer risk evaluation No diagnosis, myotonia Colon cancer risk evaluation Ovarian cancer risk evaluation Ovarian cancer risk evaluation No diagnosis, Alport syndrome excluded Deafness, consanguinity Twins, zygosity testing Neurofibromatosis type 1

The above requirements were met in the design of the instrument (see “Results”) with one exception. The final item of cross-cultural validity presented practical restraints of testing the instrument in several international centers. In accordance with previous recommendations, such as by Harris (1991) and Kessler (1979, 1982, 1989, 1990, 1992a, 1992b, 1997), the MOC attempts to measure the communication process, in particular the content, style, empathy, and skill of the communication initiated by the counselor. To measure the content of the communication, a domain for subject was selected, and to measure one


HS071-48

March 7, 2000

13:46

122



facet of style, in an objective manner, a domain for grammatical form (e.g., open or closed questions) was included. For measuring expression of empathy, advice giving, and other counselor behaviors, a domain for the function or purpose of the communication was selected. Statement A statement was defined as a sentence or part of a sentence or multiple sentences forming a complete thought or topic of discussion. Each new thought was coded as a new statement. Raters were asked to place emphasis on “main statements.” For example, a separate empathic statement may occur during a discussion of family history. The empathic statement may be regarded as a statement within the “main statement” of obtaining family history information. The coding sheet contained separate columns in each domain for distinguishing these ratings. Nonverbal expressions were also scored. Client’s statements were not coded. Only the counselor’s phrases were chosen for scoring because it is the counselor’s communication skills that are of interest. Previous studies attempted the sometimes arduous task of rating or transcribing both physicians’ and patients’ statements. For most purposes, only certain behaviors and mainly those of the physician were used for analysis. To reflect the client’s behavior, the counselor’s statements were classified by the domain “cue source,” as previously recommended by Gask et al. (1989). In summary, the domains selected for measurement of communication in a genetic counseling interview were form, function, subject, and cue source (Table I). Missed cues and multiple counselors/clients were also considered in the development of the instrument. The characteristics of the 21 videotaped sessions tested are shown in Table II. Reliability Study For determining test reliability within each statement domain, two independent facets of reliability, interrater and test-retest reliability, were compared between two or more raters. Interrater reliability was used to assess the degree to which different raters give consistent estimates of the same phenomenon. Participant raters were given a 10-minute briefing prior to viewing and rating a video using the designed scoring instrument. Ten raters participated in the reliability study for interrater reliability analysis (Tables III, IV, and V). To ensure consistency of the testing environment, they were recruited to rate the same videotaped sessions on three main occasions. The raters were genetic associates (i.e., counselors, nurses, and student genetic counselors) of varied backgrounds, including genetic counseling, nursing, ethics, psychology, and molecular genetics.


HS071-48

March 7, 2000

13:46



123

Twenty-one videotaped interviews were analyzed by the principal rater. Occasional repeat ratings were done for analysis of test-retest reliability (Table VI). Test-retest reliability was used to assess the consistency of a measure from one time to another. Correlation between two ratings depended in part on how much time elapsed between the two measurement occasions.

Cohen’s Kappa Using Cohen’s kappa (Cohen, 1960), a statistical reliability measure was calculated. A simple measure of percentage agreement between raters would not take chance agreement into consideration. For example, if out of nine observations, two raters select the same category eight times, an 89% (8/9) agreement would be observed. Cohen’s kappa is weighted for disagreement between raters and would thus provide a more accurate statistical measure. The kappa obtained in the previous example was .84. The closer Cohen’s kappa is to 1.0, the higher the agreement between different raters. The commonly accepted standard for reliability coefficients is .70 for group comparisons of interrater reliability. The instrument should be reexamined if reliability coefficients yield a value below .60. A kappa value was obtained for each statement domain, for each of the six videos assessed by more than one rater (interrater reliability), or by one rater on multiple occasions (test-retest reliability). These figures have been tabulated in Tables III–V and summarized in Table VI.

RESULTS Interrater Reliability Three videos were selected for independent rating by a total of 10 different raters. Recruitment of multiple raters of different backgrounds on fewer occasions, rather than two experienced individuals analyzing a large number of interviews, was expected to increase validity of the instrument and reduce the reliability coefficient. Differences in statement selection and rating could be explained qualitatively between different raters. No significant differences in raters’ ability to use the MOC were apparent with respect to level of training. The mean kappa obtained was .70 (Table VI). The domains “form” and “cue source” gave highest kappa values. There were no missed cues by counselors labeled by raters for any of the videos analyzed. Missed cues are those statements initiated by the client and not acknowledged by the counselor. Reliability estimates have been reported with standard deviation for all domains of the instrument, including both total and subtotal scores.


HS071-48

March 7, 2000

124

13:46


Liede, Kerzin-Storrar, and Craufurd Table III. Cohen’s Kappa for Video 2 (Interrater Reliability; Same-Day Ratings) Valid observations

Raters compared

Form

Function

Subject

Cue source

Mean

Common statements

AL × CF AL × LH AL × NB AL × RB CF × LH CF × NB CF × RB LH × NB LH × RB NB × RB

0.63 0.73 0.58 0.43 0.81 0.71 0.52 0.77 0.60 0.82

0.43 0.71 0.36 0.52 0.58 0.29 0.55 0.54 0.64 0.50

0.52 0.72 0.66 0.56 0.55 0.40 0.45 0.94 0.77 0.49

0.49 0.73 0.59 1.00 0.59 0.74 0.63 0.58 1.00 0.76

0.52 0.72 0.55 0.63 0.63 0.54 0.54 0.71 0.75 0.64

26 23 28 21 22 23 15 22 12 16

Mean Standard deviation

0.66 0.13

0.51 0.13

0.61 0.17

0.71 0.17

0.62 0.09

21

Video 2 This counseling session was rich in psychological content, where the counselor discussed results with a young woman of her Duchenne muscular dystrophy testing. The kappa values obtained for the five raters are summarized in Table III. The mean value for kappa calculated for Video 2 was .62, a value lower than for the other videos. This may be related to the length and challenging content of the counseling session. Table II shows that participants took on average 50 minutes to score this video, close to 1 hour. This 24-minute videotaped session contained statements which were somewhat difficult to distinguish, mostly in terms of “function” and “subject,” which gave the lowest kappa values, of .51 and .61. In Video 2, five main statements were labeled as psychological in content by at least four out of the five raters. The obtained kappa was explained qualitatively by reviewing the raw data and selecting statements, not simply the raters, which caused the most disagreement in coding. This video demonstrated that main statements were easier to rate and produced greater agreement. Despite the difficult nature of the counseling session, 21 (range 12–28) statements were labeled in common by the five raters for kappa analysis, and 22 were labeled by at least four out of the five raters as a statement. Video 4 This video dealt with a couple embarking on predictive testing for Huntington’s disease, which exhibited a large proportion of statements in the psychological domain. The kappa values obtained for the six raters are tabulated in Table IV. The mean value for kappa for Video 4 was .72. Table II shows participants took on average 30 minutes to score this 14-minute video. As in Video 2, the lowest


HS071-48

March 7, 2000

13:46



125

Table IV. Cohen’s Kappa for Video 4 (Interrater Reliability; Same-Day Ratings) Valid observations Raters compared

Form

Function

Subject

Cue source

Mean

Common statements

AL × LH AL × MM AL × NB AL × PC AL × RB LH × MM LH × NB LH × PC LH × RB MM × NB MM × PC MM × RB NB × PC NB × RB PC × RB Mean Standard deviation

0.76 1.00 0.59 0.77 0.83 0.63 0.75 0.76 1.00 0.40 0.63 0.76 0.76 0.76 0.82 0.75 0.15

0.69 0.66 0.88 0.67 0.74 0.53 0.53 0.59 0.58 0.71 0.46 0.60 0.51 0.61 0.49 0.62 0.11

0.84 0.87 0.86 0.45 0.91 0.54 0.56 0.39 0.66 0.40 0.32 0.87 0.48 0.67 0.11 0.60 0.24

1.00 1.00 1.00 0.63 0.76 1.00 1.00 0.62 0.62 1.00 1.00 1.00 1.00 1.00 1.00 0.91 0.16

0.82 0.88 0.83 0.63 0.81 0.67 0.71 0.59 0.72 0.63 0.60 0.81 0.69 0.76 0.61 0.72 0.10

11 13 12 13 14 9 9 10 11 9 9 12 11 11 13 11

Table V. Cohen’s Kappa for Video 5a (Interrater Reliability) Valid observations Raters compared

Form

Function

Subject

Cue source

Mean

Common statements

AL1 × LKS AL1 × RM AL1 × TC LKS × RM LKS × TC RM × TC Mean Standard deviation

0.68 0.63 0.69 0.66 0.64 0.84 0.69 0.08

0.84 1.00 0.64 0.85 1.00 0.69 0.84 0.15

0.86 0.89 0.89 0.66 0.83 0.85 0.83 0.09

0.80 0.45 1.00 0.35 0.71 0.74 0.68 0.24

0.80 0.74 0.81 0.63 0.80 0.78 0.76 0.07

10 12 11 13 15 9 12

kappa values were obtained for domains “function” and “subject,” which yielded kappas of .62 and .60, respectively. Confusion in “subject” was apparent by different raters. On average, 11 (range 9–14) statements were labeled in common by the six raters for kappa analysis, and 12 were labeled by at least five out of the six raters as a statement. Video 5a This video dealt with a follow-up appointment with a couple after the termination of a trisomy 21 fetus, detected with amniocentesis. Video 5a represented the initial 10 minutes of a 50-minute counseling session. Table V summarizes the


HS071-48

March 7, 2000

13:46

126


Liede, Kerzin-Storrar, and Craufurd Table VI. Summary Table (Cohen’s Kappa for Videos Analyzed) Valid observations

Video

Form

Function

Subject

Cue source

Mean

Common statements

Interrater reliability Video 2 Video 4 Video 5a

0.66 0.75 0.69

0.51 0.62 0.84

0.61 0.60 0.83

0.71 0.91 0.68

0.62 0.72 0.76

21 11 12

Mean

0.70

0.66

0.68

0.77

0.70

15

Test-retest reliability Video 3 Video 5a Video 6

0.64 0.69 0.90

0.64 0.86 1.00

0.43 0.84 0.87

1.00 0.92 0.72

0.68 0.83 0.87

23 13 14

Mean Total mean Standard deviation

0.74 0.72 0.09

0.83 0.75 0.18

0.71 0.70 0.18

0.88 0.82 0.14

0.79 0.75 0.09

17 16

kappa values obtained for the four raters. The mean value for kappa for Video 5a was .76. The kappa coefficients for each of the four statement domains in this video were .69, .84, .83, and .68, respectively. On average, it took raters 20 minutes to score the first 10 minutes of the video. The counseling session, particularly the first 10 minutes, was laden with psychological issues for the couple. The lowest kappa value was obtained for domain “cue source,” with a mean of .68. From the other videos analyzed, “cue source” normally provided the greatest agreement between different raters. Discrepancy could be explained by the different set of raters who participated in the rating. For Video 5a, 12 (range 9–15) statements on average were labeled in common by the four raters for kappa analysis, and 12 were labeled by at least three out of the four raters as a statement. Test-Retest Reliability Test-retest reproducibility was provided to complement, and not substitute, interrater consistency. Three videos were selected to be rated on more than one occasion by the same rater. The mean kappa for these videos measured for testretest reliability yielded a value of .79, which was greater than the mean obtained for interrater reliability (Table VI). Videos 3, 5a, and 6 were randomly selected for test-retest reliability study. The data for kappas obtained for these videos are summarized in Table VI. DISCUSSION This study describes the development and evaluation of an instrument, the Manchester Observation Code, designed for use in the study of the nature of


HS071-48

March 7, 2000


13:46


127

the genetic counseling process. The instrument comprises a multi-item rating of the counselor’s, not the client’s, statements (Table I). Coding is done primarily from videotapes. MOC allows the user to identify four specific domains of communication during a genetic counseling session. Each phrase or complete thought expressed by the counselor during genetic counseling is coded into form (how), function (why), subject (what), and origin (where). Videotape analysis enabled the identification of the sequence and frequency of events during a genetic counseling interview. The instrument successfully identified infrequent but vital exchanges such as nonverbal psychological cues by the client and cues missed by the counselor. The instrument demonstrated sufficiently high levels of interrater reliability. The reliability coefficients averaged .70 and .79 for interrater and test-retest reliability (Table VI). Interestingly, the lowest test-retest reliability coefficient of .68 observed for Video 3 was below the mean obtained for interrater reliability of .71. Overall, the “subject” domain exhibited the greatest standard deviation and disagreement. This domain contains the largest number of classification codes to choose from (Table I). From the raw data, we interpreted the disagreement originated from the inconsistency in labeling statements as dealing with psychological or social issues. This confusion in the “subject” domain may be alleviated with clear examples during the rater training session to illustrate differences in psychological and social issues. Conceptually, this domain is not difficult to classify when observing a videotaped session, despite the observed kappa values. A further recommendation may be to reduce the number of classification codes in this domain, such as combining psychological and social into “psychosocial issues”; similarly, certain applications of this method may wish to combine medical and genetics into “medical and genetics.” Interview-specific items were identified in the 21 videotaped sessions used to test the method. These included, the amount of time spent on psychological issues, the ratio of open to closed questions, the number of missed cues, and the frequency of empathic, directive, and advice-giving statements by the counselor. The instrument also allowed for the identification of subtle cues from both client and counselor. The designed instrument may be modified to measure a specific component of communication. In particular, it may hold promise for those interested in the study of directiveness. It allows for the measurement of directive counseling behaviors, such as using directive statements (“form”) or giving advice (“function”) combined with a specific “subject” domain such as medical. For example, Video 6 depicted a counselor’s home visit to a young adult at risk of FAP (familial adenomatous polyposis) who was not attending colonoscopy screening appointments. Directive counseling elements were identified. FAP is one example of a genetic condition where a better prognosis could result from medical intervention, and the counselor’s motivation was to encourage screening for early detection of cancer and to explore reasons for noncompliance. The counselor appeared to be largely nondirective in her advice giving and seemingly neutral in her educational approach. Nevertheless, the overall counseling strategy seemed to be


128

HS071-48

March 7, 2000

13:46



aimed at persuading the counselee to agree to have the colonoscopy procedure. Analysis of 14 statements identified at least 4 as “giving advice,” as well as 5 as “exploring behavior” in the “function” domain. The MOC is the first attempt to design a quantitative method for analyzing communication in genetic counseling. We have demonstrated that it is possible to measure and describe individual elements of the genetic counseling process with adequate levels of interrater reliability. However, because MOC relies on the rater’s ability to recognize when a counselor avoids or evades affective issues and when empathic interventions are employed, it still remains somewhat user-dependent. A 10-minute briefing was given to participant raters prior to viewing and rating a video. Ambiguities in classification may be alleviated with a longer training session for prospective raters. Test-retest data (not shown) suggest that one should not attempt repeat ratings until a minimum of a complete week has passed, to reduce bias. Clearly, one principal rater will become accustomed to the scoring instrument and would generate consistent scores. The instrument was designed to be succinct, applicable to one or two viewings of a videotaped session. A mean ratio of approximately 3 minutes to 5 was observed for length of video and time need for rating (Table II). In future, it may be possible to employ this instrument for rating genetic counseling sessions directly. Although the literature to date does not provide a consistent picture of the factors influencing outcome measures, the genetic counseling process itself is certainly a contributory factor to outcomes such as patients’ decision making. The instrument discussed here may be used and modified to identify important factors in the process of genetic counseling. Postcounseling questionnaires and interviews may assess measures of outcome such as client satisfaction, in relation to elements of the process of genetic counseling. However, a scheme to measure client satisfaction has yet to be fully developed. Further validation of MOC is necessary and will become available once others employ it to study the genetic counseling process.

ACKNOWLEDGMENTS We thank Rachel Belk, Nicola Bradshaw, Pam Chapman, Catherine Falconer, Lindsay Hadfield, Marion McAllister, Tara Clancy, Claire Faulkner, and Rhona MacLeod for their involvement as participant raters in the reliability study. We also thank the Department of Clinical Genetics at St. Mary’s Hospital and all the patients for providing consent to videotape their respective counseling sessions and Paula Williamson and Jean-S´ebastien Brunet for their advice on statistical considerations. This work was completed for an M.Sc. thesis in genetic counseling by Alexander Liede at the University of Manchester in England.


HS071-48

March 7, 2000

13:46



129

REFERENCES Armstrong D, Michie S, Marteau T (1998) Revealed identity: A study of the process of genetic counselling. Soc Sci Med 47:1653–1658. Bartels DM, LeRoy BS, McCarthy P, Caplan AL (1997) Nondirectiveness in genetic counseling: A survey of practitioners. Am J Med Genet 72:172–179. Benjamin CM, Colley A, Donnai D, Kingston H, Harris R, Kerzin-Storrar L (1993) Neurofibromatosis type 1 (NF1), knowledge, experience and reproductive decisions of affected patients and families. J Med Genet 30:567–574. Bensing JM, Sluijs EM (1985) Evaluation of an interview training course for general practitioners. Soc Sci Med 20:737–744. Bowman FM, Goldberg DP, Millar T, Gask L, McGrath G (1992) Improving the skills of established general practitioners: The long-term benefits of group teaching. Med Educ 26:63–68. Carter WB, Inui TS, Kukull WA, Haigh VH (1982) Outcome-based doctor-patient interaction analysis. II. Identifying effective provider and patient behavior. Med Care 20:550–566. Clarke A (1990) Genetics, ethics and audit. Lancet 335:1145–1147. Clarke A (1991) Is non-directive genetic counselling possible? Lancet 338:998–1001. Clarke A (1993) Response to: what counts as success in genetic counselling? J Med Ethics 19:47–49. Clarke A, Parsons E, Williams A (1996) Outcomes and process in genetic counselling. Clin Genet 50:462–469. Cohen J (1960) A coefficient of agreement for nominal scales. Educational Psychology and Measurement 20:37–46. Edwards S, Slattery ML, Mori M, Berry D, Caan BJ, Palmer P, Potter JD (1994) Objective system for interviewer performance evaluation for use in epidemiological studies. Am J Epidemiol 140:1020– 1028. Faulkner CL (1995) The knowledge, views and experience of myotonic dystrophy among a group of affected women. In: Master’s thesis in Medical Genetics, University of Manchester, Manchester, UK. Frets PG, Duivenvoordent HJ, Verhage F, Ketzer E, Niermeijer MF (1990) Model identifying the reproductive decision after genetic counselling. Am J Med Genet 35:503–509. Gask L, Goldberg D, Boardman J, Craig T, Goddard C, Jones O, Kiseley S, McGrath G Millar T (1991) Training general practitioners to teach psychiatric interviewing skills: An evaluation of group training. Med Educ 25:444–451. Gask L, Goldberg D, Lesser AL, Millar T (1988) Improving the psychiatric skills of the general practice trainee: An evaluation of a group teaching course. Med Educ 22:132–138. Gask L, Goldberg D, Porter R, Creed F (1989) The treatment of somatization: Evaluation of a teaching package with general practice trainees. J Psychosom Res 33:697–703. Gask L, McGrath G, Goldberg D, Millar T (1987) Improving the psychiatric skills of established general practitioners: Evaluation of group teaching. Med Educ 21:362–368. Goldberg DP, Jenkins L, Millar T, Faragher EB (1993) The ability of trainee general practitioners to identify psychological distress among their patients. Psych Med 23:185–193. Goldberg DP, Smith C, Steele JJ, Spivey L (1980a) Training family doctors to recognise psychiatric illness with increased accuracy. Lancet 2:521–523. Goldberg DP, Steele JJ, Smith C (1980b) Teaching psychiatric interviewing techniques to family doctors. Acta Psych Scand 62:41–47. Hallowell N, Murton F, Statham H, Green JM, Richards MPM (1997) Women’s need for information before attending genetic counselling for familial breast or ovarian cancer: A questionnaire, interview, and observational study. Brit J Med 314:281–283. Harris R (1991) The new genetics: A challenge to traditional medicine. Based on the Milroy Lecture 1989. J R Coll Physicians Lond 25:134–140. Inui TS, Carter WB, Kukull WA, Haigh VH (1982) Outcome-based doctor-patient interaction anaylsis. I. comparison of techniques. Med Care 20:535–549. Kaaya S, Goldberg D, Gask L (1992) Management of somatic presentations of psychiatric illness in general medical settings: Evaluation of a new training course for general practitioners. Med Educ 26:138–144.


HS071-48

March 7, 2000

13:46

130



Kessler S (1979) The processes of communication, decision making and coping in genetic counselling (Chapter 3). In: Kessler S (ed) Genetic counselling: Psychological dimensions. New York: Academic Press. Kessler S (1982) The psychological paradigm shift in genetic counseling. Soc Biol 27:167–185. Kessler S (1989) Psychological aspects of genetic counselling. VI. A critical review of the literature dealing with education and reproduction. Am J Med Genet 34:340–353. Kessler S (1990) Current psychological issues in genetic counselling. J Psychosomatics, Obstet Gynaec 11:5–18. Kessler S (1992a) Process issues in genetic counselling. In: Birth Defects: Original Article Series. New York: Wiley-Liss, pp. 1–10. Kessler S (1992b) Psychological aspects of genetic counselling. VII. Thoughts on directiveness. Genet Couns 1:9–17. Kessler S (1997) Genetic counselling is directive? Look again. Am J Hum Genet 61:466–467. Maguire P, Roe P, Goldberg D, Jones S, Hyde C, O’Dowd T (1978) The value of feedback training in teaching interviewing skills to medical students. Psych Med 8:695–704. Marteau T, Drake H, Bobrow M (1994a) Counselling following a diagnosis of a fetal abnormality, the differing approaches of obstetricians, clinical geneticists, and genetic nurses. J Med Genet 31:864–867. Marteau T, Drake H, Reid M, Feijoo M, Soares M, Nippert I, Nippert P, Bobrow M (1994b) Counselling following diagnosis of fetal abnormatility: A comparison between German, Portuguese and UK geneticists. Eur J Hum Genet 2:96–102. Michie S, French D, Allanson A, Bobrow M, Marteau TM (1997) Information recall in genetic counselling: A pilot study of its assessment. Patient Education and Counselling 32:93–100. Pearn JH (1973) Patients’ subjective interpretation of risks offered in genetic counselling. J Genet Couns 10:129–134. Preven DW, Kachur EK, Kupfer RB, Waters JA (1986) Interviewing skills of first-year medical students. J Med Educ 61:842–844. Roter DL, Stewart M, Putnam SM, Lipkin M, Stiles W, Inui TS (1997) Communication patterns of primary care physicians. JAMA 277:350–356. Seidenfeld MJ, Antley RM (1981) Genetic counselling: A comparison of counsellee’s genetic knowledge before and after. Am J Med Genet 10:107–112. Shiloh S, Avdor O, Goodman RM (1990) Satisfaction with genetic counselling: Dimensions and measurement. Am J Med Genet 37:522–529. Street RL (1992) Analyzing communication in medical consultations: Do behavioural measures correspond to patients’ perceptions? Med Care 30:976–988. Wasserman RC, Inui TS (1983) Systematic analysis of clinician-patient interactions: A critique of recent approaches with suggestions for future research. Med Care 21:279–293.

APPENDIX Manchester Observation Code (MOC) for Genetic Counseling Manual: General Instructions for Use. Please refer to Table I to follow these instructions. Please refer to these instructions when using the instrument. Table I is intended for distribution among raters. Classification of Statements. Counselor’s “statements” are to be scored. A “statement” can be in the form of a complete sentence, part of a sentence or multiple sentences forming a complete thought or topic of discussion. A statement should not necessarily be regarded as each individual gesture or phrase uttered by the counselor. Each new statement or behavior represents a new train of thought.


HS071-48

March 7, 2000

13:46



131

Whenever there is a change in the “direction” of discussion (i.e., generally a change in function, subject, or cue source), this should be counted as a new statement and a new rating should be made. Only counselors’ statements were scored because it is the counselor’s performance we are interested in measuring. Counselors’ statements are scored using four domains. The first rating domain, form, is based on the grammatical form of the statement; the second domain, function, is the purpose of the statement; the third domain is subject of discussion; the fourth domain, cue source, represents the origin of the statement (i.e., counselor-initiated or patient-initiated—developed partly to eliminate rating of client’s statements). Each new statement is rated in form, function, subject and cue source. Each statement is distinguished on the coding sheet by the time when it was initiated. The approximate time at which the statement began is recorded in minutes and seconds (may be read from VCR display). Decisions about statement selection may be marked onto the coding sheet. The primary determinants of a new statement are function, subject, and cue source of statements, designed to map out the major topics of the interview. Examples: 1. “I understand you’ve come today because you are concerned about your risk of being a carrier.” This is one statement. Obviously, if the statement is grammatically in the form of a question it is rated as such. However, sometimes a question can be complex in form, such as in the form of a statement; this would be rated as a statement made to elicit information, or as an exploring behavior.

Statement (time)

Comments (specify)

Counselor statements Form

Functn

Subjct

Cue source

Example 1

1

6

2

3

Exploring reason for coming

Example 2

4

3

2

3

Direction of session, negotiating

Min:sec

2. “I thought we could begin today by discussing what you already know or have heard about this condition and then we could talk about . . .” This is a new statement, a deliberate change in form and function.


HS071-48

March 7, 2000

13:46

132



3. “I can see that this information comes as a surprise to you.” Statement (time)

Comments (specify)


Functn

Subjct

Cue source

3:00

1

3

5

3

Inheritance risks

Example 3

1

6

6

2

“Comes as a surprise to you”— patient looked shocked

Min:sec

Subclassification. In the development of the scoring instrument, it was observed that certain statements occur within main topics of discussion. These brief statements are nested within others (“main” statements) of a different type. Example: 4. “How old was your mother when she died, following the diagnosis of breast cancer? . . . That must have been a difficult time for you.” A separate statement such as an empathic statement may occur within a main statement, comprised of multiple questions, of obtaining details of the family history. The empathic statement may be rated separately as a “substatement.” The coding sheet has separate columns in each domain to distinguish these ratings:

Statement (time) Min:sec 7:00

Example 4

Comments (specify)


Functn

Subjct

Cue source

3

2

3

3

1

5

6

Family history

1 "Difficult time for you"

Main statements represent statements of greater agreement between different raters; substatements are optional and represent statements of lower agreement. Cues Missed. Obvious cues not picked up by the counselor(s) are to be recorded on the coding sheet. Affect-laden comments (e.g., “I can’t cope”) or nonverbal cues expressed by the client not commented upon by the counselor are rated as “cue missed.” It is labeled in comments before the next rating.


HS071-48

March 7, 2000

13:46



133

Example: 5. “I am terribly frightened I will develop cancer” (client). “From epidemiological studies, your risk is . . .” (counselor). Patient’s fears are not addressed.

Statement (time) Min:sec

Comments (specify)


Functn

Subjct

Cue source Cue missed—“frightened I will develop cancer”

Example 5

20:50

1

3

5

3

Risk figures

If the counselor explains to the client that the cue will be addressed later (and is addressed later), this is not rated as a cue missed (see Examples 1, 2). Multiple Counselors, Multiple Clients. This instrument is designed to accommodate more than one counselor or client. As long as only one person speaks at one time, the same rules apply. Counselor-specific statements may be distinguished in comments on the coding sheet if necessary. Other (Code 11). If a statement cannot be adequately placed in any of the four domains, Code 11 may be used, but this should be limited to exceptional cases. Decisions about using Code 11 may be detailed in comments. Comments. As mentioned previously, a comments column is designed for various purposes. This section is designed to briefly describe what is said and/or to label certain behaviors in more detail. This section may be adapted for study-specific criteria to be assessed. Domain 1: FORM. This rating domain relates to the grammatical form of a counselor’s statement. This is the “how” of the statement. There are four possible forms: 1—STATEMENT These are statements in basic sentence form which are not questions. Examples: 1. “Your risk is . . .” 2. “Genes are instructions . . .” 2—OPEN QUESTION These are statements which elicit an open or nonspecific response from the client.


HS071-48

March 7, 2000

13:46

134



Examples: 3. “How are you getting on?” 4. “How does that risk sound to you?” Response is open. 3—CLOSED QUESTION These are statements in a closed or leading question form, which suggest a specific or limited response from the client. Examples: 5. “Are you worried about your risk?” “Yes” or “no” questions. 6. “Would you say that is a high or a low risk?” Response is limited. 4—DIRECTIVE STATEMENT Directive statements point the discussion to a specific topic area. The counselor specifies the next topic of discussion. Examples: 7. “Perhaps this is a good time to explain how the condition is inherited.” 8. “Why don’t you tell me what you already know about this condition?” Rather than a question, this statement specifies what will be discussed; the function is eliciting information.

Statement (time)

Comments (specify)


Functn

Subjct

Cue source

Example 7

4

3

5

3

Inheritance explanation— negotiating

Example 8

4

2

5

3

“What you already know about this condition”

Min:sec

Domain 2: FUNCTION. This rating domain is designed to capture the purpose behind the counselor’s statement. It is the “why” of the statement. The classification of this domain is important, and six codes have been constructed: 1—MAKING CONVERSATION The counselor exhibits general conversational skills, such as introductory or humorous remarks.


HS071-48

March 7, 2000

13:46



135

2—ELICITING INFORMATION This is a question, or statement, designed to obtain information from the client. This is the function often used when a topic is initiated or when new areas of a current topic are opened up. Checking-out questions fall into this category. 3—GIVING INFORMATION A statement designed to provide information. 4—GIVING ADVICE This is a statement giving the client a specific example of a strategy to alleviate the problem mentioned. Example: 1. “Many couples in your situation find it helpful to . . .” Provides a strategy. 5—PATIENT-CENTERED BEHAVIOR Formerly named “expressing empathy” during development of the Manchester method. Here the counselor makes a statement showing understanding of the client’s feelings or situation. Although mostly comprising empathic statements, this now extends beyond to include statements of reassurance, support, and other forms of interpersonal sensitivity. Examples: 2. “How upsetting for you.” 3. “That must have been a difficult time for you.” 4. “You seem to be coping really well.” 6—EXPLORING BEHAVIOR A statement to draw attention to thoughts, feelings, or situation. Here the counselor reflects, paraphrases what the client has stated, makes an observation about the client, or puts forward a hypothesis. The counselor may put forward a sensible and warranted guess at the client’s thoughts, feelings, or situation. This behavior is an attempt to instigate self-reflection in the client, hence helping the client define his or her own problem. It is a probing behavior, a statement exploring ambivalence. Examples: 5. “You said you were fine and no worries, but now you say you are very anxious about . . .” Exploring ambivalence. 6. “I wonder if what’s really bothering you is . . .” Hypothesis. 7. “From what you say, it appears to me that . . .” Hypothesis. 8. “You look somewhat surprised and upset by this information.” An observation.


HS071-48

March 7, 2000

13:46

136



Domain 3: SUBJECT. This is the subject or content of the counselor’s statement. This is the “what” of the statement, perhaps the main determinant of a new statement, a new topic of discussion. There are seven possible codes: 1—TRIVIAL Topics unrelated to the nature of the referral (e.g., weather, traffic). 2—ABOUT SESSION Discussion of the nature of the referral or appointment; about a past, present or future appointment; topics to be discussed during the interview. 3—FAMILY HISTORY Discussion about specific genetic condition in a family; obtaining details to construct a family tree. This statement type may be composed of multiple questions and sentences. 4—MEDICAL Discussion of previous medical history; screening or treatment options such as prenatal diagnosis; prognosis of medical condition; medical facts. 5—GENETICS Explanation of genetics such as nature of inheritance of condition, recurrence risks. May be amalgamated with medical to simplify coding. 6—PSYCHOLOGICAL ISSUES Discussion around personal issues such as coping mechanisms, adjustment to reality of condition or the reality of the risk of inheriting, developing a condition; emotional state. 7—SOCIAL AND FAMILY FACTORS To be distinguished from family history taking, this represents discussion surrounding daily life, relationships, other family members, financial worries, workplace, and other related social topics. Domain 4: CUE SOURCE. This domain is used to indicate “where” the statement originated: Who initiated the statement or topic of discussion? Is it the patient (i.e., with the counselor responding to a patient’s question—patient-initiated verbal)? Or is it the counselor (i.e., he or she is following an agenda of topics to be discussed— counselor-initiated)? Cue source was used for the scoring instrument because origin of statement is important, not necessarily the rating of the client’s statements (alleviating a potentially painstaking task). These fall into the two main categories of patient-initiated and counselor-initiated, with the further distinction of the patient-initiated classification into verbal and nonverbal: 1—PATIENT-INITIATED: VERBAL 2—PATIENT-INITIATED: NONVERBAL 3—COUNSELOR-INITIATED Most of the counselor’s statements fall into counselor-initiated, as he or she is following topics relevant to the consultation.

Genetic Counselors' Experiences with Paternal Involvement in Prenatal Genetic Counseling Sessions: An Exploratory Investigation.

Stereology: a method for analyzing images.

Genetic counseling for pregnant adolescents.

Genetic Counseling for Diabetes Mellitus.

Genetic counseling.

Genetic Counseling: A New Perspective.

From genetic counseling to "genomic counseling".

A method for analyzing resource use in ambulatory care settings.

Genetic counseling for Fanconi anemia: crosslinking disciplines.

Genetic counseling for congenital heart disease.

The life table. A method for analyzing longitudinal studies.

A method for analyzing commonalities in clinical trial target populations.

Development of a method for analyzing three-dimensional scapula kinematics.

The case for universal prenatal genetic counseling.

Therapy and genetic counseling for schizophrenia.

Genetic counseling in retinoblastoma.

Genetic Counseling for Confined Placental Mosaicism: A Case Report.

Case Report: A Systems Approach to Genetic Counseling for Albinism.

Commentary: Computers and Genetic Counseling-Time for a Dialogue?

A teaching framework for cross-cultural genetic counseling.

Bicuspid Aortic Valve: a Review with Recommendations for Genetic Counseling.

Breast Cancer Genetic Counseling: A Surgeon's Perspective.

Genetic knowledge client perspectives, and genetic counseling.

Counseling issues in genetic screening.