A Continuing Educational Program for Paramedical Personnel George Kriegman, M.D. Martin D. Harris, M.D. Edwin F. Rosinski, Ed.D.
ABSTRACT:Paramedical personnel offer a ready reservoir of partially trained personnel who can assist in combating the ever-increasing problem of emotional disorders. A multi-disciplinary program of continuing education for these allied professionals is outlined. Four courses, Personality Development, Emotional and Mental Disorders, Skills and Techniques in Working with Patients and Families, and Sociological Factors in Mental Health, have been developed. The results of three pilot projects are summarized, and future plans and problems are outlined. Evaluation of projects indicates that formal learning of mental health information occurs and greater mutual respect and unity develop among multi-disciplinary participants. The present and anticipated shortage of trained mental health personnel makes it imperative to strengthen and update the skills of allied personnel who are currently working with the emotionallydisturbed. Welfare workers, probation officers, visiting teachers, public health nurses, and others are performing a considerable degree of the initial corrective, preventive, and follow-up work with disturbed persons of all ages. Although competent in their specialties, many of the allied personnel have had limited experience with modern mental health concepts. To enhance their potential as vital parts of community and national mental health care, and to help meet the rapidly growing need for trained workers, a program of continuing education, orientation, unification, and recognition is essential. In the past, countless seminars, workshops, and lecture courses have been conducted under local or statewide sponsorships. As a rule, such programs have been "one-shot deals" aimed at a particular professional group. Their subject matter has been highly specialized, and little overall planning and follow-up Dr. Kriegman is Clinical Professor of Psychiatry, and Dr. Harris is Career Training Fellow and Instructor in Psychiatry, Medical College of Virginia, Richmond, Virginia. Dr. Rosinski is Deputy Assistant Secretary for Health Manpower, Department of Health, Education, and Welfare, Washington, D. C. The authors wish to acknowledge the help of the following colleagues: Virginia State Department of Education--W. Kuhn Barnett, Rosa Loving, Nettie T. YowelI; Virginia Department of Mental Hygiene and Hospitals--Margaret L. Cavey, Arthur Centor, James B. Funkhouse, Elisa Haga, Benedict NagIer; Virginia Department of Public Welfare and Institutions--Herbert A. Krueger; Department of Vocational Rehabilitation-R. I47. McLemore; Medical College of Virginia--Morton D. Schumann, Patricia Wiley; University of Virginia School of Medicine--Bruce Smith. Comrnuni~y Mental Health Journal, Vol. 4 (5), 1968
377
378
Community Mental Health Journal
reinforcement have been attempted. They have not shown significant longrange effectiveness, nor have they generally induded the broad spectrum of personnel who could and should benefit from them. A planned program--well organized, properly evaluated, and involving all allied mental health personnel within a specific region--would foster uniform educational opportunities, would have more far-reaching results, and would have greater long-range value to the particpants and to the communities they serve. This paper reports an effort to devise such a program and to evaluate its efficiency. BACKGROUND OF PROGRAM In -r963, The Virginia Advisory Committee on Inservice Training in Mental Health decided to develop a multi-disciplinary, continuing education program for allied mental health personnel. The objectives of the program were: I. To provide comprehensive inservice mental health training and continuing education for allied professionals on a multi-disciplinary basis. 2. To ensure that all mental health workers, within their areas of competency, provide consistent, unified, high-level mental health services through broadened mental health knowledge and principles. 3. To promote coordination and collaboration of professional workers through their equal participation in a multi-disciplinary training program. In order to develop a program that would be properly grounded in the principles of education, a Subcommittee on Methods was formed to study teaching techniques. In addition, it was decided that a leadership group should form the apex of an inverted pyramid, spreading its knowledge to the broader range of allied mental health personnel. With the assistance of Health, Education and Welfare, Region III, a threeday workshop was organized to study recent educational methods, materials, theories, and evaluating techniques. This workshop was co-sponsored by the Medical College of Virginia's Office of Research in Medical Education. Sixty training officers from the state departments of education, rehabilitation, welfare, and mental hospitals and clinics attended. With the knowledge gained from the methods workshop, the Subcommittee on Content, representing a wide range of disciplines, devised four courses which were judged to fulfill the basic needs for competency in the mental health field. These courses were entitled: Personality Development, Emotional and Mental Disorders, Skills and Techniques in Working with Patients and Families, and Sociological Factors Related to Emotional and Mental Disorders. The first three of these courses have been tested on a pilot study basis.
Pilot Project The objective of the first course (Personality Development) was to provide the understanding of human behavior necessary for dealing
G. Kriegman, M. D. Harris, and E. F. Rosinski
379
with mental health problems. Twelve sessions of a two-hour weekly meeting were conducted at Central State Hospital, Petersburg, Virginia, with co-sponsorship by the Medical College of Virginia, Department of Psychiatry. The curriculum, outlined according to lecture sequence, was: z. & 2. Introduction to the multiple factors that determine behavior; general characteristics of behavior; personality; concept of self; attitude toward death. 3. Stages of development, birth through early infancy. 4. Behavior in late infancy; physical and emotional needs. 5. Behavior levels in childhood; physical, emotional, and sexual needs. 6. Characteristics of the juvenile period; physical, emotional and sexual factors. 7. Characteristics of pre-adolescence; physical, emotional, and sexual factors. 8. Stages and characteristics of adolescence; physical, emotional, and sexual factors. 9. Definition of maturity. Io. & II. Stages of adulthood; characteristics and physical changes of each stage. Ia. Aging and senility; physical, emotional, and sexual changes. Fifty-three allied mental health workers attended the course. After each onehour lecture given by a psychiatrist, the participants divided into small discussion groups. At the conclusion of the course, a final examination (3 ~ multiple-choice questions) was administered, resulting in a mean of 6. 5 and a median of 6.o errors. Also at this time, the leaders of the discussion groups answered a questionnaire, and their questions revealed that: I. Overall, the material was comprehended, but the first three lectures (the most technical, abstract, and biologically oriented) were the most difficult for the participants. 2. The central theme varied among the discussion groups. As each group became more confident, occupational identification disappeared and a more equal interchange became evident. Each group began to focus on the lecture material. 3. No one occupational group surpassed the others. In one group, the leader felt that the occupational and recreational therapists benefitted most. In another group, the attendants and nurses seemed more activated. 4. There should be greater use of visual aids, outlines for the use by discussion groups, and further development of clinical examples in the lectures. One leader stated that the heterogeneous make-up of the group was an important factor in the much higher degree of motivation than has usually been seen in homogeneous groups, and that, in any further work of this sort, the groups be represented by all educational and job levels. 5. There was interest in determining methods for objectively measuring the
380
Community Mental Health Journal
effectiveness of this type of program, although the consensus was that the course was effective. 6. The participants wished to take future courses that emphasized sections of the course related to their individual interests. They wanted more time for discussion groups and increased structure and direction by the leadership. They also desired more audiovisual aids and supplemental reading. Pilot Project 2
The second course (Skills and Techniques in Working with Patients and Families) was given in a three-day workshop at Radford College, Radford, Virginia. Again, the Medical College of Virginia, Department of Psychiatry was co-sponsor. The objective of this course was to develop understanding of the techniques required in working with emotionally disturbed persons and their families. The three-day curriculum highlighted: 5. General principles in counseling persons with emotional disorders. 2. Understanding the patient and his family as a basis for services; nature of disorders; physical health; emotional interaction of family; living arrangements; finances; feelings and attitudes of patient and family; evaluation and planning of services. 3. Principles of relationship with patients and families; understanding of self and the role of the professional worker; essential factors in relating the patient and family; principles of interviewing; continuing contact with patients and families; professional responsibility. Lecturers were: a psychiatrist, clinical psychologist, family social worker, psychiatric nurse, and special education teacher. All but the psychiatrist acted as resource persons for discussion groups, which were conducted after each panel presentation, morning and afternoon. The 32 participants, all from southwestern Virginia, were administered identical tests of 3o multiple-choice questions at the beginning and end of the course and, as Table z shows, participants over 35 years old performed relatively poorly on both examinations and showed the least improvement between the initial test and the final test. At the completion of the course, the instructors were asked to evaluate the program. They observed that the older professionals tried to dominate the early discussion periods: personal dinical experiences, complaints, personal and family problems stimulated by the lecture material tended to dominate the discussion period in the first two or three discussion periods. The resource faculty member was minimally involved in the group. However, as group work continued, a gradual change occurred. The young workers began to be more expressive and assertive, and the elder professionals' domination diminished. Furthermore, the groups focused more and more on the lecture material and increasingly turned to the faculty resource person for assistance and advice. In the final sessions, the focus was primarily on the lecture material, and the resource person was fully integrated into the group.
G. Kriegman, M. D. Harris, and E. F. Rosinski
381
TABLE x Pre- and Post-Test Scores of Participants in the Workshop on Skills and Techniques in Working with Patients and Families Pilot Project #2 Unmatched Examinations Unmatched Examinations Examination
No.
Mean Incorrect
Range
I II
26 27
"rX.O 8.5
6.5-20.0 3.o-z6.5
Matched Examinations Post Test Mean Incorrect
Professions
No.
Age (mean)
Case or Welfare Workers Teachers
xx M-~ l::--r0
35.2 22-60"I"
xz BA
xo.7
7.2
7 M-2 F-5
52.7 34-60+
x2.3
~o.x
4 F-4
z-2 yr Coll 3-BA 3-MA 4"RN
"r162
:~o.o
BA, BD
~o.o
8.o
z-2 yr Coll x5-BA, BD 4-RN 3-MA
xx.32
8.60
~x.83 ~o.37
~o.53 5.00*
Nurses Minister
M-x
45.2 25-67t 37
Totals
23 M-4 F-z9
42.3 22-67
~5 8
+33 --33
Ed. Levcltt
Pre Test Mean Incorrect
%--Age range %~---BAor BD--college degree MA--graduate degree * t--6.7z 4 p < .ooi Pilot Project 3 The third course was composed of an eight-session, threehour weekly seminar on Emotional and Mental Disorders and was attended by 93 allied mental health workers at the Medical College of Virginia, Richmond, Virginia. Each session consisted of a one-hour lecture, a one-half hour dosedTV presentation of clinical examples, and a one-hour discussion group. Material covered during the course was: 9 . & 2. Introduction and general background related to basic syndromes and mental mechanisms. 3. Personality disturbances.
382
Community Mental Health Journal
4. Sociopathic personality; alcohol and drug addiction; sexual deviations. 5. Anxiety states and depression. 6. Hysteria. 7. Compulsive-obsessive states. 8. & 9. Manic-depressive psychosis. lo. & II. Schizophrenia. 9 2. Paranoia and paranoid conditions. 13 . Psychoses due to: infection; intoxication; drug addiction; trauma; disturbance of circulation; tumor. Five discussion groups were designated and led by a college faculty member or a senior resident in psychiatry. An observer equipped with a detailed discussion guide outline and scoring sheets was assigned to each discussion group, and each discussion group was audio-tape recorded. Every participant was given the same examination at the beginning and at the end of the seminar. This time the following procedures were used to evaluate the participant's growth of knowledge and the success of the program's objectives. I. The five discussion groups were carefully selected. The typical group consisted of six or seven teachers, four or five welfare workers, one or two ministers, one nurse, one physician or more, and one or two other related workers, such as occupational therapists or speech correctionists. There was an equal distribution in educational level, age range, and male-female ratio (3 to 4 males per group). The Negro-Caucasian ratio was consistent. 2. The discussion groups were analyzed with respect to leader behavior, group interaction, and individual behavior. Each observer was given a guide and a scoring sheet, which covered the physical components of the group, content of the meetings, leader behavior, and group roles (divided into task roles, group interaction roles, and anti-group roles). Meeting-to-meeting comparisons were also noted. 3. The audio tapes were evaluated in terms of depth of cognitive and affective learning, analysis of group processes, comparison with scoring sheets data and interactional analysis. (This latter technique as developed by N. Flanders (~959) rates primarily the percentage of activity in the group of (a) the leader and (b) the participants. This includes both direct and indirect influencing interventions by the leader and the responses to the leader by group members, as well as group initiated interactions.) 4. A 3o-question, multiple-choice, four-item test was used to evaluate the prior knowledge and the learning progress of the individual participants. It was unsigned. Reliability of the instrument was determined by the Spearman-Brown and the Kuder-Richardson reliability coefficient formuli. The rI~ values were 973 and .85, respectively, determined on the second test. The content validity of this evaluation instrument was examined to aid in evaluating present results and to improve its future validity, reliability, and objectivity. Item analysis
G. Kriegman, M. D. Harris, and E. F. Rosinski
383
was used t o investigate the difficulty and discrimination of each question. Bi-serial coefficients of correlation were not done, but an estimate of the discriminating ability showed that approximately 6o percent of the questions discriminated the top quarter from the bottom quarter. The difficulty index on the pre-test was approximately .50 and on the post-test, approximately .3o. It was not possible to determine statistical validity under the present conditions. Table : shows results of testing, with information on profession, age, and educational level. A graphic picture of the overall effects can be seen on Figure 1, which shows the shift from pre-test to post-test. Presented as the total number of incorrect responses, the mean and median of scores were fairly close and the distributions of the scores resembled those of a normal distribution. Table : also shows various professional group effects, with significant differences in the teacher group, the social work group, and the nursing group. The other groups were too small in number to show significant differences between the two tests. The overall effects of pre-test mean and post-test mean of incorrect answers were significant at a p
ABSTRACT:Paramedical personnel offer a ready reservoir of partially trained personnel who can assist in combating the ever-increasing problem of emotional disorders. A multi-disciplinary program of continuing education for these allied professionals is outlined. Four courses, Personality Development, Emotional and Mental Disorders, Skills and Techniques in Working with Patients and Families, and Sociological Factors in Mental Health, have been developed. The results of three pilot projects are summarized, and future plans and problems are outlined. Evaluation of projects indicates that formal learning of mental health information occurs and greater mutual respect and unity develop among multi-disciplinary participants. The present and anticipated shortage of trained mental health personnel makes it imperative to strengthen and update the skills of allied personnel who are currently working with the emotionallydisturbed. Welfare workers, probation officers, visiting teachers, public health nurses, and others are performing a considerable degree of the initial corrective, preventive, and follow-up work with disturbed persons of all ages. Although competent in their specialties, many of the allied personnel have had limited experience with modern mental health concepts. To enhance their potential as vital parts of community and national mental health care, and to help meet the rapidly growing need for trained workers, a program of continuing education, orientation, unification, and recognition is essential. In the past, countless seminars, workshops, and lecture courses have been conducted under local or statewide sponsorships. As a rule, such programs have been "one-shot deals" aimed at a particular professional group. Their subject matter has been highly specialized, and little overall planning and follow-up Dr. Kriegman is Clinical Professor of Psychiatry, and Dr. Harris is Career Training Fellow and Instructor in Psychiatry, Medical College of Virginia, Richmond, Virginia. Dr. Rosinski is Deputy Assistant Secretary for Health Manpower, Department of Health, Education, and Welfare, Washington, D. C. The authors wish to acknowledge the help of the following colleagues: Virginia State Department of Education--W. Kuhn Barnett, Rosa Loving, Nettie T. YowelI; Virginia Department of Mental Hygiene and Hospitals--Margaret L. Cavey, Arthur Centor, James B. Funkhouse, Elisa Haga, Benedict NagIer; Virginia Department of Public Welfare and Institutions--Herbert A. Krueger; Department of Vocational Rehabilitation-R. I47. McLemore; Medical College of Virginia--Morton D. Schumann, Patricia Wiley; University of Virginia School of Medicine--Bruce Smith. Comrnuni~y Mental Health Journal, Vol. 4 (5), 1968
377
378
Community Mental Health Journal
reinforcement have been attempted. They have not shown significant longrange effectiveness, nor have they generally induded the broad spectrum of personnel who could and should benefit from them. A planned program--well organized, properly evaluated, and involving all allied mental health personnel within a specific region--would foster uniform educational opportunities, would have more far-reaching results, and would have greater long-range value to the particpants and to the communities they serve. This paper reports an effort to devise such a program and to evaluate its efficiency. BACKGROUND OF PROGRAM In -r963, The Virginia Advisory Committee on Inservice Training in Mental Health decided to develop a multi-disciplinary, continuing education program for allied mental health personnel. The objectives of the program were: I. To provide comprehensive inservice mental health training and continuing education for allied professionals on a multi-disciplinary basis. 2. To ensure that all mental health workers, within their areas of competency, provide consistent, unified, high-level mental health services through broadened mental health knowledge and principles. 3. To promote coordination and collaboration of professional workers through their equal participation in a multi-disciplinary training program. In order to develop a program that would be properly grounded in the principles of education, a Subcommittee on Methods was formed to study teaching techniques. In addition, it was decided that a leadership group should form the apex of an inverted pyramid, spreading its knowledge to the broader range of allied mental health personnel. With the assistance of Health, Education and Welfare, Region III, a threeday workshop was organized to study recent educational methods, materials, theories, and evaluating techniques. This workshop was co-sponsored by the Medical College of Virginia's Office of Research in Medical Education. Sixty training officers from the state departments of education, rehabilitation, welfare, and mental hospitals and clinics attended. With the knowledge gained from the methods workshop, the Subcommittee on Content, representing a wide range of disciplines, devised four courses which were judged to fulfill the basic needs for competency in the mental health field. These courses were entitled: Personality Development, Emotional and Mental Disorders, Skills and Techniques in Working with Patients and Families, and Sociological Factors Related to Emotional and Mental Disorders. The first three of these courses have been tested on a pilot study basis.
Pilot Project The objective of the first course (Personality Development) was to provide the understanding of human behavior necessary for dealing
G. Kriegman, M. D. Harris, and E. F. Rosinski
379
with mental health problems. Twelve sessions of a two-hour weekly meeting were conducted at Central State Hospital, Petersburg, Virginia, with co-sponsorship by the Medical College of Virginia, Department of Psychiatry. The curriculum, outlined according to lecture sequence, was: z. & 2. Introduction to the multiple factors that determine behavior; general characteristics of behavior; personality; concept of self; attitude toward death. 3. Stages of development, birth through early infancy. 4. Behavior in late infancy; physical and emotional needs. 5. Behavior levels in childhood; physical, emotional, and sexual needs. 6. Characteristics of the juvenile period; physical, emotional and sexual factors. 7. Characteristics of pre-adolescence; physical, emotional, and sexual factors. 8. Stages and characteristics of adolescence; physical, emotional, and sexual factors. 9. Definition of maturity. Io. & II. Stages of adulthood; characteristics and physical changes of each stage. Ia. Aging and senility; physical, emotional, and sexual changes. Fifty-three allied mental health workers attended the course. After each onehour lecture given by a psychiatrist, the participants divided into small discussion groups. At the conclusion of the course, a final examination (3 ~ multiple-choice questions) was administered, resulting in a mean of 6. 5 and a median of 6.o errors. Also at this time, the leaders of the discussion groups answered a questionnaire, and their questions revealed that: I. Overall, the material was comprehended, but the first three lectures (the most technical, abstract, and biologically oriented) were the most difficult for the participants. 2. The central theme varied among the discussion groups. As each group became more confident, occupational identification disappeared and a more equal interchange became evident. Each group began to focus on the lecture material. 3. No one occupational group surpassed the others. In one group, the leader felt that the occupational and recreational therapists benefitted most. In another group, the attendants and nurses seemed more activated. 4. There should be greater use of visual aids, outlines for the use by discussion groups, and further development of clinical examples in the lectures. One leader stated that the heterogeneous make-up of the group was an important factor in the much higher degree of motivation than has usually been seen in homogeneous groups, and that, in any further work of this sort, the groups be represented by all educational and job levels. 5. There was interest in determining methods for objectively measuring the
380
Community Mental Health Journal
effectiveness of this type of program, although the consensus was that the course was effective. 6. The participants wished to take future courses that emphasized sections of the course related to their individual interests. They wanted more time for discussion groups and increased structure and direction by the leadership. They also desired more audiovisual aids and supplemental reading. Pilot Project 2
The second course (Skills and Techniques in Working with Patients and Families) was given in a three-day workshop at Radford College, Radford, Virginia. Again, the Medical College of Virginia, Department of Psychiatry was co-sponsor. The objective of this course was to develop understanding of the techniques required in working with emotionally disturbed persons and their families. The three-day curriculum highlighted: 5. General principles in counseling persons with emotional disorders. 2. Understanding the patient and his family as a basis for services; nature of disorders; physical health; emotional interaction of family; living arrangements; finances; feelings and attitudes of patient and family; evaluation and planning of services. 3. Principles of relationship with patients and families; understanding of self and the role of the professional worker; essential factors in relating the patient and family; principles of interviewing; continuing contact with patients and families; professional responsibility. Lecturers were: a psychiatrist, clinical psychologist, family social worker, psychiatric nurse, and special education teacher. All but the psychiatrist acted as resource persons for discussion groups, which were conducted after each panel presentation, morning and afternoon. The 32 participants, all from southwestern Virginia, were administered identical tests of 3o multiple-choice questions at the beginning and end of the course and, as Table z shows, participants over 35 years old performed relatively poorly on both examinations and showed the least improvement between the initial test and the final test. At the completion of the course, the instructors were asked to evaluate the program. They observed that the older professionals tried to dominate the early discussion periods: personal dinical experiences, complaints, personal and family problems stimulated by the lecture material tended to dominate the discussion period in the first two or three discussion periods. The resource faculty member was minimally involved in the group. However, as group work continued, a gradual change occurred. The young workers began to be more expressive and assertive, and the elder professionals' domination diminished. Furthermore, the groups focused more and more on the lecture material and increasingly turned to the faculty resource person for assistance and advice. In the final sessions, the focus was primarily on the lecture material, and the resource person was fully integrated into the group.
G. Kriegman, M. D. Harris, and E. F. Rosinski
381
TABLE x Pre- and Post-Test Scores of Participants in the Workshop on Skills and Techniques in Working with Patients and Families Pilot Project #2 Unmatched Examinations Unmatched Examinations Examination
No.
Mean Incorrect
Range
I II
26 27
"rX.O 8.5
6.5-20.0 3.o-z6.5
Matched Examinations Post Test Mean Incorrect
Professions
No.
Age (mean)
Case or Welfare Workers Teachers
xx M-~ l::--r0
35.2 22-60"I"
xz BA
xo.7
7.2
7 M-2 F-5
52.7 34-60+
x2.3
~o.x
4 F-4
z-2 yr Coll 3-BA 3-MA 4"RN
"r162
:~o.o
BA, BD
~o.o
8.o
z-2 yr Coll x5-BA, BD 4-RN 3-MA
xx.32
8.60
~x.83 ~o.37
~o.53 5.00*
Nurses Minister
M-x
45.2 25-67t 37
Totals
23 M-4 F-z9
42.3 22-67
~5 8
+33 --33
Ed. Levcltt
Pre Test Mean Incorrect
%--Age range %~---BAor BD--college degree MA--graduate degree * t--6.7z 4 p < .ooi Pilot Project 3 The third course was composed of an eight-session, threehour weekly seminar on Emotional and Mental Disorders and was attended by 93 allied mental health workers at the Medical College of Virginia, Richmond, Virginia. Each session consisted of a one-hour lecture, a one-half hour dosedTV presentation of clinical examples, and a one-hour discussion group. Material covered during the course was: 9 . & 2. Introduction and general background related to basic syndromes and mental mechanisms. 3. Personality disturbances.
382
Community Mental Health Journal
4. Sociopathic personality; alcohol and drug addiction; sexual deviations. 5. Anxiety states and depression. 6. Hysteria. 7. Compulsive-obsessive states. 8. & 9. Manic-depressive psychosis. lo. & II. Schizophrenia. 9 2. Paranoia and paranoid conditions. 13 . Psychoses due to: infection; intoxication; drug addiction; trauma; disturbance of circulation; tumor. Five discussion groups were designated and led by a college faculty member or a senior resident in psychiatry. An observer equipped with a detailed discussion guide outline and scoring sheets was assigned to each discussion group, and each discussion group was audio-tape recorded. Every participant was given the same examination at the beginning and at the end of the seminar. This time the following procedures were used to evaluate the participant's growth of knowledge and the success of the program's objectives. I. The five discussion groups were carefully selected. The typical group consisted of six or seven teachers, four or five welfare workers, one or two ministers, one nurse, one physician or more, and one or two other related workers, such as occupational therapists or speech correctionists. There was an equal distribution in educational level, age range, and male-female ratio (3 to 4 males per group). The Negro-Caucasian ratio was consistent. 2. The discussion groups were analyzed with respect to leader behavior, group interaction, and individual behavior. Each observer was given a guide and a scoring sheet, which covered the physical components of the group, content of the meetings, leader behavior, and group roles (divided into task roles, group interaction roles, and anti-group roles). Meeting-to-meeting comparisons were also noted. 3. The audio tapes were evaluated in terms of depth of cognitive and affective learning, analysis of group processes, comparison with scoring sheets data and interactional analysis. (This latter technique as developed by N. Flanders (~959) rates primarily the percentage of activity in the group of (a) the leader and (b) the participants. This includes both direct and indirect influencing interventions by the leader and the responses to the leader by group members, as well as group initiated interactions.) 4. A 3o-question, multiple-choice, four-item test was used to evaluate the prior knowledge and the learning progress of the individual participants. It was unsigned. Reliability of the instrument was determined by the Spearman-Brown and the Kuder-Richardson reliability coefficient formuli. The rI~ values were 973 and .85, respectively, determined on the second test. The content validity of this evaluation instrument was examined to aid in evaluating present results and to improve its future validity, reliability, and objectivity. Item analysis
G. Kriegman, M. D. Harris, and E. F. Rosinski
383
was used t o investigate the difficulty and discrimination of each question. Bi-serial coefficients of correlation were not done, but an estimate of the discriminating ability showed that approximately 6o percent of the questions discriminated the top quarter from the bottom quarter. The difficulty index on the pre-test was approximately .50 and on the post-test, approximately .3o. It was not possible to determine statistical validity under the present conditions. Table : shows results of testing, with information on profession, age, and educational level. A graphic picture of the overall effects can be seen on Figure 1, which shows the shift from pre-test to post-test. Presented as the total number of incorrect responses, the mean and median of scores were fairly close and the distributions of the scores resembled those of a normal distribution. Table : also shows various professional group effects, with significant differences in the teacher group, the social work group, and the nursing group. The other groups were too small in number to show significant differences between the two tests. The overall effects of pre-test mean and post-test mean of incorrect answers were significant at a p
