Journal of Advanced Nursing, 1978, 3 , 457-473
Discussion of a project on peer evaiuation in tiie facuity of nursing, University of Toronto, Canada Lettie Turner R.N. B.N. M.P.H. M.H. Associate Professor, Faculty of Nursing, University of Toronto, Canada Accepted for publication 14 March 1978
TtTRNER L. [igjS) Journal of Advanced Nursing 3 , 457-473
Discussion ofa project on peer evaluation in the faculty ofnursing, University ofToronto, Canada The purpose of this project was to develop guidelines and involve all faculty in the evaluation of teaching and learning. The main objective -was to provide the opportunity to assist faculty to become actively involved in self and peer evaluations. Three i day -workshops were held in 1975 involving all fulltime faculty and some part-time faculty. Two i day workshops were held in 1976 involving mainly new faculty, with a focus on peer evaluation in the clinical area. Criteria were developed and are used as guidelines in evaluating self and peers. The findings indicated that evaluation posed a threat, especially in evaluating one's peers, Ijut this threat tended to decrease through involvement in the process, which included group discussions and talking and sharing feelings both in formally devised and spontaneous informal encounters. The conferences, which brought peer partners together before and after the observation of performance, proved to be a most positive experience. Teachers identified problems in criteria selection and provided criticism. They also have expertise that they can share -with each other and they can identify areas requiring improvement, but they still require some impetus to spur them on to change in behaviour. Recommendations included that each faculty member take the responsibihty to keep self and peer evaluations in her o-wn profile and provide this data to administration if and \vhen desired.
INTRODUCTION Teaching-leaming evaluation Evaluation of teaching effectiveness is of interest and concern to three groups— administrators, students, and the teachers themselves. For an evaluation system to be effective in meeting the goals of a faculty, it must provide for the objective assessment of the skills, abilities and knowledge of each individual teacher, so that administrative decisions can be made relating to the development of the teacher, promotion, transfer, salary adjustment, and so on. Mainly, the system needs to maintain and enhance faculty development. Evaluation can also be used to obtain data for research purposes. 0309-2403/78/0900-0457 $02.00 © 1978 Blackwell Scientific Publications
457
458
L. Turner
The teacher's intent is to stimulate learning for the student, but the result is not necessarily attained. Teaching can be defined as an interaction between a teacher and one or more learners. Yet, there is no single concept of teaching. Gage (1972) said that a theory of teaching should answer three questions: (i) How teachers behave? (2) Why they behave as they do? (3) What the outcomes or results are? Theories of learning describe and explain the conditions under which learning does and does not occur. Theories of learning are considered more broad and more basic than theories of teaching. However, the two processes, teaching and learning, can be regarded as coordinate and interactive, neither one completely controlling the other. The behaviour of teachers is only one special category of the environmental situation in which learning occurs. If teaching is seen as a relationship, the teacher's personality as perceived by the student creates an atmosphere for learning. It is vital to bear in mind, however, that there is no one way to create a relationship. People appear to learn from many different teaching methods, as Bruner (1977) said, there are many diverse routes by which students get to deeper understanding. Evaluation is the systematic collection of evidence to determine if changes have occurred in the learners, as well as to find out the amount of change in each learner. Professional growth of the teacher requires that teachers secure evaluation evidence and other facts so that they can continue to improve their teaching and the students' learning.
Background of the University of Toronto project In the Autumn of 1974, the faculty of nursing set up a subcommittee on teachinglearning to study the university's newly promulgated criteria of teaching and to suggest methods by which the teaching-learning evaluative process could be implemented. This author was chairman of the subcommittee and having been named faculty fellow under a university programme focusing on the improvement of teaching and education development, decided to apply the resultant funds to a project in peer evaluation to be undertaken by the committee. The project would approach the evaluation of teaching-leaming by attempting to develop and test peer evaluation criteria and forms for use by members of the nursing faculty. The emphasis on peer evaluation or review is comparatively recent; hence, it seemed worthwhile to share the way our faculty went about it in our educational setting. The procedure included the development of a model. Models as prototypes of theories make possible early conceptualization and the study of phenomena, and useful models can lead to supportive theories. R E V I E W OF THE L I T E R A T U R E Only a small percentage of educational studies have considered teacher behaviour systematically. Many studies did not include observations of teachers and therefore
Discussion of a project on peer evaluation
459
did not provide information on teacher behaviours and student outcomes, or teacher behaviour in the classroom setting. In an extensive review of the literature relevant to evaluation effectiveness, the main points may be summarized under three aspects of the problem: (i) the teaching-learning process, (2) the evaluation process and (3) the evaluating instruments.
The teaching-learning process ' Readiness' refers to the general nature of the responses the learner has available for solving problems. The mental, physical and emotional state of the learner has much to do with the efficacy of any given learning experience. Cultural, motivational and personal factors affect the desire to learn and to undertake problemsolving and the general principle is that without appropriate readiness a learning experience will be inefficient, or learning will not occur. The principle stresses learning as a sequential process in which past learning serves as a basis for present and future learning. General mental ability and previous academic accomplishments have a significant influence on the learner's ability to acquire new material. Teaching is a complex, subtle and varying activity which may be more dependent on personality, attitudes and human relationships than on method. This interpersonal manner is the message, in McLuhan's (1967) language. Sheffield (1974) supported this premise when he said that attitudes toward students and teaching are more important than methods or technique. Of categories of criteria identified by various authors, those most frequently identified have been (i) attitude toward students, (2) presentation, (3) personal attributes, (4) class atmosphere, (5) course mechanisms and (6) attitude toward subject. Biddle (1964) said that although approximately 10 000 studies had been reported on the relationship between the characteristics of teachers, teacher behaviour and educational goals, many facts had not been identified concerning teacher effectiveness and neither had any approved way of measuring competencies been accepted, or any method of promoting teacher adequately been widely adopted. In the late 1950s, Bradford (1959) expressed his views on the teaching-learning transaction. He said that teaching is not a matter of filling a void with information but involves internal reorganization of a complex of thought patterns, perceptions, assumptions, attitudes, feelings and skills, and testing this organization out in relation to problems of living. Rogers (1959) discussed conditions which facilitated learning and stressed that one of the most important of the conditions was attitudinal quality of the interpersonal relationship between facilitator and the learner. Theobald (1976) stressed the need for effective action to create transitions into the communication era, which requires that we once again become sensitive to each other. 'We need', he said, 'to develop interpersonal understanding at levels far higher than those that now exist.'
460
L. Turner
Sheffield (1974) in Canada, and Buxton & Prichard (1975) in the USA, compiled books along similar lines in which they invited outstanding teachers to speak for themselves. Sheffield asked graduates to name professors they had known as excellent teachers and to say what they felt had made these professors and their teaching effective. The contributors to both books comprised a very diverse group of college teachers and a variety of teaching styles. The lecture was still the dominant mode of instruction, but communication and interaction with students was an essential part of the professor's style. Among the very important perceptions that the professors had were that the teacher's most important function is that of stimulating students to be active learners on their own; that effective teachers think students are important, like them, respect them, and care; that effective teachers have both a love and open enthusiasm for what they teach; and that they work hard at teaching. The students who selected teachers on their excellent performance were in agreement on their performance in five areas—they showed mastery of the subject and competence; their lectures were well-prepared and orderly; they related their subject to life, and were practical; they encouraged students' questions and opinions; and they showed enthusiasm about the subject. Ashby (1975) stated that the education industry provided excellent vocational training in a great range of skills, as well as fairly good initiation into the professions; that is, the ability to deal with things and the ability to deal with ideas. 'But', he said, 'neither school nor college or university is equipped to teach us how to deal with people, yet that today is the greatest single need among citizens of a democracy. This is the dilemma.' The need for continuing assessment by various sources, both immediate and longterm, of the enduring effect of what teachers actually do in their classrooms has been neglected and the need for studies was stressed. One of Sheffield's conclusions was that however much individual professors differ in their style, their personalities, and their methods, the resulting general patterns of effective teaching are essentially similar in all settings (the last remark indicated to the author that criteria could be developed around this pattern of similarity).
The evaluation process In the 1970s, renewed interest was generated in evaluating teacher performance. Krasno (1972) stressed the need and importance of research in teaching effectiveness based on longterm consequences. Evaluation is a cooperative process wherein the evaluator and the person being evaluated feel a mutual responsbility to centre upon performances requiring improvement as well as those showing strengths, to attempt working together to reach the best results, and to evaluate the results. Sheffield (1974) pointed out that professors, like students, learn only when they are active participants in the process, and only when they want to—imposed programmes of teacher improvement are unlikely to work, but professors themselves may initiate them if they feel it is worthwhile.
Discussion of a project on peer evaluation
461
Dornbush & Scott (1975) showed that teacher satisfaction with the evaluation is vital to the process and that knowledge of criteria, frequency of communication, evaluation, and influence on the evaluation process hy those involved, contribute greatly to such satisfaction. The relevance of beliefs and values to each perceiver and his frame of reference was highlighted. As Pearce (1973) said, 'We are the sovirce of values and choices, the source of ideas which the procedure of our reality orients', but 'we are shaped by each other. We adjust not to the reality of the world, but to the reality of other thinkers.'
Evaluation instruments Wasserman & Eggert (1976) suggested the evaluation instrument include the school's educational values, teacher behaviours which are truly prized, behaviours related to the pupils' learning, and provision for continued self-scrutiny and estimate of professional growth. Centra (1975) reported that to improve reliability of peer evaluation, faculty members needed to invest more time in visitation or training sessions. Work is visible to the extent that those evaluating the performance are able to observe relevant aspects of the task and outcomes produced by those performing. Attention was directed to having the rating scales large enough to allow for sensitivity in testing.
THE C O N C E P T OF E V A L U A T I O N
Definition of terms Evaluation is a systematic process of judging the worth, desirability and effectiveness, or adequacy, of something according to defmite criteria and purpose. The judgement is based on careful comparison of observation data with criteria as guidelines. Peer evaluation in teaching is defined as a process engaging two persons, each considered to have the necessary abilities, education and qualifications to critically assess the teaching effectiveness of the other. In some instances there is a mutual assessment of teaching effectiveness. Observing involves the intentional and the theoretical viewing of some object or activity. Observing is more than seeing and it involves planned, careful, focused, active attention by the observer. Measuring is the assigning of quantitative values to a group of people and objects, or activities, according to some established rules. It is a descriptive activity. Formal organization applies to those distinctive aspects of a project which have been consciously planned (e.g. workshops). Informal organization is conceived as being aspects that are not formally planned but that more or less spontaneously evolve from the needs of people (e.g. experience sharing).
462
L. Turner
Limitations 1 Most teachers dislike having to judge their peers. Negative effects in interpersonal relations are feared. 2 The data may represent what the evaluators think they should perceive and say about themselves and about their peers. 3 Because of the lack of visibility of teachers' work with students, peer evaluation of work in the clinical setting, in the small-group setting and in the one-to-one situation, may be viewed as less soundly based than peer evaluation in team situations and in lectures. Assiunptions 1 Peers can make a contribution to the evaluation of faculty performance. 2 Teachers will take evaluation more seriously when they are responsible for delineating their own learning needs and receiving assistance. 3 Neither the use of student, self, nor peer ratings, leads automatically to improvement in faculty development. Ratings themselves provide information only and not necessarily the desire to change. 4 Evaluations are more effective when planned cooperatively by teachers and administrators. 5 Peer evaluation takes time, work and patience to develop a working model. 6 Peer evaluation cannot be forced on any teacher. As a concept, peer evaluation takes time (relative to each individual) to internalize the concept. If the teacher does not view it as her responsibility and right, it cannot be effective. DESIGN OF T H E P R O J E C T The design of the project was flexible, building on approaches to the problem that had already been made by the committee and incorporating a nvimber of aspects that evolved as the project advanced. It may, however, be seen as having two main phases—development of evaluation criteria, forms and of a model of the evaluation process, and explaining and applying the evaluation process. It must be kept in mind that these phases overlapped considerably and faculty involvement was encouraged throughout. In the research of literature the finding of greatest importance to this and future projects, and research relating to the evaluation of teaching effectiveness was that there is a general pattern of effective teaching similar in all settings. Approach to the problem Preliminary procedures. Preliminary steps in the evaluation of the teachinglearning process had already been taken prior to this project. The committee on teaching-learning effectiveness had been examining the criteria set out by the university with a view to clarifying them for the faculty of nursing setting. A student-teacher rating form had just been developed and was being put into practice. The broad criteria for quality teaching defmed by the university were:
Discussion of a project on peer evaluation
463
1 The degree to which the teacher has demonstrated mastery of his or her subject area. 2 The degree to which the teacher has the ability to communicate well. 3 The degree to which the teacher is able to stimulate and challenge the intellectual capacity of the student. For the project, criteria i and 2 remained unaltered, criterion 3 became the degree to which the teacher can stimulate and challenge the cognitive and affective capacities of the student. In a professional faculty, the clinical expertise of the teacher had to be considered. Therefore, a fourth category was added to supplement the university criteria, namely, the degree to which the teacher demonstrates integrity as a professional person. Consideration was given to the working model of the student-teacher evaluation guidelines. One of the committee's objectives was to have the peer evaluation form parallel other forms used in evaluation. The one source of evaluation, whether student, peer or self, was not to be taken by itself as adequate; in fact each faculty member would have several sources from which to collect data, in keeping her own profile, and each source would contribute to the review process.
Evaluators The project had as its prime goals self-development by the staff, hence each individual teacher was to be a key participant. Both present and former students had an obligation to be involved as it was their learning experience. Colleagues' feedback was to be another invaluable source of input, as ongoing sharing for enhancing strengths and improving weaknesses was an aim of the process. Evaluation by administrators was also to be encouraged as they assume the fmal responsibility for the quality of teaching within the university setting.
Explaining and applying the evaluation process It was decided to hold a series of workshops for faculty members to promote understanding and acceptance of the evaluation process and to test the evaluation form. Faculty members would be asked to do evaluations, using a form incorporating any revisions found necessary. Among the aspects that evolved were conferences to be held before and after the evaluation observation periods and assistance to faculty members in making a plan for improvement.
Developing a model The model of the evaluation process developed in the process of approaching the problem is given in Figure i. It incorporates some features that developed themselves as the project advanced, as, for example, the pre-observation conference.
464
L. Turner Assigning goals
Criteria development
Conference mutual exchange
Performance observing
Outcome appraising
Conference mutual exchange
Planning for improvement
FIGURE I
Model of the evaluation process
The model of the evaluation process shows the elements in developing and applying the process. Once the steps of assigning goals and developing criteria are accomplished and expressed in guidelines, the evaluator and evaluatee meet in conferences (mutual exchange step three). They discuss the use of the guidelines (questionnaire) and how to arrange observation of performance to their mutual satisfaction. After the steps of observation of performance (whether in a lecture or clinical setting), the evaluator appraises the outcome as does the evaluatee. They then confer about the outcome and plan for improvement. The process then recommences. METHOD
Development of an evaluation form An evaluation form was developed, applicable to all faculty involved in the teaching-learning situation. All categories of criteria might not have the same
Discussion of a project on peer evaluation
465
importance for any two persons because of differences in their teaching responsibilities. The categories of criteria on the form were the same as, or as close as possible to, the wording and intent of the student-teacher, former student and administrative forms. It was hoped that this parallelism would decrease confusion and increase utilization by employing widely understood criteria. The exact grading scale was to coincide with that of the student-teacher evaluation form. Faculty members were involved in developing criteria by faculty meeting used as a vehicle for keeping faculty informed, consultation and resource materials through the faculty fellow, and workshops—three i day workshops held in February, March and April 1975 (referred to as workshop i) and two i day workshops held in March and April of 1976 (referred to as workshop 2).
The criteria The draft of criteria guidelines developed for use in the workshops consisted of six categories: 1 Mastery of subject area. 2 Application of knowledge and skills to patient care. 3 Preparation for the learning experience. 4 The learning situation. 5 Evaluation of the learning situation. 6 Student development. Within each category several areas were expanded and more clearly defined than they had been in the student-teacher form.
Workshop i; increasing faculty's understanding of teacher evaluation Workshop i was designed to assess and test the guidelines for evaluation of teaching-learning. The discussion revealed the threat in evaluation. Many questions were raised as to exactly how the result would be used and how valid the peer evaluation would be. The categories were discussed, with clarification in some areas and suggestions for rewording or change in other sections. The recommendations at the end of workshop i were to make the categories into questionnaire format and attempt to use the tool as a basis for self and peer evaluation.
Testing of the questionnaire; subsequent change and retrial Following workshop i, the guidelines were put into questionnaire form, with a 3-point grading scale for guidelines of mastery of subject and a 5-point grading scale for the remaining areas. The questionnaires were distributed to the faculty for a trial testing. The trial pointed out several problems and many questions, but it seemed to lessen anxiety about the process. The 3-point grading scale in category i (mastery) provided too narrow a range. The scale was changed for the next trial to a 5-point scale. The wording in some areas was too vague and so further revisions of those areas were made. There were several comments about the lack of criteria for
466
.
L. Turner
professional integrity, and this new area was added. The revised form is the format that has been retained for use. Faculty were asked to fill out a self-evaluation and have two peer evaluations done. New faculty had been introduced to the evaluation of teaching-learning and they were asked to participate. There were fewer comments about the form itself and the revision seemed to have corrected the initial problems. At this time, another grant was obtained by the faculty fellow and the decision was made to hold another series of workshops for the faculty. From discussion, the committee felt that though self-evaluation was less anxiety-provoking than previously, peer evaluation still carried some threat. This theme was to become the focus of the next workshop. Workshop 2; peer evaluation—openness and trust Two workshops were held and involved 24 faculty members a year following workshop i. It had been decided to work with new faculty mainly, though not entirely. The discussions revealed that evaluation still aroused anxiety and an attempt was made to encourage individuals to recognize evaluation as an ongoing learning experience. The discussion also pointed out that new staff members are ill-prepared in this area because of the limited time spent on it during their orientation and the low priority which their evaluation had in the initial weeks of employment. Recommendations arising from this workshop were valuable and some are incorporated into the committee's final report for the faculty. FINDINGS AND DISCUSSION Development of goals and selection of evaluation criteria and forms Teachers and administrators were successfully involved in the process of identifying criteria to be used in evaluating teacher effectiveness. The workshops were invaluable in helping to clarify statements and lessen the threat around evaluations. This procedure tends to delay coming to grips with the practical tasks of evaluation, but it proved to be a necessary process for cognitive and affective growth and for creating a readiness to deal with the inherent threat in evaluation. The findings of Bess (1977) are relevant. In his examination of the relationship between important needs of faculty and the process and context of teaching in institutions of higher learning, he said, 'If teaching is to be externally rewarded, it must also be internally rewarding'. If evaluation can be seen as a growth process with active participation of both persons, evaluator and evaluatee, and with effective communication and interaction, then evaluation can become a more positive process with less frustration for each person. The groups brought together formally provided a forum for sharing of thoughts and feelings. The proceedings of each workshop were formally presented to the total faculty. Informal aspects evolved spontaneously and much sharing occurred when several faculty were together. This latter aspect greatly enhanced the evaluation process.
Discussion of a project on peer evaluation
46J
Teacher evaluation and student evaluation As the teacher questionnaire contained terms parallel to the student form, it allowed teachers to identify similarities and differences in their perceptions and the students'. Both were valuable input, which were taken into consideration when teachers were assessing their strengths and weaknesses and making plans for improvement. Choosing a peer Each faculty member was free to choose two peers, or more if desired, to make her evaluations. She was also free to select them from her own or different areas of interest. Peers were often chosen on the basis of friendship or proximity. As respect and mutual trust were emphasized throughout the workshops, liking someone and feeling free to be open with them was also a criterion for some teachers' choice of partners. Mutual respect was a critical factor as a basis for successful peer relationship. Explaining and applying the evaluation process No specific time was set during the project for observation. In fact, working out a plan for observation proved to be difficult. As most of our faculty work in teams, the formal lecture did not present as many concerns as the small groups, the one-to-one interaction used in teaching-learning, and especially the work in the clinical areas. Selecting criteria appropriate for classroom as well as for small groups provided some measure of success. Colleagues suggested focusing observation by using specific criteria and different colleagues could also be used in classrooms observation, small-group observation and one-to-one interaction. Except for teachers in the team situation, the lack of visibility of the teacher's work to persons other than their students created problems in evaluation. Under these circumstances there was some hesitancy by teachers to trust the evaluation system. Conferences on appraising performances Conferences were held at some time before and after the observation period. Each teacher shared with her chosen peer her self-evaluation, using the same criteria as those used for peer evaluation. Participants were usually harder on themselves than their peers were. Students were sometimes closer to the participants' own evaluation. Colleagues' evaluation provided positive reinforcement as well as constructive criticism. Suggestions for improvement were usually balanced with praise for effective teaching. Praise by one's own peers seemed to fill a great need so that the conferences proved to be a positive factor in colleague evaluation. Suggestions were made that a pre-observation conference be held so that expectations regarding observations could be exchanged. The conferences also provided a process for discussing teacher-student interactions, a matter of great concern to the faculty.
468
L. Turner
The Improvement Plan Developing a plan for improvement is one of the vital steps in the colleague evaluation process, whereby each teacher integrates all information she has received from self-evaluation, student evaluations and peer evaluations. Strengths and weaknesses can be listed and ways defmed to improve teaching effectiveness. Minor changes were dealt with within the present system, but for some competencies and skills the organization of a resource centre seemed indicated.
PRINCIPLES AND CONCLUSIONS D R A W N F R O M THE PROJECT 1 Involve all faculty in the development of evaluation criteria. Involvement serves two purposes—the position is defmed more accurately and the morale of the faculty is enhanced. 2 Faculty involved in the evaluation process need to understand the purpose, the procedure to be used and the roles of the people involved. Faculty need to be kept informed and they need to talk and share feelings in groups. 3 Effective evaluation of teachers is dependent upon the quality of communication between teachers and administrators. 4 The guide or form for evaluation should be used on a trial basis, so that faculty become acquainted with its use. 5 Allow faculty time to work through feelings regarding evaluation. 6 Emphasize the sharing aspect between peers, observer and observee, both before and after observation. The sharing can be very rewarding and helpful for both peers. 7 Keep evaluation of teacher performance teaching-learning centred, cooperatively designed and implemented. 8 Maintain faculty accountability by having each faculty member take responsibility in keeping an up-to-date profile. Recommendations for faculty 1 The Self and Peer Teaching-Learning Evaluation Form, as developed, be retained for use for the next 2 years. In April 1978, a committee be appointed to reevaluate the effectiveness of this form. 2 Each faculty member take the responsibility to do her own self-evaluation yearly. 3 Each faculty member take the responsbility to have two peer evaluations done yearly. 4 Each faculty member take the responsbility to keep self and peer evaluations in her own profile and provide this data to administration if desired. 5 Administration take the responsibility to introduce each new faculty member to peer and self-evaluation during orientation, with a follow-up session within 3 months to further explain the criteria and process.
Discussion of a project on peer evaluation
469
Recommendations for further study The work in the teaching-learning evaluative process had indicated that the reliahility and validity of the questionnaire should be tested further before placing weight on its results. It would also be interesting to determine if there is any relevancy in comparing the peer evaluations and student evaluations of the individual teacher. Study in these areas would provide administration and the teacher with a better understanding of what these results really mean. The next and perhaps most important step is to ensure that there are resources available within the faculty or the university where any individual teacher can receive counselling or instruction to maximize her strengths and improve her weaknesses. The utilization of such resources would be the teacher's responsibility. CONCLUSION It has been proposed that teachers have expertise that they can share with each other and that they can identify areas requiring improvement in the evaluation of their peers in teaching and learning. Evaluation posed a threat, which decreased as faculty became involved in the process, for, as Eiben and Milliren (1976) pointed out, creative and self-reliant learners develop best in an atmosphere of responsible freedom wherein self-evaluation and self-criticism are fostered. Evaluation is a fundamental process necessary to effective teaching-learning situations, but teachers need to feel involved and have power to affect the evaluation system. Reilly (1959) said that there is only one thing which really trains the human mind and that is the voluntary use of the mind by man himself. You may aid him, you may guide him, you may suggest to him and above all, you may inspire him, but the only thing worth having is that which he gets by his own exertion and what he gets is proportionate to the efforts he puts into it.
References AsHBY E. (1975) Dilemmas of Modern Man. Great-West Life Assurance Company, Manitoba, Canada. BiDDLE B. (1964) The integration of teacher effectiveness. In Contemporary Research on Teacher Effectiveness, eds. Biddle B. & Ellens W., Holt, Rinehart & Winston, New York. BESS J. (1977) The motivation to teach. The Journal of Higher Education 48, 243-258. BRADFORD L . (1959) The teaching-learning transaction. Adult Leadership 8, 7. BRUNER J. (1977) The Process of Education. Harvard University Press, Massachusetts. BuxTON T. & PRICHARD K . (1975) Excellence in University Teachers. Columbia University Press, New York. CENTRA J. (1975) Colleagues as raters of classroom instruction. Journal of Higher Education, 46, 327-337DORNBUCH S. & SCOTT R . (1975) Evaluation and The Exercise of Authority.Jossey-Bass, San Francisco. EIBEN R . & MILLIREN A. (1976) Educational Change: A Humanistic Approach. University Associates, California. GAGE N . L . (1972) Teacher Effectiveness and Teacher Education: The Search for a Scientific Basis. Pacific Books, California.
470
L. Turner
KRASNO R . (1972) Accountability and research on teacher effectiveness. Administrator's Notebook 21, 1-4. McLuHAN M. (1967) The Medium is the Message. Bantam Books, New York. PEARCE J. (1973) The Crack in the Cosmic Egg. Pocket Books, New York. REILLY W . (1959) Successful Human Relations. Harper & Row, New York. ROGERS C . (1969) Freedom to Learn. Charles E. Merrill, Columbus, Ohio. SHEFFIELD E. (1974) Teaching in the University: No One Way. McGill-Queen's University Press, Montreal. THEOBALD R . (1976) Beyond Despair: Direction for American's Third Century. The New Republic Book Company, Washington D.C. WASSERMANN S. & EGGERT W . (1976) Profiles of teaching competency: a way of looking at classroom teaching performance. Canadian Journal of Education I, 67-75. APPENDIX UNIVERSITY OF TORONTO Faculty of Nursing September 13,1976 To: All Faculty Re: Self and Peer Evaluation Each faculty member has the responsibility of keeping an up to date profile of her over-all teaching-learning activities. You have three sources available within the faculty from which you can compile information, peer, student and self-evaluation. This information will be requested as required for administrative decisions and for your own academic growth. Lettie Turner Peer Evaluation (Chairman) LT/dp
U N I V E R S I T Y OF T O R O N T O FACULTY OF N U R S I N G Self and Peer Evaluation Form I The degree to which the teacher demonstrates mastery of his/her subject area. Category i: Mastery a Has a theoretical base in nursing. b Has knowledge of the content of core in the curriculum. c. Has knowledge of the content and skill (i) in speciality area (ii) in area of function (teaching) d Integrates principles from own and other disciplines in teaching. e Integrates relevant research findings and recent developments in teaching and practice.
Very Good Good Acceptable
Poor Comment
Discussion of a project on peer evaluation
471
f(i) Maintains and increases knowledge; (ii) maintains and increases nursing practice. h Influences the student through guidance, direction and/or example to maintain and improve the quality of patient care.
2 The degree to which the teacher has the ability to communicate well.
Category 2: Preparation for the Learning Situation a Prepares individually and /or as team members for (i) classes (ii) seminars (iii) labs (iv) advising b Clarifies learning goals which facilitate student learning in achieving course expectations. c Recognizes the variables affecting the learning situation. d Selects appropriate learning experiences for students; and helps students select appropriate learning experience.
Category 3: The Learning Situation a Explores with the student her goals in relation to the course goals. b Presents ideas with an understanding and awareness of the students' readiness. c Develops ideas in thoughtprovoking and stimvilating ways.
Consistently Usually Sometimes Seldom Rarely Comment
472
L. Turner
Category 4: Evaluation of the Learning Situation a
Evaluates on the basis of the learning goals.
b Uses criteria for test and assignment evaluation which can be defined to students. c
Uses methods of evaluation appropriate to the learning situation.
d Provides feedback.
constructive
3 The degree to which teacher can stimulate challenge the cognitive affective capacities of the rlpnt5 dents.
the and and stu-
Category 5: Student Development a
Encourages critical and independent thinking and action.
b Encourages student's expression of ideas, attitudes and feelings. c Encourages students' participation in developing learning experiences. d Encourages evaluation by students. e Demonstrates concern for the student as a person. f
Encourages the students' growth as a professional person.
4 The degree to which the teacher demonstrates integrity as a professional person.
Consistently Usually Sometimes Seldom Rarely Comment
Discussion of a project on peer evaluation Categoty 6: Integrity a
Demonstrates concern for the patient, his family and community.
b Communicates respect for students and colleagues as individuals. c Meets commitment to students and colleagues. d Evaluates and attempts to improve effectiveness as a teacher. e
Assumes responsibility toward her profession including continuation of personal and professional development.
473
Consistently Usually Sometimes Seldom Rarely Comment
Discussion of a project on peer evaiuation in tiie facuity of nursing, University of Toronto, Canada Lettie Turner R.N. B.N. M.P.H. M.H. Associate Professor, Faculty of Nursing, University of Toronto, Canada Accepted for publication 14 March 1978
TtTRNER L. [igjS) Journal of Advanced Nursing 3 , 457-473
Discussion ofa project on peer evaluation in the faculty ofnursing, University ofToronto, Canada The purpose of this project was to develop guidelines and involve all faculty in the evaluation of teaching and learning. The main objective -was to provide the opportunity to assist faculty to become actively involved in self and peer evaluations. Three i day -workshops were held in 1975 involving all fulltime faculty and some part-time faculty. Two i day workshops were held in 1976 involving mainly new faculty, with a focus on peer evaluation in the clinical area. Criteria were developed and are used as guidelines in evaluating self and peers. The findings indicated that evaluation posed a threat, especially in evaluating one's peers, Ijut this threat tended to decrease through involvement in the process, which included group discussions and talking and sharing feelings both in formally devised and spontaneous informal encounters. The conferences, which brought peer partners together before and after the observation of performance, proved to be a most positive experience. Teachers identified problems in criteria selection and provided criticism. They also have expertise that they can share -with each other and they can identify areas requiring improvement, but they still require some impetus to spur them on to change in behaviour. Recommendations included that each faculty member take the responsibihty to keep self and peer evaluations in her o-wn profile and provide this data to administration if and \vhen desired.
INTRODUCTION Teaching-leaming evaluation Evaluation of teaching effectiveness is of interest and concern to three groups— administrators, students, and the teachers themselves. For an evaluation system to be effective in meeting the goals of a faculty, it must provide for the objective assessment of the skills, abilities and knowledge of each individual teacher, so that administrative decisions can be made relating to the development of the teacher, promotion, transfer, salary adjustment, and so on. Mainly, the system needs to maintain and enhance faculty development. Evaluation can also be used to obtain data for research purposes. 0309-2403/78/0900-0457 $02.00 © 1978 Blackwell Scientific Publications
457
458
L. Turner
The teacher's intent is to stimulate learning for the student, but the result is not necessarily attained. Teaching can be defined as an interaction between a teacher and one or more learners. Yet, there is no single concept of teaching. Gage (1972) said that a theory of teaching should answer three questions: (i) How teachers behave? (2) Why they behave as they do? (3) What the outcomes or results are? Theories of learning describe and explain the conditions under which learning does and does not occur. Theories of learning are considered more broad and more basic than theories of teaching. However, the two processes, teaching and learning, can be regarded as coordinate and interactive, neither one completely controlling the other. The behaviour of teachers is only one special category of the environmental situation in which learning occurs. If teaching is seen as a relationship, the teacher's personality as perceived by the student creates an atmosphere for learning. It is vital to bear in mind, however, that there is no one way to create a relationship. People appear to learn from many different teaching methods, as Bruner (1977) said, there are many diverse routes by which students get to deeper understanding. Evaluation is the systematic collection of evidence to determine if changes have occurred in the learners, as well as to find out the amount of change in each learner. Professional growth of the teacher requires that teachers secure evaluation evidence and other facts so that they can continue to improve their teaching and the students' learning.
Background of the University of Toronto project In the Autumn of 1974, the faculty of nursing set up a subcommittee on teachinglearning to study the university's newly promulgated criteria of teaching and to suggest methods by which the teaching-learning evaluative process could be implemented. This author was chairman of the subcommittee and having been named faculty fellow under a university programme focusing on the improvement of teaching and education development, decided to apply the resultant funds to a project in peer evaluation to be undertaken by the committee. The project would approach the evaluation of teaching-leaming by attempting to develop and test peer evaluation criteria and forms for use by members of the nursing faculty. The emphasis on peer evaluation or review is comparatively recent; hence, it seemed worthwhile to share the way our faculty went about it in our educational setting. The procedure included the development of a model. Models as prototypes of theories make possible early conceptualization and the study of phenomena, and useful models can lead to supportive theories. R E V I E W OF THE L I T E R A T U R E Only a small percentage of educational studies have considered teacher behaviour systematically. Many studies did not include observations of teachers and therefore
Discussion of a project on peer evaluation
459
did not provide information on teacher behaviours and student outcomes, or teacher behaviour in the classroom setting. In an extensive review of the literature relevant to evaluation effectiveness, the main points may be summarized under three aspects of the problem: (i) the teaching-learning process, (2) the evaluation process and (3) the evaluating instruments.
The teaching-learning process ' Readiness' refers to the general nature of the responses the learner has available for solving problems. The mental, physical and emotional state of the learner has much to do with the efficacy of any given learning experience. Cultural, motivational and personal factors affect the desire to learn and to undertake problemsolving and the general principle is that without appropriate readiness a learning experience will be inefficient, or learning will not occur. The principle stresses learning as a sequential process in which past learning serves as a basis for present and future learning. General mental ability and previous academic accomplishments have a significant influence on the learner's ability to acquire new material. Teaching is a complex, subtle and varying activity which may be more dependent on personality, attitudes and human relationships than on method. This interpersonal manner is the message, in McLuhan's (1967) language. Sheffield (1974) supported this premise when he said that attitudes toward students and teaching are more important than methods or technique. Of categories of criteria identified by various authors, those most frequently identified have been (i) attitude toward students, (2) presentation, (3) personal attributes, (4) class atmosphere, (5) course mechanisms and (6) attitude toward subject. Biddle (1964) said that although approximately 10 000 studies had been reported on the relationship between the characteristics of teachers, teacher behaviour and educational goals, many facts had not been identified concerning teacher effectiveness and neither had any approved way of measuring competencies been accepted, or any method of promoting teacher adequately been widely adopted. In the late 1950s, Bradford (1959) expressed his views on the teaching-learning transaction. He said that teaching is not a matter of filling a void with information but involves internal reorganization of a complex of thought patterns, perceptions, assumptions, attitudes, feelings and skills, and testing this organization out in relation to problems of living. Rogers (1959) discussed conditions which facilitated learning and stressed that one of the most important of the conditions was attitudinal quality of the interpersonal relationship between facilitator and the learner. Theobald (1976) stressed the need for effective action to create transitions into the communication era, which requires that we once again become sensitive to each other. 'We need', he said, 'to develop interpersonal understanding at levels far higher than those that now exist.'
460
L. Turner
Sheffield (1974) in Canada, and Buxton & Prichard (1975) in the USA, compiled books along similar lines in which they invited outstanding teachers to speak for themselves. Sheffield asked graduates to name professors they had known as excellent teachers and to say what they felt had made these professors and their teaching effective. The contributors to both books comprised a very diverse group of college teachers and a variety of teaching styles. The lecture was still the dominant mode of instruction, but communication and interaction with students was an essential part of the professor's style. Among the very important perceptions that the professors had were that the teacher's most important function is that of stimulating students to be active learners on their own; that effective teachers think students are important, like them, respect them, and care; that effective teachers have both a love and open enthusiasm for what they teach; and that they work hard at teaching. The students who selected teachers on their excellent performance were in agreement on their performance in five areas—they showed mastery of the subject and competence; their lectures were well-prepared and orderly; they related their subject to life, and were practical; they encouraged students' questions and opinions; and they showed enthusiasm about the subject. Ashby (1975) stated that the education industry provided excellent vocational training in a great range of skills, as well as fairly good initiation into the professions; that is, the ability to deal with things and the ability to deal with ideas. 'But', he said, 'neither school nor college or university is equipped to teach us how to deal with people, yet that today is the greatest single need among citizens of a democracy. This is the dilemma.' The need for continuing assessment by various sources, both immediate and longterm, of the enduring effect of what teachers actually do in their classrooms has been neglected and the need for studies was stressed. One of Sheffield's conclusions was that however much individual professors differ in their style, their personalities, and their methods, the resulting general patterns of effective teaching are essentially similar in all settings (the last remark indicated to the author that criteria could be developed around this pattern of similarity).
The evaluation process In the 1970s, renewed interest was generated in evaluating teacher performance. Krasno (1972) stressed the need and importance of research in teaching effectiveness based on longterm consequences. Evaluation is a cooperative process wherein the evaluator and the person being evaluated feel a mutual responsbility to centre upon performances requiring improvement as well as those showing strengths, to attempt working together to reach the best results, and to evaluate the results. Sheffield (1974) pointed out that professors, like students, learn only when they are active participants in the process, and only when they want to—imposed programmes of teacher improvement are unlikely to work, but professors themselves may initiate them if they feel it is worthwhile.
Discussion of a project on peer evaluation
461
Dornbush & Scott (1975) showed that teacher satisfaction with the evaluation is vital to the process and that knowledge of criteria, frequency of communication, evaluation, and influence on the evaluation process hy those involved, contribute greatly to such satisfaction. The relevance of beliefs and values to each perceiver and his frame of reference was highlighted. As Pearce (1973) said, 'We are the sovirce of values and choices, the source of ideas which the procedure of our reality orients', but 'we are shaped by each other. We adjust not to the reality of the world, but to the reality of other thinkers.'
Evaluation instruments Wasserman & Eggert (1976) suggested the evaluation instrument include the school's educational values, teacher behaviours which are truly prized, behaviours related to the pupils' learning, and provision for continued self-scrutiny and estimate of professional growth. Centra (1975) reported that to improve reliability of peer evaluation, faculty members needed to invest more time in visitation or training sessions. Work is visible to the extent that those evaluating the performance are able to observe relevant aspects of the task and outcomes produced by those performing. Attention was directed to having the rating scales large enough to allow for sensitivity in testing.
THE C O N C E P T OF E V A L U A T I O N
Definition of terms Evaluation is a systematic process of judging the worth, desirability and effectiveness, or adequacy, of something according to defmite criteria and purpose. The judgement is based on careful comparison of observation data with criteria as guidelines. Peer evaluation in teaching is defined as a process engaging two persons, each considered to have the necessary abilities, education and qualifications to critically assess the teaching effectiveness of the other. In some instances there is a mutual assessment of teaching effectiveness. Observing involves the intentional and the theoretical viewing of some object or activity. Observing is more than seeing and it involves planned, careful, focused, active attention by the observer. Measuring is the assigning of quantitative values to a group of people and objects, or activities, according to some established rules. It is a descriptive activity. Formal organization applies to those distinctive aspects of a project which have been consciously planned (e.g. workshops). Informal organization is conceived as being aspects that are not formally planned but that more or less spontaneously evolve from the needs of people (e.g. experience sharing).
462
L. Turner
Limitations 1 Most teachers dislike having to judge their peers. Negative effects in interpersonal relations are feared. 2 The data may represent what the evaluators think they should perceive and say about themselves and about their peers. 3 Because of the lack of visibility of teachers' work with students, peer evaluation of work in the clinical setting, in the small-group setting and in the one-to-one situation, may be viewed as less soundly based than peer evaluation in team situations and in lectures. Assiunptions 1 Peers can make a contribution to the evaluation of faculty performance. 2 Teachers will take evaluation more seriously when they are responsible for delineating their own learning needs and receiving assistance. 3 Neither the use of student, self, nor peer ratings, leads automatically to improvement in faculty development. Ratings themselves provide information only and not necessarily the desire to change. 4 Evaluations are more effective when planned cooperatively by teachers and administrators. 5 Peer evaluation takes time, work and patience to develop a working model. 6 Peer evaluation cannot be forced on any teacher. As a concept, peer evaluation takes time (relative to each individual) to internalize the concept. If the teacher does not view it as her responsibility and right, it cannot be effective. DESIGN OF T H E P R O J E C T The design of the project was flexible, building on approaches to the problem that had already been made by the committee and incorporating a nvimber of aspects that evolved as the project advanced. It may, however, be seen as having two main phases—development of evaluation criteria, forms and of a model of the evaluation process, and explaining and applying the evaluation process. It must be kept in mind that these phases overlapped considerably and faculty involvement was encouraged throughout. In the research of literature the finding of greatest importance to this and future projects, and research relating to the evaluation of teaching effectiveness was that there is a general pattern of effective teaching similar in all settings. Approach to the problem Preliminary procedures. Preliminary steps in the evaluation of the teachinglearning process had already been taken prior to this project. The committee on teaching-learning effectiveness had been examining the criteria set out by the university with a view to clarifying them for the faculty of nursing setting. A student-teacher rating form had just been developed and was being put into practice. The broad criteria for quality teaching defmed by the university were:
Discussion of a project on peer evaluation
463
1 The degree to which the teacher has demonstrated mastery of his or her subject area. 2 The degree to which the teacher has the ability to communicate well. 3 The degree to which the teacher is able to stimulate and challenge the intellectual capacity of the student. For the project, criteria i and 2 remained unaltered, criterion 3 became the degree to which the teacher can stimulate and challenge the cognitive and affective capacities of the student. In a professional faculty, the clinical expertise of the teacher had to be considered. Therefore, a fourth category was added to supplement the university criteria, namely, the degree to which the teacher demonstrates integrity as a professional person. Consideration was given to the working model of the student-teacher evaluation guidelines. One of the committee's objectives was to have the peer evaluation form parallel other forms used in evaluation. The one source of evaluation, whether student, peer or self, was not to be taken by itself as adequate; in fact each faculty member would have several sources from which to collect data, in keeping her own profile, and each source would contribute to the review process.
Evaluators The project had as its prime goals self-development by the staff, hence each individual teacher was to be a key participant. Both present and former students had an obligation to be involved as it was their learning experience. Colleagues' feedback was to be another invaluable source of input, as ongoing sharing for enhancing strengths and improving weaknesses was an aim of the process. Evaluation by administrators was also to be encouraged as they assume the fmal responsibility for the quality of teaching within the university setting.
Explaining and applying the evaluation process It was decided to hold a series of workshops for faculty members to promote understanding and acceptance of the evaluation process and to test the evaluation form. Faculty members would be asked to do evaluations, using a form incorporating any revisions found necessary. Among the aspects that evolved were conferences to be held before and after the evaluation observation periods and assistance to faculty members in making a plan for improvement.
Developing a model The model of the evaluation process developed in the process of approaching the problem is given in Figure i. It incorporates some features that developed themselves as the project advanced, as, for example, the pre-observation conference.
464
L. Turner Assigning goals
Criteria development
Conference mutual exchange
Performance observing
Outcome appraising
Conference mutual exchange
Planning for improvement
FIGURE I
Model of the evaluation process
The model of the evaluation process shows the elements in developing and applying the process. Once the steps of assigning goals and developing criteria are accomplished and expressed in guidelines, the evaluator and evaluatee meet in conferences (mutual exchange step three). They discuss the use of the guidelines (questionnaire) and how to arrange observation of performance to their mutual satisfaction. After the steps of observation of performance (whether in a lecture or clinical setting), the evaluator appraises the outcome as does the evaluatee. They then confer about the outcome and plan for improvement. The process then recommences. METHOD
Development of an evaluation form An evaluation form was developed, applicable to all faculty involved in the teaching-learning situation. All categories of criteria might not have the same
Discussion of a project on peer evaluation
465
importance for any two persons because of differences in their teaching responsibilities. The categories of criteria on the form were the same as, or as close as possible to, the wording and intent of the student-teacher, former student and administrative forms. It was hoped that this parallelism would decrease confusion and increase utilization by employing widely understood criteria. The exact grading scale was to coincide with that of the student-teacher evaluation form. Faculty members were involved in developing criteria by faculty meeting used as a vehicle for keeping faculty informed, consultation and resource materials through the faculty fellow, and workshops—three i day workshops held in February, March and April 1975 (referred to as workshop i) and two i day workshops held in March and April of 1976 (referred to as workshop 2).
The criteria The draft of criteria guidelines developed for use in the workshops consisted of six categories: 1 Mastery of subject area. 2 Application of knowledge and skills to patient care. 3 Preparation for the learning experience. 4 The learning situation. 5 Evaluation of the learning situation. 6 Student development. Within each category several areas were expanded and more clearly defined than they had been in the student-teacher form.
Workshop i; increasing faculty's understanding of teacher evaluation Workshop i was designed to assess and test the guidelines for evaluation of teaching-learning. The discussion revealed the threat in evaluation. Many questions were raised as to exactly how the result would be used and how valid the peer evaluation would be. The categories were discussed, with clarification in some areas and suggestions for rewording or change in other sections. The recommendations at the end of workshop i were to make the categories into questionnaire format and attempt to use the tool as a basis for self and peer evaluation.
Testing of the questionnaire; subsequent change and retrial Following workshop i, the guidelines were put into questionnaire form, with a 3-point grading scale for guidelines of mastery of subject and a 5-point grading scale for the remaining areas. The questionnaires were distributed to the faculty for a trial testing. The trial pointed out several problems and many questions, but it seemed to lessen anxiety about the process. The 3-point grading scale in category i (mastery) provided too narrow a range. The scale was changed for the next trial to a 5-point scale. The wording in some areas was too vague and so further revisions of those areas were made. There were several comments about the lack of criteria for
466
.
L. Turner
professional integrity, and this new area was added. The revised form is the format that has been retained for use. Faculty were asked to fill out a self-evaluation and have two peer evaluations done. New faculty had been introduced to the evaluation of teaching-learning and they were asked to participate. There were fewer comments about the form itself and the revision seemed to have corrected the initial problems. At this time, another grant was obtained by the faculty fellow and the decision was made to hold another series of workshops for the faculty. From discussion, the committee felt that though self-evaluation was less anxiety-provoking than previously, peer evaluation still carried some threat. This theme was to become the focus of the next workshop. Workshop 2; peer evaluation—openness and trust Two workshops were held and involved 24 faculty members a year following workshop i. It had been decided to work with new faculty mainly, though not entirely. The discussions revealed that evaluation still aroused anxiety and an attempt was made to encourage individuals to recognize evaluation as an ongoing learning experience. The discussion also pointed out that new staff members are ill-prepared in this area because of the limited time spent on it during their orientation and the low priority which their evaluation had in the initial weeks of employment. Recommendations arising from this workshop were valuable and some are incorporated into the committee's final report for the faculty. FINDINGS AND DISCUSSION Development of goals and selection of evaluation criteria and forms Teachers and administrators were successfully involved in the process of identifying criteria to be used in evaluating teacher effectiveness. The workshops were invaluable in helping to clarify statements and lessen the threat around evaluations. This procedure tends to delay coming to grips with the practical tasks of evaluation, but it proved to be a necessary process for cognitive and affective growth and for creating a readiness to deal with the inherent threat in evaluation. The findings of Bess (1977) are relevant. In his examination of the relationship between important needs of faculty and the process and context of teaching in institutions of higher learning, he said, 'If teaching is to be externally rewarded, it must also be internally rewarding'. If evaluation can be seen as a growth process with active participation of both persons, evaluator and evaluatee, and with effective communication and interaction, then evaluation can become a more positive process with less frustration for each person. The groups brought together formally provided a forum for sharing of thoughts and feelings. The proceedings of each workshop were formally presented to the total faculty. Informal aspects evolved spontaneously and much sharing occurred when several faculty were together. This latter aspect greatly enhanced the evaluation process.
Discussion of a project on peer evaluation
46J
Teacher evaluation and student evaluation As the teacher questionnaire contained terms parallel to the student form, it allowed teachers to identify similarities and differences in their perceptions and the students'. Both were valuable input, which were taken into consideration when teachers were assessing their strengths and weaknesses and making plans for improvement. Choosing a peer Each faculty member was free to choose two peers, or more if desired, to make her evaluations. She was also free to select them from her own or different areas of interest. Peers were often chosen on the basis of friendship or proximity. As respect and mutual trust were emphasized throughout the workshops, liking someone and feeling free to be open with them was also a criterion for some teachers' choice of partners. Mutual respect was a critical factor as a basis for successful peer relationship. Explaining and applying the evaluation process No specific time was set during the project for observation. In fact, working out a plan for observation proved to be difficult. As most of our faculty work in teams, the formal lecture did not present as many concerns as the small groups, the one-to-one interaction used in teaching-learning, and especially the work in the clinical areas. Selecting criteria appropriate for classroom as well as for small groups provided some measure of success. Colleagues suggested focusing observation by using specific criteria and different colleagues could also be used in classrooms observation, small-group observation and one-to-one interaction. Except for teachers in the team situation, the lack of visibility of the teacher's work to persons other than their students created problems in evaluation. Under these circumstances there was some hesitancy by teachers to trust the evaluation system. Conferences on appraising performances Conferences were held at some time before and after the observation period. Each teacher shared with her chosen peer her self-evaluation, using the same criteria as those used for peer evaluation. Participants were usually harder on themselves than their peers were. Students were sometimes closer to the participants' own evaluation. Colleagues' evaluation provided positive reinforcement as well as constructive criticism. Suggestions for improvement were usually balanced with praise for effective teaching. Praise by one's own peers seemed to fill a great need so that the conferences proved to be a positive factor in colleague evaluation. Suggestions were made that a pre-observation conference be held so that expectations regarding observations could be exchanged. The conferences also provided a process for discussing teacher-student interactions, a matter of great concern to the faculty.
468
L. Turner
The Improvement Plan Developing a plan for improvement is one of the vital steps in the colleague evaluation process, whereby each teacher integrates all information she has received from self-evaluation, student evaluations and peer evaluations. Strengths and weaknesses can be listed and ways defmed to improve teaching effectiveness. Minor changes were dealt with within the present system, but for some competencies and skills the organization of a resource centre seemed indicated.
PRINCIPLES AND CONCLUSIONS D R A W N F R O M THE PROJECT 1 Involve all faculty in the development of evaluation criteria. Involvement serves two purposes—the position is defmed more accurately and the morale of the faculty is enhanced. 2 Faculty involved in the evaluation process need to understand the purpose, the procedure to be used and the roles of the people involved. Faculty need to be kept informed and they need to talk and share feelings in groups. 3 Effective evaluation of teachers is dependent upon the quality of communication between teachers and administrators. 4 The guide or form for evaluation should be used on a trial basis, so that faculty become acquainted with its use. 5 Allow faculty time to work through feelings regarding evaluation. 6 Emphasize the sharing aspect between peers, observer and observee, both before and after observation. The sharing can be very rewarding and helpful for both peers. 7 Keep evaluation of teacher performance teaching-learning centred, cooperatively designed and implemented. 8 Maintain faculty accountability by having each faculty member take responsibility in keeping an up-to-date profile. Recommendations for faculty 1 The Self and Peer Teaching-Learning Evaluation Form, as developed, be retained for use for the next 2 years. In April 1978, a committee be appointed to reevaluate the effectiveness of this form. 2 Each faculty member take the responsibility to do her own self-evaluation yearly. 3 Each faculty member take the responsbility to have two peer evaluations done yearly. 4 Each faculty member take the responsbility to keep self and peer evaluations in her own profile and provide this data to administration if desired. 5 Administration take the responsibility to introduce each new faculty member to peer and self-evaluation during orientation, with a follow-up session within 3 months to further explain the criteria and process.
Discussion of a project on peer evaluation
469
Recommendations for further study The work in the teaching-learning evaluative process had indicated that the reliahility and validity of the questionnaire should be tested further before placing weight on its results. It would also be interesting to determine if there is any relevancy in comparing the peer evaluations and student evaluations of the individual teacher. Study in these areas would provide administration and the teacher with a better understanding of what these results really mean. The next and perhaps most important step is to ensure that there are resources available within the faculty or the university where any individual teacher can receive counselling or instruction to maximize her strengths and improve her weaknesses. The utilization of such resources would be the teacher's responsibility. CONCLUSION It has been proposed that teachers have expertise that they can share with each other and that they can identify areas requiring improvement in the evaluation of their peers in teaching and learning. Evaluation posed a threat, which decreased as faculty became involved in the process, for, as Eiben and Milliren (1976) pointed out, creative and self-reliant learners develop best in an atmosphere of responsible freedom wherein self-evaluation and self-criticism are fostered. Evaluation is a fundamental process necessary to effective teaching-learning situations, but teachers need to feel involved and have power to affect the evaluation system. Reilly (1959) said that there is only one thing which really trains the human mind and that is the voluntary use of the mind by man himself. You may aid him, you may guide him, you may suggest to him and above all, you may inspire him, but the only thing worth having is that which he gets by his own exertion and what he gets is proportionate to the efforts he puts into it.
References AsHBY E. (1975) Dilemmas of Modern Man. Great-West Life Assurance Company, Manitoba, Canada. BiDDLE B. (1964) The integration of teacher effectiveness. In Contemporary Research on Teacher Effectiveness, eds. Biddle B. & Ellens W., Holt, Rinehart & Winston, New York. BESS J. (1977) The motivation to teach. The Journal of Higher Education 48, 243-258. BRADFORD L . (1959) The teaching-learning transaction. Adult Leadership 8, 7. BRUNER J. (1977) The Process of Education. Harvard University Press, Massachusetts. BuxTON T. & PRICHARD K . (1975) Excellence in University Teachers. Columbia University Press, New York. CENTRA J. (1975) Colleagues as raters of classroom instruction. Journal of Higher Education, 46, 327-337DORNBUCH S. & SCOTT R . (1975) Evaluation and The Exercise of Authority.Jossey-Bass, San Francisco. EIBEN R . & MILLIREN A. (1976) Educational Change: A Humanistic Approach. University Associates, California. GAGE N . L . (1972) Teacher Effectiveness and Teacher Education: The Search for a Scientific Basis. Pacific Books, California.
470
L. Turner
KRASNO R . (1972) Accountability and research on teacher effectiveness. Administrator's Notebook 21, 1-4. McLuHAN M. (1967) The Medium is the Message. Bantam Books, New York. PEARCE J. (1973) The Crack in the Cosmic Egg. Pocket Books, New York. REILLY W . (1959) Successful Human Relations. Harper & Row, New York. ROGERS C . (1969) Freedom to Learn. Charles E. Merrill, Columbus, Ohio. SHEFFIELD E. (1974) Teaching in the University: No One Way. McGill-Queen's University Press, Montreal. THEOBALD R . (1976) Beyond Despair: Direction for American's Third Century. The New Republic Book Company, Washington D.C. WASSERMANN S. & EGGERT W . (1976) Profiles of teaching competency: a way of looking at classroom teaching performance. Canadian Journal of Education I, 67-75. APPENDIX UNIVERSITY OF TORONTO Faculty of Nursing September 13,1976 To: All Faculty Re: Self and Peer Evaluation Each faculty member has the responsibility of keeping an up to date profile of her over-all teaching-learning activities. You have three sources available within the faculty from which you can compile information, peer, student and self-evaluation. This information will be requested as required for administrative decisions and for your own academic growth. Lettie Turner Peer Evaluation (Chairman) LT/dp
U N I V E R S I T Y OF T O R O N T O FACULTY OF N U R S I N G Self and Peer Evaluation Form I The degree to which the teacher demonstrates mastery of his/her subject area. Category i: Mastery a Has a theoretical base in nursing. b Has knowledge of the content of core in the curriculum. c. Has knowledge of the content and skill (i) in speciality area (ii) in area of function (teaching) d Integrates principles from own and other disciplines in teaching. e Integrates relevant research findings and recent developments in teaching and practice.
Very Good Good Acceptable
Poor Comment
Discussion of a project on peer evaluation
471
f(i) Maintains and increases knowledge; (ii) maintains and increases nursing practice. h Influences the student through guidance, direction and/or example to maintain and improve the quality of patient care.
2 The degree to which the teacher has the ability to communicate well.
Category 2: Preparation for the Learning Situation a Prepares individually and /or as team members for (i) classes (ii) seminars (iii) labs (iv) advising b Clarifies learning goals which facilitate student learning in achieving course expectations. c Recognizes the variables affecting the learning situation. d Selects appropriate learning experiences for students; and helps students select appropriate learning experience.
Category 3: The Learning Situation a Explores with the student her goals in relation to the course goals. b Presents ideas with an understanding and awareness of the students' readiness. c Develops ideas in thoughtprovoking and stimvilating ways.
Consistently Usually Sometimes Seldom Rarely Comment
472
L. Turner
Category 4: Evaluation of the Learning Situation a
Evaluates on the basis of the learning goals.
b Uses criteria for test and assignment evaluation which can be defined to students. c
Uses methods of evaluation appropriate to the learning situation.
d Provides feedback.
constructive
3 The degree to which teacher can stimulate challenge the cognitive affective capacities of the rlpnt5 dents.
the and and stu-
Category 5: Student Development a
Encourages critical and independent thinking and action.
b Encourages student's expression of ideas, attitudes and feelings. c Encourages students' participation in developing learning experiences. d Encourages evaluation by students. e Demonstrates concern for the student as a person. f
Encourages the students' growth as a professional person.
4 The degree to which the teacher demonstrates integrity as a professional person.
Consistently Usually Sometimes Seldom Rarely Comment
Discussion of a project on peer evaluation Categoty 6: Integrity a
Demonstrates concern for the patient, his family and community.
b Communicates respect for students and colleagues as individuals. c Meets commitment to students and colleagues. d Evaluates and attempts to improve effectiveness as a teacher. e
Assumes responsibility toward her profession including continuation of personal and professional development.
473
Consistently Usually Sometimes Seldom Rarely Comment
