J. ROBERT KOEWING, MDiv, STM, BARRY R. HOWES, MS, and DAVID METZ, MPA
Four characteristics of a configurational strategy for planning regional continuing education in medical allied health are identified in this article. These characteristics are restated as criteria for six design factors in the planning and evaluation of a regional continuing education program. A perceptual motor dysfunction workshop was planned and evaluated utilizing the configurational strategy.
One goal of the University of North Carolina Area Health Education Centers (AHEC) Program is the development of nine regional centers which have the capacity to respond to commonly identified health manpower educational needs and to coordi nate the utilization of existing local and statewide resources in such responses. Preliminary evaluation of the early experiences of the regional North Carolina AHEC allied health coordinators has sug gested four characteristics essential to the develop ment of regional continuing education activities for medical allied health programs. 1 These characteris tics—1) multidisciplinary focus, 2) coordinated use of available resources, 3) interdisciplinary experi ences, and 4) cost effectiveness —constitute criteria which may be used to guide the selection of an appropriate strategy for regional continuing educa tion and to evaluate its effectiveness. We have chosen to call programs which exhibit these charac teristics "configurational strategies" to emphasize the circumstances they are designed to meet. ProMr. Koewing was Assistant Professor, Department of Medical Allied Health Professions and Associate Director of the University of North Carolina Area Health Education Center Program, Chapel Hill, when this study was conducted. He is currently President, J. Robert Koewing Associates Inc, PO Box 95127, Schaumberg, IL 60195. Mr. Howes is Assistant Professor, Department of Medical Allied Health Professions and Director, Allied Health, University of North Carolina Area Health Education Center Program, Chapel Hill, NC 27514. Mr. Metz is Assistant Professor of Hospital Administration and Associate Director of the University of North Carolina Area Health Education Center Program, Chapel Hill, NC 27514. This study was supported in part by the US Department of Health, Education, and Welfare and the University of North Carolina Area Health Education Center Program in 1976. This paper was presented at the 52nd Annual Meeting of the American Physical Therapy Association, June 30, 1976, in New Orleans. 168
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
Six Design Factors for Planning and Evaluating Regional Continuing Education
grams which do not exhibit these characteristics we have chosen to call "traditional strategies." DESIGN FACTORS OF A CONFIGURATIONAL STRATEGY COMPARED WITH TRADITIONAL STRATEGY
Regardless of whether the overall strategy for a particular program is "configurational" or "tradi tional," all programs have a basic design. Character istics essential for effective regional continuing edu cation must be exhibited within the program design. Though a program design comprises many factors, six are sufficient to illustrate the difference between "traditional" and "configurational" strategies. Be cause of the widespread attention given to such design factors as needs assessment, behaviorally stated objectives, competent leadership, perfor mance based evaluation, and others, these will not be discussed here. The factors selected for this analysis are less often systematically evaluated, al though their influence upon the effectiveness of a design for regional continuing education programs cannot be overlooked. These design factors are 1) focus, 2) style, 3) emphasis, 4) outcomes, 5) effect, and 6) resources. Table 1 provides a definition of each factor and a comparison of the characteristics of "traditional" and "configurational" strategies for regional continuing education. Specific disadvantages of the traditional strategy became apparent in the comparison of the charac teristics as shown in Table 1. The major disadvan tages are as follows: 1. Traditional strategies ignore the multidiscipli nary professional experience of the participants by focusing on subject matter. PHYSICAL THERAPY
relates these objectives and their criteria to the design factor to which they correspond.
CONFIGURATIONAL PROGRAM DEVELOPMENT: AN EXAMPLE One of the nine regional Area Health Education Centers in North Carolina recently developed a workshop for professionals who work with children with perceptual motor dysfunction. Because of the regional visibility of the Charlotte AHEC and its allied health coordinator, a local professional orga nization of physical therapists sought AHEC assist ance in order to respond to a specific need they identified among health and educational profession als. The AHEC allied health coordinator responded to the request by initiating a series of planning sessions during which she assisted the physical ther apists to broaden their scope by incorporating nu merous allied health and other disciplines directly or indirectly involved with such children. Her direct link with university faculty resources soon yielded information that a similar workshop requiring the identical faculty resources was in the planning stage. In addition it was learned that other regional orga-
TABLE 1 A Comparison of the Characteristics of Configurational and Traditional Strategies for Continuing Education in Medical Allied Health Definition
Configurational Strategies
Traditional Strategies
1. Focus
. . . the person for whom edu cational experiences are in tended
Multidisciplinary
Single discipline
2. Style
. . . the experience which is intended for persons who par ticipate
Integrate disciplines in com mon educational experiences
Differentiate among disciplines for educational experiences
3. Emphasis
. . . interdisciplinary educa tional experiences addressed to common needs of participants undifferentiated by specialty or discipline
Facilitate interdisciplinary communication by emphasiz ing subject matter common to several disciplines
Facilitate disciplinary specializa tion by emphasizing subject mat ter unique to (or tailored for) a particular discipline
4. Outcomes
. . . experiences in which par ticipants achieve clarity con cerning the role and function of particular disciplines through interdisciplinary reinforcement
Promote interdisciplinary clar ity and mutual (interprofes sional) reinforcement with re spect to role and function
Promote disciplinary clarity (of ten isolation) and stimulate legal and quasi-legal (external) rein forcement for role and function
5. Effect
. . . the individual's profes sional experience
Facilitate problem solving in terms of the configuration of disciplinary resources appro priate to resolve them (Facili tate team management)
Encourage problem solving in terms of the contribution of a particular discipline
6. Resources
. . . facilitate the experience intended for participants
Promote resource sharing
Promote specialized use of re sources
Volume 58 / Number 2, February 1978
169
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
2. Discipline-specific subject matter discourages in tegration through team approaches to patient care. 3. Emphasis on "uniqueness" and "specialization" results in isolation of the allied health profes sional. 4. Focus on competency maintenance within one discipline does not enable the individual to achieve interdisciplinary reinforcement and rec ognition. 5. Problem solving by discipline ignores the context within which professionals experience the need for continuing education which is the multiprofessional complex of patient care. 6. Focus on the means, rather than on the ends, of continuing education results in duplication of resources and correspondingly decreases cost effectiveness. To correct for these disadvantages and to incor porate the characteristics we have identified as essential for effective regional continuing education in medical allied health into the design of educa tional experiences, we have developed a statement of six configurational program objectives and have identified two evaluation criteria for each. Table 2
Program Objective
Criteria
1. Focus
Program is designed with a multidisciplinary focus
1. The program is relevant to persons of several disci plines 2. The experience provided is consistent with the multidisciplinary health care environment
2. Style
Program is designed to integrate disci plines in common educational experi ences
1. The experiences are relevant to the individual's professional experience. 2. Participants gain experience in the integration of competencies and skills from other disciplines with those of their own
3. Emphasis
Program is designed to facilitate inter disciplinary communication by empha sizing subject matter common to several disciplines
1. Subject matter is defined by common needs 2. Disciplinary contributions to the multidisciplinary health care environment are articulated within an interdisciplinary framework
4. Outcomes
Program is designed to promote clarity and mutual reinforcement for role and function
1. Role and function reinforcement occurs through colleagues of other disciplines 2. Differentiation of role and function is achieved on the basis of interdisciplinary clarity
5. Effect
Program is designed to facilitate prob lem solving in terms of the configura tion of disciplinary resources appropri ate to resolve them
1. Problem solving is encouraged which utilizes appro priate interdisciplinary resources 2. Direct application of resources to participants expe rience is intended
6. Resources
Program is designed to promote re source sharing
1. Resources are not confused with program objectives 2. Resources are directed towards goals and objectives
Design Factor
nizations had initiated requests for similar technical assistance dealing with children who have percep tual motor problems. Part of the allied health coordinator's role is to relate formally and informally to regional educa tional and health service delivery institutions as well as to professional organizations. The resulting linkages facilitated the inclusion of these institutions and organizations in a cooperative planning process thus assuring their multidisciplinary input into its design and content. These linkages yielded measurable outcomes even in the planning stage. One outcome of these re gional multidisciplinary linkages was illustrated in the participation of a local school system which cosponsored the workshop and released 100 physi cal and special education and classroom teachers for all or part of the workshop. A second outcome was illustrated by the participation of professionals sponsored by a state agency which had itself con tacted the same university faculty for identical tech nical assistance. A third outcome of the multidisci plinary linkages of the allied health coordinator was the regional and statewide faculty and instructional resources which were shared for the planning and 170
implementation of this educational experience. State support was provided by the University of North Carolina School of Medicine's Department of Medical Allied Health Professions as a regular part of its contract commitment to the University's AHEC Program. These resources were utilized to back up the regional professional, clinical, and community college faculty. Criteria for configurational continuing education programs were therefore evident in the develop ment of this Charlotte AHEC regional experience for professionals working with children with percep tual motor dysfunction. The program was multidis ciplinary in its focus. It was planned to coordinate available regional and state resources. It exhibited the cost effective characteristics essential to regional programming for allied health by pooling faculty and participants and planning efforts. Finally the workshop was designed as an interdisciplinary ex perience to facilitate interprofessional communica tion and exchange. As a result of the application of these characteristics in the planning, over 190 par ticipants representing 15 disciplines with the com bined support of significantly affected local agencies registered for, and participated in, portions of a
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 2 Six Design Factors for Regional Continuing Education Stated as Configurational Program Objectives with Evaluation Criteria
OBJECTIVES-REFERENCED EVALUATION OF REGIONAL CONFIGURATIONAL CONTINUING EDUCATION: AN EXAMPLE An elusive question to continuing educators is, "What difference has this program made?" As with any educational experience, the goals and objectives for any continuing education programs must be stated in measurable terms. Numerous educators and administrators continue to debate the effective ness of continuing education programs in achieving behavior changes in adult participants and "how" to evaluate it. 2-9 Unlike academic curricula, a con tinuing education program does not take place over a sustained period of time within a reasonably con trolled environment. Equal learning potential among participants cannot be assumed because of wide differences in training, education, and experi ence. Continuing education participants are seldom screened for selection by a battery of tests typical of undergraduate and graduate academic programs even though these data are helpful in documenting baseline behaviors and isolating critical predictive variables which result in selection of homogeneous groups. There are those persons who doubt the validity of continuing education because such base line data and the research such data may propagate do not exist; it is as if these persons are willing to believe that no education is effective if it cannot be measured with their tools. On the other hand, there are those persons who do not believe continuing education should be measured —at least in the be haviors of those who participate; 10 it is as if these people are willing to say any continuing education is better than none, and the need to measure results may prevent people from participating. A helpful middle ground is found in objectivesreferenced evaluation. This form of evaluation is particularly useful where research is neither needed nor intended, and where the evaluation questions are related to the program and not the participants. The prerequisite for such evaluation is a set of program and instructional objectives which specify the intended outcomes in such a way that data may be elicited from program participants in both formal or informal ways. The assumption here is that the definition of an effective program is one which results in changes in participant behavior. The infer ence is that when such changes happen, the program
Volume 58 / Number 2, February 1978
was causal to the behavior change. The definition of an effective program, in this instance, is limited to the objectives the program was designed to accomplish. In no way can a generic statement of effectiveness be made without reference to the particular objectives intended for specific programs. The evaluation employed with the workshop on perceptual motor dysfunction was objectives-refer enced. The questions to which the evaluation was addressed were as follows: 1. Was the workshop successful in enabling the participants to demonstrate achievement of the cognitive objectives? 2. What attitudes did the workshop elicit in the participants in relation to the configurational design factors? A cognitive pretest and posttest were administered to elicit participant data to answer the first question. An attitudinal questionnaire was constructed to elicit participant feedback for the second. An eval uation of participant behavior on the job was not feasible to include in the evaluation design because of circumstances and composition. Control groups were unnecessary because the subject of the evalu ation was the program rather than the participant behaviors.
COGNITIVE EVALUATION The goals and objectives for the workshop as presented to the participants were To lay the groundwork for intervention in perceptual motor dysfunction through the utilization and coordi nation of community resources. In order to accom plish this, we hope to educate specialists in education (75 teachers of kindergarten and grades one through three and 75 elementary school physical educators), psychology, communication, physical therapy, recre ation therapy, occupational therapy, social service, and pediatrics to the latest information on multidisciplinary intervention in perceptual motor dysfunction by: 1. Defining Perceptual Motor Development —abnor mal versus normal. 2. Identifying specific evaluation tools. 3. Identifying selected management skills. 4. Identifying local community resources.
Ultimately the effectiveness of the workshop could be measured by the number of "interventions" in perceptual motor problems which utilized and co ordinated community resources. Obviously the identification of one workshop as the significant factor in achieving such interventions is nearly im possible. Therefore, a posttest was developed to 171
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
week-long continuing educational experience. Of these, 82 (43%) participated for the entire week and 108 (57%) participated in segments of the program.
Pretest Posttest
Number*
Mean
Variance
Standard De viation
Reliability KR-20
Standard Error of Measure ment
146 130
12.66 16.70
6.16 3.12
2.48 1.76
.42 •32
1.87 1.45
* 146 represents the attendance at the time the pretest was given. 130 represents the attendance at the time the posttest was given.
enable participants to demonstrate mastery of the particular cognitive objectives. A cognitive test was developed by asking each of the major presenters to submit one or two multiple choice questions which covered key or essential subject matter. The goal was to create a mini-test which contained the 20 most significant cognitive items of the workshop. The questions submitted were reviewed by other members of the technical staff and faculty for form and content. The test was limited to 20 items to help ensure completion by the participants and to facilitate quick turn-around hand scoring on the final day of the workshop so that participants could receive immediate feedback on the raw data showing their pretest and posttest scores. We recognized that 20 questions were nei ther comprehensive nor adequate for valid testing. The same test was administered as the pretest
and the posttest. Results of the pretests and the posttests are shown in Table 3. It should be noted that a decrease in the reliability estimate (using Kuder-Richardson-20) 11 is to be expected where the scores for a large proportion of the group tested are high and the standard deviation is narrow. This is a function both of the statistic which is trouble some where the explicit testing objective is not discrimination and of the small number of questions. Table 4 shows the pretest and posttest results by objective. In fact, upon examination at the conclu sion of the workshop we determined that the "man agement" objective was not well handled in terms of the needs of the participants, most of whom were teachers. For the teachers, the goal of laying the "groundwork" through objectives one and two, together with four as it related to perceived re sources, would probably have been served without
TABLE 4 Pretest and Posttest Results by Objectives Showing the Number and Percent Below the Criterion, the Percent Achieving the Criterion, the Mean, and Standard Deviation, and the Net Gain in Number and Percent Achieving the Criterion from Pretest to Posttest Items Objective I @ 85% Definition of Perceptual Motor Development: Abnormal vs Normal Pretest Posttest Net Gain Objective II @ 85% Evaluation Tools Pretest Posttest Net Gain Objective III @85% Management Skills Pretest Posttest Net Gain Objective IV @ 100% Community Resources Pretest Posttest Net Gain 172
(Criterion)
(6) (6)
#Below
120 20
%
82.20
15.39
(N = 100)
% Achiev ing
X
SD
17.81 84.62 83.33
4.2 6.4
1.44
0.86
(5) (5)
107 30 (N = 77)
73.29 23.08
26.71 76.92 71.96
3.8 5.1
1.17 0.90
(4) (4)
96 67 (N = 84)
65.76 51.54
34.25 48.46 87.50
3.0 3.5
0.97 0.79
(2) (2)
42 25 (N = 17)
28.77 19.24
71.23 80.77 40.47
1.7 1.8
0.50 0.42
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 3 Test Statistics for the Pretest and Posttest of 20 Questions
Positive (+2,+1) Number
Total Possible Re Questions sponse
Responses
Ambivalent (-1)
Negative (-2,—3)
Omit
Responses
Responses
Responses
OI£
Focus Style Emphasis Outcomes Effects Total
Number
Percent
Number
Percent
Num ber
Percent
Num ber
Percent
3 3 2 2 2
246 246 164 164 164
183 142 110 120 121
74.39 57.72 67.07 73.17 73.78
53 90 41 34 30
21.54 36.58 25.00 20.73 18.29
8 12 12 10 10
3.25 4.88 7.32 6.10 6.10
2 2 1 0 3
.82 .82 .61 0.00 1.83
12
984
676
68.70
248
25.20
52
5.29
8
.81
the technical section on management, which was of interest to those participants who were more clini cally oriented. Management was more appropriate as discipline-specific subject matter for a traditional continuing education program. ATTITUDINAL EVALUATION The questionnaire constructed to elicit participant attitudes concerning the configurational design fac tors of the perceptual motor workshop contained 26 questions requiring rating in a guided choice mode (on a narrow 5-point scale rather than a broad 10-point scale). Eleven of the questions re lated to the specific configurational design factors we have discussed above. In effect, the attitudinal evaluation amounted to a request of the participants to employ the configurational criteria to evaluate the workshop. Each of the questions contained five alternative responses. Two of the responses were generally positive, one was ambivalent, and two were clearly negative. The purpose of the attitudinal evaluation was to evaluate the workshop and not the participants. The response mode selected was deliberately biased on the negative side by assigning a negative weight to the ambivalent response which resulted in two positive and three negative weights. By assigning weights to the alternative responses and deriving means for the second results, a general picture of the success of the workshop in terms of the program objectives was obtained for the five of the configu rational design factors for which immediate feed back data were obtainable. Table 5 summarizes the results of this questionnaire. The number of ques tionnaires included in this summary is 82 which represents only those persons for whom a pretest and posttest were also analyzed. Many other ques tionnaires were returned, but for purposes of this Volume 58 / Number 2, February 1978
evaluation, participation for the entire week was a prerequisite to judge the workshop on the basis of the criteria specified for its program objectives. In general, the overall assessment of the design factors by the workshop participant/judges —using the criteria and their measures —yielded positive results at greater than 70 percent for "focus," "outcomes," and "effects." The most critical judg ment rendered by the participant/judges was to the criteria for "style" where only 58 percent of the responses were positive. These judgments led us to conclude that the method for delivering the judg ments enabled the participant/judges to discriminate among the criteria and render differential responses to discrete criteria. Overall, the workshop nearly met the 70 percent criterion, and in no case did the judges indicate that the workshop significantly failed to exhibit configurational design factors. Specific criteria which required revision for future planning were identified. The participants generally re sponded favorably to the configurational design factors for regional continuing education in medical allied health. CONCLUSION In this paper four characteristics were identified as essential to regional continuing education for medical allied health. These characteristics were restated as criteria for six of the design components of a configurational strategy for regional continuing education. These characteristics and components (called design factors) were illustrated by reporting their use in the planning, implementation, and evaluation of a multidisciplinary workshop on per ceptual motor dysfunction. The four characteristics and six design factors were found useful in the planning for, and the evaluation of, regional contin173
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 5 Attitude Questionnaire Summary of Results Showing the Number and Percent of Positive, Ambivalent, and Negative Responses by Design Factor for 82 of the Workshop Participants Who Completed both the Pretests and Posttests
4. A configurational approach to program devel opment in continuing education will serve to assist administrators to evaluate needs assess ments and assign budget and personnel on the basis of a known set of criteria which differen tiate among various providers of continuing ed ucation activities appropriate to each provider. 5. The configurational strategy facilitates a multidisciplinary approach to interdisciplinary contin uing education coordinating the use of available resources and results in cost effective program ming. This paper has been designed to encourage use of the configurational strategy in response to those educational needs where the traditional strategy is clearly inappropriate. In many instances, traditional continuing education is appropriate, especially in discipline-specific subject matter, and in no way should this discussion be taken to mean that one must employ a configurational strategy; both ap proaches are obviously required.
REFERENCES 1. Koewing JR, Howes BR, Metz D: Four characteristics for regional continuing education in medical allied health. J Allied Health 5:31-40, Fall 1976 2. Long LD: The evaluation of continuing education efforts. Am J Public Health 59:967-973, 1969 3. Barrow A: Continuing education: Why is it necessary? Cad ence Nov/Dec: 59-63, 1973 4. APTA Committee on Continuing Education. Guidelines for continuing education for components of the American Physi cal Therapy Association. Phys Ther 52:405-407, 1972 5. Lewis CE, Hassanein RS: Continuing education an epidemio logic evaluation. N Engl J Med 282:254-259, 1970
174
6. Hickey T: Continuing education in gerontology for allied health. J Allied Health 4:5-12, Summer 1975 7. Lindberg D: Professional societies and continuing education. Cadence Nov/Dec: 9-12, 1973 8. Meeth LR: Quality Education for Less Money. San Francisco, Jossey-Bass Publishers, 1974 9. Karlsen B: Educational achievement testing with adults. In Griffith WS, Hayes AP (eds): Adult Basic Education. Univer sity of Chicago Press, 1970, pp 90-107 10. Pascascio A: Continuing education for quality health care. Phys Ther 49:257-264, 1969 11. Ferguson CA: Statistical Analysis in Psychology and Educa tion, New York, McGraw-Hill, Inc, 1959
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
uing education in medical allied health of an inter disciplinary nature. Where regional efforts in continuing education are contemplated, and where geographically defined areas have been determined, the following implica tions may be drawn: 1. The four characteristics for regional continuing education constitute practical criteria for the evaluation of proposals for special and demon stration projects for regional continuing educa tion. 2. Employment of the configurational strategy at the local and regional level will reduce duplicate program efforts and facilitate more efficient use of the available program money for competencyand discipline-focused continuing education. 3. Employment of the configurational strategy fa cilitates appropriate use of in-service and other resources towards those educational needs not the primary responsibility of regional or profes sional organizations and educational institutes.
Four characteristics of a configurational strategy for planning regional continuing education in medical allied health are identified in this article. These characteristics are restated as criteria for six design factors in the planning and evaluation of a regional continuing education program. A perceptual motor dysfunction workshop was planned and evaluated utilizing the configurational strategy.
One goal of the University of North Carolina Area Health Education Centers (AHEC) Program is the development of nine regional centers which have the capacity to respond to commonly identified health manpower educational needs and to coordi nate the utilization of existing local and statewide resources in such responses. Preliminary evaluation of the early experiences of the regional North Carolina AHEC allied health coordinators has sug gested four characteristics essential to the develop ment of regional continuing education activities for medical allied health programs. 1 These characteris tics—1) multidisciplinary focus, 2) coordinated use of available resources, 3) interdisciplinary experi ences, and 4) cost effectiveness —constitute criteria which may be used to guide the selection of an appropriate strategy for regional continuing educa tion and to evaluate its effectiveness. We have chosen to call programs which exhibit these charac teristics "configurational strategies" to emphasize the circumstances they are designed to meet. ProMr. Koewing was Assistant Professor, Department of Medical Allied Health Professions and Associate Director of the University of North Carolina Area Health Education Center Program, Chapel Hill, when this study was conducted. He is currently President, J. Robert Koewing Associates Inc, PO Box 95127, Schaumberg, IL 60195. Mr. Howes is Assistant Professor, Department of Medical Allied Health Professions and Director, Allied Health, University of North Carolina Area Health Education Center Program, Chapel Hill, NC 27514. Mr. Metz is Assistant Professor of Hospital Administration and Associate Director of the University of North Carolina Area Health Education Center Program, Chapel Hill, NC 27514. This study was supported in part by the US Department of Health, Education, and Welfare and the University of North Carolina Area Health Education Center Program in 1976. This paper was presented at the 52nd Annual Meeting of the American Physical Therapy Association, June 30, 1976, in New Orleans. 168
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
Six Design Factors for Planning and Evaluating Regional Continuing Education
grams which do not exhibit these characteristics we have chosen to call "traditional strategies." DESIGN FACTORS OF A CONFIGURATIONAL STRATEGY COMPARED WITH TRADITIONAL STRATEGY
Regardless of whether the overall strategy for a particular program is "configurational" or "tradi tional," all programs have a basic design. Character istics essential for effective regional continuing edu cation must be exhibited within the program design. Though a program design comprises many factors, six are sufficient to illustrate the difference between "traditional" and "configurational" strategies. Be cause of the widespread attention given to such design factors as needs assessment, behaviorally stated objectives, competent leadership, perfor mance based evaluation, and others, these will not be discussed here. The factors selected for this analysis are less often systematically evaluated, al though their influence upon the effectiveness of a design for regional continuing education programs cannot be overlooked. These design factors are 1) focus, 2) style, 3) emphasis, 4) outcomes, 5) effect, and 6) resources. Table 1 provides a definition of each factor and a comparison of the characteristics of "traditional" and "configurational" strategies for regional continuing education. Specific disadvantages of the traditional strategy became apparent in the comparison of the charac teristics as shown in Table 1. The major disadvan tages are as follows: 1. Traditional strategies ignore the multidiscipli nary professional experience of the participants by focusing on subject matter. PHYSICAL THERAPY
relates these objectives and their criteria to the design factor to which they correspond.
CONFIGURATIONAL PROGRAM DEVELOPMENT: AN EXAMPLE One of the nine regional Area Health Education Centers in North Carolina recently developed a workshop for professionals who work with children with perceptual motor dysfunction. Because of the regional visibility of the Charlotte AHEC and its allied health coordinator, a local professional orga nization of physical therapists sought AHEC assist ance in order to respond to a specific need they identified among health and educational profession als. The AHEC allied health coordinator responded to the request by initiating a series of planning sessions during which she assisted the physical ther apists to broaden their scope by incorporating nu merous allied health and other disciplines directly or indirectly involved with such children. Her direct link with university faculty resources soon yielded information that a similar workshop requiring the identical faculty resources was in the planning stage. In addition it was learned that other regional orga-
TABLE 1 A Comparison of the Characteristics of Configurational and Traditional Strategies for Continuing Education in Medical Allied Health Definition
Configurational Strategies
Traditional Strategies
1. Focus
. . . the person for whom edu cational experiences are in tended
Multidisciplinary
Single discipline
2. Style
. . . the experience which is intended for persons who par ticipate
Integrate disciplines in com mon educational experiences
Differentiate among disciplines for educational experiences
3. Emphasis
. . . interdisciplinary educa tional experiences addressed to common needs of participants undifferentiated by specialty or discipline
Facilitate interdisciplinary communication by emphasiz ing subject matter common to several disciplines
Facilitate disciplinary specializa tion by emphasizing subject mat ter unique to (or tailored for) a particular discipline
4. Outcomes
. . . experiences in which par ticipants achieve clarity con cerning the role and function of particular disciplines through interdisciplinary reinforcement
Promote interdisciplinary clar ity and mutual (interprofes sional) reinforcement with re spect to role and function
Promote disciplinary clarity (of ten isolation) and stimulate legal and quasi-legal (external) rein forcement for role and function
5. Effect
. . . the individual's profes sional experience
Facilitate problem solving in terms of the configuration of disciplinary resources appro priate to resolve them (Facili tate team management)
Encourage problem solving in terms of the contribution of a particular discipline
6. Resources
. . . facilitate the experience intended for participants
Promote resource sharing
Promote specialized use of re sources
Volume 58 / Number 2, February 1978
169
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
2. Discipline-specific subject matter discourages in tegration through team approaches to patient care. 3. Emphasis on "uniqueness" and "specialization" results in isolation of the allied health profes sional. 4. Focus on competency maintenance within one discipline does not enable the individual to achieve interdisciplinary reinforcement and rec ognition. 5. Problem solving by discipline ignores the context within which professionals experience the need for continuing education which is the multiprofessional complex of patient care. 6. Focus on the means, rather than on the ends, of continuing education results in duplication of resources and correspondingly decreases cost effectiveness. To correct for these disadvantages and to incor porate the characteristics we have identified as essential for effective regional continuing education in medical allied health into the design of educa tional experiences, we have developed a statement of six configurational program objectives and have identified two evaluation criteria for each. Table 2
Program Objective
Criteria
1. Focus
Program is designed with a multidisciplinary focus
1. The program is relevant to persons of several disci plines 2. The experience provided is consistent with the multidisciplinary health care environment
2. Style
Program is designed to integrate disci plines in common educational experi ences
1. The experiences are relevant to the individual's professional experience. 2. Participants gain experience in the integration of competencies and skills from other disciplines with those of their own
3. Emphasis
Program is designed to facilitate inter disciplinary communication by empha sizing subject matter common to several disciplines
1. Subject matter is defined by common needs 2. Disciplinary contributions to the multidisciplinary health care environment are articulated within an interdisciplinary framework
4. Outcomes
Program is designed to promote clarity and mutual reinforcement for role and function
1. Role and function reinforcement occurs through colleagues of other disciplines 2. Differentiation of role and function is achieved on the basis of interdisciplinary clarity
5. Effect
Program is designed to facilitate prob lem solving in terms of the configura tion of disciplinary resources appropri ate to resolve them
1. Problem solving is encouraged which utilizes appro priate interdisciplinary resources 2. Direct application of resources to participants expe rience is intended
6. Resources
Program is designed to promote re source sharing
1. Resources are not confused with program objectives 2. Resources are directed towards goals and objectives
Design Factor
nizations had initiated requests for similar technical assistance dealing with children who have percep tual motor problems. Part of the allied health coordinator's role is to relate formally and informally to regional educa tional and health service delivery institutions as well as to professional organizations. The resulting linkages facilitated the inclusion of these institutions and organizations in a cooperative planning process thus assuring their multidisciplinary input into its design and content. These linkages yielded measurable outcomes even in the planning stage. One outcome of these re gional multidisciplinary linkages was illustrated in the participation of a local school system which cosponsored the workshop and released 100 physi cal and special education and classroom teachers for all or part of the workshop. A second outcome was illustrated by the participation of professionals sponsored by a state agency which had itself con tacted the same university faculty for identical tech nical assistance. A third outcome of the multidisci plinary linkages of the allied health coordinator was the regional and statewide faculty and instructional resources which were shared for the planning and 170
implementation of this educational experience. State support was provided by the University of North Carolina School of Medicine's Department of Medical Allied Health Professions as a regular part of its contract commitment to the University's AHEC Program. These resources were utilized to back up the regional professional, clinical, and community college faculty. Criteria for configurational continuing education programs were therefore evident in the develop ment of this Charlotte AHEC regional experience for professionals working with children with percep tual motor dysfunction. The program was multidis ciplinary in its focus. It was planned to coordinate available regional and state resources. It exhibited the cost effective characteristics essential to regional programming for allied health by pooling faculty and participants and planning efforts. Finally the workshop was designed as an interdisciplinary ex perience to facilitate interprofessional communica tion and exchange. As a result of the application of these characteristics in the planning, over 190 par ticipants representing 15 disciplines with the com bined support of significantly affected local agencies registered for, and participated in, portions of a
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 2 Six Design Factors for Regional Continuing Education Stated as Configurational Program Objectives with Evaluation Criteria
OBJECTIVES-REFERENCED EVALUATION OF REGIONAL CONFIGURATIONAL CONTINUING EDUCATION: AN EXAMPLE An elusive question to continuing educators is, "What difference has this program made?" As with any educational experience, the goals and objectives for any continuing education programs must be stated in measurable terms. Numerous educators and administrators continue to debate the effective ness of continuing education programs in achieving behavior changes in adult participants and "how" to evaluate it. 2-9 Unlike academic curricula, a con tinuing education program does not take place over a sustained period of time within a reasonably con trolled environment. Equal learning potential among participants cannot be assumed because of wide differences in training, education, and experi ence. Continuing education participants are seldom screened for selection by a battery of tests typical of undergraduate and graduate academic programs even though these data are helpful in documenting baseline behaviors and isolating critical predictive variables which result in selection of homogeneous groups. There are those persons who doubt the validity of continuing education because such base line data and the research such data may propagate do not exist; it is as if these persons are willing to believe that no education is effective if it cannot be measured with their tools. On the other hand, there are those persons who do not believe continuing education should be measured —at least in the be haviors of those who participate; 10 it is as if these people are willing to say any continuing education is better than none, and the need to measure results may prevent people from participating. A helpful middle ground is found in objectivesreferenced evaluation. This form of evaluation is particularly useful where research is neither needed nor intended, and where the evaluation questions are related to the program and not the participants. The prerequisite for such evaluation is a set of program and instructional objectives which specify the intended outcomes in such a way that data may be elicited from program participants in both formal or informal ways. The assumption here is that the definition of an effective program is one which results in changes in participant behavior. The infer ence is that when such changes happen, the program
Volume 58 / Number 2, February 1978
was causal to the behavior change. The definition of an effective program, in this instance, is limited to the objectives the program was designed to accomplish. In no way can a generic statement of effectiveness be made without reference to the particular objectives intended for specific programs. The evaluation employed with the workshop on perceptual motor dysfunction was objectives-refer enced. The questions to which the evaluation was addressed were as follows: 1. Was the workshop successful in enabling the participants to demonstrate achievement of the cognitive objectives? 2. What attitudes did the workshop elicit in the participants in relation to the configurational design factors? A cognitive pretest and posttest were administered to elicit participant data to answer the first question. An attitudinal questionnaire was constructed to elicit participant feedback for the second. An eval uation of participant behavior on the job was not feasible to include in the evaluation design because of circumstances and composition. Control groups were unnecessary because the subject of the evalu ation was the program rather than the participant behaviors.
COGNITIVE EVALUATION The goals and objectives for the workshop as presented to the participants were To lay the groundwork for intervention in perceptual motor dysfunction through the utilization and coordi nation of community resources. In order to accom plish this, we hope to educate specialists in education (75 teachers of kindergarten and grades one through three and 75 elementary school physical educators), psychology, communication, physical therapy, recre ation therapy, occupational therapy, social service, and pediatrics to the latest information on multidisciplinary intervention in perceptual motor dysfunction by: 1. Defining Perceptual Motor Development —abnor mal versus normal. 2. Identifying specific evaluation tools. 3. Identifying selected management skills. 4. Identifying local community resources.
Ultimately the effectiveness of the workshop could be measured by the number of "interventions" in perceptual motor problems which utilized and co ordinated community resources. Obviously the identification of one workshop as the significant factor in achieving such interventions is nearly im possible. Therefore, a posttest was developed to 171
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
week-long continuing educational experience. Of these, 82 (43%) participated for the entire week and 108 (57%) participated in segments of the program.
Pretest Posttest
Number*
Mean
Variance
Standard De viation
Reliability KR-20
Standard Error of Measure ment
146 130
12.66 16.70
6.16 3.12
2.48 1.76
.42 •32
1.87 1.45
* 146 represents the attendance at the time the pretest was given. 130 represents the attendance at the time the posttest was given.
enable participants to demonstrate mastery of the particular cognitive objectives. A cognitive test was developed by asking each of the major presenters to submit one or two multiple choice questions which covered key or essential subject matter. The goal was to create a mini-test which contained the 20 most significant cognitive items of the workshop. The questions submitted were reviewed by other members of the technical staff and faculty for form and content. The test was limited to 20 items to help ensure completion by the participants and to facilitate quick turn-around hand scoring on the final day of the workshop so that participants could receive immediate feedback on the raw data showing their pretest and posttest scores. We recognized that 20 questions were nei ther comprehensive nor adequate for valid testing. The same test was administered as the pretest
and the posttest. Results of the pretests and the posttests are shown in Table 3. It should be noted that a decrease in the reliability estimate (using Kuder-Richardson-20) 11 is to be expected where the scores for a large proportion of the group tested are high and the standard deviation is narrow. This is a function both of the statistic which is trouble some where the explicit testing objective is not discrimination and of the small number of questions. Table 4 shows the pretest and posttest results by objective. In fact, upon examination at the conclu sion of the workshop we determined that the "man agement" objective was not well handled in terms of the needs of the participants, most of whom were teachers. For the teachers, the goal of laying the "groundwork" through objectives one and two, together with four as it related to perceived re sources, would probably have been served without
TABLE 4 Pretest and Posttest Results by Objectives Showing the Number and Percent Below the Criterion, the Percent Achieving the Criterion, the Mean, and Standard Deviation, and the Net Gain in Number and Percent Achieving the Criterion from Pretest to Posttest Items Objective I @ 85% Definition of Perceptual Motor Development: Abnormal vs Normal Pretest Posttest Net Gain Objective II @ 85% Evaluation Tools Pretest Posttest Net Gain Objective III @85% Management Skills Pretest Posttest Net Gain Objective IV @ 100% Community Resources Pretest Posttest Net Gain 172
(Criterion)
(6) (6)
#Below
120 20
%
82.20
15.39
(N = 100)
% Achiev ing
X
SD
17.81 84.62 83.33
4.2 6.4
1.44
0.86
(5) (5)
107 30 (N = 77)
73.29 23.08
26.71 76.92 71.96
3.8 5.1
1.17 0.90
(4) (4)
96 67 (N = 84)
65.76 51.54
34.25 48.46 87.50
3.0 3.5
0.97 0.79
(2) (2)
42 25 (N = 17)
28.77 19.24
71.23 80.77 40.47
1.7 1.8
0.50 0.42
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 3 Test Statistics for the Pretest and Posttest of 20 Questions
Positive (+2,+1) Number
Total Possible Re Questions sponse
Responses
Ambivalent (-1)
Negative (-2,—3)
Omit
Responses
Responses
Responses
OI£
Focus Style Emphasis Outcomes Effects Total
Number
Percent
Number
Percent
Num ber
Percent
Num ber
Percent
3 3 2 2 2
246 246 164 164 164
183 142 110 120 121
74.39 57.72 67.07 73.17 73.78
53 90 41 34 30
21.54 36.58 25.00 20.73 18.29
8 12 12 10 10
3.25 4.88 7.32 6.10 6.10
2 2 1 0 3
.82 .82 .61 0.00 1.83
12
984
676
68.70
248
25.20
52
5.29
8
.81
the technical section on management, which was of interest to those participants who were more clini cally oriented. Management was more appropriate as discipline-specific subject matter for a traditional continuing education program. ATTITUDINAL EVALUATION The questionnaire constructed to elicit participant attitudes concerning the configurational design fac tors of the perceptual motor workshop contained 26 questions requiring rating in a guided choice mode (on a narrow 5-point scale rather than a broad 10-point scale). Eleven of the questions re lated to the specific configurational design factors we have discussed above. In effect, the attitudinal evaluation amounted to a request of the participants to employ the configurational criteria to evaluate the workshop. Each of the questions contained five alternative responses. Two of the responses were generally positive, one was ambivalent, and two were clearly negative. The purpose of the attitudinal evaluation was to evaluate the workshop and not the participants. The response mode selected was deliberately biased on the negative side by assigning a negative weight to the ambivalent response which resulted in two positive and three negative weights. By assigning weights to the alternative responses and deriving means for the second results, a general picture of the success of the workshop in terms of the program objectives was obtained for the five of the configu rational design factors for which immediate feed back data were obtainable. Table 5 summarizes the results of this questionnaire. The number of ques tionnaires included in this summary is 82 which represents only those persons for whom a pretest and posttest were also analyzed. Many other ques tionnaires were returned, but for purposes of this Volume 58 / Number 2, February 1978
evaluation, participation for the entire week was a prerequisite to judge the workshop on the basis of the criteria specified for its program objectives. In general, the overall assessment of the design factors by the workshop participant/judges —using the criteria and their measures —yielded positive results at greater than 70 percent for "focus," "outcomes," and "effects." The most critical judg ment rendered by the participant/judges was to the criteria for "style" where only 58 percent of the responses were positive. These judgments led us to conclude that the method for delivering the judg ments enabled the participant/judges to discriminate among the criteria and render differential responses to discrete criteria. Overall, the workshop nearly met the 70 percent criterion, and in no case did the judges indicate that the workshop significantly failed to exhibit configurational design factors. Specific criteria which required revision for future planning were identified. The participants generally re sponded favorably to the configurational design factors for regional continuing education in medical allied health. CONCLUSION In this paper four characteristics were identified as essential to regional continuing education for medical allied health. These characteristics were restated as criteria for six of the design components of a configurational strategy for regional continuing education. These characteristics and components (called design factors) were illustrated by reporting their use in the planning, implementation, and evaluation of a multidisciplinary workshop on per ceptual motor dysfunction. The four characteristics and six design factors were found useful in the planning for, and the evaluation of, regional contin173
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
TABLE 5 Attitude Questionnaire Summary of Results Showing the Number and Percent of Positive, Ambivalent, and Negative Responses by Design Factor for 82 of the Workshop Participants Who Completed both the Pretests and Posttests
4. A configurational approach to program devel opment in continuing education will serve to assist administrators to evaluate needs assess ments and assign budget and personnel on the basis of a known set of criteria which differen tiate among various providers of continuing ed ucation activities appropriate to each provider. 5. The configurational strategy facilitates a multidisciplinary approach to interdisciplinary contin uing education coordinating the use of available resources and results in cost effective program ming. This paper has been designed to encourage use of the configurational strategy in response to those educational needs where the traditional strategy is clearly inappropriate. In many instances, traditional continuing education is appropriate, especially in discipline-specific subject matter, and in no way should this discussion be taken to mean that one must employ a configurational strategy; both ap proaches are obviously required.
REFERENCES 1. Koewing JR, Howes BR, Metz D: Four characteristics for regional continuing education in medical allied health. J Allied Health 5:31-40, Fall 1976 2. Long LD: The evaluation of continuing education efforts. Am J Public Health 59:967-973, 1969 3. Barrow A: Continuing education: Why is it necessary? Cad ence Nov/Dec: 59-63, 1973 4. APTA Committee on Continuing Education. Guidelines for continuing education for components of the American Physi cal Therapy Association. Phys Ther 52:405-407, 1972 5. Lewis CE, Hassanein RS: Continuing education an epidemio logic evaluation. N Engl J Med 282:254-259, 1970
174
6. Hickey T: Continuing education in gerontology for allied health. J Allied Health 4:5-12, Summer 1975 7. Lindberg D: Professional societies and continuing education. Cadence Nov/Dec: 9-12, 1973 8. Meeth LR: Quality Education for Less Money. San Francisco, Jossey-Bass Publishers, 1974 9. Karlsen B: Educational achievement testing with adults. In Griffith WS, Hayes AP (eds): Adult Basic Education. Univer sity of Chicago Press, 1970, pp 90-107 10. Pascascio A: Continuing education for quality health care. Phys Ther 49:257-264, 1969 11. Ferguson CA: Statistical Analysis in Psychology and Educa tion, New York, McGraw-Hill, Inc, 1959
PHYSICAL THERAPY
Downloaded from https://academic.oup.com/ptj/article-abstract/58/2/168/4559689 by University of Texas at Dallas - McDermott Library user on 14 February 2019
uing education in medical allied health of an inter disciplinary nature. Where regional efforts in continuing education are contemplated, and where geographically defined areas have been determined, the following implica tions may be drawn: 1. The four characteristics for regional continuing education constitute practical criteria for the evaluation of proposals for special and demon stration projects for regional continuing educa tion. 2. Employment of the configurational strategy at the local and regional level will reduce duplicate program efforts and facilitate more efficient use of the available program money for competencyand discipline-focused continuing education. 3. Employment of the configurational strategy fa cilitates appropriate use of in-service and other resources towards those educational needs not the primary responsibility of regional or profes sional organizations and educational institutes.
