Evaluation: A Keystone of Comprehensive Health Planning Jerry Osterweil, Ph.D.

ABSTRACT: This paper describes the implications for mental health programming of the emphasis on evaluation throughout the regulations for comprehensive health planning and health service development in the Partnership for Health legislation. A new and enlarged planning effort was initiated through the Comprehensive Health Planning and Public Health Service Amendments of ~966. This effort has created new needs for the development of adequate mental health baseline statistics and evaluation measures. Two journal articles have offered pointed observations about the Government's need for statistics and evaluation, if those responsible for managing Federal programs are to make sense out of the rapid proliferation of program development. For example, in the spring z967 issue of the Journal of Public Interest, Daniel P. Moynihan wrote an article entitled "A Crisis of Confidence." In it he proposed that the Government set up an independent office of legislative evaluation to make systematic statistical assessments of the impact and long-range effects of various proposals coming out of new legislation. Also, in the May ~2, ~967 issue of Science there was an article by Arthur Kantrowitz, entitled "A Proposal for an Institution for Scientific Judgment." This artide suggested that Congress set up a mechanism for the review of important scientific programs. A common theme that runs through both of these articles is a recognition of the need for better statistics and better methods and measures of evaluation. It is a theme which will undoubtedly be forcefully reechoed in the course of further developments issuing from comprehensive health planning. This artide will review the major sections of this legislation with a particular emphasis on the statistical, evaluative implications of each portion of the law. Dr. Osterweil is deputy director, Division of Manpower and Training Programs, National Institute of Mental Health, 5454 Wisconsin Ave., Chevy Chase, Md. 2o2o3. Earlier versions of this paper were presented to the National Conference for Mental Health Statisticians in Chicago, Ill., May I967, and the Conference of Psychologists and Consultants in State, Federal, and Territorial Mental Health Programs in Washington, D.C., September I967. Community Mental Health Journal, Vol. 5 (2), 1969

121

122

Community Mental Health Journal

The Comprehensive Health Planning and Public Health Service Amendments of x966 completed a second program year during fiscal year z968 which ended in June 1968. Because there was not sufficient time for detailed consideration of this legislation during its initial hearing in i966, Congress decided to authorize this program at that time for only one fiscal year. As a result, during z967 additional hearings were held regarding the extension of the legislation by the Committee on Interstate and Foreign Commerce of the House of Representatives under the chairmanship of Representative Staggers of West Virginia. The Partnership for Health Amendments of :r967 (PL 9o-I74) were enacted and signed December 3, ~967, amending and extending the basic legislation. This article will review the level of funding provided by this legislation and describe the various sections of the legislation. FORMULA GRANTS TO STATES FOR COMPREHENSIVE HEALTH PLANNING Let us first consider in detail each of the sections of the original bill. Section 3~4(a) of the legislation authorized funding of $2. 5 million for fiscal year :t967 to provide formula grants to the states for state-level comprehensive health planning. The ~967 amendments increased the authorization for i968 from $5 million to $7 million. This level of funding was increased to Szo million in z969 and to $z5 million in z97 o. The actual levels of appropriation were $5 million in fiscal year z968 and $7.375 million in fiscal year

:r969. To qualify for funds under Section 3z4(a), states are required to designate a single state health planning agency, which may be interdepartmental, to administer this planning. The states also need to set up a state-level health planning advisory council composed of representative's of state and local public agencies as well as nongovernmental organizations and groups concerned with health. In order to be eligible for this formula grant .support, the states were required initially to present a work design which indicated what areas of planning activity they proposed to undertake with this assistance during fiscal years :r967 and ~968. Many of the major activities that the state is expected to undertake under comprehensive health planning have obvious statistical and evaluative implications. 9. The state, for example, is expected to select and apply measures for evaluating the health of the population and for assessing the impact of environmental, socioeconomic, and other demographicfactors on health and mental health. 2. The state is also expected to undertake studies to define the scope and nature of health and mental health problems and to identify and assess the resources available and necessary to meet them. 3. In addition, the state is expected to select goals and priorities for meeting identified health problems through the use of available resources or through the development of new resources. The selection of goals obviously implies an ordering of various health program activities, and this ordering presumably should be based not only on the best professional judgment but on the best available indexes for measuring the significance of various health problems.

Jerry Osterweil

123

4. The state is expected to develop current as well as long-range policy and action recommendations for meeting the health needs of the people of the state through both public and private efforts. 5. As a final example, the state is expected to develop criteria for evaluating health programs and their contribution to the goals established through comprehensive health planning. These criteria of evaluation may be based initially on subjective measures but it is expected that they will be substantially reinforced by appropriate statistical measures as they become available. A major concern in the training of health planners under Section 3x4(c) is that adequate technical competence be developed so that there can be more adequate statistics to underpin the continued development of comprehensive health planning. PROJECT GRANTS FOR AREA-WIDE PLANNING Section 314(b) of the Public Health Service Act authorizes project grants for area-wide health planning to public or nonprofit private agencies. These funds can cover up to 75% of the cost for developing comprehensive regional, metropolitan area, or other local area plans for the coordination of existing and planned health services. Applications for these grants must be approved first by the State Comprehensive Health Planning Agency before they can be funded by the Public Health Service. The extension of the legislation authorized appropriations of up to $7.5 million for fiscal year i968 and increased the authorization to up to $Io million in fiscal year "r969 and $I 5 million in fiscal year I97o. The actual appropriation for fiscal year I969 was $7.5 million with $I 5 million authorized but not yet appropriated for fiscal year i97o. The charge to the new area-wide health planning groups is similar at the regional level to the required activities for the state-level comprehensive health planning. Again, if one surveys some of the activities that are expected to be accomplished through area-wide health planning, one is immediately struck with the vast expansion of the statistical and evaluative capability that will be needed to carry out this effort effectively. The area-wide agency, for example, is expected to perform the following functions. 2. It is to establish a system for gathering and analyzing data on the characteristics and health problems of the entire population of the area, as well as on the availability, development, and utilization of health services, facilities, and manpower. 2. The area-wide health planning agency is expected to review existing laws, studies, and data; to conduct studies, surveys, and by other means to determine the environmental hazards and deficiencies and physical and mental health problems of the entire population, particularly of target groups within this population. In addition, there will be an extensive development of inventory data similar to that with which we have been familiar in the mental health planning field. This will include the number and nature of health services, facilities, and manpower available in any area. The agency will be expected to evaluate the financial, organizational, jurisdictional, or other impediments to the effective use of existing resources or the development of new ones. It will also be the responsible agency for analyzing health

124

Community Mental Health Journal

program development needs and the opportunities afforded through Federal, state, and local legislation that can help to fulfill these planning and program objectives in the area. 3. It will recommend goals and policies for developing and improving the system which provide physical, mental, and environmental health services for the area. 4. In cooperation with the appropriate public agencies, voluntary organizations, and health-related institutions, it will prepare plans and procedures for the attainment of health goals for the coordination of health programs. 5- The last set of activities of the area-wide agency, which has especially obvious statistical and evaluative implication, is that the area-wide agency will conduct periodic evaluations of all health planning within the area which contributes to the state planning effort. It will prepare data and studies which will aid in the conduct of comprehensive state health planning, and it will recommend projects which meet the area's needs, which will assist in local implementation of the state plan. It is expected that previous activities and specialized planning such as mental health planning will be continued at the state level and it is, therefore, altogether appropriate that some portion of area-wide planning efforts also be devoted to the coordinating of mental health services and the interrelating of mental health services to the spectrum of health care program development. Continued awareness of the uses and implications of this program by personnel in the mental health field can make a great deal of difference in how effectively this new legislative instrumentality will be used to address some of the critical mental health problems in our country. PROJECT GRANTS FOR TRAINING, STUDIES, AND DEMONSTRATIONS One of the more interesting portions of the new legislation falls under Section 314(c) which provides project grants for training, studies, or demonstrations looking toward the development of improved and more effective comprehensive health planning. The sum of $I. 5 million was authorized for fiscal year 2967, $2. 5 million for I968, $5 million for 1969, and $7.5 million for I97o. The actual appropriation was $2. 5 million for 2968 and $4.125 million for 2969 . It is anticipated that this section will provide for the development of various levels of additional training for people to more effectively carry out comprehensive health planning. One level will provide for short-term, continuing education courses to improve the skills of people already working in the field. Orientation and inservice training courses can be provided through this mechanism. At another level it will provide for the development of specific courses and curriculums in academic centers that will increase the planning capability of various professional groups connected with planning, and at the fullest level of development it will provide the funding for centers of comprehensive health planning which will have full academic curriculums at

Jerry Osterweil

125

various degree levels. This section of the legislation will enable academic centers not only to conduct training but to be a resource for the surrounding area and its own planning activities. It is of interest that the funds available under Section 314(c) will provide for a wide range of support, including stipends, faculty salaries, equipment, curriculum development, and the indirect cost for setting up a large-scale training center. This section of the legislation will also support the development of demonstration programs to increase the effectiveness of approaches to planning. Some of these demonstration programs will be integrated into the ongoing training activities in these selected centers. FORMULA GRANTS FOR PUBLIC HEALTH SERVICES Section 3z4(d) of the Public Health Service Act authorized formula grants for state health and mental health authorities to assist in establishing and maintaining adequate public health services including the training of personnel for public health work. During 2967, this portion of the program was carried under existing legislative authority and was not given specific funding authorization under this legislation until fiscal year 5968. The amount authorized for i968 was $7 ~ million, which was increased to $9 ~ million for I969 and to $1oo million for I97o. The actual amount appropriated was $6o.25 million for fiscal year 1968 and $66.o3z million for fiscal year I969 . Since a large majority of states have organized their mental health services in departments other than state health departments, the legislation provides that at least i5% of the states' allotment for public health services be made available to the state mental health authority. This then corresponds to the money that has been available in the past for mental health services under the mental health formula grant-in-aid to the states. It is especially significant that under the new requirements for formula grants under Section 3z4(d) there has been an increased effort to delineate programmatic dimensions and to use systematic ways of describing the nature of particular programs. Each of these areas in the narrative description for the state plan for the use of formula grants also has obvious statistical and evaluative implications. It is expected, for example, that the plan will describe the particular health problems to which a program is directed and that related supporting data which provide quantitative evidence of a particular health need of the people should be included in this description. The program should then define a set of specific objectives in terms of anticipated impact on the health status of the people. It is expected that these objectives will be based on quantitative measures, whenever possible. The state agency is then expected to describe the actual methodology, that is, the public health measures, that will be used to reach the above-stated objectives. The narrative should indicate the geographic location; where the activities will be performed; the agency, institution, organization, individ-

126

Community Mental Health Journal

uals who perform these services; and a description of the people who will be served and how the public will be informed of the availability of services. Finally, it is anticipated that each program will have described some methods to be used in the evaluation of the progress in reaching the stated program objectives, including changing the health status of the people and the effectiveness and efficiency of program operations. It should be obvious that the language that is used in this portion of the instructions for Section 3~4(d) requires development of evaluative measures which will provide some indication of the effectiveness of alternative strategies and alternative programs. In other words, it is an effort to reach not only more quantitative data but also data that can ultimately evaluate the effect of this program on the health status of people. As these trends become more fully implemented, it is obvious that the need for additional statistical and evaluative information will become more and more critical, and the weaknesses in our existing methods of statistical reporting techniques and evaluation will also become more apparent. PROJECT GRANTS FOR PUBLIC HEALTH SERVICES The final section of the legislation under Section 3~4(e) authorized project grants to any public or nonprofit private agency, institution, or organization to cover part of the cost of projects for health service development. The amount authorized was $9 ~ million for fiscal year 2968. The authorizations for Section 3z4(e) were $95 million in fiscal year I969 and $8o million in fiscal year ~97 o. The amounts authorized in fiscal years 2968 and I969 include Sao million for rat control projects. The amounts actually appropriated were $6a.5 million for fiscal year 1968 and $7 ~ million for fiscal year i969 , with $15 million in fiscal year 1969 for rat control projects. These projects should be concerned with providing services, including related training, to meet health needs of limited geographic scope or specialized regional or national significance. It will provide for developing and supporting for an initial period new programs for health services, including related training. It is anticipated that some of these funds can be used to support services in the mental health area. It could provide a vehicle whereby worthy demonstration projects could be replicated, particularly those which are developed as a part of comprehensive health services in areas of greatest need. Here again, the obvious need for the development of more effective evaluative measures is very apparent and the linkage of these demonstration efforts with other considerations of area-wide and state-level comprehensive health planning will require a relating of overall state and regional data to the specific data that are being generated at a specific locality. The need for biometric statistics to reinforce this effort is again perfectly obvious. NEW FEATURES OF THE "r967 AMENDMENTS The Partnership for Health Amendments of I967 extend-

Jerry OsterweiI

127

ed the authorization provided in PL 89-749 for two more years, fiscal years

:~969 and 297o, and added a number of new features to the original legislation, including the following: 2. In addition to extending the period of authorization for the planning and public health service grant programs, the amendments also provided for modest increases in the grant ceilings. 2. The amendments require that at least 7o% of the funds allotted to the state health and mental health agencies for public health services under Section 3~4(d) of the Public Health ServiceAct be made available only to support services in communities. A number of other provisions were included which are not germane to the subject of this paper, including a new Section 3o4 of the Public Health Service Act dealing with research and demonstrations relating to health facilities and services, a new Section 353 concerned with licensing of laboratories, and revisions of other sections of the Public Health Service Act to deal with volunteer services, and the sharing of medical care facilities and resources. FUTURE PROSPECTS The massive planning effort under comprehensive health planning that has been described is of special significance because it is not viewed as a time-limited activity. The expectation is that it will be a continuous effort to develop long-range professional capability so that states will ultimately have available a cadre of personnel who can conduct this kind of program on a continuous basis. The advantages in development of more thoroughgoing conceptual schemes, of more systematic evaluative approaches, and of better formulations of research ideas for this program cannot be underestimated. The selection of problems and the intensity with which public attention is fixed on various problems will be supported, dramatized, and substantiated by the use of evaluative techniques as well as a variety of new methods of statistical and program analysis. It is up to those in the mental health field to see that the spotlight does not move too readily past the mental health problems so that they are lost in the shadows and overlooked in this new planning effort. It is their job to see that the spotlight focuses on the vast array of mental health problems that still need further attention. One important contribution that can be made in bringing this about is the refinement and improvement of methods of evaluation and mental health statistics. In the long run, the potential use of these funds for mental health programs will be dependent on the extent to which mental health effectively develops specific program dimensions so that the funds can be defined and the impact of these funds can be adequately described. As the program comes under further Congressional review, an important determinant of its continued support, and particularly of the support of mental health activities under this legislation, will be the extent to which we are able to assess more effectively the needs for a wide variety of mental health services. It will also be contingent on our ability to generate adequate statistical and evaluative description of the programmatic impact and effectiveness of

128

Community Mental Health Journal

these service programs. It is in this effort that the role of the mental health researcher is likely to be enlarged. We are moving, we hope, into a period when at the very beginning of the programs, throughout the course of their development, and in the evaluation of completed programs, mental health evaluation will be an integral part of the entire program operation. For in order to meet the requirements that this legislation specifically describes in each of its sections, the basic underpinnings of adequate statistics and improved evaluation will be a necessary part of the enlarging effort to provide more effective health and mental health services to our population.

A measure of professional orientation COMMUNITY MENTAL HEALTH IDEOLOGY SCALE (CMHIS) FRANK BAKER, PrI.D., and HErt~r C. SCH~L~X~C,PH.D. Laboratory of Community Psychiatry, Harvard Medical School A standardized, 38-item questionnaire for measuring the opinions of professionals about innovative practices in mental health, including manual reporting validity, reliability, and item analysis. Scale discriminates between groups highly oriented to community mental health ideology and random samples of mental health professionals. PRICES: Test booklets

pkg of 25 $ 4.75 pg 100 15.30 pg 500 68.85

Preliminary manual $ 1.50 Scoring key .50 Spedmen set (with manual and keys) 2.50

Behavioral Publicafons

2852 Broadway-Morningside Heights New York, N.Y. "10025

ai|mmm nmaala Nllam

ll lil lal l l r IIIIlll|

Evaluation: A keystone of Comprehensive Health Planning.

This paper describes the implications for mental health programming of the emphasis on evaluation throughout the regulations for comprehensive health ...
505KB Sizes 0 Downloads 0 Views