Public Health Nursing Vol. 9 No. 2, pp. 133-137 0737-1209/92/$6.00 0 1992 Blackwell Scientific Publications, Inc.
A Public Health Quality Assurance System Cheryl Zlotnick, R.N., Dr. P.H.
Abstract Quality assurance (QA) systems are commonplace among hospitals. They are even found in home health and health maintenance organizations, but they are less common among public health agencies. This article enumerates the discrepancies between the design of traditional QA and that necessary to meet the needs of a department of health’s public health nursing service. I t characterizes the subsequent changes that must be inserted into the QA system for it to comply with the mission and services of a public health setting. The system and instruments presented are used in the Baltimore County Department of Health, Public Health Nursing Services.
Quality assurance is a widely accepted system that compares the care provided to institutionally held standards, evaluates data, identifies problems, plans and implements activities to alleviate the problems, and determines whether the activities achieved the desired results (Rowland & Rowland, 1987). Due to regulations, the majority of hospitals have intricate quality assurance (QA) programs. Such programs are also beginning to proliferate in health maintenance organizations and home health agencies, in large part to achieve thirdparty payer status for government programs (i.e., Medicare and Medicaid) (Rowland & Rowland, 1987). One type of health organization in which QA is less common is the local government department of health. With the increasing demands for health care regulation and ever-present litigious climate, health departments have begun to establish QA systems. Most of the existing material on QA is from facilities that provide curative health care, yet most local depart-
Cheryl Zlotnick is director of evaluation and research, Baltimore County Department of Health, Public Health Nursing Services, The Investment Building-10th floor, One Investment Place, Baltimore, M D 21204.
ments of health are geared toward disease prevention and health promotion (Institute of Medicine, 1988). Thus, the conventional model is not always a perfect fit. Basic adjustments are required so that QA can be adapted to public health nursing. Since no regulations yet exist for QA in public health, the system includes only those components necessary to provide the clearest depiction of health care dispensed. As public health programs expand and change, the criteria of the QA system has undergone revisions, but the structure of the system, with its inherent checks and balances, has remained intact.
INCONGRUITIES OF THE QA PROCESS WITH PUBLIC HEALTH Generally, the following nine steps of the Joint Commission of Accreditation for Health Organizations (JCAHO, formerly JCAH) are widely accepted as the gold standard for QA systems. 1. Assign one individual to be responsible for all monitoring, tabulations, and reports. 2. Specify the scope of services for which the QA system is directed. 3. Identify the most important areas of care. 4. Delineate standards and thresholds for the most important areas of care. 5. Establish measurable criteria to serve as estimates of the standards of care. 6. Collect and tabulate data, then compare observed findings with the expected thresholds. 7. Identify problem areas, and plan and implement actions to remedy the problems. 8. Assess the effectiveness of the plan. 9. Communicate the results of the steps to all staff involved in providing patient care (Rowland & Rowland, 1987). 133
134 Public Health Nursing
Volume 9 Number 2 June 1992
Most of these steps can be incorporated into health departments without much difficulty. However, two of them do not conform easily. The first problematic step is that of delineating the scope of care. Traditionally, QA monitors care provided for patients within a specific setting, usually a hospital, where there is no question regarding the nature and presence of the health care provider. However, this may not be the case in public health departments. Local departments of health usually identify all individuals within an entire geographic area as their “patients.” Yet, several individuals in these areas may be completely unaware of a health department’s existence, let alone its services. Others might be aware of the services, but opt for private providers. Regardless of the instance, the state’s statutes judge all those within the geographic borders as being within the department’s domain of care (Institute of Medicine, 1988). The second problem is with monitoring tools. Typically, QA tools or instruments-consist of structural, process, and outcome criteria. Outcomes are particularly difficult with public health, since public health often directs its services to health promotion and disease prevention. A possible suggestion for the outcome criteria of disease prevention are decreased morbidity and mortality rates. A less likely suggestion for outcome criteria of health promotion might be increased levels of health or well-being, neither of which would be acceptable. For health-promotion services, outcomes evaluating levels of well-being are indisputably ambiguous, and therefore not easily measured. Also, morbidity and mortality rates are less than ideal outcomes, since they may or may not result from the institution’s activities, and health care outcomes or results usually are presumed to be the consequence of process criteria or specific nursing activities. Although health departments consider it their mission to help all individuals in a designated area, they direct their activities (processes) to target populations that may be subsets of the population. Thus, it is unreasonable and frustrating to expect fluctuations with an entire area’s rate of disease or death. Accordingly, departments of health must decide carefully on which outcomes would be both meaningful and reasonable.
vices, even though they may be eligible for them and are included in the target population. Often agencies incorporate standards taken from traditional hospital and ambulatory care settings, or those that are created from policy and procedure manuals. Some of these are applicable to public health, although in many cases new standards and thresholds must be generated. Of course, generic community health nursing standards are available from the American Nurses’ Association, as well as some specialty standards, such as maternal-child health (Council of Community Health Nurses, 1986; Division of Maternal and Child Health Nursing Practices, 1983). In addition, the National League for Nursing provides standards of practice as well as for use of QA in community and public health agencies; however, the standards must be operationalized and thresholds must be applied (Mitchell & Storfjell, 1989).
PUBLIC HEALTH NURSING QA SYSTEM
The QA system conceived in Baltimore County Public Health Nursing Services was designed with the assumption that it would evaluate care provided to individuals who sought care from its resources, but not to those who did not use the services, but were eligible. Second, standards were adopted from the American Nurses’ Association community health and maternal health standards (Council of Community Health Nurses, 1986; Division of Maternal and Child Health Nursing Practice, 1983). Thresholds were also chosen for each respective standard. Thresholds refer to the expected level of patient care performance (Rowland & Rowland, 1987). The performance level is calculated by dividing the number of acceptable cases of patient care (numerator) by the number of relevant cases reviewed (denominator). In Baltimore County Public Health Nursing, the thresholds were chosen based on experience and preference. They are 90% for assessment, family evaluation, and planning; 85% for follow-up; and 75% for patients outcome. Because a simple audit provides only a retrospective view, other perspectives were necessary to obtain a clear picture of care. A concurrent view of patient care was obtained through observation of the health center environment, and eliciting patients’ opinions of their FITTING QA TO PUBLIC HEALTH care and staff‘s perceptions about the care that they proSince health departments are responsible for entire vide. By combining these viewpoints, the evaluator accommunities, it becomes prudent to define the scope of quired a more comprehensive understanding of quality the QA program as encompassing only individuals who of care. Thus, the system collects data obtained from have sought their services. This alteration in scope staff, patients, peer review, and documentation, and leads to the assumption that the QA program will not employs both concurrent and retrospective approaches. address individuals who do not avail themselves of serThe information is tabulated and consolidated into a
Zlotnick: QA System 135
report, and is shared with all staff at regional meetings. In addition, to ensure that the staff understand the operations of the QA system, work sessions have been set up in which the staff review one of their own records using the QA instruments. These sessions have been useful. The staff has the opportunity to learn how the QA system works. In addition, they can provide input regarding the instruments. Through these activities, the staff can gain insight and become invested in the system. THE FIVE QA INSTRUMENTS
Since the QA system already contains tools that elicit subjective and judgmental evaluation, the QMT was designed to promote maximum impartiality. To this end, all criteria were written in an explicit manner, attempting to eliminate the need for opinions or explanations. To ensure that criteria were interpreted in the same way, two reviewers independently evaluated the same group of randomly selected patient records. In this way, they avoided influencing each others' responses. The responses of the reviewers were then compared for interrater reliability to ensure that the same criteria would lead to the same responses when used in the same records (Donabedian, 1985). To calculate the levels of interrater reliability (difference between reviewers' responses), a procedure called Kappa was used (Fleiss, 1981). Some of the criteria were practical for all nine programs; however, many programs were dissimilar, especially in areas such as assessment and patient outcome. In these cases, different criteria were designed for each program (Figs. 1 and 2).
Six instruments make up this QA system. In four of them the questions are subjective and require the reviewer's judgment. They obtain views from the staff, patients, and peers. The advantage of subjective over objective instruments is that questions are answered more fully, often providing examples and explanations. However, even though different perspectives have advantages, more objective and less biased input is also required. Thus, the remaining two instruments tend to be obEnvironmental Survey jective. One of them measures different characteristics of the workplace, and the other monitors documenta- The environmental survey was designed to provide objective responses in evaluating the workplace for safety, tion in patient records. comfort, and distractions. Generally, this type of instruQuality Monitoring Tool ment uses structural criteria, which focus on function, The tool auditing documentation in patients' records is such as equipment, surroundings, and tools for work called the quality monitoring tool (QMT). After exam- (e.g., forms, patient charts). Poor equipment, lighting, ining the nine public health programs carefully, five noise, and cleanliness can affect the quality of patient areas were specified as defining the domain of care: as- care services. This tool can include any of these areas. To measure distraction, the number of telephone calls sessment, family evaluation, planning, follow-up, and or unscheduled walk-in visits that occur during a clinic patient outcome. For each of the areas, process and/or session are tallied. Other activities might also cause disoutcome criteria were developed (Lehmann, 1989; Matraction, but this criteria was used as the surrogate mearek, 1989; Mates & Sidel, 1981). When possible, the sure. In this way, the evaluator can count the number same criteria were used.
-
A. ASSESSMENT CRITERIA
I
2 . In the notes (DHMH-210 Continuation Sheet, Ped. Flow Sheet, NS-140 to 147 DHPM-39a. DWWH-3213,NS-93 NS-94, D h - 3 7 8 5 , D k 378513788, NS-96, NS-80 NS-85, NS-6.0, NS-6.2),
a.
all visitslphone contacts in the last two pages are dated and have initials or signatures.
A. ASSESSJMENT YES
COMMENTS CRITERIA 4. Physical assessment is noted: in the record, as seen by completion of NS-94 boxes "Intermenstrual b1eeding"and "Vaginal discharge". in the Pr.n.t.l record as noted by completion of DHMH-3784 box,"Menstrual Hxn& "Previous Pregnancies". =PI--U=-_==EPPI=----P
rTE COHMENTS
Figure 1. Example of QMT criteria used for all programs. Figure 2. Example of QMT program-specific process criteria. (Baltimore County Department of Health, Public Health Nurs- (Baltimore County Department of Health, Public Health Nursing, 1990, p. Q-24.) ing, 1990, p. Q-22.)
136 Public Health Nursing
Volume 9 Number 2 June 1992
+
====-==r=n-mrrrz3==~==~=~-=~===-~-=-====*~---=-
PATI6WT VISITS
HOW by by by did you
1.
CLINIC I N T W U P T I O N S TILEPEONS CALLS
did you get to the Health Center? ) 1 . Did you wear a seat belt? car taxi bus walk 2) Was a seat belt available?
yesno-
2. Did YOU receive any information or teaching from the nurse?
1. Is the temperature of the Health Center Waiting Room between 65O-75OF according to the nearest thermostat? 2.
No Uncertain Yes
-2
Is there any evidence of leaking ceilings in the:
a) What was it?
b)
Was it helpful? Yes -
uncerEin= --------PPII------=----=====-==-==---=-=-=I=======~====
a. b. c.
examining rooms laboratory waiting rooms
=ZLP===-==EEEEE=EEP_==========P=====-===============~====
Figure 3. Selectred criteria from the environmental survey. (Baltimore County Department of Health, Public Health Nursing, 1990, pp. Q-1, Q-2.)
of interruptions that occur during different time periods and clinic sessions (Fig. 3). Staff Evaluation Survey The morale, feelings, and beliefs of the staff can strongly influence the quality of care, a point that is often forgotten in QA. Staff evaluation surveys are useful in characterizing morale, and identifying problem areas in policies and procedures. Another use for them is to obtain opinions of program changes before they are instituted (Fig. 4). Patient Input The patient input survey obtains patients' perceptions of care and their public health behaviors. By way of the survey, the health care provider can look at the needs
AGREE
DO NOT AGREE
Figure 5. Questions from the patient input survey. (Baltimore County Department of Health, Public Health Nursing, 1990, PP. Q-5, Q-6.)
of a patient population to consider future programs or changes. Health promotion is primarily achieved through patient education. By using this instrument, it is possible to determine whether patients believe that they received any helpful information; and even more important, how much of the information the patients understand (Fig. 5). Utilization/Peer Review Forum The last two sources of data for the QA system are furnished through the Utilization and Peer Review Forum Committee. This committee consists of fellow public health professionals from surrounding counties and cities. The forum's criteria seek their professional judgment and so are subjective or implicit. Therefore, whereas the objective QMT asks about extremely specific items in the patient's record, the utilization and peer review forum encourages committee members to review the entire patient record. The peer review forum is more subjective, and attempts to extract perceptions of the quality of care. The utilization review obtains information pertaining to underuse or overuse of resources (Figs. 6 and 7).
1. I feel that my work contributes to Nursing Services.
EVACUATION OF CARE
2. My boss encourages me and reinforces me positively.
2.
Were the interventions of your discipline (if you are a nurse, address nursing care; if you are a physician, address the physician's care etc) appropriate for the patient's problems? If not describe the inconsistencies.
3.
Were tho interventions of your discipline (if you are a nurse, address nursing care; if you are a physician, address the physician's care etc) appropriate for the family's problems (if no interventions were indicated, put "NA")? If not describe the inconsistencies.
3. The Nursing Services staff
work well as a team. 4. The Communicable Diseases Program standardsjprocedures are clear and easy to follow.
5. The Communicable Diseases Program Manual is a clear and useful reference. ~
===_==__===============P_n___3_P_==E_r=======
Figure 4. Excerpt from the staff evaluation survey. (Baltimore County Department of Health, Public Health Nursing, pp. Q-57, Q-58.)
=SD===PE==P=n=E=EEP-=-=-====E==~=-=====~===========
Figure 6. Questions from the peer review forum. (Baltimore County Department of Health, Public Health Nursing, 1990, p. Q-19.)
Zlotnick: QA System
~
-
tabulating a year’s data, the annual report is generated. From this report the QA committee can decide which areas emerge as problem areas as a result of several instruments’ data. A plan is subsequently prepared and implemented, and all changes that occur are monitored. If, after a year, the plan is judged ineffectual, a different strategy is devised and the QA cycle continues. After all, what is the purpose of QA if not to seek better and more efficient ways to provide the highest quality care?
-
Criteria Provide Not PIv A.
4.
Not Prv
Provide
Problem List
Doctor’s care
1
I
U
I
137
I
REFERENCES Baltimore County Department of Health, Public Health Nursing. (1991). Baltimore County Department of Health, Public Health Nursing’s quality assurance system manual.
CONCLUSION Using these six instruments, the organization obtains perspectives from staff, patients, peers, the work environment, and documents. Incorporated into these instruments are forums for subjective and expansive discussions on quality of care, and tools to elicit objective responses about health centers and documentation. Certain methods use retrospective approaches, such as those based on documentation, whereas others use concurrent techniques, eliciting information from patients, staff, and the work environment. With data from different perspectives and different approaches, the problems identified from different tools can be validated, thus corroborating the findings. Not all instruments are administered monthly. The utilization and peer review forum are performed quarterly. The environmental survey is executed biannually; but the staff evaluation survey is distributed annually. The QMT is performed monthly by the QA committee. The QA committee is composed of fellow supervisors and staff. The time expenditure for QA work, other than monthly meetings, is roughly 40 minutes per member. The data from the six instruments are tabulated, and where applicable, monthly reports are generated. After
Baltimore: Author. Council of Community Health Nurses. (1986). Standards of community health nursing practice. Kansas City, MO: American Nurses’ Association. Division of Maternal and Child Health Nursing Practice. (1983). Standards of maternal and child health nursing practice.
Kansas City, MO: American Nurses’ Association. Donabedian, A. (1985). The methods offindings ofquality assessment and monitoring. Ann Arbor Health Administration Press. Fleiss, J. L. (1981). Statistical methods for rates andproportions, 2nd ed. New York: John Wiley & Sons. Institute of Medicine. (1988). The future of public health. Washington, DC: National Academy Press. Lehmann, R. (1989). Forum on clinical indicator development: A discussion of the use and development of indicators. Quality Review Bulletin, 15(7), 223-227. Marek, K. D. (1989). Outcome measurement in nursing. Journal of Nursing Quality Assurance, 4(1), 1-9. Mates, S., & Sidel, V. W. (1981). Quality assessment by process and outcome methods: Evaluation of emergency room care of asthmatic adults. American Journal of Public Health, 71(7), 687-693. Mitchell, M. K . , & Storfjell, J. L. (Eds.). (1989). Standards of excellence for community health organizations. New York: National League for Nursing, Community Health Accreditation Program. Rowland, H. S., & Rowland, B. L. (1987). The manual of nursing quality assurance. Rockville, MD: Aspen.
A Public Health Quality Assurance System Cheryl Zlotnick, R.N., Dr. P.H.
Abstract Quality assurance (QA) systems are commonplace among hospitals. They are even found in home health and health maintenance organizations, but they are less common among public health agencies. This article enumerates the discrepancies between the design of traditional QA and that necessary to meet the needs of a department of health’s public health nursing service. I t characterizes the subsequent changes that must be inserted into the QA system for it to comply with the mission and services of a public health setting. The system and instruments presented are used in the Baltimore County Department of Health, Public Health Nursing Services.
Quality assurance is a widely accepted system that compares the care provided to institutionally held standards, evaluates data, identifies problems, plans and implements activities to alleviate the problems, and determines whether the activities achieved the desired results (Rowland & Rowland, 1987). Due to regulations, the majority of hospitals have intricate quality assurance (QA) programs. Such programs are also beginning to proliferate in health maintenance organizations and home health agencies, in large part to achieve thirdparty payer status for government programs (i.e., Medicare and Medicaid) (Rowland & Rowland, 1987). One type of health organization in which QA is less common is the local government department of health. With the increasing demands for health care regulation and ever-present litigious climate, health departments have begun to establish QA systems. Most of the existing material on QA is from facilities that provide curative health care, yet most local depart-
Cheryl Zlotnick is director of evaluation and research, Baltimore County Department of Health, Public Health Nursing Services, The Investment Building-10th floor, One Investment Place, Baltimore, M D 21204.
ments of health are geared toward disease prevention and health promotion (Institute of Medicine, 1988). Thus, the conventional model is not always a perfect fit. Basic adjustments are required so that QA can be adapted to public health nursing. Since no regulations yet exist for QA in public health, the system includes only those components necessary to provide the clearest depiction of health care dispensed. As public health programs expand and change, the criteria of the QA system has undergone revisions, but the structure of the system, with its inherent checks and balances, has remained intact.
INCONGRUITIES OF THE QA PROCESS WITH PUBLIC HEALTH Generally, the following nine steps of the Joint Commission of Accreditation for Health Organizations (JCAHO, formerly JCAH) are widely accepted as the gold standard for QA systems. 1. Assign one individual to be responsible for all monitoring, tabulations, and reports. 2. Specify the scope of services for which the QA system is directed. 3. Identify the most important areas of care. 4. Delineate standards and thresholds for the most important areas of care. 5. Establish measurable criteria to serve as estimates of the standards of care. 6. Collect and tabulate data, then compare observed findings with the expected thresholds. 7. Identify problem areas, and plan and implement actions to remedy the problems. 8. Assess the effectiveness of the plan. 9. Communicate the results of the steps to all staff involved in providing patient care (Rowland & Rowland, 1987). 133
134 Public Health Nursing
Volume 9 Number 2 June 1992
Most of these steps can be incorporated into health departments without much difficulty. However, two of them do not conform easily. The first problematic step is that of delineating the scope of care. Traditionally, QA monitors care provided for patients within a specific setting, usually a hospital, where there is no question regarding the nature and presence of the health care provider. However, this may not be the case in public health departments. Local departments of health usually identify all individuals within an entire geographic area as their “patients.” Yet, several individuals in these areas may be completely unaware of a health department’s existence, let alone its services. Others might be aware of the services, but opt for private providers. Regardless of the instance, the state’s statutes judge all those within the geographic borders as being within the department’s domain of care (Institute of Medicine, 1988). The second problem is with monitoring tools. Typically, QA tools or instruments-consist of structural, process, and outcome criteria. Outcomes are particularly difficult with public health, since public health often directs its services to health promotion and disease prevention. A possible suggestion for the outcome criteria of disease prevention are decreased morbidity and mortality rates. A less likely suggestion for outcome criteria of health promotion might be increased levels of health or well-being, neither of which would be acceptable. For health-promotion services, outcomes evaluating levels of well-being are indisputably ambiguous, and therefore not easily measured. Also, morbidity and mortality rates are less than ideal outcomes, since they may or may not result from the institution’s activities, and health care outcomes or results usually are presumed to be the consequence of process criteria or specific nursing activities. Although health departments consider it their mission to help all individuals in a designated area, they direct their activities (processes) to target populations that may be subsets of the population. Thus, it is unreasonable and frustrating to expect fluctuations with an entire area’s rate of disease or death. Accordingly, departments of health must decide carefully on which outcomes would be both meaningful and reasonable.
vices, even though they may be eligible for them and are included in the target population. Often agencies incorporate standards taken from traditional hospital and ambulatory care settings, or those that are created from policy and procedure manuals. Some of these are applicable to public health, although in many cases new standards and thresholds must be generated. Of course, generic community health nursing standards are available from the American Nurses’ Association, as well as some specialty standards, such as maternal-child health (Council of Community Health Nurses, 1986; Division of Maternal and Child Health Nursing Practices, 1983). In addition, the National League for Nursing provides standards of practice as well as for use of QA in community and public health agencies; however, the standards must be operationalized and thresholds must be applied (Mitchell & Storfjell, 1989).
PUBLIC HEALTH NURSING QA SYSTEM
The QA system conceived in Baltimore County Public Health Nursing Services was designed with the assumption that it would evaluate care provided to individuals who sought care from its resources, but not to those who did not use the services, but were eligible. Second, standards were adopted from the American Nurses’ Association community health and maternal health standards (Council of Community Health Nurses, 1986; Division of Maternal and Child Health Nursing Practice, 1983). Thresholds were also chosen for each respective standard. Thresholds refer to the expected level of patient care performance (Rowland & Rowland, 1987). The performance level is calculated by dividing the number of acceptable cases of patient care (numerator) by the number of relevant cases reviewed (denominator). In Baltimore County Public Health Nursing, the thresholds were chosen based on experience and preference. They are 90% for assessment, family evaluation, and planning; 85% for follow-up; and 75% for patients outcome. Because a simple audit provides only a retrospective view, other perspectives were necessary to obtain a clear picture of care. A concurrent view of patient care was obtained through observation of the health center environment, and eliciting patients’ opinions of their FITTING QA TO PUBLIC HEALTH care and staff‘s perceptions about the care that they proSince health departments are responsible for entire vide. By combining these viewpoints, the evaluator accommunities, it becomes prudent to define the scope of quired a more comprehensive understanding of quality the QA program as encompassing only individuals who of care. Thus, the system collects data obtained from have sought their services. This alteration in scope staff, patients, peer review, and documentation, and leads to the assumption that the QA program will not employs both concurrent and retrospective approaches. address individuals who do not avail themselves of serThe information is tabulated and consolidated into a
Zlotnick: QA System 135
report, and is shared with all staff at regional meetings. In addition, to ensure that the staff understand the operations of the QA system, work sessions have been set up in which the staff review one of their own records using the QA instruments. These sessions have been useful. The staff has the opportunity to learn how the QA system works. In addition, they can provide input regarding the instruments. Through these activities, the staff can gain insight and become invested in the system. THE FIVE QA INSTRUMENTS
Since the QA system already contains tools that elicit subjective and judgmental evaluation, the QMT was designed to promote maximum impartiality. To this end, all criteria were written in an explicit manner, attempting to eliminate the need for opinions or explanations. To ensure that criteria were interpreted in the same way, two reviewers independently evaluated the same group of randomly selected patient records. In this way, they avoided influencing each others' responses. The responses of the reviewers were then compared for interrater reliability to ensure that the same criteria would lead to the same responses when used in the same records (Donabedian, 1985). To calculate the levels of interrater reliability (difference between reviewers' responses), a procedure called Kappa was used (Fleiss, 1981). Some of the criteria were practical for all nine programs; however, many programs were dissimilar, especially in areas such as assessment and patient outcome. In these cases, different criteria were designed for each program (Figs. 1 and 2).
Six instruments make up this QA system. In four of them the questions are subjective and require the reviewer's judgment. They obtain views from the staff, patients, and peers. The advantage of subjective over objective instruments is that questions are answered more fully, often providing examples and explanations. However, even though different perspectives have advantages, more objective and less biased input is also required. Thus, the remaining two instruments tend to be obEnvironmental Survey jective. One of them measures different characteristics of the workplace, and the other monitors documenta- The environmental survey was designed to provide objective responses in evaluating the workplace for safety, tion in patient records. comfort, and distractions. Generally, this type of instruQuality Monitoring Tool ment uses structural criteria, which focus on function, The tool auditing documentation in patients' records is such as equipment, surroundings, and tools for work called the quality monitoring tool (QMT). After exam- (e.g., forms, patient charts). Poor equipment, lighting, ining the nine public health programs carefully, five noise, and cleanliness can affect the quality of patient areas were specified as defining the domain of care: as- care services. This tool can include any of these areas. To measure distraction, the number of telephone calls sessment, family evaluation, planning, follow-up, and or unscheduled walk-in visits that occur during a clinic patient outcome. For each of the areas, process and/or session are tallied. Other activities might also cause disoutcome criteria were developed (Lehmann, 1989; Matraction, but this criteria was used as the surrogate mearek, 1989; Mates & Sidel, 1981). When possible, the sure. In this way, the evaluator can count the number same criteria were used.
-
A. ASSESSMENT CRITERIA
I
2 . In the notes (DHMH-210 Continuation Sheet, Ped. Flow Sheet, NS-140 to 147 DHPM-39a. DWWH-3213,NS-93 NS-94, D h - 3 7 8 5 , D k 378513788, NS-96, NS-80 NS-85, NS-6.0, NS-6.2),
a.
all visitslphone contacts in the last two pages are dated and have initials or signatures.
A. ASSESSJMENT YES
COMMENTS CRITERIA 4. Physical assessment is noted: in the record, as seen by completion of NS-94 boxes "Intermenstrual b1eeding"and "Vaginal discharge". in the Pr.n.t.l record as noted by completion of DHMH-3784 box,"Menstrual Hxn& "Previous Pregnancies". =PI--U=-_==EPPI=----P
rTE COHMENTS
Figure 1. Example of QMT criteria used for all programs. Figure 2. Example of QMT program-specific process criteria. (Baltimore County Department of Health, Public Health Nurs- (Baltimore County Department of Health, Public Health Nursing, 1990, p. Q-24.) ing, 1990, p. Q-22.)
136 Public Health Nursing
Volume 9 Number 2 June 1992
+
====-==r=n-mrrrz3==~==~=~-=~===-~-=-====*~---=-
PATI6WT VISITS
HOW by by by did you
1.
CLINIC I N T W U P T I O N S TILEPEONS CALLS
did you get to the Health Center? ) 1 . Did you wear a seat belt? car taxi bus walk 2) Was a seat belt available?
yesno-
2. Did YOU receive any information or teaching from the nurse?
1. Is the temperature of the Health Center Waiting Room between 65O-75OF according to the nearest thermostat? 2.
No Uncertain Yes
-2
Is there any evidence of leaking ceilings in the:
a) What was it?
b)
Was it helpful? Yes -
uncerEin= --------PPII------=----=====-==-==---=-=-=I=======~====
a. b. c.
examining rooms laboratory waiting rooms
=ZLP===-==EEEEE=EEP_==========P=====-===============~====
Figure 3. Selectred criteria from the environmental survey. (Baltimore County Department of Health, Public Health Nursing, 1990, pp. Q-1, Q-2.)
of interruptions that occur during different time periods and clinic sessions (Fig. 3). Staff Evaluation Survey The morale, feelings, and beliefs of the staff can strongly influence the quality of care, a point that is often forgotten in QA. Staff evaluation surveys are useful in characterizing morale, and identifying problem areas in policies and procedures. Another use for them is to obtain opinions of program changes before they are instituted (Fig. 4). Patient Input The patient input survey obtains patients' perceptions of care and their public health behaviors. By way of the survey, the health care provider can look at the needs
AGREE
DO NOT AGREE
Figure 5. Questions from the patient input survey. (Baltimore County Department of Health, Public Health Nursing, 1990, PP. Q-5, Q-6.)
of a patient population to consider future programs or changes. Health promotion is primarily achieved through patient education. By using this instrument, it is possible to determine whether patients believe that they received any helpful information; and even more important, how much of the information the patients understand (Fig. 5). Utilization/Peer Review Forum The last two sources of data for the QA system are furnished through the Utilization and Peer Review Forum Committee. This committee consists of fellow public health professionals from surrounding counties and cities. The forum's criteria seek their professional judgment and so are subjective or implicit. Therefore, whereas the objective QMT asks about extremely specific items in the patient's record, the utilization and peer review forum encourages committee members to review the entire patient record. The peer review forum is more subjective, and attempts to extract perceptions of the quality of care. The utilization review obtains information pertaining to underuse or overuse of resources (Figs. 6 and 7).
1. I feel that my work contributes to Nursing Services.
EVACUATION OF CARE
2. My boss encourages me and reinforces me positively.
2.
Were the interventions of your discipline (if you are a nurse, address nursing care; if you are a physician, address the physician's care etc) appropriate for the patient's problems? If not describe the inconsistencies.
3.
Were tho interventions of your discipline (if you are a nurse, address nursing care; if you are a physician, address the physician's care etc) appropriate for the family's problems (if no interventions were indicated, put "NA")? If not describe the inconsistencies.
3. The Nursing Services staff
work well as a team. 4. The Communicable Diseases Program standardsjprocedures are clear and easy to follow.
5. The Communicable Diseases Program Manual is a clear and useful reference. ~
===_==__===============P_n___3_P_==E_r=======
Figure 4. Excerpt from the staff evaluation survey. (Baltimore County Department of Health, Public Health Nursing, pp. Q-57, Q-58.)
=SD===PE==P=n=E=EEP-=-=-====E==~=-=====~===========
Figure 6. Questions from the peer review forum. (Baltimore County Department of Health, Public Health Nursing, 1990, p. Q-19.)
Zlotnick: QA System
~
-
tabulating a year’s data, the annual report is generated. From this report the QA committee can decide which areas emerge as problem areas as a result of several instruments’ data. A plan is subsequently prepared and implemented, and all changes that occur are monitored. If, after a year, the plan is judged ineffectual, a different strategy is devised and the QA cycle continues. After all, what is the purpose of QA if not to seek better and more efficient ways to provide the highest quality care?
-
Criteria Provide Not PIv A.
4.
Not Prv
Provide
Problem List
Doctor’s care
1
I
U
I
137
I
REFERENCES Baltimore County Department of Health, Public Health Nursing. (1991). Baltimore County Department of Health, Public Health Nursing’s quality assurance system manual.
CONCLUSION Using these six instruments, the organization obtains perspectives from staff, patients, peers, the work environment, and documents. Incorporated into these instruments are forums for subjective and expansive discussions on quality of care, and tools to elicit objective responses about health centers and documentation. Certain methods use retrospective approaches, such as those based on documentation, whereas others use concurrent techniques, eliciting information from patients, staff, and the work environment. With data from different perspectives and different approaches, the problems identified from different tools can be validated, thus corroborating the findings. Not all instruments are administered monthly. The utilization and peer review forum are performed quarterly. The environmental survey is executed biannually; but the staff evaluation survey is distributed annually. The QMT is performed monthly by the QA committee. The QA committee is composed of fellow supervisors and staff. The time expenditure for QA work, other than monthly meetings, is roughly 40 minutes per member. The data from the six instruments are tabulated, and where applicable, monthly reports are generated. After
Baltimore: Author. Council of Community Health Nurses. (1986). Standards of community health nursing practice. Kansas City, MO: American Nurses’ Association. Division of Maternal and Child Health Nursing Practice. (1983). Standards of maternal and child health nursing practice.
Kansas City, MO: American Nurses’ Association. Donabedian, A. (1985). The methods offindings ofquality assessment and monitoring. Ann Arbor Health Administration Press. Fleiss, J. L. (1981). Statistical methods for rates andproportions, 2nd ed. New York: John Wiley & Sons. Institute of Medicine. (1988). The future of public health. Washington, DC: National Academy Press. Lehmann, R. (1989). Forum on clinical indicator development: A discussion of the use and development of indicators. Quality Review Bulletin, 15(7), 223-227. Marek, K. D. (1989). Outcome measurement in nursing. Journal of Nursing Quality Assurance, 4(1), 1-9. Mates, S., & Sidel, V. W. (1981). Quality assessment by process and outcome methods: Evaluation of emergency room care of asthmatic adults. American Journal of Public Health, 71(7), 687-693. Mitchell, M. K . , & Storfjell, J. L. (Eds.). (1989). Standards of excellence for community health organizations. New York: National League for Nursing, Community Health Accreditation Program. Rowland, H. S., & Rowland, B. L. (1987). The manual of nursing quality assurance. Rockville, MD: Aspen.
