Family Practice, 2014, Vol. 31, No. 5, 607–621 doi:10.1093/fampra/cmu021 Advance Access publication 21 May 2014

Refinement of indicators and criteria in a quality tool for assessing quality in primary care in Canada: a Delphi Panel study Cheryl A Levitt*, Kalpana Nair, Lisa Dolovich, David Price and Linda Hilts Department of Family Medicine, McMaster University, Hamilton, Canada.

Received December 3 2013; revised March 25 2014; Accepted April 14 2014.

Abstract Purpose.  Primary care is the cornerstone of the health care system and increasingly countries are developing indicators for assessing quality in primary care practices. The ‘Quality Tool’, developed in Ontario, Canada, provides a framework for assessing practices and consists of indicators and criteria. The purpose of this study was to validate the indicators and simplify the Quality Tool. Methods.  This study involved a systematic comparison of indicators in the Quality Tool with those in other local and international tools to determine common indicators to include as valid in the Quality Tool. A Delphi process was used to help reach consensus for inclusion of any indicators that were not included in the comparison exercise. Setting.  Primary care in Ontario, Canada. Subjects.  Key informants were those with known expertise and experience in quality assessment in primary care. Main outcome.  Validated set of indicators for inclusion in an updated Quality Tool. Results.  Twenty-three stakeholders participated in the Delphi panel. Forty-four indicators were included as valid after the systematic comparison of similar indicators in other assessment tools. Of the 63 indicators brought to the Delphi panel, 37 were included as valid, 15 were excluded and 11 became criteria for other included indicators. Conclusions.  The study resulted in a set of 81 validated primary care indicators. The validation of the indicators provided a strong foundation for the next version of the Quality Tool and may be used for quality assessment in primary care. Key words:  Delphi technique, family practice, general practice, primary health care, quality improvement, quality indicators.

Introduction

Internationally, many countries have adopted accreditation programs to promote quality improvement. These programs externally assess and monitor standards or indicators of quality in primary care. General practice accreditation programs have been widely implemented in Australia and New Zealand(4,5), in some countries in Europe(6–8) and are proposed in the UK (9). Although not considered an accreditation program, the Quality and Outcomes Framework in the UK was initiated in 2003 to improve patient access to care and patient health through a

Most health care is provided in family practice settings where the prevention, diagnosis, management and outcomes of care depend on the quality of care provided(1). Decision makers, funders and clinicians themselves have been challenged to provide an environment that fosters the best possible care(2). In Canada, the Quality in Family Practice Program (Quality Program) was developed in 2005, to assess the quality of care provided in family practice settings, through a voluntary accreditation program(3).

© The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: [email protected].

607

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

*Correspondence to Cheryl A Levitt, Department of Family Medicine, McMaster University, McMaster Innovation Park, 175 Longwood Rd S, Suite 201A, Hamilton L8P 0A1, Canada; E-mail: [email protected]

Family Practice, 2014, Vol. 31, No. 5

608

Including as face valid the same or similar indicators from other tools The Quality Tool had 80 indicators and over 300 criteria. Indicators represented standards of practice performance. Criteria were processes and interpretations that could be counted or measured to assess the performance of the indicator(23). For example indicator B.5.6 ‘[t]he practice ensures that only authorised people have access to prescription medications’, has an associated criterion B.5.6.6 ‘[p]rescription pads are not accessible to unauthorised persons’ and an interpretation ‘[t]here are no prescription pads left in the examining rooms where patients can access them’ (23). First, we listed and compared the Quality Tool indicators and criteria with indicators from three validated/consensus-derived tools [European Practice Assessment (EPA) (24) Quality and Outcomes Framework (QOF) (25), and CIHI Pan-Canadian Primary Care Indicators] (18). The Australia and New Zealand tools were not examined as the Quality Tool was mostly a derivative of these tools (4,5). Three members of the research team with indicator expertise reviewed the indicators for face validity. All disagreements were resolved and agreement reached on indicators that were face valid. We use the terms face validity and content validity as technical descriptions that the judgements were reasonable(26). We included an indicator in the Quality Tool as face valid if, on the face of it, it was the same or similar to an indicator in any of the other tools examined. Content validity is a judgement that the included indicators were appropriate for the intended purpose. An example of a face valid indicator in the Quality Tool is, ‘There is a system to manage patient test results and medical reports’ and the indicator in the EPA tool, ‘There is a procedure for managing patient information regarding outgoing requests (tests, referrals, and requests from third parties)’(3). Occasionally a criterion in the Quality Tool aligned with an indicator in another tool examined, in which case the Quality Tool indicator was considered face valid.

Identifying and listing unique indicators

Methods There were three main objectives to this validation study: 1. To include as face valid the indicators in the Quality Tool that were the same or similar to indicators in other validated/ consensus-derived primary care quality assessment tools, both in Canada and internationally. 2. To list unique indicators from the Quality Tool and other Canadian and international tools examined, and new indicators from emerging programs in Canada. 3. To perform a Delphi panel exercise on the unique and new indicators, and include the indicators that met our definition of consensus, as content valid, and remove the indicators that fell below this cut point.

The remaining indicators that were only found in the Quality Tool or in one of the other tools were listed as ‘unique’ indicators for the Delphi panel. For completeness, in addition to the unique indicators, we added some ‘new’ indicators for the Delphi panel consideration. These new indicators were chosen because they had recently prompted funding of special primary care programs in Canada: colorectal cancer screening (27), access to a family doctor (28) and care of older adults(29).

The Delphi method We used a Delphi method to further validate the unique and new indicators(30). We chose a Delphi method as a pragmatic group facilitation technique to help guide the participating individuals

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

pay-for-performance system(10). Systematic reviews of studies of accreditation have reported limited and inconclusive findings about the effects of accreditation(11). There have been criticisms that these quality programs are too prescriptive, do not take into account the complexity of practice, drive clinical behaviour for financial reward, promote evidence rather than clinical judgement and reduce continuity of care(12,13). There is also evidence to show that these initiatives have had a positive effect on chronic conditions(13). In each program mentioned above, assessment tools have been developed to measure the quality of performance of the practice through external assessment(14–17). One way to address these concerns is to ensure that the authoritative standards and indicators to formally assess practices are comprehensive, meaningful and foster continuity of care. The Quality program, developed a comprehensive quality assessment tool, called the Quality Tool in 2005 as a practical guide for assessment of family practices(3). At the same time in Canada, the Canadian Institute of Health Information (CIHI) undertook a collaborative consensus process with experts to develop primary health care indicators in 2006(18). The Quality Tool was based on New Zealand’s ‘Aiming for Excellence’ tool that had its roots in Australia’s Standards for General Practice(4,5). The Quality Tool was refined for the Canadian context, and attempted to address the criticisms about quality programs by developing indicators that incorporated the four principles of family medicine(19,20). In both the pilot and the field test of the Quality Program and Tool, participants found the implementation of the Quality Program and Quality Tool to be a positive, useful and valuable experience and suggested simplifying the tool(21,22). The Quality Tool indicators, while based on other established programs that had been subjected to peer-review or validation had not undergone similar study. The purpose of this project was to validate a set of indicators to be included for the next version of the Quality Tool and simplify the tool.

Refinement of indicators and criteria in a quality tool

towards concurrance(31). We used a quasi-anonymity method, in that the respondents were known to the researchers and to each other, but their survey response judgements and opinions remained anonymous(32). A  similar method was used in developing the standards for the Quality and Outcomes framework(33). In our process, we did not ask open ended questions in the survey. Instead we provided pre-existing indicators for rating and invited comments. This may have limited the available options, but enabled the participants to focus on the task and kept them engaged(34).

Identifying key informants and Research Ethics Approval Our team identified, invited and sought consent from key informants agreeing to participate. Our pre-Delphi group had clinical, administrator and patient expertise from those involved in the development and testing of the Quality Tool and were selected from a list developed by the investigator team(21,22). The preDelphi group became Delphi panelists. In addition, other family physician leaders and a pharmacist, who were academics and decision makers, with expertise in quality improvement, were selected and invited to take part in the Delphi panel. We purposefully included an over-representation of family physicians, as family physicians are by far the most representative of clinicians in family practice, work in solo, group, rural and urban locations, play a leadership role in quality improvement and own and manage family practices in most settings in Canada. Our team felt that the voice of other primary care providers was well represented in the pre-Delphi group with the addition of the pharmacist. Delphi panel participants were provided with a small honorarium for taking part in the study. Research Ethics Board Approval [08-369] was received from McMaster University. Consensus Although the literature provides few clear guidelines for measuring consensus, numerical consensus is commonly used(37). Our rating level cut-points for ‘included’, ‘excluded’ and ‘undecided’ indicators were loosely based on a published report of a Delphi process in primary care(38). For our purposes we chose to define consensus to participants as concurrence or agreement on which indicator should be included, excluded or undecided. An indicator was included as content valid if 80% or more agreed, excluded if 50% or less agreed and undecided if between 51% and 79% agreed.

Delphi process The Delphi panel was convened for ~4 months (November 2008 to March 2009). The Delphi process involved two rounds of online surveys(39) with teleconference calls following each survey to discuss findings, and one final face-to-face meeting to resolve any outstanding issues. In the first round, panelists rated and commented on all of the unique and new indicators written as they appeared in the tool from which they originated. A report was produced of the indicators that were included, excluded and undecided. In the second round, participants were encouraged to review their ratings of the undecided indicators arising from the first round and a second report produced. The results of the second round undecided indicators were brought to the face-toface meeting. At the face-to-face meeting, participants discussed each remaining undecided indicator and then voted in a paper survey either yes or no on whether to include this indicator in the revised Quality Tool. Paper surveys were collected at the meeting or electronically following the meeting. Delphi participants who could not attend the face-to-face meeting were invited to complete the survey electronically. At this meeting, in addition to Delphi panel participants, three experts in quality assessment from Germany, Australia and New Zealand contributed to the discussion about the undecided indicators. Following the each round and the face-to-face meeting, some of the undecided indicators were neither included nor excluded but recommended to become criteria in the revised Quality Tool, thereby contributing to reducing the number of indicators and simplifying the tool. Analysis of inclusion principles Participants were asked to rate each indicator, in each round of on-line surveys, against the set of four inclusion principles using a five-point Likert scale with the anchors strongly disagree to strongly agree. These inclusion principles were presented to Delphi panelists to provide context to their decision-making. After reflecting on the inclusion principles and rating them, participants were then asked to indicate whether the indicator should be included by answering yes or no to the statement, ‘This indicator should be part of the revised Quality Tool’. The yes answers were combined to form the percentage used for consensus in each round. An analysis of the ratings of each of the inclusion principles was completed to determine if the decision to include or not include the indicator was significantly influenced by the principles. Round 1 of the Delphi was chosen for analysis, as Round 2, and the face-to-face meeting included only the undecided indicators. Using an independent sample t-test, the mean scores of the four inclusion principles were compared for included (80% voted to include) and not included (all others) indicators in the first round. A P value (two-sided) of less than 0.05 was considered statistically significant. The distributions of the ratings of the inclusion principles were also visually compared to determine if the differences were really significant.

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Development of indicator inclusion principles Prior to convening the Delphi panel, a small group of key informants, called the pre-Delphi group, developed a set of inclusion principles from a comprehensive review of indicator selection criteria used by other reporting bodies and projects that could assist the Delphi panel members in rating the value of an indicator for the Quality Tool (35,36).

609

Family Practice, 2014, Vol. 31, No. 5

610

Results Including the same or similar indicators as face valid and identifying and listing unique and new indicators Forty-four of the 80 indicators in the Quality Tool were included as face valid because they were in at least one of the other tools examined. (Table 1) This resulted in 63 unique and new indicators (36 from Quality Tool and 27 from other tools and newly funded programs) that were brought to the first Delphi process.

Delphi method

Delphi Indicators Table  3 summarizes the number included, excluded, recommended to be criteria and undecided indicators from each round of the Delphi process. Overall, 37/63 indicators were included, 15 were excluded and 11 were recommended to be criteria. Tables 4, 5 and 6 list the included, recommended to be criteria and excluded indicators. These tables also include the indicator

Table 1.  Indicators included by comparing the indicators and criteria in the Quality Tool with the indicators in CIHI, EPA, and QOF prior to the Delphi process (although some criteria in the Quality Tool matched indicators in the other tools, they became indicators to be validated in the Delphi process) Indicator No. (n = 44)

(Indicator No.) Description in Quality Toola (3)

(Domain) Description in EPA (24)

1

(A.1.3) The practice encourages patient suggestions and feedback into service planning

2

(A.2.1) Information about practice services is available for patients

3

(A.2.2) The practice makes provision to ensure patients are able to access 24-hour care, 7 days a week

(QUALITY & SAFETY: Patient Perspective) The practice has a suggestion box for patients on a clearly visible place (INFORMATION: Information for patients about non-clinical issues) The practice has a practice information sheet & If the practice has an information sheet, it contains relevant information (INFRASTRUCTURE: Accessibility and availability) The practice has a clear phone message when phoning the practice out of hours, or direct connection to the deputizing service/ own GP

(Indicator No.) Description in CIHIb (18)

(Indicator Ref.) Description in QOFb (25) (PE 2 Patient surveys 1)  The practice will have undertaken an approved patient survey each year

(31) Average number of extended hours (beyond 9:00 a.m. to 5:00 p.m., Monday to Friday), provided by PHC organizations per month, by PHC organization.

(Records 3) The practice has a system for transferring and acting on information about patients seen by other doctors out of hours

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Development of indicator inclusion principles The pre-Delphi included three family physicians and three administrators, two nurses, a social worker, a dietician and a patient. Eleven completed an on-line survey about which of nine proposed principles should be included to assist the Delphi panel in its deliberations. A teleconference was held to discuss survey findings and there was consensus that four principles should be used to provide context when considering the inclusion of each indicator (Table 2).

Delphi panel member characteristics The Delphi panel consisted of 23 stakeholders, 11 pre-Delphi panel members, 11 additional family physicians and a pharmacist. All Delphi members completed each on-line survey. In total, 18/23 (78%) participants attended the first and 16/23 (70%) the second round conference calls. Eighteen (78%) attended the final face-to-face meeting and completed the final ratings on paper or sent in their final ratings electronically. Of the inter-professional group participating, a nurse missed the first round teleconference and a practice manager, a nurse and social worker missed the second round teleconference, but all four attended the faceto-face meeting. One practice manager attended each teleconference but missed the face-to-face meeting and sent her final ratings electronically. The patient expert participated in each Delphi round and the face-to-face meeting.

Refinement of indicators and criteria in a quality tool

611

Table 1.  Continued (Indicator No.) Description in Quality Toola (3)

(Domain) Description in EPA (24)

4

(A.2.4) The practice uses a system that assists the practice team to identify and provide an appropriate response to emergencies/ urgent medical conditions

(INFRASTRUCTURE: Accessibility and availability) Reception staff have been trained to recognize and respond appropriately to urgent medical matters

5

(A.2.5) The practice team ensures that patients are provided with information to enable them to make informed decisions about their care

6

(A.2.6) The practice provides educational information on health promotion and disease prevention to patients

7

(A.2.7) Initial and repeat prescribing in the absence of a visit is accurate, appropriate and timely

8

(A.3.1) Patients can easily access the practice using the telephone system

9

(A.3.2) Patients can access the practice for advice or appointments appropriately

(Indicator No.) Description in CIHIb (18)

(Indicator Ref.) Description in QOFb (25) (Education 1) There is a record of all practiceemployed clinical staff having attended training/ updating in basic life support skills in the preceding 18 months

(76) % of PHC clients/ patients, 18 years and over, who were involved in clinical decision-making regarding their health, with their regular PHC provider, over the past 12 months. (25) % of PHC clients/ patients, 18 years and over, with a chronic health condition(s), whose PHC organization provided them with resources to support self-management or self-help groups.

(INFRASTRUCTURE: Accessibility and availability) The practice has a booking system

(33) % of PHC clients/ patients, 18 years and over, who are satisfied with wait time to obtain an appointment with their regular PHC provider for an emergent but minor health problem.

(Records 9) For repeat medicines, an indication for the drug can be identified in the records (for drugs added to the repeat prescription with effect from 1 April 2004). Minimum Standard 80% (Information 7) Patients are able to access a receptionist via telephone and face to face in the practice, for at least 45 hours over 5 days, Monday to Friday, except where agreed with the primary care organization (PE 7 Patient experience of access) The percentage of patients who, in the appropriate national survey, indicate that they were able to obtain a consultation with a GP (in England) or appropriate health care professional (in Scotland, Wales and NI) within two working days (in Wales this will be within 24 hours).

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Indicator No. (n = 44)

Family Practice, 2014, Vol. 31, No. 5

612

Table 1.  Continued (Indicator No.) Description in Quality Toola (3)

10

(A.3.3) Patient visits are effective and efficient

11

(B.5.1) The practice waiting area is comfortable and sufficient to accommodate patients and their family members who wait for services (B.5.3) The practice has appropriate disinfection and sterilization facilities available for infection control and follows (B.5.7) The practice demonstrates a commitment to the Workplace Safety and Insurance Act 1997

12

13

(Domain) Description in EPA (24)

(Indicator No.) Description in CIHIb (18)

(Indicator Ref.) Description in QOFb (25)

(27) % of PHC clients/ patients, 18 years and over, with a chronic condition(s), who had sufficient time in most visits to confide their healthrelated feelings, fears and concerns to their PHC provider.

(PE 1 Length of Consultations) The length of routine booked appointments with the doctors in the practice is not less than 10 minutes. (If the practice routinely sees extras during booked surgeries, then the average booked consultation length should allow for the average number of extras seen in a surgery session. If the extras are seen at the end, then it is not necessary to make this adjustment).

(INFRASTRUCTURE: Premises) There is sufficient seating in the waiting room

(QUALITY AND SAFETY: Safety of the staff and patients) The practice has procedures for infection control (90) % of PHC providers who had a workplace related injury over the past 12 months, by type of PHC provider.

14

(B.6.1) Medical equipment and resources are appropriate, available and maintained

(QUALITY AND SAFETY: Quality policy) The practice has lists/ inventories of medical equipment and drugs

15

(C.8.1) Smoking cessation

(INFORMATION: Clinical data) The medical record contains smoking status

(13) Tobacco use (20) % of population, 12 years and over, who are current smokers.

(Management 7) The practice has systems in place to ensure regular and appropriate inspection, calibration, maintenance and replacement of equipment including: • A defined responsible person • Clear recording • Systematic pre-planned schedules • Reporting of faults (Information 5) The practice supports smokers in stopping smoking by a strategy which includes providing literature and offering appropriate therapy

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Indicator No. (n = 44)

Refinement of indicators and criteria in a quality tool

613

Table 1.  Continued (Indicator Ref.) Description in QOFb (25)

(C.8.2) Diabetes mellitus

(7) % of PHC organizations who currently have specific programs for PHC clients/ patients with specific chronic conditions.

17

(C.8.3) Hypertension

(54) % of PHC clients/ patients, 18 years and over, who had their blood pressure measured within the past 24 months.

(DM 19) The practice can produce a register of all patients aged 17 years and over with diabetes mellitus, which specifies whether the patient has Type 1 or Type 2 diabetes (BP 1) The practice can produce a register of patients with established hypertension

18

(C.8.4) Stroke or transient ischaemic attacks (TIAs)

19

(C.8.5) Secondary prevention in coronary heart disease (CHD)

20

(C.8.6) Anti-coagulation medication

21

(C.8.7) Mental Health

22

(C.8.10) Asthma care

23

(C.8.11) Chronic obstructive pulmonary disease (COPD)

24

(C.8.12) Epilepsy

(Indicator No.) Description in Quality Toola (3)

16

(Domain) Description in EPA (24)

(61) % of PHC clients/ patients, 18 years and over, with established CAD and elevated LDL-C (i.e. >2.5 mmol/l) who were offered lifestyle advice and/or lipid lowering medication.

(64) % of PHC clients/ patients, 18 years and over, with depression who were offered treatment (pharmacological and/or non-pharmacological) or referral to a mental health provider. (37) % of PHC clients/ patients, ages 6 to 55 years, with asthma who visited the emergency department in the past 12 months.

(Stroke 1) The practice can produce a register of patients with Stroke or TIA (CHD 1) The practice can produce a register of patients with coronary heart disease

(AF 3) The percentage of patients with atrial fibrillation who are currently treated with anti-coagulation drug therapy or an anti-platelet therapy (MH 8) The practice can produce a register of people with schizophrenia, bipolar disorder and other psychoses

(Asthma 1) The practice can produce a register of patients with asthma, excluding patients with asthma who have been prescribed no asthma-related drugs in the previous 12 months (COPD 1) Practice can produce a register of patients with COPD (Epilepsy 5) The practice can produce a register of patients aged 18 and over receiving drug treatment for epilepsy

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

(Indicator No.) Description in CIHIb (18)

Indicator No. (n = 44)

Family Practice, 2014, Vol. 31, No. 5

614

Table 1.  Continued

25

(C.8.13) Hypothyroidism

26

(C.8.14) Cancer

27

(C.9.1) The practice provides comprehensive care for children

28

(C.9.4) The practice provides comprehensive care for cervical screening

(50) % of women PHC clients/patients, ages18 to 69 years, who received papanicolaou smear within the past 3 years.

29

(C.9.6) The practice provides comprehensive care for breast cancer screening

30

(C.9.9) The practice provides comprehensive care for adult patients (C.9.10) The practice provides palliative end-of-life care to patients

(49) % of women PHC clients/patients, ages 50 to 69, who received mammography and clinical breast examination within the past 24 months. (12) Non-urgent routine care (e.g. well care (baby, child, woman and/or man) (12) End-of-life care

32

(D.11.1) Medical records and documents are stored or filed safely

(INFORMATION: Confidentiality) Patient medical records or other files containing patient information are not stored or left visible in places where patients could gain access to them or read them.

(Indicator Ref.) Description in QOFb (25) (Thyroid 1) The practice can produce a register of patients with hypothyroidism (Cancer 1) The practice can produce a register of all cancer patients defined as a ‘register of patients with a diagnosis of cancer excluding non-melanotic skin cancers from 1 April 2003 (CHS 1) Child development checks are offered at intervals that are consistent with national guidelines and policy (CS 7) The practice has a protocol that is in line with national guidance and practice for the management of cervical screening, which includes staff training, management of patient call/ recall, exception reporting and the regular monitoring of inadequate smear rates

(PC 3) The practice has a complete register available of all patients in need of palliative care/support irrespective of age.

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

(Indicator No.) Description in Quality Toola (3)

31

(Domain) Description in EPA (24)

(Indicator No.) Description in CIHIb (18)

Indicator No. (n = 44)

Refinement of indicators and criteria in a quality tool

615

Table 1.  Continued (Indicator No.) Description in Quality Toola (3)

(Domain) Description in EPA (24)

(Indicator No.) Description in CIHIb (18)

33

(D.11.2) There is a system to manage patient test results and medical reports

(INFORMATION: Management of external patient data) The practice has a procedure for managing external patient data

(83) % of PHC FPs/GPs/NPs who repeated tests because findings were unavailable over the past month.

34

(D.11.3) Registration of new patients and transfer of medical records is patient-friendly

35

(D.12.1) The practice provides care that is integrated with other care agencies and community services to improve individual patient care

36

(D.12.2) The practice provides services to patients and families to meet patients with complex needs (high users, regular emergency users, patients often in crisis and patients with multiple problems) (D.13.1) All members of the practice team are qualified or trained for their position (D.13.2) All members of the practice team have contracts and current job description, and management structures are in place

37

38

(Indicator Ref.) Description in QOFb (25)

(Information 4) If a patient is removed from a practice’s list, the practice provides an explanation of the reasons in writing to the patient and information on how to find a new practice, unless it is perceived that such an action would result in a violent response by the patient (Records 19) 80% of newly registered patients have had their notes summarized within 8 weeks of receipt by the practice (80) % of PHC organizations who currently have collaborative care arrangements with provider organizations beyond the health care sector (e.g. housing, justice, police, education) (10) % of PHC organizations who currently provide specialized programs for vulnerable/special needs population groups.

(PEOPLE: Personnel) The practice checks certificates when a new employee is appointed (PEOPLE: Personnel) Staff have a signed contract & Staff have a job description

(Management 10) There is a written procedures manual that includes staff employment policies including equal opportunities, bullying and harassment and sickness absence (including illegal drugs, alcohol and stress), to which staff have access

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Indicator No. (n = 44)

Family Practice, 2014, Vol. 31, No. 5

616

Table 1.  Continued (Indicator No.) Description in Quality Toola (3)

39

(E.15.1) The practice promotes continuous quality improvement (CQI)

40

(E.15.2) The practice promotes continuing professional development (CPD)

41

(E.15.3) The practice has a morbidity and mortality management system to address serious or potentially serious practice problems (adverse incidents, near misses, etc.) (E.15.4) A range of educational resources and materials are available for reference purposes to members of the practice (E.15.5) The practice is aware of the contractual obligations and makes every effort to avoid financial mishaps (E.16.1) The practice promotes a healthy balance of work and home life

42

43

44

(Domain) Description in EPA (24)

(PEOPLE: Education and training) Staff has been on training course related to their work in the past 12 months

(INFORMATION: Information for staff) Every GP has access to the internet & Every GP has access to e-mail (FINANCE: Financial leadership and responsibilities) Responsibilities for financial management are clearly defined (PEOPLE: Perspective of GPs on working conditions) GPs experience a positive work satisfaction

(Indicator No.) Description in CIHIb (18) (69) % of PHC organizations who implemented at least one or more changes in clinical practice as a result of quality improvement initiatives over the past 12 months. (72) % of PHC providers and support staff whose PHC organization provided them with support to participate in continuing professional development within the past 12 months, by type of PHC provider and support staff. (67) % of PHC providers whose PHC organization has processes and structures in place to support a non-punitive approach to medication incident reduction.

(Indicator Ref.) Description in QOFb (25)

(Education 10) The practice has undertaken a minimum of three significant event reviews within the last year

(92) % of PHC providers who were satisfied with the overall quality of work life balance over the past 12 months, by type of PHC provider.

CAD, Coronary Artery Disease; COPD, Chronic Obstructive Pulmonary Disease; GP, General Practitioner; NI, National Insurance; NPs, Nurse Practitioners; PHC, Primary Health Care; TIA, Transient Ischemic Attack. a For more details about the indicators and criteria see Quality Tool (23). b Most appropriate indicator was used in this table, other indicators may also apply.

Table 2.  Inclusion principles and definitions for contextual rating of indicators in Round 1and Round 2 of the Delphi process Inclusion principle

Definition

Value added Measurable

The indicator is value added: it reflects an area of assessment that is not covered by any other process. The indicator is measurable at the patient, practice or population level and changes in the indicator can be clearly identified and compared over time. The indicator’s criteria would be considered a standard for family practice, including what is formally required by law. The indicator reflects an important or emerging issue that impacts on primary health care or primary health care delivery and provides information that can be used to inform policy decisions or change the behaviour of health service providers.

Standard Important

Each assessed using a 5-point Likert scale: strongly disagree = 1; disagree = 2; neutral = 3; agree = 4; strongly agree = 5.

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Indicator No. (n = 44)

617

Refinement of indicators and criteria in a quality tool

Table 3.  Summary of included/excluded/criteria/undecided indicators in Delphi process Delphi Round

n

Included

Excluded

Criteria

Undecided

Round 1 Round 2 Face-to-face meeting Total

63 28 17 63

30

5 4 6 15

7 4 11

28 17 0 0

7 37

Table 4.  Delphi Panel’s included indicator, description, indicator origin, number and percentage ratings Yes, and round or face-to-face meeting in which they were held Description

Indicator origin

n/N (%) Yes

Round or face to face included

1

The practice demonstrates its commitment to respecting the needs and rights of its practice population Fire risk is minimized by demonstrating a commitment to relevant legislation and Codes of Practice relating to fire safety, disasters or other emergencies There is an audit of medication reviews Screening for TB status, and immunization status for measles, rubella, polio, influenza, tetanus, diphtheria and pneumococcus is reviewed on a regular basis, according to CPSO Infection Control Guidelines section 1.4 The practice can produce a register of patients with heart failure. Patients who have a current diagnosis of heart failure due to LVD are treated with ACE inhibitors or ARBs if there is no contraindication. Patients with prescription or illicit drug use problems are offered, provided, or directed to treatment by their PHC provider The practice provides comprehensive care for older adults Patients on the CKD register have had their blood pressure recorded in the previous 15 months. Patients on the CKD register with hypertension and proteinuria are treated with an angiotensin converting enzyme inhibitor (ACE-I) or angiotensin receptor blocker (ARB) (unless a contraindication or side effects are recorded) Mandatory reporting occurs in accordance with legislation in the family practice The practice has an effective system to monitor waiting times for investigations and referrals to ensure that patients are receiving them in a timely manner The practice continuously improves team functioning Patients with alcohol problems have received specific help or information to manage their alcohol consumption The practice can produce a register of patients aged 18 years and over with CKD Examination areas for assessment and management ensure patient comfort The practice has a protocol/policy and procedures in case of an accidental needle-stab or other type of exposure The practice has an effective system to identify and record adolescent immunizations Patients with dementia have had their care reviewed in the last 15 months. Colorectal cancer screening for patients 50–74 years and over has taken place via FOBT in the last 2 years or completed colonoscopy in the last 5 years. The practice premises are clearly signposted and physically accessible Biomedical waste (includes anatomical waste, blood, non-anatomical waste, and other waste) is safety disposed of in accordance with local regulations

Quality Tool

19/23 (82.6)

Round 1

Quality Tool

19/23 (82.6)

Round 1

Quality Tool Quality Tool

19/23 (82.6) 19/23 (82.6)

Round 1 Round 1

QOF QOF

19/23 (82.6) 19/23 (82.6)

Round 1 Round 1

CIHI

19/23 (82.6)

Round 1

New indicator QOF

19/23 (82.6) 19/23 (82.6)

Round 1 Round 1

QOF

19/23 (82.6)

Round 1

Quality Tool

20/23 (86.9)

Round 1

Quality Tool

19/23 (82.6)

Round 1

Quality Tool CIHI

20/23 (86.96) 20/23 (86.96)

Round 1 Round 1

QOF

20/23 (87.0)

Round 1

Quality Tool Quality Tool

20/23 (87.0) 21/23 (91.3)

Round 1 Round 1

Quality Tool

21/23 (91.3)

Round 1

QOF New indicator

21/23 (91.3) 21/23 (91.3)

Round 1 Round 1

Quality Tool Quality Tool

22/23 (95.7) 22/23 (95.7)

Round 1 Round 1

2 3 4

5 6 7 8 9 10

11 12

13 14 15 16 17 18 19 20

21 22

Downloaded from http://fampra.oxfordjournals.org/ at Michigan State University on July 2, 2015

Indicator (n = 37)

Family Practice, 2014, Vol. 31, No. 5

618

Table 4.  Continued Description

Indicator origin

n/N (%) Yes

Round or face to face included

23

Office procedures are only performed after suitable training and in accordance with accepted guidelines The practice uses evidence-based clinical practice guidelines to ensure consistent high quality health care The practice has a policy for how it provides maternity services to its patients Continuity of care is promoted by the practice Screened adults with positive stool for occult blood are notified of results and referral for colonoscopy within 2 weeks of the test results. The practice maintains the privacy of patient information in accordance with Bill 31 Examination areas for assessment and management ensure patient safety The practice has appropriate vaccine storage and maintains the Cold Chain in line with provincial guidelines The practice has a shared care model in psychiatry, obstetrics and chronic disease management that promotes continuity of care The practice honours its commitment to recognizing the diversity of its patients The practice has a system to keep track of and manage patients that are hospitalized, in rehabilitation, and following discharge Patients are seen by the third next available appointment by provider The practice ensures that only authorized people have access to prescription medication kept within the practice The practice has a policy for prevention, investigation, management, and referral for sexually transmitted diseases The practice respects patients’ rights to formally complain

Quality Tool

22/23 (95.7)

Round 1

Quality Tool

22/23 (95.7)

Round 1

Quality Tool Quality Tool New indicator

22/23 (95.7) 22/23 (95.7) 22/23 (95.7)

Round 1 Round 1 Round 1

Quality Tool

23/23 (100)

Round 1

Quality Tool Quality Tool

23/23 (100) 23/23 (100)

Round 1 Round 1

Quality Tool

15/18 (83.3)

Face to face

Quality Tool

15/18 (83.3)

Face to face

Quality Tool

16/18 (88.9)

Face to face

New indicator Quality Tool

15/18 (83.3) 17/18 (94.4)

Face to face Face to face

Quality Tool

15/18 (83.3)

Face to face

Quality Tool

16/18 (88.9)

Face to face

24 25 26 27 28 29 30 31 32 33 34 35 36 37

ACE, angiotensin-converting enzyme; ARBs, angiotensin receptor blockers; CKD, chronic kidney disease; CPSO, College of Physicians and Surgeons of Ontario; FOBT, Fecal Occult Blood Test; LVD, Left Ventricular Dilatation; n, number responding; N, of total group; PHC, Primary Health Care.

origin, and the round in which inclusion/exclusion occurred. Of the final 37/63 unique and new indicators included, 25 of 36 indicators originated from the Quality Tool, 0 of 1 from EPA, 1 of 7 from CIHI, 6 of 12 from QOF and 4 of 6 from new indicators. An example of an indicator becoming a criterion is the indicator from the QOF(10), ‘[Percentage of] Patients on the diabetes register and/or the CHD register for whom case finding for depression has been undertaken on one occasion during the previous 15 months using two standard screening questions’ became a criterion under the indicator ‘Diabetes Mellitus, …the percentage of patients with diabetes mellitus who in the past 15 months have a record of …depression screening’ (40).

Contribution of Inclusion Principles to Overall Inclusion Mean ratings for indicators that were included compared to indicators that were not included in the first round were statistically significantly higher for the measurable (3.95, 0.29 versus 3.71, 0.29; P 

Refinement of indicators and criteria in a quality tool for assessing quality in primary care in Canada: a Delphi panel study.

Primary care is the cornerstone of the health care system and increasingly countries are developing indicators for assessing quality in primary care p...
502KB Sizes 2 Downloads 3 Views