ORIGINAL RESEARCH: CLINICAL TRIAL

Is it possible to strengthen psychiatric nursing staff’s clinical supervision? RCT of a meta-supervision intervention Henrik Gonge & Niels Buus Accepted for publication 19 September 2014

Correspondence to H. Gonge: e-mail: [email protected] Henrik Gonge MSc PhD Psychologist Mental Health Services in the Region of Southern Denmark, Odense, Denmark Niels Buus MScN PhD RN Associate Professor Health, Man and Society Institute of Public Health, University of Southern Denmark, Odense, Denmark

G O N G E H . & B U U S N . ( 2 0 1 5 ) Is it possible to strengthen psychiatric nursing staff’s clinical supervision? RCT of a meta-supervision intervention. Journal of Advanced Nursing 71(4), 909–921. doi: 10.1111/jan.12569

Abstract Aim. To test the effects of a meta-supervision intervention in terms of participation, effectiveness and benefits of clinical supervision of psychiatric nursing staff. Background. Clinical supervision is regarded as a central component in developing mental health nursing practices, but the evidence supporting positive outcomes of clinical supervision in psychiatric nursing is not convincing. Design. The study was designed as a randomized controlled trial. All permanently employed nursing staff members at three general psychiatric wards at a Danish university hospital (n = 83) were allocated to either an intervention group (n = 40) receiving the meta-supervision in addition to attending usual supervision or to a control group (n = 43) attending usual supervision. Methods. Self-reported questionnaire measures of clinical supervision effectiveness and benefits were collected at base line in January 2012 and at follow-up completed in February 2013. In addition, a prospective registration of clinical supervision participation was carried out over 3 months subsequent to the intervention. Results. The main result was that it was possible to motivate staff in the intervention group to participate significantly more frequently in sessions of the ongoing supervision compared with the control group. However, more frequent participation was not reflected in the experienced effectiveness of the clinical supervision or in the general formative or restorative benefits. Conclusions. The intervention had a positive effect on individuals or wards already actively engaged in clinical supervision, which suggested that individuals and wards without well-established supervision practices may require more comprehensive interventions targeting individual and organizational barriers to clinical supervision. Keywords: clinical supervision, mental health, professional development, psychiatric nursing, randomized controlled trials

© 2014 John Wiley & Sons Ltd

909

H. Gonge and N. Buus

Why is this research needed? ● Clinical supervision is a common practice widely regarded as having normative, formative and/or restorative effects on nursing staffs. ● There is a paucity of studies documenting measurable effects of clinical supervision. ● To date only few studies of clinical supervision in mental health nursing have applied rigid research designs and methods.

What are the key findings? ● This RCT documents that it is possible to motivate nursing staff to participate significantly more frequently in clinical supervision through reflection on their supervision practices. ● The RCT did not document that more frequent participation increased experienced effectiveness of clinical supervision or the general formative or restorative benefits.

How should the findings be used to influence practice/ education? ● Attention should be devoted to help staff overcome individual and organizational barriers to clinical supervision participation. ● Recognizing differences in organizational and individual attitudes towards clinical supervision is vital to successful organization of supervision.

Introduction Clinical supervision can be defined as a formalized regular activity, where nursing staff members reflect on their clinical practice under the guidance of a trained supervisor aiming at developing quality care (Cassedy 2010, Cleary et al. 2010). There are several acclaimed benefits of clinical supervision, such as development of professional skills and competence, prevention of stress, burnout or reduced sick leave and improved job satisfaction (Wright 2012). However, these accounts remain to a large extent unsubstantiated (White & Winstanley 2010b) and anecdotal (Sloan & Grant 2012). The most recent systematic review of empirical studies of supervision in mental health nursing concluded that the applied research designs and methods were suffering from several weaknesses and that the evidence of measurable effects of clinical supervision was inconclusive (Buus & Gonge 2009), which in the wider context of allied health professionals are conclusions confirmed in a recent review (Dawson et al. 2013).

910

Background Supervision interventions Details about trial interventions, such as aim, methods, duration and frequency are important for readers to evaluate and possibly replicate the intervention. Previous intervention studies on clinical supervision in mental health settings vary with regard to types of supervision. Moreover, they vary as to whether they are implemented as standardized supervision interventions on clinical nursing staff or standardized educational interventions on the supervisors’ practices (Buus et al. 2013). In the 1990s, a group of Swedish researchers carried out three projects in different psychiatric settings investigating various effects of 12 months regular (2 hours every 2nd or 3rd week) psychodynamically oriented clinical supervision in groups with a particular focus on interpersonal issues (Hallberg & Norberg 1993, Berg et al. 1994, Hallberg 1994, Hallberg et al. 1994, Edberg et al. 1996, 1999, Berg & Hallberg 1999). All three projects applied pre-and postintervention data collection, but only one of the projects had a control group. Bradshaw et al. (2007) implemented workplace cognitive behavioural clinical supervision (1½ hours every 2nd week in groups of two) for 9 months to nurses attending a psychosocial intervention training programme. A quasi-experimental design was applied to test the effect of the intervention by collecting data from students before and after the training programme and by using students from the previous year as a historical control group. White and Winstanley (2009, 2010a,b) implemented an intervention by educating nurses to provide clinical supervision for other nurses. The intervention consisted of a manualised 4-day training course in clinical supervision for the supervisor nurses. During the subsequent 1-year intervention period, the supervisor nurses offered supervision to nursing staff (supervisees) in the participating mental health facilities. The researchers assessed the effect of the training by having supervisor and supervisee nurses providing data before, during and after the intervention period. The study was designed as a randomized controlled trial (RCT) as the participating sites were randomized into nine sites receiving the intervention and seven sites practicing supervision as usual. The Swedish studies and the study by Bradshaw et al. (2007) identified various effects of clinical supervision, but suffered from methodological limitations eroding the reliability of the findings (Buus & Gonge 2009). White and Winstanley’s study applied a more rigid design, but was

© 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL

modest in concluding evidence of effect from the intervention (White & Winstanley 2010b). A model for measuring supervision A RCT is the golden standard for intervention studies. The decision about how to measure effect is an integral part of designing an RCT. Gonge and Buus (2011) introduced a model guiding levels of measurement in research on clinical supervision. The model suggested three hierarchical levels of measurement: First, actual participation in clinical supervision must be assessed as the basis of possible benefits. This is because effectiveness and/or benefits cannot be expected if there is limited or no participation. Second, it should be investigated whether the participation elicited an experience of effectiveness, as participation may not always lead to an experience of clinical supervision being effective. Third, experienced effectiveness may result in various individual or organizational benefits such as improved aspects of individual (professional and personal) or organizational functioning. A limit to the model is that it does not include the ultimate level of measurement, namely, the outcome clinical supervision may have on the quality of care and on patients’ well-being. This level of impact is one that tantalizingly remains out of reach posing tremendous challenges to the assessment (Butterworth et al. 2008), although attempts have been made in previous studies (Edberg et al. 1999, Bradshaw et al. 2007, White & Winstanley 2010b). Participation is assessed by the number of sessions each individual supervisee has attended. Previous intervention studies tended to be very explicit about how many supervision sessions specific interventions included. However, the actual level of participation in these sessions has been expressed in more general terms, which makes it difficult to interpret the outcome of the interventions tested in these studies. Assessment of effectiveness reflects how an individual experiences the process of participating in clinical supervision. A lack of validated psychometric instruments for these measurements has been a major challenge for research in this field. The development (Winstanley 2000) and refinement (Winstanley & White 2011) of the empirically derived Manchester Clinical Supervision Scale (MCSS) advanced the field and it is the most internationally applied measure of clinical supervision effectiveness to date. Benefits are the consequences of clinical supervision effectiveness and a plethora of benefits has been suggested in clinical supervision research (Buus & Gonge 2009). Likewise, several models attempt to describe benefits in general categories, however, Proctor’s model categorizing © 2014 John Wiley & Sons Ltd

A RCT of a meta-supervision intervention

benefits as normative, formative and/or restorative (Proctor 1987) has widely gained recognition in the literature (Hawkins & Shohet 2012) e.g. with a review study supporting the structure of benefits (Brunero & Stein-Parbury 2008). Researchers have primarily operationalized the formative and restorative categories by selecting individual measures of stress, burn out (Berg et al. 1994, Hallberg 1994, Hyrk€ as 2005, Edwards et al. 2006), job satisfaction (Hallberg 1994, Hallberg et al. 1994) (restorative) and coping (Berg & Hallberg 1999) and professional growth and autonomy (Hallberg et al. 1994) (formative). Less attention has been devoted to aspects of normative organizational control.

The study The purpose of this study was to test a manualized supervision intervention. The intervention was focused on strengthening supervisees’ active engagement in developing their own supervision practices and not, as in previous intervention studies, emphasizing the provision of clinical supervision per se. Staff members were encouraged to systematically reflect on how they could overcome the individual and organizational constraints they experienced, as limiting the effectiveness of their own clinical supervision (see further details in (Buus et al. 2013)). The intervention was tested in a longitudinal RCT adhering to the abovementioned model measuring participation, effectiveness and benefits with validated instruments.

Aims The aims of the study were to test whether there would be a significant difference between the intervention group and the control group in terms of:

• •

The meta-supervision intervention will increase participation in clinical supervision. The meta-supervision intervention will lead to increased clinical supervision effectiveness and increased benefits.

Design The study was conducted as a randomized controlled trial. Two sources of data were included in the study: (a) Prospective registration of participation in clinical super-

vision subsequent to the intervention. (b) Self-report questionnaires at baseline and at follow-up

after 1 year. 911

H. Gonge and N. Buus

91 nursing staff at three psychiatric hospital wards assigned for eligibility Excluded: 8 staff excluded due to irrelevant job functions Population: 83 permanently employed nursing staff at three psychiatric hospital wards

Group randomization (n = 83): Intervention group n = 40, Control group n = 43

Sample for analyses of participation (3 months prospective registration)

Allocated to intervention n = 40 Left job before intervention n = 2

Allocated to control n = 43 Left job before intervention n = 7

Allocated to intervention n = 40 Non-respondents at baseline n = 9

Allocated to control n = 43 Non-respondents at baseline n = 10

Lost at follow-up n=3

Lost at follow-up n = 10

Attended intervention n = 23 Did not attend intervention n = 5

Attended intervention n = 25 Did not attend intervention n = 13

Analyzed n = 25

Sample for analyses of effectiveness and benefits (questionnaires at baseline and follow-up)

Analyzed n = 36

Analyzed n = 23

Analyzed n = 23

Figure 1 Flow chart indicating how samples were derived from the total population. Sample The sample consisted of psychiatric nursing staff members at a Danish university hospital serving the purpose of testing the intervention. All staff members (n = 83) on three psychiatric wards (A, B and C) holding permanent positions were invited to participate in the study (eight staff were ineligible due to irrelevant job functions and therefore not invited). The study was carried out parallel to a larger staff training programme on Motivational Interviewing defining the sample size of this study. Randomization The staff training programme was organized round three training groups with staff from each of the three 912

participating wards assigned for the three groups consecutively in alphabetical order of first names. Part of the training took place in subgroups with the implication that each of the three training groups was divided into two subgroups – in total six subgroups. By blinded randomization (flip of a coin) at group-level conducted by an administrative staff with no connection to the study, one subgroup from each training group was assigned for the meta-supervision intervention while the other was assigned as the control group (group randomization). The hospital provided basic demographical data on all staff members, which allowed statistical analyses of bias in the randomization of intervention and control groups and no significant differences were found in respect of gender, education, work shift, ward and age. © 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL

A RCT of a meta-supervision intervention

new experiences (Hawkins & Shohet 2012). Depending on the participants’ evaluations, these reflections would lead to a corroboration of the personal strategies, an adaption of the strategies and/or to the formulation of additional strategies. The intervention was implemented by the first author who is a psychologist and approved specialist in psychotherapy with extensive experience in providing clinical supervision to psychiatric nursing staff. We decided that participation in the major introductory seminar qualified as adhering to the manual (25 staff members, 65
8%) and we did not request participants to have attended one (11 staff members) or both (seven staff members) of the two shorter follow-up sessions (see Figure 1). Comparing the intervention group’s 25 participants’ and the 13 non-participants’ demographical characteristics (gender, education and age) and work shift did not indicate bias while there was significant difference in participation between the wards indicating bias (v2, 2 d.f., P = 0
002). Participation rates of staff from each of the three wards were A: 14
3%, B: 88
2% and C: 64
3%.

Following the randomization, 40 staff members were enrolled in our intervention programme with the remaining 43 staff members constituting the control group (receiving training in Motivational Interviewing only). Staff from both intervention and control groups were offered the usual clinical supervision at their respective wards. Nine staff members were lost due to maternity leave and external educational programmes, which reduced the sample to 74 staff members. Thus, 38 and 36 participants stayed in the intervention and control groups (Figure 1).

Intervention The intervention was designed to strengthen nursing staff members’ clinical supervision practices through educational preparation for supervision and through systematic reflection on their already existing supervision. This ‘meta-supervision’ intervention was designed as a supplement to usual supervision. Supervisees were encouraged to reflect on personal and organizational barriers to their supervision and to develop and implement strategies to overcome these barriers. The intervention was pilot tested and manualized (Buus et al. 2013). The manual described an intervention with three sessions: An introductory session lasting 3 hours and, with circa 6–8 weeks interval, two follow-up sessions lasting 1¼ hours each. The introductory session started with the supervisor setting the agenda and introducing the purpose of the intervention. Part 1 was a teaching session defining core characteristics and basic functions of clinical supervision. In part 2, the participants described how much supervision they had participated in and the type and structure of the present supervision on the ward. In parts 3 and 4, the participants explored experienced benefits and barriers to the potential they saw in supervision. Part 5 had a problem-solving focus and participants were asked to formulate strategies to overcome barriers and strengthen the potential benefits of their supervision. The follow-up sessions were based on action research principles aimed at developing ‘second loop learning’ by systematically re-exploring earlier insight in the context of

Data collection Prospective registration of clinical supervision participation The registration of participation in clinical supervision was administered in close cooperation with the usual supervisors. Initially, each supervisor submitted a plan for clinical supervision sessions. In advance of each session the supervisor received a registration form with blank lines for names, which was filled in by the participating nursing staff and returned by the supervisor immediately after each session. Participation registration took place over 3 months (November 2012–February 2013) subsequent to the intervention. To our knowledge, registration of participants in the clinical supervision was complete. In total, 10 sessions were conducted (at Christmas supervision was suspended for 2 weeks). Presence at the 10 sessions adds up to 77 cases of participation in total from which the 44 cases could be referred to participants of this study. The number of sessions attended by the participants is indicated in

Table 1 Number of clinical supervision sessions each participant attended over three months. Ward A (n = 12)

Ward B (n = 24)

Ward C (n = 25)

Total (n = 61)

Sessions*

Frequency

%

Frequency

%

Frequency

%

Frequency

%

0 1 2 3

7 4 1 –

58
3 33
3 8
3 –

4 10 5 5

16
7 41
7 20
8 20
8

22 3 – –

88
0 12
0 – –

33 17 6 5

54
1 27
9 9
8 8
2

*Sessions conducted: A:2 sessions, B:7 sessions, C:1 session. © 2014 John Wiley & Sons Ltd

913

H. Gonge and N. Buus

Table 1 stating number of sessions attended by the staff in total and separately for each ward. In addition to the actual clinical supervision attendance, we calculated the participation rate for each participant by dividing the number of sessions each participant had attended with the number of sessions that had been available in his or her ward during the 3 months (Table 2). In a few cases, the number of sessions was adjusted to reflect that some clinical supervision sessions were only offered to subgroups in a ward, for instance, at ward B night shift staff were offered special supervision at night time. The participation rate was intended to provide a more reliable measure for participation than the actual number of sessions attended by overcoming the fact that there was considerable variation in the number of clinical supervision sessions offered in the participating wards (Ward A: 2 sessions, Ward B: 7 sessions and Ward C: 1 session). In Danish psychiatric settings, clinical supervision of nursing staff is mostly conducted by psychologists and psychiatrists. Five clinical psychologists conducted the usual supervision; one of the supervisors was internal to the ward, two were internal to the hospital and two were external to the hospital. The hospital did not give the supervisors instructions on how to supervise, which meant that they could define their own supervisory methods and practices. Facilitation of reflective processes was described as the main component of their supervision, at times supplemented by teaching. All sessions were conducted in groups and sessions lasted on average 100 minutes (range 60–180 minutes). Questionnaire Procedure. The second author conducted the collection of questionnaires. He visited each ward and introduced the study orally and in writing and encouraged staff to

participate. The staff were asked to answer the baseline questionnaire prior to the intervention (January 2012) and at follow-up 1 year later (January 2013). After 1 month, a second copy of the questionnaire was sent to non-respondents both at baseline and at follow-up. There were 64 respondents at baseline (response rate: 77
1%), which was reduced to 51 respondents at follow-up (response rate: 61
4%) (Figure 1). The reduction was mostly because nine staff members left their position during the study period, two were on long-term sick leave and two did not respond. Finally, in the statistical analyses, the sample was further reduced to 46 participants (55
4%), as five of the respondents did not attend the intervention they were assigned to. The 46 participants were evenly distributed as both groups included 23 participants. At baseline the 64 respondents and the 19 non-respondents did not differ significantly in respect of gender (v2, 1 d.f., P = 0
05), education (v2, 2 d.f., P = 0
78), work shift (v2, 2 d.f., P = 0
06), ward (v2, 2 d.f., P = 0
06) or age (P = 0
98). However, the analyses warranted suspicion of bias because of the higher response rates from Ward B than wards A and C, because women responded less than men and because day shift workers responded less than the rest of the population. Content. The questionnaire generated self-reported data covering clinical supervision effectiveness and benefits of clinical supervision (Table 3). Supervision effectiveness. A Danish translation of the Manchester Clinical Supervision Scale-26 including six subscales (Winstanley 2000, Winstanley & White 2011, Buus & Gonge 2013) (MCSS) was applied to measure the effectiveness of the clinical supervision.

Table 2 Rate of available clinical supervision sessions each participant attended over 3 months. Ward A (n = 12)

Ward B (n = 24)

Ward C (n = 25)

Total (n = 61)

Participation rate

Frequency

%

Frequency

%

Frequency

%

Frequency

%

0 0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 1
0

7 – – – – 4 – – – – 1

58
3

4 – 7 – 4 5 3 – – – 1

16
7

22 – – – – – – – – – 3

88
0

33 – 7 – 4 9 3 – – – 5

54
1

914

33
3

8
2

29
2 16
7 20
8 12
5

4
2

12
0

11
5 6
6 14
8 4
9

8
2

© 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL

A RCT of a meta-supervision intervention

Benefits. In line with previous studies, we applied a range of measures intended to cover ‘restorative’ benefits; General Health, Vitality, Mental health from SF-36 (Bjørner et al. 1997), Maslach Burnout Inventory (MBI) (Maslach & Jackson 1986, Kristensen & Borritz 1998) and General, Somatic and Cognitive stress measured by The Copenhagen Psychosocial Questionnaire (COPSOQ) (Kristensen et al. 2005) and ‘formative’ benefits; Job satisfaction, Social support from COPSOQ and the Coping Styles Questionnaire (CSQ) (Roger et al. 1993, Elklit 1996). Validity In general, the scales presented satisfactory and consistent Cronbach’s alpha coefficients at baseline and follow-up

(Table 3) except for: MCSS Importance/value of CS (a = 0
72 & 0
54), MBI Depersonalization (a = 0
66 & 0
58) COPSOQ Social support (a = 0
54 & 0
69) and CSQ Rational coping (a = 0
69 & 0
82) and Detached coping (a = 0
69 & 0
54). The few above-mentioned scales with coefficients below 0
70 at one or both measurements, indicate questionable internal consistency.

Ethical considerations All the staff invited to participate in the study received information about the study’s purpose and procedures verbally and in writing before deciding whether to participate. Participants provided their responses voluntarily and were not associated with the researchers in any way.

Table 3 Base line measures of clinical supervision effectiveness (Manchester Clinical Supervision Scale) and benefits measured by scales from SF-36, Maslach Burnout Inventory (MBI), Coping Styles Questionnaire (CSQ) and Copenhagen Psychosocial Questionnaire (COPSOQ): Mean, standard deviation, range and Cronbach’s a at baseline and follow-up.

Clinical supervision effectiveness MCSS* Trust/rapport Supervisor advice/support Improved care/skills Importance/value of CS Finding time Reflection MCSS total Benefits SF-36 General health† Vitality† Mental health† MBI Emotional exhaustion Depersonalization Personal accomplishment CSQ Rational coping Emotional coping Detached coping Avoidance coping COPSOQ Social Support Job satisfaction Stress Somatic stress Cognitive stress

Items

Response categories

n

5 5 4 5 4 3 26

5 5 5 5 5 5 5

1 4 5

Cronbach’s a

Mean

Standard deviation

Range

Baseline

Follow-up

46 46 46 46 46 46 46

19
4 18
6 15
5 21
0 11
6 12
4 98
4

2
8 2
9 2
4 3
4 3
3 1
9 11
2

12–24 9–25 8–20 9–25 4–20 6–15 64–122

0
76 0
80 0
76 0
72 0
77 0
84 0
87

0
83 0
81 0
79 0
54 0
77 0
73 0
89

5 6 6

46 45 45

77
1 67
1 82
7

18
9 20
6 11
2

25–100 20–100 52–100

– 0
84 0
75

– 0
84 0
75

9 5 8

7 7 7

46 46 46

9
5 3
8 39
1

7
2 3
4 7
4

1–30 0–15 11–47

0
86 0
66 0
74

0
88 0
58 0
81

11 10 6 10

4 4 4 4

44 45 43 45

30
3 14
9 13
4 19
0

3
7 3
6 3
0 3
6

24–40 10–27 7–21 13–29

0
69 0
86 0
69 0
70

0
82 0
83 0
54 0
71

3 4 4 4 4

5 4 5 5 5

46 46 45 45 45

6
7 7
9 15
7 17
2 16
8

1
4 1
8 3
2 3
0 3
1

4–10 4–12 4–20 4–20 4–20

0
54 0
70 0
87 0
78 0
87

0
69 0
81 0
90 0
82 0
85

*Responses to the 26 items are indicated in five categories ranging from strongly disagree to strongly agree. † Aggregate scores were converted into a score ranging from 0–100. © 2014 John Wiley & Sons Ltd

915

H. Gonge and N. Buus

Table 4 Effect of the intervention in 3 months prospective registration of clinical supervision participation measured by sessions attended and participation rate for the intervention group compared with the control group (means, medians and Mann–Whitney U-test probabilities)

Sessions attended Intervention (n = 25)† Control (n = 36)‡ Participation rate Intervention (n = 25)† Control (n = 36)‡

Total (n = 61)

Ward A (n = 12) Mean (Median)

Ward B (n = 24) Mean (Median)

Ward C (n = 25) Mean (Median)

Mean (Median)

P

0 (0) 0
55 (0)

1
67 (1) 1
11 (1)

0
11 (0) 0
13 (0)

1
04 (1) 0
50 (0)

0
04*

0 (0) 0
27 (0)

0
41 (0
4) 0
24 (0
2)

0
11 (0) 0
13 (0)

0
28 (0
2) 0
20 (0)

0
13

*P < 0
05. † Ward A: n = 1, ward B: n = 15, ward C n = 9. ‡ Ward A: n = 11, ward B: n = 9, ward C n = 16.

Confidentiality was assured as only the authors had access to the responses and no individuals or groups were recognizable in the published results. We complied with the compulsory rules for processing personal data set out by the Danish Data Protection Agency and adhered to the guidelines of the regional ethical committee.

Data analysis The population and dependent variables are presented by descriptive statistics including means, standard deviations and ranges. Bias in randomization, responses to the questionnaires and participation in the intervention group was controlled by the application of Pearson’s chi-square analysis for categorical variables and a rank-sum analysis (Mann–Whitney U-test) of the continuous variable ‘age’. Dependent variables applied for analysing effects – defined as significant difference between the intervention and control groups – were (a) the prospective registration of participation in clinical supervision and (b) individual changes in questionnaire responses from base line to follow-up about effectiveness and benefits of clinical supervision. Regarding the latter variables, we tested that scores of the intervention group did not deviate significantly from the control group at baseline. Scores of the variables measuring participation in clinical supervision departed from the criteria for normal distribution, which was also the case with several of the variables applied for measuring effectiveness and benefits. Furthermore, several variables were not characterized by homogenous variance in intervention and control groups. As assumptions for the application of parametric tests were not met, we applied the non-parametric Mann–Whitney’s rank-sum (U-test) in the analyses of effect. Results are reported as probabilities of rejecting the null hypothesis at a 5% level of significance. 916

All statistical analyses were conducted using Stata IC 10
0 (Juul 2006).

Results Initially the sample included 31 nurses (37
4%), 51 auxiliary nurses (61
5%) and one social worker (1
2%). There were 72 women (86
8%) and 11 men (13
3%). 47 staff members worked predominantly day shifts (56
6%), 18 predominantly evening shifts (21
7%) and 18 predominantly night shifts (21
7%). Their ages ranged from 25–62 years with an average age of 43
7 years. Table 4 indicated an effect of the intervention on the staffs’ participation in the usual supervision over 3 months, as the difference between the intervention group (mean 1
04 sessions) and the control group (mean 0
50 sessions) was significant (P = 0
04). Wards A and C did not contribute to this significant result, on the contrary the control groups participated more in the supervision (means: 0
55 and 0
13 sessions) than the intervention groups (means: 0 and 0
11 sessions) on these wards. The significance of the effect for the total population probably derived from Ward B with staff from the intervention group participating more in the supervision (mean: 1
67 sessions) than staff from the control group (mean: 1
11 sessions). The analysis of the participation rate controls for the variation in the number of sessions available on each ward and the differences between the intervention group and the control group lose statistical significance (Table 4). The results indicated that the intervention group continued to have a higher participation rate (mean 0
28) compared with the control group (mean 0
20). Once more, this difference could be identified as stemming from a positive effect in Ward B with no positive effect on wards A and C. In Ward B, the participation rate was higher in the intervention © 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL

A RCT of a meta-supervision intervention

Table 5 Effect of the intervention measured as change from base-line to follow-up questionnaire responses analysed by comparing the intervention and control groups (mean, standard deviation and Mann–Whitney U-test probabilities) Intervention (n = 23) Mean (SD) Supervision effectiveness MCSS MCSS total (n = 45) Trust/rapport (n = 45) Supervisor advice/support (n = 45) Improved care/skills (n = 45) Importance/value of CS (n = 45) Finding time (n = 45) Reflection (n = 45) Benefits SF-36 General health† (n = 45) Vitality† Mental health† (n = 44) MBI Emotional exhaustion (n = 46) Depersonalization (n = 46) Personal accomplishment (n = 42) CSQ Rational coping (n = 44) Emotional coping (n = 45) Detached coping (n = 43) Avoidance coping (n = 45) COPSOQ Social Support (n = 46) Job satisfaction (n = 45) Stress(n = 44) Somatic stress (n = 44) Cognitive stress (n = 44)

1
43 0
18 0
22 0
70 0
52 0
96 0
35

(9
99) (2
69) (2
26) (2
20) (3
31) (2
84) (2
06)

Control (n = 23) Mean (SD)

Effect P

2
57(8
29) 0
14 (2
19) 0
32 (3
11) 0
18 (2
30) 0
39 (3
20) 1
95 (2
70) 0
23 (1
41)

0
39 0
45 0
54 0
06 0
84 0
32 0
19

2
05 (13
94) 2
05 (16
67) 0
68 (11
93)

5
22 (19
97) 5
23 (18
99) 8
94 (13
82)

0
96 0
44 0
12

1
57 (3
38) 0
98 (2
56) 0
13 (3
82)

0
23 (7
06) 1
45 (2
10) 0
66 (8
54)

0
51 0
00* 0
72

0
87 0
25 0
24 0
38

(3
20) (1
98) (1
84) (3
42)

0
65 0
45 0
52 0
13

(4
16) (3
27) (3
49) (2
29)

0
88 0
24 0
20 0
81

4
35 0
38 4
85 3
33 2
27

(15
05) (13
23) (26
78) (24
07) (28
12)

0
04 2
17 0
57 4
26 0
28

(14
54) (15
12) (13
21) (14
61) (18
14)

0
34 0
72 0
66 0
45 0
50

*P < 0
01. † Aggregate scores were converted into a score ranging from 0–100.

group (mean 0
41), which was considerably more compared with the participation rate in the control group (mean 0
24). The effect of the intervention on the participation was not reflected in the self-reported supervision ‘effectiveness’ as measured by the MCSS presented in Table 5. Neither the total score of the MCSS (P = 0
39) nor the six subscales constituting the MCSS (P = 0
45, 0
54, 0
06, 0
84, 0
32, 0
19) indicated any significant effect of the intervention. Likewise, differences in means did not reveal consistently larger positive – or less negative – changes in the intervention group compared with the control group and, quite controversially, there seems to be an overall negative change in the self-reported experience of supervision effectiveness. Additional analyses conducted for each of the three wards separately did not provide any statistically significant results either. © 2014 John Wiley & Sons Ltd

As we did not find any significant changes in effectiveness, it was expected that analyses of ‘benefits’ from supervision would reveal similar insignificant results (Table 5). None of the subscales from SF36, MBI, CSQ and COPSOQ providing measures of possible benefits of supervision indicated any significant effect of the intervention compared with the control group. The only exception was the result for the subscale Depersonalization from MBI (P = 0
001). However, we believe that this result either was spurious because this scale presented low Cronbach’s alpha coefficients, which indicated uncertain internal consistency, or because it was a chance finding (a type I error).

Discussion Despite an intervention with a short timeframe, this study indicated a significant positive effect of the meta-supervision 917

H. Gonge and N. Buus

intervention as staff from the intervention group attended more clinical supervision sessions over a period of 3 months than did the staff from the control group. It was noticeable that the effect on participation could be attributed to one of the three wards, while no positive change was found in the other two wards. Interestingly, the ward producing the significant change was the ward with the highest number of intervention attendants and the contextual virtue of being the ward providing by far the highest number of clinical supervision sessions throughout the 3 months of registration (7 sessions vs. 1 and 2 sessions respectively). Adjusting for the variation in number of clinical supervision sessions conducted in the three wards produces participation rates as a more reliable measure of effect. Analysing participation rates the difference in participation between intervention and control groups dropped below level of significance while the tendency was retained. The meaning we drew from these results was that it was possible to motivate more frequent participation in clinical supervision; however, the effect of the intervention appeared to be influenced by existing clinical supervision routines and possibly by organizational characteristics of the ward in general. Our impression from the intervention was that the introduction seminar worked quite well, while participation in the follow-up sessions was disappointing. This was probably attributable to practical and motivational factors. Interpreting the results of this study it should be kept in mind that the main part of the intervention group only did 3 hours training with only seven participants attending the total 5½ hours intervention. More time for the introduction seminar and stronger emphasis on participation in the follow-up sessions would be recommendable and is likely to create more substantial changes (cf. Butterworth et al. 2008). More frequent participation was an important effect of the intervention because, according to the applied model, participation functions as the prerequisite for any effectiveness or benefits of clinical supervision. It is essential to motivate and monitor participation, as the importance of frequent and consistent clinical supervision cannot be underestimated. If clinical supervision participation is not continuous it is unlikely to identify a positive effect, which may lead to ambivalence about the value of clinical supervision (Cleary et al. 2010). Evaluating the material participants presented during the intervention sessions indicated to us that it was meaningful to test an intervention consisting of a process of systematic reflection on staff members’ own experiences of clinical supervision focusing on barriers and potential benefits. Participants described individual barriers such as defensiveness 918

and reluctance (Wright 2012) and organizational obstacles such as work shifts and duty roster (Buus et al. 2010, 2011). The formulation of strategies to overcome these barriers proved efficient for some, while others lacked the motivation to pursue the aims they had formulated. In any case, it appeared promising to create space – physically and psychologically – allowing staff open and critical reflection on their supervision inspiring them to take ownership of how the supervision could be better adapted to their individual needs in the specific workplace (cf. Cleary et al. 2010). An intriguing observation from our study was that the intervention apparently had the strongest effect on those staff already engaged in clinical supervision or staff employed in wards with a progressive attitude towards clinical supervision. We agree with the findings of White and Winstanley (2010b) that interventions promoting clinical supervision probably have the best chances of a positive result when given to staff with a prepared mind employed in wards actively engaged in clinical supervision. On the contrary, the most stressed, defensive, anxious or even hostile individuals or wards are probably those least likely to engage themselves in the process of self-disclosure and reflection that would be required to find clinical supervision useful (Butterworth et al. 2008). Following the change in participation, we hoped to find more positive evaluations of the clinical supervision effectiveness in the intervention group compared with the control group, but this was not the case. When no change was found in the experienced effectiveness it came as no surprise that general benefits could not be detected either, however, limitations of this study may contribute to this finding.

Limitations First, an intervention with a formal duration of 5½ hours when the main body of participants only attended the first 3 hours obviously has limitations to the extent of change that can be produced. We believe that a more comprehensive introduction seminar combined with full attendance to the follow-up sessions would have stronger impact. Second, general methodological issues should be considered. Contrary to the direct observation of clinical supervision participation the analyses of effectiveness relied on subjective questionnaire reports at baseline and follow-up. The interrogation of bias in the questionnaire responses indicated that staff from the ward with frequent supervision had a considerably higher response rate compared with the other wards, which is in line with findings from a previous study (Gonge & Buus 2010). We find reason to believe that © 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL

the validity of studies of clinical supervision effectiveness will be impaired if they are based on self-reported data, because staff engaged in continuous supervision will be more likely to provide data than those who are not. Another methodological issue observed in the pilot study was that participation in the intervention probably affects the staff’s ambitions about clinical supervision (Buus et al. 2013), which in turn is likely to affect how they rate supervision effectiveness. In other words, the intervention may make participants expect more from the supervision, which affect their criteria for evaluating supervision. Third, circumstantial limitations applied to this study. As it was embedded in a larger study, we had to randomize the staff in groups across three wards and not ward by ward. Not only did this condition result in uneven numbers of staff from each ward being appointed to the intervention group, but participants enrolled in the intervention group also had to implement their supervision promoting strategies among colleagues from the control group, which may have led to confluence in the measures of effect. An illustrious example is that some intervention group participants successfully took the initiative to request supervision at night time allowing night shift staff a realistic opportunity to participate, which equally benefitted night shift colleagues from the control group. Fourth, during the study period a very violent event with close to fatal consequences took place in one of the participating wards leaving staff traumatized at the specific ward and the hospital in general. From a scientific perspective this impairs comparability of baseline and follow-up measures and is likely to have confounded any possible effect of the intervention. As a consequence of the incident, a previously planned initiative to arrange supplementary clinical supervision sessions was postponed at one of the wards, who, therefore, offered fewer sessions. Fifth, dropouts resulting in a relatively small sample size (analyses of participation n = 61, 73
5% and analyses of effectiveness and benefits n = 46, 55
4%) are also a limitation likely to have affected the results of this study. Furthermore, analyses were conducted ‘as per protocol’ as staff – that did not attend the intervention despite being assigned for intervention group – were ignored (34
2%). We gave priority to analysing the possible effect among the attendants, but we recognize that an ‘intention to treat’ approach, including the non-attendants, could have affected the results. Sixth, it was beyond the scope of this study to control for individual differences in the practice of the supervisors, which may contribute to variation in supervision culture across the participating wards. © 2014 John Wiley & Sons Ltd

A RCT of a meta-supervision intervention

Conclusion There is a lack of well-conducted intervention studies investigating outcome of clinical supervision in mental health nursing and this study reported results from a randomized controlled trial aiming at testing a meta-supervision intervention. Despite the robust design of the study and the conservative selection of validated measurement instruments, we did experience serious methodological challenges leading to the conclusion that this area of research does not easily lend itself to scientific investigation. In the light of these obstacles, it was interesting that the meta-supervision intervention produced a significant change in the nominal clinical supervision participation of those who attended the intervention and participated more than the control group. The implication of this study is that individual and organizational barriers to involvement in clinical supervision should be addressed and doing so it is recommended that attention is paid to individual and organizational attitudes towards clinical supervision. Such contextual variation poses a challenge to the research designs in studies of clinical supervision as the applied methodology must be sensitive to reflect organizational diversity in supervision cultures.

Acknowledgements The Psychiatric Research Fund of the Mental Health Services in the Region of Southern Denmark funded the study. The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Funding This study was supported by grants from The Psychiatric Research Fund of the Mental Health Services in the Region of Southern Denmark. The funding source provided financial support without any involvement in the outcome of the study.

Conflict of interest No conflict of interest has been declared by the authors.

Author contributions All authors have agreed on the final version and meet at least one of the following criteria [recommended by the ICMJE (http://www.icmje.org/ethical_1author.html)]: 919

H. Gonge and N. Buus

• •

substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content.

References Berg A. & Hallberg I.R. (1999) Effects of systematic clinical supervision on psychiatric nurses’ sense of coherence, creativity, work-related strain, job satisfaction and view of the effects from clinical supervision: a pre-post test design. Journal of Psychiatric and Mental Health Nursing 6(5), 371–381. Berg A., Hansson U.W. & Hallberg I.R. (1994) Nurses’ creativity, tedium and burnout during 1 year of clinical supervision and implementation of individually planned nursing care: comparisons between a ward for severely demented patients and a similar control ward. Journal of Advanced Nursing 20(4), 742–749. Bjørner JB., Damsgaard M., Watt T., Bech P., Rasmussen NK., Kristensen T. S., Modvig J. & Thuneborg K. (1997) Dansk Manual til SF-36. Lægemiddel Industri Foreningen, Copenhagen. Bradshaw T., Butterworth A. & Mairs H. (2007) Does structured clinical supervision during psychosocial intervention education enhance outcome for mental health nurses and the service users they work with? Journal of Psychiatric and Mental Health Nursing 14(1), 4–12. Brunero S. & Stein-Parbury J. (2008) The effectiveness of clinical supervision in nursing: an evidenced based literature review. Australian Journal of Advanced Nursing 25(3), 86–94. Butterworth T., Bell L., Jackson C. & Pajnkihar M. (2008) Wicked spell or magic bullet? A review of the clinical supervision literature 2001–2007. Nurse Education Today 28, 264–272. Buus N. & Gonge H. (2009) Empirical studies of clinical supervision in psychiatric nursing: a systematic literature review and methodological critique. International Journal of Mental Health Nursing 18(4), 250–264. Buus N. & Gonge H. (2013) Translation of the Manchester Clinical Supervision Scale (MCSS) into Danish and a preliminary psychometric validation. International Journal of Mental Health Nursing 22(2), 145–153. Buus N., Angel S., Traynor M. & Gonge H. (2010) Psychiatric hospital nursing staff’s experiences of participating in groupbased clinical supervision: an interview study. Issues in Mental Health Nursing 31(10), 654–661. Buus N., Angel S., Traynor M. & Gonge H. (2011) Psychiatric nursing staff members’ reflections on participating in groupbased clinical supervision: a semistructured interview study. International Journal of Mental Health Nursing 20(2), 95–101. Buus N., Cassedy P. & Gonge H. (2013) Developing a manual for strengthening mental health nurses’ clinical supervision. Issues in Mental Health Nursing 34(5), 344–349. Cassedy P. (2010) First Steps in Clinical Supervision. A Guide for Healthcare Professionals, 1st edn. Open University Press, Maidenhead. Cleary M., Horsfall J. & Happell B. (2010) Establishing clinical supervision in acute mental health inpatient units:

920

acknowledging the challenges. Issues in Mental Health Nursing 31(8), 525–531. Dawson M., Phillips B. & Leggat S. (2013) Clinical supervision for allied health professionals. A systematic review. Journal of Allied Health 42(1), 65–73. Edberg A.K., Hallberg I.R. & Gustafson L. (1996) Effects of clinical supervision on nurse-patient cooperation quality: a controlled study in dementia care. Clinical Nursing Research 5 (2), 127–146. Edberg A.K., Norberg A. & Hallberg I.R. (1999) Mood and general behavior of patients with severe dementia during one year of supervised, individualized planned care and systematic clinical supervision. Comparison with a similar control group. Aging (Milano.) 11(6), 395–403. Edwards D., Burnard P., Hannigan B., Cooper L., Adams J., Jugessur T., Fothergil A. & Coyle D. (2006) Clinical supervision and burnout: the influence of clinical supervision for community mental health nurses. Journal of Clinical Nursing 15(8), 1007– 1015. Elklit A. (1996) Coping styles questionnaire: a contribution to the validation of a scale for measuring coping strategies. Personality and Individual Differences 21(5), 809–812. Gonge H. & Buus N. (2010) Individual and workplace factors that influence psychiatric nursing staff’s participation in clinical supervision: a survey study and prospective longitudinal registration. Issues in Mental Health Nursing 31(5), 345–354. Gonge H. & Buus N. (2011) Model for investigating the benefits of clinical supervision in psychiatric nursing: a survey study. International Journal of Mental Health Nursing 20(2), 102– 111. Hallberg I.R. (1994) Systematic clinical supervision in a child psychiatric ward: satisfaction with nursing care, tedium, burnout and the nurses’ own report on the effects of it. Archives of Psychiatric Nursing 8(1), 44–52. Hallberg I.R. & Norberg A. (1993) Strain among nurses and their emotional reactions during 1 year of systematic clinical supervision combined with the implementation of individualized care in dementia nursing. Journal of Advanced Nursing 18(12), 1860–1875. Hallberg I.R., Welander-Hansson U. & Axelsson K. (1994) Satisfaction with nursing care and work during a year of clinical supervision and individualized care. Comparison between two wards for the care of severely demented patients. Journal of Nursing Management 1, 297–307. Hawkins P. & Shohet R. (2012) Supervision in the Helping Professions. Open University Press, Maidenhead. Hyrk€as K. (2005) Clinical supervision, burnout and job satisfaction among mental health and psychiatric nurses in Finland. Issues in Mental Health Nursing 26(5), 531–556. Juul S. (2006) An Introduction to Stata for Health Researchers. Stata Press, College Station, TX. Kristensen T.S. & Borritz M. (1998) Forebyggelse af udbrændthed. Arbejdsmiljøfondet, Copenhagen. Kristensen T.S., Hannerz H., Hogh A. & Borg V. (2005) The Copenhagen Psychosocial Questionnaire – a tool for the assessment and improvement of the psychosocial work environment. Scandinavian Journal of Work, Environment & Health 31(6), 438–449.

© 2014 John Wiley & Sons Ltd

JAN: ORIGINAL RESEARCH: CLINICAL TRIAL Maslach C. & Jackson S. (1986) Maslach Burnout Inventory. Manual, 2nd edn. Consulting Psychologists Press, Palo Alto, CA. Proctor B. (1987) Supervision: a co-operative exercise in accountability. In Enabling & Ensuring. Supervision in Practice (Marken M. & Payne M., eds), National Youth Bureau and the Council for Education and Training in Youth and Community Work, Leicester, pp. 21–34. Roger D., Jarvis G. & Najarian B. (1993) Detachment and coping: the construction and validation of a new scale for measuring coping strategies. Personality and Individual Differences 15(6), 619–626. Sloan G. & Grant A. (2012) A rationale for a clinical supervision database for mental health nursing in the UK. Journal of Psychiatric and Mental Health Nursing 19, 466–473. White E. & Winstanley J. (2009) Implementation of Clinical Supervision: educational preparation and subsequent diary accounts of the practicalities involved, from an Australian mental

A RCT of a meta-supervision intervention health innovation. Journal of Psychiatric and Mental Health Nursing 16(10), 895–903. White E. & Winstanley J. (2010a) A randomised controlled trial of clinical supervision: selected findings from a novel Australian attempt to establish the evidence base for causal relationships with quality of care and patient outcomes, as an informed contribution to mental health nursing practice development. Journal of Research in Nursing 15(2), 151–167. White E. & Winstanley J. (2010b) Clinical Supervision for mental health professionals: the evidence base. Social Work & Social Sciences Review 14(3), 73–90. Winstanley J. (2000) Manchester Clinical Supervision Scale User Guide. Osman Consulting Pty Ltd, Sydney. Winstanley J. & White E. (2011) The MCSS-26: revision of the manchester clinical supervision scale using the Rasch measurement model. Journal of Nursing Measurement 19, 160–178. Wright J. (2012) Clinical supervision: a review of the evidence base. Nursing Standard 27(3), 44–49.

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of evidence-based nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance and with potential to advance knowledge for practice, education, management or policy. JAN publishes research reviews, original research reports and methodological and theoretical papers. For further information, please visit JAN on the Wiley Online Library website: www.wileyonlinelibrary.com/journal/jan Reasons to publish your work in JAN:

• High-impact forum: the world’s most cited nursing journal, with an Impact Factor of 1·527 – ranked 14/101 in the 2012 ISI Journal Citation Reports © (Nursing (Social Science)).

• Most read nursing journal in the world: over 3 million articles downloaded online per year and accessible in over 10,000 libraries worldwide (including over 3,500 in developing countries with free or low cost access).

• • • •

Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jan. Positive publishing experience: rapid double-blind peer review with constructive feedback. Rapid online publication in five weeks: average time from final manuscript arriving in production to online publication. Online Open: the option to pay to make your article freely and openly accessible to non-subscribers upon publication on Wiley Online Library, as well as the option to deposit the article in your own or your funding agency’s preferred archive (e.g. PubMed).

© 2014 John Wiley & Sons Ltd

921

This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.