Clinical Psychology and Psychotherapy Clin. Psychol. Psychother. 23, 77–86 (2016) Published online 11 December 2014 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/cpp.1935

Assessment

Development and Validation of a Short Version of the Supervisory Relationship Questionnaire Tom Cliffe, Helen Beinart and Myra Cooper* Oxford Doctoral Course in Clinical Psychology, Oxford Institute of Clinical Psychology Training, Harris Manchester College, University of Oxford, Oxford, UK The Supervisory Relationship Questionnaire (SRQ) is one of the few theoretically sound and psychometrically valid questionnaires for measuring the SR within clinical supervision. However, its length can make it difficult to use in clinical practice and research. This study aimed to produce a shorter version of the SRQ (the Short Supervisory Relationship Questionnaire: S-SRQ) that retained its reliability and psychometric validity. The SRQ’s 67 items were initially reduced using the criteria of external, internal and face validity. Two hundred and three UK trainee clinical psychologists then completed a series of online questionnaires including the S-SRQ and other clinical supervision measures. A Principal Component Analysis identified three components of the S-SRQ: ‘safe base’, ‘reflective education’ and ‘structure’, and an 18-item version was produced. Analyses revealed that the S-SRQ has high internal reliability, adequate test–retest reliability and good convergent, divergent and predictive validity. Participants also rated the S-SRQ as easy to use and potentially helpful for providing feedback on the SR in supervision. The S-SRQ (three subscales, 18 items) is a valid and reliable measure of the SR from the supervisee perspective. The current findings also support aspects of existing models of the SR. The S-SRQ is a promising measure for use in clinical, training and research settings. Copyright © 2014 John Wiley & Sons, Ltd. Key Practitioner Message: • The Short Supervisory Relationship Questionnaire (S-SRQ) is a psychometrically reliable and valid 18-item measure of the supervisory relationship based on the SRQ. • Clinically, the measure represents a quick and accessible means for supervisees to assess the quality of their supervisory relationship and discuss this with their supervisors. • It can also be used in conjunction with the supervisor-completed Supervisory Relationship Measure to support a dyadic discussion about clinical supervision and the supervisory relationship. Keywords: clinical supervision, supervisory relationship, measure

INTRODUCTION Clinical Supervision Clinical supervision has become increasingly emphasized and researched in the training of mental health professionals over the last two decades (Watkins, 2012a). Defined as, ‘the formal provision, by approved supervisors, of a relationship-based education and training that is case-focused and which manages, supports, develops and evaluates the work of junior colleagues’ (Milne 2007,

*Correspondence to: Dr Myra Cooper, Oxford Doctoral Course in Clinical Psychology, Isis Education Centre, Warneford Hospital, Oxford, OX3 7JX, UK. E-mail: [email protected] [Correction added on 21 December 2015 after first online publication: The article was previously published as “Research Article”and this has been changed to “Assessment”]

Copyright © 2014 John Wiley & Sons, Ltd.

p. 440), it is a significant part of professional development and monitoring. Indeed, Fleming and Steen (2012) state that supervision can be used to ‘improve future practice and ultimately have a positive impact on client outcome’ (p. 10). Recently, there has been progress in the UK and other countries towards competence-based and evidencebased supervision (Milne, 2009; Watkins, 2012b) to ensure delivery is monitored and unacceptable variability is lowered (Milne & Reiser, 2012). This has required defining and operationalizing clinical supervision, followed by the development of a research base investigating and revising such definitions (Falender & Shafranske, 2012).

The Supervisory Relationship One major finding has been that successful supervision is substantially linked to the supervisory relationship (SR) (Beinart, 2012; Inman & Ladany, 2008; Milne, 2009),

78 regardless of the supervision model adopted (Ladany, Ellis & Friedlander, 1999a). Emerging consensus suggests that a ‘good’ SR represents a safe, secure base established by a consistent, responsive supervisor sensitive to their supervisee’s needs (Beinart & Clohessy, 2009) who supports supervisees to ‘explore’ and develop their competencies (Watkins & Riggs, 2012; White & Queener, 2003).

Supervisory Relationship Models Bordin’s (1983) model of the ‘supervisory working alliance’ contains three components central to developing an effective SR emphasizing mutual agreement and understanding of supervisory goals and tasks, underpinned by an emotional bond. This model has been relatively well researched due to having a corresponding measure. However, as a translational measure from the therapeutic alliance literature, it is argued that it misses important elements of the supervisory process. These include its educational, involuntary and evaluative functions (Palomo, Beinart, & Cooper, 2010; Inman & Ladany, 2008). Holloway (1995) proposes that the SR has three components: interpersonal relationship between supervisor and supervisee, phase of the relationship and supervisory contract. This model has not been fully tested empirically; it lacks a validated measure of its constructs. Beinart’s model (Beinart, 2002; 2012), derived from qualitative research, argues that relational elements such as boundaries and trust must be facilitated before the tasks of supervision (e.g., education and evaluation) can become effective. Other SR qualities include support, respect, commitment, sensitivity to needs and collaboration. This model has support from a number of studies highlighting these characteristics (Cushway & Knibbs, 2004; Weaks, 2002). Using quantitative research, Palomo et al. (2010) expanded Beinart’s model and conceptualized the SR as three ‘facilitative’ components (safe base, commitment and structure) and three ‘educative’ components (supervisor acting as a role model, initiating reflective education and delivering formative feedback) that support supervisee development.

The Supervisory Relationship and Satisfaction with Supervision Supervisee satisfaction with supervision has been widely used as an outcome measure in supervision research (Ellis & Ladany, 1997). The SR is a good predictor of supervisee satisfaction (Cheon, Blumer, Shih, Murphy, & Sato, 2009; Ramos-Sanchez et al., 2002). Britt and Gleaves (2011) found that ‘collaboration and mutual understanding’, key components of a good SR, was the best predictor of satisfaction with supervision. Palomo et al. (2010) also found that SR ratings predicted satisfaction with Copyright © 2014 John Wiley & Sons, Ltd.

T. Cliffe et al. supervision and its perceived effectiveness. Limitations of using satisfaction to assess supervision include the notion that supervisees may not always find corrective feedback and increasingly challenging supervisory goals as ‘satisfying’, although these processes may lead to ‘effective’ supervision (O’Donovan & Kavanagh, 2014). Additionally, supervisee resistance to the SR and process (e.g., to the supervisor’s role and influence) may lead to a negative satisfaction rating despite there being effective aspects to the supervision. Therefore, satisfaction with supervision should not be the only index of supervision quality (Goodyear & Bernard, 1998).

Clinical Supervision and Clinical Outcomes Few studies have examined client outcomes and clinical supervision (Inman & Ladany, 2008). Indeed, studying the supervisor–supervisee–client triad requires complex methodologies and measures (Ladany, Walker, PateCarolan, & Gray, 2008) that have limited this research (Milne & Reiser, 2012). However, two studies have produced rigorous, if equivocal, relevant research. Bambling, King, Raue, Schweitzer and Lambert (2006) demonstrated that clients in supervised treatment rated their therapeutic alliance and treatment satisfaction higher and symptoms lower than clients in unsupervised treatment. Conversely, White and Winstanley (2010) found that supervision was not related to client satisfaction with treatment.

Measures of Supervision and the Supervisory Relationship Measures of supervision are important for testing definitions, models, and facilitating and generating research (Watkins, 1997). Indeed, Bernard and Goodyear (2014) argue that assessment is an essential aspect of supervision and can aid the development of inexperienced supervisors (Crook-Lyon, Heppler, Leavitt, & Fisher, 2008). Existing measures of the SR, for example, the Working Alliance Inventory–Trainee Form (WAI-T; Bahrick, 1990), are associated with supervisee satisfaction (Ladany, LehrmanWaterman, Molinaro & Wolgast, 1999b) and have been used to develop other supervision measures (LehrmanWaterman & Ladany, 2001; Palomo et al., 2010). The Role Conflict and Role Ambiguity Inventory (RCRA; Olk & Friedlander, 1992) has shown that higher role conflict and ambiguity within supervision are associated with lowerrated SRs and lower satisfaction with supervision (Palomo et al., 2010). A recent review of clinical supervision measures (Ellis, D’Iuso, & Ladany, 2008) concluded that most were the product of studies with multiple limitations, including psychometric limitations. The dearth of psychometrically valid measures has had a negative impact on supervision research (Ellis & Ladany; 1997, Milne, 2009). Clin. Psychol. Psychother. 23, 77–86 (2016)

Short Supervisory Relationship Questionnaire One psychometrically valid and reliable measure derived from a theoretically robust SR model (Beinart, 2002; 2012) is the SR Questionnaire (SRQ) (Palomo et al., 2010). Supervisee-completed, it has a coherent factor structure and has been used to monitor the supervision of UK clinical psychologists. However, it has been suggested that it would be more widely adopted if it had fewer than its existing 67 items (Wainwright, 2010). Additionally, some items may not be completed due to their sensitivity, (e.g., ‘I respected my supervisor as a person’). Although the questionnaire meets Ellis and Ladany’s (1997) recommendations for use in supervision research, there is a definite need for a shorter version.

AIMS The present study will first shorten the SRQ and, second, determine the psychometric properties of the short form. In light of previous supervision research, it will investigate whether the short form predicts supervisee ratings of satisfaction with supervision and its effectiveness.

METHOD Participants These were trainee clinical psychologists on doctoral programmes in the UK. Twenty out of the 30 UK training programmes responded to an initial contact. Trainees of the 18 courses who agreed to participate were then contacted. Only trainees who had experienced an SR of 4 months or longer during clinical training were included.

Measures Measures were chosen on the basis of sound psychometric properties (Ellis et al., 2008) and use in previous research.

Draft Short Supervisory Relationship Questionnaire A 28-item draft Short Supervisory Relationship Questionnaire (S-SRQ), based on the original SRQ, was administered, developed and validated as part of the present study. Its development is detailed in the Procedure section.

Short Supervisory Relationship Questionnaire Feedback Questionnaire This four-item measure asked for feedback on the S-SRQ. Questions (on a five-point scale) asked how easy it was to complete and understand, how comfortable trainees would feel discussing their answers with their supervisor and how helpful it would be for providing feedback on the SR in supervision. This measure was created and used in the present Copyright © 2014 John Wiley & Sons, Ltd.

79 study in order to explore the practical utility of the S-SRQ, rather than its psychometric validity.

Role Conflict and Role Ambiguity Inventory (Olk & Friedlander, 1992) The RCRA is a 29-item measure of negative experiences that a supervisee may encounter in supervision. Rated on a five-point scale, from 1 (not at all) to 5 (very much), it contains two subscales: role conflict (supervisee and supervisor differing expectations for the supervisee’s behaviour) and role ambiguity (supervisee lack of clarity regarding supervisor expectations and evaluation of their performance). Both subscales have good internal reliability and adequate convergent validity (Olk & Friedlander, 1992). The measure has been recommended for use in clinical and research contexts (Ellis & Ladany, 1997; Wheeler, Aveline, & Barkham, 2011). It was chosen for the present study in order to assess the S-SRQ’s convergent validity, having also been used by Palomo et al. (2010) for the same purpose.

Working Alliance Inventory Trainee Form (Bahrick, 1990) The WAI-T is a 36-item measure of the supervisory working alliance based on Bordin’s (1983) model. It measures three components (agreement on supervision goals, tasks and supervisee–supervisor bond) using three subscales on a seven-point scale (from ‘never’ to ‘always’). It has good internal reliability (α ≥ 0.90) for all subscales in several studies (e.g., Ladany & Friedlander, 1995). It has construct validity and has been widely used to measure the SR (Palomo et al., 2010). It is negatively related to the RCRA subscales (Ladany & Friedlander, 1995). The measure was chosen for the present study to analyse convergent validity.

Supervisory Satisfaction Questionnaire Ladany, Hill, Corbett, & Nutt, 1996 The Supervisory Satisfaction Questionnaire (SSQ) is an eight-item questionnaire that asks supervisees to rate their satisfaction with aspects of supervision on a four-point scale. On the basis of the Client Satisfaction Questionnaire (Larsen, Attkisson, Hargreaves, & Nguyen, 1979), it has a consistent single factor and high internal consistency (α = 0.96) (Ladany et al., 1996). The SSQ was chosen for the present study to assess the predictive validity of the S-SRQ.

Short-scale Eysenck Personality Questionnaire Revised (Eysenck, Eysenck, & Barrett 1985) The short-scale Eysenck Personality Questionnaire Revised (EPQ-R) is a short version of the EPQ-R, itself based on the EPQ. It has 48 items rated on a two-point scale (Yes; No) and four subscales measuring the personality dimensions of extraversion, neuroticism and psychoticism, alongside a ‘lie’ subscale. Internal reliability of the subscales varies from α = 0.61 to α = 0.88. This measure was Clin. Psychol. Psychother. 23, 77–86 (2016)

T. Cliffe et al.

80 chosen in order to ascertain the S-SRQ’s divergent validity, given that it theoretically measures a different construct.

Indices of Supervision Outcome (Friedlander & Ward, 1984) An adapted version of this measure was used by Palomo et al. (2010) and further modified here. It has five items rated on a seven-point scale (extremely negative to extremely positive), asking supervisees to rate the extent to which they believe their supervisor is affecting their personal and professional development, their therapeutic work and client’s progress. Questions were removed here if they overlapped with SSQ questions. The indices of supervision outcome (IoS) were utilized alongside the SSQ to determine the predictive validity of the S-SRQ.

Supervisory Relationship Questionnaire (Palomo et al., 2010) The SRQ is a 67-item measure asking supervisees to rate their SR on a seven-point scale from ‘strongly disagree’ to ‘strongly agree’. It contains six subscales: safe base, structure, commitment, reflective education, role model and formative feedback. Internal reliability is extremely high (α = 0.98), and test–retest reliability is good. The measure has good construct, predictive and divergent validity and was used for the present study to determine the S-SRQ’s convergent and predictive validity.

similar when item removed and relatively low discriminatory α parameter values, flatter Item Response Curves compared with other items in that subscale [internal item quality]; lower likelihood of more ‘sensitive’ items being completed compared with less ‘sensitive’ items and higher likelihood of socially desirable answers for certain items). Two phases of item reduction were undertaken, with the SRQ shortened from 67 to 28 items. An experienced supervision researcher (D. Milne) assessed the face validity of the 28-item S-SRQ positively, using a supervision measure evaluation form (Milne, Reiser, Cliffe, & Raine, 2011).

Validation of the Draft Short Supervisory Relationship Questionnaire Directors of 30 UK Clinical Psychology training programmes were contacted by email to ask permission to contact their trainees to request participation. If Directors approved, an email was sent to eligible trainees with a link to the Internet survey. Two hundred and three participants completed the S-SRQ and 86 completed the S-SRQ a second time to determine test–retest reliability.

Ethical Approval The study received ethical approval from the Central University Research Ethics Committee, University of Oxford.

Demographic data Participants were asked for their gender, months into training and preferred theoretical orientation, and the same information about their supervisors.

RESULTS Descriptive Data Response Rate

Procedure Shortening the Satisfaction Relationship Questionnaire The SRQ was shortened using best practice recommendations for scale reduction (Stanton, Sinar, Balzer, & Smith, 2002). Using the original SRQ data containing 284 participants (Palomo et al., 2010), external item quality was determined by correlations between the SRQ and other measures. Internal item quality was determined by calculating Cronbach’s alpha values, interitem-total and item-total correlations and item factor loadings for each SRQ item and subscale. Item Response Theory (Embretson, & Reise, 2000; van der Linden & Hambleton, 2010) has been recommended by Ellis et al. (2008) for informing internal item quality. Item Response Curves and Item Parameter Estimates were therefore generated with a Graded Response Model using IRTPRO software, version 2.1 (Cai, Thissen, & du Toit, 2013). Face validity of items was ascertained through discussion with the coauthors of the SRQ. Items were removed iteratively (e.g., relatively higher and lower correlations with supervision measures than expected [external item quality]; Cronbach’s α remaining Copyright © 2014 John Wiley & Sons, Ltd.

Thirty doctoral training courses in clinical psychology were approached, and 18 (60%) agreed to participate. A total of 220 trainee clinical psychologists gave consent to participate in the study. Two hundred and three of these completed the S-SRQ and S-SRQ feedback questionnaire, and 177 completed all supervision measures.

Supervisee Characteristics

One hundred and fifty-seven participants (88.7%) were female and 20 male (11.3%). The mean number of months into training was 19.5 (standard deviation = 8.69, range = 4–38). Participant’s preferred theoretical orientations were primarily eclectic/integrative (44.1%), cognitive–behavioural (23.2%) or systemic (14.7%).

Supervisor Characteristics Most participants rated their SR with a female supervisor (75.1%). Most supervisors were experienced, supervising 10 trainees (24.3%), 6–10 trainees (27.7%), 3–5 trainees (24.9%) or 1–2 trainees (16.9%). Some supervisors (6.2%) had not previously supervised a trainee. The most common Clin. Psychol. Psychother. 23, 77–86 (2016)

Short Supervisory Relationship Questionnaire

81

supervisor theoretical orientations were cognitive– behavioural (40.7%) or eclectic/integrative (23.7%), followed by systemic (11.3%) and psychodynamic (8.5%).

Supervision Measures Table 1 shows participants’ scores on the supervision measures. On average, participants rated their SRs positively, with high scores on the S-SRQ, SRQ and WAI-T and low scores on the RCRA.

Feedback on the 28-Item Short Supervisory Relationship Questionnaire Most participants rated the 28-item S-SRQ as very easy (n = 80, 39.4%) or easy (n = 107, 52.7%) to complete. Most participants found the S-SRQ very easy (n = 117, 57.6%) or easy (n = 83, 40.9%) to understand. Forty-four participants (21.7%) would feel very comfortable discussing S-SRQ responses with their supervisors. Seventy-one participants (35%) would feel comfortable, 17 participants (8.4%) neither comfortable nor uncomfortable, 56 participants (27.6%) uncomfortable and 15 participants (7.4%) very uncomfortable. Most participants rated the S-SRQ as being somewhat helpful for providing SR feedback within supervision (n = 132, 65%). Thirty-two participants (15.8%) rated it as very helpful.

Factor Analysis A Principal Component Analysis (PCA) was used for factor extraction, with Varimax rotation and Kaiser normalization. Stanton et al. (2002) have recommended using PCA for shortening measures. This type of factor analysis was also chosen due to having a smaller pool of items than in the original Palomo et al. (2010) analysis, and

therefore, the original six-factor structure would be unlikely to be replicated. Additionally, the measure in question is still in development, and it was felt that an exploratory analysis was most suitable. Kaiser–Meyer–Olkin measure of sampling adequacy was acceptable at 0.952, and Bartlett’s test of sphericity was significant (p < 0.001). The Scree plot and Kaiser criterion (eigenvalues >1; Cattell, 1978) suggested three components. These accounted for 69.3% of the variance in S-SRQ scores. Two-factor and four-factor solutions were tested but demonstrated overlap between items and components and were discounted (Tabachnick & Fidell, 2003).

Revised Questionnaire Items were kept if they loaded greater than 0.45 on a single component and below 0.45 on the other components. For the first component, items with the eight lowest loadings were removed because the component had many high loading items. The three components were agreed with the coauthors of the SRQ and given titles related to corresponding components on the SRQ. The ‘Safe Base’ component accounted for the largest proportion of variance (57.45%) with items relating to a collaborative, open and safe environment from which supervision can take place. The ‘Reflective Education’ component accounted for 7.1% of the variance. These items concern the supervisor’s ability to facilitate supervisee reflection and pay attention to the process of supervision and different theoretical models. The third component, ‘Structure’, accounted for 4.8% of the variance in S-SRQ scores. Its items relate to the practical elements of supervision and the supervisor’s ability to ensure these are managed appropriately. The final version of the S-SRQ contained 18 items (Appendix) and was used for all subsequent analyses.

Alpha Reliabilities Table 1. Participant’s scores on supervision measures Measure

S-SRQ SRQ RCRA Conflict subscale Ambiguity subscale WAI-T Bond subscale Goal subscale Task subscale

Sample n (missing)

Mean

SD

Range

203 (0) 177 (26)

99.49 379.41

21.02 74.23

21–126 129–466

196 (7) 196 (7)

15.85 37.22

6.65 15.13

10–39 19–86

191 (12) 191 (12) 191 (12)

61.54 65.03 64.72

14.92 13.39 13.06

16–84 22–84 23–84

S-SRQ = Short Supervisory Relationship Questionnaire. SRQ = Supervisory Relationship Questionnaire. RCRA = Role Conflict and Role Ambiguity Inventory. WAI-T = Working Alliance Inventory Trainee Form. SD = standard deviation.

Copyright © 2014 John Wiley & Sons, Ltd.

The S-SRQ and its three subscales had high internal reliability. The internal consistency of the overall scale was high (α = 0.96). Item-total correlations ranged from 0.53 to 0.87. Alpha values for subscales (and ranges for item-total correlations) were as follows: safe base α = 0.97 (range 0.79 to 0.90), reflective education α = 0.89 (range 0.67 to 0.80) and structure subscale α = 0.88 (range 0.69 to 0.78).

Test–Retest Reliability The S-SRQ demonstrated acceptable test–retest reliability. Scores did not fluctuate dramatically over time, but findings suggested that the measure is still sensitive enough to detect some change. Eighty-six participants completed the S-SRQ a second time. Original scores were highly and significantly Clin. Psychol. Psychother. 23, 77–86 (2016)

T. Cliffe et al.

82 correlated with scores approximately 2–4 weeks later (r(84) = 0.94, p < 0.001, two-tailed). A Wilcoxon Signed Rank test showed a moderately significant difference between the scores at both time points (p = 0.04).

Convergent Validity The S-SRQ has good convergent validity due to significant positive correlations with measures of the SR (WAI-T and SRQ) and significant negative correlations with a measure of supervisory role conflict and role ambiguity (RCRA). Two-tailed Pearson correlation coefficients (chosen as relevant assumptions were met) demonstrated a significant positive correlation with the WAI-T total score (r(189) = 0.92, p < 0.001), WAI-T Bond subscale (r(189) = 0.88, p < 0.001), WAI-T Goals subscale (r(189) = 0.89, p < 0.001), WAI-T Tasks subscale (r(189) = 0.88, p < 0.001) and SRQ total score (r(175) = 0.95, p < 0.001). It was significantly negatively correlated with the RCRA Conflict subscale (r(194) = 0.68, p < 0.001) and RCRA Ambiguity subscale (r(194) = 0.73, p < 0.001). The S-SRQ and WAI-T thus measure a similar construct (the SR), and role conflict and ambiguity are associated with a poorer SR. The S-SRQ subscales generated similar results, providing further evidence for its convergent validity.

Divergent Validity The S-SRQ demonstrated good divergent validity by not being significantly related to EPQ-R subscale scores. This indicates that it measures a different construct (the SR) to the EPQ-R (personality). Pearson correlation coefficients (chosen because relevant assumptions were met) showed that the S-SRQ was not significantly correlated with any EPQ-R subscales: psychoticism (r(184) = 0.05, p = 0.48), extraversion (r(184) = 0.08, p = 0.26), neuroticism (r(184) = 0.05, p = 0.54) and lie subscale (r(184) = 0.13, p = 0.07).

Predictive Validity The S-SRQ had good predictive validity in relation to ratings of supervision effectiveness and satisfaction. Simple and multiple stepwise linear regression analyses were undertaken to predict SSQ and IoS total scores from the supervision measures and demographic information. Relevant assumptions were met for these analyses. For SSQ total scores, the model that predicted most variance was a combination of S-SRQ scores, supervision measures and demographic variables (R2 = 0.88, F(17,174) = 75.77, p < 0.001). S-SRQ scores predicted satisfaction at a similar level as SRQ scores (R2 = 0.83, F(1,187) = 948.62, p < 0.001 and R2 = 0.85, F(1,175) = 983.78, p < 0.001, respectively) and compared with other supervision measures Copyright © 2014 John Wiley & Sons, Ltd.

alone (R2 = 0.85, F(5,183) = 217.53, p < 0.001). For IoS total scores, SRQ scores predicted most variance (R2 = 0.77, F(1,175) = 586.34, p < 0.001). S-SRQ scores predicted the same amount of variance as other supervision measures alone and other supervision measures combined with demographic variables (R2 = 0.74, F(1,182) = 511.89, p < 0.001).

DISCUSSION Main Findings The current study aimed to develop a clinically useful measure of the SR by shortening the SRQ and then determining the psychometric properties of this shorter measure. Findings suggest that the S-SRQ is a reliable, valid and acceptable supervisee-completed measure of the SR. PCA indicated that the S-SRQ has three components, which were labelled as three subscales: ‘safe base’, ‘reflective education’ and ‘structure’. The S-SRQ and its subscales demonstrated high internal reliability, and the measure also has acceptable test–retest reliability. The S-SRQ demonstrated good convergent, divergent and predictive validity, which shows that it measures a specific construct (the SR). Scores on the S-SRQ predicted satisfaction with supervision and perceived effectiveness of supervision. Supervisees found the measure easy to read and understand, with the majority rating it as being either helpful or somewhat helpful for providing feedback on the SR in supervision. Participants predicted varying levels of ‘comfort’ when using the measure.

Three Components of the Supervisory Relationship Results of this study suggest that the SR has three components. The ‘Safe Base’ component accounted for over half the variance in the S-SRQ and includes the ‘core’ characteristics of a facilitative relationship such as collaboration, respect, non-judgement and feeling safe. These factors support the supervisee to openly discuss their concerns in supervision. Numerous studies have highlighted the importance of these factors in a good SR (e.g., Beinart & Clohessy, 2009; Ladany et al., 1999a; Watkins & Riggs, 2012; White & Queener, 2003), further validating it as a key component. There are parallels between these ‘safe base’ supervisory elements and those which allow for the formation of a secure attachment (Bowlby, 1988). The extension of attachment theory to the SR has recently developed. For example, studies have shown that a supervisor’s attachment pattern can influence the quality of the SR, with less secure attachment styles associated with lower quality SRs (Dickson, Moberly, Marshall, & Reilly, 2011; Riggs & Bretz, 2006). Clin. Psychol. Psychother. 23, 77–86 (2016)

Short Supervisory Relationship Questionnaire Productive and secure attachments are also important within supervision because they can enable emotional containment for supervisees (Scaturo & Watkins, 2014). Emotional containment itself is necessary due to the emphasis on ‘experiencing’ and reflecting on the emotional content of clinical work in order to enhance learning and exploration in supervision (Milne et al., 2011). These ‘safe base’ elements of the SR gain added significance when conceptualized through an attachment framework. The ‘Reflective Education’ component refers to the processes of reflection and learning, which are important in facilitating effective supervision. They are associated with higher quality SRs (Milne, 2009). This component includes items about the supervisor’s ability to utilize a range of theoretical models and encourage their supervisee to reflect on their practice and learning needs. Also in this component is the supervisor’s ability to pay attention to the process of supervision and supervisee’s unspoken feelings and anxieties. Such ‘reflective’ factors are known to be associated with effective supervision (Bennett-Levy, McManus, Westling, & Fennell, 2009a; Watkins, 2012b) and can act as the ‘engine’ for the continual acquisition and enhancement of knowledge and skills (Bennett-Levy, Thwaites, Chaddock & Davis, 2009b). Indeed, reflective practice in supervision is seen as the primary method to develop ‘reflective practitioners’, a position related to more ‘expert’ therapists (Bennett-Levy, et al., 2009a). The ‘Structure’ component relates to the organizational aspects of supervision, and whether sessions are sufficiently focussed, structured and uninterrupted. Boundaries and structure around supervision are seen as part of effective SRs (Barnett, Erickson Cornish, Goodyear, & Lichtenberg, 2007); they may allow for a ‘containing’ safe base to be established (Palomo et al., 2010).

Support for Supervisory Relationship Models The current findings somewhat support Bordin’s (1983) supervisory working alliance model. The ‘Safe Base’ component is comparable with the ‘bond’ element, with its emphasis on ‘core’ interpersonal factors. Bordin’s ‘goal’ and ‘task’ factors are tentatively present in the ‘Safe Base’ and ‘Reflective Education’ components, with their identification of learning/training needs (‘goals’) and feedback and reflection on practice (‘tasks’). The results also support Beinart’s model (2002; 2012). This is not surprising given that items for the original SRQ were generated using this model. However, it provides empirical support for Beinart’s assumption that multiple interrelated factors help to create an effective SR. The model of Palomo et al. (2010) is also supported, with evidence for safe base, structural and reflective education components. Commitment and formative feedback Copyright © 2014 John Wiley & Sons, Ltd.

83 components were subsumed into the ‘Safe Base’ component of the S-SRQ.

Supervision Outcomes and Satisfaction As measured by the S-SRQ, the SR was associated with satisfaction with supervision and IoS and effectiveness and accounted for a substantial and significant amount of variance in these variables. This supports previous research showing that good SRs are important in determining satisfaction with supervision and higher perceived effectiveness of supervision (Britt & Gleaves, 2011; Cheon et al., 2009; Ladany et al., 1996; Palomo et al., 2010; Watkins, 1997, 2011). Supervisee’s satisfaction with supervision thus remains an important outcome to measure in supervision research (Ellis & Ladany, 1997).

Self-disclosure and the Supervisory Relationship Results indicated that approximately one-third of the participants would feel either uncomfortable or very uncomfortable discussing results of the S-SRQ within supervision. This is not wholly unexpected given research indicating that feedback and disclosure can be difficult for supervisees (Ladany et al., 1996). Lemoir (2013) found that non-disclosure can have a negative influence on the SR, supervisee learning and clinical practice. This grounded theory study also found that the quality of the SR is the most significant factor in determining whether disclosure in supervision is facilitated or not. These findings indicate that supervisors need to discuss expectations about disclosure within supervision and also in relation to using the S-SRQ. The supervision contract (Beinart, 2014) represents a concrete way of facilitating such a discussion, which ultimately would look to lower ‘discomfort’ around using the S-SRQ. Additionally, the development process of the S-SRQ, based on prior feedback on the SRQ and also the author’s judgement, omitted the most personally exposing questions. As supervisees generally do not disclose sensitive information to supervisors unless they are in ‘safe’ SRs (which can take time to develop), it was felt that more personally sensitive items would not be shared and would reduce the likelihood of completion. Although the omission of these items could be seen as a limitation of the S-SRQ, it could also be argued that both its psychometric robustness and clinical utility have been improved by lack of missing data.

Implications for Research, Clinical Practice and Training The S-SRQ could be used by training programmes to monitor the quality of SRs. It provides a method for Clin. Psychol. Psychother. 23, 77–86 (2016)

T. Cliffe et al.

84 supervisees to openly discuss their supervision and any potential difficulties experienced. However, as the SR is a dyadic process (Beinart & Clohessy, 2009), supervisor perspectives also need to be considered. The SR Measure (Pearce, Beinart, Clohessy, & Cooper, 2013) represents a reliable and valid measure to gather such information. The S-SRQ also lends itself to tracking supervisor development and adherence to SR components of supervision frameworks (Watkins, 2012b). An important issue is investigating the impact of supervision effectiveness on client outcomes, research that would add to the evidence base of supervision (Milne & Reiser, 2012). The S-SRQ could be used as part of wider research investigating the impact of supervision effectiveness and the SR on client outcomes (Milne & Reiser, 2012). Lastly, in relation to the ‘Safe Base’ component of the S-SRQ and its theoretical similarities to a secure attachment found in attachment theory, future research could further explore the links between certain attachment patterns and scores on the S-SRQ.

Limitations This study has some limitations. The measures were chosen due to their solid psychometric properties; however, others are also recommended (Inman & Ladany, 2008). The SSQ and IoS are not likely to be particularly sensitive or specific measures of client outcome and therapist behaviour. This highlights the general dearth of research and measures for studying clinical supervision and therapeutic outcomes (Watkins, 2011). Ultimately, the measures were chosen on the basis of the constructs they measured balancing this against their length and completion time. It could be argued that the S-SRQ has only one factor. The eigenvalues of the ‘reflective education’ and ‘structure’ components were above one, yet not substantially. However, accepted methods of determining the number of components were used (Cattell, 1978). The S-SRQ may also not fully capture some of the SR components in Palomo et al. (2010). There are fewer S-SRQ items regarding supervisee’s level of development and how this subsequently impacts on the SR. Perhaps the SRQ could be used if this information is needed. Concurrently, a mixed method research design may have best captured the full range of factors associated with the SR. For example, there is the potential that an unstructured, qualitative response elicited by a prompt question could capture these factors. However, it is also worth noting that all of the original questions stem from qualitative literature (Beinart, 2002). The S-SRQ is a self-report measure, which means the usual limitations of completion bias and social desirability apply. Its generalizability has not been fully explored and such research would prove useful. It remains to be seen, for example, if non-UK clinical psychology supervisees, Copyright © 2014 John Wiley & Sons, Ltd.

other applied psychology trainees or qualified clinical psychologists respond similarly to the measure. The decision to conduct a PCA may have led to bias in terms of the factors produced. It is arguable that this type of analysis may have meant that it was less likely to replicate the SRQs original factors. Results, including strength of the predictive factors, could have been influenced by the predominance of female participants, and responses may therefore be more typical of female than male supervisees. However, due to the relatively small number of male participants, analyses exploring any potential gender differences in measure scores were deemed unfeasible. Additionally, the gender balance of participants in this study represents the general gender balance in Clinical Psychology.

Conclusion This paper has detailed the development and validation of a short version of the SRQ, the S-SRQ, which is a supervisee-completed measure of the SR. The S-SRQ is a reliable, valid and acceptable measure that answers the call for more rigorously developed supervision measures and supports previous literature on the SR. The findings suggest that the SR can predict the effectiveness of, and satisfaction with, supervision. There are three components of the SR as measured by the S-SRQ: safe base, structure and reflective education. The S-SRQ is a viable measure for use in the growing area of supervision research in which few comprehensively reliable and valid measures exist. Clinically, the measure represents a quick and accessible means for assessing and discussing the quality of SRs. The S-SRQ may be useful for clinical training programmes and general clinical supervisory training.

ACKNOWLEDGEMENT The authors would like to thank Derek Milne for his face validity rating of the draft S-SRQ and Paul Griffiths for his statistical support.

REFERENCES Bahrick, A. S. (1990). Role induction for counselor trainees: Effects on the supervisory working alliance. Dissertation Abstracts International, 51, 1484–1484 (Abstract No. 1991-51645). Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006). Clinical supervision: Its influence on client-rated working alliance and client symptom reduction in the brief treatment of major depression. Psychotherapy Research, 16, 317–331. DOI: 10.1080/10503300500268524 Barnett, J. E., Erickson Cornish, J. A., Goodyear, R. K., & Lichtenberg, J. W. (2007). Commentaries on the ethical and effective practice of clinical supervision. Professional Psychology:

Clin. Psychol. Psychother. 23, 77–86 (2016)

Short Supervisory Relationship Questionnaire Research and Practice, 38, 268–275. DOI: 10.1037/07357028.38.3.268 Beinart, H. (2002). An exploration of the factors which predict the quality of the relationship in clinical supervision. (Unpublished D.Clin.Psych. thesis). U.K.: Open University/British Psychological Society. Beinart, H. (2012). Models of supervision and the supervisory relationship. In I. Fleming, & L. Steen (Eds.), Supervision and clinical psychology: Theory, practice and perspectives (pp. 36–50). Hove: Brunner Routledge. Beinart, H. (2014). Building and sustaining the supervisory relationship. In C. E. Watkins, & D. L. Milne (Eds.), The Wiley international handbook of clinical supervision. (pp. 257–281). Chichester: Wiley-Blackwell. Beinart, H., & Clohessy, S. (2009). Supervision. In H. Beinart, P. Kennedy, & S. Llewelyn (Eds.), Clinical psychology in practice (pp. 319–335). Chichester: British Psychological Society/Blackwell. Bennett-Levy, J., McManus, F., Westling, B. E., & Fennell, M. (2009a). Acquiring and refining CBT skills and competencies: Which training methods are perceived to be most effective? Behavioural and Cognitive Psychotherapy, 37, 571–583. DOI: 10.1017/S1352465809990270 Bennett-Levy, J., Thwaites, R., Chaddock, A., & Davis, M. (2009b). Reflective practice in cognitive-behavioural therapy: The engine of lifelong learning. In R. Dallos, & J. Stedmon (Eds.), Reflection in psychotherapy and counselling. Maidenhead: Open University Press. Bernard, J. M., & Goodyear, R. K. (2014). Fundamentals of clinical supervision. USA: Pearson Education. Bordin, E. S. (1983). A working alliance model of supervision. Counseling Psychologist, 11, 35–42. DOI: 10.1177/ 0011000083111007 Bowlby, J. (1988). A secure base. New York: Basic Books. Britt, E., & Gleaves, D. H. (2011). Measurement and prediction of clinical psychology students’ satisfaction with clinical supervision. The Clinical Supervisor, 30, 172–182. DOI: 10.1080/ 07325223.2011.604274 Cai, L., Thissen, & du Toit, S. (2013). IRTPRO (Version 2.1) [Computer software]. Skokie, IL: Scientific Software International, Inc. Available from http://www.ssicentral.com/irt/ Cattell, R. B. (1978). The scientific use of factor analysis. New York: Plenum. Cheon, H.-S., Blumer, M. L. C., Shih, A.-T., Murphy, M. J., & Sato, M.. (2009). The influence of supervisor and supervisee matching, role conflict, and supervisory relationship on supervisee satisfaction. Contemporary Family Therapy, 31, 52–67. DOI: 10.1007/s10591-008-9078-y Crook-Lyon, R., Heppler, A., Leavitt, L., & Fisher, L. (2008). Supervisory training experiences and overall supervisory development in predoctoral interns. The Clinical Supervisor, 27, 268–284. DOI: 10.1080/07325220802490877 Cushway, D., & Knibbs, J. (2004). Trainees’ and supervisors’ perceptions of supervision. In I. Fleming, & L. Steen (Eds.), Supervision and clinical psychology: Theory, practice and perspectives (pp. 163–187). Hove: Brunner Routledge. Dickson, J., Moberly, N., Marshall, Y., & Reilly, J. (2011). Attachment style and its relationship to working alliance in the supervision of British clinical psychology trainees. Clinical Psychology & Psychotherapy, 18, 322–330. DOI: 10.1002/cpp.715 Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and clients in clinical supervision: An integrative review. In C. E. Watkins (Ed.). Handbook of psychotherapy supervision (pp. 447–507). New York: Wiley.

Copyright © 2014 John Wiley & Sons, Ltd.

85 Ellis, M. V., D’Iuso, N., & Ladany, N. (2008). State of the art in the assessment, measurement, and evaluation of clinical supervision. In A. K. Hess, K. D. Hess, & T. H. Hess (Eds.), Psychotherapy supervision: Theory, research, and practice (pp. 473–499). New York: Wiley. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: Lawrence Erlbaum. Eysenck, S. B. G., Eysenck, H. J., & Barrett, P. (1985). A revised version of the psychoticism scale. Personality and Individual Differences, 6, 21–29. DOI: 10.1016/0191-8869(85)90026-1 Falender, C. A., & Shafranske, E. P. (2012). The importance of competency-based clinical supervision and training in the twenty-first century: Why bother? Journal of Contemporary Psychotherapy, 42, 129–137. DOI: 10.1007/s10879-011-9198-9 Fleming, I., & Steen, L. (2012). Supervision and clinical psychology: Theory, practice and perspectives. Hove: Brunner Routledge. Friedlander, M. L., & Ward, L. G. (1984). Development and validation of the supervisory styles inventory. Journal of Counseling Psychology, 31, 541–557. DOI: 10.1037//0022-0167.31.4.541 Goodyear, R. K., & Bernard, J. M. (1998). Clinical supervision: Lessons from the literature. Counselor Education and Supervision, 38, 6–17. DOI: 10.1002/j.1556-6978.1998.tb00553.x Holloway, E. L. (1995). Clinical supervision: A systems approach. Thousand Oaks, CA: Sage. Inman, A. G., & Ladany, N. (2008). Research: The state of the field. In A. K. Hess, K. D. Hess, & T. H. Hess (Eds.), Psychotherapy supervision: Theory, research, and practice (pp. 500–517). New York: Wiley Ladany, N., Ellis, M. V., & Friedlander, M. L. (1999a). The supervisory working alliance, trainee self-efficacy and satisfaction. Journal of Counseling and Development, 77, 447–455. DOI: 10.1002/j.1556-6676.1999.tb02472.x Ladany, N., & Friedlander, M. L. (1995). The relationship between the supervisory working alliance and supervisee role conflict and role ambiguity. Counselor Education and Supervision, 34, 220–231. Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature, extent, and importance of what psychotherapy trainees do not disclose to their supervisors. Journal of Counseling Psychology, 43, 10–24. DOI: 10.1037//0022-0167.43.1.10 Ladany, N., Lehrman-Waterman, D., Molinaro, M., & Wolgast, B. (1999b). Psychotherapy supervisor ethical practices: Adherence to guidelines, the supervisory working alliance, and supervisee satisfaction. The Counseling Psychologist, 27, 443–475. DOI: 10.1177/0011000099273008 Ladany, N., Walker, J. A., Pate-Carolan, L., & Gray Evans, L. (2008). Experiencing counseling and psychotherapy: Insights from psychotherapy trainees, their clients, and their supervisors. New York: Taylor & Francis. Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: Development of a general scale. Evaluation and Program Planning, 2, 197–207. DOI: 10.1016/0149-7189(79)90094-6 Lehrman-Waterman, D., & Ladany, N. (2001). Development and validation of the evaluation process within supervision inventory. Journal of Counseling Psychology, 48, 168–177. DOI: 10.1037//0022-0167.48.2.168 Lemoir, V. (2013). (Non) disclosure in the supervision of trainee clinical psychologists: A grounded theory analysis. (Unpublished D. Clin.Psych. thesis). Oxford: University of Oxford. Milne, D. (2007). An empirical definition of clinical supervision. British Journal of Clinical Psychology, 46, 437–447. DOI: 10.1348/014466507X197415

Clin. Psychol. Psychother. 23, 77–86 (2016)

T. Cliffe et al.

86 Milne, D. L. (2009). Evidence-based clinical supervision: Principles and practice. Malden, MA: British Psychological Society/Blackwell Milne, D., & Reiser, R. P. (2012). A rationale for evidence-based clinical supervision. Journal of Contemporary Psychotherapy, 42, 139–149. DOI: 10.1007/s10879-011-9199-8 Milne, D. L., Reiser, R. P., Cliffe, T., & Raine, R. (2011). SAGE: Preliminary evaluation of an instrument for observing competence in CBT supervision. The Cognitive Behaviour Therapist, 4, 123–138. DOI: 10.1017/S1754470X11000079 O’Donovan, A., & Kavanagh, D. J. (2014). Measuring competence in supervisees and supervisors: Satisfaction and related reactions in supervision. In C. E. Watkins, & D. L. Milne (Eds.), The Wiley international handbook of clinical supervision (pp. 458–467). Chichester: Wiley-Blackwell. Olk, M., & Friedlander, M. L. (1992). Trainees’ experience of role conflict and role ambiguity in supervisory relationships. Journal of Counseling Psychology, 39, 389–397. DOI: 10.1037//00220167.39.3.389 Palomo, M., Beinart, H., & Cooper, M. (2010). Development and validation of the supervisory relationship questionnaire (SRQ) in UK trainee clinical psychologists. British Journal of Clinical Psychology, 49, 131–149. DOI: 10.1348/014466509X441033 Pearce, N., Beinart, H., Clohessy, S., & Cooper, M. (2013). Development and validation of the supervisory relationship measure: A self-report questionnaire for use with supervisors. British Journal of Clinical Psychology, 52, 249–268. DOI: 10.1111/ bjc.12012 Ramos-Sanchez, L., Esnil, E., Goodwin, A., Riggs, S., Touster, L. O., Wright, L. K., Ratanasiripong, P., Rodolfa, E. (2002). Negative supervisory events: Effects on supervision satisfaction and supervisory alliance. Professional Psychology: Research and Practice, 33, 197–202. DOI: 10.1037//0735-7028.33.2.197 Riggs, S., & Bretz, K. (2006). Attachment processes in the supervisory relationship: An exploratory investigation. Professional Psychology: Research & Practice, 37, 558–566. DOI: 10.1037/ 0735-7028.37.5.558 Scaturo, D. J., & Watkins, C. E. (2014). Supervising integrative and eclectic psychotherapies. In C. E. Watkins, & D. L. Milne (Eds.), The Wiley international handbook of clinical supervision (pp. 552–575). Chichester: Wiley-Blackwell. Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55, 167–194. DOI: 10.1111/j.17446570.2002.tb00108.x Tabachnik, B. G., & Fidell, L. S. (2003). Using multivariate statistics. New York: Wiley. Van der Linden, W. J., & Hambleton, R. K. (2010). Handbook of modern item response theory. New York: Springer. Wainwright, N. (2010). The development of the Leeds alliance in supervision scale (LASS): A brief sessional measure of the supervisory alliance (Unpublished D.Clin.Psych. thesis). Leeds: University of Leeds. Watkins, C. E. (Ed.). (1997). Handbook of psychotherapy supervision. New York: Wiley. Watkins, C. E. (2011). Does psychotherapy supervision contribute to patient outcomes? Considering thirty years of research. The Clinical Supervisor, 30, 235–256. DOI: 10.1080/ 07325223.2011.619417 Watkins, C. E. (2012a). Contemporary visions of psychotherapy supervision: Sharing perspective, identifying need, and charting possibility. Journal of Contemporary Psychotherapy, 42, 125–127. DOI: 10.1007/s10879-011-9203-3

Copyright © 2014 John Wiley & Sons, Ltd.

Watkins, C. E. (2012b). Psychotherapy supervision in the new millennium: Competency-based, evidence-based, particularized, and energized. Journal of Contemporary Psychotherapy, 42, 193–203. DOI: 10.1007/s10879-011-9202-4 Watkins, C. E., & Riggs, S. A. (2012). Psychotherapy supervision and attachment theory: Review, reflections, and recommendations. Clinical Supervisor, 31, 256–289. DOI: 10.1080/ 07325223.2012.743319 Weaks, D. (2002). Unlocking the secrets of ‘good supervision’: A phenomenological exploration of experienced counsellors’ perceptions of good supervision. Counselling and Psychotherapy Research, 2, 33–39. DOI: 10.1080/14733140212331384968 Wheeler, S., Aveline, M., & Barkham, M. (2011). Practice-based supervision research: A network of researchers using a common toolkit. Counselling and Psychotherapy Research, 11, 88–96. DOI: 10.1080/14733145.2011.562982 White, V. E., & Queener, J. (2003). Supervisor and supervisee attachments and social provisions related to the Supervisory Working Alliance. Counselor Education and Supervision, 42, 203–218. DOI: 10.1002/j.1556-6978.2003.tb01812 White, E., & Winstanley, J. (2010). A randomised controlled trial of clinical supervision: Selected findings from a novel Australian attempt to establish the evidence base for causal relationships with quality of care and patient outcomes, as an informed contribution to mental health nursing practice development. Journal of Research in Nursing, 15, 151–167. DOI: 10.1177/ 1744987109357816

APPENDIX 1:

Clin. Psychol. Psychother. 23, 77–86 (2016)

Development and Validation of a Short Version of the Supervisory Relationship Questionnaire.

The Supervisory Relationship Questionnaire (SRQ) is one of the few theoretically sound and psychometrically valid questionnaires for measuring the SR ...
3MB Sizes 1 Downloads 5 Views