This article was downloaded by: [The University of Manchester Library] On: 13 October 2014, At: 15:18 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Intellectual and Developmental Disability Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/cjid20
The Equity Perception Scale – Intellectual Disability Services (EPS-IDS): Evaluating the reliability and validity of a new measure a
a
b
Philip Disley , Chris Hatton & Dave Dagnan a
Lancaster University, UK
b
West Cumberland Hospital, Whitehaven, Cumbria, UK Published online: 09 Jul 2013.
To cite this article: Philip Disley, Chris Hatton & Dave Dagnan (2013) The Equity Perception Scale – Intellectual Disability Services (EPS-IDS): Evaluating the reliability and validity of a new measure, Journal of Intellectual and Developmental Disability, 38:4, 292-300, DOI: 10.3109/13668250.2013.807909 To link to this article: http://dx.doi.org/10.3109/13668250.2013.807909
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Journal of Intellectual & Developmental Disability, 2013 Vol. 38, No. 4, 292–300, http://dx.doi.org/10.3109/13668250.2013.807909
ORIGINAL ARTICLE
The Equity Perception Scale – Intellectual Disability Services (EPSIDS): Evaluating the reliability and validity of a new measure
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
PHILIP DISLEY1, CHRIS HATTON1 & DAVE DAGNAN2 1
Lancaster University, UK, and 2West Cumberland Hospital, Whitehaven, Cumbria, UK
Abstract Background The aim of this paper was to evaluate the reliability and validity of the Equity Perception Scale – Intellectual Disability Services (EPS-IDS), a detailed measure of staff equity perceptions in services for people with intellectual disability (ID). The EPS-IDS contains 3 relationship indices that assess staff relationships with their employing organisation, their co-workers, and service users. Method Postal questionnaires were used to collect quantitative data from a purposive sample of ID service staff. Results The EPS-IDS was found to have good internal consistency reliability. The overall test–retest reliability of the measure, however, was found to be less satisfactory. With the exception of the co-worker relationship index, the EPS-IDS indices were found to have promising validity. Conclusions The findings from the study suggest that the EPS-IDS shows some promise as a measure of the equity perceptions of ID service staff. Further refinements to the relationship indices, however, may be required.
Keywords: Equity theory, intellectual disability, staff
Introduction A small number of studies involving staff working in services for people with intellectual disability (ID) have focused upon their perceptions of how equitably, or fairly, they believe they are treated, or rewarded, in their work-based relationships (e.g., Thomas & Rose, 2010; van Dierendonck, Schaufeli, & Buunk, 1996, 1998; van Yperen, 1996; van Yperen, Buunk, & Schaufeli, 1992). Typically, these studies use equity theory (Adams, 1965) as their theoretical framework. Briefly, this theory proposes that individuals evaluate their social relationships by comparing their perceived ratio of inputs (i.e., investments) and outcomes (i.e., gains) to the corresponding ratio of a comparison other. An individual will view a relationship as equitable if they perceive a match between the input/ outcome ratio of themselves and the input/outcome ratio of their comparison other. If a mismatch is perceived, the individual will experience distress and will be motivated to try to restore equity. According to Adams (1965), the individual can restore equity
in a number of ways, including (a) attempting to alter their own or their comparison other’s inputs and outcomes, (b) changing their comparison other, or (c) leaving the field. In research involving ID service staff, staff equity perceptions have been found to be related to intention to leave (van Dierendonck et al., 1998; van Yperen, 1996), absenteeism (van Dierendonck et al., 1998), and burnout (e.g., Thomas & Rose, 2010; van Dierendonck et al., 1996, 1998; van Yperen, 1996; van Yperen et al., 1992). Research has also found that an intervention program utilising the principles of equity theory had a positive impact upon the wellbeing (level of burnout) and behaviour (length of absence duration) of ID service staff (van Dierendonck et al., 1998). Recent reviews in the ID literature have suggested that equity theory may have some utility as a theoretical framework for integrating various strands of research on staff (Disley, Hatton, & Dagnan, 2009) and that further research using equity theory as a theoretical framework is warranted (Devereux, Hastings, & Noone, 2009).
Correspondence: Philip Disley, Division of Health Research, Lancaster University, LA1 4YG, UK. E-mail: [email protected] © 2013 Australasian Society for Intellectual Disability, Inc.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale Previous research investigating the equity perceptions of ID service staff has used global, often single-item, measures of equity and detailed measures of equity that have been developed for other occupational groupings. As yet, however, no detailed measure of equity perceptions designed specifically for ID service staff is currently available. Such a measure may have a number of advantages over existing equity measures. Compared to global measures, a detailed measure, designed specifically for ID service staff, may be considered a more reliable measure and may capture more information. For example, unlike single-item measures, multi-item measures can provide data on their internal consistency reliability (Churchill, 1979) and, according to classical test theory, are less susceptible to random measurement error (Nunnally & Bernstein, 1994). Unlike multi-item detailed measures of equity, global measures do not provide information on specific inputs and outcomes. Consequently, the reasons why staff perceive their work-based relationships to be unbalanced cannot be ascertained. Within the context of ID research, the main advantage a detailed equity measure developed specifically for ID service staff would have over measures developed for other occupational groupings is that it is likely to have better content validity (i.e., it is more likely to contain items that are relevant to ID service staff and less likely to contain items that are not relevant). In light of these advantages, a detailed equity measure developed specifically for ID service staff may be considered a more appealing alternative to other available equity measures.
293
significantly, with their global counterparts. The discriminant validity of the EPS-IDS was assessed in two ways. First, the discriminant validity of the EPS-IDS was tested by investigating intercorrelations between (a) the EPS-IDS relationship indices, (b) the EPS-IDS input subscales, and (c) the EPSIDS outcome subscales. Second, the degree to which the EPS-IDS scales correlated with a measure of social desirability was investigated. It was predicted that only small correlations would be found between the variables when the discriminant validity of the EPS-IDS was tested. The concurrent validity of the EPS-IDS was tested by investigating the degree to which scores on the EPS-IDS relationship indices correlated with variables that had previously been found to be related to equity perceptions: staff burnout, intention to leave, organisational commitment, staff emotion (positive and negative affect), and job satisfaction (e.g., Geurts, Schaufeli, & Rutte, 1999; Thomas & Rose, 2010; van Dierendonck et al., 1998). The predictive validity of the EPS-IDS was determined by investigating the relationship of the measure with staff performance. It was predicted that scores on the EPS-IDS relationship indices would be positively associated with organisational commitment, personal accomplishment, positive affect, job satisfaction, and performance, and negatively associated with emotional exhaustion, depersonalisation, negative affect, and intention to leave.
Method Design
The aim of the current study The aim of the current study was to evaluate the reliability and validity of the Equity Perception Scale – Intellectual Disability Services (EPS-IDS), a closed format self-report measure designed to assess the equity perceptions of staff working in ID services. Both the internal consistency reliability and the test–retest reliability of the EPS-IDS were assessed. It was assumed that the measure would have reasonable internal consistency and that, barring changes in their circumstances, the scores of staff on the measure would remain stable over the test–retest period. With respect to the validity of the EPS-IDS, the convergent validity of the measure was tested by comparing the degree to which the EPS-IDS scales (the relationship indices and the subscales) correlated with similar global measures of equity. It was predicted that the EPS-IDS relationship indices and subscales would correlate positively, and
A within-participants design was used that had crosssectional and longitudinal components. Data were collected using postal questionnaires. Questionnaires were completed by both staff and their line managers.
Participants Main staff survey. Four hundred and nine ID service staff from the north of England were invited to participate in the research. Of the 409 individuals who were invited to take part in the study, 194 agreed to participate. Of the 194 staff who agreed to participate, 146 staff completed and returned the questionnaire they were sent. Three of the returned questionnaires were excluded because they were filled in incorrectly, leaving 143 usable questionnaires (response rate = 35%). Seventy-nine percent of the participants were female and the mean age was 41 years. The mean lengths of time participants had been working in their current posts, working
294
P. Disley et al.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
for their current employer, and working in services for people with ID were 3 years and 10 months, 6 years and 4 months, and 8 years, respectively. The sample comprised 109 support staff and 34 staff in other job roles (e.g., nurses, occupational therapists, and managers). Test–retest reliability. One hundred and two staff were sent a second copy of the EPS-IDS to complete approximately 4 weeks after they had returned the main staff survey. Seventy-nine questionnaires were returned, of which 78 were usable (response rate = 80%). Seventy-seven percent of the participants were female and the mean age was 42 years. The mean lengths of time participants had been working in their current posts, working for their current employer, and working in services for people with ID were 3 years and 10 months, 6 years and 3 months, and 7 years and 6 months, respectively. The sample comprised 60 support staff and 18 staff in other job roles (e.g., nurses, occupational therapists, and managers). Performance assessment. A subset of 45 housing support staff, who took part in the main survey, had their performance at work assessed. Eighty-nine percent of the participants were female and the mean age was 42 years. The mean lengths of time participants had been working in their current posts, working for their current employer, and working in services for people with ID were 3 years and 8 months, 5 years and 9 months, and 7 years and 3 months, respectively. Measures The Equity Perception Scale – Intellectual Disability Service (EPS-IDS). The EPS-IDS contains 53 items (the scale is available from the first author upon request). Items for the scale were generated from semistructured interviews with 15 ID service staff comprising seven support staff and eight staff from other job roles (i.e., nurses, occupational therapists, and managers). A draft version of the EPS-IDS was then reviewed by seven ID service staff comprising three support staff and four in nursing and managerial roles (see Disley, 2011, and Disley, Hatton, & Dagnan, 2012, for more detailed accounts of the development of the scale). The scale contains three relationship indices (relationships with the employing organisation, co-workers, and service users), each of which contains two subscales: one assessing staff inputs and one to assess staff outcomes. The employer, co-worker, and service user inputs subscales contain, respectively, 15, nine, and five
items. All the outcomes subscales contain eight items. All the items are rated on a 5-point scale, which contains word anchors at each point (1= very little and 5 = very much). Examples of items from the subscales are: “How much do you give your employer in terms of the effort you put into your job” (inputs/employer); “How much do you give your co-workers in terms of encouragement?” (inputs/co-worker); “How much do you give service users in terms of the support you give them?” (inputs/service user); “How much do you get from your employer in terms of the job security you have?” (outcomes/employer); “How much do you get from your co-workers in terms of advice about work issues?” (outcomes/co-worker); “How much do you get from service users in terms of appreciation?” (outcomes/service user). Each relationship index is scored separately. Scores are calculated by dividing the percentage score on the outcomes subscale by the percentage score on the inputs subscale. For example, a participant scores 35 out of 45 and 28 out of 40 on the input and outcomes subscales of the co-worker index, respectively. The percentage score for the input subscale would be 78% (35/45 × 100). The percentage score for the outcome subscale would be 70% (28/40 × 100). Dividing the percentage score of the outcome subscale (70%) by the percentage score on the input subscale (78%) would produce an index score of 0.90. A ratio score lower than 1.00 indicates that the individual perceives themselves to be under-benefited in the relationship (the lower the score the more under-benefited the individual perceives themselves to be). A ratio score of 1.00 indicates that the individual perceives the relationship to be equitable. A ratio score higher than 1.00 indicates that the individual perceives themselves to be over-benefited in the relationship (the higher the score the more over-benefited the individual perceives themselves to be). Other measures used in the study The following measures were completed by staff who took part in the main survey (see Disley, 2011, for more detailed descriptions of the measures). Organisational commitment. The nine-item Organisational Commitment Questionnaire (Mowday, Steers, & Porter, 1979) was used to assess staff loyalty to their employing organisation. Global equity measure. Staff global equity perceptions were assessed with a six-item measure developed by van Horn, Schaufeli, and Enzmann (1999). Each work-based relationship (employer, co-workers, and
The Equity Perception Scale service users) is assessed by two items: a global investment item and a global outcome item.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
Burnout. Burnout was assessed using the 22-item Human Services version of the Maslach Burnout Inventory (MBI-HS; Maslach, Jackson, & Leiter, 1996). The MBI-HS contains three separately scored subscales: emotional exhaustion (MBI-EE; nine items), depersonalisation (MBI-D; five items), and personal accomplishment (MBI-PA; eight items). Staff emotions. The affective state of participants was assessed with the 20-item Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). The PANAS contains two 10-item subscales: positive affect and negative affect. Job satisfaction. A six-item shortened version of a scale developed by Brayfield and Rothe (1951) was used to measure job satisfaction. Intention to leave. Intention to leave the job was assessed using the two-item “propensity to leave” scale. The scale utilises items developed by Lyons (1971) and Seashore, Lawler, Mirvis, and Cammann (1982).
295
complete, and a Freepost return envelope. All completed forms and questionnaires were returned directly to the researchers. Direct line managers (performance assessment). After the main staff questionnaire had been returned, a questionnaire pack was posted to the line managers of the housing support staff who had agreed to have their performance assessed. The pack contained a cover letter, one or more questionnaires, and a stamped addressed envelope. Each questionnaire contained a tear-off sheet that contained the name of the member of staff who had agreed to have their performance assessed. Completed questionnaires were returned directly to the researchers. Upon receipt of the questionnaire, the researchers replaced the tear-off sheet with a personal identification number.
Ethical approval. Ethical approval for the study was sought and obtained from a National Health Service Local Research Ethics Committee (Cumbria and Lancashire Research Ethics Committee A).
Data analysis Social desirability. A short version (11 items) of the Marlowe-Crowne Social Desirability Scale developed by Ballard (1992; Scale 1) was used to assess social desirability. Demographic information. Single-item measures were used to collect demographic information from staff (e.g., gender, age, length of time working in current post, length of time working for current employer, length of time working in ID services, and job title). The following measure was completed by the line managers of housing support staff. Job performance rating. The performance of housing support staff was assessed with the 23-item job performance measure developed by Hatton, Wigham, and Craig (2009). Procedure Staff (main survey/test–retest). Questionnaire packs, containing a cover letter, the main survey, and a Freepost return envelope, were distributed to participants by post. Approximately 4 weeks after they had returned the main staff survey, a subset of staff was sent another questionnaire pack. This pack contained a cover letter, a second copy of the EPS-IDS to
All data were analysed using SPSS Version 14.0 (SPSS Inc., Chicago, IL, USA). Given the number of statistical tests conducted in this study, a p value of < .01 was taken to be statistically significant. The distribution of scores on several of the scales failed to meet parametric assumptions; consequently, nonparametric statistical tests were used in the analysis. A combination of one-tailed and two-tailed tests were used. One-tailed tests were used in cases where directional hypotheses had been made (e.g., when the convergent, concurrent, and predictive validity of the EPS-IDS was being tested). Two-tailed tests were conducted when no explicit directional hypothesis was forwarded (e.g., when the discriminant validity of the EPS-IDS was being assessed). In order to assess the internal consistency reliability of the EPS-IDS, both Cronbach’s alpha coefficients (Cronbach, 1951) and interitem correlations were calculated for each of the measure’s six subscales. Spearman rank order correlations and Wilcoxon signed-rank tests were used to assess the test–retest reliability of the EPS-IDS. Spearman rank order correlations were calculated to determine the convergent validity, discriminant validity, concurrent validity, and predictive validity of the EPSIDS.
296
P. Disley et al. all cases the total score at Time 2 was significantly lower than at Time 1.
Results Descriptive statistics The means, standard deviations (SD), and ranges for the EPS-IDS relationship indices and subscales are presented in Table 1.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The internal consistency of the EPS-IDS All of the EPS-IDS subscales were found to have good internal consistency reliability (see Table 1). The Cronbach’s alphas for the subscales were all higher than the threshold of .70 recommended by Kline (1993). The mean interitem correlations for the subscales were all higher than the threshold of .30 recommended by Robinson, Shaver, and Wrightsman (1991).
The convergent validity of the EPS-IDS With the exception of the co-worker relationship index (r = .07), all of the EPS-IDS relationship indices (employer, r = .49, and service user, r = .36) and subscales (inputs/employer, r = .41, inputs/co-worker, r = .41, inputs/service user, r = .40, outcomes/ employer, r = .55, outcomes/co-worker, r = .27, and outcomes/service user, r = .45) were found to correlate positively and significantly with their global counterparts. These findings suggest that, with the exception of the co-worker relationship index, all of the EPSIDS scales (the relationship indices and the subscales) and their global counterparts were measuring similar constructs.
The test–retest reliability of the EPS-IDS The mean average duration between receipt of the first questionnaire by the researchers and the receipt of the second questionnaire was 41 days (SD = 4.91, range: 33–54 days). All of the relationship indices and the outcome subscales were found to have satisfactory test–retest reliabilities (see Table 1). The test–retest reliabilities for the inputs subscales, however, were found to be less satisfactory. Wilcoxon signed-rank tests found significant differences in the test–retest scores for the co-worker relationship index, the co-worker outcomes subscale, and the service user inputs subscale (see Table 1). In
The discriminant validity of the EPS-IDS The first test of the discriminant validity of the EPSIDS involved calculating the intercorrelations between (a) the EPS-IDS relationship indices, (b) the EPS-IDS inputs subscales, and (c) the EPSIDS outcomes subscales. The correlations between the EPS-IDS relationship indices were all within the range of −.20 to .20 recommended by Mitchell and Jolley (2009) (employer/co-worker, r = −.11; employer/service user, r = .18; co-worker/service user, r = .02). These findings suggest the three EPS-IDS
Table 1. Descriptive, internal consistency reliability, and test–retest statistics for EPS-IDS
EPS-IDS relationship scales
Actual mean (percentage mean)
EPS-IDS (Employer) Inputs 68.97 (91.9%) Outcomes 30.17 (75.4%) Relationship 0.81 index EPS-IDS (Co-worker) Inputs 41.67 (92.6%) Outcomes 33.70 (84.3%) Relationship 0.91 index EPS-IDS (Service user) Inputs 23.45 (93.8%) Outcomes 27.81 (69.5%) Relationship 0.76 index
Mean score Time 1 (test– retest)
Mean score Time 2 (test– retest)
Cronbach’s alpha
Mean interitem correlation
Spearman’s rank order correlations (test–retest)
Wilcoxon signed-rank test (test– retest)
6.12 49–75 6.17 11–60 0.14 0.47–1.07
.93 .84 n/a
.48 .38 n/a
.49∗ .77∗ .89∗
−0.24 −0.66 −0.34
91.95 75.35 0.83
92.00 76.23 0.82
3.51 33–45 5.17 16–40 0.10 0.71–1.14
.90 .93 n/a
.51 .63 n/a
.42∗ .61∗ .62∗
−1.28 −2.66∗ −3.30∗
92.77 85.09 0.92
91.57 81.82 0.89
2.33 14–25 6.62 11–40 0.15 0.46–1.05
.84 .90 n/a
.58 .52 n/a
.41∗ .73∗ .76∗
−4.13∗ −0.46 −1.42
94.76 71.05 0.76
90.87 70.27 0.78
SD
Range
Note. Actual mean (percentage mean), SD, range, Cronbach’s alpha, and mean interitem correlation, n =143; all others, n = 78. p < .01, two-tailed.
∗
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale relationship indices are measuring conceptually distinct relationships. The correlations between the EPS-IDS inputs subscales were all outside the recommended range (employer/co-worker r = .68; employer/service user, r = .57; co-worker/service user, r = .59). With regard to the EPS-IDS outcomes subscales, only the two correlations involving the co-workers subscale were within the recommended range (employer/co-worker, r = .01; employer/service user, r = .44; co-worker/service user, Spearman’s r = .04). These data suggest that the outcomes/ co-workers subscale is the only subscale measuring a conceptually distinct relationship. The second test of the discriminant validity of the EPS-IDS involved investigating relationships between the EPS-IDS indices and subscales and a measure of social desirability. The correlations found for all three of the indices (employer relationship index, r = −.02, co-worker relationship index, r = .02, and service user relationship index, r = −.14) and four of the six subscales (inputs/co-worker, r = .19, outcomes/employer, r = .11, outcomes/coworker, r = .10, and outcomes/service user, r = −.01) were found to be within the recommended range of −.20 to .20. The correlations found for inputs/employer subscale (r = .27) and the inputs/ service user subscale (r = .23) were found to be outside the recommended range. These findings suggest that only the inputs/employer and inputs/service user subscales are affected by social desirability bias. The concurrent validity of the EPS-IDS The means, standard deviations, and internal consistency reliabilities for the measures used to assess the concurrent (and predictive) validity of the EPSIDS are shown in Table 2.
297
Scores on the EPS-IDS employer index were found to be positively correlated with organisational commitment and negatively correlated with emotional exhaustion and intention to leave (see Table 2). Staff who scored higher on the EPS-IDS employer relationship index (i.e., felt less under-benefited) had higher levels of organisational commitment, lower levels of emotional exhaustion, and had less intention to leave their job. Scores on the co-worker relationship index were found to be positively correlated with personal accomplishment and intention to leave and negatively correlated with organisational commitment (see Table 2). Staff who scored higher on the EPS-IDS co-worker relationship index (i.e., felt less under-benefited) had higher levels of personal accomplishment, lower levels of organisational commitment, and a greater intention to leave the job. Scores on the EPS-IDS service user relationship index were found to be positively correlated with organisational commitment, job satisfaction, and positive affect, and negatively correlated with emotional exhaustion, negative affect, and intention to leave (see Table 2). Staff that scored higher on the service user relationship index (i.e., felt less under-benefited) had higher levels of organisational commitment, job satisfaction, and positive affect; lower levels of emotional exhaustion and negative affect; and had less intention to leave their job. The findings suggest that both the employer and service user relationship indices of the EPS-IDS have acceptable concurrent validity. The concurrent validity of the co-worker relationship index was poor because two out of three of the correlations (organisational commitment and intention to leave) were in the opposite direction to what was predicted.
Table 2. Correlations to assess the concurrent and predictive validity of the ESP-IDS relationship indices Mean (SD) Organisational Commitment Emotional Exhaustion Depersonalisation Personal Accomplishment Positive Affect Negative Affect Job Satisfactiona Intention to Leave Performance
34.63 (8.03) 13.92 (8.40) 1.84 (2.21) 36.66 (7.44) 38.37 (6.85) 15.29 (4.59) 12.60 (4.43) 4.41 (2.22) 43.38 (6.35)
EPS-IDS Cronbach’s alpha coefficient EPS-IDS (Employer) EPS-IDS (Co-worker) (Service user) .94 .91 .72 .83 .92 .84 .86 .88 .93
0.55∗ −0.31∗ 0.04 0.09 0.08 −0.01 −0.04 −0.42∗ 0.36∗
−0.25∗ 0.17 0.10 0.25∗ −0.04 0.11 0.11 0.20∗ −0.06
Note. All of the tests n =143 except performance (n = 45). Statistical test used: Spearman’s rank order correlations. Higher scores = less job satisfaction. ∗ Correlation is significant at the .01 level (one-tailed). a
0.27∗ -0.35∗ −0.14 0.05 0.34∗ −0.31∗ −0.24∗ −0.29∗ 0.03
298
P. Disley et al.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The predictive validity of the EPS-IDS The mean average duration between receipt of the EPS-IDS data by the researchers and the receipt of the performance data was 41 days (SD = 4.56, range: 33–51 days). Only one of the three EPS-IDS relationship indices was found to have acceptable predictive validity (see Table 2.). A significant positive correlation was found between the employer relationship index and performance rating—the less underbenefited staff perceived themselves to be in their relationship with their employer the better their assessed performance at work was. Scores on the coworker and service user relationship indices were not significantly correlated with performance rating.
Discussion This study provides preliminary data on the reliability and validity of the EPS-IDS, a self-report measure that assesses the equity perceptions of staff working in ID services. Overall, the findings relating to the psychometric properties of the EPS-IDS are encouraging. The reliability analysis of the EPS-IDS found that all six of the EPS-IDS subscales had good internal consistency reliability. This finding suggests that the items used in each of the EPS-IDS subscales measure the same underlying constructs and consequently can be used as scales. The data relating to the test–retest reliability of the EPS-IDS, however, was less convincing. None of the three relationship indices were found to have entirely satisfactory test– retest reliability. The test–retest reliability coefficients for all of the input subscales of the EPS-IDS were lower than expected. Also, the test–retest scores for the co-worker relationship index, coworker outcomes subscale, and service user inputs subscale were found to differ significantly, suggesting that the scores on these measures do not remain stable over time. It is possible, however, that that differences found in the scores at Time 1 and Time 2 may be the result of actual changes in staff equity perceptions rather than random error. It may be advisable for future studies investigating the test–retest reliability of the EPS-IDS to monitor participants over the course of the study to determine whether there have been any changes between administrations in their perceptions of their work-based relationships. With regard to the validity of the EPS-IDS, both the service user and employer scales were found to have acceptable convergent validity. The findings relating to the convergent validity of the EPS-IDS co-worker scales, however, were less positive. Only the co-worker inputs subscale was found to correlate
significantly with its global counterpart. The correlation found between the EPS-IDS co-worker relationship index and its global counterpart was nonsignificant, and the correlation found between the co-worker outcomes subscale and its global counterpart was significant but relatively small. It is possible that the items in the co-worker outcomes subscale were not entirely relevant to some, or all, of the participants, and this had an impact on the relationships found for the co-worker outcomes subscale and the co-worker index. It should be pointed out, however, that a number of steps were taken during the development process to ensure that the content of the co-worker index was relevant to ID service staff (see the Measures section). With respect to the discriminant validity of the EPSIDS, many of the correlations between the measure’s subscales were found to be significant and of a relatively large magnitude. These findings suggest that the EPS-IDS subscales are often measuring similar, as opposed to distinct, constructs and consequently appear to have poor discriminant validity. It is possible, however, that the poor discriminant validity of the EPS-IDS may be the result of conceptual issues within equity theory research in general rather than deficits within the EPS-IDS. Specifically, staff may view their work-based relationships in a different way to how they are traditionally conceptualised in equity research (i.e., staff in the current study might have used broader categories for their work-based relationships than the traditionally used categories of “relationship with employer,” “relationship with coworkers,” and “relationship with service users”). Both the employer inputs and service user inputs subscales were found to correlate significantly with a measure of social desirability. An important implication from this finding is that researchers who use the EPS-IDS (especially the employer and service user relationship indices) would be advised to control for social desirability effects. The concurrent validity of the EPS-IDS was established by investigating the relationships between its indices and a number of variables that have previously been found to be correlated with staff equity perceptions. As with previous research (e.g., Geurts et al., 1999; Thomas & Rose, 2010; van Dierendonck et al., 1998), staff equity perceptions, as measured by the EPS-IDS, were found to be related to organisational commitment, emotional exhaustion, personal accomplishment, positive affect, negative affect, job satisfaction, and intention to leave. Only the employer(s) and service user indices of the EPSIDS, however, appear to have acceptable concurrent validity. Scores on these two indices were found to correlate in the predicted direction, with scores on
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale a number of variables, as found in previous research, to be significantly related to equity perceptions. Two of the three significant correlations found for the coworker relationship index, however, were in the opposite direction to what was predicted. Counterto-predictions scores on the co-worker index were negatively correlated with organisational commitment and positively correlated with intention to leave. These findings are perplexing and difficult to accommodate within the framework of traditional equity theory. One possible explanation, outside the framework of equity theory, is that when staff perceive the work environment in a negative way they look for more support from, and give more support to, their co-workers. In terms of the predictive validity of the EPS-IDS, the finding that only the employer relationship index correlated significantly with performance was disappointing but not entirely unsurprising. The evaluation of the predictive validity of the EPS-IDS requires further research. Given that the concurrent validity data on the EPS-IDS suggest that its relationship indices (a) correlate with a broad range of staff variables and (b) vary in terms of what staff variables they correlate with, it is entirely possible that the predictive validity findings would have been more positive if a broader range of staff variables had been used. The performance measure used in the current study also focused upon a relatively small number of the core competencies needed by support staff in ID services (see Hatton et al., 2009). It is possible, therefore, that a different, more positive pattern of findings would have emerged if a wider range of staff competencies had been assessed. The current study had a number of limitations. For example, design issues, such as the sampling strategy used, mean that one needs to be cautious about generalising the study’s findings. The low response rate found in the current study may have had an impact upon the power of its statistical analysis. The use of nonparametric tests in the study meant that possible confounding variables (such as social desirability) could not be controlled for. Data relating to the predictive validity of the EPS-IDS was only obtained from housing support staff. Future studies on the predictive validity of measure should recruit ID service staff from a broader range of roles. Future studies on the EPS-IDS could also replicate and extend on the findings reported in the current study. For example, the reliability and validity of the EPS-IDS could be assessed independently, in different settings, on larger samples, and on specific occupational groupings working in ID services. The test–retest reliability of the scale also needs to be assessed over longer periods.
299
In conclusion, the current study aimed to evaluate the reliability and validity of the EPS-IDS, a measure designed to assess the equity perceptions of staff working in ID services. The findings from the study suggest that the EPS-IDS shows some promise as a measure of the equity perceptions of ID service staff. Further refinements to the relationship indices, however, may be required. Once refined, the EPS-IDS should prove to be a measure that is easy to complete, quick to do (taking approximately 10 minutes to complete), and easy to score. The EPS-IDS has a number of potential uses. First, it can be used in research that investigates the equity perceptions of staff working in ID services. Second, the measure can be used by operators of ID services to gain insights into the equity perceptions of their staff, to determine the impact of existing and/or new procedures on staff equity perceptions, and can be used as a guide for planning. Third, the measure can be used to guide and evaluate interventions that aim to improve the equity perceptions of staff working in ID services. Author note This paper was funded by a grant from the Economic and Social Research Council, UK. The funding body has placed no restrictions on the access to, or publication of, the research data. Conflict of interest: None. References Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 2, pp. 267–299). New York, NY: Academic Press. Ballard, R. (1992). Short forms of the Marlowe-Crowne Social Desirability Scale. Psychological Reports, 71, 1155–1160. Brayfield, A. H., & Rothe, H. F. (1951). An index of job satisfaction. Journal of Applied Psychology, 35, 307–311. doi:10.1037/ h0055617 Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 64–73. doi:10.2307/3150876 Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. doi:10.1007/BF02310555 Devereux, J., Hastings, R., & Noone, S. (2009). Staff stress and burnout in intellectual disability services: Work stress theory and its application. Journal of Applied Research in Intellectual Disabilities, 22, 561–573. doi:10.1111/j.1468-3148.2009. 00509.x Disley, P. (2011). Applying equity theory to staff in learning disability services (Unpublished doctoral dissertation). Lancaster University, Lancashire, UK. Disley, P., Hatton, C., & Dagnan, D. (2009). Applying equity theory to staff working with individuals with intellectual disabilities. Journal of Intellectual & Developmental Disability, 34, 55– 66. doi:10.1080/13668250802684701
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
300
P. Disley et al.
Disley, P., Hatton, C., & Dagnan, D. (2012). Inputs and outcomes: What do staff in services for people with intellectual disabilities perceive they bring to and receive from their work-based relationships? Journal of Intellectual Disabilities, 16, 297–306. doi:10.1177/1744629512466793 Geurts, S. A., Schaufeli, W. B., & Rutte, C. G. (1999). Absenteeism, turnover intention and inequity in the employment relationship. Work & Stress, 13, 253–267. doi:10.1080/ 026783799296057 Hatton, C., Wigham, S., & Craig, J. (2009). Developing measures of job performance for support staff in housing services for people with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 22, 54–64. doi:10.1111/j.14683148.2008.00439.x Kline, P. (1993). The handbook of psychological testing. London, UK: Routledge. Lyons, T. F. (1971). Role clarity, need for clarity, satisfaction, tension, and withdrawal. Organizational Behavior and Human Performance, 6, 99–110. doi:10.1016/0030-5073(71)90007-9 Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. Mitchell, M. L., & Jolley, J. A. (2009). Research design explained (7th ed.). Belmont, CA: Wadsworth. Mowday, R. T., Steers, R. M., & Porter, L.W. (1979). The measurement of organizational commitment. Journal of Vocational Behavior, 14, 224–247. doi:10.1016/0001-8791 (79)90072-1 Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.) New York, NY: McGraw-Hill. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Criteria for scale selection and evaluation. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality
and social psychological attitudes (Vol. 1, pp. 1–15). San Diego, CA: Academic Press. Seashore, S. E., Lawler, E. E., Mirvis, P., & Cammann, C. (1982). Observing and measuring organisational change: A guide to field practice. New York, NY: Wiley. Thomas, C., & Rose, J. (2010). The relationship between reciprocity and the emotional and behavioural responses of staff. Journal of Applied Research in Intellectual Disabilities, 23, 167– 178. doi:10.1111/j.1468-3148.2009.00524.x van Dierendonck, D., Schaufeli, W. B., & Buunk, B. P. (1996). Inequity amongst human service professionals: Measurement and relation to burnout. Basic and Applied Social Psychology, 18, 429–451. doi:10.1207/s15324834basp1804_5 van Dierendonck, D., Schaufeli, W. B., & Buunk, B. P. (1998). The evaluation of an individual burnout intervention program: The role of inequity and social support. Journal of Applied Psychology, 83, 392–407. doi:10.1037/0021-9010.83.3.392 van Horn, J. E., Schaufeli, W. B., & Enzmann, D. (1999). Teacher burnout and lack of reciprocity. Journal of Applied Social Psychology, 29, 91–108. doi:10.1111/j.1559-1816.1999.tb01376.x van Yperen, N. W. (1996). Communal orientation and the burnout syndrome among nurses: A replication and extension. Journal of Applied Social Psychology, 26, 338–354. doi:10.1111/ j.1559-1816.1996.tb01853.x van Yperen, N. W., Buunk, B. P., & Schaufeli, W. B. (1992). Communal orientation and the burnout syndrome among nurses. Journal of Applied Social Psychology, 22, 173–189. doi:10.1111/j.1559-1816.1992.tb01534.x Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. doi:10.1037/0022-3514.54. 6.1063
Journal of Intellectual and Developmental Disability Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/cjid20
The Equity Perception Scale – Intellectual Disability Services (EPS-IDS): Evaluating the reliability and validity of a new measure a
a
b
Philip Disley , Chris Hatton & Dave Dagnan a
Lancaster University, UK
b
West Cumberland Hospital, Whitehaven, Cumbria, UK Published online: 09 Jul 2013.
To cite this article: Philip Disley, Chris Hatton & Dave Dagnan (2013) The Equity Perception Scale – Intellectual Disability Services (EPS-IDS): Evaluating the reliability and validity of a new measure, Journal of Intellectual and Developmental Disability, 38:4, 292-300, DOI: 10.3109/13668250.2013.807909 To link to this article: http://dx.doi.org/10.3109/13668250.2013.807909
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Journal of Intellectual & Developmental Disability, 2013 Vol. 38, No. 4, 292–300, http://dx.doi.org/10.3109/13668250.2013.807909
ORIGINAL ARTICLE
The Equity Perception Scale – Intellectual Disability Services (EPSIDS): Evaluating the reliability and validity of a new measure
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
PHILIP DISLEY1, CHRIS HATTON1 & DAVE DAGNAN2 1
Lancaster University, UK, and 2West Cumberland Hospital, Whitehaven, Cumbria, UK
Abstract Background The aim of this paper was to evaluate the reliability and validity of the Equity Perception Scale – Intellectual Disability Services (EPS-IDS), a detailed measure of staff equity perceptions in services for people with intellectual disability (ID). The EPS-IDS contains 3 relationship indices that assess staff relationships with their employing organisation, their co-workers, and service users. Method Postal questionnaires were used to collect quantitative data from a purposive sample of ID service staff. Results The EPS-IDS was found to have good internal consistency reliability. The overall test–retest reliability of the measure, however, was found to be less satisfactory. With the exception of the co-worker relationship index, the EPS-IDS indices were found to have promising validity. Conclusions The findings from the study suggest that the EPS-IDS shows some promise as a measure of the equity perceptions of ID service staff. Further refinements to the relationship indices, however, may be required.
Keywords: Equity theory, intellectual disability, staff
Introduction A small number of studies involving staff working in services for people with intellectual disability (ID) have focused upon their perceptions of how equitably, or fairly, they believe they are treated, or rewarded, in their work-based relationships (e.g., Thomas & Rose, 2010; van Dierendonck, Schaufeli, & Buunk, 1996, 1998; van Yperen, 1996; van Yperen, Buunk, & Schaufeli, 1992). Typically, these studies use equity theory (Adams, 1965) as their theoretical framework. Briefly, this theory proposes that individuals evaluate their social relationships by comparing their perceived ratio of inputs (i.e., investments) and outcomes (i.e., gains) to the corresponding ratio of a comparison other. An individual will view a relationship as equitable if they perceive a match between the input/ outcome ratio of themselves and the input/outcome ratio of their comparison other. If a mismatch is perceived, the individual will experience distress and will be motivated to try to restore equity. According to Adams (1965), the individual can restore equity
in a number of ways, including (a) attempting to alter their own or their comparison other’s inputs and outcomes, (b) changing their comparison other, or (c) leaving the field. In research involving ID service staff, staff equity perceptions have been found to be related to intention to leave (van Dierendonck et al., 1998; van Yperen, 1996), absenteeism (van Dierendonck et al., 1998), and burnout (e.g., Thomas & Rose, 2010; van Dierendonck et al., 1996, 1998; van Yperen, 1996; van Yperen et al., 1992). Research has also found that an intervention program utilising the principles of equity theory had a positive impact upon the wellbeing (level of burnout) and behaviour (length of absence duration) of ID service staff (van Dierendonck et al., 1998). Recent reviews in the ID literature have suggested that equity theory may have some utility as a theoretical framework for integrating various strands of research on staff (Disley, Hatton, & Dagnan, 2009) and that further research using equity theory as a theoretical framework is warranted (Devereux, Hastings, & Noone, 2009).
Correspondence: Philip Disley, Division of Health Research, Lancaster University, LA1 4YG, UK. E-mail: [email protected] © 2013 Australasian Society for Intellectual Disability, Inc.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale Previous research investigating the equity perceptions of ID service staff has used global, often single-item, measures of equity and detailed measures of equity that have been developed for other occupational groupings. As yet, however, no detailed measure of equity perceptions designed specifically for ID service staff is currently available. Such a measure may have a number of advantages over existing equity measures. Compared to global measures, a detailed measure, designed specifically for ID service staff, may be considered a more reliable measure and may capture more information. For example, unlike single-item measures, multi-item measures can provide data on their internal consistency reliability (Churchill, 1979) and, according to classical test theory, are less susceptible to random measurement error (Nunnally & Bernstein, 1994). Unlike multi-item detailed measures of equity, global measures do not provide information on specific inputs and outcomes. Consequently, the reasons why staff perceive their work-based relationships to be unbalanced cannot be ascertained. Within the context of ID research, the main advantage a detailed equity measure developed specifically for ID service staff would have over measures developed for other occupational groupings is that it is likely to have better content validity (i.e., it is more likely to contain items that are relevant to ID service staff and less likely to contain items that are not relevant). In light of these advantages, a detailed equity measure developed specifically for ID service staff may be considered a more appealing alternative to other available equity measures.
293
significantly, with their global counterparts. The discriminant validity of the EPS-IDS was assessed in two ways. First, the discriminant validity of the EPS-IDS was tested by investigating intercorrelations between (a) the EPS-IDS relationship indices, (b) the EPS-IDS input subscales, and (c) the EPSIDS outcome subscales. Second, the degree to which the EPS-IDS scales correlated with a measure of social desirability was investigated. It was predicted that only small correlations would be found between the variables when the discriminant validity of the EPS-IDS was tested. The concurrent validity of the EPS-IDS was tested by investigating the degree to which scores on the EPS-IDS relationship indices correlated with variables that had previously been found to be related to equity perceptions: staff burnout, intention to leave, organisational commitment, staff emotion (positive and negative affect), and job satisfaction (e.g., Geurts, Schaufeli, & Rutte, 1999; Thomas & Rose, 2010; van Dierendonck et al., 1998). The predictive validity of the EPS-IDS was determined by investigating the relationship of the measure with staff performance. It was predicted that scores on the EPS-IDS relationship indices would be positively associated with organisational commitment, personal accomplishment, positive affect, job satisfaction, and performance, and negatively associated with emotional exhaustion, depersonalisation, negative affect, and intention to leave.
Method Design
The aim of the current study The aim of the current study was to evaluate the reliability and validity of the Equity Perception Scale – Intellectual Disability Services (EPS-IDS), a closed format self-report measure designed to assess the equity perceptions of staff working in ID services. Both the internal consistency reliability and the test–retest reliability of the EPS-IDS were assessed. It was assumed that the measure would have reasonable internal consistency and that, barring changes in their circumstances, the scores of staff on the measure would remain stable over the test–retest period. With respect to the validity of the EPS-IDS, the convergent validity of the measure was tested by comparing the degree to which the EPS-IDS scales (the relationship indices and the subscales) correlated with similar global measures of equity. It was predicted that the EPS-IDS relationship indices and subscales would correlate positively, and
A within-participants design was used that had crosssectional and longitudinal components. Data were collected using postal questionnaires. Questionnaires were completed by both staff and their line managers.
Participants Main staff survey. Four hundred and nine ID service staff from the north of England were invited to participate in the research. Of the 409 individuals who were invited to take part in the study, 194 agreed to participate. Of the 194 staff who agreed to participate, 146 staff completed and returned the questionnaire they were sent. Three of the returned questionnaires were excluded because they were filled in incorrectly, leaving 143 usable questionnaires (response rate = 35%). Seventy-nine percent of the participants were female and the mean age was 41 years. The mean lengths of time participants had been working in their current posts, working
294
P. Disley et al.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
for their current employer, and working in services for people with ID were 3 years and 10 months, 6 years and 4 months, and 8 years, respectively. The sample comprised 109 support staff and 34 staff in other job roles (e.g., nurses, occupational therapists, and managers). Test–retest reliability. One hundred and two staff were sent a second copy of the EPS-IDS to complete approximately 4 weeks after they had returned the main staff survey. Seventy-nine questionnaires were returned, of which 78 were usable (response rate = 80%). Seventy-seven percent of the participants were female and the mean age was 42 years. The mean lengths of time participants had been working in their current posts, working for their current employer, and working in services for people with ID were 3 years and 10 months, 6 years and 3 months, and 7 years and 6 months, respectively. The sample comprised 60 support staff and 18 staff in other job roles (e.g., nurses, occupational therapists, and managers). Performance assessment. A subset of 45 housing support staff, who took part in the main survey, had their performance at work assessed. Eighty-nine percent of the participants were female and the mean age was 42 years. The mean lengths of time participants had been working in their current posts, working for their current employer, and working in services for people with ID were 3 years and 8 months, 5 years and 9 months, and 7 years and 3 months, respectively. Measures The Equity Perception Scale – Intellectual Disability Service (EPS-IDS). The EPS-IDS contains 53 items (the scale is available from the first author upon request). Items for the scale were generated from semistructured interviews with 15 ID service staff comprising seven support staff and eight staff from other job roles (i.e., nurses, occupational therapists, and managers). A draft version of the EPS-IDS was then reviewed by seven ID service staff comprising three support staff and four in nursing and managerial roles (see Disley, 2011, and Disley, Hatton, & Dagnan, 2012, for more detailed accounts of the development of the scale). The scale contains three relationship indices (relationships with the employing organisation, co-workers, and service users), each of which contains two subscales: one assessing staff inputs and one to assess staff outcomes. The employer, co-worker, and service user inputs subscales contain, respectively, 15, nine, and five
items. All the outcomes subscales contain eight items. All the items are rated on a 5-point scale, which contains word anchors at each point (1= very little and 5 = very much). Examples of items from the subscales are: “How much do you give your employer in terms of the effort you put into your job” (inputs/employer); “How much do you give your co-workers in terms of encouragement?” (inputs/co-worker); “How much do you give service users in terms of the support you give them?” (inputs/service user); “How much do you get from your employer in terms of the job security you have?” (outcomes/employer); “How much do you get from your co-workers in terms of advice about work issues?” (outcomes/co-worker); “How much do you get from service users in terms of appreciation?” (outcomes/service user). Each relationship index is scored separately. Scores are calculated by dividing the percentage score on the outcomes subscale by the percentage score on the inputs subscale. For example, a participant scores 35 out of 45 and 28 out of 40 on the input and outcomes subscales of the co-worker index, respectively. The percentage score for the input subscale would be 78% (35/45 × 100). The percentage score for the outcome subscale would be 70% (28/40 × 100). Dividing the percentage score of the outcome subscale (70%) by the percentage score on the input subscale (78%) would produce an index score of 0.90. A ratio score lower than 1.00 indicates that the individual perceives themselves to be under-benefited in the relationship (the lower the score the more under-benefited the individual perceives themselves to be). A ratio score of 1.00 indicates that the individual perceives the relationship to be equitable. A ratio score higher than 1.00 indicates that the individual perceives themselves to be over-benefited in the relationship (the higher the score the more over-benefited the individual perceives themselves to be). Other measures used in the study The following measures were completed by staff who took part in the main survey (see Disley, 2011, for more detailed descriptions of the measures). Organisational commitment. The nine-item Organisational Commitment Questionnaire (Mowday, Steers, & Porter, 1979) was used to assess staff loyalty to their employing organisation. Global equity measure. Staff global equity perceptions were assessed with a six-item measure developed by van Horn, Schaufeli, and Enzmann (1999). Each work-based relationship (employer, co-workers, and
The Equity Perception Scale service users) is assessed by two items: a global investment item and a global outcome item.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
Burnout. Burnout was assessed using the 22-item Human Services version of the Maslach Burnout Inventory (MBI-HS; Maslach, Jackson, & Leiter, 1996). The MBI-HS contains three separately scored subscales: emotional exhaustion (MBI-EE; nine items), depersonalisation (MBI-D; five items), and personal accomplishment (MBI-PA; eight items). Staff emotions. The affective state of participants was assessed with the 20-item Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). The PANAS contains two 10-item subscales: positive affect and negative affect. Job satisfaction. A six-item shortened version of a scale developed by Brayfield and Rothe (1951) was used to measure job satisfaction. Intention to leave. Intention to leave the job was assessed using the two-item “propensity to leave” scale. The scale utilises items developed by Lyons (1971) and Seashore, Lawler, Mirvis, and Cammann (1982).
295
complete, and a Freepost return envelope. All completed forms and questionnaires were returned directly to the researchers. Direct line managers (performance assessment). After the main staff questionnaire had been returned, a questionnaire pack was posted to the line managers of the housing support staff who had agreed to have their performance assessed. The pack contained a cover letter, one or more questionnaires, and a stamped addressed envelope. Each questionnaire contained a tear-off sheet that contained the name of the member of staff who had agreed to have their performance assessed. Completed questionnaires were returned directly to the researchers. Upon receipt of the questionnaire, the researchers replaced the tear-off sheet with a personal identification number.
Ethical approval. Ethical approval for the study was sought and obtained from a National Health Service Local Research Ethics Committee (Cumbria and Lancashire Research Ethics Committee A).
Data analysis Social desirability. A short version (11 items) of the Marlowe-Crowne Social Desirability Scale developed by Ballard (1992; Scale 1) was used to assess social desirability. Demographic information. Single-item measures were used to collect demographic information from staff (e.g., gender, age, length of time working in current post, length of time working for current employer, length of time working in ID services, and job title). The following measure was completed by the line managers of housing support staff. Job performance rating. The performance of housing support staff was assessed with the 23-item job performance measure developed by Hatton, Wigham, and Craig (2009). Procedure Staff (main survey/test–retest). Questionnaire packs, containing a cover letter, the main survey, and a Freepost return envelope, were distributed to participants by post. Approximately 4 weeks after they had returned the main staff survey, a subset of staff was sent another questionnaire pack. This pack contained a cover letter, a second copy of the EPS-IDS to
All data were analysed using SPSS Version 14.0 (SPSS Inc., Chicago, IL, USA). Given the number of statistical tests conducted in this study, a p value of < .01 was taken to be statistically significant. The distribution of scores on several of the scales failed to meet parametric assumptions; consequently, nonparametric statistical tests were used in the analysis. A combination of one-tailed and two-tailed tests were used. One-tailed tests were used in cases where directional hypotheses had been made (e.g., when the convergent, concurrent, and predictive validity of the EPS-IDS was being tested). Two-tailed tests were conducted when no explicit directional hypothesis was forwarded (e.g., when the discriminant validity of the EPS-IDS was being assessed). In order to assess the internal consistency reliability of the EPS-IDS, both Cronbach’s alpha coefficients (Cronbach, 1951) and interitem correlations were calculated for each of the measure’s six subscales. Spearman rank order correlations and Wilcoxon signed-rank tests were used to assess the test–retest reliability of the EPS-IDS. Spearman rank order correlations were calculated to determine the convergent validity, discriminant validity, concurrent validity, and predictive validity of the EPSIDS.
296
P. Disley et al. all cases the total score at Time 2 was significantly lower than at Time 1.
Results Descriptive statistics The means, standard deviations (SD), and ranges for the EPS-IDS relationship indices and subscales are presented in Table 1.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The internal consistency of the EPS-IDS All of the EPS-IDS subscales were found to have good internal consistency reliability (see Table 1). The Cronbach’s alphas for the subscales were all higher than the threshold of .70 recommended by Kline (1993). The mean interitem correlations for the subscales were all higher than the threshold of .30 recommended by Robinson, Shaver, and Wrightsman (1991).
The convergent validity of the EPS-IDS With the exception of the co-worker relationship index (r = .07), all of the EPS-IDS relationship indices (employer, r = .49, and service user, r = .36) and subscales (inputs/employer, r = .41, inputs/co-worker, r = .41, inputs/service user, r = .40, outcomes/ employer, r = .55, outcomes/co-worker, r = .27, and outcomes/service user, r = .45) were found to correlate positively and significantly with their global counterparts. These findings suggest that, with the exception of the co-worker relationship index, all of the EPSIDS scales (the relationship indices and the subscales) and their global counterparts were measuring similar constructs.
The test–retest reliability of the EPS-IDS The mean average duration between receipt of the first questionnaire by the researchers and the receipt of the second questionnaire was 41 days (SD = 4.91, range: 33–54 days). All of the relationship indices and the outcome subscales were found to have satisfactory test–retest reliabilities (see Table 1). The test–retest reliabilities for the inputs subscales, however, were found to be less satisfactory. Wilcoxon signed-rank tests found significant differences in the test–retest scores for the co-worker relationship index, the co-worker outcomes subscale, and the service user inputs subscale (see Table 1). In
The discriminant validity of the EPS-IDS The first test of the discriminant validity of the EPSIDS involved calculating the intercorrelations between (a) the EPS-IDS relationship indices, (b) the EPS-IDS inputs subscales, and (c) the EPSIDS outcomes subscales. The correlations between the EPS-IDS relationship indices were all within the range of −.20 to .20 recommended by Mitchell and Jolley (2009) (employer/co-worker, r = −.11; employer/service user, r = .18; co-worker/service user, r = .02). These findings suggest the three EPS-IDS
Table 1. Descriptive, internal consistency reliability, and test–retest statistics for EPS-IDS
EPS-IDS relationship scales
Actual mean (percentage mean)
EPS-IDS (Employer) Inputs 68.97 (91.9%) Outcomes 30.17 (75.4%) Relationship 0.81 index EPS-IDS (Co-worker) Inputs 41.67 (92.6%) Outcomes 33.70 (84.3%) Relationship 0.91 index EPS-IDS (Service user) Inputs 23.45 (93.8%) Outcomes 27.81 (69.5%) Relationship 0.76 index
Mean score Time 1 (test– retest)
Mean score Time 2 (test– retest)
Cronbach’s alpha
Mean interitem correlation
Spearman’s rank order correlations (test–retest)
Wilcoxon signed-rank test (test– retest)
6.12 49–75 6.17 11–60 0.14 0.47–1.07
.93 .84 n/a
.48 .38 n/a
.49∗ .77∗ .89∗
−0.24 −0.66 −0.34
91.95 75.35 0.83
92.00 76.23 0.82
3.51 33–45 5.17 16–40 0.10 0.71–1.14
.90 .93 n/a
.51 .63 n/a
.42∗ .61∗ .62∗
−1.28 −2.66∗ −3.30∗
92.77 85.09 0.92
91.57 81.82 0.89
2.33 14–25 6.62 11–40 0.15 0.46–1.05
.84 .90 n/a
.58 .52 n/a
.41∗ .73∗ .76∗
−4.13∗ −0.46 −1.42
94.76 71.05 0.76
90.87 70.27 0.78
SD
Range
Note. Actual mean (percentage mean), SD, range, Cronbach’s alpha, and mean interitem correlation, n =143; all others, n = 78. p < .01, two-tailed.
∗
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale relationship indices are measuring conceptually distinct relationships. The correlations between the EPS-IDS inputs subscales were all outside the recommended range (employer/co-worker r = .68; employer/service user, r = .57; co-worker/service user, r = .59). With regard to the EPS-IDS outcomes subscales, only the two correlations involving the co-workers subscale were within the recommended range (employer/co-worker, r = .01; employer/service user, r = .44; co-worker/service user, Spearman’s r = .04). These data suggest that the outcomes/ co-workers subscale is the only subscale measuring a conceptually distinct relationship. The second test of the discriminant validity of the EPS-IDS involved investigating relationships between the EPS-IDS indices and subscales and a measure of social desirability. The correlations found for all three of the indices (employer relationship index, r = −.02, co-worker relationship index, r = .02, and service user relationship index, r = −.14) and four of the six subscales (inputs/co-worker, r = .19, outcomes/employer, r = .11, outcomes/coworker, r = .10, and outcomes/service user, r = −.01) were found to be within the recommended range of −.20 to .20. The correlations found for inputs/employer subscale (r = .27) and the inputs/ service user subscale (r = .23) were found to be outside the recommended range. These findings suggest that only the inputs/employer and inputs/service user subscales are affected by social desirability bias. The concurrent validity of the EPS-IDS The means, standard deviations, and internal consistency reliabilities for the measures used to assess the concurrent (and predictive) validity of the EPSIDS are shown in Table 2.
297
Scores on the EPS-IDS employer index were found to be positively correlated with organisational commitment and negatively correlated with emotional exhaustion and intention to leave (see Table 2). Staff who scored higher on the EPS-IDS employer relationship index (i.e., felt less under-benefited) had higher levels of organisational commitment, lower levels of emotional exhaustion, and had less intention to leave their job. Scores on the co-worker relationship index were found to be positively correlated with personal accomplishment and intention to leave and negatively correlated with organisational commitment (see Table 2). Staff who scored higher on the EPS-IDS co-worker relationship index (i.e., felt less under-benefited) had higher levels of personal accomplishment, lower levels of organisational commitment, and a greater intention to leave the job. Scores on the EPS-IDS service user relationship index were found to be positively correlated with organisational commitment, job satisfaction, and positive affect, and negatively correlated with emotional exhaustion, negative affect, and intention to leave (see Table 2). Staff that scored higher on the service user relationship index (i.e., felt less under-benefited) had higher levels of organisational commitment, job satisfaction, and positive affect; lower levels of emotional exhaustion and negative affect; and had less intention to leave their job. The findings suggest that both the employer and service user relationship indices of the EPS-IDS have acceptable concurrent validity. The concurrent validity of the co-worker relationship index was poor because two out of three of the correlations (organisational commitment and intention to leave) were in the opposite direction to what was predicted.
Table 2. Correlations to assess the concurrent and predictive validity of the ESP-IDS relationship indices Mean (SD) Organisational Commitment Emotional Exhaustion Depersonalisation Personal Accomplishment Positive Affect Negative Affect Job Satisfactiona Intention to Leave Performance
34.63 (8.03) 13.92 (8.40) 1.84 (2.21) 36.66 (7.44) 38.37 (6.85) 15.29 (4.59) 12.60 (4.43) 4.41 (2.22) 43.38 (6.35)
EPS-IDS Cronbach’s alpha coefficient EPS-IDS (Employer) EPS-IDS (Co-worker) (Service user) .94 .91 .72 .83 .92 .84 .86 .88 .93
0.55∗ −0.31∗ 0.04 0.09 0.08 −0.01 −0.04 −0.42∗ 0.36∗
−0.25∗ 0.17 0.10 0.25∗ −0.04 0.11 0.11 0.20∗ −0.06
Note. All of the tests n =143 except performance (n = 45). Statistical test used: Spearman’s rank order correlations. Higher scores = less job satisfaction. ∗ Correlation is significant at the .01 level (one-tailed). a
0.27∗ -0.35∗ −0.14 0.05 0.34∗ −0.31∗ −0.24∗ −0.29∗ 0.03
298
P. Disley et al.
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The predictive validity of the EPS-IDS The mean average duration between receipt of the EPS-IDS data by the researchers and the receipt of the performance data was 41 days (SD = 4.56, range: 33–51 days). Only one of the three EPS-IDS relationship indices was found to have acceptable predictive validity (see Table 2.). A significant positive correlation was found between the employer relationship index and performance rating—the less underbenefited staff perceived themselves to be in their relationship with their employer the better their assessed performance at work was. Scores on the coworker and service user relationship indices were not significantly correlated with performance rating.
Discussion This study provides preliminary data on the reliability and validity of the EPS-IDS, a self-report measure that assesses the equity perceptions of staff working in ID services. Overall, the findings relating to the psychometric properties of the EPS-IDS are encouraging. The reliability analysis of the EPS-IDS found that all six of the EPS-IDS subscales had good internal consistency reliability. This finding suggests that the items used in each of the EPS-IDS subscales measure the same underlying constructs and consequently can be used as scales. The data relating to the test–retest reliability of the EPS-IDS, however, was less convincing. None of the three relationship indices were found to have entirely satisfactory test– retest reliability. The test–retest reliability coefficients for all of the input subscales of the EPS-IDS were lower than expected. Also, the test–retest scores for the co-worker relationship index, coworker outcomes subscale, and service user inputs subscale were found to differ significantly, suggesting that the scores on these measures do not remain stable over time. It is possible, however, that that differences found in the scores at Time 1 and Time 2 may be the result of actual changes in staff equity perceptions rather than random error. It may be advisable for future studies investigating the test–retest reliability of the EPS-IDS to monitor participants over the course of the study to determine whether there have been any changes between administrations in their perceptions of their work-based relationships. With regard to the validity of the EPS-IDS, both the service user and employer scales were found to have acceptable convergent validity. The findings relating to the convergent validity of the EPS-IDS co-worker scales, however, were less positive. Only the co-worker inputs subscale was found to correlate
significantly with its global counterpart. The correlation found between the EPS-IDS co-worker relationship index and its global counterpart was nonsignificant, and the correlation found between the co-worker outcomes subscale and its global counterpart was significant but relatively small. It is possible that the items in the co-worker outcomes subscale were not entirely relevant to some, or all, of the participants, and this had an impact on the relationships found for the co-worker outcomes subscale and the co-worker index. It should be pointed out, however, that a number of steps were taken during the development process to ensure that the content of the co-worker index was relevant to ID service staff (see the Measures section). With respect to the discriminant validity of the EPSIDS, many of the correlations between the measure’s subscales were found to be significant and of a relatively large magnitude. These findings suggest that the EPS-IDS subscales are often measuring similar, as opposed to distinct, constructs and consequently appear to have poor discriminant validity. It is possible, however, that the poor discriminant validity of the EPS-IDS may be the result of conceptual issues within equity theory research in general rather than deficits within the EPS-IDS. Specifically, staff may view their work-based relationships in a different way to how they are traditionally conceptualised in equity research (i.e., staff in the current study might have used broader categories for their work-based relationships than the traditionally used categories of “relationship with employer,” “relationship with coworkers,” and “relationship with service users”). Both the employer inputs and service user inputs subscales were found to correlate significantly with a measure of social desirability. An important implication from this finding is that researchers who use the EPS-IDS (especially the employer and service user relationship indices) would be advised to control for social desirability effects. The concurrent validity of the EPS-IDS was established by investigating the relationships between its indices and a number of variables that have previously been found to be correlated with staff equity perceptions. As with previous research (e.g., Geurts et al., 1999; Thomas & Rose, 2010; van Dierendonck et al., 1998), staff equity perceptions, as measured by the EPS-IDS, were found to be related to organisational commitment, emotional exhaustion, personal accomplishment, positive affect, negative affect, job satisfaction, and intention to leave. Only the employer(s) and service user indices of the EPSIDS, however, appear to have acceptable concurrent validity. Scores on these two indices were found to correlate in the predicted direction, with scores on
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
The Equity Perception Scale a number of variables, as found in previous research, to be significantly related to equity perceptions. Two of the three significant correlations found for the coworker relationship index, however, were in the opposite direction to what was predicted. Counterto-predictions scores on the co-worker index were negatively correlated with organisational commitment and positively correlated with intention to leave. These findings are perplexing and difficult to accommodate within the framework of traditional equity theory. One possible explanation, outside the framework of equity theory, is that when staff perceive the work environment in a negative way they look for more support from, and give more support to, their co-workers. In terms of the predictive validity of the EPS-IDS, the finding that only the employer relationship index correlated significantly with performance was disappointing but not entirely unsurprising. The evaluation of the predictive validity of the EPS-IDS requires further research. Given that the concurrent validity data on the EPS-IDS suggest that its relationship indices (a) correlate with a broad range of staff variables and (b) vary in terms of what staff variables they correlate with, it is entirely possible that the predictive validity findings would have been more positive if a broader range of staff variables had been used. The performance measure used in the current study also focused upon a relatively small number of the core competencies needed by support staff in ID services (see Hatton et al., 2009). It is possible, therefore, that a different, more positive pattern of findings would have emerged if a wider range of staff competencies had been assessed. The current study had a number of limitations. For example, design issues, such as the sampling strategy used, mean that one needs to be cautious about generalising the study’s findings. The low response rate found in the current study may have had an impact upon the power of its statistical analysis. The use of nonparametric tests in the study meant that possible confounding variables (such as social desirability) could not be controlled for. Data relating to the predictive validity of the EPS-IDS was only obtained from housing support staff. Future studies on the predictive validity of measure should recruit ID service staff from a broader range of roles. Future studies on the EPS-IDS could also replicate and extend on the findings reported in the current study. For example, the reliability and validity of the EPS-IDS could be assessed independently, in different settings, on larger samples, and on specific occupational groupings working in ID services. The test–retest reliability of the scale also needs to be assessed over longer periods.
299
In conclusion, the current study aimed to evaluate the reliability and validity of the EPS-IDS, a measure designed to assess the equity perceptions of staff working in ID services. The findings from the study suggest that the EPS-IDS shows some promise as a measure of the equity perceptions of ID service staff. Further refinements to the relationship indices, however, may be required. Once refined, the EPS-IDS should prove to be a measure that is easy to complete, quick to do (taking approximately 10 minutes to complete), and easy to score. The EPS-IDS has a number of potential uses. First, it can be used in research that investigates the equity perceptions of staff working in ID services. Second, the measure can be used by operators of ID services to gain insights into the equity perceptions of their staff, to determine the impact of existing and/or new procedures on staff equity perceptions, and can be used as a guide for planning. Third, the measure can be used to guide and evaluate interventions that aim to improve the equity perceptions of staff working in ID services. Author note This paper was funded by a grant from the Economic and Social Research Council, UK. The funding body has placed no restrictions on the access to, or publication of, the research data. Conflict of interest: None. References Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 2, pp. 267–299). New York, NY: Academic Press. Ballard, R. (1992). Short forms of the Marlowe-Crowne Social Desirability Scale. Psychological Reports, 71, 1155–1160. Brayfield, A. H., & Rothe, H. F. (1951). An index of job satisfaction. Journal of Applied Psychology, 35, 307–311. doi:10.1037/ h0055617 Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 64–73. doi:10.2307/3150876 Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. doi:10.1007/BF02310555 Devereux, J., Hastings, R., & Noone, S. (2009). Staff stress and burnout in intellectual disability services: Work stress theory and its application. Journal of Applied Research in Intellectual Disabilities, 22, 561–573. doi:10.1111/j.1468-3148.2009. 00509.x Disley, P. (2011). Applying equity theory to staff in learning disability services (Unpublished doctoral dissertation). Lancaster University, Lancashire, UK. Disley, P., Hatton, C., & Dagnan, D. (2009). Applying equity theory to staff working with individuals with intellectual disabilities. Journal of Intellectual & Developmental Disability, 34, 55– 66. doi:10.1080/13668250802684701
Downloaded by [The University of Manchester Library] at 15:18 13 October 2014
300
P. Disley et al.
Disley, P., Hatton, C., & Dagnan, D. (2012). Inputs and outcomes: What do staff in services for people with intellectual disabilities perceive they bring to and receive from their work-based relationships? Journal of Intellectual Disabilities, 16, 297–306. doi:10.1177/1744629512466793 Geurts, S. A., Schaufeli, W. B., & Rutte, C. G. (1999). Absenteeism, turnover intention and inequity in the employment relationship. Work & Stress, 13, 253–267. doi:10.1080/ 026783799296057 Hatton, C., Wigham, S., & Craig, J. (2009). Developing measures of job performance for support staff in housing services for people with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 22, 54–64. doi:10.1111/j.14683148.2008.00439.x Kline, P. (1993). The handbook of psychological testing. London, UK: Routledge. Lyons, T. F. (1971). Role clarity, need for clarity, satisfaction, tension, and withdrawal. Organizational Behavior and Human Performance, 6, 99–110. doi:10.1016/0030-5073(71)90007-9 Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. Mitchell, M. L., & Jolley, J. A. (2009). Research design explained (7th ed.). Belmont, CA: Wadsworth. Mowday, R. T., Steers, R. M., & Porter, L.W. (1979). The measurement of organizational commitment. Journal of Vocational Behavior, 14, 224–247. doi:10.1016/0001-8791 (79)90072-1 Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.) New York, NY: McGraw-Hill. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Criteria for scale selection and evaluation. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality
and social psychological attitudes (Vol. 1, pp. 1–15). San Diego, CA: Academic Press. Seashore, S. E., Lawler, E. E., Mirvis, P., & Cammann, C. (1982). Observing and measuring organisational change: A guide to field practice. New York, NY: Wiley. Thomas, C., & Rose, J. (2010). The relationship between reciprocity and the emotional and behavioural responses of staff. Journal of Applied Research in Intellectual Disabilities, 23, 167– 178. doi:10.1111/j.1468-3148.2009.00524.x van Dierendonck, D., Schaufeli, W. B., & Buunk, B. P. (1996). Inequity amongst human service professionals: Measurement and relation to burnout. Basic and Applied Social Psychology, 18, 429–451. doi:10.1207/s15324834basp1804_5 van Dierendonck, D., Schaufeli, W. B., & Buunk, B. P. (1998). The evaluation of an individual burnout intervention program: The role of inequity and social support. Journal of Applied Psychology, 83, 392–407. doi:10.1037/0021-9010.83.3.392 van Horn, J. E., Schaufeli, W. B., & Enzmann, D. (1999). Teacher burnout and lack of reciprocity. Journal of Applied Social Psychology, 29, 91–108. doi:10.1111/j.1559-1816.1999.tb01376.x van Yperen, N. W. (1996). Communal orientation and the burnout syndrome among nurses: A replication and extension. Journal of Applied Social Psychology, 26, 338–354. doi:10.1111/ j.1559-1816.1996.tb01853.x van Yperen, N. W., Buunk, B. P., & Schaufeli, W. B. (1992). Communal orientation and the burnout syndrome among nurses. Journal of Applied Social Psychology, 22, 173–189. doi:10.1111/j.1559-1816.1992.tb01534.x Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. doi:10.1037/0022-3514.54. 6.1063
