ORIGINAL ARTICLE
Measuring Self-Rated Productivity Factor Structure and Variance Component Analysis of the Health and Work Questionnaire Ulrica von Thiele Schwarz, PhD, Anders Sj¨oberg, PhD, Henna Hasson, PhD, and Susanne Tafvelin, PhD
Objective: To test the factor structure and variance components of the productivity subscales of the Health and Work Questionnaire (HWQ). Methods: A total of 272 individuals from one company answered the HWQ scale, including three dimensions (efficiency, quality, and quantity) that the respondent rated from three perspectives: their own, their supervisor’s, and their coworkers’. A confirmatory factor analysis was performed, and common and unique variance components evaluated. Results: A common factor explained 81% of the variance (reliability 0.95). All dimensions and rater perspectives contributed with unique variance. The final model provided a perfect fit to the data. Conclusions: Efficiency, quality, and quantity and three rater perspectives are valid parts of the self-rated productivity measurement model, but with a large common factor. Thus, the HWQ can be analyzed either as one factor or by extracting the unique variance for each subdimension.
R
elating the effects of occupational health and safety (OHS) interventions to business outcomes such as productivity has become increasingly important for establishing business cases for OHS interventions.1–3 This, in turn, is essential for motivating organizations to invest in OHS interventions.4 Several instruments have been developed to measure the impact of illness on productivity.5–8 These instruments assess absence from work (absenteeism) and presenteeism (ie, on-the-job productivity loss because of ill-health).5 The magnitude of how illness relates to productivity losses and of the cost that illnesses carry for organizations has been an important contribution of this research.1 The instruments have been shown to be useful for descriptive, comparative, and evaluative purposes when the aim is to capture consequences on productivity relating to illnesses and symptoms.9,10 Nevertheless, this limits the assessment of productivity to health impairments. Thus, they are less well suited for investigations when the aim is to assess productivity improvements. One important reason for broadening the measurement of productivity to include productivity improvements is to make it possible to detect effects of interventions that have a broader scope than simply decreasing ill-health. In these cases, limiting the measurement of productivity to productivity loss may be too narrow. At least two recent developments within the OHS field contribute to the need to measure improvements in productivity rather than productivity
From the Medical Management Centre (Drs von Thiele Schwarz, Hasson, and Tafvelin), Department of Learning, Informatics, Management and Ethics, Karolinska Institutet; Department of Psychology (Drs von Thiele Schwarz and Sj¨oberg), Stockholm University; Centre for Epidemiology and Community Medicine (Dr Hasson), Stockholm County Council; and Department of Psychology (Dr Tafvelin), Ume˚a University, Sweden. The data collection was supported by the occupational health company AB Previa. The writing of this article was supported by a grant from AFA Insurance (Ref. No. 090043). The authors declare no conflicts of interest. Address correspondence to: Ulrica von Thiele Schwarz, PhD, Karolinska Institutet, Medical Management Centre, LIME, Sweden, Tomtebodav¨agen 18A, 17177 Stockholm, Sweden ([email protected]). C 2014 by American College of Occupational and Environmental Copyright Medicine DOI: 10.1097/JOM.0000000000000267
1302
losses: (1) an integration between prevention and promotion and (2) an increase in organizational-level interventions. First, it has been acknowledged that OHS interventions aiming at prevention and primarily physical hazards on the one hand and workplace-based health promotion aimed at individual lifestyle behaviors on the other are often performed in isolation from each other.11,12 This has led to a call for a closer integration between OHS interventions and health promotion interventions.12–16 With such an integration, the aim of the OHS and health promotion interventions is clearly not only a reduction in ill-health and a corresponding reduction in productivity losses but also health improvements and, subsequently, productivity improvements. Second, there is a movement from interventions targeting individual workers toward interventions conducted at an organizational level and aiming to improve health and well-being through system improvements.13 In current European legislation, organizationallevel OHS intervention is the recommended strategy for improving employee health.17 Subsequently, such interventions are becoming increasingly common.18–21 Organizational-level interventions are defined as planned, behavioral, theory-based actions involving changes in the design, organization, and management of work.13,18 A review of job stress interventions suggests that organizational-level interventions may be more effective than individual-level interventions for organizational outcomes such as turnover, absenteeism, and productivity.13 One reason for this may be that organizational-level interventions not only target the source of a work-related health problem but also involve changes in how work is performed, and thus affect the efficiency of work processes.16 In addition, through the participatory design that characterizes organizational-level OHS interventions,13,22 these types of organizations also often increase employee job motivation.16 Thus, organizational-level OHS interventions may have three ways of affecting productivity: (1) decreased productivity losses through the prevention of ill-health, and increased productivity as a result of, (2) increased employee job motivation, and (3) improved work processes that increase possibilities to work efficiently. The need to evaluate the effects of organizational-level OHS interventions on organizational aspects such as productivity has recently been highlighted.23 In view of this, there is a need for instruments assessing productivity without limiting it to productivity loss in relation to health impairments. One promising instrument in this regard is the Health and Work Questionnaire (HWQ), a multidimensional instrument of selfrated productivity and health developed to evaluate the impact of workplace interventions.24 The HWQ focuses on three aspects of productivity: efficiency, quantity, and quality. In an attempt to minimize social desirability, respondents are asked to rate their work quality, quantity, and efficiency from their supervisor’s and co-workers’ perspectives as well as their own (Fig. 1). Nevertheless, because different viewers are known to value and notice different aspects of performance,25 being asked to rate productivity viewed from different standpoints may have the added benefit of also inviting the employee to take into account a wider range of aspects in their ratings of productivity. The subjective ratings of productivity in the HWQ have been related to objective performance, and the scale has been found to discriminate between smokers and nonsmokers,24 JOEM r Volume 56, Number 12, December 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
Measuring Self-Rated Productivity
Participants and Recruitment Procedures
All the company’s line managers (n = 101) participated in the leadership training program, which included a multisource (360degree) feedback study of the line managers’ leadership. This entailed that the managers were asked to invite five of their subordinates to rate their leadership behaviors and the other questionnaire items. The line managers were instructed to ask both subordinates to whom they felt close and those they perceived as more distant to participate. Thus, the sample of the study consists of the 345 subordinates who were invited by the managers and agreed to participate in research. Of these, 68 had completely missing data on the productivity items and five had systematically partially missing data, giving a total sample of 272 individuals.
Data Collection
FIGURE 1. The conceptual model underlying the measurement of self-rated productivity in the Health and Work Questionnaire, where the respondents are asked to rate their work quality, quantity, and efficiency from their supervisor’s and co-workers’ perspectives as well as their own. adherence and nonadherence to epilepsy medication,26 and patients with controlled versus inadequately controlled asthma.27 Nevertheless, although the HWQ is not limited to measuring health-related productivity, few studies have used it for other purposes. An exception to this is a recent study investigating differences in perceptions of organizational learning between supervisors and employees, which were found to be negatively correlated with the efficiency dimension of the HWQ.28 In sum, despite the growing use of the HWQ, the instrument has only been subject to exploratory factor analyses, which revealed a one-factor model.24 Nevertheless, demonstrating that the items are one-dimensional (or multidimensional) with exploratory factor analyses does not necessarily mean that the total score, or the subscores, is adequate indicators of the intended construct.29 To better understand the psychometric properties of the HWQ, it is better to use structural equation modeling.30–32 To further validate the HWQ, this study use confirmatory factor analysis (CFA) using a bifactor model31 of the structure of the HWQ to estimate the unique components of the HWQ to better understand the usefulness of the scale from both a practical and a theoretical perspective. This study contributes with new knowledge about the HWQ by answering the following questions: To what degree does the total score of the HWQ reflect reliable variation on the single construct of productivity? To what degree do the subscores of efficiency, quantity, and quality reflect reliable variation on the productivity construct? To what degree do the rating sources self, manager, and coworker reflect systematic variation in the HWQ?
METHOD Design and Setting The study was set within a forest industry company in Sweden with approximately 800 employees, and is based on a cross-sectional questionnaire survey among a selection of employees in the company. The survey was used as baseline measurement in an intervention study investigating the effects of a leadership training program on productivity, leadership, safety, and organizational learning.
An e-mail including an introductory letter outlining the aim of the study and a personal link to a web-based questionnaire was sent to participants. It was emphasized that participation was voluntary, and all respondents gave their informed consent in writing. The response time was 3 weeks, during which two reminders were sent. The study was approved by the local ethical review board.
Measures Self-rated productivity was assessed using a Swedish version of the productivity subscales from the HWQ, which includes three productivity dimensions—efficiency, quality, and quantity— and three perspectives.24 For each item, the respondent is asked to make ratings from three different perspectives (Fig. 1). For instance, efficiency was measured with the item “How would you and the following persons describe your work efficiency during the last week?” with separate response alternatives for ratings from the perspective of the respondent and of his/her supervisor and coworkers. The response alternatives were on a 10-point Likert scale, with “my worst ever” and “my best ever” as endpoints. The nine items were translated from English to Swedish using a procedure inspired by the Guidelines for Best Practice in Crosscultural Surveys.33 This procedure focuses on evaluating the translation produced in the target language directly, rather than indirectly through back-translation. First, two persons knowledgeable in both languages as well as the research field jointly translated the nine items and the response alternatives to Swedish. The translations and the wording were then discussed with a team of researchers accustomed with productivity measurement, as well as piloted with employees and managers from several different organizations and sectors. This process resulted in very little changes to the original translation.
Statistical Analysis The nine indicators of productivity were subjected to a CFA using maximum-likelihood estimation.32 A bifactor model was specified.34,35 This type of model can be used to partition the variance of each indicator into common and unique components. The unique components can be further decomposed into indicator-specific and random error components. According to this model, all factors are uncorrelated with each other, and the factors are uncorrelated with unique components, and unique components are uncorrelated.36 The CFA model were specified as a model with one productivity factor, with all nine indicators loading on the same factor, three perspective factors (ie, self, supervisor, and coworker), and three factors describing three different productivity dimensions (ie, efficiency, quantity, and quality). We used χ 2 to evaluate our model.32 The correlations in the specified model are compared with the correlations in the empirical data; a nonsignificant χ 2 value means that there is a perfect fit between our specified model and the result of the CFA. In addition, a number of coefficients were used to estimate how much variance can be attributed to each component: (1) omega (ω) is an estimate of the percentage of variance in observed scores
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1303
1304
.62* .74* .56* .63* .73* .72* .48* .81* .51* 7.97 (1.55) 7.63 (1.68) 7.72 (1.58) 8.07 (1.57) 7.62 (1.83) 7.80 (1.63) 8.29 (1.53) 7.91 (1.63) 8.04 (1.58) Efficiency 1 (self) Efficiency 2 (manager) Efficiency 3 (coworker) Quantity 1 (self) Quantity 2 (manager) Quantity 3 (coworker) Quality 1 (self) Quality 2 (manager) Quality 3 (coworker)
HWQ, Health and Work Questionnaire; SD, Standard Deviation. *p < 0.001.
Efficiency 3 Efficiency 2 Efficiency 1 Means (SD) Variable
DISCUSSION In this study, we analyzed the variance components of the HWQ productivity scale and showed that one factor, denoted productivity, accounts for most of the variance. Nevertheless, each of the three productivity dimensions and the three perspectives explained unique variance, but very low part of the variance in the HWQ. The general factor explained 85% of the variance. The finding illustrates that the HWQ is a large common factor that can be interpreted as a main productivity factor and the subfactors and perspective factors contribute a relatively low part of the variance after controlling for the general factor. This has implications for future development of productivity measures as well as the theoretical development of the productivity concept. In line with the first validation study of the HWQ instrument, the productivity scale resulted in a large common factor rather than three distinct subscales. We have chosen to denote this factor as productivity, indicating our hypothesis that general productivity is
TABLE 1. Correlations, Means, and Standard Deviations for HWQ Productivity Items
The correlations between all indicators are presented in Table 1. The standardized factor loadings (λ) are given in Table 2 along with the nine different indicators of the HWQ. The chi-square value (χ 2 [9] = 16.81; P > 0.05) indicates a perfect fit between the factor model and the data. This result suggests that it is possible to answer our questions about how much systematic variance that is hidden in the overall factor, the productivity dimension scores, and the perspective scores. Table 2 also presents the three omega (ω) coefficients. The coefficients were calculated from the lambda estimates (see formulas 3, 4, and 6 in Reise, Bonifay, and Haviland29 ). The general productivity factor in the CFA model was reliable, and the ω estimate shows that 93% of the variance in the observed scores was due to all sources of the common variance. This answers our first question—yes, there is a huge common factor that shows systematic differences between individuals in self-rated productivity. This general factor alone accounted for 85% (ωH = 0.85) of the variance, whereas both the productivity dimensions and the perspective factors show very low ωH coefficients. These results tell us that these factors are systematic but very small parts of the total variance of the HWQ. The same interpretation of the HWQ data is supported when we look at the ωS estimate, the proportion of reliable score variance of indictors measuring a specific factor or perspective after controlling for the general productivity factor. None of these ωS values is near the cut of value of 0.50, suggested by Reise, Bonifay, and Haviland.29
Quantity 1
Variance Components
.71* .82* .62* .55* .54*
Quantity 2
Of the sample, 83% were men. The respondents’ mean age was 47 years (standard deviation, 8.8 years), and they had worked within the organization for a mean of 22.6 years (standard deviation, 10.4 years).
.62* .64* .78* .58* .66* .74*
Respondents
.70* .58* .85* .60* .50* .81* .51*
RESULTS
.68* .72* .69* .63* .60* .69* .61* .54*
Quantity 3
Quality 1
Quality 2
because of all sources of the common variance—this coefficient is analogous to coefficient alpha for the total score; (2) omega hierarchical (ωH) is an estimate of the proportion of the total score variance that can be attributed to a single factor, and omega subscale (ωS) is an estimate of the proportion of the reliable score variance of indictors measuring a specific factor after controlling for the general productivity factor.29 When ωS is interpreted as the reliability of specific factors after controlling for the effect of other factors, following the recommendations of Reise, Bonifay, and Haviland,29 values higher than 0.50 were viewed to indicate that the information in a specific factor was systematic enough to be interpreted separately.
.69*
JOEM r Volume 56, Number 12, December 2014
von Thiele Schwarz et al
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
Measuring Self-Rated Productivity
TABLE 2. CFA Bifactor Model of HWQ
Efficiency (self) Efficiency (supervisor) Efficiency (coworker) Quantity (self) Quantity (supervisor) Quantity (coworker) Quality (self) Quality (supervisor) Quality (coworker) ω ωH ωS
λProductivity
λSelf
0.79 0.78 0.83 0.74 0.76 0.79 0.67 0.76 0.72 0.93 0.85
0.38
λSupervisor
λCoworker
λEfficiency
0.35
0.23 0.19 0.29
0.49 0.27
λQuantity
0.53 0.28 0.44
0.52 0.36 0.44
0.48 0.29 0.53
0.46 0.39 0.02 0.16
0.04 0.28
0.02 0.20
λQuality
0.01 0.08
0.03 0.22
0.03 0.26
CFA, confirmatory factor analysis; HWQ, Health and Work Questionnaire.
an important part of the common factor. Nevertheless, importantly, this study does not show what the common factor consists of. The variance that makes up the common factor may, to a greater or lesser extent, involve other factors than productivity, for example common method bias. Common method bias has been suggested to account for about one fourth of the variance in self-ratings.37–39 The proportion is similar across studies using different comparisons, including common method variance in a measure (using data from several disciplines),37 the extent to which method variance explains relationships between variables,38 in self-ratings of affects and perceptions at work,39 and in measures of job performance.37 Given the size of the common factor in this study, accounting for more than 80% of the variance, it seems unlikely that common method variance is the whole story. Regardless, unpacking the common factor is an important subject for future research, and includes measuring productivity with other sources of data than self-ratings, for example objective data, to validate the interpretation of the common factor as productivity. The overlap between the productivity dimensions and perspectives that result in the common factor has some theoretical support. For example, some factors are likely to influence all dimensions to some extent. For instance, organizational citizenship behaviors are likely to explain variance in not only efficiency and quality but also quantity of work performed.40 Similarly, organizational factors may also exert similar influence on all productivity dimensions. Nevertheless, quality on the one hand and quantity/efficacy on the other are often described as distinct and even conflicting dimensions of productivity.41 In line with this, previous studies have concluded that they are, indeed, distinct and positively correlated constructs of productivity that share less than 50% of their variance, and are related to different antecedents.41 Nevertheless, previous studies have not investigated the existence of a common factor or the variance components of the model. This also shows the benefit of examining variance components, with a CFA bifactor model of scores in a measurement model, not only the model’s factor structure. Although all dimensions and perspectives do independently contribute to explain variance in the model, the large variance explained by the common factor clearly shows that the subscales should not be analyzed independently without taking into account the large shared common component. Specifically, every subscale would reflect the common factor (denoted productivity) to a much larger extent than it would reflect the subscale or the different perspectives. Thus, the variance component analysis helps delineate the measurement and analysis of self-rated productivity. It also contributes to an understanding of
the theoretical construct of productivity, suggesting that similarly to intelligence, self-rated productivity consists of a common factor (ie, the general factor in intelligence) with unique contributions of the subdimensions efficiency, quality, and quantity and the subperspectives self, supervisor, and coworker (ie, spatial and verbal in intelligence).42 Regarding the specific factors, we found that these explained a small but unique proportion of the variance in productivity ratings. First, the distinctiveness of the different perspectives (ie, self, supervisor, and coworker) indicates that respondents do discriminate between their own, their supervisor’s, and their coworkers’ views on their productivity. Comparing mean levels of self-rated productivity with the two other perspectives reveals that respondents anticipated that their supervisor would perceive their productivity somewhat lower than they themselves did. This is in line with leadership studies using a 360-degree perspective, which commonly suggests that ratings by external observers are generally lower than self-ratings.43 As expected from this line of research, taking the perspective of coworkers yielded productivity ratings between the supervisor and self-perspectives. This might be explained by the fact that respondents interact with coworkers more frequently than with their supervisor, and that co-workers’ ratings of the respondents’ productivity would therefore reflect a value closer to their own. Another explanation may be that workers and coworkers view and value similar aspects of productivity, in line with the socioanalytic approach.44 Interestingly, the ratings from the managers’ perspective were more reliable than those from the respondents’ own perspective. This could indicate that this focus factor would correlate more highly with measures of objective work performance than the other factors, given the larger amount of variance explained and higher reliability. Compared with the supervisor’s perspective, the self-perspective probably has more of an overlap with the general productivity factor and less unique variance left, and therefore less variance was left to explain, resulting in a lower reliability coefficient. The three productivity dimensions (ie, efficiency, quantity, and quality) also explained unique variance, but only a small proportion of it. This suggests that these factors can be meaningful specific factors, but more systematic variance is needed. Among these factors, efficiency was the dimension with the lowest reliability and variance, indicating that the quantity and quality dimensions explain more variance after the general productivity factor is accounted for. Thus, efficiency has a greater overlap with general productivity. Given the small but unique contribution of each of the three productivity
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1305
JOEM r Volume 56, Number 12, December 2014
von Thiele Schwarz et al
dimensions, they may be important in understanding the relationship to specific narrow factors, for example, the quality dimension after a quality improvement intervention. Nevertheless, our results strongly suggest that before using them, it is important to correct for the general productivity factor to avoid confusing the specific factor with the general productivity factor (ie, “clean” the specific indicators). This can be done by using a full structural equation model with different outcomes. Investigating whether the specific cleaned indicators predict the specific, matched outcome is an important topic for future research.
Limitations The HWQ uses an innovative rating strategy where the rater is asked to rate productivity taking three different perspectives, and the result supports the validity argument that this shall be interpreted as a general productivity factor. Nevertheless, it is important to remember that the ratings are still made by one person, and thus, that the results are likely to be different if the rater perspective was represented by coworkers and managers. This study is focused on the reliability of the productivity subscales of the HWQ. An important next step is to establish the validity of the instrument, that is, validate that it is actually productivity that is measured. This is preferably made by using objective productivity data. Nevertheless, it is not an uncomplicated process given that objective productivity data come with its own challenges. One challenge is that productivity is seldom measured on an individual level, and when it is, its reliability and validity may be questioned. Another challenge is that productivity measures generally are specific to a certain setting, thus making comparison across sectors difficult. This study is based on data from an industrial company in a small town in Sweden. This may have implications for the generalizability of the findings, and future research in other sectors and settings is warranted. Also, the small sample size (272) is another limitation to consider.
CONCLUSIONS Effects on productivity are an important part of evaluations of OHS interventions. Examination of the productivity subscales of the HWQ showed that efficiency, quality, and quantity and three rater perspectives (ie, self, supervisor, and coworker) are all unique parts of the productivity construct, but similarly to the intelligence construct, self-rated productivity consists of a large common factor. Thus, in measuring productivity with the HWQ, the dimensions and perspectives should be analyzed as one factor, and further research is needed before using the subscales.
ACKNOWLEDGMENT We are indebted to all employees who volunteered participation. We also thank those who assisted us in various parts of the data collection.
REFERENCES 1. Loeppke R, Taitel M, Haufle V, Parry T, Kessler RC, Jinnett K. Health and productivity as a business strategy: a multiemployer study. J Occup Environ Med. 2009;51:411–428. 2. Loeppke R, Taitel M, Richling D, et al. Health and productivity as a business strategy. J Occup Environ Med. 2007;49:712–721. 3. Shearn P. Case Examples: Business Benefits Arising From Health and Safety Interventions. Sheffield: Health and Safety Laboratory; 2003. 4. Miller P, Haslam C. Why employers spend money on employee health: interviews with occupational health and safety professionals from British industry. Saf Sci. 2009;47:163–169. 5. Johns G. Presenteeism in the workplace: a review and research agenda. J Organ Behav. 2010;31:519–542.
1306
6. Mattke S, Balakrishnan A, Bergamo G, Newberry SJ. A review of methods to measure health-related productivity loss. Am J Manag C. 2007;13:211– 217. 7. Prasad M, Wahlqvist P, Shikiar R, Shih Y-CT. A review of self-report instruments measuring health-related work productivity: a patient-reported outcomes perspective. Pharmacoeconomics. 2004;22:225–244. 8. Lofland JH, Pizzi L, Frick KD. A review of health-related workplace productivity loss instruments. Pharmacoeconomics. 2004;22:165–184. 9. Lenneman J, Schwartz S, Giuseffi DL, Wang C. Productivity and health. J Occup Environ Med. 2011;53:55–61. 10. Schwartz SM, Riedel J. Productivity and health: best practices for better measures of productivity. J Occup Environ Med. 2010;52:865–871. 11. Baker E, Israel BA, Schurman S. The integrated model: Implications for worksite health promotion and occupational health and safety practice. Health Educ Behav. 1996;23:175–190. 12. Hymel PA, Loeppke RR, Baase CM, et al. Workplace health protection and promotion: a new pathway for a healthier—and safer—workforce. J Occup Environ Med. 2011;53:695–702. 13. Lamontagne AD, Keegel T, Louie AM, Ostry A, Landsbergis PA. A systematic review of the job-stress intervention evaluation literature, 1990–2005. Int J Occup Environ Health. 2007;13(3):268–280. 14. European Network for Workplace Health P. The Luxembourg Declaration on workplace health promotion in the European Union. 2005. 15. Stenfors HT, Hasson H, Augustsson H, Hvitfeldt FH, von Thiele SU. Merging occupational health, safety and health promotion with Lean: an integrated systems approach (the LeanHealth project). In Burke R, Cooper C, Biron C, eds. Creating Healthy Workplaces: Stress Reduction, Improved Wellbeing, and Organizational Effectiveness United Kingdom. Gower Publishing; 2014. 16. von Thiele Schwarz U, Hasson H. Alignment for achieving a healthy organization. In Bauer GF, Jenny GJ, eds. Salutogenic Organizations and Change. The Netherlands: Springer; 2013:107–25. 17. EU-OSHA. European Survey of Enterprises on New and Emerging Risks: Managing Safety and Health at Work. Luxembourg: European Agency for Safety and Health at Work; 2010. 18. Nielsen K, Randall R, Holten A-L, Gonzalez ER. Conducting organizationallevel occupational health interventions: what works? Work Stress. 2010;24:234–259. 19. Kompier MAJ, Kristensen TS, eds. Organizational Work Stress Interventions in a Theoretical, Methodological and Practical Context. London: Whurr Publishers; 2001. 20. Semmer NK. Job stress interventions and the organization of work. Scand J Work Environ Health. 2006;32:515–527. 21. Egan M, Bambra C, Petticrew M, Whitehead M. Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health. 2009;63:4–11. 22. Nielsen K, Randall R. The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work Stress. 2012;26:91–111. 23. Nielsen K, Abildgaard JS. Organizational interventions: a research-based framework for the evaluation of both process and effects. Work Stress. 2013:278–297. 24. Shikiar R, Halpern M, Rentz A, Khan Z. Development of the Health and Work Questionnaire (HWQ): an instrument for assessing workplace productivity in relation to worker health. Work. 2004;22:219–229. 25. Oh I-S, Berry CM. The five-factor model of personality and managerial performance: Validity gains through the use of 360 degree performance ratings. J Appl Psychol. 2009;94:1498. 26. Hovinga CA, Asato MR, Manjunath R, et al. Association of non-adherence to antiepileptic drugs and seizures, quality of life, and productivity: survey of patients with epilepsy and physicians. Epilepsy Behavior. 2008;13:316– 322. 27. Schmier JK, Manjunath R, Halpern MT, Jones ML, Thompson K, Diette GB. The impact of inadequately controlled asthma in urban children on quality of life and productivity. Ann Allergy Asthma Immunol. 2007;98:245–251. 28. Hasson H, Tafvelin S, von Thiele Schwarz U. Comparing employees and managers’ perceptions of organizational learning, health, and work performance. Adv Dev Human Res. 2013;15:163–176. 29. Reise SP, Bonifay WE, Haviland MG. Scoring and modeling psychological measures in the presence of multidimensionality. J Pers Assess. 2013;95: 129–140. 30. Holzinger KJ, Swineford F. The bi-factor method. Psychometrika. 1937;2: 41–54.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
31. Holzinger KJ, Swineford F. A Study in Factor Analysis: The Stability of a Bifactor Solution. Supplementary Educational Monograph, no. 48. Chicago: University of Chicago Press; 1939. 32. Bollen KA. Structural Equations With Latent Variables. New York: Wiley; 1989. 33. Survey Research Center. Guidelines for Best Practice in Cross-cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan; 2010. 34. Gustafsson J-E. Measurement from a hierarchical point of view. In: Braun HI, Jackson DN, Wiley DE, eds. The Role of Constructs in Psychological and Educational Measurement. London: Lawrence Erlbaum Associates; 2002: 73–95. 35. Reise SP, Moore TM, Haviland MG. Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. J Person Assess. 2010; 92:544–559. 36. Reise SP. The rediscovery of bifactor measurement models. Multivariate Behav Res. 2012;47:667–696. 37. Cote JA, Buckley MR. Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. J Market Res. 1987:315–318.
Measuring Self-Rated Productivity
38. Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J Appl Psychol. 2003;88:879–903. 39. Williams LJ, Cote JA, Buckley MR. Lack of method variance in selfreported affect and perceptions at work: Reality or artifact? J Appl Psychol. 1989;74:462. 40. Podsakoff PM, Ahearne M, MacKenzie SB. Organizational citizenship behavior and the quantity and quality of work group performance. J Appl Psychol. 1997;82:262. 41. Singh J. Performance productivity and quality of frontline employees in service organizations. J Market. 2000:15–34. 42 Jensen AR. The g Factor. The Science of Mental Ability. Westport, CT: Praeger; 1998. 43. Fleenor JW, Smither JW, Atwater LE, Braddy PW, Sturm RE. Selfother rating agreement in leadership: A review. Leadersh Q. 2010;21: 1005–1034. 44. Hogan J, Holland B. Using theory to evaluate personality and jobperformance relations: A socioanalytic perspective. J Appl Psychol. 2003;88: 1041.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1307
Measuring Self-Rated Productivity Factor Structure and Variance Component Analysis of the Health and Work Questionnaire Ulrica von Thiele Schwarz, PhD, Anders Sj¨oberg, PhD, Henna Hasson, PhD, and Susanne Tafvelin, PhD
Objective: To test the factor structure and variance components of the productivity subscales of the Health and Work Questionnaire (HWQ). Methods: A total of 272 individuals from one company answered the HWQ scale, including three dimensions (efficiency, quality, and quantity) that the respondent rated from three perspectives: their own, their supervisor’s, and their coworkers’. A confirmatory factor analysis was performed, and common and unique variance components evaluated. Results: A common factor explained 81% of the variance (reliability 0.95). All dimensions and rater perspectives contributed with unique variance. The final model provided a perfect fit to the data. Conclusions: Efficiency, quality, and quantity and three rater perspectives are valid parts of the self-rated productivity measurement model, but with a large common factor. Thus, the HWQ can be analyzed either as one factor or by extracting the unique variance for each subdimension.
R
elating the effects of occupational health and safety (OHS) interventions to business outcomes such as productivity has become increasingly important for establishing business cases for OHS interventions.1–3 This, in turn, is essential for motivating organizations to invest in OHS interventions.4 Several instruments have been developed to measure the impact of illness on productivity.5–8 These instruments assess absence from work (absenteeism) and presenteeism (ie, on-the-job productivity loss because of ill-health).5 The magnitude of how illness relates to productivity losses and of the cost that illnesses carry for organizations has been an important contribution of this research.1 The instruments have been shown to be useful for descriptive, comparative, and evaluative purposes when the aim is to capture consequences on productivity relating to illnesses and symptoms.9,10 Nevertheless, this limits the assessment of productivity to health impairments. Thus, they are less well suited for investigations when the aim is to assess productivity improvements. One important reason for broadening the measurement of productivity to include productivity improvements is to make it possible to detect effects of interventions that have a broader scope than simply decreasing ill-health. In these cases, limiting the measurement of productivity to productivity loss may be too narrow. At least two recent developments within the OHS field contribute to the need to measure improvements in productivity rather than productivity
From the Medical Management Centre (Drs von Thiele Schwarz, Hasson, and Tafvelin), Department of Learning, Informatics, Management and Ethics, Karolinska Institutet; Department of Psychology (Drs von Thiele Schwarz and Sj¨oberg), Stockholm University; Centre for Epidemiology and Community Medicine (Dr Hasson), Stockholm County Council; and Department of Psychology (Dr Tafvelin), Ume˚a University, Sweden. The data collection was supported by the occupational health company AB Previa. The writing of this article was supported by a grant from AFA Insurance (Ref. No. 090043). The authors declare no conflicts of interest. Address correspondence to: Ulrica von Thiele Schwarz, PhD, Karolinska Institutet, Medical Management Centre, LIME, Sweden, Tomtebodav¨agen 18A, 17177 Stockholm, Sweden ([email protected]). C 2014 by American College of Occupational and Environmental Copyright Medicine DOI: 10.1097/JOM.0000000000000267
1302
losses: (1) an integration between prevention and promotion and (2) an increase in organizational-level interventions. First, it has been acknowledged that OHS interventions aiming at prevention and primarily physical hazards on the one hand and workplace-based health promotion aimed at individual lifestyle behaviors on the other are often performed in isolation from each other.11,12 This has led to a call for a closer integration between OHS interventions and health promotion interventions.12–16 With such an integration, the aim of the OHS and health promotion interventions is clearly not only a reduction in ill-health and a corresponding reduction in productivity losses but also health improvements and, subsequently, productivity improvements. Second, there is a movement from interventions targeting individual workers toward interventions conducted at an organizational level and aiming to improve health and well-being through system improvements.13 In current European legislation, organizationallevel OHS intervention is the recommended strategy for improving employee health.17 Subsequently, such interventions are becoming increasingly common.18–21 Organizational-level interventions are defined as planned, behavioral, theory-based actions involving changes in the design, organization, and management of work.13,18 A review of job stress interventions suggests that organizational-level interventions may be more effective than individual-level interventions for organizational outcomes such as turnover, absenteeism, and productivity.13 One reason for this may be that organizational-level interventions not only target the source of a work-related health problem but also involve changes in how work is performed, and thus affect the efficiency of work processes.16 In addition, through the participatory design that characterizes organizational-level OHS interventions,13,22 these types of organizations also often increase employee job motivation.16 Thus, organizational-level OHS interventions may have three ways of affecting productivity: (1) decreased productivity losses through the prevention of ill-health, and increased productivity as a result of, (2) increased employee job motivation, and (3) improved work processes that increase possibilities to work efficiently. The need to evaluate the effects of organizational-level OHS interventions on organizational aspects such as productivity has recently been highlighted.23 In view of this, there is a need for instruments assessing productivity without limiting it to productivity loss in relation to health impairments. One promising instrument in this regard is the Health and Work Questionnaire (HWQ), a multidimensional instrument of selfrated productivity and health developed to evaluate the impact of workplace interventions.24 The HWQ focuses on three aspects of productivity: efficiency, quantity, and quality. In an attempt to minimize social desirability, respondents are asked to rate their work quality, quantity, and efficiency from their supervisor’s and co-workers’ perspectives as well as their own (Fig. 1). Nevertheless, because different viewers are known to value and notice different aspects of performance,25 being asked to rate productivity viewed from different standpoints may have the added benefit of also inviting the employee to take into account a wider range of aspects in their ratings of productivity. The subjective ratings of productivity in the HWQ have been related to objective performance, and the scale has been found to discriminate between smokers and nonsmokers,24 JOEM r Volume 56, Number 12, December 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
Measuring Self-Rated Productivity
Participants and Recruitment Procedures
All the company’s line managers (n = 101) participated in the leadership training program, which included a multisource (360degree) feedback study of the line managers’ leadership. This entailed that the managers were asked to invite five of their subordinates to rate their leadership behaviors and the other questionnaire items. The line managers were instructed to ask both subordinates to whom they felt close and those they perceived as more distant to participate. Thus, the sample of the study consists of the 345 subordinates who were invited by the managers and agreed to participate in research. Of these, 68 had completely missing data on the productivity items and five had systematically partially missing data, giving a total sample of 272 individuals.
Data Collection
FIGURE 1. The conceptual model underlying the measurement of self-rated productivity in the Health and Work Questionnaire, where the respondents are asked to rate their work quality, quantity, and efficiency from their supervisor’s and co-workers’ perspectives as well as their own. adherence and nonadherence to epilepsy medication,26 and patients with controlled versus inadequately controlled asthma.27 Nevertheless, although the HWQ is not limited to measuring health-related productivity, few studies have used it for other purposes. An exception to this is a recent study investigating differences in perceptions of organizational learning between supervisors and employees, which were found to be negatively correlated with the efficiency dimension of the HWQ.28 In sum, despite the growing use of the HWQ, the instrument has only been subject to exploratory factor analyses, which revealed a one-factor model.24 Nevertheless, demonstrating that the items are one-dimensional (or multidimensional) with exploratory factor analyses does not necessarily mean that the total score, or the subscores, is adequate indicators of the intended construct.29 To better understand the psychometric properties of the HWQ, it is better to use structural equation modeling.30–32 To further validate the HWQ, this study use confirmatory factor analysis (CFA) using a bifactor model31 of the structure of the HWQ to estimate the unique components of the HWQ to better understand the usefulness of the scale from both a practical and a theoretical perspective. This study contributes with new knowledge about the HWQ by answering the following questions: To what degree does the total score of the HWQ reflect reliable variation on the single construct of productivity? To what degree do the subscores of efficiency, quantity, and quality reflect reliable variation on the productivity construct? To what degree do the rating sources self, manager, and coworker reflect systematic variation in the HWQ?
METHOD Design and Setting The study was set within a forest industry company in Sweden with approximately 800 employees, and is based on a cross-sectional questionnaire survey among a selection of employees in the company. The survey was used as baseline measurement in an intervention study investigating the effects of a leadership training program on productivity, leadership, safety, and organizational learning.
An e-mail including an introductory letter outlining the aim of the study and a personal link to a web-based questionnaire was sent to participants. It was emphasized that participation was voluntary, and all respondents gave their informed consent in writing. The response time was 3 weeks, during which two reminders were sent. The study was approved by the local ethical review board.
Measures Self-rated productivity was assessed using a Swedish version of the productivity subscales from the HWQ, which includes three productivity dimensions—efficiency, quality, and quantity— and three perspectives.24 For each item, the respondent is asked to make ratings from three different perspectives (Fig. 1). For instance, efficiency was measured with the item “How would you and the following persons describe your work efficiency during the last week?” with separate response alternatives for ratings from the perspective of the respondent and of his/her supervisor and coworkers. The response alternatives were on a 10-point Likert scale, with “my worst ever” and “my best ever” as endpoints. The nine items were translated from English to Swedish using a procedure inspired by the Guidelines for Best Practice in Crosscultural Surveys.33 This procedure focuses on evaluating the translation produced in the target language directly, rather than indirectly through back-translation. First, two persons knowledgeable in both languages as well as the research field jointly translated the nine items and the response alternatives to Swedish. The translations and the wording were then discussed with a team of researchers accustomed with productivity measurement, as well as piloted with employees and managers from several different organizations and sectors. This process resulted in very little changes to the original translation.
Statistical Analysis The nine indicators of productivity were subjected to a CFA using maximum-likelihood estimation.32 A bifactor model was specified.34,35 This type of model can be used to partition the variance of each indicator into common and unique components. The unique components can be further decomposed into indicator-specific and random error components. According to this model, all factors are uncorrelated with each other, and the factors are uncorrelated with unique components, and unique components are uncorrelated.36 The CFA model were specified as a model with one productivity factor, with all nine indicators loading on the same factor, three perspective factors (ie, self, supervisor, and coworker), and three factors describing three different productivity dimensions (ie, efficiency, quantity, and quality). We used χ 2 to evaluate our model.32 The correlations in the specified model are compared with the correlations in the empirical data; a nonsignificant χ 2 value means that there is a perfect fit between our specified model and the result of the CFA. In addition, a number of coefficients were used to estimate how much variance can be attributed to each component: (1) omega (ω) is an estimate of the percentage of variance in observed scores
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1303
1304
.62* .74* .56* .63* .73* .72* .48* .81* .51* 7.97 (1.55) 7.63 (1.68) 7.72 (1.58) 8.07 (1.57) 7.62 (1.83) 7.80 (1.63) 8.29 (1.53) 7.91 (1.63) 8.04 (1.58) Efficiency 1 (self) Efficiency 2 (manager) Efficiency 3 (coworker) Quantity 1 (self) Quantity 2 (manager) Quantity 3 (coworker) Quality 1 (self) Quality 2 (manager) Quality 3 (coworker)
HWQ, Health and Work Questionnaire; SD, Standard Deviation. *p < 0.001.
Efficiency 3 Efficiency 2 Efficiency 1 Means (SD) Variable
DISCUSSION In this study, we analyzed the variance components of the HWQ productivity scale and showed that one factor, denoted productivity, accounts for most of the variance. Nevertheless, each of the three productivity dimensions and the three perspectives explained unique variance, but very low part of the variance in the HWQ. The general factor explained 85% of the variance. The finding illustrates that the HWQ is a large common factor that can be interpreted as a main productivity factor and the subfactors and perspective factors contribute a relatively low part of the variance after controlling for the general factor. This has implications for future development of productivity measures as well as the theoretical development of the productivity concept. In line with the first validation study of the HWQ instrument, the productivity scale resulted in a large common factor rather than three distinct subscales. We have chosen to denote this factor as productivity, indicating our hypothesis that general productivity is
TABLE 1. Correlations, Means, and Standard Deviations for HWQ Productivity Items
The correlations between all indicators are presented in Table 1. The standardized factor loadings (λ) are given in Table 2 along with the nine different indicators of the HWQ. The chi-square value (χ 2 [9] = 16.81; P > 0.05) indicates a perfect fit between the factor model and the data. This result suggests that it is possible to answer our questions about how much systematic variance that is hidden in the overall factor, the productivity dimension scores, and the perspective scores. Table 2 also presents the three omega (ω) coefficients. The coefficients were calculated from the lambda estimates (see formulas 3, 4, and 6 in Reise, Bonifay, and Haviland29 ). The general productivity factor in the CFA model was reliable, and the ω estimate shows that 93% of the variance in the observed scores was due to all sources of the common variance. This answers our first question—yes, there is a huge common factor that shows systematic differences between individuals in self-rated productivity. This general factor alone accounted for 85% (ωH = 0.85) of the variance, whereas both the productivity dimensions and the perspective factors show very low ωH coefficients. These results tell us that these factors are systematic but very small parts of the total variance of the HWQ. The same interpretation of the HWQ data is supported when we look at the ωS estimate, the proportion of reliable score variance of indictors measuring a specific factor or perspective after controlling for the general productivity factor. None of these ωS values is near the cut of value of 0.50, suggested by Reise, Bonifay, and Haviland.29
Quantity 1
Variance Components
.71* .82* .62* .55* .54*
Quantity 2
Of the sample, 83% were men. The respondents’ mean age was 47 years (standard deviation, 8.8 years), and they had worked within the organization for a mean of 22.6 years (standard deviation, 10.4 years).
.62* .64* .78* .58* .66* .74*
Respondents
.70* .58* .85* .60* .50* .81* .51*
RESULTS
.68* .72* .69* .63* .60* .69* .61* .54*
Quantity 3
Quality 1
Quality 2
because of all sources of the common variance—this coefficient is analogous to coefficient alpha for the total score; (2) omega hierarchical (ωH) is an estimate of the proportion of the total score variance that can be attributed to a single factor, and omega subscale (ωS) is an estimate of the proportion of the reliable score variance of indictors measuring a specific factor after controlling for the general productivity factor.29 When ωS is interpreted as the reliability of specific factors after controlling for the effect of other factors, following the recommendations of Reise, Bonifay, and Haviland,29 values higher than 0.50 were viewed to indicate that the information in a specific factor was systematic enough to be interpreted separately.
.69*
JOEM r Volume 56, Number 12, December 2014
von Thiele Schwarz et al
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
Measuring Self-Rated Productivity
TABLE 2. CFA Bifactor Model of HWQ
Efficiency (self) Efficiency (supervisor) Efficiency (coworker) Quantity (self) Quantity (supervisor) Quantity (coworker) Quality (self) Quality (supervisor) Quality (coworker) ω ωH ωS
λProductivity
λSelf
0.79 0.78 0.83 0.74 0.76 0.79 0.67 0.76 0.72 0.93 0.85
0.38
λSupervisor
λCoworker
λEfficiency
0.35
0.23 0.19 0.29
0.49 0.27
λQuantity
0.53 0.28 0.44
0.52 0.36 0.44
0.48 0.29 0.53
0.46 0.39 0.02 0.16
0.04 0.28
0.02 0.20
λQuality
0.01 0.08
0.03 0.22
0.03 0.26
CFA, confirmatory factor analysis; HWQ, Health and Work Questionnaire.
an important part of the common factor. Nevertheless, importantly, this study does not show what the common factor consists of. The variance that makes up the common factor may, to a greater or lesser extent, involve other factors than productivity, for example common method bias. Common method bias has been suggested to account for about one fourth of the variance in self-ratings.37–39 The proportion is similar across studies using different comparisons, including common method variance in a measure (using data from several disciplines),37 the extent to which method variance explains relationships between variables,38 in self-ratings of affects and perceptions at work,39 and in measures of job performance.37 Given the size of the common factor in this study, accounting for more than 80% of the variance, it seems unlikely that common method variance is the whole story. Regardless, unpacking the common factor is an important subject for future research, and includes measuring productivity with other sources of data than self-ratings, for example objective data, to validate the interpretation of the common factor as productivity. The overlap between the productivity dimensions and perspectives that result in the common factor has some theoretical support. For example, some factors are likely to influence all dimensions to some extent. For instance, organizational citizenship behaviors are likely to explain variance in not only efficiency and quality but also quantity of work performed.40 Similarly, organizational factors may also exert similar influence on all productivity dimensions. Nevertheless, quality on the one hand and quantity/efficacy on the other are often described as distinct and even conflicting dimensions of productivity.41 In line with this, previous studies have concluded that they are, indeed, distinct and positively correlated constructs of productivity that share less than 50% of their variance, and are related to different antecedents.41 Nevertheless, previous studies have not investigated the existence of a common factor or the variance components of the model. This also shows the benefit of examining variance components, with a CFA bifactor model of scores in a measurement model, not only the model’s factor structure. Although all dimensions and perspectives do independently contribute to explain variance in the model, the large variance explained by the common factor clearly shows that the subscales should not be analyzed independently without taking into account the large shared common component. Specifically, every subscale would reflect the common factor (denoted productivity) to a much larger extent than it would reflect the subscale or the different perspectives. Thus, the variance component analysis helps delineate the measurement and analysis of self-rated productivity. It also contributes to an understanding of
the theoretical construct of productivity, suggesting that similarly to intelligence, self-rated productivity consists of a common factor (ie, the general factor in intelligence) with unique contributions of the subdimensions efficiency, quality, and quantity and the subperspectives self, supervisor, and coworker (ie, spatial and verbal in intelligence).42 Regarding the specific factors, we found that these explained a small but unique proportion of the variance in productivity ratings. First, the distinctiveness of the different perspectives (ie, self, supervisor, and coworker) indicates that respondents do discriminate between their own, their supervisor’s, and their coworkers’ views on their productivity. Comparing mean levels of self-rated productivity with the two other perspectives reveals that respondents anticipated that their supervisor would perceive their productivity somewhat lower than they themselves did. This is in line with leadership studies using a 360-degree perspective, which commonly suggests that ratings by external observers are generally lower than self-ratings.43 As expected from this line of research, taking the perspective of coworkers yielded productivity ratings between the supervisor and self-perspectives. This might be explained by the fact that respondents interact with coworkers more frequently than with their supervisor, and that co-workers’ ratings of the respondents’ productivity would therefore reflect a value closer to their own. Another explanation may be that workers and coworkers view and value similar aspects of productivity, in line with the socioanalytic approach.44 Interestingly, the ratings from the managers’ perspective were more reliable than those from the respondents’ own perspective. This could indicate that this focus factor would correlate more highly with measures of objective work performance than the other factors, given the larger amount of variance explained and higher reliability. Compared with the supervisor’s perspective, the self-perspective probably has more of an overlap with the general productivity factor and less unique variance left, and therefore less variance was left to explain, resulting in a lower reliability coefficient. The three productivity dimensions (ie, efficiency, quantity, and quality) also explained unique variance, but only a small proportion of it. This suggests that these factors can be meaningful specific factors, but more systematic variance is needed. Among these factors, efficiency was the dimension with the lowest reliability and variance, indicating that the quantity and quality dimensions explain more variance after the general productivity factor is accounted for. Thus, efficiency has a greater overlap with general productivity. Given the small but unique contribution of each of the three productivity
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1305
JOEM r Volume 56, Number 12, December 2014
von Thiele Schwarz et al
dimensions, they may be important in understanding the relationship to specific narrow factors, for example, the quality dimension after a quality improvement intervention. Nevertheless, our results strongly suggest that before using them, it is important to correct for the general productivity factor to avoid confusing the specific factor with the general productivity factor (ie, “clean” the specific indicators). This can be done by using a full structural equation model with different outcomes. Investigating whether the specific cleaned indicators predict the specific, matched outcome is an important topic for future research.
Limitations The HWQ uses an innovative rating strategy where the rater is asked to rate productivity taking three different perspectives, and the result supports the validity argument that this shall be interpreted as a general productivity factor. Nevertheless, it is important to remember that the ratings are still made by one person, and thus, that the results are likely to be different if the rater perspective was represented by coworkers and managers. This study is focused on the reliability of the productivity subscales of the HWQ. An important next step is to establish the validity of the instrument, that is, validate that it is actually productivity that is measured. This is preferably made by using objective productivity data. Nevertheless, it is not an uncomplicated process given that objective productivity data come with its own challenges. One challenge is that productivity is seldom measured on an individual level, and when it is, its reliability and validity may be questioned. Another challenge is that productivity measures generally are specific to a certain setting, thus making comparison across sectors difficult. This study is based on data from an industrial company in a small town in Sweden. This may have implications for the generalizability of the findings, and future research in other sectors and settings is warranted. Also, the small sample size (272) is another limitation to consider.
CONCLUSIONS Effects on productivity are an important part of evaluations of OHS interventions. Examination of the productivity subscales of the HWQ showed that efficiency, quality, and quantity and three rater perspectives (ie, self, supervisor, and coworker) are all unique parts of the productivity construct, but similarly to the intelligence construct, self-rated productivity consists of a large common factor. Thus, in measuring productivity with the HWQ, the dimensions and perspectives should be analyzed as one factor, and further research is needed before using the subscales.
ACKNOWLEDGMENT We are indebted to all employees who volunteered participation. We also thank those who assisted us in various parts of the data collection.
REFERENCES 1. Loeppke R, Taitel M, Haufle V, Parry T, Kessler RC, Jinnett K. Health and productivity as a business strategy: a multiemployer study. J Occup Environ Med. 2009;51:411–428. 2. Loeppke R, Taitel M, Richling D, et al. Health and productivity as a business strategy. J Occup Environ Med. 2007;49:712–721. 3. Shearn P. Case Examples: Business Benefits Arising From Health and Safety Interventions. Sheffield: Health and Safety Laboratory; 2003. 4. Miller P, Haslam C. Why employers spend money on employee health: interviews with occupational health and safety professionals from British industry. Saf Sci. 2009;47:163–169. 5. Johns G. Presenteeism in the workplace: a review and research agenda. J Organ Behav. 2010;31:519–542.
1306
6. Mattke S, Balakrishnan A, Bergamo G, Newberry SJ. A review of methods to measure health-related productivity loss. Am J Manag C. 2007;13:211– 217. 7. Prasad M, Wahlqvist P, Shikiar R, Shih Y-CT. A review of self-report instruments measuring health-related work productivity: a patient-reported outcomes perspective. Pharmacoeconomics. 2004;22:225–244. 8. Lofland JH, Pizzi L, Frick KD. A review of health-related workplace productivity loss instruments. Pharmacoeconomics. 2004;22:165–184. 9. Lenneman J, Schwartz S, Giuseffi DL, Wang C. Productivity and health. J Occup Environ Med. 2011;53:55–61. 10. Schwartz SM, Riedel J. Productivity and health: best practices for better measures of productivity. J Occup Environ Med. 2010;52:865–871. 11. Baker E, Israel BA, Schurman S. The integrated model: Implications for worksite health promotion and occupational health and safety practice. Health Educ Behav. 1996;23:175–190. 12. Hymel PA, Loeppke RR, Baase CM, et al. Workplace health protection and promotion: a new pathway for a healthier—and safer—workforce. J Occup Environ Med. 2011;53:695–702. 13. Lamontagne AD, Keegel T, Louie AM, Ostry A, Landsbergis PA. A systematic review of the job-stress intervention evaluation literature, 1990–2005. Int J Occup Environ Health. 2007;13(3):268–280. 14. European Network for Workplace Health P. The Luxembourg Declaration on workplace health promotion in the European Union. 2005. 15. Stenfors HT, Hasson H, Augustsson H, Hvitfeldt FH, von Thiele SU. Merging occupational health, safety and health promotion with Lean: an integrated systems approach (the LeanHealth project). In Burke R, Cooper C, Biron C, eds. Creating Healthy Workplaces: Stress Reduction, Improved Wellbeing, and Organizational Effectiveness United Kingdom. Gower Publishing; 2014. 16. von Thiele Schwarz U, Hasson H. Alignment for achieving a healthy organization. In Bauer GF, Jenny GJ, eds. Salutogenic Organizations and Change. The Netherlands: Springer; 2013:107–25. 17. EU-OSHA. European Survey of Enterprises on New and Emerging Risks: Managing Safety and Health at Work. Luxembourg: European Agency for Safety and Health at Work; 2010. 18. Nielsen K, Randall R, Holten A-L, Gonzalez ER. Conducting organizationallevel occupational health interventions: what works? Work Stress. 2010;24:234–259. 19. Kompier MAJ, Kristensen TS, eds. Organizational Work Stress Interventions in a Theoretical, Methodological and Practical Context. London: Whurr Publishers; 2001. 20. Semmer NK. Job stress interventions and the organization of work. Scand J Work Environ Health. 2006;32:515–527. 21. Egan M, Bambra C, Petticrew M, Whitehead M. Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health. 2009;63:4–11. 22. Nielsen K, Randall R. The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work Stress. 2012;26:91–111. 23. Nielsen K, Abildgaard JS. Organizational interventions: a research-based framework for the evaluation of both process and effects. Work Stress. 2013:278–297. 24. Shikiar R, Halpern M, Rentz A, Khan Z. Development of the Health and Work Questionnaire (HWQ): an instrument for assessing workplace productivity in relation to worker health. Work. 2004;22:219–229. 25. Oh I-S, Berry CM. The five-factor model of personality and managerial performance: Validity gains through the use of 360 degree performance ratings. J Appl Psychol. 2009;94:1498. 26. Hovinga CA, Asato MR, Manjunath R, et al. Association of non-adherence to antiepileptic drugs and seizures, quality of life, and productivity: survey of patients with epilepsy and physicians. Epilepsy Behavior. 2008;13:316– 322. 27. Schmier JK, Manjunath R, Halpern MT, Jones ML, Thompson K, Diette GB. The impact of inadequately controlled asthma in urban children on quality of life and productivity. Ann Allergy Asthma Immunol. 2007;98:245–251. 28. Hasson H, Tafvelin S, von Thiele Schwarz U. Comparing employees and managers’ perceptions of organizational learning, health, and work performance. Adv Dev Human Res. 2013;15:163–176. 29. Reise SP, Bonifay WE, Haviland MG. Scoring and modeling psychological measures in the presence of multidimensionality. J Pers Assess. 2013;95: 129–140. 30. Holzinger KJ, Swineford F. The bi-factor method. Psychometrika. 1937;2: 41–54.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 12, December 2014
31. Holzinger KJ, Swineford F. A Study in Factor Analysis: The Stability of a Bifactor Solution. Supplementary Educational Monograph, no. 48. Chicago: University of Chicago Press; 1939. 32. Bollen KA. Structural Equations With Latent Variables. New York: Wiley; 1989. 33. Survey Research Center. Guidelines for Best Practice in Cross-cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan; 2010. 34. Gustafsson J-E. Measurement from a hierarchical point of view. In: Braun HI, Jackson DN, Wiley DE, eds. The Role of Constructs in Psychological and Educational Measurement. London: Lawrence Erlbaum Associates; 2002: 73–95. 35. Reise SP, Moore TM, Haviland MG. Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. J Person Assess. 2010; 92:544–559. 36. Reise SP. The rediscovery of bifactor measurement models. Multivariate Behav Res. 2012;47:667–696. 37. Cote JA, Buckley MR. Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. J Market Res. 1987:315–318.
Measuring Self-Rated Productivity
38. Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J Appl Psychol. 2003;88:879–903. 39. Williams LJ, Cote JA, Buckley MR. Lack of method variance in selfreported affect and perceptions at work: Reality or artifact? J Appl Psychol. 1989;74:462. 40. Podsakoff PM, Ahearne M, MacKenzie SB. Organizational citizenship behavior and the quantity and quality of work group performance. J Appl Psychol. 1997;82:262. 41. Singh J. Performance productivity and quality of frontline employees in service organizations. J Market. 2000:15–34. 42 Jensen AR. The g Factor. The Science of Mental Ability. Westport, CT: Praeger; 1998. 43. Fleenor JW, Smither JW, Atwater LE, Braddy PW, Sturm RE. Selfother rating agreement in leadership: A review. Leadersh Q. 2010;21: 1005–1034. 44. Hogan J, Holland B. Using theory to evaluate personality and jobperformance relations: A socioanalytic perspective. J Appl Psychol. 2003;88: 1041.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
1307
