Evaluation and Program Planning 43 (2014) 48–54

Contents lists available at ScienceDirect

Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan

Implementation of a school-wide prevention programme-teachers’ and headmasters’ perceptions of organizational capacity Maria Ingemarson a,*, Birgitta Rubenson b, Maria Bodin c, Karin Guldbrandsson b a

Department of Clinical Neuroscience, Karolinska Institutet, Box 6031, 102 31 Stockholm, Sweden Dept. of Public Health Sciences, Karolinska Institutet, Stockholm, Sweden c National Board of Health and Welfare, Sweden b

A R T I C L E I N F O

A B S T R A C T

Article history: Received 18 April 2013 Received in revised form 11 October 2013 Accepted 29 October 2013

This study focuses on schools’ organizational capacity to implement a Swedish school-wide programme, Prevention in School (PS). It is based on semi-structured interviews with seven headmasters and 13 teachers from seven Swedish schools. The interviews were analyzed by the use of qualitative content analysis. The findings show that the adoption of a comprehensive intervention like PS challenges the school organization as the staff encountered a variety of organizational barriers when implementing the programme. Factors connected to lack of consensus, collaboration and insufficient programme management were the main barriers that were identified. Teachers wanted a more extensive support from their headmasters in terms of participation in different programme activities. It was emphasized that peer coaches need to be prepared for their task, although the headmasters found it difficult to be able to choose those teachers who they perceived as the most suitable. It is concluded that leadership, coaching and staff selection need particular attention when implementing a programme like PS, since those factors have been defined as important implementation drivers, both in this study and previously. ß 2013 Elsevier Ltd. All rights reserved.

Keywords: Implementation Organizational capacity Prevention programme School-wide programme Teachers Headmasters

1. Introduction A variety of preventive school programmes have been developed during the last decades (Catalano et al., 2003) and the school setting is considered as a natural arena for interventions that target problematic behaviour among youth (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Greenberg et al., 2003). Initially prevention researchers and practitioners focused on single problem behaviours and measures aimed at individuals (Catalano, Hawkins, Berglund, Pollard, & Arthur, 2002) This has changed over time and today comprehensive programmes with aims of altering social regularities are advocated; much since they have an impact on all children and it is difficult to know which children are at risk of getting problems in the future (Berryhill & Prinz, 2003; Catalano et al., 2002; Flay, 2002). In general, there are important contextual aspects to consider during implementation of prevention programmes in school settings. The general demands of the schools have increased (Greenberg et al., 2003) and lead to an intensified workload and feelings of stress among teachers (Ballet & Kelchtermans, 2009;

* Corresponding author. Tel.: +46 736822685 E-mail address: [email protected] (M. Ingemarson). 0149-7189/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.evalprogplan.2013.10.005

Day, 2002; Stenla˚s, 2009). Adding a new programme to an already stressful work situation may be perceived as overwhelming and frustrating (Ballet & Kelchtermans, 2009). Implementation of preventive interventions also requires certain organizational capacities (Durlak & DuPre, 2008; Fixsen, Blase, Naoom, & Wallace, 2009; Greenberg et al., 2003; Greenhalgh, 2005; Payne, Gottfredson, & Gottfredson, 2006). School-wide programmes are particularly challenging, as they put high demands on the school and are more dependent on the whole organization than more narrowly focused programmes. School-wide programmes require all teachers’ engagement and collegiality, common planning time (Berryhill & Prinz, 2003) and supportive headmasters (Payne, 2009). 1.1. Theoretical implementation frameworks Based on the last decades of implementation research, several theoretical frameworks for the conceptualization of implementation determinants have been developed. One of these describes the process of implementation as progressing through four stages: exploration, installation, initial implementation and full implementation, though the process is not always linear and the stages are to be seen as interconnected (Fixsen et al., 2009). The model (shown in Fig. 1) holds that implementation drivers (also known as

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

49

Fig. 1. Implementation stages (Fixsen et al., 2009; Bertram et al., 2013).

core implementation components) are needed at each stage throughout the process. The drivers, which constitute the required infrastructure for implementation, are defined as organization drivers (systems level intervention, facilitative administration, decision support data systems), competency drivers (coaching, training, staff selection) and leadership drivers (technical and adaptive leadership) (Bertram et al., 2013; Fixsen et al., 2009). With regard to the adoption and institutionalization of prevention programmes in particular, a comprehensive review by Durlak and DuPre (2008) has identified several domains of importance, one of which pertains to organizational capacity. Table 1 gives an overview of the types of factors falling within this domain; i.e. general organizational factors, specific practices and processes and specific staffing considerations (Durlak & DuPre, 2008). 1.2. PS – a Swedish school-wide programme Prevention in School (PS) is a school-wide programme at the universal prevention level, developed by Swedish researchers and practitioners to improve the school climate and to prevent problem behaviour such as disorder in class (Sundell et al., 2007). The programme targets all students in the school years 4–9 (age 11–16), and builds on the involvement of all staff. As reported elsewhere (Bodin et al., 2013) the development of PS was inspired by the Norwegian PALS (Ogden et al., 2012) a programme adhering to the School-Wide Positive Behaviour Support model (Sugai & Horner, 2002) The present study was performed together with a randomized controlled trial, in a research project funded by the Development Centre for Child Mental Health at the National Board of Health and Welfare, Sweden. The trial was commissioned with the primary aim to study effects of the PS-programme on classroom climate and student’s problem behaviour (Bodin et al., 2013). A second aim, which is also the aim of the present and a forthcoming study on programme and provider characteristics, was to investigate factors related to implementation of the PS-programme. The programme is described in further detail below.

If the school management shows interest in PS the decision making process starts with an introduction for all staff held by the external consultants (Sundell et al., 2007). Then a survey (Hellqvist & Sundell, 2007) about the school environment is distributed to staff and students, to guide a decision whether to adopt the programme or not. As PS is about establishing a common approach a clear majority need to be positive. If 80% agree to an implementation a final decision to adopt PS can be taken. A local steering group with teacher- and management representatives is created for the planning and monitoring of the PS-work. The external consultants train and supervise the teachers and work together with the steering group. They spend approximately 50–60 h during 1–1.5 years at each school, holding scheduled lectures, meetings and workshops about the three core components. Between the seminars, the teachers get assignments to solve and try out the PS approach in the classrooms. The consultants also train teacher representatives to coach their peers in the Positive leadership component. The training of the peer coaches takes place in parallel with the coaches’ training their peers. The monitoring of the use of techniques connected to the components is conducted by discussions within the teacher teams. After the initial 1.5 years, the programme is supposed to be sustained through peer coaching and the PS-steering group (Sundell et al., 2007). 1.4. Study preconditions Although parts of the PS programme had been tried in collaboration with four schools during development, it had not been used in its full, final format before the trial. Prior to implementation, programme modifications were done in terms of additional training sessions, specification of manuals and new techniques, as well as classroom observations and a questionnaire for further monitoring of the work. Due to the contracted time for the research project, the schools were given approximately three weeks to consider whether to participate or not, instead of one semester as stipulated in the programme. Thus one of the major structural features of PS was not fulfilled.

1.3. Programme structure and components 1.5. Study aims and research questions In PS, schools work under supervision of external consultants. The major structural features are shown in Fig. 2, and can be summarized as (a) a decision making process on whether to implement PS or not, lasting approximately one semester and where 80% of the staff need to be in favour of implementation, (b) the creation of a local steering group responsible for the programme implementation, supervised by programme consultants, (c) the three core components Norm work (work with school rules and consequences for norm-breaking behaviour), Positive leadership (positive behaviour support in class) and a termly Parental forum.

The core assumption in this study is that the school organization is crucial when implementing a school-wide programme like PS. The study aims at exploring teachers’ and headmasters’ perceptions of the schools’ organization and their ability to implement the PS-programme. The following research questions guide the study:  Which school organizational factors are crucial to the implementation of PS?  How do these factors influence the implementation of PS?

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

50

Introduction to school management

Introduction to staff

Survey on school factors to students and staff + feed-back on results

Decision (80% majority required) and stakeholder approval

Formation of local steering group, responsible for implementation

specifically on preventive interventions (Table 1). Further, the interview guide focused on the programme features of PS (Fig. 2). The interview guide (see Appendix 1) was divided into eight different question areas; (1) introduction and contextual factors, (2) the decision to participate in the PS-project, (3) the PSprogramme, (4) training and support, (5) the programme work, (6) management, communication and administration (7) summing up and (8) conclusions. The guide was constructed to be as logical and chronological as possible. The initial questions within each question area were usually more general, such as ‘‘Can you describe how you were trained in the PS-programme?’’ The subsequent questions were more specific and targeted the implementation concepts, for example. ‘‘Did you need more support than originally planned’’? The aim with structuring the guide in this manner was to build an opportunity for the interviewees to reflect both freely and more structured upon the process of the PS implementation, thus enabling a deeper understanding of context specific conditions. 2.2. Interviewees and data collection

Steering group works 1-1.5 years under supervision from consultants

1st component

2nd component

3rd component

Definition of 4-6 positively framed school norms and coupled consequences for norm-breaking behavior

PS-consultants educate one teacher per team to educate their colleagues in positive behavior support in class.

PS-consultants educate teachers (4 hours) in Parental forums: presentations during termly parental meetings

Suggested by teachers and students Decided on by steering group Documented Taught to staff and students Mailed to parents

5 sessions + inbetween practice Systematic praise, encouragement and reward for normconsistent behavior

Presentations bring up importance of: Children’s leisure time activities

Applying consequences for norm-breaking

Restrictivity towards normbreaking behavior such as youth alcohol drinking

Effective instruction-giving

Class agreements

Ignoring minor problems.

Parent-child time

Fig. 2. Decision process and implementation of the core PS components. Adapted from Sundell et al. (2007).

Within the RCT part of the research project, 13 schools were randomized to become programme schools. For the qualitative part that is presented here, interviewees were selected from the first six programme schools that had completed training in the core PS components and from one school which had chosen to terminate their participation during on-going training. To interview staff from this seventh school was considered as important to include the experience of a less successful implementation of PS. The selection of interviewees in each school was based on the degree of engagement in the PS-work. It was made by purposively choosing (Patton, 2002) teachers from the PS-steering teams, as the team members were assumed to have the best insight into the implementation process. The main author (MI) attended PSsteering team meetings in each school to recruit two teachers who worked in the school years 4–9. Teachers not attending this meeting were not asked to participate. In one school, the teachers were included after a drawing of lots, as more than two possible interviewees were present at the meeting. Two schools did not have any active steering groups at the time for the interviews. Here teachers who had been members of the local steering group and working in the school years 4–9 were contacted by In one of the schools only one teacher agreed to participate. When recruiting interviewees among school leaders their engagement in the programme work was decisive. The main headmaster or the assistant headmaster in each school was asked to participate, depending on their degree of engagement. All interviews were made by the first author (MI) and each of them lasted between 1 and 1.5 h. Most participants preferred to be interviewed at their school, but two of them came to the first authors’ office. The tapes were transcribed and entered into the Nvivo-software, resulting in an empirical material consisting of transcripts of interviews with seven school leaders and 13 teachers. 2.3. Analyses

2. Methods A qualitative approach was chosen as this is especially useful when evaluating processes in general and programme implementation in particular (Patton, 2002). 2.1. Development of an interview guide Concepts from a review on implementation (Durlak & DuPre, 2008) were used to frame the interview guide as it focused

Teacher and headmaster interviews were analyzed together. Thematic content analysis was performed in order to reduce data and identify core patterns by using the principle of analytic induction (Patton, 2002). In a first deductive step the main author (MI) extracted larger units of text from all interviews and coded the content according to organizational capacity as described by Durlak and DuPre (2008). Then the third and the last authors coded a sample of the interviews and consensus discussions were held among all authors and a code-book was developed by the first

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

51

Table 1 Factors connected to organizational capacity, based on Durlak and DuPre (2008). Organizational capacity General organizational factors Positive work climate (moral, trust and resolving disagreements) Organizational norms regarding change (collective willingness to try new approaches, risk taking) Integration of new programming (the possibility to incorporate the programme into existing practices) Shared vision (consensus and commitment among organizational members)

Specific practices and processes Shared decision making (about what is to be implemented) Communication (within the organization) Formulation of tasks (strategic planning, clear roles and responsibilities)

author (MI). After this, the first author (MI) checked the coherence against the code-book and re-coded when needed. In the second inductive phase the first author (MI) read the coded text multiple times to search for regularities, differences and relationships (Patton, 2002). Based on the patterns that were found, a model with an overall theme, categories and subcategories was developed and verified through several text readings by the first author (MI) until it was found solid and coherent by the first (MI) and second author (BR). Quotes connected to the categories where coded manually by the main author (MI) according to the model. When presenting the findings the quotes illustrate the categories or subcategories. 2.4. The research team The research team consists of three co-authors and public health researchers: one specialized in qualitative methods (BR), one with extensive involvement in the field of implementation science (KG) and one with knowledge and experience from experimental programme evaluations (MB). The first author (MI) is a PhD student with previous experience of evaluation and qualitative methods in social work. Ethics approval was given in 2010 by the Stockholm regional ethics board.

3. Findings The seven interviewed headmasters/assistant headmasters had been active as teachers between 10 and 30 years and as headmasters between 3 and 12 years. The 13 teachers had a teaching experience between 3 and 36 years. They described the general school setting as open to change and highly influenced by a variety of external ideas, where new projects seldom get rooted and only last during shorter periods of time. In the analysis three subthemes were identified: Lack of consensus, which concerns the lack of agreement among school staffs regarding needs and assignments. Collaboration barriers is a theme on the conditions that complicated the possibility to gather and shape a comprehensive approach. Insufficient process management designates the need for and the lack of an extensive headmaster support, as well as the perceived difficulties associated with peer coaching. The study findings are summarized and formulated in the overall

Specific staffing considerations Leadership (establishing consensus and managing the process) Programme champion (a trusted individual that promotes implementation) Support (managerial and supervisory support and encouragement, administrative support)

theme: ‘‘Organizational barriers to programme commitment’’. The categories and sub categories are described in Table 2 and under the headings below. 3.1. Lack of consensus Even though the schools had officially declared that they had made a majority decision regarding the implementation of PS, the decision procedure was described as both ambiguous and hasty by several interviewees. It was stressed that less than 80% actually did want the programme. Fear of not being socially accepted if voting against programme adoption, the short amount of time for consideration and headmasters trying to persuade the teachers to say yes, were factors believed to have affected the decision: The school management really wanted to participate (in the PSproject). They pushed and I felt that you more or less had to agree to (. . .). (Teacher) We got them on board without them really understanding what a big job it was. (Headmaster) Such a quasi-democratic decision was described to make teachers feel misled, to stir up resistance among opposing teachers and to reinforce tensions. Teachers also had different views on their professional assignments. Teachers who had been in the profession for a long time were sometimes described as loners, as opposed to younger teachers who were perceived as more prone to open the classroom door. Such differences in attitudes might, according to one headmaster, lead to younger teachers leaving: The younger teachers may stay for three or four years, some won’t even stay that long, because this is not how they look at the school and school improvement. So they hit the road, while those who like to work isolated in their own fortresses, they stay. (Headmaster). The theme on different needs in different teacher groups was recurrent. Teachers in the different school years were described to differ in their needs, as the requirements on younger students are not equal to those on the older. This was described to affect the work with the school norm-component, as teachers wanted to

Table 2 The overall theme, the three categories and their related subcategories. Organizational barriers to programme commitment Lack of consensus Pseudo democratic decision Different views on professional assignments Different needs

Collaboration barriers Teachers spatially divided Lack of time Participation refusal

Insufficient process management Not enough headmaster support Unprepared coaches Pragmatically selected coaches

52

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

formulate different types of school norms. A more experienced teacher meant that the PS-training rather belonged in the teacher education, since it did not provide much new. 3.2. Collaboration barriers Even though the PS programme stipulates regular steeringgroup meetings, the interviewees expressed a need for additional opportunities to gather and discuss the PS-work in more informal ways. That kind of collaboration was difficult to establish since teachers were spatially divided: Every teacher has an own specific schedule, teachers work with different age groups in different areas of the school and are sometimes even located in different buildings. This made it difficult to meet and spontaneously reflect on the work. The experience of a general lack of time affected the implementation in a number of ways. At the weekly conferences there was a pent-up need to talk about individual students, due to this the discussions about student related topics tended to be prioritized and the PS work to be postponed. Nevertheless, some teachers pointed out that the discussions concerning students sometimes got out of hand and got too problem-oriented and saw PS as a useful tool to bring structure to the conferences. Further, policies and regulations were generally described to put pressure on the school organization, which further reinforced the feeling of lacking time. The increasing general demands on documentation were perceived as stressful and to force staff to make difficult prioritizations, as these new duties were not compensated with the corresponding time: (. . .) More is added to be documented. There is less time (. . .) It’s not like the teachers used to hang around drinking coffee two hours a day before (the increased demands on documentation). You have to take that time from something else (. . .). How are you supposed to keep a high quality education when the teachers, not only perceive that they have, but really do have less time to develop their work? It’s an immense challenge. (Headmaster) The lack of compensation for the PS-work was also described to lower teachers’ motivation. The programme became yet another additional task to be completed without getting extra time to do it. A further perceived collaboration barrier was that some teachers refused to participate in the PS-activities, by not using the techniques or not showing up at the PS-meetings. Such actions lead to frustration among teachers who were committed to use the programme: - I don’t care about that-’’. ‘‘- I don’t want to work with that-’’. But (. . .) we are in this together and we did agree to it! (. . .) We have chosen this even if YOU didn’t vote for it. You have to adapt and work with this. Then you notice that some -just to show that they are not going to do this- have neglected it. Then it fails. What kind of message do we send to the students? What does it say about us? We have to be consistent! (Teacher) 3.3. Insufficient process management The PS programme contains several features to promote the implementation process, such as the PS-steering teams with teacher representatives, regular meetings and peer-coaching. Despite this there was an expressed need, especially among the teachers, for a more extensive implementation management. The theme on insufficient process management concerns the need for internal support and should not be confused with the support from the external consultants.

The majority of the teachers wanted more headmaster support and that the formal school leader should take action in a variety of ways. A headmaster simply emphasizing the importance of the programme was not perceived as enough. It was suggested that headmasters have to participate in every PS-activity as it would give them the necessary understanding of both the programme and the process of implementing it. By participating they would also be able to pursue control over resistant teachers. One teacher stressed that this kind of control needed to be performed by the formal leader, who has access to means of pressure: The school management said, alright, this (the program) is so good. Go for it! But they haven’t been sufficiently knowledgeable and familiar with the methods or what’s going on in the different teams. If the headmaster showed up and did participate sometimes, people wouldn’t show up without having done their homework. If the person who determines your salary-that’s where we end up-is sitting there, then people will be well read. Then the input for discussions will be better. Instead of someone (a colleague) sitting there acting as a defiant teenager (. . .). (Teacher) When comparing teacher and headmaster responses, though, it is clear that the views on the responsibility for the programme implementation differed between them. The teachers emphasized the need for a more extensive support from their headmasters, whereas headmasters were more oriented towards sharing the responsibility with the teachers. Further, the general impression is that the teachers were inexperienced in peer coaching and unprepared for the task and it was stressed that peer coaches need to have certain skills and personal qualities. One teacher described the importance of selecting the right people for the right task: As a school leader you would need to look at what kind of teachers you have. What resources do they have? Which teachers would make a dream team? (. . .) The (school-) organization is so slim. We can’t afford not using the real strengths. (Teacher) However, it could be difficult to pick the right peer coaches. The headmasters described that they sometimes had to make pragmatic selections of coaches as the most suited teachers tended to be overburdened and could not always be chosen. They felt forced to pick volunteers, those who lacked additional assignments or even the reluctant teachers. Such a selection was unfortunate though, since teachers who were not readily prepared could end up as peer coaches: Like everyone in the steering team I became a coach automatically (. . .) I feel like ‘‘zero coach’’ (. . .) I don’t feel that it (the role as coach) is in me. Therefore it is difficult to supervise and give feedback to the colleagues. (Teacher) Some teachers described how their colleagues seemed reluctant towards being subordinated and coached by a peer. One teacher who was selected to be a coach discussed the possibility of not having the right skills, but ended up in the conclusion that lack of mandate was the crucial point: I have no mandate to control them, and I am not their employer. Then it becomes problematic. (. . .) ‘‘Who do you think you are?’’ ‘‘You think you are so amazingly talented’’. (Teacher) In summary, this study explores teachers’ and headmasters’ perceptions of their schools’ organizational capacity, what factors they perceived as crucial and how these factors affected the implementation of the PS programme. As shown in the results the interviewee statements primarily focused on factors perceived as organizational barriers. The main areas of concern were lack of

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

consensus, barriers to collaboration and an insufficient programme management. 4. Discussion When designing the study, factors in a review on preventive interventions and implementation (Durlak & DuPre, 2008) were used as a framework of the interview guide. In the review article the authors particularly stress the importance of two factors: Collaboration (non-hierarchical relationships, trust, open communication, shared responsibilities and efforts to establish consensus) and staffing factors (leadership and programme champions). Our findings clearly relate to those specific factors. Programmes like PS can be described as a rather demanding undertaking for a school, as they require a full commitment from all staff members (Payne, 2009). Our findings confirm what implementation researchers often highlight as an important aspect of organizational capacity, namely the importance of an extensive support from the formal leader (Durlak & DuPre, 2008; Fixsen et al., 2009; Handler et al., 2007; Rohrbach, Grana, Sussman, & Valente, 2006). Leadership is an implementation driver needed both to provide routines and to establish consensus (Bertram et al., 2013; Fixsen et al., 2009). It appears that the interviewed teachers had high expectations and were quite dependent of their headmasters. The headmasters in turn wanted more of a shared responsibility and were, due to time limits, not able to respond to the teachers’ requests. The study findings also suggest that the headmasters did not fully understand how important they were to the implementation process. Nevertheless, the teachers expressed that a headmaster has to participate in all programme activities in order to gain an understanding of the on-going work, to monitor and sometimes control the work process. The findings also indicate, in line with earlier research, that lack of such an active support might generate resistance, whereas a headmaster who takes part might be able to reduce opposition (Handler et al., 2007; Lohrmann, Forman, Martin, & Palmieri, 2008). It has been found that implementation of school programmes require leaders at many levels (Rowling & Samdal, 2011). Although PS by the peer coaching feature is built on the principle of a shared leadership, it did not fully work in practice. Teachers can be reluctant to be subordinated their peers, which might be explained by the somewhat hierarchical school culture, with teachers being highly dependent on the headmaster and unaccustomed to a decentralized leadership. It is also possible that norms of equality among the teachers (Ballet & Kelchtermans, 2009) negatively affects teachers’ willingness to be led by their peers. Coaching is a driver that has been found as essential to programme implementation, since theory, demonstration and practice is not enough to reach sustainable change (Bertram et al., 2013; Fixsen et al., 2009; Joyce & Showers, 2002). Mutual trust and exchange of ideas among colleagues facilitate peer coaching (Donegan, Ostrosky, & Fowler, 2000; Slater & Simmons, 2001; Wanzare & da Costa, 2000), whereas reluctance and teachers doubting their own abilities to coach preclude it (Slater & Simmons, 2001). Consequently, peer coaching requires both specific individual skills (Wanzare & da Costa, 2000) and a radical change in relationships among teachers and school administrators (Slater & Simmons, 2001). Given all this, it seems especially important to strategically choose skilled and trusted peer coaches. However our findings show that headmasters, due to lack of resources, could not always choose the most suitable ones. The identified category ‘‘collaboration barriers’’ is strongly connected to communication, as the subcategories ‘‘spatial division of teachers’’ and ‘‘lack of time’’ mainly concern the lack of opportunities to communicate about the programme work. Generally, programme implementation requires effective commu-

53

nication among stakeholders (Durlak & DuPre, 2008; Fixsen et al., 2009). It is particularly important when implementing a comprehensive programme like PS, where common agreements on different levels are essential (Rowling & Samdal, 2011). But schools communication patterns tend to be fragmented (Handler et al., 2007) and needs for significant changes in the design of school schedules have been identified in order to increase teachers’ opportunities to collaborate (Slater & Simmons, 2001) The PS feature of steering teams can be regarded as a way to create such a formalized structure. It is worth noting though, that the steering teams are not the same thing as ‘‘implementation teams’’ (Fixsen, Blase, & Van Dyke, 2011). Even though there were external consultants, their function cannot be seen as fully equivalent to the intended function of implementation teams; as such teams operate at the community level and work with drivers in a more structured way. Given the time of the interviews, it seems reasonable that the schools were somewhere in the stages of installation and/or initial implementation (Fig. 2; Fixsen et al., 2009). It is important to convey that implementation is known to take time and that all staff members could not possibly be committed when the interviews were conducted (Fixsen et al., 2009; Rogers, 2003). Nevertheless, this study shows what kind of organizational barriers that might occur when implementing a programme like PS and what needs to be solved during the early stages. 4.1. Study limitations There are no rules about sample size in qualitative studies (Patton, 2002). Given the time frame of the research project, this study had to be initiated at a time when only seven of thirteen PSschools had completed the introduction and training in the basic components. Nevertheless, the number of selected teachers was considered as satisfactory. The headmaster interviewees, were lower in number, but as the analysis was conducted on the statements from both teachers and headmasters, the material as a whole was considered as saturated (Patton, 2002). Since the interviewees were selected among the most active teachers and headmasters, they were probably more likely to be positive towards the programme implementation than their colleagues. On the other hand, given the short period the schools had worked with PS, teachers that were not that active in the implementation process probably would have had less to contribute. The results might have looked different and led to a deeper insight if the interviews had been carried out at a later stage of the implementation process, but there would have been risks connected to such an approach. Some schools could have chosen to quit and staff turnover could have made it difficult to capture the initial stages of the implementation. Further, the fact that the schools were given a very short period to consider programme adoption most likely affected the decision, and perhaps even evoked a more non-democratic procedure. It is important for both funders, researchers and external consultants to be self-critical about that, even though it is a common problem that projects has to be speeded up due to limited resources (Fixsen et al., 2011). 4.2. Conclusions and implications The implementation of a comprehensive programme like PS puts high demands on the school organization. Hence, the process of implementation needs to be thoroughly planned. The findings in this study reveal a variety of substantial barriers to the establishment of consensus and collaboration as well as barriers to an effective management of the overall process. It seems

54

M. Ingemarson et al. / Evaluation and Program Planning 43 (2014) 48–54

necessary with a further development of the structural features of the PS programme to be able to overcome such barriers. The selection of peer coaches needs to be more formalized and the training to focus more on the coaching ability in itself. Leadership, staffing and coaching factors need to be taken into account when planning future implementation of comprehensive interventions in schools, since these factors have been identified as essential implementation drivers, both in this study and in previous research. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.evalprogplan. 2013.10.005. References Ballet, K., & Kelchtermans, G. (2009). Struggling with workload: Primary teachers’ experience of intensification. Teaching and Teacher Education, 25, 1150–1157. Berryhill, J. C., & Prinz, R. J. (2003). Environmental interventions to enhance student adjustment: Implications for prevention. Prevention Science, 4, 65–87. Bertram, M. B., Blase, K. A., & Fixsen, D. L. (2013). Improving programs and outcomes. Implementation frameworks 2013. School of Social Work, University of MissouriKansas City. (submitted manuscript) http://www.sw.uh.edu/news/events/ 05292012-bridging%20the%20gap%202013/Bertram-Blase-Fixsen_Improving%20Programs%20and%20Outcomes%20Implementation%20Frameworks_2013.pdf. Bodin, M. C., South, S., & Ingemarson, M. (2013). A quasi-randomized trial of a schoolwide universal program for preventing problem behavior and improving the learning climate in swedish schools. (Manuscript submitted for publication). Catalano, R. F., Hawkins, J. D., Berglund, M. L., Pollard, J. A., & Arthur, M. W. (2002). Prevention science and positive youth development: Competitive or cooperative frameworks? Journal of Adolescent Health., 31, 230–239. Catalano, R. F., Mazza, J. J., Harachi, T. W., Abbott, R. D., Haggerty, K. P., & Fleming, C. B. (2003). Raising healthy children through enhancing social development in elementary school: Results after 1.5 years. Journal of School Psychology, 41, 143–164. Day, C. (2002). School reform and transitions in teacher professionalism and identity. International Journal of Educational Research, 37, 677–692. Donegan, M. M., Ostrosky, M. M., & Fowler, S. A. (2000). Peer coaching: Teachers supporting teachers. Young Exceptional Children, 3, 9–16. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19, 531–540. Fixsen, D. L., Blase, K. A., & Van Dyke, M. K. (2011). Mobilizing communities for implementing evidence-based youth violence prevention programming: A commentary. American Journal of Community Psychology, 48, 133–137. Flay, B. R. (2002). Positive youth development requires comprehensive health promotion programs. American Journal of Health Behavior, 26, 407–424. Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., et al. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466–474. Greenhalgh, T. (2005). Diffusion of innovations in health service organisations: A systematic literature review. Malden, MA: Blackwell. Handler, M. W., Rey, J., Connell, J., Thier, K., Feinberg, A., & Putnam, R. (2007). Practical considerations in creating school-wide positive behavior support in public schools. Psychology in the Schools, 44, 29–39.

Hellqvist, A., & Sundell, K. S. (2007). Elevers och la¨rares upplevelser av skolan: En studie av 1310 elever i a˚rskurs 4 till 9 och 118 la¨rare fra˚n fyra grundskolor i Stockholm [Student’s and teachers experiences of their school: A study of 1310 students in school-years 4 to 9 and 118 teachers from four schools in Stockholm, in Swedish]. FoU-rapport. Stockholm: The Unit of Research and Development, Stockholm. Joyce, B. R., & Showers, B. (2002). Student achievement through staff development. Alexandria, VA: Association for Supervision and Curriculum Development. Lohrmann, S., Forman, S., Martin, S., & Palmieri, M. (2008). Understanding school personnel’s resistance to adopting schoolwide positive behavior support at a universal level of intervention. Journal of Positive Behavior Interventions, 10, 256–269. Ogden, T., Bjornebekk, G., Kjobli, J., Patras, J., Christiansen, T., Taraldsen, K., et al. (2012). Measurement of implementation components ten years after a nationwide introduction of empirically supported programs – A pilot study. Implementation Science: IS, 7, 49. Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage Publications. Payne, A. (2009). Do predictors of the implementation quality of school-based prevention programs differ by program type? Prevention Science, 10, 151–167. Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science, 7, 225–237. Rogers, E. M. (2003). Diffusion of innovations. New York: Free Press. Rohrbach, L. A., Grana, R., Sussman, S., & Valente, T. W. (2006). Type II translation: Transporting prevention interventions from research to real-world settings. Evaluation & The Health Professions, 29, 302–333. Rowling, L., & Samdal, O. (2011). Filling the black box of implementation for healthpromoting schools. Health Education, 111, 347–366. Slater, C. L., & Simmons, D. L. (2001). The design and implementation of a peer coaching program. American Secondary Education, 29, 67–76. Stenla˚s, N. (2009). En ka˚r i kla¨m- La¨raryrket mellan professionella ideal och statliga reformideologier [The teaching profession, between professional ideals and governmental reform ideologies; In Swedish] Rapport till expertgruppen fo¨r studier i offentlig ekonomi. Sugai, G., & Horner, R. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24(1–2), 23–50. Sundell, K., Koutakis, N., Hellqvist, A., Andre´n, J., Karlberg, M., Forster, M., et al. (2007). Prevention i Skolan: Forskningsbaserat preventionsprogram fo¨r grundskolans a˚r 4-9 [Prevention in School: A research-based preventionprogram for school years 4-9; in Swedish] FoU-rapport. Stockholm: Forsknings-och Utvecklingsenheten Stockholms stad. Wanzare, Z., & da Costa, J. L. (2000). Supervision and staff development: Overview of the literature. NASSP Bulletin, 84, 47–54.

Maria Ingemarson is a PhD student at the Department of Clinical Neuro Science. Her thesis involves studies on programme effects and the process of implementing the Swedish school wide programme PS. She has a bachelor of science in social work and has been involved in several evaluation projects targeting the social work practice.

Birgitta Rubenson is a Senior Lecturer at the Department of Public Health Sciences at Karolinska Institutet in Stockholm. She has a PhD in International Health and as a qualitative researcher she focuses on studying the health and rights of children, their development, resilience and vulnerability. She teaches courses on Qualitative Research Methods and on Health and Human Rights.

Maria Bodin, PhD, is currently employed as a researcher at the Department of Knowledge Based Policy and Guidance, National Board of Health and Welfare in Stockholm, Sweden. Her research has predominantly concerned effectiveness trials of prevention programmes targeting youth.

Karin Guldbrandsson, MPH, PhD, is working at the Swedish National Institute of Public Health and affiliated to the Department of Public Health Sciences at the Karolinska Institutet in Sweden. Her main research area is implementation of public health interventions.