PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
Space Planners’ Perception of an Assessment Instrument for Briefs in the Pre-Design Phase of New Healthcare Environments Marie Elf, RN, PhD, and Helle Wijk, RN, DMSc
ABSTRACT OBJECTIVE: The main purpose of the study was to investigate the usability of a newly developed instrument designed to assess the content and quality briefs (CQB-I) in programs for new healthcare environments. We studied the perception of using the instrument on a group of space planners. BACKGROUND: The study is part of a larger project designed to ensure the quality of the planning of new healthcare environments with a focus on documents (programs or briefs) created in the early phases of the planning process. In this study, we used an instrument that measures the clarity of the mission statement for the project based on user needs and care activities that will take place in the new healthcare environment. The instrument further evaluates whether there are clear patient-related outcome measures specified and whether the information in the documents is person-oriented and evidence-based.
AUTHOR AFFILIATIONS: Marie Elf is an associate professor and a senior lecturer in the Department of Nursing in the School of Education, Health and Social Studies in Högskolan Dalarna, Falun, Sweden, and at Chalmers University of Technology, Department of Architecture, Göteborg, Sweden, as well as an associate in the HELD Research Group at the Karolinska Institutet. Helle Wijk is an associate professor in nursing at the Sahlgrenska Academy, Institute of Health and Care Sciences at Gothenburg University, Göteborg, Sweden.
METHODS: The study used a mixed-method design where the relevance and usability of the instrument was estimated and a focus group interview was conducted. RESULTS: The study showed that a CQB-I is perceived to be a relevant instrument primarily as a guide for programming, but also for monitoring the produced programs. CONCLUSIONS: Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the early phases of a planning process by orienting planning participants to user needs and facilitating the discussion and articulation of clear performance indicators. KEYWORDS: Design process, evidence-based design, outcomes, planning, project management
ACKNOWLEDGMENTS: The study was partly funded by grants from SKL and Högskolan Dalarna. Thanks to Maria Svedbo Engström, PhD, student, School of Health and Social Studies, Falun, Sweden, for data collection. PREFERRED CITATION: Elf, M., & Wijk, H. (2014). Space planners’ perception of an assessment instrument for briefs in the pre-design phase of new healthcare environments. Health Environments Research & Design Journal, 8(1), 67–80.
CORRESPONDING AUTHOR: Marie Elf, [email protected]
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 67
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
D
ecisions about the design of new healthcare environments are extremely important because such decisions can influence people and work processes several years into the future and often require large financial commitments (Rybkowski, 2009). The implementation of new methods and models of care to achieve effective, safe, and person-centered care is largely dependent on the layout of the space (Henriksen, Isaacson, Sadler, & Zimring, 2007). The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented. The aim of the planning process is to create a healthcare environment that contributes to patient health and well being and that allows for high quality and safe care (Gesler, Bell, Curtis, Hubbard, & Francis, 2004; Huisman, Morales, van Hoof, & Kort, 2012). Healthcare environments must also be designed so that patients, families, and healthcare professionals feel welcome, visible, and included. Balancing and satisfying these different requirements can be challenging. P-DP for a new healthcare environment thus involves a thorough analysis of the project’s patient-related objectives and the processes (care activities) and space conditions that are necessary to achieve these objectives (Curtis, Gesler, Priebe, & Francis, 2009; Steinke, Webster, & Fontaine, 2010; Ulrich, Berry, Quan, & Parish, 2010; Vischer & Zeisel, 2008). This approach illuminates the need for the analysis to be a shared responsibility among the various stakeholders, including those involved in the healthcare, architecture, planning, and construction facets of the project.
The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented.
The P-DP is often led by a space planner (programmer) with support from the real estate organization. During this stage, an initial outline brief (or program) is written prior to the presentation of a detailed layout proposal. This outline brief is a key document for decision making in the P-DP process (Szigeti & Davis, 2002) and is considered essential for high quality construction projects; it should inform and control the project from the planning to construction phases (Ryd & Fristedt, 2007). Therefore, this document must be systematic, person-oriented (based on user requirements), and evidence-based (Preiser & Vischer, 2005; Vischer & Zeisel, 2008).
In this study, we explored the perceptions of a group of space planners who were using a new instrument designed to assess the content and quality of initial briefs created in P-DP for new healthcare environments. This study was part of a larger project designed to ensure the quality of the P-DP of new healthcare environments, with a focus on the outline brief created during P-DP. We pre-tested an instrument called the Content and Quality in Briefs Instrument (CQB-I; see Table 1) to determine whether a project has a clear mission statement based on the needs of the users and the care activities that will occur in the new healthcare environment (Elf, Svedbo Engström, & Wijk, 2011). Furthermore, this instrument can be used to determine whether clear, patient-related outcome measures have been specified and whether the information in the
68 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
Table 1. Content and Quality in Briefs Instrument (CQB-I) and the Assessed Relevance Measured Using a Content Validity Index (CVI) (n = 9)
Quality Level
CVI
1. Is there a description of why a new healthcare environment is needed?
0
1
2
1.0
2. Are the needs of the patients included in the description of why a new healthcare environment is being investigated?
0
1
2
0.75
3. Are the needs of the significant others included in the description of why a new healthcare environment is being investigated?
0
1
2
0.5
4. Are the needs of the healthcare personnel included in the description of why a new healthcare environment is being investigated?
0
1
2
1.0
5. Is there a description of the organization?
0
1
2
0.88
6. Is there a description of the healthcare-related activities that are to take place in the new healthcare environment?
0
1
2
0.88
7. Is there a description of the relationships among spaces that are to be housed in the new healthcare environment?
0
1
2
1.0
8. Are there explicitly described outcomes for the new healthcare environment?
0
1
2
0.88
9. Are the needs of patients included in the outcomes for the new healthcare environment?
0 1 2 0.75
10. Are the needs of the significant others included in the outcomes for the new healthcare environment?
0
1
2
0.75
11. Are the needs of the healthcare personnel included in the outcomes for the new healthcare environment?
0
1
2
1.0
12. Is there a description of the types of post-occupancy evaluation of the new healthcare environment?
0
1
2
0.63
13. Is there any reference to evidence?
0
1
2
0.63
14. Is there any reference to the national quality indicators for high-quality healthcare?
0
1
2
0.25
∑ 0.79
documents is person oriented and evidence based. The CQB-I was developed through a deductive and iterative process. The items were generated through an extensive literature review and an expert evaluation on what should be included in briefs. The development of the instrument has been reported earlier by Elf, Svedbo Engström, and Wijk (2011). This instrument has been validated by experts (architects, space planners, and researchers) (Elf, Svedbo Engström, & Wijk, 2011, 2012); however, space planners’ perceptions of the instrument’s usability have not been studied. Therefore, we wanted to explore and pre-test the usability of the instrument among space planners with daily involvement in planning of new healthcare environments.
Background The creation of the CQB-I arose from the expressed need to develop methods and processes for quality control in the early planning process of a new healthcare environment before a detailed layout is presented (Ryd & Fristedt, 2007;
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 69
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Szigeti & Davis, 2002). An effective and quality-driven planning process has been described as essential for achieving a good result—one that is functional, sustainable, meets the needs of the individuals in the environment, and is cost-effective over the long term (Ulrich et al., 2008). Research has revealed deficiencies in the quality of the planning process in several areas: combining knowledge from healthcare and architecture (evidence-based design), focusing on the needs of users (personal orientation) and clarifying patient-related outcome measures (PROMs) for later evaluations of the building (Preiser, 1995; Ulrich et al., 2008; Vischer, 2009). For example, in Sweden the National Board of Health, criteria for good healthcare—knowledge-derived from health research—can be transferred to the premises planning of new care environments to contribute to person-oriented, safe, and effective care (Henriksen, Isaacson, Sadler, & Zimring, 2007).
The Need to Assess Briefs There is a need to assess the content and quality of briefs as part of the continuous quality improvement process (Vischer & Zeisel, 2008). Research in architecture, planning, and construction has demonstrated the importance of well-developed briefs that are user-oriented, evidence-based, and related to the strategic plan of the healthcare organization (Barrett & Balderly, 2003; Barrett, Hudson, & Stanley, 1999; Blyth & Worthington, 2001; Preiser & Vischer, 2005; Vischer & Zeisel, 2008). In general, the brief should include explicit information about the project objectives and specifications (functionality and the ability to support the activities to be performed in the An effective and quality-driven new environment) based on end-user requirements and needs planning process has been (Barrett & Stanley, 1999; Blyth & Worthington, 2001; Kelly & Hunter, 2005; Preiser & Vischer, 2005; Ryd & Fristedt, described as essential for 2007). It is important that the initial brief relates to the straachieving a good result—one that tegic brief of the organization and describes how the design of is functional, sustainable, meets the building will contribute to the organization’s stated strategic goals (Kelly & Hunter, 2005; Blyth & Worthington, 2001). the needs of the individuals in the environment, and is cost-effective In addition, there is an increasing requirement that planning and written briefs include measurable outcomes and evidenceover the long term. based information (Vischer, 2008). High quality briefs contain explicit outcomes that can serve as tools for evaluating the building when in use (post-occupancy evaluation) and are a natural component of an evidence-based approach (Baird, 2001). Despite numerous recommendations on what a brief should contain, there is a dearth of studies that have actually studied the content and quality of brief (Elf, Svedbo Engström, & Wijk, 2012). Elf and Malmqvist (2009) conducted a review of the information contained in briefs created during the planning process for healthcare environments. The results showed that only a few briefs incorporated explicit patient-focused goals for the project (2 out of 21 briefs), measurable patient outcomes (2 of 21), or references to new evidence (10 of 21). Less than half of the audited briefs contained a clear description of the objectives of the organization and the activities that were to occur in the planned healthcare environment. Furthermore, the results showed that the team perspective was
70 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
frequently omitted, and the care process was often described solely from a medical perspective. This review was performed using a qualitative approach (content analysis) due to the absence of an objective evaluating instrument. Similar results were found in a recent nationwide study of briefs in Sweden (Elf, Svedbo Engström, & Wijk, 2012).
Briefs as Part of Evidence-Based Design An appropriate brief provides a basis for an evidence-based design (EBD) process (Preiser & Vischer, 2005; Steinke, Webster, & Fontaine, 2010). EBD can be defined as a critical and reflective process in which decisions about the design of the environment are based on research, experience, and the user’s experience of the building (Stankos & Schwartz, 2007; Vischer, 2009; Ulrich, Berry, Quan, & Parish, 2010). EBD is closely related to continuous quality improvement models, in which objectives within the healthcare environment are based on the best available research, knowledge, and experience and in which explicit evaluation measures are presented at the beginning of a project. The results of these evaluations are used as experience and input in the subsequent planning processes. Presentation of the Instrument The CQB-I can be used to determine whether the project includes a clear mission statement based on user needs, a description of the care activities that will occur in the new healthcare environment, clear patient-related outcome measures, and evidence-based information (Table 2). Furthermore, the instrument design incorporates a systems thinking approach, that is, it is important to first analyze the mission of the care and the types of activities that will occur in the environment and then to apply the results of this analysis when determining what types of facilities are required. The instrument includes a manual to support this assessment (Table 2).
Table 2. Example of a Manual to Support Assessments with the CQB-I 1. Is there a description of why a new healthcare environment is needed? 0: Not at all. 1: There is limited information on why a new healthcare environment is being investigated, though not clearly formulated. 2: There is a clearly worded description of why a new healthcare environment is being investigated. 2. Are the needs of patients included in the description of why the new healthcare environment is being investigated? 0: Not at all. 1: There is limited information on the patient’s needs in the description of why the new health care environment is investigated. 2: There is a clear description of the needs of the patients in the description of why a new healthcare environment is being investigated.
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 71
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Methods A mixed-method design (Morse & Niehaus, 2007) was used to explore the perspectives of the space planners regarding the CQB-I and its usability. Mixedmethods research is defined as research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches and methods in a single study or a brief of inquiry (Creswell & Tashakkori, 2007). Data were collected using observations and field notes, in addition to a questionnaire concerning the participants’ perspectives on the relevance of the instrument and the accompanying manual by using Content Validity Index (CVI) (Polit & Beck, 2006). To obtain further information about the usability of the CQB-I, a focus group interview with space planners was conducted and videotaped.
Participants and Selection The participants (n = 9) all had a position as a space planner, with backgrounds in nursing (n = 8) or engineering (n = 1), and were affiliated with nine different planning and construction organizations in Sweden. The participants formed a convenience sample (they were chosen because they were gathered for a space planner meeting). Informed consent was obtained before the study began. Data Collection and Procedures The data collection occurred at a regular meeting for planners of new healthcare environments. The participants were asked to individually review a pre-selected brief using the CQB-I and to then assess the relevance of the instrument and the related manual. A data collector familiar with the CQB-I was present, observed the process and took field notes. The participants were able to ask questions throughout the review process. In particular, the data collector noted the length of time required to review the brief and whether the participants showed any uncertainty or hesitation about any of the questions in the CQB-I or the associated manual.
Outcomes and Instruments The perceptions of the planners regarding the relevance of the instrument were assessed using a content validity index (CVI) (Polit & Beck, 2006), a method of assessing content validity. Assessment of the Relevance of the Instrument CVI is a guide for further analysis of the relevance of items and can be used on small groups (5–10 people are recommended). The important thing is that the group reflects experts in the field. The relevance of the entire instrument (Scale Content Validity Index, S-CVI) and of each item in the instrument (I-CVI) was ascertained. The participants were asked to individually specify the relevance of
72 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
each sub-question on the instrument using a 4-point scale, where 0 = “not relevant,” 1 = “somewhat relevant,” 2 = “relevant,” and 3 = “very relevant.” In addition, the participants were asked to indicate the reasons for their judgments and were invited to provide comments, to suggest revisions to improve the instrument’s content and clarity, and to note any questions that should be added.
Assessment of the Relevance of the Associated Manual The manual that accompanies the CQB-I was also assessed using a questionnaire specifically developed for the assessment of manuals (Polit & Beck, 2006). The participants were asked to assess the extent to which the manual supported the evaluation of the brief by choosing between several response options. The scoring involved a 3-point scale, where 0 = “not at all,” 1 = “to a certain extent, but the difference between the response alternatives is unclear,” and 2 = “to a great extent; there is a clear difference between the response alternatives.” In addition, the participants were asked to provide any further explanations of their ratings, any comments and suggestions for revision, and how the differences between the alternatives might be made clearer. Planners’ Perceptions Regarding the Instrument After the participants had tested the CQB-I, a semi-structured focus group interview using an interview guide with open-ended questions was conducted to assess the usefulness of the CQB-I. The questions were: 1. How and in what way do you think the CQB-I could be useful for your work? 2. Are there any important components missing from the CQB-I? The interview lasted for approximately 1 hour; it was videotaped and transcribed verbatim.
Data Analysis The focus group interview and the field note data were analyzed according to the principles of qualitative content analysis with a manifest approach (Graneheim & Lundman, 2004). The quantitative data were analyzed using descriptive statistics. The content analysis was performed in three stages by two independent researchers. An initial reading of the text was performed to obtain an overview of the content. Subsequently, the text was systematically read to identify units of greater meaning with common content. The next step was to systematize the data into a number of descriptive categories.
Analysis of the Relevance of the CQB-I The relevance of the entire instrument (S-CVI) was calculated by adding the mean values for all of the questions and dividing the sum by the number of ques-
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 73
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
tions. The relevance of each item (I-CVI) (presumably, quite relevant or very relevant) was calculated by dividing the user assessment by the total number of participants (Polit & Beck, 2006). The limits were ≥ 0.78 for I-CVI and ≥ 0.90 for S-CVI.
Results The analysis of the focus group interview results revealed three main categories: 1. Usability and relevance of the instrument, 2. Limitations of the instrument, and 3. Areas for improvement. The findings from the focus group interview are presented together with the results from the questionnaires.
Usability and Relevance of the CQB-I The participants expressed that the CQB-I could be relevant and valuable for their work. The instrument as a whole and the individual items were generally considered to be relevant (see Table 1, above). The planners indicated that the instrument could be applied as an assessment tool of the content and quality of the briefs but also as a guide in the planning process. They stated that the instrument could be especially valuable for highlighting the patient’s perspective in a P-DP and the associated brief, as well as for helping to provide the stakeholders in the healthcare organization with valuable questions important to consider in a P-DP. Participants also expressed that the instrument could be an especially beneficial resource for a new planner gaining familiarity with a new role. They noted that the instrument could provide a basis for achieving a national consensus on the content in briefs for healthcare environments. The participants emphasized that the instrument could serve The instrument could be applied as a proactive framework used to prevent important issues in as an assessment tool of the the process from being ignored, for example, by elucidating the healthcare environment objectives from the perspective of content and quality of the briefs patients and their relatives. According to the space planners, but also as a guide in the healthcare organizations rarely consider the patient-based perplanning process. spective when planning a new healthcare environment—as one participant expressed, “There is something else that causes a change, and then it is not the patient-based perspective that comes first; it is in the background.” Other concerns, such as the needs of professionals, infection control requirements, or the prevention of overcrowded units, typically dominate the healthcare environment planning discussion instead. The participants considered that a self-audit of briefs (and, indirectly, of their planning process) would be key to ensuring important aspects of the process are not neglected and to maintaining focus on the patient-based perspective:
74 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
As a checklist, when looking through a brief, we look at the perspectives of the patients and relatives, if available; to ensure that everything is the way it’s supposed to be to help the business. We say that now you will have to provide figures, and you have to get to these parts so that all parts are included. [The instrument] can be a great way to increase the quality of the briefs we develop. There may be a way to put a little more focus on the business and that they need to work more with their business, organization, etc.
The planners scored several questions in the instrument to be less relevant, such as questions 3, 12, 13, and 14 (see Table “There is something else that 1, above). At the focus group interview, several of the group’s causes a change, and then it is not members questioned whether the documents should include a the patient-based perspective that description of the organization and the activities planned for in the environment. They considered that it was better and eas- comes first; it is in the background.” ier to refer to the strategic plan of the organization than mention this in the briefs. In addition, some planners expressed that the question concerning the quality indicators of good healthcare was irrelevant, and others questioned why the instrument included that item at all. The results were confirmed by estimating the relevance of the items using CVI (see Table 1, above).
Limitations of the Instrument Although the group considered the instrument useful, they also noted several limitations, such as the risk that checklists may not be completed and used in their daily work. The group expressed workload comments, including, “You definitely need checklists for self-monitoring, but they are often not used.” Additionally, it was mentioned that it is important to consider what type of brief is being evaluated, that is, at what point in the process is the evaluation being written and what is its purpose. The brief became increasingly detailed as the process continued, and the audit instrument then also needed more detailed questions:
“You definitely need checklists for self-monitoring, but they are often not used.”
Briefs must be examined from multiple perspectives because different professionals in the planning process focus on different details in a construction project. For example, healthcare professionals should perhaps study how health hygiene is considered in a brief; engineers may review briefs from a technology perspective.
The Practical Usability The results from the field notes indicated that the participants did not experience difficulty assessing the briefs with the help of the CQB-I. The participants showed no uncertainty or disagreement over the use of the instrument and had no concerns during the assessment. Additionally, the majority of the participants
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 75
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
used all questions in the instrument. The duration of the assessment was approximately 20 minutes. The subsequent focus group interview revealed a great commitment to the results and the group expressed that the CQB-I was easy to use, “just to look if the issues were or were not included.” The participants reported that the manual provided support for using the instrument. Most of the questions in the manual were considered largely or very supportive.
Improvement Areas Generally, the planners agreed that the instrument fulfilled its purpose, could contribute to the standardization of briefs, and could increase the overall quality of briefs, but that it lacked questions related to cost-benefit analyses and the impact on cost of various design proposals. Comments included, “It’s all about money in the end,” and “Health economics is not just about saving money; it is also about using money in the right way.” The test situation led to reflections on other types of audit instruments for various briefs: You could compare various audit documents written from different perspectives if you have an audit instrument that you can quickly check that has questions that are absolutely important if you are a relative, or an instrument with the issues that are most important to patients, employees, technology, policy makers, or economists. Decision makers need information on finances, goals, and benefits … they cannot make decisions without looking at them from the owner’s perspective. If the brief is reasonable in terms of economics and technology, it will be as efficient as we have imagined.
There were also suggestions that brief must consider more “soft” dimensions in terms of interior design, environment, colors, and the atmosphere encountered upon entering a room—these issues should be addressed with an assessment instrument. The participants also expressed that the instrument failed to consider issues of technology solutions in a healthcare environment. Finally, the participants noted that the use of the instrument would be enhanced by its integration of a computer platform.
Discussion This study showed that the CQB-I was perceived to be a relevant instrument primarily as a guide for P-DP but also for assessing the content of written briefs. The instrument was originally developed for a Swedish context but can be translated and adopted for international contexts. The study was small and had a limited selection of participants; therefore, the results cannot be generalized. By focusing on prospective space planners, that is, users of such instrument and their perceptions of the instrument, this study nevertheless provides valuable guidance
76 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
for the further development of the CQB-I and similar instruments. Instrument development is a lengthy, iterative process (Polit & Beck, 2006) that requires several validity and reliability tests before the instrument can be used in practice; usability is a key consideration. The CQB-I was designed primarily for assessing the content and quality written briefs created in the P-DP of new healthcare environments for a Swedish context, when the goal is to determine the needs and requirements and should be addressed in the briefs. That the participants thought that the instrument could act as a guide for the space planner or team planning for new healthcare environments was unexpected. The original idea behind CQB-I is that the instrument enables an assessment of whether the planning team has analyzed and formulated the goals and needs of the new healthcare environment based on user needs and evidence-based knowledge, which have been identified as essential for a planning process to achieve good results (Becker & Parson, 2007; Vischer, 2009; Yu, Shen, Kelly, & Hunter, 2007). The use of an instrument such as the CQB-I for briefs can guide discussions of the content and quality, and thus the planning process, toward a more user-oriented and knowledge-based process. There is no equivalent tool for the assessment of briefs, which suggests that such studies have not been the focus in the past. The planners did not consider that a brief should include a description of the healthcare organization; instead, they thought that a reference to existing documents would be sufficient. Their argument was that briefs at this stage are overall descriptions and should only contain relevant information. While it is true that a brief should be short and concise, relevant information must be included. References to documents can be included and are valuable. However, if the descriptions of the organization and activities are not up to date or easily accessible, for example, problems can result, and therefore a warning flag should be raised. Planning for a new healthcare environment involves being a part of the strategic development of healthcare, and the goals and activities of the organization must be analyzed when planning the design of spaces (Becker & Parson, 2007; Blyth & Worthington, 2001). However, we need to explore the value of including a description of the organization in the briefs further on. It was interesting that the planners reported that they often had to introduce the patient-based perspective to the planning process, although they considered this task to be the primary responsibility of the healthcare organization. The group stressed, however, that it was the responsibility of the planners to summarize the discussion and gather any information necessary for the continued planning and design process. Therefore, the planners need information about the patient group and the activities that will occur in the building. The group also stated that key issues discussed in the planning process were not always documented, which might have negative or even dangerous consequences for the entire construction process. Therefore, it might be beneficial to have a checklist to ensure that the key issues were addressed during the process. The planners did not consider the quality indicators of good health worthy of mention in a brief. However, we emphasize that these indicators should be a
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 77
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
basis for P-DP. These indicators were developed from research in healthcare, and their implementation in local planning is necessary for the process to be evidence-based. This finding may indicate that the space planners were not familiar with the indicators when the study was performed and therefore did not consider the indicators relevant. The issue of indicators arose during the subsequent focus group interview, and the participants changed their position slightly upon understanding the reasoning behind the indicators. The group also discussed the difficulties of using a single audit tool for different stages of the planning process. Various real estate organizations design the planning and design process differently in terms of who writes the brief (a member of the healthcare organization or a planner employed by the real estate organization), which people are involved in the process, how the process is designed, the decision level addressed by the brief and the recipients of the brief. Although the group’s concerns are relevant, the CQB-I is generic and indicates important areas to be included in the brief for patient healthcare spaces. The participants also remarked that there was no defined name for a brief produced in the early stages of the planning process. These documents are variously called “business descriptions for space designs,” “requirements analyses,” “feasibility studies,” and “functional briefs.” Thus, the names and contents of the briefs can vary. However, several studies have demonstrated that although briefs in the early stages have different names, their purposes and contents are relatively similar (Ryd, 2001). Briefing in the early stages is a complex process, and one must always ask whether it is being performed efficiently. Research has shown that ensuring the quality of the early stages of the planning process is cost effective (Ulrich, Berry, Quan, & Parish, 2010). In addition to costs and benefits, society is increasingly focused on the importance of the implementation of evidence-based knowledge in healthcare, with a particular focus on factors that support patient-centered care. An instrument such as the CQB-I can be used to evaluate and develop briefs, thus making it more effective. The CQB-I may contribute to the continuous improvement process of briefs to identify areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the process more visible. Briefs can be compared over time and between places and situations. The instrument clarifies the principles of quality management and an evidence-based approach to planning by enabling an assessment of the documents that result directly and indirectly from the process.
Conclusion In summary, this study showed that the CQB-I satisfied its purpose and was easy to use. It is well known that it is difficult to transfer research results into healthcare services and into briefings. The CQB-I is an instrument that can help to bring research results into the planning process and contribute to discussions and collaboration among the stakeholders during the early stages. This instrument includes a user-oriented approach, enables clear discussions and documentation of outcomes, and consequently supports quality improvement in the planning process of new healthcare environments. However, this instrument should be developed and tested in larger groups.
78 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
Implications for Practice • There is a need to assess the content and quality of briefs as part of the continuous quality improvement process. • An instrument such as the CQB-I can be used to evaluate and develop briefs, by identifying areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the planning process more visible. • Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the planning process by orienting to the user needs, facilitating discussion and articulation of clear performance and related quality improvement indicators.
References Baird, G. (2001). Post-occupancy evaluation and probe: A New Zealand perspective. Building Research & Information, 29(6), 469–472. Barrett, P., & Baldery, D. (2003). Facilities management: Towards best practice. Oxford, England: Blackwell Science. Barrett, P., & Stanley, C. (1999). Better construction briefing. Oxford, England: Blackwell Science. Barrett, P. S., Hudson, J., & Stanley, C. (1999). Good practice in briefing: The limits of rationality. Automation in Construction, 8(6), 633–642. Becker, F., & Parson, S. K. (2007). Hospital facilities and the role of evidence-based design. Journal of Facility Management, 5(4), 263–274. Blyth, A., & Worthington, J. (2001). Managing the brief for better design. London, England: Spoon Press. Creswell, J. W., & Tashakkori, A. (2007). Developing publishable mixed methods manuscripts. Journal of Mixed Methods, 1, 107–111. Curtis, S., Gesler, W., Priebe, S., & Francis, S. (2009). New spaces of inpatient care for people with mental illness: A complex ‘rebirth’ of the clinic? Health & Place, 15(1), 340–348. Elf, M., & Malmqvist, I. (2009). An audit of content and quality in briefs in Sweden. Journal of Facility Management, 7(3), 198–211. Elf, M., Svedbo Engström, M., & Wijk, H. (2011). Development of the content and quality in briefs instrument (CQB-I). Health Environments Research & Design Journal, 5(3), 74–88. Elf, M., Svedbo Engström, M., & Wijk, H. (2012). An assessment of briefs used for designing healthcare environments: A survey in Sweden. Construction Management and Economics, 30(10), 835–844. Gesler, W., Bell, M., Curtis, S., Hubbard, P., & Francis, S. (2004). Therapy by design: Evaluating the UK hospital building program. Health & Place, 10(2), 117–128.
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 79
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Graneheim, U., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. Henriksen, K., Isaacson, S., Sadler, B. L., & Zimring, C. M. (2007). The role of the physical environment in crossing the quality chasm. The Joint Commission Journal on Quality and Patient Safety, 33(11 Suppl), 68–80. Huisman, E. R. C. M., Morales, E., van Hoof, J., & Kort, H. S. M. (2012, December). Healing environment: A review of the impact of physical environmental factors on users. Building and Environment, 58, 70–80. Kelly, J, and Hunter, K. (2005). Briefing from facilities management perspective. Facilities, 23(7/8), 356–367. Morse, J., & Niehaus, L. (2007). Combining qualitative and quantitative methods for mixed method designs. In P. Munhall (Ed.), Nursing research: A qualitative perspective (4th ed.) (pp. 541– 554). Boston, MA: Jones & Bartlett. Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing and Health, 29(5), 489–497. Preiser, W. (1995). Post-occupancy evaluation: How to make buildings work better. Facilities, 13(11), 19–28. Preiser, W., & Vischer, J. (2005). Assessing building performance. London, England: Elsevier Butterworth-Heinemann. Rybkowski, Z. K. (2009). The application of root cause analysis and target value design to evidence-based design in the capital planning of healthcare facilities. (Dissertation). University of California, Berkeley, Berkeley, CA. ProQuest, UMI Dissertation Publishing. Ryd, N. (2001). Byggnadsbrief som informationsbärare i byggprocessen. En studie av byggnadsbriefmering. [The architectural brief as carrier of client-information during the building process.] Göteborg, Sweden: Chalmers University of Technology. Ryd, N., & Fristedt, S. (2007). Transforming strategic briefing into project briefs. A case study about client and contractor collaboration. Facilities, 25(5/6), 185–202. Steinke, C., Webster, L., & Fontaine, M. (2010). Evaluating building performance in healthcare facilities: An organizational perspective. Health Environments Research & Design Journal, 3(2), 63–83. Szigeti, F., & Davis, G. (2002). Forum: The turning point for linking briefing and POE? Building Research & Information, 30(1), 47–53. Ulrich, R. S., Berry, L. L., Quan, X., & Parish, J. T. (2010). A conceptual framework for the domain of evidence-based design. Health Environments Research & Design Journal, 4(1), 95–114. Ulrich, R. S., Zimring, C., Zhu, X., DuBose, J., Seo, H.-B., Choi, Y.-S., … Joseph, A. (2008). A review of the research literature on evidence-based healthcare design. Health Environments Research & Design Journal, 1(3), 61–125. Vischer, J. (2009). Applying knowledge on building performance: From evidence to intelligence. Intelligent Building International, 1(4), 239–248. Vischer, J., & Zeisel, J. (2008, July). Process management: Bridging the gap between research and design. Design & Health Scientific Review, 57–61. Yu, T. W. A., Shen, Q., Kelly, J., & Hunter, K. (2007). An empirical study of the variables affecting construction project briefing/architectural briefing. International Journal of Project Management, 25(2), 198–212.
80 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
RESEARCH
Space Planners’ Perception of an Assessment Instrument for Briefs in the Pre-Design Phase of New Healthcare Environments Marie Elf, RN, PhD, and Helle Wijk, RN, DMSc
ABSTRACT OBJECTIVE: The main purpose of the study was to investigate the usability of a newly developed instrument designed to assess the content and quality briefs (CQB-I) in programs for new healthcare environments. We studied the perception of using the instrument on a group of space planners. BACKGROUND: The study is part of a larger project designed to ensure the quality of the planning of new healthcare environments with a focus on documents (programs or briefs) created in the early phases of the planning process. In this study, we used an instrument that measures the clarity of the mission statement for the project based on user needs and care activities that will take place in the new healthcare environment. The instrument further evaluates whether there are clear patient-related outcome measures specified and whether the information in the documents is person-oriented and evidence-based.
AUTHOR AFFILIATIONS: Marie Elf is an associate professor and a senior lecturer in the Department of Nursing in the School of Education, Health and Social Studies in Högskolan Dalarna, Falun, Sweden, and at Chalmers University of Technology, Department of Architecture, Göteborg, Sweden, as well as an associate in the HELD Research Group at the Karolinska Institutet. Helle Wijk is an associate professor in nursing at the Sahlgrenska Academy, Institute of Health and Care Sciences at Gothenburg University, Göteborg, Sweden.
METHODS: The study used a mixed-method design where the relevance and usability of the instrument was estimated and a focus group interview was conducted. RESULTS: The study showed that a CQB-I is perceived to be a relevant instrument primarily as a guide for programming, but also for monitoring the produced programs. CONCLUSIONS: Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the early phases of a planning process by orienting planning participants to user needs and facilitating the discussion and articulation of clear performance indicators. KEYWORDS: Design process, evidence-based design, outcomes, planning, project management
ACKNOWLEDGMENTS: The study was partly funded by grants from SKL and Högskolan Dalarna. Thanks to Maria Svedbo Engström, PhD, student, School of Health and Social Studies, Falun, Sweden, for data collection. PREFERRED CITATION: Elf, M., & Wijk, H. (2014). Space planners’ perception of an assessment instrument for briefs in the pre-design phase of new healthcare environments. Health Environments Research & Design Journal, 8(1), 67–80.
CORRESPONDING AUTHOR: Marie Elf, [email protected]
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 67
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
D
ecisions about the design of new healthcare environments are extremely important because such decisions can influence people and work processes several years into the future and often require large financial commitments (Rybkowski, 2009). The implementation of new methods and models of care to achieve effective, safe, and person-centered care is largely dependent on the layout of the space (Henriksen, Isaacson, Sadler, & Zimring, 2007). The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented. The aim of the planning process is to create a healthcare environment that contributes to patient health and well being and that allows for high quality and safe care (Gesler, Bell, Curtis, Hubbard, & Francis, 2004; Huisman, Morales, van Hoof, & Kort, 2012). Healthcare environments must also be designed so that patients, families, and healthcare professionals feel welcome, visible, and included. Balancing and satisfying these different requirements can be challenging. P-DP for a new healthcare environment thus involves a thorough analysis of the project’s patient-related objectives and the processes (care activities) and space conditions that are necessary to achieve these objectives (Curtis, Gesler, Priebe, & Francis, 2009; Steinke, Webster, & Fontaine, 2010; Ulrich, Berry, Quan, & Parish, 2010; Vischer & Zeisel, 2008). This approach illuminates the need for the analysis to be a shared responsibility among the various stakeholders, including those involved in the healthcare, architecture, planning, and construction facets of the project.
The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented.
The P-DP is often led by a space planner (programmer) with support from the real estate organization. During this stage, an initial outline brief (or program) is written prior to the presentation of a detailed layout proposal. This outline brief is a key document for decision making in the P-DP process (Szigeti & Davis, 2002) and is considered essential for high quality construction projects; it should inform and control the project from the planning to construction phases (Ryd & Fristedt, 2007). Therefore, this document must be systematic, person-oriented (based on user requirements), and evidence-based (Preiser & Vischer, 2005; Vischer & Zeisel, 2008).
In this study, we explored the perceptions of a group of space planners who were using a new instrument designed to assess the content and quality of initial briefs created in P-DP for new healthcare environments. This study was part of a larger project designed to ensure the quality of the P-DP of new healthcare environments, with a focus on the outline brief created during P-DP. We pre-tested an instrument called the Content and Quality in Briefs Instrument (CQB-I; see Table 1) to determine whether a project has a clear mission statement based on the needs of the users and the care activities that will occur in the new healthcare environment (Elf, Svedbo Engström, & Wijk, 2011). Furthermore, this instrument can be used to determine whether clear, patient-related outcome measures have been specified and whether the information in the
68 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
Table 1. Content and Quality in Briefs Instrument (CQB-I) and the Assessed Relevance Measured Using a Content Validity Index (CVI) (n = 9)
Quality Level
CVI
1. Is there a description of why a new healthcare environment is needed?
0
1
2
1.0
2. Are the needs of the patients included in the description of why a new healthcare environment is being investigated?
0
1
2
0.75
3. Are the needs of the significant others included in the description of why a new healthcare environment is being investigated?
0
1
2
0.5
4. Are the needs of the healthcare personnel included in the description of why a new healthcare environment is being investigated?
0
1
2
1.0
5. Is there a description of the organization?
0
1
2
0.88
6. Is there a description of the healthcare-related activities that are to take place in the new healthcare environment?
0
1
2
0.88
7. Is there a description of the relationships among spaces that are to be housed in the new healthcare environment?
0
1
2
1.0
8. Are there explicitly described outcomes for the new healthcare environment?
0
1
2
0.88
9. Are the needs of patients included in the outcomes for the new healthcare environment?
0 1 2 0.75
10. Are the needs of the significant others included in the outcomes for the new healthcare environment?
0
1
2
0.75
11. Are the needs of the healthcare personnel included in the outcomes for the new healthcare environment?
0
1
2
1.0
12. Is there a description of the types of post-occupancy evaluation of the new healthcare environment?
0
1
2
0.63
13. Is there any reference to evidence?
0
1
2
0.63
14. Is there any reference to the national quality indicators for high-quality healthcare?
0
1
2
0.25
∑ 0.79
documents is person oriented and evidence based. The CQB-I was developed through a deductive and iterative process. The items were generated through an extensive literature review and an expert evaluation on what should be included in briefs. The development of the instrument has been reported earlier by Elf, Svedbo Engström, and Wijk (2011). This instrument has been validated by experts (architects, space planners, and researchers) (Elf, Svedbo Engström, & Wijk, 2011, 2012); however, space planners’ perceptions of the instrument’s usability have not been studied. Therefore, we wanted to explore and pre-test the usability of the instrument among space planners with daily involvement in planning of new healthcare environments.
Background The creation of the CQB-I arose from the expressed need to develop methods and processes for quality control in the early planning process of a new healthcare environment before a detailed layout is presented (Ryd & Fristedt, 2007;
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 69
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Szigeti & Davis, 2002). An effective and quality-driven planning process has been described as essential for achieving a good result—one that is functional, sustainable, meets the needs of the individuals in the environment, and is cost-effective over the long term (Ulrich et al., 2008). Research has revealed deficiencies in the quality of the planning process in several areas: combining knowledge from healthcare and architecture (evidence-based design), focusing on the needs of users (personal orientation) and clarifying patient-related outcome measures (PROMs) for later evaluations of the building (Preiser, 1995; Ulrich et al., 2008; Vischer, 2009). For example, in Sweden the National Board of Health, criteria for good healthcare—knowledge-derived from health research—can be transferred to the premises planning of new care environments to contribute to person-oriented, safe, and effective care (Henriksen, Isaacson, Sadler, & Zimring, 2007).
The Need to Assess Briefs There is a need to assess the content and quality of briefs as part of the continuous quality improvement process (Vischer & Zeisel, 2008). Research in architecture, planning, and construction has demonstrated the importance of well-developed briefs that are user-oriented, evidence-based, and related to the strategic plan of the healthcare organization (Barrett & Balderly, 2003; Barrett, Hudson, & Stanley, 1999; Blyth & Worthington, 2001; Preiser & Vischer, 2005; Vischer & Zeisel, 2008). In general, the brief should include explicit information about the project objectives and specifications (functionality and the ability to support the activities to be performed in the An effective and quality-driven new environment) based on end-user requirements and needs planning process has been (Barrett & Stanley, 1999; Blyth & Worthington, 2001; Kelly & Hunter, 2005; Preiser & Vischer, 2005; Ryd & Fristedt, described as essential for 2007). It is important that the initial brief relates to the straachieving a good result—one that tegic brief of the organization and describes how the design of is functional, sustainable, meets the building will contribute to the organization’s stated strategic goals (Kelly & Hunter, 2005; Blyth & Worthington, 2001). the needs of the individuals in the environment, and is cost-effective In addition, there is an increasing requirement that planning and written briefs include measurable outcomes and evidenceover the long term. based information (Vischer, 2008). High quality briefs contain explicit outcomes that can serve as tools for evaluating the building when in use (post-occupancy evaluation) and are a natural component of an evidence-based approach (Baird, 2001). Despite numerous recommendations on what a brief should contain, there is a dearth of studies that have actually studied the content and quality of brief (Elf, Svedbo Engström, & Wijk, 2012). Elf and Malmqvist (2009) conducted a review of the information contained in briefs created during the planning process for healthcare environments. The results showed that only a few briefs incorporated explicit patient-focused goals for the project (2 out of 21 briefs), measurable patient outcomes (2 of 21), or references to new evidence (10 of 21). Less than half of the audited briefs contained a clear description of the objectives of the organization and the activities that were to occur in the planned healthcare environment. Furthermore, the results showed that the team perspective was
70 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
frequently omitted, and the care process was often described solely from a medical perspective. This review was performed using a qualitative approach (content analysis) due to the absence of an objective evaluating instrument. Similar results were found in a recent nationwide study of briefs in Sweden (Elf, Svedbo Engström, & Wijk, 2012).
Briefs as Part of Evidence-Based Design An appropriate brief provides a basis for an evidence-based design (EBD) process (Preiser & Vischer, 2005; Steinke, Webster, & Fontaine, 2010). EBD can be defined as a critical and reflective process in which decisions about the design of the environment are based on research, experience, and the user’s experience of the building (Stankos & Schwartz, 2007; Vischer, 2009; Ulrich, Berry, Quan, & Parish, 2010). EBD is closely related to continuous quality improvement models, in which objectives within the healthcare environment are based on the best available research, knowledge, and experience and in which explicit evaluation measures are presented at the beginning of a project. The results of these evaluations are used as experience and input in the subsequent planning processes. Presentation of the Instrument The CQB-I can be used to determine whether the project includes a clear mission statement based on user needs, a description of the care activities that will occur in the new healthcare environment, clear patient-related outcome measures, and evidence-based information (Table 2). Furthermore, the instrument design incorporates a systems thinking approach, that is, it is important to first analyze the mission of the care and the types of activities that will occur in the environment and then to apply the results of this analysis when determining what types of facilities are required. The instrument includes a manual to support this assessment (Table 2).
Table 2. Example of a Manual to Support Assessments with the CQB-I 1. Is there a description of why a new healthcare environment is needed? 0: Not at all. 1: There is limited information on why a new healthcare environment is being investigated, though not clearly formulated. 2: There is a clearly worded description of why a new healthcare environment is being investigated. 2. Are the needs of patients included in the description of why the new healthcare environment is being investigated? 0: Not at all. 1: There is limited information on the patient’s needs in the description of why the new health care environment is investigated. 2: There is a clear description of the needs of the patients in the description of why a new healthcare environment is being investigated.
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 71
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Methods A mixed-method design (Morse & Niehaus, 2007) was used to explore the perspectives of the space planners regarding the CQB-I and its usability. Mixedmethods research is defined as research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches and methods in a single study or a brief of inquiry (Creswell & Tashakkori, 2007). Data were collected using observations and field notes, in addition to a questionnaire concerning the participants’ perspectives on the relevance of the instrument and the accompanying manual by using Content Validity Index (CVI) (Polit & Beck, 2006). To obtain further information about the usability of the CQB-I, a focus group interview with space planners was conducted and videotaped.
Participants and Selection The participants (n = 9) all had a position as a space planner, with backgrounds in nursing (n = 8) or engineering (n = 1), and were affiliated with nine different planning and construction organizations in Sweden. The participants formed a convenience sample (they were chosen because they were gathered for a space planner meeting). Informed consent was obtained before the study began. Data Collection and Procedures The data collection occurred at a regular meeting for planners of new healthcare environments. The participants were asked to individually review a pre-selected brief using the CQB-I and to then assess the relevance of the instrument and the related manual. A data collector familiar with the CQB-I was present, observed the process and took field notes. The participants were able to ask questions throughout the review process. In particular, the data collector noted the length of time required to review the brief and whether the participants showed any uncertainty or hesitation about any of the questions in the CQB-I or the associated manual.
Outcomes and Instruments The perceptions of the planners regarding the relevance of the instrument were assessed using a content validity index (CVI) (Polit & Beck, 2006), a method of assessing content validity. Assessment of the Relevance of the Instrument CVI is a guide for further analysis of the relevance of items and can be used on small groups (5–10 people are recommended). The important thing is that the group reflects experts in the field. The relevance of the entire instrument (Scale Content Validity Index, S-CVI) and of each item in the instrument (I-CVI) was ascertained. The participants were asked to individually specify the relevance of
72 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
each sub-question on the instrument using a 4-point scale, where 0 = “not relevant,” 1 = “somewhat relevant,” 2 = “relevant,” and 3 = “very relevant.” In addition, the participants were asked to indicate the reasons for their judgments and were invited to provide comments, to suggest revisions to improve the instrument’s content and clarity, and to note any questions that should be added.
Assessment of the Relevance of the Associated Manual The manual that accompanies the CQB-I was also assessed using a questionnaire specifically developed for the assessment of manuals (Polit & Beck, 2006). The participants were asked to assess the extent to which the manual supported the evaluation of the brief by choosing between several response options. The scoring involved a 3-point scale, where 0 = “not at all,” 1 = “to a certain extent, but the difference between the response alternatives is unclear,” and 2 = “to a great extent; there is a clear difference between the response alternatives.” In addition, the participants were asked to provide any further explanations of their ratings, any comments and suggestions for revision, and how the differences between the alternatives might be made clearer. Planners’ Perceptions Regarding the Instrument After the participants had tested the CQB-I, a semi-structured focus group interview using an interview guide with open-ended questions was conducted to assess the usefulness of the CQB-I. The questions were: 1. How and in what way do you think the CQB-I could be useful for your work? 2. Are there any important components missing from the CQB-I? The interview lasted for approximately 1 hour; it was videotaped and transcribed verbatim.
Data Analysis The focus group interview and the field note data were analyzed according to the principles of qualitative content analysis with a manifest approach (Graneheim & Lundman, 2004). The quantitative data were analyzed using descriptive statistics. The content analysis was performed in three stages by two independent researchers. An initial reading of the text was performed to obtain an overview of the content. Subsequently, the text was systematically read to identify units of greater meaning with common content. The next step was to systematize the data into a number of descriptive categories.
Analysis of the Relevance of the CQB-I The relevance of the entire instrument (S-CVI) was calculated by adding the mean values for all of the questions and dividing the sum by the number of ques-
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 73
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
tions. The relevance of each item (I-CVI) (presumably, quite relevant or very relevant) was calculated by dividing the user assessment by the total number of participants (Polit & Beck, 2006). The limits were ≥ 0.78 for I-CVI and ≥ 0.90 for S-CVI.
Results The analysis of the focus group interview results revealed three main categories: 1. Usability and relevance of the instrument, 2. Limitations of the instrument, and 3. Areas for improvement. The findings from the focus group interview are presented together with the results from the questionnaires.
Usability and Relevance of the CQB-I The participants expressed that the CQB-I could be relevant and valuable for their work. The instrument as a whole and the individual items were generally considered to be relevant (see Table 1, above). The planners indicated that the instrument could be applied as an assessment tool of the content and quality of the briefs but also as a guide in the planning process. They stated that the instrument could be especially valuable for highlighting the patient’s perspective in a P-DP and the associated brief, as well as for helping to provide the stakeholders in the healthcare organization with valuable questions important to consider in a P-DP. Participants also expressed that the instrument could be an especially beneficial resource for a new planner gaining familiarity with a new role. They noted that the instrument could provide a basis for achieving a national consensus on the content in briefs for healthcare environments. The participants emphasized that the instrument could serve The instrument could be applied as a proactive framework used to prevent important issues in as an assessment tool of the the process from being ignored, for example, by elucidating the healthcare environment objectives from the perspective of content and quality of the briefs patients and their relatives. According to the space planners, but also as a guide in the healthcare organizations rarely consider the patient-based perplanning process. spective when planning a new healthcare environment—as one participant expressed, “There is something else that causes a change, and then it is not the patient-based perspective that comes first; it is in the background.” Other concerns, such as the needs of professionals, infection control requirements, or the prevention of overcrowded units, typically dominate the healthcare environment planning discussion instead. The participants considered that a self-audit of briefs (and, indirectly, of their planning process) would be key to ensuring important aspects of the process are not neglected and to maintaining focus on the patient-based perspective:
74 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
As a checklist, when looking through a brief, we look at the perspectives of the patients and relatives, if available; to ensure that everything is the way it’s supposed to be to help the business. We say that now you will have to provide figures, and you have to get to these parts so that all parts are included. [The instrument] can be a great way to increase the quality of the briefs we develop. There may be a way to put a little more focus on the business and that they need to work more with their business, organization, etc.
The planners scored several questions in the instrument to be less relevant, such as questions 3, 12, 13, and 14 (see Table “There is something else that 1, above). At the focus group interview, several of the group’s causes a change, and then it is not members questioned whether the documents should include a the patient-based perspective that description of the organization and the activities planned for in the environment. They considered that it was better and eas- comes first; it is in the background.” ier to refer to the strategic plan of the organization than mention this in the briefs. In addition, some planners expressed that the question concerning the quality indicators of good healthcare was irrelevant, and others questioned why the instrument included that item at all. The results were confirmed by estimating the relevance of the items using CVI (see Table 1, above).
Limitations of the Instrument Although the group considered the instrument useful, they also noted several limitations, such as the risk that checklists may not be completed and used in their daily work. The group expressed workload comments, including, “You definitely need checklists for self-monitoring, but they are often not used.” Additionally, it was mentioned that it is important to consider what type of brief is being evaluated, that is, at what point in the process is the evaluation being written and what is its purpose. The brief became increasingly detailed as the process continued, and the audit instrument then also needed more detailed questions:
“You definitely need checklists for self-monitoring, but they are often not used.”
Briefs must be examined from multiple perspectives because different professionals in the planning process focus on different details in a construction project. For example, healthcare professionals should perhaps study how health hygiene is considered in a brief; engineers may review briefs from a technology perspective.
The Practical Usability The results from the field notes indicated that the participants did not experience difficulty assessing the briefs with the help of the CQB-I. The participants showed no uncertainty or disagreement over the use of the instrument and had no concerns during the assessment. Additionally, the majority of the participants
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 75
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
used all questions in the instrument. The duration of the assessment was approximately 20 minutes. The subsequent focus group interview revealed a great commitment to the results and the group expressed that the CQB-I was easy to use, “just to look if the issues were or were not included.” The participants reported that the manual provided support for using the instrument. Most of the questions in the manual were considered largely or very supportive.
Improvement Areas Generally, the planners agreed that the instrument fulfilled its purpose, could contribute to the standardization of briefs, and could increase the overall quality of briefs, but that it lacked questions related to cost-benefit analyses and the impact on cost of various design proposals. Comments included, “It’s all about money in the end,” and “Health economics is not just about saving money; it is also about using money in the right way.” The test situation led to reflections on other types of audit instruments for various briefs: You could compare various audit documents written from different perspectives if you have an audit instrument that you can quickly check that has questions that are absolutely important if you are a relative, or an instrument with the issues that are most important to patients, employees, technology, policy makers, or economists. Decision makers need information on finances, goals, and benefits … they cannot make decisions without looking at them from the owner’s perspective. If the brief is reasonable in terms of economics and technology, it will be as efficient as we have imagined.
There were also suggestions that brief must consider more “soft” dimensions in terms of interior design, environment, colors, and the atmosphere encountered upon entering a room—these issues should be addressed with an assessment instrument. The participants also expressed that the instrument failed to consider issues of technology solutions in a healthcare environment. Finally, the participants noted that the use of the instrument would be enhanced by its integration of a computer platform.
Discussion This study showed that the CQB-I was perceived to be a relevant instrument primarily as a guide for P-DP but also for assessing the content of written briefs. The instrument was originally developed for a Swedish context but can be translated and adopted for international contexts. The study was small and had a limited selection of participants; therefore, the results cannot be generalized. By focusing on prospective space planners, that is, users of such instrument and their perceptions of the instrument, this study nevertheless provides valuable guidance
76 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
for the further development of the CQB-I and similar instruments. Instrument development is a lengthy, iterative process (Polit & Beck, 2006) that requires several validity and reliability tests before the instrument can be used in practice; usability is a key consideration. The CQB-I was designed primarily for assessing the content and quality written briefs created in the P-DP of new healthcare environments for a Swedish context, when the goal is to determine the needs and requirements and should be addressed in the briefs. That the participants thought that the instrument could act as a guide for the space planner or team planning for new healthcare environments was unexpected. The original idea behind CQB-I is that the instrument enables an assessment of whether the planning team has analyzed and formulated the goals and needs of the new healthcare environment based on user needs and evidence-based knowledge, which have been identified as essential for a planning process to achieve good results (Becker & Parson, 2007; Vischer, 2009; Yu, Shen, Kelly, & Hunter, 2007). The use of an instrument such as the CQB-I for briefs can guide discussions of the content and quality, and thus the planning process, toward a more user-oriented and knowledge-based process. There is no equivalent tool for the assessment of briefs, which suggests that such studies have not been the focus in the past. The planners did not consider that a brief should include a description of the healthcare organization; instead, they thought that a reference to existing documents would be sufficient. Their argument was that briefs at this stage are overall descriptions and should only contain relevant information. While it is true that a brief should be short and concise, relevant information must be included. References to documents can be included and are valuable. However, if the descriptions of the organization and activities are not up to date or easily accessible, for example, problems can result, and therefore a warning flag should be raised. Planning for a new healthcare environment involves being a part of the strategic development of healthcare, and the goals and activities of the organization must be analyzed when planning the design of spaces (Becker & Parson, 2007; Blyth & Worthington, 2001). However, we need to explore the value of including a description of the organization in the briefs further on. It was interesting that the planners reported that they often had to introduce the patient-based perspective to the planning process, although they considered this task to be the primary responsibility of the healthcare organization. The group stressed, however, that it was the responsibility of the planners to summarize the discussion and gather any information necessary for the continued planning and design process. Therefore, the planners need information about the patient group and the activities that will occur in the building. The group also stated that key issues discussed in the planning process were not always documented, which might have negative or even dangerous consequences for the entire construction process. Therefore, it might be beneficial to have a checklist to ensure that the key issues were addressed during the process. The planners did not consider the quality indicators of good health worthy of mention in a brief. However, we emphasize that these indicators should be a
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 77
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
basis for P-DP. These indicators were developed from research in healthcare, and their implementation in local planning is necessary for the process to be evidence-based. This finding may indicate that the space planners were not familiar with the indicators when the study was performed and therefore did not consider the indicators relevant. The issue of indicators arose during the subsequent focus group interview, and the participants changed their position slightly upon understanding the reasoning behind the indicators. The group also discussed the difficulties of using a single audit tool for different stages of the planning process. Various real estate organizations design the planning and design process differently in terms of who writes the brief (a member of the healthcare organization or a planner employed by the real estate organization), which people are involved in the process, how the process is designed, the decision level addressed by the brief and the recipients of the brief. Although the group’s concerns are relevant, the CQB-I is generic and indicates important areas to be included in the brief for patient healthcare spaces. The participants also remarked that there was no defined name for a brief produced in the early stages of the planning process. These documents are variously called “business descriptions for space designs,” “requirements analyses,” “feasibility studies,” and “functional briefs.” Thus, the names and contents of the briefs can vary. However, several studies have demonstrated that although briefs in the early stages have different names, their purposes and contents are relatively similar (Ryd, 2001). Briefing in the early stages is a complex process, and one must always ask whether it is being performed efficiently. Research has shown that ensuring the quality of the early stages of the planning process is cost effective (Ulrich, Berry, Quan, & Parish, 2010). In addition to costs and benefits, society is increasingly focused on the importance of the implementation of evidence-based knowledge in healthcare, with a particular focus on factors that support patient-centered care. An instrument such as the CQB-I can be used to evaluate and develop briefs, thus making it more effective. The CQB-I may contribute to the continuous improvement process of briefs to identify areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the process more visible. Briefs can be compared over time and between places and situations. The instrument clarifies the principles of quality management and an evidence-based approach to planning by enabling an assessment of the documents that result directly and indirectly from the process.
Conclusion In summary, this study showed that the CQB-I satisfied its purpose and was easy to use. It is well known that it is difficult to transfer research results into healthcare services and into briefings. The CQB-I is an instrument that can help to bring research results into the planning process and contribute to discussions and collaboration among the stakeholders during the early stages. This instrument includes a user-oriented approach, enables clear discussions and documentation of outcomes, and consequently supports quality improvement in the planning process of new healthcare environments. However, this instrument should be developed and tested in larger groups.
78 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
PERCEPTIONS OF ASSESSMENT INSTRUMENT IN THE PRE-DESIGN PHASE
RESEARCH
Implications for Practice • There is a need to assess the content and quality of briefs as part of the continuous quality improvement process. • An instrument such as the CQB-I can be used to evaluate and develop briefs, by identifying areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the planning process more visible. • Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the planning process by orienting to the user needs, facilitating discussion and articulation of clear performance and related quality improvement indicators.
References Baird, G. (2001). Post-occupancy evaluation and probe: A New Zealand perspective. Building Research & Information, 29(6), 469–472. Barrett, P., & Baldery, D. (2003). Facilities management: Towards best practice. Oxford, England: Blackwell Science. Barrett, P., & Stanley, C. (1999). Better construction briefing. Oxford, England: Blackwell Science. Barrett, P. S., Hudson, J., & Stanley, C. (1999). Good practice in briefing: The limits of rationality. Automation in Construction, 8(6), 633–642. Becker, F., & Parson, S. K. (2007). Hospital facilities and the role of evidence-based design. Journal of Facility Management, 5(4), 263–274. Blyth, A., & Worthington, J. (2001). Managing the brief for better design. London, England: Spoon Press. Creswell, J. W., & Tashakkori, A. (2007). Developing publishable mixed methods manuscripts. Journal of Mixed Methods, 1, 107–111. Curtis, S., Gesler, W., Priebe, S., & Francis, S. (2009). New spaces of inpatient care for people with mental illness: A complex ‘rebirth’ of the clinic? Health & Place, 15(1), 340–348. Elf, M., & Malmqvist, I. (2009). An audit of content and quality in briefs in Sweden. Journal of Facility Management, 7(3), 198–211. Elf, M., Svedbo Engström, M., & Wijk, H. (2011). Development of the content and quality in briefs instrument (CQB-I). Health Environments Research & Design Journal, 5(3), 74–88. Elf, M., Svedbo Engström, M., & Wijk, H. (2012). An assessment of briefs used for designing healthcare environments: A survey in Sweden. Construction Management and Economics, 30(10), 835–844. Gesler, W., Bell, M., Curtis, S., Hubbard, P., & Francis, S. (2004). Therapy by design: Evaluating the UK hospital building program. Health & Place, 10(2), 117–128.
© 2014 VENDOME GROUP LLC
HEALTH ENVIRONMENTS RESEARCH & DESIGN JOURNAL 79
RESEARCH
FALL 2014 • VOL. 8 NO. 1, pp. 67–80
Graneheim, U., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. Henriksen, K., Isaacson, S., Sadler, B. L., & Zimring, C. M. (2007). The role of the physical environment in crossing the quality chasm. The Joint Commission Journal on Quality and Patient Safety, 33(11 Suppl), 68–80. Huisman, E. R. C. M., Morales, E., van Hoof, J., & Kort, H. S. M. (2012, December). Healing environment: A review of the impact of physical environmental factors on users. Building and Environment, 58, 70–80. Kelly, J, and Hunter, K. (2005). Briefing from facilities management perspective. Facilities, 23(7/8), 356–367. Morse, J., & Niehaus, L. (2007). Combining qualitative and quantitative methods for mixed method designs. In P. Munhall (Ed.), Nursing research: A qualitative perspective (4th ed.) (pp. 541– 554). Boston, MA: Jones & Bartlett. Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing and Health, 29(5), 489–497. Preiser, W. (1995). Post-occupancy evaluation: How to make buildings work better. Facilities, 13(11), 19–28. Preiser, W., & Vischer, J. (2005). Assessing building performance. London, England: Elsevier Butterworth-Heinemann. Rybkowski, Z. K. (2009). The application of root cause analysis and target value design to evidence-based design in the capital planning of healthcare facilities. (Dissertation). University of California, Berkeley, Berkeley, CA. ProQuest, UMI Dissertation Publishing. Ryd, N. (2001). Byggnadsbrief som informationsbärare i byggprocessen. En studie av byggnadsbriefmering. [The architectural brief as carrier of client-information during the building process.] Göteborg, Sweden: Chalmers University of Technology. Ryd, N., & Fristedt, S. (2007). Transforming strategic briefing into project briefs. A case study about client and contractor collaboration. Facilities, 25(5/6), 185–202. Steinke, C., Webster, L., & Fontaine, M. (2010). Evaluating building performance in healthcare facilities: An organizational perspective. Health Environments Research & Design Journal, 3(2), 63–83. Szigeti, F., & Davis, G. (2002). Forum: The turning point for linking briefing and POE? Building Research & Information, 30(1), 47–53. Ulrich, R. S., Berry, L. L., Quan, X., & Parish, J. T. (2010). A conceptual framework for the domain of evidence-based design. Health Environments Research & Design Journal, 4(1), 95–114. Ulrich, R. S., Zimring, C., Zhu, X., DuBose, J., Seo, H.-B., Choi, Y.-S., … Joseph, A. (2008). A review of the research literature on evidence-based healthcare design. Health Environments Research & Design Journal, 1(3), 61–125. Vischer, J. (2009). Applying knowledge on building performance: From evidence to intelligence. Intelligent Building International, 1(4), 239–248. Vischer, J., & Zeisel, J. (2008, July). Process management: Bridging the gap between research and design. Design & Health Scientific Review, 57–61. Yu, T. W. A., Shen, Q., Kelly, J., & Hunter, K. (2007). An empirical study of the variables affecting construction project briefing/architectural briefing. International Journal of Project Management, 25(2), 198–212.
80 WWW.HERDJOURNAL.COM
© 2014 VENDOME GROUP LLC
