Nurse Researcher
Development of a factorial survey to explore restricting a child’s movement for a clinical procedure Cite this article as: Brenner M (2013) Development of a factorial survey to explore restricting a child’s movement for a clinical procedure. Nurse Researcher. 21, 2, 40-48. Date of submission: August 1 2012. Date of acceptance: February 19 2013. Correspondence to Maria Brenner [email protected] Maria Brenner PhD, MSc, BSc, RCN, RGN, RNT is a lecturer at the School of Nursing, Midwifery and Health Systems, University College Dublin, Ireland Peer review This article has been subject to double-blind review and checked using antiplagiarism software Author guidelines http://nr.rcnpublishing.com
Abstract Aim To report on the development of a factorial survey to explore nurses’ participation in restricting children’s movement for clinical procedures in hospital. Background Exploration of implicit practices, such as restriction, is essential in ensuring that the care delivered to children addresses their developmental needs. Data sources A questionnaire was developed that consisted of two sections: vignettes and the professional and personal characteristics of the nurse. It was sent to 166 nurses and 105 questionnaires were returned, resulting in a response rate of 63.3 per cent. Review methods The development of the factorial survey included identifying and determining the levels of the independent variables, identifying the dependent variable, writing the vignette frame and associated questions, and randomly generating vignettes. Discussion Reliability of the tool was established
Introduction Restricting a child’s movement for a clinical procedure is a frequent and often implicit aspect of practice in children’s nursing. According to emerging knowledge on the emotional and physiological development of children, restriction has the potential to cause long-term effects on the physical and emotional wellbeing of the child (Valiente et al 2003, Herba and Phillips 2004). Exploration of practices such as restriction is essential in ensuring that the best care is provided to children in hospital. Despite emerging 40 November 2013 | Volume 21 | Number 2
as a significant correlation was found for responses with Pearson’s r=0.80. There was a small correlation between five of the 14 variables and the dependent variable ‘likelihood of restricting a child for a clinical procedure’. Conclusion A factorial survey was found to be a robust tool in exploring a sensitive issue, allowing for the inclusion of multiple variables for consideration in the analysis. Implications for research/practice The factorial survey enables rigorous exploration of the influences of personal and professional characteristics of a profession across a broad spectrum of clinical scenarios, and offers in-depth insight into the effect of differentiations in these characteristics on a variety of care delivery situations and how they influence behavioural intentions. Keywords Children’s nursing, factorial design, factorial survey, orthogonality, vignettes knowledge on the effects of restriction and on the developmental needs of children, there is a lack of research exploring restriction. This paper reports on the development of a factorial survey to explore nurses’ participation in restricting children’s movement for clinical procedures in hospital.
Background A factorial survey was identified as the most appropriate way to explore the effects of the potential © RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 40
24/10/2013 14:26
Questionnaire factors that may influence the restriction of a child’s movements for a clinical procedure. It is a hybrid technique that incorporates the use of vignettes with multiple factors (Rossi and Nock 1982). This afforded the researcher the opportunity to explore the interaction of various factors on the dependent variable (DV) – the likelihood of a nurse restricting a child for a clinical procedure. It could also produce effects that could not be predicted from individually exploring the relationship between the DV and each factor (Moore 2010). This decision acknowledges that multiple factors may be responsible for influencing any action taken, such as restricting a child for a clinical procedure (Rossi and Nock 1982). The factorial survey was initially trialled in the 1950s, then further developed into a more applicable tool for social research in the late 1970s and 1980s (Rossi and Nock 1982). The rationale for the inception of this design was acknowledgement that ‘human evaluations are in part socially determined (that is, shared with others) and in part governed by individuality, the mix varying from topic to topic’ (Rossi and Anderson 1982). A factorial survey is based on a factorial design and collects data in two parts: vignettes and characteristics of the respondent (Ludwick et al 2004). The vignettes may be developed from practice, qualitative research, literature review or a combination of these sources. Based on conventional practice and following an extensive review of the literature, it was identified that factors that influence restriction are inextricably linked with each other (Selekman and Snyder 1995, Robinson and Collier 1997, Brenner et al 2007). The interaction effects of these factors could be explored in a factorial survey, because it uses the concept of orthogonality (no association of factors) from the experimental tradition (Shadish et al 2002). This means that it is possible to examine the influence of each factor on nurses’ likelihood to restrict a child’s movement for a procedure, while controlling for other, identified factors. A factorial design would typically be referred to using the number of categories for each factor: for example, a ‘2x3’ design refers to two factors with two levels in one category and three levels in the other (Ludwick et al 2004). However, a factorial survey has the capacity for a larger number of factors and levels, increasing the proportion of surveys and, consequently, the proportion of observations for analysis. In clinical settings, the use of a factorial design would present ethical problems because of the difficulty of manipulating care delivery. However, a factorial survey that uses vignettes is a suitable © RCN PUBLISHING / NURSE RESEARCHER
alternative (Richman and Mercer 2002). The complexity of measuring the factors associated with the restriction of a child for a clinical procedure was therefore a major influence in the decision to use a factorial survey, because a factorial design is useful for explaining an intricate set of relationships (Shadish et al 2002, Moore 2010). In a factorial survey, vignettes are used to present the factors to be explored by the participants (Richman and Mercer 2002, Taylor 2006, Taylor et al 2009). Vignettes have been used predominantly in politics and marketing, but they have recently been used for a variety of reasons in social and healthcare research into the care of terminally ill infants (Kodadek and Feeg 2002), judgements of self-neglect (Lauder 2002), recognition of confusion and the need to restrain the elderly (Ludwick and Zeller 2001), pre-operative education of healthcare professionals (Spalding and Phillips 2007), clinical risk (Brown and Pritchard 2008), patient preferences in shared decision making (Müller-Engelmann et al 2008), the risk of cardiovascular disease (Brauer et al 2009), and practitioner assessments of parenting (Taylor et al 2009). Most researchers who used vignettes did not describe in detail how they developed them, although most vignettes were derived from practice situations (Ludwick and Zeller 2001, Lauder 2002, Brown and Pritchard 2008). In this study, it was anticipated that the use of hypothetical situations would make questions about restricting a child’s movement less personally threatening, potentially elucidating more truthful answers, than asking nurses to comment on their personal practices (Taylor et al 2009).
Research design Item development The measurement instrument was a questionnaire that consisted of two sections: ■■ The vignettes, ■■ The professional and personal characteristics of the nurse. Identifying independent variables It is necessary to ensure that all possible associated factors are included when writing vignettes (Brauer et al 2009). The independent variables (IVs) for inclusion in the vignettes were initially identified from the literature review, and by conducting focus groups with nurses and interviews with parents. Purposive sampling was used to identify parents who had experience of their children being restricted for clinical procedures and nurses caring for children in a children’s hospital who had more than three years’ experience. November 2013 | Volume 21 | Number 2 41
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 41
24/10/2013 14:26
Nurse Researcher Figure 1 Conceptual framework ■■ Nursing qualification. ■■ Nurse’s level of education. ■■ Nurse’s age. ■■ Effect of nurse’s education on emotional development. ■■ Nurse’s training on safe holding techniques. ■■ Nurse’s length of experience in children’s nursing.
Professional and personal attributes of the nurse
Characteristics of the child
■■ Age of child. ■■ Cognitive ability of child.
Restricting a child for a clinical procedure Dynamics of family-centred care
■■ Parental presence. ■■ Child’s refusal or request not to be restricted. ■■ Environment of care (including availability of play therapist, reduced staffing, order from consultant, time pressure, order from senior nurse).
Characteristics of the clinical procedure
■■ Type of procedure (including nasogastric tube insertion, cannulation, surgical wound dressing, removal of sutures, administration of an enema, female catheterisation, administration of oral medication). ■■ Frequency of procedure.
42 November 2013 | Volume 21 | Number 2
Ethical approval was obtained from the research ethics committee in the tertiary care centre. Participants were informed that it was a voluntary study and were asked to sign a consent form before being interviewed. The researcher recognised that exploring the restriction of a child’s movement required great sensitivity and that parents or nurses might get distressed, so they arranged for further support to be available for any participant if required. None of the participants sought this support. Both focus groups and parent interviews were analysed using a thematic network analysis framework (Attride-Stirling 2001, Brenner 2013, Brenner et al 2013). The potential influencing factors that collectively emerged from the data are presented in four categories in the conceptual framework (Figure 1). The recommended number of IVs in vignettes is between five and ten (Ludwick et al 2004, Taylor 2006). In this study, 13 main factors arose from the conceptual framework. The six factors from the category, ‘professional and personal attributes of the nurse’, were explored at the end of the survey, leaving seven main factors for inclusion in the vignettes. These were ■■ Age of the child. ■■ Cognitive ability of the child (characteristics of the child). ■■ Parental presence. ■■ Child’s refusal or request to stop the procedure. ■■ Environment of care (dynamics of family-centred care). ■■ Type of procedure. ■■ Number of attempts at procedure (characteristics of the procedure). Determining the levels of the independent variables The second step was to determine the potential levels of the IVs in each factor for inclusion in the vignettes. For example, with the IV ‘age’, the associated levels reflected different age groups. The rationale for inclusion of levels was that the factorial survey afforded the researcher the opportunity to examine the interaction of all levels in each factor. The factors and associated levels are presented in Table 1. Identifying the dependent variable The dependent variable refers to the subject of the associated question (Taylor 2006). The subject of the associated question in this study was the likelihood that the nurse would restrict a child’s movement for a clinical procedure. © RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 42
24/10/2013 14:26
Questionnaire Writing the vignette framework A vignette typically consists of a fixed series of skeletal sentences designed to explore a combination of IVs that may affect the DV (Richman and Mercer 2002, Ludwick et al 2004, Taylor 2006). In most published factorial surveys, the preference has been to avoid overburdening the respondents, by using short, three- to four-sentence descriptions of clinical scenarios (Finch 1987, Ludwick et al 1999, Ludwick et al 2004, Taylor et al 2009). In this study, a framework vignette was written in which the level of each factor could be randomly assigned to produce various vignettes. It comprised three sentences, written with guidance from two experienced children’s nurses to ensure that the scenario would contain sufficient clinically relevant information about the setting, participants, the problem and the interacting dimensions for respondents to understand the situation (Spalding and Phillips 2007). The skeletal vignette was: ‘At the start of your shift, a ten-year-old girl requires reinsertion of a nasogastric tube. This girl has a cognitive impairment and this is the second attempt to carry out this procedure. She is refusing to co-operate. Her parents are present and there is a staff shortage.’ The structure of this vignette allowed interchangeable levels of the independent variables.
Writing the associated question In this study, the participants were asked after each vignette one question about their likelihood to restrict a child’s movement for a clinical procedure. The response format for the question was an anchored ten-point analogue scale (1-10), with higher scores representing greater likelihood of the nurse restricting a child’s movement for a clinical procedure. Randomly generating vignettes The individual vignette is the unit of analysis in a factorial survey. Therefore, random assignment of levels of factors is a crucial feature in the development of vignettes (Tabachnik and Fidell 2007, Field 2009). The challenge of randomly generating vignettes was to create scenarios that would give each IV and all its associated levels an equal chance of being included. This allowed for the assessment of the independent contribution of each factor affecting whether a nurse would restrict a child’s movement for a clinical procedure. Because a simple 2x2 factorial design would yield four cells, the design in this study – a 8x2x2x2x5x7x4 design – would yield 8,960 cells. A unique vignette could be produced for each cell, once variables and levels of independent variables were identified. The vignettes were produced using Excel and SPSS 15, guided by Winchell (2003). Excel’s RAND function was used to create an m x n matrix of random numbers, where m is the number of
Table 1 Factors and levels of factors for inclusion in vignettes Independent factors
Levels
Value of independent factors
Age of the child
8
2, 4, 6, 8, 10, 12, 14, 16
Cognitive ability of the child
2
Cognitive impairment, no cognitive impairment
Parental presence during the procedure
2
Yes, No
Child’s refusal or request to stop
2
Yes, No
Environment of care
5
Reduced staff numbers, time pressure, order from more senior nurse, order from consultant, play therapist available
Type of clinical procedure
7
Nasogastric tube insertion, cannulation, surgical wound dressing, removal of sutures, administration of an enema, female catheterisation, administration of oral medication
Frequency of procedure
4
1, 2, 3, 4
© RCN PUBLISHING / NURSE RESEARCHER
November 2013 | Volume 21 | Number 2 43
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 43
24/10/2013 14:26
Nurse Researcher independent variables, and n the number of cases to be generated. Using SPSS’s ‘visual binning’ option, category values were created for each of the levels of the IVs, to generate a random table of numbers that represented each level of the IVs. Following this, an SPSS program was created to substitute textual versions of IVs for numeric values. It is acknowledged that it is difficult to decide on the number of vignettes for each participant in a factorial survey (Lauder 2002, Ludwick et al 2004). There are no definitive guidelines on how many vignettes should be given to each participant, and previous studies have presented between three and 40 vignettes. In this study, ten vignettes were given to each participant. This decision was based on a number of suggestions: ■■ That the number of vignettes per person should exceed the total number of factors in the study (Müller-Engelmann et al 2008). ■■ That the survey burden may be greater for participants in this survey as it explores a sensitive issue (Ludwick et al 2004). ■■ That the survey may take longer to complete because the staff were more likely to be unfamiliar with scoring vignettes, as a factorial survey has not been previously used to explore care delivery in children’s nursing. The final part of the survey focused on the professional and personal attributes of the nurse, and included questions on: ■■ Qualifications. ■■ Level of education. ■■ Age. ■■ Effect of level of education on emotional development. ■■ Training on safe holding and length of experience in children’s nursing. These questions were placed at the end of the survey, as respondents are more likely to follow through and answer questions that can be perceived as sensitive (Dillman et al 2009).
administrator to print out the exact number of names required for the survey. One advantage of using a factorial survey is that a small sample size can produce a large number of vignettes for analysis (Ludwick et al 2004), although inadequate sample size can threaten the external validity of the study. One approach to determining sample size is to use the ratio of the number of independent variables to the total number of participants (Tabacknik and Fidell 2007). However, the inclusion of too many variables for each participant has been criticised as unnecessary when using multiple regression, as is the case in this study (Munro 2005). Furthermore, it is unnecessary to include every possible combination of every factor (Munro 2005, Taylor 2006). Therefore, to ensure an adequate number of nurses in this study, the estimated sample size for multiple regression studies was determined using power analysis. To estimate the sample size, the researcher clarified the levels of risk of a type I (the possibility of observing a difference when there is none) or type II (failing to observe a difference when one exists) error. The risk of a type I error was set at α =0.05 and the risk of a type II error was set at b =0.10, which are conventional error standards for healthcare research (Polit and Tatano Beck 2008). Sample size was also influenced by the effect size, which refers to the magnitude of the relationship between the research variables (Pedhazer and Pedhazur Schmelkin 1991). The effect size (r 2) for this study was set at ‘moderate’, r 2=0.13, where r is Pearson’s correlation, which is the most commonly used setting in nursing research (Polit and Tatano Beck 2008). On the basis of the parameters described above, the sample size for this multiple regression study was determined using a power analysis formula suggested by Polit and Tatano Beck (2008), as follows:
Sample
where N is the estimated number of subjects required, y is the estimated effect size and k is the number of predictors. The value for L was derived from a table that presents values for studies with a varying number of predictors and power (1-b) (Polit and Tatano Beck 2008). In this study, there were seven predictors that a child would be restricted for a clinical procedure presented in each vignette. Using this number of predictors and with 1–b=0.90, gave L a value of 18.28. y was arrived at by dividing r 2 by 1–r 2. In this case, the valyue of y was 0.149, so the estimated
The target population was registered nurses caring for children in a large children’s hospital. The sample frame consisted of 450 nurses working in the hospital. Participants were selected using simple random sampling, which involved arranging a random selection of the appropriate number of nurses from the sample frame of all registered nurses. This process was accomplished using SPSS, which had the capacity to randomly assign numbers to all the names. The list was then randomly rearranged from the lowest to the highest numbers, and the researcher instructed the 44 November 2013 | Volume 21 | Number 2
L N= y + k + 1
© RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 44
24/10/2013 14:26
Questionnaire sample size needed was 138 nurses. However, because this was a postal survey, De Vaus (2002) suggests increasing the initial sample by 20 per cent to address the possibility of non-response. The final sample size was, consequently, 166, so 1,660 vignettes were required. However, Winchell (2003) and Taylor (2006) advocate generating more vignettes than required, to allow for the removal of unrealistic scenarios that become apparent when the vignettes are generated. As a result, 10 per cent more vignettes were generated, taking the total number of randomly generated vignettes to 1,826.
Reliability and validity In this study, a measure of stability, ‘test-retest’, was used to determine reliability. Stability refers to the extent to which similar results are obtained over time; test-retest involves administering vignettes to a sample of the population on two separate occasions and comparing the scores attained (Bennett et al 2002). Pearson’s r can range from –1.00 to 1.00, but the recommended range to confirm stability in test-retest is between 0.7 and 1.00 (Tabachnik and Fidell 2007). Although test-retest has been used in previous studies to measure the stability of vignettes, both tests were administered at the same time in most of these studies. This has the potential to give false high correlations and subsequently overinflate reliability, and has been advised against, with a time frame of one to two weeks preferred (Pedhazur and Pedhazur-Schmelkin 1991). In this study, ten sample vignettes were initially administered to 10 per cent of the total population (n =16), two weeks apart, and r computed for responses given to a combination of all ten vignettes. Significant correlation was found for responses, with a difference of r =0.80 between the groups when the results from the first and second occasion were compared. Although ‘face validity’ does not provide strong evidence of validity, it is a helpful way to encourage participation in the study (Moore 2010). Face validity was established by consulting other researchers and staff nurses in children’s nursing to determine professional appearance and layout. To address content validity and ensure that the vignettes adequately represented issues related to restricting a child’s movement, the vignettes were sent to two experts who had experience in developing vignettes.
Ethical considerations Following ethical approval from the participating hospital, potential participants were provided with © RCN PUBLISHING / NURSE RESEARCHER
detailed written information about the study and an invitation to contact the researcher if they had any questions that were not addressed by the information provided. They were informed in a cover letter that it was a voluntary study and that they could withdraw at any time. Return of the factorial survey to the researcher was deemed evidence of consent to participate. Anonymity and confidentiality was assured for each participant.
Results Surveys were sent to 166 nurses and 105 returned, resulting in a response rate of 63.3 per cent. The initial response rate was 52 per cent (n =87). However, this was increased by 11 per cent (n =18) after further responses to follow-up reminders. Outliers and residuals Data were screened for outliers and residuals to ensure extreme cases did not affect the prediction of influencing factors. No outliers were detected from the scatter plot, as no cases had a standardised residual of more than 3.3 or less than -3.3 (Field 2009). Furthermore, inspection of Mahalanobis distances, which provide relative measures of data points’ distances from a common point, indicated that there were no influential outliers. The highest value in the study was 15.07, which is less than the maximum suggested value of 22.46 (Tabachnik and Fidell 2007). Finally, Cook’s distance, which estimates the influence of a data point, was measured at 0.01, suggesting no negative effect of any single independent factor. Correlations between variables The relationships between each variable were investigated using Pearson’s r (Table 2, page 46) to assess for multicollinearity. Many of the results were identified as statistically significant, with p values ranging from 0.05 to 0.001. However the strength of association between variables was generally small, with only a few reaching a moderate correlation. There was a small correlation between five of the 14 variables and the DV ‘likelihood to restrict a child for a clinical procedure’. Two of these had a small negative relationship, indicating that younger children were more likely to be restricted (r = –0.10, p
Development of a factorial survey to explore restricting a child’s movement for a clinical procedure Cite this article as: Brenner M (2013) Development of a factorial survey to explore restricting a child’s movement for a clinical procedure. Nurse Researcher. 21, 2, 40-48. Date of submission: August 1 2012. Date of acceptance: February 19 2013. Correspondence to Maria Brenner [email protected] Maria Brenner PhD, MSc, BSc, RCN, RGN, RNT is a lecturer at the School of Nursing, Midwifery and Health Systems, University College Dublin, Ireland Peer review This article has been subject to double-blind review and checked using antiplagiarism software Author guidelines http://nr.rcnpublishing.com
Abstract Aim To report on the development of a factorial survey to explore nurses’ participation in restricting children’s movement for clinical procedures in hospital. Background Exploration of implicit practices, such as restriction, is essential in ensuring that the care delivered to children addresses their developmental needs. Data sources A questionnaire was developed that consisted of two sections: vignettes and the professional and personal characteristics of the nurse. It was sent to 166 nurses and 105 questionnaires were returned, resulting in a response rate of 63.3 per cent. Review methods The development of the factorial survey included identifying and determining the levels of the independent variables, identifying the dependent variable, writing the vignette frame and associated questions, and randomly generating vignettes. Discussion Reliability of the tool was established
Introduction Restricting a child’s movement for a clinical procedure is a frequent and often implicit aspect of practice in children’s nursing. According to emerging knowledge on the emotional and physiological development of children, restriction has the potential to cause long-term effects on the physical and emotional wellbeing of the child (Valiente et al 2003, Herba and Phillips 2004). Exploration of practices such as restriction is essential in ensuring that the best care is provided to children in hospital. Despite emerging 40 November 2013 | Volume 21 | Number 2
as a significant correlation was found for responses with Pearson’s r=0.80. There was a small correlation between five of the 14 variables and the dependent variable ‘likelihood of restricting a child for a clinical procedure’. Conclusion A factorial survey was found to be a robust tool in exploring a sensitive issue, allowing for the inclusion of multiple variables for consideration in the analysis. Implications for research/practice The factorial survey enables rigorous exploration of the influences of personal and professional characteristics of a profession across a broad spectrum of clinical scenarios, and offers in-depth insight into the effect of differentiations in these characteristics on a variety of care delivery situations and how they influence behavioural intentions. Keywords Children’s nursing, factorial design, factorial survey, orthogonality, vignettes knowledge on the effects of restriction and on the developmental needs of children, there is a lack of research exploring restriction. This paper reports on the development of a factorial survey to explore nurses’ participation in restricting children’s movement for clinical procedures in hospital.
Background A factorial survey was identified as the most appropriate way to explore the effects of the potential © RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 40
24/10/2013 14:26
Questionnaire factors that may influence the restriction of a child’s movements for a clinical procedure. It is a hybrid technique that incorporates the use of vignettes with multiple factors (Rossi and Nock 1982). This afforded the researcher the opportunity to explore the interaction of various factors on the dependent variable (DV) – the likelihood of a nurse restricting a child for a clinical procedure. It could also produce effects that could not be predicted from individually exploring the relationship between the DV and each factor (Moore 2010). This decision acknowledges that multiple factors may be responsible for influencing any action taken, such as restricting a child for a clinical procedure (Rossi and Nock 1982). The factorial survey was initially trialled in the 1950s, then further developed into a more applicable tool for social research in the late 1970s and 1980s (Rossi and Nock 1982). The rationale for the inception of this design was acknowledgement that ‘human evaluations are in part socially determined (that is, shared with others) and in part governed by individuality, the mix varying from topic to topic’ (Rossi and Anderson 1982). A factorial survey is based on a factorial design and collects data in two parts: vignettes and characteristics of the respondent (Ludwick et al 2004). The vignettes may be developed from practice, qualitative research, literature review or a combination of these sources. Based on conventional practice and following an extensive review of the literature, it was identified that factors that influence restriction are inextricably linked with each other (Selekman and Snyder 1995, Robinson and Collier 1997, Brenner et al 2007). The interaction effects of these factors could be explored in a factorial survey, because it uses the concept of orthogonality (no association of factors) from the experimental tradition (Shadish et al 2002). This means that it is possible to examine the influence of each factor on nurses’ likelihood to restrict a child’s movement for a procedure, while controlling for other, identified factors. A factorial design would typically be referred to using the number of categories for each factor: for example, a ‘2x3’ design refers to two factors with two levels in one category and three levels in the other (Ludwick et al 2004). However, a factorial survey has the capacity for a larger number of factors and levels, increasing the proportion of surveys and, consequently, the proportion of observations for analysis. In clinical settings, the use of a factorial design would present ethical problems because of the difficulty of manipulating care delivery. However, a factorial survey that uses vignettes is a suitable © RCN PUBLISHING / NURSE RESEARCHER
alternative (Richman and Mercer 2002). The complexity of measuring the factors associated with the restriction of a child for a clinical procedure was therefore a major influence in the decision to use a factorial survey, because a factorial design is useful for explaining an intricate set of relationships (Shadish et al 2002, Moore 2010). In a factorial survey, vignettes are used to present the factors to be explored by the participants (Richman and Mercer 2002, Taylor 2006, Taylor et al 2009). Vignettes have been used predominantly in politics and marketing, but they have recently been used for a variety of reasons in social and healthcare research into the care of terminally ill infants (Kodadek and Feeg 2002), judgements of self-neglect (Lauder 2002), recognition of confusion and the need to restrain the elderly (Ludwick and Zeller 2001), pre-operative education of healthcare professionals (Spalding and Phillips 2007), clinical risk (Brown and Pritchard 2008), patient preferences in shared decision making (Müller-Engelmann et al 2008), the risk of cardiovascular disease (Brauer et al 2009), and practitioner assessments of parenting (Taylor et al 2009). Most researchers who used vignettes did not describe in detail how they developed them, although most vignettes were derived from practice situations (Ludwick and Zeller 2001, Lauder 2002, Brown and Pritchard 2008). In this study, it was anticipated that the use of hypothetical situations would make questions about restricting a child’s movement less personally threatening, potentially elucidating more truthful answers, than asking nurses to comment on their personal practices (Taylor et al 2009).
Research design Item development The measurement instrument was a questionnaire that consisted of two sections: ■■ The vignettes, ■■ The professional and personal characteristics of the nurse. Identifying independent variables It is necessary to ensure that all possible associated factors are included when writing vignettes (Brauer et al 2009). The independent variables (IVs) for inclusion in the vignettes were initially identified from the literature review, and by conducting focus groups with nurses and interviews with parents. Purposive sampling was used to identify parents who had experience of their children being restricted for clinical procedures and nurses caring for children in a children’s hospital who had more than three years’ experience. November 2013 | Volume 21 | Number 2 41
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 41
24/10/2013 14:26
Nurse Researcher Figure 1 Conceptual framework ■■ Nursing qualification. ■■ Nurse’s level of education. ■■ Nurse’s age. ■■ Effect of nurse’s education on emotional development. ■■ Nurse’s training on safe holding techniques. ■■ Nurse’s length of experience in children’s nursing.
Professional and personal attributes of the nurse
Characteristics of the child
■■ Age of child. ■■ Cognitive ability of child.
Restricting a child for a clinical procedure Dynamics of family-centred care
■■ Parental presence. ■■ Child’s refusal or request not to be restricted. ■■ Environment of care (including availability of play therapist, reduced staffing, order from consultant, time pressure, order from senior nurse).
Characteristics of the clinical procedure
■■ Type of procedure (including nasogastric tube insertion, cannulation, surgical wound dressing, removal of sutures, administration of an enema, female catheterisation, administration of oral medication). ■■ Frequency of procedure.
42 November 2013 | Volume 21 | Number 2
Ethical approval was obtained from the research ethics committee in the tertiary care centre. Participants were informed that it was a voluntary study and were asked to sign a consent form before being interviewed. The researcher recognised that exploring the restriction of a child’s movement required great sensitivity and that parents or nurses might get distressed, so they arranged for further support to be available for any participant if required. None of the participants sought this support. Both focus groups and parent interviews were analysed using a thematic network analysis framework (Attride-Stirling 2001, Brenner 2013, Brenner et al 2013). The potential influencing factors that collectively emerged from the data are presented in four categories in the conceptual framework (Figure 1). The recommended number of IVs in vignettes is between five and ten (Ludwick et al 2004, Taylor 2006). In this study, 13 main factors arose from the conceptual framework. The six factors from the category, ‘professional and personal attributes of the nurse’, were explored at the end of the survey, leaving seven main factors for inclusion in the vignettes. These were ■■ Age of the child. ■■ Cognitive ability of the child (characteristics of the child). ■■ Parental presence. ■■ Child’s refusal or request to stop the procedure. ■■ Environment of care (dynamics of family-centred care). ■■ Type of procedure. ■■ Number of attempts at procedure (characteristics of the procedure). Determining the levels of the independent variables The second step was to determine the potential levels of the IVs in each factor for inclusion in the vignettes. For example, with the IV ‘age’, the associated levels reflected different age groups. The rationale for inclusion of levels was that the factorial survey afforded the researcher the opportunity to examine the interaction of all levels in each factor. The factors and associated levels are presented in Table 1. Identifying the dependent variable The dependent variable refers to the subject of the associated question (Taylor 2006). The subject of the associated question in this study was the likelihood that the nurse would restrict a child’s movement for a clinical procedure. © RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 42
24/10/2013 14:26
Questionnaire Writing the vignette framework A vignette typically consists of a fixed series of skeletal sentences designed to explore a combination of IVs that may affect the DV (Richman and Mercer 2002, Ludwick et al 2004, Taylor 2006). In most published factorial surveys, the preference has been to avoid overburdening the respondents, by using short, three- to four-sentence descriptions of clinical scenarios (Finch 1987, Ludwick et al 1999, Ludwick et al 2004, Taylor et al 2009). In this study, a framework vignette was written in which the level of each factor could be randomly assigned to produce various vignettes. It comprised three sentences, written with guidance from two experienced children’s nurses to ensure that the scenario would contain sufficient clinically relevant information about the setting, participants, the problem and the interacting dimensions for respondents to understand the situation (Spalding and Phillips 2007). The skeletal vignette was: ‘At the start of your shift, a ten-year-old girl requires reinsertion of a nasogastric tube. This girl has a cognitive impairment and this is the second attempt to carry out this procedure. She is refusing to co-operate. Her parents are present and there is a staff shortage.’ The structure of this vignette allowed interchangeable levels of the independent variables.
Writing the associated question In this study, the participants were asked after each vignette one question about their likelihood to restrict a child’s movement for a clinical procedure. The response format for the question was an anchored ten-point analogue scale (1-10), with higher scores representing greater likelihood of the nurse restricting a child’s movement for a clinical procedure. Randomly generating vignettes The individual vignette is the unit of analysis in a factorial survey. Therefore, random assignment of levels of factors is a crucial feature in the development of vignettes (Tabachnik and Fidell 2007, Field 2009). The challenge of randomly generating vignettes was to create scenarios that would give each IV and all its associated levels an equal chance of being included. This allowed for the assessment of the independent contribution of each factor affecting whether a nurse would restrict a child’s movement for a clinical procedure. Because a simple 2x2 factorial design would yield four cells, the design in this study – a 8x2x2x2x5x7x4 design – would yield 8,960 cells. A unique vignette could be produced for each cell, once variables and levels of independent variables were identified. The vignettes were produced using Excel and SPSS 15, guided by Winchell (2003). Excel’s RAND function was used to create an m x n matrix of random numbers, where m is the number of
Table 1 Factors and levels of factors for inclusion in vignettes Independent factors
Levels
Value of independent factors
Age of the child
8
2, 4, 6, 8, 10, 12, 14, 16
Cognitive ability of the child
2
Cognitive impairment, no cognitive impairment
Parental presence during the procedure
2
Yes, No
Child’s refusal or request to stop
2
Yes, No
Environment of care
5
Reduced staff numbers, time pressure, order from more senior nurse, order from consultant, play therapist available
Type of clinical procedure
7
Nasogastric tube insertion, cannulation, surgical wound dressing, removal of sutures, administration of an enema, female catheterisation, administration of oral medication
Frequency of procedure
4
1, 2, 3, 4
© RCN PUBLISHING / NURSE RESEARCHER
November 2013 | Volume 21 | Number 2 43
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 43
24/10/2013 14:26
Nurse Researcher independent variables, and n the number of cases to be generated. Using SPSS’s ‘visual binning’ option, category values were created for each of the levels of the IVs, to generate a random table of numbers that represented each level of the IVs. Following this, an SPSS program was created to substitute textual versions of IVs for numeric values. It is acknowledged that it is difficult to decide on the number of vignettes for each participant in a factorial survey (Lauder 2002, Ludwick et al 2004). There are no definitive guidelines on how many vignettes should be given to each participant, and previous studies have presented between three and 40 vignettes. In this study, ten vignettes were given to each participant. This decision was based on a number of suggestions: ■■ That the number of vignettes per person should exceed the total number of factors in the study (Müller-Engelmann et al 2008). ■■ That the survey burden may be greater for participants in this survey as it explores a sensitive issue (Ludwick et al 2004). ■■ That the survey may take longer to complete because the staff were more likely to be unfamiliar with scoring vignettes, as a factorial survey has not been previously used to explore care delivery in children’s nursing. The final part of the survey focused on the professional and personal attributes of the nurse, and included questions on: ■■ Qualifications. ■■ Level of education. ■■ Age. ■■ Effect of level of education on emotional development. ■■ Training on safe holding and length of experience in children’s nursing. These questions were placed at the end of the survey, as respondents are more likely to follow through and answer questions that can be perceived as sensitive (Dillman et al 2009).
administrator to print out the exact number of names required for the survey. One advantage of using a factorial survey is that a small sample size can produce a large number of vignettes for analysis (Ludwick et al 2004), although inadequate sample size can threaten the external validity of the study. One approach to determining sample size is to use the ratio of the number of independent variables to the total number of participants (Tabacknik and Fidell 2007). However, the inclusion of too many variables for each participant has been criticised as unnecessary when using multiple regression, as is the case in this study (Munro 2005). Furthermore, it is unnecessary to include every possible combination of every factor (Munro 2005, Taylor 2006). Therefore, to ensure an adequate number of nurses in this study, the estimated sample size for multiple regression studies was determined using power analysis. To estimate the sample size, the researcher clarified the levels of risk of a type I (the possibility of observing a difference when there is none) or type II (failing to observe a difference when one exists) error. The risk of a type I error was set at α =0.05 and the risk of a type II error was set at b =0.10, which are conventional error standards for healthcare research (Polit and Tatano Beck 2008). Sample size was also influenced by the effect size, which refers to the magnitude of the relationship between the research variables (Pedhazer and Pedhazur Schmelkin 1991). The effect size (r 2) for this study was set at ‘moderate’, r 2=0.13, where r is Pearson’s correlation, which is the most commonly used setting in nursing research (Polit and Tatano Beck 2008). On the basis of the parameters described above, the sample size for this multiple regression study was determined using a power analysis formula suggested by Polit and Tatano Beck (2008), as follows:
Sample
where N is the estimated number of subjects required, y is the estimated effect size and k is the number of predictors. The value for L was derived from a table that presents values for studies with a varying number of predictors and power (1-b) (Polit and Tatano Beck 2008). In this study, there were seven predictors that a child would be restricted for a clinical procedure presented in each vignette. Using this number of predictors and with 1–b=0.90, gave L a value of 18.28. y was arrived at by dividing r 2 by 1–r 2. In this case, the valyue of y was 0.149, so the estimated
The target population was registered nurses caring for children in a large children’s hospital. The sample frame consisted of 450 nurses working in the hospital. Participants were selected using simple random sampling, which involved arranging a random selection of the appropriate number of nurses from the sample frame of all registered nurses. This process was accomplished using SPSS, which had the capacity to randomly assign numbers to all the names. The list was then randomly rearranged from the lowest to the highest numbers, and the researcher instructed the 44 November 2013 | Volume 21 | Number 2
L N= y + k + 1
© RCN PUBLISHING / NURSE RESEARCHER
Downloaded from RCNi.com by ${individualUser.displayName} on Aug 25, 2015. For personal use only. No other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
40-48 NR Nov 2013x.indd 44
24/10/2013 14:26
Questionnaire sample size needed was 138 nurses. However, because this was a postal survey, De Vaus (2002) suggests increasing the initial sample by 20 per cent to address the possibility of non-response. The final sample size was, consequently, 166, so 1,660 vignettes were required. However, Winchell (2003) and Taylor (2006) advocate generating more vignettes than required, to allow for the removal of unrealistic scenarios that become apparent when the vignettes are generated. As a result, 10 per cent more vignettes were generated, taking the total number of randomly generated vignettes to 1,826.
Reliability and validity In this study, a measure of stability, ‘test-retest’, was used to determine reliability. Stability refers to the extent to which similar results are obtained over time; test-retest involves administering vignettes to a sample of the population on two separate occasions and comparing the scores attained (Bennett et al 2002). Pearson’s r can range from –1.00 to 1.00, but the recommended range to confirm stability in test-retest is between 0.7 and 1.00 (Tabachnik and Fidell 2007). Although test-retest has been used in previous studies to measure the stability of vignettes, both tests were administered at the same time in most of these studies. This has the potential to give false high correlations and subsequently overinflate reliability, and has been advised against, with a time frame of one to two weeks preferred (Pedhazur and Pedhazur-Schmelkin 1991). In this study, ten sample vignettes were initially administered to 10 per cent of the total population (n =16), two weeks apart, and r computed for responses given to a combination of all ten vignettes. Significant correlation was found for responses, with a difference of r =0.80 between the groups when the results from the first and second occasion were compared. Although ‘face validity’ does not provide strong evidence of validity, it is a helpful way to encourage participation in the study (Moore 2010). Face validity was established by consulting other researchers and staff nurses in children’s nursing to determine professional appearance and layout. To address content validity and ensure that the vignettes adequately represented issues related to restricting a child’s movement, the vignettes were sent to two experts who had experience in developing vignettes.
Ethical considerations Following ethical approval from the participating hospital, potential participants were provided with © RCN PUBLISHING / NURSE RESEARCHER
detailed written information about the study and an invitation to contact the researcher if they had any questions that were not addressed by the information provided. They were informed in a cover letter that it was a voluntary study and that they could withdraw at any time. Return of the factorial survey to the researcher was deemed evidence of consent to participate. Anonymity and confidentiality was assured for each participant.
Results Surveys were sent to 166 nurses and 105 returned, resulting in a response rate of 63.3 per cent. The initial response rate was 52 per cent (n =87). However, this was increased by 11 per cent (n =18) after further responses to follow-up reminders. Outliers and residuals Data were screened for outliers and residuals to ensure extreme cases did not affect the prediction of influencing factors. No outliers were detected from the scatter plot, as no cases had a standardised residual of more than 3.3 or less than -3.3 (Field 2009). Furthermore, inspection of Mahalanobis distances, which provide relative measures of data points’ distances from a common point, indicated that there were no influential outliers. The highest value in the study was 15.07, which is less than the maximum suggested value of 22.46 (Tabachnik and Fidell 2007). Finally, Cook’s distance, which estimates the influence of a data point, was measured at 0.01, suggesting no negative effect of any single independent factor. Correlations between variables The relationships between each variable were investigated using Pearson’s r (Table 2, page 46) to assess for multicollinearity. Many of the results were identified as statistically significant, with p values ranging from 0.05 to 0.001. However the strength of association between variables was generally small, with only a few reaching a moderate correlation. There was a small correlation between five of the 14 variables and the DV ‘likelihood to restrict a child for a clinical procedure’. Two of these had a small negative relationship, indicating that younger children were more likely to be restricted (r = –0.10, p
