Original Article Validity and Reliability of the Critical Care Pain Observation Tool: A Replication Study ---

-

From the Maine Medical Center, Portland, Maine, and Boston College, Chestnut Hill, Massachusetts. Address correspondence to Kathleen M. Keane, PhD (c), MS, BSN, CNL, CCRN, Cardiothoracic Intensive Care Unit, Maine Medical Center, 22 Bramhall St., Portland, ME 04102. E-mail: [email protected] Received November 18, 2010; Revised January 12, 2012; Accepted January 15, 2012. Supported by an American Association of Critical Care Nurses Small Grants Award. 1524-9042/$36.00 Ó 2013 by the American Society for Pain Management Nursing doi:10.1016/j.pmn.2012.01.002

Kathleen Marie Keane, PhD (c), MS, BSN, CNL, CCRN

ABSTRACT:

Critically ill patients are often not able to self-report the presence of pain. Currently there is no generally accepted assessment tool for this population. The Critical-Care Pain Observation Tool (CPOT) was developed for pain assessment of critically ill patients. The purpose of this study was to replicate the findings of the Gelinas et al. (2006) CPOT reference study and examine the interrater reliability (IRR), discriminant validity (DV), and criterion validity (CV) of the CPOT. This quantitative study used a repeated measures design with a convenience sample of 21 postoperative open heart surgery patients cared for in a tertiary-care teaching hospital. Testing for IRR in this sample showed a range of results resulting in fair to almost perfect IRR; the findings of this study suggest that the instrument’s IRR is acceptable but variable. Testing for DV demonstrated a significant difference in mean scores between noxious (painful) and nonnoxious (nonpainful) procedures. Testing for CV showed a weak nonsignificant Spearman correlation of 0.26 (P < .312) between CPOT scores and patient selfreport during repositioning after extubation. This replication study adds to four studies that have examined psychometric attributes of the instrument and contributes to the process of translating the use of this instrument to the clinical setting. Ó 2013 by the American Society for Pain Management Nursing

Critically ill patients can be intubated, sedated, and minimally interactive to noninteractive. For these patients, the ‘‘gold standard’’ of patient self-report is not available to the clinician (Pasero & McCaffery, 2011). Based on the recommendations of clinical guidelines (Herr, Coyne, Key, Manworren, McCaffery, Merkel, & Wild, 2006; Jacobi, Fraser, Coursin, Riker, Fontaine, Wittbrodt, & Lum, 2002), clinicians observe the patient for cues that indicate the patient may be experiencing pain and treat the patient based on those observations. Interdisciplinary clinical guidelines have called for the development of standardized instruments to aid in the assessment of pain in those who cannot self-report (Herr et al., 2006; Jacobi et al., 2002). In response to this need, instruments have been developed to aid the clinician in assessment of pain in critically ill adults (Gelinas, Fillion, Puntillo, Viens, & Fortier, 2006; Mateo & Krenzischek, 1992; Odhner, Wegman, Freeland, Steinmetz, & Ingersoll, 2003; Payen, Bru, Bosson, Lagrasta, Novel, Deschaux, & Jacquot, 2001; Pain Management Nursing, Vol 14, No 4 (December), 2013: pp e216-e225

CPOT Validity and Reliability

Puntillo, Stannard, Miaskowski, Kehrle, & Gleeson, 2002). The charge of research now is to further test and validate the instruments and thereby to recommend an instrument for general clinical use. The purpose of the present study was to reproduce the findings of one study (henceforth referred to as the Gelinas reference study) that developed an instrument to assess pain in critically ill adults, the Critical-Care Pain Observation Tool (CPOT) (Gelinas et al., 2006). The aim was to contribute to research that examines the validity and reliability of the CPOT instrument and to contribute to translation of research findings to the clinical setting.

BACKGROUND There is general consensus that an individual’s experience of pain is subjective and complex and that patients’ themselves can most accurately measure their pain experience; this is referred to as a patient’s selfreport (Herr et al., 2006; Jacobi et al., 2002; Pasero & McCaffery, 2011; Puntillo, Pasero, Li, Mularski, Grap, Erstad, & Sessler, 2009). Critically ill patients may be intubated, sedated, and minimally interactive to noninteractive and assessing pain in these patients is problematic, because the gold standard of patient selfreport is not available to the clinician (Pasero & McCaffery, 2011). Poor pain management directly affects patient care; it is linked to poor patient outcomes, increased cost of care, and decreased quality of life (Dalton, Brown, Carlson, McNutt, & Greer, 2000; Granja, Lopes, Moreira, Dias, Costa-Pereira, & Carneiro, 2005; Mularski, Curtis, Billings, Burt, Byock, Fuhrman, & Levy, 2006). Some consequences of undertreated pain include impaired immune response, respiratory complications such as atelectasis and infection, decreased mobilization and venous thromboembolism, the development of chronic pain syndromes, and psychologic disorders, such as posttraumatic stress disorder (Jones, Griffiths, Humphris, & Skirrow, 2001; Norman, Stein, Dimsdale, & Hoyt, 2008; Page, 2005; Pasero & McCaffery, 2011). Undertreated pain decreases patient and family quality of life and is a cause of moral distress in caregivers (Desbiens & Wu, 2000; Elpern, Covert, & Kleinpell, 2005; Ferrell, 2005). Pain assessment in the critically ill is a concern because multiple studies have recognized that pain is prevalent in critically ill patients (Gelinas, 2007; Puntillo, Arai, Cohen, Gropper, Neuhaus, Paul, & Miaskowski, 2010; Puntillo, Morris, Thompson, Stanik-Hutt, White, & Wild, 2004; Puntillo, White, Morris, Perdue, StanikHutt, Thompson, & Wild, 2001). Acute pain is often underrecognized, underassessed, and undertreated (Arroyo-Novoa, Figueroa-Ramos, Puntillo, Stanik-Hutt,

e217

Thompson, White, & Wild, 2008; Carr, Reines, Schaffer, Polomano, & Lande, 2005; Gelinas, Fortier, Viens, Fillion, & Puntillo, 2004; Marquie, Raufaste, Lauque, Marine, Ecoiffier, & Sorum, 2003; Stanik-Hutt, Soeken, Belcher, Fontaine, & Gift, 2001). This was illustrated by a seminal descriptive study (Thunder II) of procedural pain in adult intensive care unit (ICU) patients (n ¼ 5,957) that was conducted at multiple sites (n ¼ 169) in the United States, Canada, Great Britain, and Australia (Puntillo et al., 2001). Self-report of pain intensity using the numeric rating scale of 0 (no pain) to 10 (highest level of pain) was elicited from adults who underwent one of six common procedures (mean pain intensity is noted in parentheses after each procedure): turning (4.93), wound drain removal (4.67), wound care (4.42), tracheal suctioning (3.94), central venous catheter insertion (2.72), and femoral sheath removal (2.65) (Puntillo et al., 2004). Thunder II made clear the experience of procedural pain in the critically ill; patients’ reported turning to be the most distressing and painful of the procedures studied, in addition, 600 beds) located in the northeastern United States. As in the Gelinas reference study, data were collected on postoperative open heart surgery patients in a cardiothoracic intensive care setting. A convenience sample was used; 23 patients were enrolled. The patient population consisted of adults who were scheduled for open heart surgery procedures. Inclusion criteria were that patients: 1) be >21 years old; 2) be English speaking; and 3) require cardiothoracic surgery. Exclusion criteria were: 1) a left ventricular ejection fraction 2 are indicative of pain; however, scores that are >1 may be indicative of pain in patients before exposure to noxious stimuli (Gelinas, Harel, Fillion, Puntillo, & Johnston, 2009). The Confusion Assessment Method–Intensive Care Unit (CAM-ICU) was used to screen for delirium. The CAM-ICU (Ely, Inouye, Bernard, Gordon, Francis, May, et al., 2001) instrument has been well studied and validated for use in the critically ill population. Participant sedation levels were measured with the use of the Ramsay scale (Ramsay, Savege, Simpson, & Goodwin, 1974). This scale assesses patient responsiveness and agitation; it is scored as follows: 1 (anxious and agitated, restless, or both), 2 (cooperative, oriented, and tranquil), 3 (responds to commands only), 4 (exhibits brisk response to light glabellar tap or loud auditory stimulus), 5 (exhibits a sluggish response to light glabellar tap or loud auditory stimulus), or 6 (exhibits no response) (Ramsay et al., 1974). The PDS is a self-report instrument that patients can use to rate their level of pain. The scale consists of five verbal descriptors of pain (none, mild, moderate, severe, and unbearable), each assigned a numeric value (0-4, respectively). The PDS has been previously studied and has been shown to be a reliable and valid instrument for pain measurement in postoperative patients (Mateo & Krenzischek, 1992). This self-report measure was used to correlate with the CPOT scores when assessing for instrument CV. Study Procedure Each study patient had assessments taken with the CPOT instrument at three different times on the day of their surgery (referred to as postoperative day 0). During the second and third assessment periods, if patients were interactive, they were prompted to self-report their pain scores. The study procedure is depicted in Figure 1. Two nurse observers performed the assessments with the CPOT independently and were blinded to each other’s scores. The nurses who participated in reliability testing were given one educational session on the instrument which consisted of viewing a standardized videotape of patient scenarios that was obtained from the CPOT’s author. Scores were then reviewed with reference scores of the videotaped patient scenarios until there was $90% agreement between scores.

e220

Keane

FIGURE 1.

-

Diagram of study design and data collection.

Assessment Period One The first assessment period took place in the intensive care unit 1 hour after the participant’s arrival from the operating room. Participants were assessed with the CPOT instrument at rest (time [T] 1), with repositioning (T2), and again at rest (T3). Two nurse observers performed the assessments and were blinded to each other’s ratings. A criterion for this time period was that the participant be intubated and unconscious as indicated by a Ramsay score of 5 or 6. Of the 21 participants assessed during this period, 15 participants met the criteria for evaluation. Assessment Period Two The second assessment period took place when the participant was still intubated and had become conscious, as evidenced by a Ramsey Scale score of 2-4. The participant was assessed with the CPOT by two nurse observers at rest (T4), with repositioning (T5), and again at rest (T6). In addition, the participant’s self-report was solicited immediately after repositioning. A simple self-report system was used during this time; patients were asked to indicate if pain were present or absent by nodding their head yes or no. This method of self-report was used because patients were generally too sleepy to use the more complex numeric PDS. Assessment Period Three The final assessment period was performed when participants were extubated and awake. Subjects were assessed at rest (T7), during positioning (T8), and again at rest (T9). They were also asked to indicate if pain was present and to self-report using the PDS. As in the Gelinas reference study, subjects in the study were screened for delirium after extubation; those

that screened positive for delirium were excluded from further study participation. Analytic Methods Deidentified data were collected and statistical calculations conducted in MedCalc for Windows, version 10.02.0 (MedCalc Software, Mariakerke, Belgium), and version 15.0 of SPSS for Windows (SPSS, Chicago, Illinois). The DV of an instrument refers to the ability of an instrument to measure one intended variable and not another, in this case pain versus no pain. The DV of the CPOT instrument was tested by comparing the mean CPOT scores of patients before and immediately after turning in bed (repositioning is considered to be a noxious procedure). Paired Student t tests were performed to compare the mean scores of the testing periods T1 and T2, T4 and T5, and T7 and T8. CV refers to the ability of an instrument to accurately measure the phenomenon of interest, in this case the measurement of pain, and was evaluated by correlating the observed CPOT score to the ‘‘gold’’ standard of pain measurement, the patient self-report (using the PDS). IRR measures the extent to which different users of the instrument can, under similar circumstances, obtain like measurements. IRR of the CPOT was measured by comparing the blinded independent assessments made by two nurse observers at T1-T9. Weighted kappa measurements are able to account for the degree of congruence of measurements that have multiple items in a scale (Landis & Koch, 1977).

RESULTS Sample Characteristics Thirty-three patient charts were screened for potential enrollment in the study based on study inclusion

e221

CPOT Validity and Reliability

and exclusion criteria, and 29 of these patients were invited to participate. Six patients declined to participate in the study. Five of those six patients stated that they were too anxious to consider participation; the sixth patient stated that he was not interested in participating in research. Of the 23 subjects enrolled, one patient withdrew from the study citing anxiety as a factor in his decision; a second patient was dropped from the study owing to hemodynamic instability. Data from 21 subjects were used in this analysis. The ages of patients enrolled in the study ranged from 44 to 85 years, and the mean age was 64 years.

TABLE 2. Discriminant Validity: Differences in CPOT Scores of Participants at Rest (T1, T4, and T7) Compared with Participant CPOT Scores with Repositioning (T2, T5, and T8, respectively) Score at Rest/ No. of Position observations T1/T/2 T4/T5 T7/T8

30 40 32

95% Confidence Interval t

df

Lower

Upper

5.784* 29 3.92543 1.87457 5.785* 39 2.26064 1.08936 7.662 31 2.61148 1.51352

*P < .0001.

Discriminant Validity The mean observed CPOT scores at each assessment of T1-T9 are represented in Figure 2. When comparing mean CPOT scores during nonnocioceptive periods (the rest periods of T1, T4, and T7), and periods of nocioception (the positioning periods of T2, T5, and T8), statistically significant differences were noted. Table 2 provides the details of this analysis.

Criterion Validity The Spearman coefficient was used to test the association between patient self-report (PDS) after extubation and the average CPOT score. A weak nonsignificant Spearman association of 0.26 (P

Validity and reliability of the critical care pain observation tool: a replication study.

Critically ill patients are often not able to self-report the presence of pain. Currently there is no generally accepted assessment tool for this popu...
470KB Sizes 0 Downloads 0 Views