532136
research-article2014
PENXXX10.1177/0148607114532136Journal of Parenteral and Enteral NutritionKoretz
Journal Club
JPEN Journal Club 8. Concealment of Allocation Ronald L. Koretz, MD1
Journal of Parenteral and Enteral Nutrition Volume 38 Number 8 November 2014 1007–1008 © 2014 American Society for Parenteral and Enteral Nutrition DOI: 10.1177/0148607114532136 jpen.sagepub.com hosted at online.sagepub.com
Keywords nutrition support practice; outcomes research/quality; pancreatic disease; research and diseases
The question being addressed in the articles to be discussed1,2 is again fairly straightforward. The investigators saw a number of patients with severe pancreatitis who required surgical interventions and wondered if immediate postoperative enteral nutrition (EN) delivered through a tube placed in the jejunum during the surgery would improve the clinical outcomes. To answer the question, Pupelis et al1,2 stated that they randomized 60 patients into 1 of 2 interventions; as we have noted before, randomization is a preferred way to assign subjects to treatment groups because it should avoid issues created by confounding factors.3 However, to be noted, these investigators did not describe how they developed the randomization sequence or why they decided to stop after they had entered 60 patients; this was in stark contrast to the trial by Sandstrom et al4 that we discussed last time, in which we were told that a computer was employed and a sample size was calculated when the protocol was written. Two papers appeared to result from this single trial. I mention this, and asked you to assume this fact when I made the assignment to read it, because some confusion has occurred in this regard. In a systematic review published in 2006, Steve McClave reported that he had written to Dr Pupelis and asked him how many trials he had performed; McClave reported that Pupelis had told him that there were 2.5 However, a close examination of both studies indicates that the described trials began at the same time (January 1997), were conducted at the same single center in Latvia, had the same protocol, had the earlier report carry the label of “preliminary report,” and even had the same description of the single death in the 1 EN patient who had died. In other words, it seemed very likely that these were both reports of the same trial. I subsequently wrote to Pupelis and asked him whether the 2 reports came from a single trial or 2 different ones, and I also added additional questions such as, if there were 2 different trials, how did the group of investigators decide into which one to enter a particular patient? I never received a response and was not able to confirm McClave’s report. I have thus concluded that both of these papers are from the same trial.6 Since the second paper was accepted over a year after the first one, it is unfortunate that these investigators did not cite that first one in that second paper, since this might have further clarified the matter. The investigators concluded that the EN did improve mortality and reduced the need for reoperation for unresolved
peritonitis. However, before we just take these observations at face value, let us consider the data more closely. I have already noted that the method of generating the randomization schedule was not explained and, in that regard, there was an interesting peculiarity. At the end, there was a 1:1 ratio of patients assigned to each arm, so it is likely that it was the intent of the randomization sequence to create such a distribution. However, for unexplained reasons, at the time of the preliminary report, 11 patients were in the EN arm and 18 were assigned to the control arm. This means that, of the final 31 individuals assigned to the trial, 19 were placed in the EN arm and 12 in the control arm. Assuming that there is a 50:50 chance that any given patient would be assigned to either arm, the chances of having such a distribution occur just by chance (at least 18 of the first 29 randomized to the control arm, but no more than 12 of the last 31 being so assigned) is actually less than 4%. (It is equivalent to having a coin come up either heads or tails at least 18 out of the first 29 times, but then only 12 out of the next 31 times.) However, in this case, the reason for the disparities in entry into the trial in the 2 groups was not due to chance; rather, it was a methodological issue. The description in the Methods section in the second paper informs us that the nasojejunal tube was introduced intraoperatively when early jejunal feeding was feasible; presumably, if the surgeon/investigator did not believe that the patient could tolerate the intrajejunal infusion of nutrient solutions, the patient was then excluded and not counted even though he or she had already been randomized into the EN arm. Additional inclusion criteria for patients assigned to EN were the patient’s consent, presence of a properly positioned and well-functioning nasojejunal tube, and no intraoperarative evidence of ileus. Thus, even a postoperative factor (a nonfunctioning nasojejunal tube) was a reason for From the 1Olive View–UCLA Medical Center, David Geffen–UCLA School of Medicine, Sylmar and Los Angeles, California. Financial disclosure: None declared. Received for publication March 25, 2014; accepted for publication March 27, 2014. Corresponding Author: Ronald L. Koretz, MD, Olive View–UCLA Medical Center, 14445 Olive View Dr, Sylmar, CA 91342, USA. Email: [email protected]
Downloaded from pen.sagepub.com at UNIV OF GEORGIA LIBRARIES on June 24, 2015
1008
Journal of Parenteral and Enteral Nutrition 38(8)
exclusion. In other words, in order for a patient to be entered into the EN arm, additional criteria had to be met that were not required for entry into the control arm; patients could be entered into the control arm even if they would not have been eligible for the EN arm. Since the treated group needed to pass more criteria, patients assigned to that group would be expected to be admitted to the trial more slowly; hence, the 30 slots in the control arm were filled first, explaining the disparity in the numbers assigned at the midpoint in the trial. Given the reasons for excluding patients from the EN arm, it is not surprising that the control group had more residual peritonitis and a higher mortality rate. This confounding bias, rather than a true effect of the EN, is very likely responsible for the differences in the outcomes. The methodologic problem was that the investigators knew, before the patient was formally assigned to the EN group, what the assignment was going to be. In technical terms, allocation of treatment (ie, the assignment to a particular treatment arm) was not concealed. We briefly discussed the issue of concealment of allocation in the last column.7 The bias associated with a failure to employ concealment of allocation has been shown to inflate the estimated treatment effect.8 Finally, I would remind you that the word concealment does not refer to formal blinding of the intervention; blinding means that the involved personnel are not aware of the nature of the intervention during the conduct of the trial and collection of the data. Because of the use of EN, it would have been challenging to blind this study. The loss of concealment of allocation is the principal reason for methodologists to frown on the use of quasi-randomization. Quasi-randomization is the use of a patient record number, birth date or day of the week, or Social Security number to determine group assignment. While such numbers or dates may seem to be random events to the naive reader, there is a problem. While it is true that neither the investigator nor the patient has any control over that number or date, all of those involved individuals will know what that number or date is prior to entry into the trial. Thus, if the subject has a number that would put him or her into the control arm, he or she may decline to enroll for the trial. On the other hand, if the patient was scheduled to be entered into the treatment arm, but that patient appears a little sicker than average, the investigator
may (perhaps even subconsciously) look a little harder for some reason to exclude that potential study participant from the trial (which seems to have been the case in the study we are considering now). While this is not to say that Pupelis et al1,2 used a quasi-randomization scheme, they did employ some method of randomization such that the surgeon/investigator did know, at the time of randomization in the operating room, whether the patient would be assigned to the EN arm. For the next installment, please read the following article: O’Daly BJ, Walsh BC, Quinlan JA, et al. Serum albumin and total lymphocyte count as predictors of outcome in hip fractures. Clin Nutr. 2010;29:89-93.
Acknowledgment The author acknowledges the ongoing support of GIIssues, Inc, a nonprofit organization dedicated to the practice and dissemination of evidence-based medicine. GIIssues, Inc supports educational expenses for the author as well as any expenses related to the creation of papers and other educational products. GIIssues, Inc does not provide any salary support to the author. There are no research materials that are related to this article that can be accessed other than the stated references.
References 1. Pupelis G, Austrums E, Jansone A, Spruce R, Wehbi H. Randomised trial of safety and efficacy of postoperative enteral feeding in patients with severe pancreatitis: preliminary report. Eur J Surg. 2000;166:383-387. 2. Pupelis G, Selga G, Austrums E, Kaminski A. Jejunal feeding, even when instituted late, improves outcomes in patients with severe pancreatitis and peritonitis. Nutrition. 2001;17:91-94. 3. Koretz RL. JPEN Journal Club 3. When and why to randomize. JPEN J Parenter Enteral Nutr. 2014;38:400-401. 4. Sandstrom R, Drott C, Hyltander A, et al. The effect of postoperative intravenous feeding (TPN) on outcome following major surgery evaluated in a randomized study. Ann Surg. 1993;217:185-195. 5. McClave SA, Chang WK, Dhaliwal R, Heyland DK. Nutrition support in acute pancreatitis: a systematic review of the literature. JPEN J Parenter Enteral Nutr. 2006;30:143-156. 6. Koretz RL. Artificial nutrition in critical care: a critical look at the evidence. Br J Intensive Care. 2011;21:13-18. 7. Koretz RL. JPEN Journal Club 7. Subgroup analysis. JPEN J Parenter Enteral Nutr. 2014;38:905-906. 8. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-412.
Downloaded from pen.sagepub.com at UNIV OF GEORGIA LIBRARIES on June 24, 2015
research-article2014
PENXXX10.1177/0148607114532136Journal of Parenteral and Enteral NutritionKoretz
Journal Club
JPEN Journal Club 8. Concealment of Allocation Ronald L. Koretz, MD1
Journal of Parenteral and Enteral Nutrition Volume 38 Number 8 November 2014 1007–1008 © 2014 American Society for Parenteral and Enteral Nutrition DOI: 10.1177/0148607114532136 jpen.sagepub.com hosted at online.sagepub.com
Keywords nutrition support practice; outcomes research/quality; pancreatic disease; research and diseases
The question being addressed in the articles to be discussed1,2 is again fairly straightforward. The investigators saw a number of patients with severe pancreatitis who required surgical interventions and wondered if immediate postoperative enteral nutrition (EN) delivered through a tube placed in the jejunum during the surgery would improve the clinical outcomes. To answer the question, Pupelis et al1,2 stated that they randomized 60 patients into 1 of 2 interventions; as we have noted before, randomization is a preferred way to assign subjects to treatment groups because it should avoid issues created by confounding factors.3 However, to be noted, these investigators did not describe how they developed the randomization sequence or why they decided to stop after they had entered 60 patients; this was in stark contrast to the trial by Sandstrom et al4 that we discussed last time, in which we were told that a computer was employed and a sample size was calculated when the protocol was written. Two papers appeared to result from this single trial. I mention this, and asked you to assume this fact when I made the assignment to read it, because some confusion has occurred in this regard. In a systematic review published in 2006, Steve McClave reported that he had written to Dr Pupelis and asked him how many trials he had performed; McClave reported that Pupelis had told him that there were 2.5 However, a close examination of both studies indicates that the described trials began at the same time (January 1997), were conducted at the same single center in Latvia, had the same protocol, had the earlier report carry the label of “preliminary report,” and even had the same description of the single death in the 1 EN patient who had died. In other words, it seemed very likely that these were both reports of the same trial. I subsequently wrote to Pupelis and asked him whether the 2 reports came from a single trial or 2 different ones, and I also added additional questions such as, if there were 2 different trials, how did the group of investigators decide into which one to enter a particular patient? I never received a response and was not able to confirm McClave’s report. I have thus concluded that both of these papers are from the same trial.6 Since the second paper was accepted over a year after the first one, it is unfortunate that these investigators did not cite that first one in that second paper, since this might have further clarified the matter. The investigators concluded that the EN did improve mortality and reduced the need for reoperation for unresolved
peritonitis. However, before we just take these observations at face value, let us consider the data more closely. I have already noted that the method of generating the randomization schedule was not explained and, in that regard, there was an interesting peculiarity. At the end, there was a 1:1 ratio of patients assigned to each arm, so it is likely that it was the intent of the randomization sequence to create such a distribution. However, for unexplained reasons, at the time of the preliminary report, 11 patients were in the EN arm and 18 were assigned to the control arm. This means that, of the final 31 individuals assigned to the trial, 19 were placed in the EN arm and 12 in the control arm. Assuming that there is a 50:50 chance that any given patient would be assigned to either arm, the chances of having such a distribution occur just by chance (at least 18 of the first 29 randomized to the control arm, but no more than 12 of the last 31 being so assigned) is actually less than 4%. (It is equivalent to having a coin come up either heads or tails at least 18 out of the first 29 times, but then only 12 out of the next 31 times.) However, in this case, the reason for the disparities in entry into the trial in the 2 groups was not due to chance; rather, it was a methodological issue. The description in the Methods section in the second paper informs us that the nasojejunal tube was introduced intraoperatively when early jejunal feeding was feasible; presumably, if the surgeon/investigator did not believe that the patient could tolerate the intrajejunal infusion of nutrient solutions, the patient was then excluded and not counted even though he or she had already been randomized into the EN arm. Additional inclusion criteria for patients assigned to EN were the patient’s consent, presence of a properly positioned and well-functioning nasojejunal tube, and no intraoperarative evidence of ileus. Thus, even a postoperative factor (a nonfunctioning nasojejunal tube) was a reason for From the 1Olive View–UCLA Medical Center, David Geffen–UCLA School of Medicine, Sylmar and Los Angeles, California. Financial disclosure: None declared. Received for publication March 25, 2014; accepted for publication March 27, 2014. Corresponding Author: Ronald L. Koretz, MD, Olive View–UCLA Medical Center, 14445 Olive View Dr, Sylmar, CA 91342, USA. Email: [email protected]
Downloaded from pen.sagepub.com at UNIV OF GEORGIA LIBRARIES on June 24, 2015
1008
Journal of Parenteral and Enteral Nutrition 38(8)
exclusion. In other words, in order for a patient to be entered into the EN arm, additional criteria had to be met that were not required for entry into the control arm; patients could be entered into the control arm even if they would not have been eligible for the EN arm. Since the treated group needed to pass more criteria, patients assigned to that group would be expected to be admitted to the trial more slowly; hence, the 30 slots in the control arm were filled first, explaining the disparity in the numbers assigned at the midpoint in the trial. Given the reasons for excluding patients from the EN arm, it is not surprising that the control group had more residual peritonitis and a higher mortality rate. This confounding bias, rather than a true effect of the EN, is very likely responsible for the differences in the outcomes. The methodologic problem was that the investigators knew, before the patient was formally assigned to the EN group, what the assignment was going to be. In technical terms, allocation of treatment (ie, the assignment to a particular treatment arm) was not concealed. We briefly discussed the issue of concealment of allocation in the last column.7 The bias associated with a failure to employ concealment of allocation has been shown to inflate the estimated treatment effect.8 Finally, I would remind you that the word concealment does not refer to formal blinding of the intervention; blinding means that the involved personnel are not aware of the nature of the intervention during the conduct of the trial and collection of the data. Because of the use of EN, it would have been challenging to blind this study. The loss of concealment of allocation is the principal reason for methodologists to frown on the use of quasi-randomization. Quasi-randomization is the use of a patient record number, birth date or day of the week, or Social Security number to determine group assignment. While such numbers or dates may seem to be random events to the naive reader, there is a problem. While it is true that neither the investigator nor the patient has any control over that number or date, all of those involved individuals will know what that number or date is prior to entry into the trial. Thus, if the subject has a number that would put him or her into the control arm, he or she may decline to enroll for the trial. On the other hand, if the patient was scheduled to be entered into the treatment arm, but that patient appears a little sicker than average, the investigator
may (perhaps even subconsciously) look a little harder for some reason to exclude that potential study participant from the trial (which seems to have been the case in the study we are considering now). While this is not to say that Pupelis et al1,2 used a quasi-randomization scheme, they did employ some method of randomization such that the surgeon/investigator did know, at the time of randomization in the operating room, whether the patient would be assigned to the EN arm. For the next installment, please read the following article: O’Daly BJ, Walsh BC, Quinlan JA, et al. Serum albumin and total lymphocyte count as predictors of outcome in hip fractures. Clin Nutr. 2010;29:89-93.
Acknowledgment The author acknowledges the ongoing support of GIIssues, Inc, a nonprofit organization dedicated to the practice and dissemination of evidence-based medicine. GIIssues, Inc supports educational expenses for the author as well as any expenses related to the creation of papers and other educational products. GIIssues, Inc does not provide any salary support to the author. There are no research materials that are related to this article that can be accessed other than the stated references.
References 1. Pupelis G, Austrums E, Jansone A, Spruce R, Wehbi H. Randomised trial of safety and efficacy of postoperative enteral feeding in patients with severe pancreatitis: preliminary report. Eur J Surg. 2000;166:383-387. 2. Pupelis G, Selga G, Austrums E, Kaminski A. Jejunal feeding, even when instituted late, improves outcomes in patients with severe pancreatitis and peritonitis. Nutrition. 2001;17:91-94. 3. Koretz RL. JPEN Journal Club 3. When and why to randomize. JPEN J Parenter Enteral Nutr. 2014;38:400-401. 4. Sandstrom R, Drott C, Hyltander A, et al. The effect of postoperative intravenous feeding (TPN) on outcome following major surgery evaluated in a randomized study. Ann Surg. 1993;217:185-195. 5. McClave SA, Chang WK, Dhaliwal R, Heyland DK. Nutrition support in acute pancreatitis: a systematic review of the literature. JPEN J Parenter Enteral Nutr. 2006;30:143-156. 6. Koretz RL. Artificial nutrition in critical care: a critical look at the evidence. Br J Intensive Care. 2011;21:13-18. 7. Koretz RL. JPEN Journal Club 7. Subgroup analysis. JPEN J Parenter Enteral Nutr. 2014;38:905-906. 8. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-412.
Downloaded from pen.sagepub.com at UNIV OF GEORGIA LIBRARIES on June 24, 2015
