This article was downloaded by: [134.117.10.200] On: 29 November 2014, At: 05:11 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Biopharmaceutical Statistics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/lbps20

Adaptive Sequential Testing for Multiple Comparisons a

b

Ping Gao , Lingyun Liu & Cyrus Mehta

bc

a

The Medicines Company, Parsippany, New Jersey, USA

b

Cytel Inc., Cambridge, Massachusetts, USA

c

Harvard School of Public Health, Boston, Massachusetts, USA Accepted author version posted online: 13 Jun 2014.Published online: 11 Aug 2014.

To cite this article: Ping Gao, Lingyun Liu & Cyrus Mehta (2014) Adaptive Sequential Testing for Multiple Comparisons, Journal of Biopharmaceutical Statistics, 24:5, 1035-1058, DOI: 10.1080/10543406.2014.931409 To link to this article: http://dx.doi.org/10.1080/10543406.2014.931409

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

Journal of Biopharmaceutical Statistics, 24: 1035–1058, 2014 Copyright © Cytel, Inc. ISSN: 1054-3406 print/1520-5711 online DOI: 10.1080/10543406.2014.931409

ADAPTIVE SEQUENTIAL TESTING FOR MULTIPLE COMPARISONS Ping Gao1 , Lingyun Liu2 , and Cyrus Mehta2,3 1

The Medicines Company, Parsippany, New Jersey, USA Cytel Inc., Cambridge, Massachusetts, USA 3 Harvard School of Public Health, Boston, Massachusetts, USA

Downloaded by [134.117.10.200] at 05:11 29 November 2014

2

We propose a Markov process theory-based adaptive sequential testing procedure for multiple comparisons. The procedure can be used for confirmative trials involving multi-comparisons, including dose selection or population enrichment. Dose or subpopulation selection and sample size modification can be made at any interim analysis. Type I error control is exact. Key Words: Adaptive sequential testing; Exact type I error control; Markov process; Multiple comparisons; Transition density function.

1. INTRODUCTION The majority of phase 3 trials compare one dose of the new drug with a placebo. However, there are situations in which an optimal dose has not been identified prior to the Phase 3 trial. In these situations, it may be desirable to start the trial with several doses and conduct interim analyses to assess the efficacy of each dose. Ineffective doses may be dropped according to the results of the interim analyses. Sometimes there is potential that a drug is more effective in some subpopulation than in others, and it is desirable to be able to select such subpopulations based on interim analyses results. Further, it may also be desirable to reestimate the sample size after dose or subpopulation selection. Jennison and Turnbull (1997) showed that the accumulative data for a single comparison in clinical trials, in the form of score function, are typically approximate Brownian motions (which is a one-dimensional Markov process). Hence, the theory of the onedimensional Markov process is the most suitable mathematical tool for calculating the probabilities associated with the accumulative data in a single comparison and to derive statistical inferences such as p-values, estimates, and confidence intervals. Further, a vector of score functions (each score function describes a dose vs. control comparison) can be described by a multidimensional Markov process, and the theory of multidimensional Markov process is the most suitable mathematical tool for calculating the probabilities associated with the accumulative data for multiple dose comparisons and to derive statistical inferences such as p-values, estimates, and confidence intervals. The Markov process approach also allows for data-dependent sample size re-estimation. Our Markov process approach only borrows the intuition from Markov process theory, but does not require any prior knowledge or familiarity with the Markov process theory. Received July 12, 2013; Accepted December 4, 2013 Address correspondence to Dr. Ping Gao, Biostatistics, The Medicines Company, 8 Sylvan Way, Parsippany, NJ 07054, USA; E-mail: [email protected]

1035

Downloaded by [134.117.10.200] at 05:11 29 November 2014

1036

GAO ET AL.

This approach is parallel to the one-dimensional Markov process approach for adaptive sequential design (see Gao, Ware, and Mehta, 2008; Gao, Liu, and Mehta, 2013). A drop-the-loser approach proposed by Chen et al. (2010) allows for interim analyses, and uses fixed futility boundaries to drop ineffective doses. The concept of group sequential test procedure is similar to ours, but their method does not provide p-values and estimates, and there are no sample-size reestimations. Stallard and Friede (2008) also uses a score function to design group sequential testing. This method controls the family-wise error very conservatively. It also requires the number of doses to be dropped at any given interim analysis to be prespecified. As with all non-Markov-process approaches (including, e.g., D. Magirr et al., 2012), these methods do not provide statistical inferences such as p-value, point estimate, confidence interval, and sample size reestimation at interim analyses. In addition, our method can also be used for subpopulation enrichment, but it requires that the subpopulations be prespecified. An enrichment strategy that does not require prespecification of subpopulations is discussed in Mehta and Gao (2011). For multiple-dose comparison, our procedure does not require sample sizes for each dose group to be the same. Further, our procedure is not limited to comparing different doses of the same drug; it can be used to compare multiple drugs. In this article, we present the design of the adaptive group sequential test procedure. The issues of point estimation and confidence interval calculation require in-depth discussion and will be presented in another article. In the appendix, we construct “discrete time” Markov processes from a hypothetical trial to illustrate this approach. 2. MULTIDIMENSIONAL MARKOV PROCESSES Suppose that M comparisons (M doses of drug, or M treatments versus a common control, or M subpopulations) are being  evaluated in a clinical trial. The accumulative data   t = Wt,1 , . . . , Wt,M (where t = (t1 , . . . , tM ) are information times), with have the form W each component Wt,m = Wtm ,m ∼ N (θm tm , tm ) a normally distributed variable, and a joint density function ⎧    ⎫ ⎪   −1 x − t ◦ θ ⎪ ⎨ ⎬    −  x − t ◦ θ M 1 t , p x, t, θ, t = (2π )− 2 (det (t))− 2 exp ⎪ ⎪ 2 ⎩ ⎭

  where, θ = (θ1 , . . . , θM ), t ◦ θ = (θ1 t1 , . . . , θM tM ) , t = σij,t i,j=1,. . .,M .   Let ti = ti,1 , . . . , ti,M , be such that ti,m ≤ ti+1,m and denote this relationship as ti ≺ ti+1 (This means that ti is “before” ti+1 . Then    ti+1 − W  ti = Wti+1 ,1 − Wti ,1 , . . . , Wti+1 ,M − Wti ,M , W and Wti+1,m − Wti,m ∼ N    ti = xti = xti ,1 , . . . , xti ,M , Given W



  ti+1,m − ti,m θm , ti+1,m − ti,m .

ADAPTIVE SEQUENTIAL TESTING

1037

      Wti+1 ,m = xti ,m + Wti+1 ,m − Wti ,m ∼ N xti ,m + ti+1,m − ti,m θm , ti+1,m − ti,m .  Let ti ,ti+1 = (σ  lm,ti ,ti+1 ) be the covariance matrix, where σlm,ti ,ti+1 = cov Wti+1 ,l − Wti ,l , Wti+1 ,m − Wti ,m . (The calculation of the covariance matrix is discussed in the appendix.)  ti = xi is  ti+1 given W The conditional probability density for W    − 12 M  p xi , y, ti , ti+1 , ti ,ti+1 , θ = (2π)− 2 det ti ,ti+1

Downloaded by [134.117.10.200] at 05:11 29 November 2014

× exp

⎧          ⎫  ⎪  ⎪ ⎨ − y − x + ti+1 − ti ◦ θ t−1 ⎬   − t  y −  x + t i+1 i ◦θ i ,ti+1 ⎪ ⎩

2

⎪ ⎭

.

  Let t1 ≺ . . . ≺ ti+1 . Then similarly,  given Wt1 = x1 , . .. , Wti = xi , the conditional  ti+1 is again p xi , y, ti , ti+1 , ti ,ti+1 , θ , which is independent of probability density of W      t1 , . . . , W  ti−1 . Hence W  t = Wt1 ,1 , . . . WtM ,M = Wt,1 , . . . , Wt,M is a multidimensional W    t0 = x0 , the conditional probMarkov process. Let t0 ≺ t1 ≺ . . . ≺ tk . Given that W    tk = (x1 , . . . , xk ), which is the same probability den t1 , . . . , W ability density of W    t1 = x1 , then to  t0 = x0 , then to W sity that the Markov process moves from W  k−1   tk = xk , which is then i=0 p xi , xi+1 , ti , ti+1 , ti ,ti+1 , θ . Let  t2 = x2 , . . ., then to W W      ti ∈ ai =  ai,1 , . . . , ai,M , bi = bi,1 , . . . , bi,M , with ai,m < bi,m , m = 1, .., M. Denote W     M ai , bi = m=1 Wti ,m ∈ ai,m , bi,m . Then P



k i=1



        ti ∈ ai , bi W  t0 = x0 W

b1,1

=

 ...

a1,1



a1,M bi,1

×

  p x0 , x1 , t0 , t1 , t0 ,t1 , θ dx1,1 . . . dx1,M . . . ×

b1,M

 ...

ai,1

 ...

bi,M

  p xi−1 , xi , ti−1 , ti , ti−1 ,ti , θ dxi,1 . . . dxi,M . . .

ai,M bk,1

ak,1

 ...

bk,M

  p xk−1 , xk , tk−1 , tk , tk−1 ,tk , θ dxk,1 . . . dxk,M

aik,M

    t0 = t0 , which is equivalent to W  (0) = 0, Without the condition W P



i=1

 ×

    ti ∈ ai , bi W =



k

bi,1

a1,1

 ...

ai,1

 ...

b1,1

bi,M

 ...

b1,M a1,M

   x1 , 0,  t1 ,   , θ dx1,1 . . . dx1,M . . . × p 0, 0,t1

  p xi−1 , xi , ti−1 , ti , ti−1 ,ti , θ dxi,1 . . . dxi,M . . .

ai,M bk,1

ak,1

 ...

bk,M

ak,M

  p xk−1 , xk , tk−1 , tk , tk−1 ,tk , θ dxk,1 . . . dxk,M

1038

GAO ET AL.

3. EXIT BOUNDARY DETERMINATION IN A GROUP SEQUENTIAL DESIGN

Downloaded by [134.117.10.200] at 05:11 29 November 2014

Suppose that a sequential testing procedure is desired for a clinical trial in which M comparisons will be investigated. Each comparison is associated with a normal variable Wm (tm ) = Wt,m ∼ N (θm tm , tm ) , m = 1, . . . , M. Let  t1 ≺ . . . ≺ tk be a sequence  of  ti = Wti ,1 , . . . , Wti ,M . Let   . Let W = t , . . . , t information time points (vectors), and t i i,1 i,M     Then the conditi ,ti+1 = σlm,ti ,ti+1 , where σlm,ti ,ti+1 = cov Wti+1 ,l − Wti ,l , Wti+1 ,m − Wti ,m .     tional probability density for Wti+1 given Wti = x is p x, y, ti , ti+1 , ti ,ti+1 , θ . Let Hm,0 be the null hypothesis that θm = 0, and Hm,a be the alternative hypothesis that θm > 0. The null hypothesis here is that M drug T is not better = than drug C, in any of the comparisons, that is, H 0 m=1 Hm,0 , and the alterc  M M native hypothesis is Ha = = m=1 Hm,a (at least one θm > 0). Let m=1 Hm,0 e1 , . . . , eK be some exit In a group sequential test, e1 , . . . , eK are boundaries.    α K  chosen such that P max W ≥ e ≤ , . . . , W under the null i ti ,1 ti ,M i=1 2  (θ1 , . . . , θM ) = (0, . . . , 0). The null hypothesis is rejected hypothesis  of θ =    α K max W ≥ e > 2 . If ti ,ti+1 are known, then , . . . , W if P i ti ,1 ti ,M i=1       K can be calculated for each combination P i=1 max Wti ,1 , . . . , Wti ,M ≥ ei e1 , . . . , eK . Then e1 , . . . , eK can be selected such that Pθ=0  = ≤

   K   max Wti ,1 , . . . , Wti ,M ≥ ei = P Ai,θ θ=0

K  i=1

K i=1

i=1

P



i−1

      max Wtj ,1 , . . . , Wtj ,M < ej max Wti ,1 , . . . , Wti ,M ≥ ei



j=1

  θ=0

α 2

is satisfied, where Ai,θ = and 

  P Ai,θ =  ×

−∞



...



... 

× 1−

j=1



e1 −∞

ei−1

i−1

ei

−∞

e1 −∞

ei−1 −∞



(max{Wti ,1 , . . . , Wti ,M } ≥ ei ),

   x1 , 0,  t1 ,   , θ dx1,1 . . . dx1,M . . . p 0, 0,t1

  p xi−2 , xi−1 , ti−2 , ti−1 , ti−2 ,ti−1 , θ dxi−1,1 . . . dxi−1,M

 ...

(max{Wtj ,1 , . . . , Wtj ,M } < ej )

ei −∞

      p xi−1 , xi , ti−1 , ti , ti−1 ,ti , θ dxi,1 . . . dxi,M .

Let 1 ≤ L < K; using the transitional density probability function, P



K i=L+1

=P

Ai,θ

L  j=1

   K max Wtj,1 , . . . , Wtj,M < ej ∩

i=L+1

   max Wti,1 , . . . , Wti,M ≥ ei



ADAPTIVE SEQUENTIAL TESTING



 e1 −∞

=

 ×

P

... eL

−∞

e1 −∞ p

 ...

K  

eL

−∞



1039

  x1 , 0,  t1 ,   , θ dx1,1 . . . dx1,M . . . 0, 0,t1

  p xL−1 , xL , tL−1 , tL ,

tL−1 ,tL

    tL =xL max Wti ,1 , . . . , Wti ,M ≥ ei |W

 , θ ! dxL,1 . . . dxL,.

(1)

i=L+1

Downloaded by [134.117.10.200] at 05:11 29 November 2014

Equation (1) plays an essential role in controlling both conditional and overall type I error and calculating conditional power when adaptive changes are made to the sequential testing procedure

4. CRITICAL BOUNDARY SELECTION IN CLINICAL TRIAL DESIGN Suppose that M comparisons are  being investigated,  and interim analyses are planned at sample sizes n1 ≺ . . . ≺ nK ni = ni,1 , . . . ni,M , where ni,m is the sample size involved in the m-th comparison) with information times t1 ≺ . . . ≺ tK . Let T =  max tK,1 , . . . , tK,M . Let si = Tti , i = 0, 1, . . . , K. The si are often referred as “information fractions.” Usually, sample sizes are proportional to the information times (e.g., Whitehead, 1997), and nNi = Tti = si , i = 0, 1, . . . , K. The exit boundaries e1 , . . . , eK are associated with information times. In designing clinical trials, the precise information time is often not known. Hence, it is difficult to select the exit boundaries. However, another quantity, “critical” boundaries, can be determined using the “information fractions” without knowing the exact value of the information time i , m = 1, . . . , M, i = 1, . . . , K. points. The critical boundaries can be defined as ci,m = √eti,m √ √ (Note that ci,1 ti,1 = . . . = ci,M ti,M = ei .) With the scaling property (see appendix), under the null hypothesis of θ = 0, P

"

K i=1

M m=1



wt ,m ≥ ci , m √i ti , m

# =P

"

K i=1

M m=1



w ˜ si ,m ≥ ci , m √ si , m

#

That is, the “critical” boundaries ci,m , m = 1, . . . , M, i = 1, . . . , K are “scaling invariant.” Since si does not rely on the actual information time, ci,m , m = 1, . . . , M, i = 1, . . . , K can be selected using si instead of ti , when ti−1 ,ti (hence si−1 ,si are known). In order to be able √ √ to define ei , ci , 1 si,1 = . . . = ci,M si,M should be satisfied. But, si,1 = . . . = si,M = si is not required (e.g., Chen et al, 2010).

5. POWER CALCULATION AND SAMPLE SIZE DETERMINATION Suppose that a clinical trial is being designed. Suppose that M comparisons will be investigated in a group testing procedure. Suppose that the “information fraction” si i = 1, . . . , K have been determined, and that critical boundaries ci,m , m = 1, . . . , M, i = 1, . . . , K have been determined.√Let the information √ time at the final analysis be denoted as T, and let ti = Tsi , ei = ci,1√si,1 T = . . . = ci,M√si,M T .

1040

GAO ET AL.

The power will be calculated under the alternative hypothesis θ = 0 as Pθ,T = P = =

i=1

K i=1

i=1



Downloaded by [134.117.10.200] at 05:11 29 November 2014

P

K 

×

   max Wti ,1 , . . . , Wti ,M ≥ ei

K 

ei−1

−∞



"

i=1

       max Wtj ,1 , . . . , Wtj ,1 < ej ∩ max Wti ,1 , . . . , Wti ,M ≥ ei

j=1



e1

−∞

... 

... 

× 1−

ei

−∞

#



e1

−∞

ei−1

−∞

  p xi−2 , xi−1 , ti−2 , ti−1 ,  ti−2 ,ti−1 , θ dxi−1,1 . . . dxi−1,M ×

 ...

   x1 , 0,  t1 ,   , θ dx1,1 . . . dx1,M . . . × p 0, 0,t1

ei

−∞

    p xi−1 , xi , ti−1 , ti ,  ti−1 ,ti , θ dxi,1 . . . dxi,M

The desired power of the trial can be obtained by choosing large enough T, such that Pθ,T ≥ 1 − β. Sample size nK can then be determined by the relationship between T and nK . Then n1 , . . . , nK can be determined given s(1) i .

6. p-VALUES The calculation of p-values is based on the ordering≺MS discussed in the appendix. Suppose that thetrial is terminated at I-th interim analysis (information time tI ), with max WtI ,1 , . . . , WtI ,M = xI . Let θ = 0 and calculate p = Pθ=0

""

I−1  i=1

##

       max Wti ,1 , . . . , Wti ,M ≥ ei ∪ max WtI ,1 , . . . , WtI ,M ≥ xI

.

This is an exact p-value for the sequential test. If p ≤ α2 , then the null hypothesis of θ = 0 is rejected.

7. DROPPING COMPARISONS AND/OR SAMPLE SIZE MODIFICATION Suppose that M (1) comparisons will be investigated in a sequential testing procedure, and interim analyses are planned at time points (vectors) t1(1) ≺   . .. ≺ (1) (1) (1) (1) (1) (1)  ti(1) = tK1 , where ti = ti,1 , . . . , ti,M(1) , with exit boundaries e1 , . . . , eK1 . Let W       (1)  ti+1  (1) (1) = W  ti(1) . Suppose that at the L(1) -th −W Wt(1) ,1 , . . . , Wt(1) ,M(1) , and W t , t i i i i+1   interim analysis at tL(1)(1) , with information time vector tL(1)(1) = tL(1)(1) .1 , . . . , tL(1)(1) ,M(1) , there are observed values           t(1)(1) = W t(1)(1) , . . . , W t(1)(1) (1) = x(1)(1) , . . . , x(1)(1) (1) = x(1)(1) , W L L ,1 L ,M L ,1 L ,M L

ADAPTIVE SEQUENTIAL TESTING

1041

     = 0 is calcuwith max xL(1)(1) ,1 , . . . , xL(1)(1) ,M(1) < e(1) . The conditional type I error θ (1) L lated as P

K1



      (1) (1)   max Wt(1) ,1 , . . . , Wt(1) ,M(1) ≥ e(1) | W t =  x (1) (1) i L L

i=L(1) +1

=

i



K1 i=L(1) +1

e(1)(1)

+1

L

−∞

i

 ...

e(1)(1) L

−∞

+1

 p xL(1)(1) , yL(1)(1) +1 , tL(1)(1) , tL(1)(1) +1 , t(1) L

(1) (1) ,t (1) L

+1

, 0



dyL(1) +1,1 . . . dyL(1) +1,M(1) . . . 

Downloaded by [134.117.10.200] at 05:11 29 November 2014

...

(1) ei−1

−∞

$

 ...



× 1−

  (1) (1) p yi−2 , yi−1 , ti−2 , ti−1 , t(1) ,t(1) , 0 dyi−1,1 . . . dyi−1,M(1) ×

(1) ei−1

i−2 i−1

−∞

ei(1)

−∞

 ...

ei(1)

−∞

%   (1) (1)  p yi−1 , yi , ti−1 , ti , t(1) ,t(1) , 0 dyi,1 . . . dyi,M(1) . i−1 i

The overall type I error is (see equation (1))  (1)    K1 K1 L Ai,θ = P Ai,θ + P P i=1

=P  +

L(1) i=1

−∞

 ×



e1(1)

 Ai,θ

 ...

e(1)(1) L

−∞

×P

i=L(1) +1

i=1

   x1 , 0,  t1 ,   , 0 dx1,1 . . . dx1,M(1) . . . × p 0, 0,t1

e1(1) −∞

 ...

 Ai,θ

e(1)(1) L

−∞

K1 i=L(1) +1



  p xL(1) −1 , xL(1) , tL(1) −1 , tL(1) −1 ,tL(1) , 0 dxL(1) ,1 . . . dxL(1) ,M(1) ×

      (1) (1)   max Wt(1) ,1 , . . . , Wt(1) ,M(1) ≥ e(1) | W t =  x (1) (1) i L L i

i

= P1 + P2 Suppose that after the L(1) -th interim analysis, the number of comparisons (e.g., dropping some doses, or subgroups) will be reduced from M (1) to M (2) , and the sample size will be modified. (2) , . . . , mM Denote the remaining comparisons as m(2) (2) (This is a rearranged subset of   1    (2) (2) (1) 1, . . . , M , denoted as m1 , . . . , mM(2) ⊆ 1, . . . , M (1) ). Suppose that the remaining interim analyses are rescheduled at information times t1(2) ≺ . . . ≺ tK(2)2 .   (2) (2) (2)  t,M(2) = Let x0,M(2) = x (2) , . . . , x (2) be the observed value of W 0,m1 0,m (2) M " #     Wt,m(2) , . . . , Wt,m(2) at tL(1) ; then x(2) (2) , . . . , x(2) (2) is a subset of xL(1)(1) ,1 , . . . , xL(1)(1) ,M(1) , 0,m 0,m (2) 1 M(2) M #  1 "  (2) (2) (1) (1) denoted as x (2) , . . . , x (2) ⊂ xL(1) ,1 , . . . , xL(1) ,M(1) . 0,m1

0,m

M (2)

1042

GAO ET AL.

  Let t0(2) = t(2) (2) , . . . , t(2) (2) be the corresponding information time for the remain0,m1 0,m (2) M " #   (1) (2) (2) (2)  ing M comparisons at tL(1) . Then t (2) , . . . , t (2) ⊂ tL(1)(1) ,1 , . . . , tL(1)(1) ,M(1) . ...

0,m1 0,m (2) M (2) (2) Let e1 , . . . , eK2 be the new exit boundaries (for O’Brien–Fleming boundaries, e1(2) = e(2) K2 ); the new conditional type I error will then be

P

K2 

       t(1)(1) = x(1)(1) max Wt(2) ,m(2) , . . . , Wt(2) ,m(2) ≥ ei(2) |W L L

i=1

i

K2 

=

Downloaded by [134.117.10.200] at 05:11 29 November 2014

=

i=1

 ...

(2) ei−1

−∞

$



e1(2)

...

−∞

 ...



× 1−

...

  (2) (2) (2)    (2) (2) p x0,M ,  y , t , t ,  , 0 dy1,1 . . . dy1,M(2) . . . 1 0 (2) t ,t 1 0

1

  (2) (2) p yi−2 , yi−1 , ti−2 , ti−1 , t(2) ,t(2) , 0 dyi−1,1 . . . dyi−1,M(2) ×

(2) ei−1

i−2 i−1



−∞

e1(2)

−∞

−∞

ei(2)

M (2)

i

1

ei(2)

−∞

%   (2) (2) p yi−1 , yi , ti−1 , ti , t(2) ,t(2) , 0 dyi,1 . . . dyi,M(2) . i−1 i

The new overall type I error after adaptation (dropping comparison/modify sample size) will be P

 "     L(1) Ai,θ + P max Wt (1) ,1 , . . . , Wt (1) ,M(1) < e(1) i



L(1) i=1

i=1

i

i

K2  i=1

   max Wt (2) ,m(2) , . . . , Wt (2) ,m(2) ≥ e(2) i i

1

i

M (2)

= P1 + P2 where P2 = P

"

L(1)



   max Wt (1) ,1 , . . . , Wt (1) ,M(1) < e(1) i

i=1

i

i

K2  i=1

 =

−∞

 ×

e1(1)

e(1)(1) L −∞

×P

 ...

   max Wt (2) ,m(2) , . . . , Wt (2) ,m(2) ≥ ei(2) i

e1(1) −∞

 ...

−∞

M (2)

   x1 , 0,  t1 ,  , 0 dx1,1 . . . dx1,M . . . × p 0, 0,t

e(1)(1) L

i

1

  p xL(1) −1 , xL(1) , tL(1) −1 , tL(1) , tL(1) −1 ,tL(1) , 0 dx1,L(1) . . . dxM,L(1) ×

  K2  i=1

       t(1)(1) = x(1)(1) max Wt (2) ,m(2) , . . . , Wt (2) ,m(2) ≥ ei(2) |W L L i

1

i

M (2)

(2)  To maintain type I error control, we seek e(2) 1 , . . . , eK2 such that P2 = P2 , or

ADAPTIVE SEQUENTIAL TESTING

P

1043

 K2 

       t(1) max Wt (2) ,m(2) , . . . , Wt (2) ,m(2) ≥ ei(2) |W xL(1)(1) L(1) = 

i=1

=P

i

K1

M (2)

i

1



     (1) (1)   max Wt (1) ,1 , . . . , Wt (1) ,M(1) ≥ e(1) | W t =  x . (1) i L L(1)

i=L(1) +1



i

i

8. CONDITIONAL POWER CALCULATION

Downloaded by [134.117.10.200] at 05:11 29 November 2014

(1) time tL(1) , with observed value the L(1) -th interim  Suppose  analysis at information    t(1) W xL(1)(1) , such that max xL(1)(1) ,1 , . . . , xL(1)(1) ,M(1) < eL(1)(1) . L(1) =  Under the alternative hypothesis of θ = 0, the conditional power is calculated as

P

K1



      (1) (1)   max Wt (1) ,1 , . . . , Wt (1) ,M(1) ≥ e(1) | W t =  x (1) (1) i L L

i=L(1) +1

=

i



K1 i=L(1) +1

e(1)(1)

+1

L

−∞

i

 ...

e(1)(1) L

−∞

+1

 (1) (1) p xL(1) , y(1) , tL(1) , tL(1) +1 , t (1) +1 ,t (1) (1) L(1) +1 L(1)

L(1) +1

  ,θ

dyL(1) +1,1 . . . dyL(1) +1,M(1) . . .  ...

(1) ei−1

−∞

$

 ...



× 1−

i−2 i−1

−∞

ei(1)

−∞

  (1) (1) p yi−2 , yi−1 , ti−2 , ti−1 , t (1) ,t (1) , θ dyi−1,1 . . . dyi−1,M(1) ×

(1) ei−1

 ...

ei(1)

−∞

%   (1) (1)  p yi−1 , yi , ti−1 , ti , t (1) ,t (1) , θ dyi,1 . . . dyi,M(1) . i−1 i

Suppose that after the L(1) -th interim analysis, the comparisons will be reduced from M to M (2) , and/or sample size will be modified. Suppose the remaining interim analysis (2) (2) are rescheduled at information times t1 ≺ . . . ≺ tK2 . (2) Suppose that new exit boundaries have been selected as e(2) 1 , . . . , eK2 . Then the conditional power with the rescheduled interim analysis is calculated as (1)

P

K2  i=1

=

       t(1)(1) = x(1)(1) max Wt (2) ,m(2) , . . . , Wt (2) ,m(2) ≥ e(2) |W i L L i

K2  i=1

 ...

(2) ei−1

−∞

$ × 1−

e1(2)

−∞

 ...



ei(2)

−∞

 ...

e1(2)

−∞

M (2)

  (2) (2) (2) p x0,M y1 , t0 , t1 , t (2) , t (2) , θ dy1,1 . . . dy1,M(2) . . . (2) ,  0

1

  (2) (2) p yi−2 , yi−1 , ti−2 , ti−1 , t (2) ,t (2) , θ dyi−1,1 . . . dyi−1,M(2) ×

(2) ei−1

i−2 i−1

−∞

 ...

i

1

ei(2)

−∞

%   (2) (2)  p yi−1 , yi , ti−1 , ti , t (2) ,t (2) , θ dyi,1 . . . dyi,M(2) . i−1 i

1044

GAO ET AL. (2)

(2)

Then t1 , . . . , tK2 may be selected to obtain desired conditional power of 1 − β. Then the associated sample sizes n1(2) , . . . , nK(2)2 can be calculated according to the relationship between t and n.

Downloaded by [134.117.10.200] at 05:11 29 November 2014

9. p-VALUE WITH ADAPTATION The p-values and confidence intervals are calculated based on the ordering ≺AMS , which is discussed in the appendix. Suppose that in a multiple comparison sequential testing procedure, M (1) comparisons are being investigated. Suppose that the (1) (1) interim analyses are planned at time points (vectors) t1 ≺ . . . ≺ tK1 , with exit bound  (1)  (1) aries e1(1) , . . . , e(e) xL(1)(1) (with K1 . Suppose that at the L -th interim analysis, W tL(1) =        (1) , . . . , W   (1) was observed). was observed (hence W max xL(1)(1) ,1 , . . . , xL(1)(1) ,M(1) < e(1) t1 tL(1) L(1) Suppose that an adaptation is executed such that the number of comparisons dropped from M (1) to M (2) , and/or sample size were also modified. Suppose that the remaining (2) (2) interim analyses are rescheduled at information times t1 ≺ . . . ≺ tK2 , with adjusted exit (2) boundaries e(2) at the I (2) -th interim anal1 , . . . , eK2 . Suppose that the trial was terminated   (2) (2)  t(2) (2) (2) , . . . , w (2) (2) , and max w =  x ysis (information time tI (2) ), with W (2) (2) I tI (2) .m1 t (2) .m (2) = I     I M (2) (1)  t(1) t(1) xL(1)(1) (see be the backward image of tI (2) = xI(2) xI(2) (2) . Let (1) , x (1) (2) , given W L(1) =  Jθ

Jθ

appendix). Let θ = 0 and calculate p=P

& J0(1) −1  i=1

$ & ' %'    (1) (1) max Wt (1) ,1 , . . . , Wt (1) ,M (1) ≥ ei ∪ max Wt (1) ,1 , . . . , Wt (1) ,M (1) ≥ x (1) i

i

(1) J0

(1) J0

J0

This is the exact p-value for the sequential test. If p ≤ α2 , then the null hypothesis of θ = 0 is rejected. 10. SIMULATIONS We provide examples of the design of multiple dose comparisons. 10.1. Critical Boundary Selection and Type I Error Control Example 1. We consider two designs for a trial that compares two doses of a new drug with a control.

r In one design there will be one interim analysis at 50% enrollment (information fraction s1 = 0.5) and a final analysis (information fraction s2 = 1). We use an O’Brien–Fleming type critical boundary (Lan and DeMets, 1983) (Table 1). r In the second design, there will be one interim analysis at information fraction s1 = 1(3, ( one at s2 = 2 3, and a final analysis at s3 = 1. The boundary is an O’Brien–Fleming type critical boundary, (Table 2).

ADAPTIVE SEQUENTIAL TESTING

1045

Table 1 Critical boundary with two doses and two looks Look 1 2

Information fraction

Cumulative alpha

Critical boundary

0.5 1.0

0.002 0.025

3.163 2.221

Table 2 Critical boundary with two doses and three looks Look

Downloaded by [134.117.10.200] at 05:11 29 November 2014

1 2 3

Information fraction

Cumulative alpha

Critical boundary

0.333 0.667 1.000

0.0001 0.0060 0.0250

3.882 2.733 2.247

Table 3 Type I error control with two doses and with two and three looks (θ1 , θ2 )

Two look design

Three look design

(0,0) (0,0.2) (0,0.5) (0,0.8) (0,1)

0.0252 0.0127 0.0065 0.0014 0.0008

0.0253 0.0104 0.0040 0.0011 0.0004

r Type I error simulations are performed using the already-described selected O’Brien– Fleming boundaries for a study that enrolls 300 patients randomized to the two dose groups and the control group in a ratio of 1:1:1. The efficacy variable from each patient is normally distributed. Patients from the control have an N (0, 1) distribution, the low-dose group has an N (θ1 , 1) distribution, and the high-dose group has an N (θ2 , 1) distribution. The repetition for the simulation is 100,000. The probability is the rejection rate when the Z-score associated with a dose with θi = 0 crossed the critical boundary (a type I error) (Table 3). Example 2. We consider two designs for a trial that compares three doses of a new drug with a control.

r In one design there will be one interim analysis at 50% enrollment (information fraction s1 = 0.5) and a final analysis (information fraction s2 = 1). We use an O’Brien–Fleming type critical boundary (Table 4). r In the second design, there will be one interim analysis at information fraction s1 = 1(3, ( one at s2 = 2 3, and a final analysis at s3 = 1. The boundary is an O’Brien–Fleming type critical boundary (Table 5). r Type I error simulations are performed using the earlier selected O’Brien–Fleming boundaries for a study that enrolls 400 patients randomized to the three dose groups and the control group in a ratio of 1:1:1:1. The efficacy variable from each patient is normally

1046

GAO ET AL. Table 4 Critical boundary with three doses and two looks Look 1 2

Information fraction

Cumulative alpha

Critical boundary

0.5 1.0

0.002 0.025

3.274 2.358

Table 5 Critical boundary with three doses and three looks Look

Downloaded by [134.117.10.200] at 05:11 29 November 2014

1 2 3

Information fraction

Cumulative alpha

Critical boundary

0.333 0.667 1.000

0.0001 0.0060 0.0250

3.978 2.855 2.384

Table 6 Type I error control with three doses and with two and three looks (θ1 , θ2 , θ3 ) (0,0,0) (0,0,0.5) (0,0,1) (0,0,2) (0,0.5,1) (0,1,2) (0,1.5,2)

Two look design

Three look design

0.0247 0.0107 0.0012 0.0011 0.0007 0.0006 0.0005

0.0259 0.0071 0.0012 0.0001 0.0005 < 0.0005 0.0001

distributed. Patients from the control have an N (0, 1)distribution, the low-dose group has an N (θ1 , 1) distribution, the median-dose group has an N (θ2 , 1) distribution, and the high-dose has an N (θ3 , 1) distribution. The repetition for the simulation is 100,000. Type I error is the rejection rate when the Z-score associated with a dose with θi = 0 crossed the critical boundary (Table 6).

10.2. Type I Error Control With Sample Size Modification Simulations for type I error control with sample size modification were performed using the design in table 1 for a study that enrolls 300 patients randomized to the two dose groups and the control group in a ratio of 1:1:1. The efficacy variable from each patient is normally distributed. Patients from the control have an N(0,1) distribution, the low-dose group has an N (θ1 , 1), and the high-dose group has an N (θ2 , 1) distribution. The repetition for the simulation is 100,000. An interim analysis was performed when 50% of subjects (50 subjects in each arm) have been enrolled. The sample size was doubled to a total of 600 patients if the conditional power at the interim analysis was in a “promising zone” between 0.3 and 0.9. Type I error is the rejection rate when the Z-score associated with a dose with θi = 0 crossed the critical boundary (Table 7).

ADAPTIVE SEQUENTIAL TESTING

1047

Table 7 Type I error control with sample size increase (θ1 , θ2 )

Type I error

(0,0) (0,0.5) (0,1) (0,2)

0.0251 0.0072 0.0009 0.0008

APPENDIX

Downloaded by [134.117.10.200] at 05:11 29 November 2014

A.1. Multiple Subpopulation Comparison A.1.1. Covariance Matrix of “Discrete-Time” Markov Processes. Suppose that several subpopulations are investigated in the comparison of two drugs T and C for reduction of blood pressure. The drug that results in larger reduction in blood pressure is considered to be more effective. The reductions in blood pressure (change from baseline) were denoted as XT,i for the i-th subject in arm T and XC,j for the j-th subject in C. Let Di = XT,i − XC,i ∼ N μ, σ 2 . Let Ei = Diσ−μ ∼ N (0, 1). Let θ = μσ be the stan dardized mean. Let Wn = ni=0 Dσi (W0 = 0). Then Wn is the sum of the differences in blood pressure reduction between the first n pair of subjects from the two drug arms, and

Wn ∼ N (nθ, n). Let Bn = ni=0 Ei (B0 = 0). Then Bn ∼ N (0, n), and Wn = Bn + nθ , n = 1, 2, . . . Bn is a standard Brownian motion on discrete time points and Wn , n = 1, 2, . . . is a Brownian motion (on discrete time points) with drift. Let the subgroups be denoted as Gs , s = 1, . . . , S. Let n = (n0 , n1 , . . . , nS ) be the vector of sample sizes, and ns be the number of pairs of patients in the subgroup Gs . Let Wn,s = Bn,s + θs ns be the Brownian motion with drift associated with each subpopulation Gs . Let the overall population be denoted as G0 . The null hypothesis here is that drug T is not better than drug C in any of the subgroups, that is, H0 = ∩Ss=0 Hs,0 (all θs = 0, s = 1, . . . , S) and the alternative hypothesis is Ha = ∪Ss=0 Hs,a (at least  one θs > 0). The data collected at n = (n0 , n1 , . . . , nS ) are ¯ n = Wn,0 , Wn,1 , . . . , Wn,S . W For any subgroups Gsi and Gsj , let Gsi ∩sj = Gsi ∩ Gsj , and let ni∩j be the number of patients enrolled into each arm of the subpopulation Gsi ∩sj . Then 

 Dsi ,l nsj Dsj ,m cov Wn,i , Wn,j = cov , t=0 σ m=0 σ     nsi ∩sj Dsi ∩sj ,l nsi ∩sj Dsi ∩sj ,m = cov , = var Wnsi ∩sj ,si ∩sj = ni∩j . t=0 m=0 σ σ 



nsi

  where Wn,0 , Wn,1 , . . . , Wn,S has a joint density function ⎧   ⎫  ⎪   −1 x − nθ ⎪ ⎨ ⎬   −  x − n  θ S 1 p x, , θ, n = (2π)− 2 (det ())− 2 exp ⎪ ⎪ 2 ⎩ ⎭       where  = σij , σii = var Wn,i = ni , and σij = cov Wni ,si , Wnj ,sj = ni∩j , for i = j.

1048

GAO ET AL.

   The data   be Wn = Wn,0 , Wn,1 , . . . , Wn,S . The covariance matrix  will  observed  var Wn,i = ni , and σij = cov Wn,i , Wn,j = ni∩j , for i = j. Let ni = is  = σij , σii = ni,0 , ni,1 , . . . , ni,S , be such that ni,s ≤ ni+1,s and denote this relationship as ni ≺ ni+1 . Then    ni +1 − W  n1 = Wni +1,0 − Wni,0 , Wni+1,1 − Wni,1 , . . . , Wni+1,S − Wni ,S . W and ni+1,s

   Dl,s ∼ N ni+1,s − ni,s θs , ni+1,s − ni,s . σ    ni = xni = xni,0 , xni,1 , . . . , xni ,S , Given W

Downloaded by [134.117.10.200] at 05:11 29 November 2014

Wni+1,s − Wni,s =

Wni+1,s = xni,s +

l=ni,s +1

n i +1,s l=ni,s

    Dl,s ∼ N xni,s + ni+1,s − ni,s θs , ni+1,s − ni,s . σ +1

Let   cov Wni+1,m − Wni,m , Wni+1,l − Wni,l = σni ,ni+1 ,ml . 

Then, σni ,ni+1 ,ll = ni+1,l − ni,l , and σni, ni+1, ml = ni+1,m∩l − ni,m∩l . Let ni ,n+1 =   ni+1 given W  ni = xi is σni ,ni+1, ml . The conditional probability density for W    − 12 S  p xi , y, ni , ni+1 , ni ,ni+1 , θ = (2π)− 2 det n1 ,n2

× exp

⎧      ⎫  −1 ⎪ ⎪  ⎨ − y − xi + (ni+1 − ni ) ◦ θ ⎬ + n − n   y −  x ◦ θ ( ) i i+1 i ni ,ni+1 ⎪ ⎩

2

⎪ ⎭

 n1 = x1 , . . . , W  given W Let n1 ≺ . . . ≺ nk . Then similarly,   nk−1 = xk−1 , the condi nk is p xk−1 , y, nk−1 , nk , nk−1 ,nk , θ , which is independent of tional probability density of W    n1 , . . . , W  nk−2 . Hence W  n = Wn,0 , Wn,1 , . . . , Wn,S is a multidimensional Markov process W  (n0 ) = x0 , the conditional on a “discrete time point.” Let n0 ≺ n1 ≺ . . . nk . Given that W   nk = (x1 , . . . , xk ), which is the same probability density  n1 , . . . , W probability density of W    (n0 ) = x0 , then to W that the Markov process moves from W  n1 = x1 , then to Wn2 = x2 , . . . k−1   nk = xk , is then i=0 p xi , xi+1 , ni , ni+1 , ni ,ni+1 , θ . then to W A.1.2. Covariance Matrix for Markov Processes. Suppose that several subpopulations are investigated in the comparison of two drugs T and C. Let the subgroups be denoted as Gs , s = |1, . . . , S ; let t = (t1 , . . . , tS ) be the vector of information times. Let Wt,s = Bt,s + θs ts be the Brownian motion with drift associated with each subpopulation Gs . Let the overall population be denoted as G0 . The null hypothesis here S is, H0 = S is that drug T is not better than drug C in any of the subgroups, that s=0 Hs,0 (all θs = 0, s = 1, . . . , S), and the alternative hypothesis is Ha = s=0 Hs,a (at   t = Wt,0 , Wt,1 , . . . , Wt,S . least one θs > 0). The data collected at t = (t0 , t1 , . . . , tS ) are W

ADAPTIVE SEQUENTIAL TESTING

1049

For any subgroups Gsi and Gsj , let Gsi ∩sj = Gsi ∩ Gsj , and let ti∩j be the information time associated with the sub population Gsi ∩sj . Then, similar to the discrete case,     cov Wt,m , Wt,l = var Wt,Gm ∩Gl = tm∩l where Wt,Gm ∩Gl is the Brownian motion with drift associated with each subpopulation Gm ∩ time. Gl and tm∩l is its associated information  Let ti = ti,0 , ti,1 , . . . , ti,S be such that ti,s ≤ ti+1,s and denote this relationship as ti ≺ ti+1 . Then    ti+1 − W  ti = Wti+1,0 − Wti,0 , Wti+1,1 − Wti,1 , . . . , Wti+1,S − Wti,S . W

Downloaded by [134.117.10.200] at 05:11 29 November 2014

and Wti+1,s − Wti,s ∼ N



  ti+1,s − ti,s θs , ti+1,s − ti,s .

 ti = xti = (xti,0 , xti,1 , . . . , xti,S ), Wti+1,s = xti,s + (Wti+1,s − Wti,s ) ∼ N(xti,s + (ti+1,s − Given W ti,s ) θs , ti+1,s − ti,s ). Let cov(Wti+1,m − Wti,m , Wti+1,l − Wti,l ) = σti ,ti+1 ,ml . Then σti ,ti+1 ,ll = ti+1,l − ti,l , and σti ,ti+1 ,ml = ti+1,m∩l − ti,m∩l . Let ti, ti+1 = (σti, ti+1, ml ). The Conditional  ti+1 given W  ti = xi is probability density for W    − 12 S  p xi , y, ti , ti+1 , ti, ti+1 , θ = (2π)− 2 det t1, t2

× exp

⎧      −1    ⎫   ⎪ ⎨ − y − xi + ti+1 − ti ◦ θ ⎬   ◦ θ ⎪ + t − t  y −  x i i+1 i ti, ti+1 ⎪ ⎩

2

⎪ ⎭

 t1 = x1 , . . . , W  tk−1 = xk−1 , the conditional probaLet t1 ≺ . . . ≺ tk . Then similarly, given W   tk is p xk−1 , y, tk−1 , tk , tk−1, tk , θ , which is independent of W  tk−2 .  t1 , . . . , W bility density of W    t = Wt,0 , Wt,1 , . . . , Wt,S is a multidimensional Markov process. Let t0 ≺ t1 ≺ Hence W      t0 = x0 , the conditional probability density of W  t1 , . . . , W  tk = . . . ≺ tk . Given that W (x1 , . . . , xk ), which is the same probability density that the Markov process moves  t1 = x1 , then to W  t2 = x2 , . . . then to W  tk = xk , is then  t0 = x0 , then to W from W  k−1  xi , xi+1 , ti , ti+1 , ti, ti+1 , θ . i=0 p  A.2. Multiple Dose Comparisons A.2.1. Covariance Matrix of “Discrete-Time” Markov Processes. Suppose that several doses of drug T will be tested, and the dose with the best observed efficacy will be selected for the second stage testing. Let the dose groups be denoted as Gd , d = 1, . . ., D. Suppose that n patients were randomized to each dosing group (nonequal randomization is possible, but the calculation  for the covariance matrix will be complicated) and the control group. Let Xd,i ∼ N μd , σd2 , and XC,i ∼ N μC , σC2 be the efficacy measurement for dose group Gd and the control arm, respectively. For simplicity, the variances for all dose groups are assumed to be equal, that is, σd2i = σD2 . Let σ 2 = σD2 + σC2 .

1050

GAO ET AL.

Let Ed,i =

Xd,i −μd σd

∼ N (0, 1), Ec,i =

Xc,i −μc σc

Xd,i −Xc,i −μd−c ∼ N (0, 1). Let σ Xd,i −Xc,i −μd−c Let Dd−c,i = ∼ N (θd−c , 1). σ Hd−c,a be the alternative hypothesis that

∼ N (0, 1), Ed−c,i =

θd−c = μσd−c be the standardized mean difference. Let Hd,0 be the null hypothesis that θd−c = 0, and θd−c > 0. The null hypothesis here is that drug T is not better than drug C in any of the dose c  D groups, that is, H0 = D = d=1 Hd,0 , and the alternative hypothesis is Ha = d=1 Hd,0  

n Dd−c,i D ∼ N (nθd−c , n) · W0,d = 0 . Let d,a (at least one θd−c > 0). Let Wn,d = d=1 H i=0 σ   Bn,d = ni=0 Ed−c,i ∼ N (0, n) B0,d = 0 . Then Wn,d = Bn,d + nθd−c . Bn,d , n = 1, 2, . . ., is a standard Brownian motion on discrete time points and Wn,d , n = 1, 2, . . ., is a Brownian motion (on discrete time points) with drift θd−c . Note that if i = j, then n  n     Edi −c,l , Edj −c,m cov Wn,di , Wn,dj = cov Bn,di , Bn,dj = cov

Downloaded by [134.117.10.200] at 05:11 29 November 2014

l=0

= cov



n l=1

Xc,l n Xc,m , m=1 σ σ

 =

n l=1

m=0

 var

Xc,l σ

 =n

σC2 σ2

   n = Wn,1 , . . . , Wn,D . The covariance matrix is Hence the observed data will be W     σ2  = σij , σii = var Wdi −c,n = n, and σij = n σC2 , for i = j. Let n1 < . . . < nk . Then    ni+1 − W  ni = Wni+1 ,1 − Wni ,1 , . . . , Wni+1 ,D − Wni ,D W and Wni+1 ,dj − Wni ,dj =

ni+1 l=ni +1

  Ddj −c,l ∼ N (ni+1 − ni ) θdj −c , ni+1 − ni . σ





 ni = xi = xi,1 , . . . xi,D , Given W Wni+1 ,j = xi,j +

ni+1    Ddj −c,l ∼ N xi,j + (ni+1 − ni ) θdj −c , ni+1 − ni . σ l=n +1 i

Let

  cov Wdl −c,ni+1 − Wdl −c,ni , Wdm −c,ni+1 − Wdm −c,ni = σlm,ni ,ni+1 .

  σ2 Then σll,ni ,ni+1 = ni+1 − ni , and σlm,ni ,ni+1 = (ni+1 − ni ) σC2 . Let ni ,ni+1 = σlm,ni ,ni+1 . The  ni = xi is  ni+1 given W conditional probability density for W    − 12 D  p xi , y, ni , ni+1 , ni ,ni+1 , θ = (2π)− 2 det ni+1 ,ni

× exp

⎧       ⎫ −1 ⎪ ⎪  ⎨ − y − xi + (ni+1 − ni ) ◦ θ ⎬ + − n  y −  x ◦ θ (n ) i i+1 i ni ,ni+1 ⎪ ⎩

2

⎪ ⎭

.

 n1 = x1 , . . . , W  nk−1 = xk−1 , the conditional Let n1 < . . . nk . Then similarly, given W  probability density of Wnk is

ADAPTIVE SEQUENTIAL TESTING

1051

  p xk−1 , y, nk−1 , nk , nk−1 ,nk , θ

Downloaded by [134.117.10.200] at 05:11 29 November 2014

   nk−2 . Hence W  n = Wd1 −c,n , . . . , WdD −c,n is a  n, . . . , W which is independent of W  (n0 ) = x0 , the multidimensional Markov processon a “discretetime point.” Given that W   conditional probability density of Wn1 , . . . , Wnk = (x1 , . . . , xk ), which is the same prob  (n0 ) = x0 , then to W ability density that the Markov process moves from W  n1 = x1 , then to k−1     Wn2 = x2 , . . . then to Wnk = xk , is then i=0 p xi , xi+1 , ni , ni+1 , ni ,ni+1 , θ . A.2.2. Covariance Matrix of Markov Processes. Suppose that several doses of drug T will be tested, and the dose with the best observed efficacy will be selected for the second-stage testing. Let the dose groups be denoted as Gd , d = 1, . . . , D. Let t = (t1 , . . . , tD ) be the information time for each dose comparison. Unlike in the discrete case already discussed, the information times ti may not necessarily be equal (e.g., if the outcome is a binary variable, and the comparison is based on difference of rate of success or failure). Let Wt,d = Bt,d + θd td be the Brownian motion with drift associated with each dose comparison. The null hypothesis here is that drug T is not better than drug C at any dose. that is, H0 = ∩D d=0 Hd,0 (all θs = 0, s = 1, . . . , S), and the alternative hypothHd,a (at least one θd > 0). The data collected at t = (t1 , . . . , tD ) are esis is Ha = ∪D d=0    t = Wt,1 , . . . , Wt,D . Let W   cov Wt,m , Wt,l = σt,ml .   Let ti = ti,1 , . . . , ti,D be such that ti,d ≤ ti+1,d and denote this relationship as ti ≺ ti+1 . Then    ti+1 − W  ti = Wti+1,1 − Wti,1 , . . . , Wti +1,D − Wti,D W and Wti+1,d − Wti,d ∼ N  ti = xti = (xti ,1 , . . . , xti ,D ), Given W ti,d )θd , ti+1,d − ti,d ). Let



  ti+1,d − ti,d θd , ti+1,d − ti,d .

Wti+1,d = xti ,d + (Wti+1 ,d − wti ,d ) ∼ N(xti ,d + (ti+1,d −

  cov Wti+1,m − Wti,m , Wti+1,l − Wti,l = σti ,ti+1 ,ml . Then, σti ,ti+1 ,ll = ti+1,l − ti,l . The σti ,ti+1 ,ml will need to be calculated depending on the endpoint (e.g., difference in means,  difference in success rates, log-odds ratio, or log ti+1 given hazards ratio). Let ti ,ti+1 = σti ,ti+1 ,ml . The conditional probability density for W  Wti = xi is    − 12 D  p xi , y, t, ti+1 , ti ,ti+1 , θ = (2π)− 2 det t1 ,t2 ⎧         ⎫   ⎪ ⎨ − y − xi + ti+1 − ti ◦ θ t−1 ⎬   ◦ θ ⎪ + t − t  y −  x i i+1 i i ,ti+1 × exp . ⎪ ⎪ 2 ⎩ ⎭

1052

GAO ET AL.

  Let t1 ≺ . . . ≺ tk . Then similarly,  given Wt1 = x1 , . . . , W  tk−1 = xk−1 , the condi tk is p xk−1 , y, tk−1 , tk , tk−1 ,tk , θ , which is independent tional probability density of W    tk−2 . Hence W  t = Wt,1 , . . . , Wt,D is a multidimensional Markov process.  t1 , . . . , W of W    t0 = x0 , the conditional probability density of Let t0 ≺ t1 ≺ · · · ≺ tk . Given that W    tk = (x1 , . . . , xk ), which is the same probability density that the Markov pro t1 , . . . , W W       t0 = x0 , then to W cess moves from W   t1 = x1 , then to Wt2 = x2 , . . . then to Wtk = xk , is k−1 then i=0 p xi , xi+1 , ti , ti+1 , ti ,ti+1 , θ .

Downloaded by [134.117.10.200] at 05:11 29 November 2014

A.3. Scaling Property and Critical Boundary Selection In the planning of clinical trials, the sample sizes at interim analyses can be selected. However, the actual information time at interim analyses are often not known. Hence, it is difficult to select the exit boundaries for interim analyses. With the scaling property, it is possible and convenient to select the “critical” boundaries, which does not rely on the actual values of the information times.       ti . Let T > 0, and si+1 = ti+1 . Let W  ti+1 − W  # (si ) = W√(ti ) , and  ti ,ti+1 = W Let W T T  # (si + 1) − W  # (si ). Ws#i ,si+1 = W Let     σl,m,t1 = cov Wt1 ,l , Wt1 ,m , σl,m,t (1) ,t(1) = cov Wti+1 ,l − Wti ,l , Wti+1 ,m − Wti ,m i



Wt ,l Wt ,m = cov √ 1 , √1 T T



1 σl,m,t1 , T     Wti+1 ,l − Wti ,l Wti+1 ,m − Wti ,m = cov Ws#i+1 ,l − Ws#i ,l , Ws#i+1 ,m − Ws#i ,m = cov , √ √ T T

σl,m,s1 = cov σl,m,si ,si+1



i+1



Ws#1 ,l , Ws#1 ,m

=

1 σl,m,ti ,ti+1 T   si ,si+1 = σl,m,si ,si+1 , =

Then ti ,ti+1 = Tsi ,si+1     det ti ,ti+1 = T M det si ,si+1 

ti ,ti+1

−1

 −1 = T −1 si ,si+1

Let x = (x1 , . . . , xM ), y = (y1 , . . . , yM ),   M p x, y, ti , ti+1 , ti ,ti+1 , 0 = (2π)− 2

$



det ti ,ti+1

− 12

!%  −1 (y − x) (y − x) ti ,ti+1 exp − 2

ADAPTIVE SEQUENTIAL TESTING

1053

Let  #  x y (x1 , . . . , xM )  # (y1 , . . . , yM )  # # x# = √ = = x1 , . . . , xD(1) = y1 , . . . , y#M , y = √ = √ √ T T T T   − 1  M p x# , y# , si , si+1 , si ,si+1 , 0 = (2π )− 2 det si ,si+1 2  #   −1  # ! y − x# si ,si+1 y − x# exp − 2

Downloaded by [134.117.10.200] at 05:11 29 November 2014

    M T − 2 p x# , y# , si , si+1 , si ,si+1 , 0 = p x, y, ti , ti+1 , ti ,ti+1 , 0    − 12 M  p x# , y# , ti , ti+1 , ti ,ti+1 , 0 = (2π)− 2 det ti ,ti+1 ⎡          ⎤    y − x + ti+1 − ti ◦ θ t−1 ◦ θ − t  y −  x + t i+1 1  i ,ti +1 ⎢ ⎥ exp ⎣− ⎦ 2  √  M = T − 2 p x# , y# , si , si+1 , si ,si+1 , T θ Per the scaling property,   P t1 , . . . , tK1 , e1 . . . , eK1 , θ =

 K1  i=1



−∞

...

−∞





× 1−

i=1



e √1 T

−∞



−∞

  p yi−2 , yi−1 , ti−2 , ti−1 , ti−2 ,ti−1 , θ dyi−1,1 . . . dyi−1,M×

...

...

e √i T

−∞

e √1 T

−∞ ei−1 √ T

−∞

 ...

    p yi−1 , yi , ti−1 , ti , ti−1 ,ti , θ dyi,1 . . . dyi,M

ei

−∞

 

   y1 , 0,  t1 ,   , θ dy1,1 . . . dy1,M . . . p 0, 0,t1

e1



...

−∞

× 1−

−∞ ei

ei−1 √ T

$

ei−1

−∞

K1 

...

...



ei−1

..

=



e1

 √   y#1 , 0,  s1 ,  , T θ dy#1,1 . . . dy#1,M .. p 0, 0,s1

 √  p y#i−2 , y#i−1 , si−2 , si−1 , si−2 ,si−1 , T θ dy#i−1,1 . . . dy#i−1,M × e √i T

−∞

  p y#i−1 , y#i , si−1 , si , si−1 ,si , θ dy#i,1 . . . dy#i,M

  e1 eK √ = P s1 , . . . , sk1 , √ . . . , √ 1 , T θ T T

%

1054

GAO ET AL.

Downloaded by [134.117.10.200] at 05:11 29 November 2014

A.4. Ordering for Observations Under the Group Sequential Procedure The probability and p value associated with an observation in a statistical testing are calculated for events that are “more extreme” then the observed event. For example, suppose that a statistical testing involves a normally distributed variable Z ∼ N (μ, 1), with the null hypothesis H0 , μ = 0, and the one-sided alternative hypothesis H a , μ > 0,. Suppose that Z = z is observed. Then the associated one sided p value is calculated as p = P (Z ≥ z) (under the null hypothesis), instead of P (Z = z) (which is zero). The probability P (Z ≥ z) is the probability of all possible observations Z = x with x ≥ z, or all events that are “more extreme” than Z = z. Mathematically, the relationship ≥ is a (total) order. A total order, usually denoted  in mathematics, has basically the same properties as ≥. The relationship ≥ orders real numbers, while in general  orders a collection or a set of “elements.” Let’s denote the set as , and let a, b, c be any three elements in  (in the case of ≥,  = (−∞, ∞), elements in  are the real numbers). Then a total order has the following properties:

r If a  b and b  a, then a = b (antisymmetry); r If a  b and b  c, then a  c (transitivity); r Either a  b or b  a (totality). The relationship a  b can also be equivalently expressed as b  a . And we describe the relationship as b is “before” a, and a is “after” b, and a is “more extreme” than b. In order to define and calculate probabilities and p values in a multiple comparison sequential testing, an ordering must be defined. Suppose that in sequential testing for multiple interim analyses were planned at information time points t1 ≺ . . . ≺  comparison,   tK ti = t1,i , . . . , tM,i , with exit boundaries e1 , . . . , eK . Suppose that the trial terminated    t1 is observed.  t1 , . . . , W at the I-th interim analysis. Then the sequence of the form W Further, either t1 ≺ tK , with max wtI ,1 , . . . wtI,M ≥ eI (early stopping), or t1 = tK , and   ˜ , . . . , W ˜ ,  tK = xK (no early stopping). Suppose another observed sequence is W W t1 t1˜   ˜ ˜   with the testing stopped at t = tI(1) ˜ , such that either tI˜ ≺ tK , with max WtI˜ ,1 , . . . WtI˜,M ≥ ˜ = y (no early stopping). Then a sequential e (early stopping), or t = t , and W I˜

I˜

tK

K

K

MS stands for “multicomparison sequential”) can be defined testing order  “≺ ” (“MS”        MS  t1 , . . . , W  t1 , . . . , W ˜ t˜ ≺  tI , or W  tI is more “extreme” than ˜ t1 , . . . , W W such that W I   ˜ , if ˜ , . . . , W W t1

tI˜

r tI ≺ tI˜ or r tI ≺ tI˜ , and max Wt 1 , . . . , Wt M  ≥ max W˜ t ,1 , . . . , W˜ t ,M . I,

I,

I˜

I˜

    ˜ ˜ , . . . , W MS   tI as Wt1 , . . . , W For convenience, we may refer to the relationship W t1 tI ≺         ˜ , . . . , W ˜ ˜ . ˜ , . . . , W  t1 , . . . , W  tI or W  tI “is after” W  t1 , . . . , W “is before” W W t1 tI t1 tI     ˜ . It is straight˜ , . . . , W  tI MS W  t1 , . . . , W This relationship can also be written as W t1 tI forward to verify that this order “≺MS  ” is a total order.  With this ordering, if the observed   tI with max WtI ,1 , . . . , WtI ,M = xI , then the probability under the  t1 , . . . , W sequence is W    tI is  t1 , . . . , W multiple comparison sequential testing procedure of observing W

ADAPTIVE SEQUENTIAL TESTING

P

""

1055

##



       max wti, 1 , . . . , wti,M ≥ ei ∪ max wtI ,1 , . . . , wtI,M ≥ xI

I−1 i=1

,

   t1 , . . . , W  tI , which is exactly the probability of all events that are more “extreme” than W that is, P

""

I−1











##



max Wti ,1 , . . . , Wti ,M ≥ ei ∪ max WtI ,1 , . . . , WtI ,M ≥ xI

i=1

=P

Downloaded by [134.117.10.200] at 05:11 29 November 2014



" (W t1 ,. . .,W tI )≺MS



˜ ,. . .,W ˜ W t1 t˜





˜ ˜ , . . . , W W t1 tI˜



# .

I

It can be verified that ≺MS is a total order.

A.5. Ordering for Observations With Adaptation A.5.1. Backward Image. Suppose at i(2) th interim anal  that the trial terminated   (2) (2) (2)  t (2) = x (2) , and max W(2) (2) , . . . , W(2) (2) = ysis (information time tI (2) ), with W t ,m t ,m I I I (2)

I (2)

1

(1) (1)  xI(2) (2) . Then for each θ there is a unique J  , and a unique x (1) , such that θ

P

" (2) I

Jθ

−1



 max W

ti(2) ,m1(2)

i=1



,. . .,W

ti(2) ,m(2)(2) M

≥

ei(2)

M (2)



        t(1)(1) = x(1)(1) |W ∪ max Wt(2) ,m(2) , . . . , Wt(2) ,m(2) ≥ xI(2) (2) L L I (2)

=P $

"

Jθ(1)−1  i=L(1) +1

I (2)

1



   max Wt(1) ,1 , . . . , Wt(1) ,M(1) ≥ ei(1) i

i

&

'

∪ max Wt(1) ,1 , . . . , Wt(1) (1) J θ



M (2)

, x(1) t(1) Jθ(1) Jθ(1)  

(1) ,M J θ

(1)

% ≥

x(1) Jθ(1) 



and the pair is the backward image of     (2)  t(1)(1) = x(1)(1) . tI(2) , given W (2) , xI (2) L L

   t(1)(1) = x(1)(1) |W L L

'

  tI(2) xI(2) (or equivalently, of (2) ,  (2)

A.5.2. Ordering. Suppose that in a multiple comparison sequential testing procedure, M (1) comparisons are being investigated. Suppose that the interim analyses are planned at time points (vectors) t1(1) ≺ . . . ≺ tK(1)1 , with exit bound  (1) (1)  (1) xL(1)(1) (with aries e(1) 1 , . . . , eK1 . Suppose that at the L -th interim analysis, W tL(1) =       (1) , . . . , W  (1) was observed). ) was observed (hence W max xL(1)(1) ,1 , . . . , xL(1)(1) ,M(1) < e(1) t1 t (1) L(1) L Suppose that an adaptation is executed such that the number of comparisons dropped from M (1) to M (2) and/or sample size were also modified. Suppose that the remaining interim analyses are rescheduled at information times t1(2) ≺ . . . ≺ tK(2)2 , with adjusted exit

Downloaded by [134.117.10.200] at 05:11 29 November 2014

1056

GAO ET AL.

(2) boundaries e(2) the trial was terminated at the I (2) -th interim anal1 , . . . , eK2 . Suppose that      t(2) (2) , . . . , W (2) (2) = xI(2) = ysis (information time tI(2) (2) ), with W  (2) , and max W  ,m ,m t(2) t I (2) 1 I (2) I (2) M (2)    (2) , . . . , W  (2) (each W  (2) is an M (2) -dimensional vector) was observed W xI(2) (2) . Hence, tI t (2) t1i I after the adaptation. Thus, a typical sequence in a multiple  comparison sequential  (1) , . . . , W  (1) , W  (2) , . . . , W  (2) . Let another sequence testing may have the form W t1 t (1) t1 t (2) L I        ˜ t(1) = x˜ (1) (with  ˜ (1) , W ˜ (2) , . . . , W ˜ (2) . With such an observation,W ˜ (1) , . . . , W be W t1 t ˜ (1) ˜t1 ˜t˜(2) L˜ (1) L˜ (1) L I     (1) (1) (1) (1) (1) (1) max x˜ L˜ (1) ,1 , . . . , x˜ L˜ (1) ,M(1) < eL˜ (1) ) (this implies that max xi,1 , . . . , xi,M(1) < ei for i < L˜ (1) ) is observed at the L˜ (1) -th (L˜ (1) ) may not be equal to L(1) ) interim analysis, the num˜ (2) (M ˜ (2) may not be equal to M (2) ), and the ber of comparisons dropped from M (l) to M (1) sample size is modified, such that the remaining interim analyses at tL(1) ˜ (1) +1 ≺. . . ≺ tK1 are  ˜ ˜(2) ˜(2) rescheduled to time points ˜t1(2) < . . . < ˜tK(2) ˜ 2 (K2 may not be equal to K 2 , t1 , . . . , tK˜ 2   (may not be may not be equal to t1(2) , . . . , tK(2)2 ), with new exit boundaries e˜ 1(2) , . . . , e˜ (2) K˜ 2

equal to e1(2) , . . . , eK(2)2 ). The trial terminates at the I˜(2) -th (I˜(2) may not be equal to I (2) )     ˜ ˜t(2) = y˜ (2) , and max W (2) (2) , . . . , W (2) (2) = y(2) . Hence interim analysis, with W   (2) (2) ˜ ˜t˜(2) ,m ˜t˜(2) .m ˜1 ˜ ˜ (2) I˜ I˜(2) I I M   I    ˜ ˜t(2) ˜ ˜t(2) , . . . , W ˜ (2) -dimensional vector) was observed after  (2) is an M (each W W 1 I˜(2)     ti (1) (1) (1) , x , y and t be the “backward” images the adaptation. For each θ, let ˜t(1) Jθ(1) Jθ(1) J˜ θ(1) J˜ θ(1)         (2) (2) AMS (“AMS” stands for and ˜tI(2) of ˜tI(2) (2) , xI (2) ˜(2) , yI˜(2) , respectively. Then an ordering ≺ “adaptive multiple comparison sequential testing) can be defined such that

r

    ˜ (1) , W ˜ (2) , . . . , W ˜ (2) ˜ (1) , . . . , W  (1) , . . . , W  (1) , W  (2) , . . . , W  (2) ≺AMS W W   t t t t t t ˜t ˜t 1

L(1)

1

I (2)

L˜ (1)

1

I˜(2)

1

if and only if (1) (1) (1) (1) ˜ (1) J˜ θ(1)  < Jθ , or Jθ = Jθ and x (1) ≤ y ˜ (1) Jθ

r

Jθ

    ˜ (1) , W ˜ (2) , . . . , W ˜ (2) ˜ (1) , . . . , W   (1) , W   (2) , . . . , W   (2) ∼AMS W   (1) , . . . , W W   ˜t ˜t t t t t t t 1

L(1)

1

I (2)

L˜ (1)

1

if and only if

1

I˜(2)

(1) (1) (1) J˜ θ(1)  = Jθ and x (1) = y ˜ (1) . Jθ

Jθ

It can be verified that ≺AMS is a total order. A.6. Repeated Sample Size Modification Suppose that in a multiple comparison sequential design, interim analyses are planned at n1(1) ≺ . . . ≺ nK(1)1 (with information time points t1(1) ≺ . . . ≺ tK(1)1 ) with exit (1) boundaries e(1) 1 , . . . , eK1 . Suppose that the sample size has been modified after interim anal(1) (2) ysis tL(1) , with a new sample size n(2) K2 (and information time tK2 and planned interim analyses (2) (2) at n(2) K(2)2 (and information time points t1 ≺ . . . ≺ tK2 ), with exit boundaries 1 ≺...≺n

Downloaded by [134.117.10.200] at 05:11 29 November 2014

ADAPTIVE SEQUENTIAL TESTING

1057

(2) (2) (2) (2) e(2) 1 , . . . , eK2 . At any interim analysis tL(2) , with tL(2) ≺ tK2 , sample size may be modi(3) fied again to nK(3)3 (with information time tK3 ), with new interim analyses scheduled at (3) (3) (3) t1 ≺ . . . ≺ tK3 . Then the exit boundaries e1 , . . . , eK(3)3 can be selected in the same manner (2) as that of e(2) 1 , . . . , eK2 . This process can be further repeated if necessary. Suppose that the trial is modified N − 1 times, resulting in interim time  analyses  (i) (i) (i) (i)  points t1 ≺ . . . ≺ tKi , i = 1, . . . , N − 1, and with observations W tL(i) = xL(i) , i =   (N)  t(N) = xI(N) 1, . . . , N − 1. Suppose that the trial is terminated at tI (N) , with W (N) , and I (N)   (N) (N) max Wt (N) ,m(N) , . . . , Wt (N) ,m (N) = xI (N) = xi∗,I (N) . Let θ = θ1 = θ2 = . . . = θM and θ = 1 I (N) I (N) M (N) configuration). Then (θ1 , . . . , θM ) = (θ1 , . . . , θM(1) ) = (θ, . . . , θ) (this is the least  favorable   (N−1) (N−1) (N−2) (N−2) (1) (1) the “backward” images tJ (N−1) , x (N−1) , tJ (N−2) , x (N−2) , . . . , tJ (1) , x (1) can be succesJθ Jθ Jθ θ θ  θ   (N−1) (N−1) (N)  sively obtained, such that tJ (N−1) , x (N−1) is the backward image of tI (N) , xI(N) given (N) Jθ θ     (N−1) (m−1) (N−1) (m−1)  t (N−1) = x (N−1) , and for 2 ≤ N, then t (m−1) , x (m−1) is the backward image of W L L Jθ   Jθ     (m) (m) (1) (1) (m−1)  t(m−1)  tJ (m) , x (m) given W . After t , x =  x is obtained, (1) L(m−1) L(m−1) J (1) θ

Jθ

Jθ

θ



 x(N) f (θ ) = p θ, I (N) = Pθ



"

$

Jθ(1)  −1

i=1



 max Wt (1) ,1 , . . . , W (1) i

&



ti

,M (1)

≥

x(1) Jθ(1) 

'

∪ max Wt(1) ,1 , . . . , Wt(1) (1) J θ

(1) ,M J θ

(1)

≥ e(1) i



%'

can be calculated. The final (exact) p value is then f (0).

REFERENCES Armitage, P., McPherson, C. K., and Rowe, B. C. (1969). Repeated significance tests on accumulating data. Journal of the Royal Statistical Society (General) 132(2): 235–244. Bauer, P. (1989). Multistage testing with adaptive designs (with discussion). Biometrie und Informatik in Medizin und Biologie 20: 130–148. Bauer, P., and Kieser, M. (1999). Combining different phases in the development of medical treatments within a single trial. Statistics in Medicine 18: 1833–1848. Betensky, R. A. (1996). An O’Brien–Fleming sequential trial for comparing three treatments. Annals of Statistics 24(4): 1765–1791. Chen, Y. H. DeMets, D. L, Lan, K. K. G. (2010). Some drop-the-loser designs for monitoring multiple doses. Statistics in Medicine 29:1793–1807. Chung, K. L. (1982). Lectures From Markov Processes to Brownian Motion. New York, NY: Springer-Verlag. Doob, J. L. (1953). Stochastic Processes. New York, NY: Wiley. Gao, P., Ware, J. H., Mehta, C. (2008), Sample size re-estimation for adaptive sequential designs. Journal of Biopharmaceutical Statistics 18:1184–1196. Gao, P., Liu, L. Y., and Mehta, C. (2013b). Exact inference for adaptive group sequential designs. Statistics in Medicine, May (online). doi: 10.1002/sim.5847.

Downloaded by [134.117.10.200] at 05:11 29 November 2014

1058

GAO ET AL.

Jennison, C., Turnbull, B. W. (1997). Group sequential analysis incorporating covariance information. Journal of the American Statistical Association, 92: 1330–1441. Jennison, C., Turnbull, B. W. (2000). Group Sequential Methods With Applications to Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC. Käonig, F., Brannath, W., Bretz, F., Posch, M., (2008). Adaptive Dunnett tests for treatment selection. Statistics in Medicine 17:1612–1625. Kelly, P. J, Stallard, N., Todd, S., (2005). An adaptive group sequential design for phase II/III clinical trials that select a single treatment from several. Journal of Biopharmaceutical Statistics 15:641–658. Lan, K. K. G., DeMets, D. L. (1983). Discrete sequential boundaries for clinical trials. Biometrika 70: 659–663. Liu, Q., Pledger, G. W. (2005). Phase 2 and 3 combination designs to accelerate drug development. Journal of the American Statistical Association 100(470):493–502. Magirr, D., Jaki, T., Whitehead, J. (2012). A generalized Dunnett test for multi-arm multi-stage clinical studies with treatment selection. Biometrika, 99(2): 494–501. Mehta, C., Gao, P. (2011). Population enrichment designs: case study of a large multinational trial. Journal of Biopharmaceutical Statistics 21(4):831–845. Müller, H. H., Schäfer, H. (2001). Adaptive group sequential designs for clinical trials: combining the advantages of adaptive and of classical group sequential approaches. Biometrics 57: 886–891. Pocock, S. J. (1977), Group sequential methods in the design and analysis of clinical trials. Biometrika, 64: 191–199. Shen, L. (2001). An improved method of evaluating drug effect in a multiple dose clinical trial. Statistics in Medicine 20:1913–1929. Stallard, N., Todd, S. (2003). Sequential designs for phase III clinical trials incorporating treatment selection. Statistics in medicine 22:689–703. Stallard, N., Friede, T. (2008). A group-sequential design for clinical trials with treatment selection. Statistics in Medicine 27:6209–6227. Thall, P., Simon, R. Ellenburg, S. S. (1988). Two-stage selection and testing designs for comparative clinical trials. Biometrika 75(2):303–310. Thall, P., Simon, R. Ellenburg, S. S. (1989). A two-stage design for choosing among several experimental treatments and a control in clinical trials. Biometrics 45:537–547. Todd, S., Stallard, N. (2005). A new clinical trial design combining phase 2 and 3: Sequential designs with treatment selection and a change of endpoint. Drug Information Journal 39:109–118. Tsiatis, A. A., Rosner, G. L., Mehta, C. R. (1984). Exact confidence intervals following a group sequential test. Biometrics 40:797–803. Whitehead, J. (1986). Sample sizes for phase II and phase III clinical trials: An integrated approach. Statistics in Medicine 5:459–464. Whitehead, J. (1997). The Design and Analysis of Sequential Clinical Trials, rev. 2nd ed. West Sussex, England: John Wiley & Sons Ltd.