This article was downloaded by: [Tulane University] On: 21 January 2015, At: 19:32 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Biopharmaceutical Statistics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/lbps20
Statistical Considerations in Setting Product Specifications a
a
a
Xiaoyu Dong , Yi Tsong & Meiyu Shen a
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration Accepted author version posted online: 30 Oct 2014.
Click for updates To cite this article: Xiaoyu Dong, Yi Tsong & Meiyu Shen (2014): Statistical Considerations in Setting Product Specifications, Journal of Biopharmaceutical Statistics, DOI: 10.1080/10543406.2014.972511 To link to this article: http://dx.doi.org/10.1080/10543406.2014.972511
Disclaimer: This is a version of an unedited manuscript that has been accepted for publication. As a service to authors and researchers we are providing this version of the accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proof will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to this version also.
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Statistical Considerations in Setting Product Specifications Xiaoyu Dong, Yi Tsong, Meiyu Shen
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences,
Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD 20993;
M an
Center for Drug Evaluation and Research,
*
ed
Email: [email protected]
pt
This article reflects the views of the author and should not be constructed to represent Food and Drug Administration’s views or policies.
ce
ABSTRACT
According to ICH Q6A (1999), a specification is defined as a list of tests, references to analytical
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Xiaoyu Dong, PhD,
us
Correspondence should be addressed to:
cr ip
t
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration
procedures, and appropriate acceptance criteria, which are numerical limits, ranges, or other criteria for the tests described. For drug products, specifications usually consist of test methods and acceptance criteria for Assay, Impurities, pH, Dissolution, Moisture, and Microbial Limits, etc., depending on the dosage forms. They are usually proposed by the manufacturers, and
1
subject to the regulatory approval for use. When the acceptance criteria in product specifications cannot be pre-defined based on prior knowledge, the conventional approach is to use data from a limited number of clinical batches during the clinical development phases. Often in time, such
t
acceptance criteria is set as an interval bounded by the sample mean plus and minus two to four
cr ip
standard deviations. This interval may be revised with the accumulated data collected from
released batches after drug approval. In this paper, we describe and discuss statistical issues of
us
including reference interval, (Min, Max) method, tolerance interval, and confidence limit of
M an
percentiles. We also compare their performance in terms of interval width and the intended coverage. Based on our study results and review experiences, we make some recommendations on how to select appropriate statistical methods in setting product specifications to better ensure the product quality.
ed
Key Words: Specification; Quality Control; Tolerance Interval; Confidence limit; Percentile.
ce
pt
1.INTRODUCTION
Specifications are critical quality standards and requirements for drug substances and
drug products. According to ICH Q6A, a specification is defined as a list of tests, references to
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
commonly used approaches in setting or revising specifications (usually tighten the limits),
analytical procedures, and appropriate acceptance criteria, which are numerical limits, ranges, or other criteria for the tests described. For drug products, specifications usually consist of test methods and acceptance criteria for Assay, Impurities, pH, Dissolution, Water Content, and Microbial Limits, et. al. depending on the dosage form. The testing results of these quality
2
attributes from the sample will be compared to the acceptance criteria. If all the results are within the acceptance criteria, then the product batch can be released. Otherwise, thorough investigation may be needed to identify the root cause of out-of-specification (OOS) observations. Thus,
t
specifications are key elements to ensure the product quality as well as the consistency of the
cr ip
manufacturing process.
us
subject to regulatory approval for use during the new drug application process. When setting specifications, there are many important points to be considered, such as dosage form, clinical
M an
efficacy and safety, process controls, batch analyses, analytical procedures. Many literatures provide excellent introduction and overview of this area, including EBE Concept Paper (2013), EMA (2011), DiFeo (2003), Kumar and Palmieri (2009). Specifically, for some quality attributes, their specifications can be pre-specified by prior knowledge or clinical justification.
ed
For a few other quality attributes, specifications are given by USP general chapters. For example,
pt
the specification for content uniformity is provided USP; and the specification for dissolution is provided by USP. When no prior information is available, statistical analysis plays an important
ce
role in setting specifications. In this case, specification is often set as an interval bounded by the sample mean plus and minus two to four standard deviations. Sometimes, minimum (Min) and maximum from the sample are also used to establish specifications. As mentioned earlier, the
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
In general, specifications are proposed by manufacturers at the Phase III of development, and
original specification is usually determined using a limited number of clinical batches in Phase
III. Thus, the data may not be fully representative of the product and usually results in unpractical intervals. For this reason, after drug approval, original specifications may be revised by statistical analysis of the accumulated data collected from released batches or stability studies.
3
This post-marketing change is also subject to the approval from regulatory agencies. Tolerance interval method is a commonly used method in revision of specifications. In short, a tolerance interval is an interval cover at least p% of the population with a given confidence level. This
t
concept was introduced in earlier work in Wilks (1941), Wilks (1942), and Wald (1943) for
cr ip
normal distributions. However, tolerance interval approaches in general lead to over-wide intervals. Thus, in this paper, we also consider an alternative method, confidence limits of
us
specifications. For example, Wessels, et. al. (1997) presents a confidence limit based statistical
M an
approach to utilize stability data and a given shelf life in setting specifications. Yang (2013) presents a multivariate statistical model to establish specifications for correlated quality attributes.
In order to identify proper statistical methods to set meaningful and reasonable specifications,
ed
this paper provides in depth discuss on statistical properties of above commonly used methods,
pt
including Mean +/- k∙SD, (Min, Max), Tolerance Interval, and confidence limits of percentile. Then, these methods are compared with each other in terms of interval width and coverage for
ce
large sample sizes through theoretical and simulation studies. We also address the issues of consumer’s risk and manufacturer’s risk. Conclusions are presented in the last section.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
percentiles, in order to avoid the over-wide problem. Other methods are also proposed in setting
4
2.
STATISTICAL
METHODS
FOR
SETTING
t
SPECIFICATION
cr ip
For a given quality attribute, assume the testing results from batches, X1, X2, …, Xn, are random samples from a normal distribution with mean µ and variance σ2. If the desired specification can
us
0< p 50. These observations show that confidence limits of percentiles is a more stringent method than the tolerance interval method. It can reduce the risk of passing low quality product and thus better assure the product
cr ip
t
quality.
us
SAMPLES
M an
To set specifications which are close to the intended interval, all methods discussed in this paper are problematic when sample size is small. As presented earlier, for small samples, reference interval and (Min, Max) cannot provide assurance on the coverage due to the nature of these two intervals. Tolerance interval gives a too-wide interval with inflated consumer’s risk. Conversely,
ed
the confidence limits of percentiles provide a narrower interval with well controlled consumer’s risk. These findings are consistent with the fundamental principles of statistics as well as the
pt
nature of each method.
ce
Can large sample size solve all these problems? To answer this question, we compare the performance of above methods when the sample size n is no less than 30. Considering (Min, Max) is not suitable to set specifications for small and large samples, we will not include this
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
3. COMPARISON OF METHODS ON LARGE
method in our following discussion in this section. With n = 30, 50, 100, and 500 and the
intended coverage of 95%, Figure 7 shows the box plots of coverage from 105 Monte Carlo
simulations from the standard normal distribution using Reference Interval (RI), (97.5%, 95%) one-sided Tolerance Interval (TI) and 90% Confident Limits of Percentiles (CL of Perc.). From
12
Figure 7, we have following findings. First, a coverage distribution obtained from the reference interval method is almost symmetric about the target value of 95% when n = 500 with much reduced variability. Second, in general, the coverage from the tolerance interval method is the
t
highest among the all methods and larger than the target value most of time. Third, the coverage
cr ip
from the confidence limits of percentiles is the lowest in general and is lower than 95% mostly.
Although for n = 500, all methods produces similar results which are all close to 95%, the above
us
We also plot the percentiles of the interval limits in Figure 8 using the three methods. As can be
M an
seen, the interval obtained from tolerance interval method is the widest, while that from the confidence limits of percentiles is the tightest. As expected, interval limits from all three methods are similar and also are very close to the targeted interval of (-1.96, 1.96) when n = 500.
ed
Moreover, variability of the interval limits are similar.
To further evaluate the performance of each method, we study two types of misclassification
pt
probabilities. The first one is the inflation of consumer’s risk which is computed as the
ce
probability of been classified as normal data when it is actually an OOS data. The other one is the inflation of manufacturer’s risk defined as the probability of been classified as OOS data when it is actually a normal data. Please note, these two types of inflation do not include the
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
patterns are still presented.
parts caused by sampling plans. By the above definitions, these two probabilities can be formulated as (6) and (7), respectively.
13
Pinflate.C= I (θˆlow < θ1 ) Pr(θˆlow < X < θ1 | θˆlow ) + I (θˆup > θ 2 ) Pr(θ 2 < X < θˆup | θˆup )
(6)
Pinflate.M= I (θˆlow > θ1 ) Pr(θ1 < X < θˆlow | θˆlow ) + I (θˆup < θ 2 ) Pr(θˆup < X < θ 2 | θˆup )
(7)
t
Applying (6) and (7), Figures 9 and 10 provide the box plots of inflation rate of consumer’s risk
cr ip
and manufacturer’s risk for n = 30, 50, 100, and 500 using the same simulation scenarios as in
Figures 7 and 8. More specifically, the inflation of consumer’s risk is computed via the following
us
M an
Step 1: Generate X ~ N (0,1/ n) , and S 2 ~ χ n2−1 / (n − 1) independely;
Step 2: Compute the specification interval ( θˆlow , θˆup ) based on a given method;
1 Step 3: Let the underlying true specification interval be (θ1 , θ 2 ) , if θˆlow < θ1 , then I (θˆlow < θ1 ) =
ed
and compute the proportion of misclassification due to the underestimate of the lower specification limit as (1 − P) / 2 − Φ (θˆlow ) . Here, we use P as the targeted central coverage.
ce
pt
Similarly computation is applied to obtain the value of I (θˆup > θ 2 ) Pr(θ 2 < X < θˆup | θˆup ) . Step 4: Obtain Pinflate.C from the results in (2); Step 5: Repeat Steps 1 to 4 for a large number of times, say 105 times, to obtain the boxplot.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
simulation scheme. Here, we use P as the targeted central coverage.
A similar simulation scheme can be used to compute the inflation of manufacturer’s risk defined in (7).
14
4. SAMPLE SIZE CALCULATION In this section, we propose a sample size calculation method in the context of setting
cr ip
t
specifications. Particularly, if the intended central coverage is 100p%, say 95%, then the sample size can be obtained so that the probability of the estimated coverage within an acceptable range
us
as confidence level.
PX , S p − δ ≤ PX ( X − K .S < X < X + K .S | X , S ) ≤ p + δ ≥ γ
M an
(8)
, where the factor K depends on the statistical interval applied. For reference interval, K = Z(1+p)/2; for tolerance interval K= k= tn −1,γ ( Z (1+ p )/2 n ) / n ; and for confidence limits of percentiles
ed
K =CZ (1+ p )/ 2 − Z1−α 1/ n + Z (12 + p )/ 2 (C 2 − 1) . Similar idea of such sample size calculation has been
discussed in earlier work of Faulkenberry and Weeks (1968), Faulkenberry and Daly (1970),
pt
Guenther (1972) and Krishnamoorthy and Mathew (2009). Extending the derivation in
ce
Krishnamoorthy and Mathew (2009), it can be shown that equation (8) is equivalent to χ1,2 p −δ ( X n2 ) χ1,2 p +δ ( X n2 ) 2 < < EX n P U ≥ γ n kn2 kn2
(9)
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
of (p – δ, p + δ) is at least γ, where δ is a small number between 0 and 1 and γ can be interpreted
Where Xn ~ N(0, 1/n), U n2 ~ χ n2−1 / (n − 1) , χ df2 is the chi-square distribution with degrees of freedom of df, and χ df2 , p (θ ) is the 100p% percentile of χ df2 and non-centrality parameter of θ. Then the sample size n can be determined as the smallest number satisfying the inequality (9).
15
5. SUMMARY The purpose of this paper is to identify proper statistical methods for setting a specification
cr ip
t
interval which is close to the intended interval with target coverage. This is an important part in the product control strategy to assure the product quality and manufacturing process consistency.
us
impact on product quality at release and in-process control are considered in this paper. One is the coverage; the other is the width of the interval. In this paper, we discussed three common
M an
approaches, including reference interval, (Min, Max), and tolerance interval. We also propose an alternative method using confidence limits of percentiles. From our in-depth study and review experiences, we do not recommend using (Min, Max) to set specification. We also do not encourage change specifications using small samples. With relative large sample sizes, although
ed
all three other methods would generate similar results, tolerance interval mostly likely gives an interval wider than the intended and a most inflated consumer’s risk; while confidence limits of
pt
percentiles would have an interval slightly tighter than the intended and the least inflated
ce
consumer’s risk. Most importantly, all specifications need to be scientifically meaningful. REFERENCES
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
To evaluate the suitability of common statistical methods, two critical aspects with a direct
(1)
S. Chakraborti and J. Li (2007): Confidence Interval Estimation of a Normal Percentile,
The American Statistician, 61:4, 331-336.
16
(2)
Thomas, J. DiFeo. (2003). Drug product development: a technical review of chemistry,
manufacturing, and controls information for the support of pharmaceutical compound licensing activities. Drug Development and Industrial Pharmacy. 29: 939-958.
t
ICH Guideline Q6A, “Specifications: test procedures and acceptance criteria for new drug
cr ip
(3)
substances and new drug procedures” chemical substances”. 1999
us
EBE Concept Paper (2013), “Considerations in Setting Specifications”, European
Biopharmaceutical Enterprises.
EMA (2011), “Report on the expert workshop on setting specifications for biotech
M an
(5)
products”. European Medicines Agency, London, 9 September 2011. (6)
Guenther, W.C. (1972). Tolerance intervals for univariate distributions. Naval Research
(7)
ed
Logistic Quarterly, 19, 309-333.
Faulkenberry, G.D. and Daly, J.C. (1970). Sample size for tolerance limits on a normal
Faulkenberry, G.D. and Weeks, D. L. (1968). Sample size determination for tolerance
ce
(8)
pt
distribution. Technometrics, 12, 813-821.
limits. Technometrics, 43, 147-155.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
(4)
(9) Krishnamoorthy, K. and Mathew, T. (2009). Statistical tolerance regions – Theory, Applications and computations. Wiley series in probability and statistics. (10) Kumar, R. and Palmieri Jr, M. J. (2009). Points to consider when establishing drug product specifications for parenteral microspheres. The AAPS Journal. 12: 27-32.
17
(11)
Shen, M., Dong, X., Lee, Y. and Tsong, Y. (2014). Statistical evaluation of several
methods for cut point determination of immunogenicity screening assay. (Under peer review) (12)
Wald, A. (1943). An extension of Wilks’s method for setting tolerance limits. Annals of
cr ip
(13)
t
Mathematical Statistics, 13, 45-55.
Wessels, P., Holz, M., Erni, F., Krummen, K., and Ogorka, J. (1997). Statistical
Wilks, S. S. (1941). Determination of sample sizes for setting tolerance-limit factors for
M an
(14)
us
Development and Industrial Pharmacy. 23: 427-439.
normal distribution. Annals of Mathematical Statistics, 12, 91 – 96. (15)
Wilks, S. S. (1942). Statistical prediction with special reference to the problem of
(16)
ed
tolerance limits. Annals of Mathematical Statistics, 13, 400 – 409.
Yang, H. (2013). Setting specifications of correlated quality attributes through
ce
543.
pt
multivariate statistical modelling. Journal of Pharmaceutical Science and Technology. 67: 533-
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
evaluation of stability data of pharmaceutical products for specification setting. Drug
18
68%
87%
95%
99%
>99%
Z(1+p)/2
1
1.5
2
2.5
3
ce
pt
ed
M an
us
cr ip
t
p
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 1 – Percentiles for Commonly Used Coverage of the Standard Normal Distribution
19
p% = 95%,
p% = 99%,
p% = 99.7%,
Z(1+p)/2 = 1.64
Z(1+p)/2 = 1.96
Z(1+p)/2 = 2.58
Z(1+p)/2 = 3.00
n = 10
0.5
0.55
0.68
n = 20
0.35
0.39
n = 30
0.28
n = 50
0.22
cr ip
us
M an
0.47
0.53
0.38
0.43
0.24
0.3
0.33
0.21
0.23
ed 0.15
0.76
0.31
0.17
ce
pt
n = 100
t
p% = 90%,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 2 – Standard Deviation of X ± Z (1+ p )/2 × S for Various Values of Sample Size n and Intended Coverage of p
20
Percentiles
n = 10
n = 20
n = 30
n = 50
n = 100
0%
16
48.1
53.1
68.2
75.5
t
25%
79.3
83.8
85.4
86.7
87.8
89.3
50%
86.9
88.4
89
75%
92.4
100%
99.9
cr ip
us 89.3
89.6
89.9
91.5
91.2
90.4
99.5
99.2
98.2
97.5
92.6
n = 20
n = 30
n = 50
n = 100
n = 1000
27.2
46.9
64.7
75.2
84.3
92.6
25%
86.9
90.6
91.8
92.8
93.6
94.6
50%
92.9
94
94.3
94.6
94.8
95
n = 10
pt 0%
ce Zp = 1.96
85.9
91.9
Percentiles
p = 0.95,
n = 1000
92.1
ed
Zp = 1.64
M an
p = 0.9,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 3 – Percentiles of Coverage of X ± Z p × S with Intended Coverage of 100p% based on 105 Monte Carlo Simulations from N(0,1)
21
96.6
96.4
96.3
96.1
95.9
95.3
100%
100
99.9
99.8
99.5
99
96.8
Percentiles
n = 10
n = 20
n = 30
n = 50
n = 100
n = 1000
0%
36.1
65.5
77.5
86.5
93.7
98.1
25%
95.4
97.3
97.8
50%
98.3
75%
99.5
100
cr ip
us
98.9
98.8
98.9
99
99
99.4
99.4
99.3
99.3
99.1
100
100
99.9
99.5
100
ce
pt
100%
98.5
98.7
ed
Zp = 2.58
98.2
M an
p = 0.99,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
75%
22
n = 10
n = 30
n = 50
n = 50
n = 500 n = 1000
90%
1.64
2.90
2.21
2.06
1.92
1.76
95%
1.96
3.40
2.61
2.43
2.28
2.09
99%
2.58
4.39
3.38
3.16
99.7%
3.00
5.01
us 2.97
M an 3.62
ce
pt
ed
3.87
23
1.72
t
Z(p+1)/2
cr ip
100p%
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 4 – One-sided (β, 95%) Tolerance Factors k for a given intended central coverage of 100p%and sample size n with β = (1+p)/2
3.40
2.05
2.74
2.69
3.15
3.10
Table 5 – Percentiles of Coverage of the One-sided (97.5%, 95%) Tolerance Interval from 105 Monte Carlo Simulations with a Intended Coverage of p = 95% n = 10
n = 30
n = 50
n = 100
n = 500
n = 1000
k = 3.40
k =2.61
k =2.43
k =2.28
k =2.09
k = 2.05
0%
51.4
77.2
83.9
88.4
93.0
93.7
5%
95
95.4
95.5
10%
97.3
25%
99.2
t
cr ip
us 95.2
95.2
96
95.5
95.4
97.9
97.4
96.8
95.9
95.7
98.9
98.3
97.6
96.3
96
100
99.4
98.9
98.2
96.7
96.3
95%
100
99.8
99.5
98.9
97.2
96.7
50%
ed
96.3
pt
M an
95.4
96.6
ce
75%
99.8
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Percentile
24
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 1 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 5 Random Samples from the Standard Normal Distribution
25
I
-2.8
2.07
H
-2.18
2.01
G
-2.38
2.22
F
-2.22
2.65
E
-2.04
1.79
D
-1.6
2.77
C
-2.3
B
-1.95
A
-2.19
2.46
M an
2.34 1.55
-2
0
2
(Min, Max)
ce
pt
ed
-4
t
2.33
cr ip
-1.85
us
J
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Samples
Figure 2 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 50 Random Samples from the Standard Normal Distribution
26
4
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 3 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 100 Random Samples from the Standard Normal Distribution
27
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 4 – Box Plot of the Lower and Upper Bounds of Specification using (97.5%, 95%) Onesided Tolerance Interval from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96)
28
Figure 5 – Box Plot of Coverage using 90% Confidence Limits of Percentiles
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Coverage of p = 95%
29
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 6 – Box Plot of the Lower and Upper Bounds of Specification Interval using 90% Confidence Limits of Percentile from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96)
30
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 7 – Box Plot of the Lower and Upper Bounds of Specification Interval from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Coverage of 95% using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and 90% Confidence Limits of Percentiles (CL of Perc.)
31
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 8 – Box Plot of the Lower and Upper Bounds of Specification from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and 90% Confidence Limits of Percentiles (CL of Perc.)
32
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 9 – Box Plots of the Inflation Rate of Consumer’s Risk from 105 Monte Carlo Simulations Generated on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and Confidence Limits of Percentiles (CL of Perc.)
33
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 10 – Box Plot of the Inflation Rate of Manufacturer’s Risk from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and Confidence Limits of Percentiles (CL of Perc.)
34
Journal of Biopharmaceutical Statistics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/lbps20
Statistical Considerations in Setting Product Specifications a
a
a
Xiaoyu Dong , Yi Tsong & Meiyu Shen a
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration Accepted author version posted online: 30 Oct 2014.
Click for updates To cite this article: Xiaoyu Dong, Yi Tsong & Meiyu Shen (2014): Statistical Considerations in Setting Product Specifications, Journal of Biopharmaceutical Statistics, DOI: 10.1080/10543406.2014.972511 To link to this article: http://dx.doi.org/10.1080/10543406.2014.972511
Disclaimer: This is a version of an unedited manuscript that has been accepted for publication. As a service to authors and researchers we are providing this version of the accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proof will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to this version also.
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Statistical Considerations in Setting Product Specifications Xiaoyu Dong, Yi Tsong, Meiyu Shen
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences,
Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD 20993;
M an
Center for Drug Evaluation and Research,
*
ed
Email: [email protected]
pt
This article reflects the views of the author and should not be constructed to represent Food and Drug Administration’s views or policies.
ce
ABSTRACT
According to ICH Q6A (1999), a specification is defined as a list of tests, references to analytical
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Xiaoyu Dong, PhD,
us
Correspondence should be addressed to:
cr ip
t
Division of Biometrics VI, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration
procedures, and appropriate acceptance criteria, which are numerical limits, ranges, or other criteria for the tests described. For drug products, specifications usually consist of test methods and acceptance criteria for Assay, Impurities, pH, Dissolution, Moisture, and Microbial Limits, etc., depending on the dosage forms. They are usually proposed by the manufacturers, and
1
subject to the regulatory approval for use. When the acceptance criteria in product specifications cannot be pre-defined based on prior knowledge, the conventional approach is to use data from a limited number of clinical batches during the clinical development phases. Often in time, such
t
acceptance criteria is set as an interval bounded by the sample mean plus and minus two to four
cr ip
standard deviations. This interval may be revised with the accumulated data collected from
released batches after drug approval. In this paper, we describe and discuss statistical issues of
us
including reference interval, (Min, Max) method, tolerance interval, and confidence limit of
M an
percentiles. We also compare their performance in terms of interval width and the intended coverage. Based on our study results and review experiences, we make some recommendations on how to select appropriate statistical methods in setting product specifications to better ensure the product quality.
ed
Key Words: Specification; Quality Control; Tolerance Interval; Confidence limit; Percentile.
ce
pt
1.INTRODUCTION
Specifications are critical quality standards and requirements for drug substances and
drug products. According to ICH Q6A, a specification is defined as a list of tests, references to
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
commonly used approaches in setting or revising specifications (usually tighten the limits),
analytical procedures, and appropriate acceptance criteria, which are numerical limits, ranges, or other criteria for the tests described. For drug products, specifications usually consist of test methods and acceptance criteria for Assay, Impurities, pH, Dissolution, Water Content, and Microbial Limits, et. al. depending on the dosage form. The testing results of these quality
2
attributes from the sample will be compared to the acceptance criteria. If all the results are within the acceptance criteria, then the product batch can be released. Otherwise, thorough investigation may be needed to identify the root cause of out-of-specification (OOS) observations. Thus,
t
specifications are key elements to ensure the product quality as well as the consistency of the
cr ip
manufacturing process.
us
subject to regulatory approval for use during the new drug application process. When setting specifications, there are many important points to be considered, such as dosage form, clinical
M an
efficacy and safety, process controls, batch analyses, analytical procedures. Many literatures provide excellent introduction and overview of this area, including EBE Concept Paper (2013), EMA (2011), DiFeo (2003), Kumar and Palmieri (2009). Specifically, for some quality attributes, their specifications can be pre-specified by prior knowledge or clinical justification.
ed
For a few other quality attributes, specifications are given by USP general chapters. For example,
pt
the specification for content uniformity is provided USP; and the specification for dissolution is provided by USP. When no prior information is available, statistical analysis plays an important
ce
role in setting specifications. In this case, specification is often set as an interval bounded by the sample mean plus and minus two to four standard deviations. Sometimes, minimum (Min) and maximum from the sample are also used to establish specifications. As mentioned earlier, the
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
In general, specifications are proposed by manufacturers at the Phase III of development, and
original specification is usually determined using a limited number of clinical batches in Phase
III. Thus, the data may not be fully representative of the product and usually results in unpractical intervals. For this reason, after drug approval, original specifications may be revised by statistical analysis of the accumulated data collected from released batches or stability studies.
3
This post-marketing change is also subject to the approval from regulatory agencies. Tolerance interval method is a commonly used method in revision of specifications. In short, a tolerance interval is an interval cover at least p% of the population with a given confidence level. This
t
concept was introduced in earlier work in Wilks (1941), Wilks (1942), and Wald (1943) for
cr ip
normal distributions. However, tolerance interval approaches in general lead to over-wide intervals. Thus, in this paper, we also consider an alternative method, confidence limits of
us
specifications. For example, Wessels, et. al. (1997) presents a confidence limit based statistical
M an
approach to utilize stability data and a given shelf life in setting specifications. Yang (2013) presents a multivariate statistical model to establish specifications for correlated quality attributes.
In order to identify proper statistical methods to set meaningful and reasonable specifications,
ed
this paper provides in depth discuss on statistical properties of above commonly used methods,
pt
including Mean +/- k∙SD, (Min, Max), Tolerance Interval, and confidence limits of percentile. Then, these methods are compared with each other in terms of interval width and coverage for
ce
large sample sizes through theoretical and simulation studies. We also address the issues of consumer’s risk and manufacturer’s risk. Conclusions are presented in the last section.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
percentiles, in order to avoid the over-wide problem. Other methods are also proposed in setting
4
2.
STATISTICAL
METHODS
FOR
SETTING
t
SPECIFICATION
cr ip
For a given quality attribute, assume the testing results from batches, X1, X2, …, Xn, are random samples from a normal distribution with mean µ and variance σ2. If the desired specification can
us
0< p 50. These observations show that confidence limits of percentiles is a more stringent method than the tolerance interval method. It can reduce the risk of passing low quality product and thus better assure the product
cr ip
t
quality.
us
SAMPLES
M an
To set specifications which are close to the intended interval, all methods discussed in this paper are problematic when sample size is small. As presented earlier, for small samples, reference interval and (Min, Max) cannot provide assurance on the coverage due to the nature of these two intervals. Tolerance interval gives a too-wide interval with inflated consumer’s risk. Conversely,
ed
the confidence limits of percentiles provide a narrower interval with well controlled consumer’s risk. These findings are consistent with the fundamental principles of statistics as well as the
pt
nature of each method.
ce
Can large sample size solve all these problems? To answer this question, we compare the performance of above methods when the sample size n is no less than 30. Considering (Min, Max) is not suitable to set specifications for small and large samples, we will not include this
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
3. COMPARISON OF METHODS ON LARGE
method in our following discussion in this section. With n = 30, 50, 100, and 500 and the
intended coverage of 95%, Figure 7 shows the box plots of coverage from 105 Monte Carlo
simulations from the standard normal distribution using Reference Interval (RI), (97.5%, 95%) one-sided Tolerance Interval (TI) and 90% Confident Limits of Percentiles (CL of Perc.). From
12
Figure 7, we have following findings. First, a coverage distribution obtained from the reference interval method is almost symmetric about the target value of 95% when n = 500 with much reduced variability. Second, in general, the coverage from the tolerance interval method is the
t
highest among the all methods and larger than the target value most of time. Third, the coverage
cr ip
from the confidence limits of percentiles is the lowest in general and is lower than 95% mostly.
Although for n = 500, all methods produces similar results which are all close to 95%, the above
us
We also plot the percentiles of the interval limits in Figure 8 using the three methods. As can be
M an
seen, the interval obtained from tolerance interval method is the widest, while that from the confidence limits of percentiles is the tightest. As expected, interval limits from all three methods are similar and also are very close to the targeted interval of (-1.96, 1.96) when n = 500.
ed
Moreover, variability of the interval limits are similar.
To further evaluate the performance of each method, we study two types of misclassification
pt
probabilities. The first one is the inflation of consumer’s risk which is computed as the
ce
probability of been classified as normal data when it is actually an OOS data. The other one is the inflation of manufacturer’s risk defined as the probability of been classified as OOS data when it is actually a normal data. Please note, these two types of inflation do not include the
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
patterns are still presented.
parts caused by sampling plans. By the above definitions, these two probabilities can be formulated as (6) and (7), respectively.
13
Pinflate.C= I (θˆlow < θ1 ) Pr(θˆlow < X < θ1 | θˆlow ) + I (θˆup > θ 2 ) Pr(θ 2 < X < θˆup | θˆup )
(6)
Pinflate.M= I (θˆlow > θ1 ) Pr(θ1 < X < θˆlow | θˆlow ) + I (θˆup < θ 2 ) Pr(θˆup < X < θ 2 | θˆup )
(7)
t
Applying (6) and (7), Figures 9 and 10 provide the box plots of inflation rate of consumer’s risk
cr ip
and manufacturer’s risk for n = 30, 50, 100, and 500 using the same simulation scenarios as in
Figures 7 and 8. More specifically, the inflation of consumer’s risk is computed via the following
us
M an
Step 1: Generate X ~ N (0,1/ n) , and S 2 ~ χ n2−1 / (n − 1) independely;
Step 2: Compute the specification interval ( θˆlow , θˆup ) based on a given method;
1 Step 3: Let the underlying true specification interval be (θ1 , θ 2 ) , if θˆlow < θ1 , then I (θˆlow < θ1 ) =
ed
and compute the proportion of misclassification due to the underestimate of the lower specification limit as (1 − P) / 2 − Φ (θˆlow ) . Here, we use P as the targeted central coverage.
ce
pt
Similarly computation is applied to obtain the value of I (θˆup > θ 2 ) Pr(θ 2 < X < θˆup | θˆup ) . Step 4: Obtain Pinflate.C from the results in (2); Step 5: Repeat Steps 1 to 4 for a large number of times, say 105 times, to obtain the boxplot.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
simulation scheme. Here, we use P as the targeted central coverage.
A similar simulation scheme can be used to compute the inflation of manufacturer’s risk defined in (7).
14
4. SAMPLE SIZE CALCULATION In this section, we propose a sample size calculation method in the context of setting
cr ip
t
specifications. Particularly, if the intended central coverage is 100p%, say 95%, then the sample size can be obtained so that the probability of the estimated coverage within an acceptable range
us
as confidence level.
PX , S p − δ ≤ PX ( X − K .S < X < X + K .S | X , S ) ≤ p + δ ≥ γ
M an
(8)
, where the factor K depends on the statistical interval applied. For reference interval, K = Z(1+p)/2; for tolerance interval K= k= tn −1,γ ( Z (1+ p )/2 n ) / n ; and for confidence limits of percentiles
ed
K =CZ (1+ p )/ 2 − Z1−α 1/ n + Z (12 + p )/ 2 (C 2 − 1) . Similar idea of such sample size calculation has been
discussed in earlier work of Faulkenberry and Weeks (1968), Faulkenberry and Daly (1970),
pt
Guenther (1972) and Krishnamoorthy and Mathew (2009). Extending the derivation in
ce
Krishnamoorthy and Mathew (2009), it can be shown that equation (8) is equivalent to χ1,2 p −δ ( X n2 ) χ1,2 p +δ ( X n2 ) 2 < < EX n P U ≥ γ n kn2 kn2
(9)
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
of (p – δ, p + δ) is at least γ, where δ is a small number between 0 and 1 and γ can be interpreted
Where Xn ~ N(0, 1/n), U n2 ~ χ n2−1 / (n − 1) , χ df2 is the chi-square distribution with degrees of freedom of df, and χ df2 , p (θ ) is the 100p% percentile of χ df2 and non-centrality parameter of θ. Then the sample size n can be determined as the smallest number satisfying the inequality (9).
15
5. SUMMARY The purpose of this paper is to identify proper statistical methods for setting a specification
cr ip
t
interval which is close to the intended interval with target coverage. This is an important part in the product control strategy to assure the product quality and manufacturing process consistency.
us
impact on product quality at release and in-process control are considered in this paper. One is the coverage; the other is the width of the interval. In this paper, we discussed three common
M an
approaches, including reference interval, (Min, Max), and tolerance interval. We also propose an alternative method using confidence limits of percentiles. From our in-depth study and review experiences, we do not recommend using (Min, Max) to set specification. We also do not encourage change specifications using small samples. With relative large sample sizes, although
ed
all three other methods would generate similar results, tolerance interval mostly likely gives an interval wider than the intended and a most inflated consumer’s risk; while confidence limits of
pt
percentiles would have an interval slightly tighter than the intended and the least inflated
ce
consumer’s risk. Most importantly, all specifications need to be scientifically meaningful. REFERENCES
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
To evaluate the suitability of common statistical methods, two critical aspects with a direct
(1)
S. Chakraborti and J. Li (2007): Confidence Interval Estimation of a Normal Percentile,
The American Statistician, 61:4, 331-336.
16
(2)
Thomas, J. DiFeo. (2003). Drug product development: a technical review of chemistry,
manufacturing, and controls information for the support of pharmaceutical compound licensing activities. Drug Development and Industrial Pharmacy. 29: 939-958.
t
ICH Guideline Q6A, “Specifications: test procedures and acceptance criteria for new drug
cr ip
(3)
substances and new drug procedures” chemical substances”. 1999
us
EBE Concept Paper (2013), “Considerations in Setting Specifications”, European
Biopharmaceutical Enterprises.
EMA (2011), “Report on the expert workshop on setting specifications for biotech
M an
(5)
products”. European Medicines Agency, London, 9 September 2011. (6)
Guenther, W.C. (1972). Tolerance intervals for univariate distributions. Naval Research
(7)
ed
Logistic Quarterly, 19, 309-333.
Faulkenberry, G.D. and Daly, J.C. (1970). Sample size for tolerance limits on a normal
Faulkenberry, G.D. and Weeks, D. L. (1968). Sample size determination for tolerance
ce
(8)
pt
distribution. Technometrics, 12, 813-821.
limits. Technometrics, 43, 147-155.
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
(4)
(9) Krishnamoorthy, K. and Mathew, T. (2009). Statistical tolerance regions – Theory, Applications and computations. Wiley series in probability and statistics. (10) Kumar, R. and Palmieri Jr, M. J. (2009). Points to consider when establishing drug product specifications for parenteral microspheres. The AAPS Journal. 12: 27-32.
17
(11)
Shen, M., Dong, X., Lee, Y. and Tsong, Y. (2014). Statistical evaluation of several
methods for cut point determination of immunogenicity screening assay. (Under peer review) (12)
Wald, A. (1943). An extension of Wilks’s method for setting tolerance limits. Annals of
cr ip
(13)
t
Mathematical Statistics, 13, 45-55.
Wessels, P., Holz, M., Erni, F., Krummen, K., and Ogorka, J. (1997). Statistical
Wilks, S. S. (1941). Determination of sample sizes for setting tolerance-limit factors for
M an
(14)
us
Development and Industrial Pharmacy. 23: 427-439.
normal distribution. Annals of Mathematical Statistics, 12, 91 – 96. (15)
Wilks, S. S. (1942). Statistical prediction with special reference to the problem of
(16)
ed
tolerance limits. Annals of Mathematical Statistics, 13, 400 – 409.
Yang, H. (2013). Setting specifications of correlated quality attributes through
ce
543.
pt
multivariate statistical modelling. Journal of Pharmaceutical Science and Technology. 67: 533-
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
evaluation of stability data of pharmaceutical products for specification setting. Drug
18
68%
87%
95%
99%
>99%
Z(1+p)/2
1
1.5
2
2.5
3
ce
pt
ed
M an
us
cr ip
t
p
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 1 – Percentiles for Commonly Used Coverage of the Standard Normal Distribution
19
p% = 95%,
p% = 99%,
p% = 99.7%,
Z(1+p)/2 = 1.64
Z(1+p)/2 = 1.96
Z(1+p)/2 = 2.58
Z(1+p)/2 = 3.00
n = 10
0.5
0.55
0.68
n = 20
0.35
0.39
n = 30
0.28
n = 50
0.22
cr ip
us
M an
0.47
0.53
0.38
0.43
0.24
0.3
0.33
0.21
0.23
ed 0.15
0.76
0.31
0.17
ce
pt
n = 100
t
p% = 90%,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 2 – Standard Deviation of X ± Z (1+ p )/2 × S for Various Values of Sample Size n and Intended Coverage of p
20
Percentiles
n = 10
n = 20
n = 30
n = 50
n = 100
0%
16
48.1
53.1
68.2
75.5
t
25%
79.3
83.8
85.4
86.7
87.8
89.3
50%
86.9
88.4
89
75%
92.4
100%
99.9
cr ip
us 89.3
89.6
89.9
91.5
91.2
90.4
99.5
99.2
98.2
97.5
92.6
n = 20
n = 30
n = 50
n = 100
n = 1000
27.2
46.9
64.7
75.2
84.3
92.6
25%
86.9
90.6
91.8
92.8
93.6
94.6
50%
92.9
94
94.3
94.6
94.8
95
n = 10
pt 0%
ce Zp = 1.96
85.9
91.9
Percentiles
p = 0.95,
n = 1000
92.1
ed
Zp = 1.64
M an
p = 0.9,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 3 – Percentiles of Coverage of X ± Z p × S with Intended Coverage of 100p% based on 105 Monte Carlo Simulations from N(0,1)
21
96.6
96.4
96.3
96.1
95.9
95.3
100%
100
99.9
99.8
99.5
99
96.8
Percentiles
n = 10
n = 20
n = 30
n = 50
n = 100
n = 1000
0%
36.1
65.5
77.5
86.5
93.7
98.1
25%
95.4
97.3
97.8
50%
98.3
75%
99.5
100
cr ip
us
98.9
98.8
98.9
99
99
99.4
99.4
99.3
99.3
99.1
100
100
99.9
99.5
100
ce
pt
100%
98.5
98.7
ed
Zp = 2.58
98.2
M an
p = 0.99,
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
75%
22
n = 10
n = 30
n = 50
n = 50
n = 500 n = 1000
90%
1.64
2.90
2.21
2.06
1.92
1.76
95%
1.96
3.40
2.61
2.43
2.28
2.09
99%
2.58
4.39
3.38
3.16
99.7%
3.00
5.01
us 2.97
M an 3.62
ce
pt
ed
3.87
23
1.72
t
Z(p+1)/2
cr ip
100p%
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Table 4 – One-sided (β, 95%) Tolerance Factors k for a given intended central coverage of 100p%and sample size n with β = (1+p)/2
3.40
2.05
2.74
2.69
3.15
3.10
Table 5 – Percentiles of Coverage of the One-sided (97.5%, 95%) Tolerance Interval from 105 Monte Carlo Simulations with a Intended Coverage of p = 95% n = 10
n = 30
n = 50
n = 100
n = 500
n = 1000
k = 3.40
k =2.61
k =2.43
k =2.28
k =2.09
k = 2.05
0%
51.4
77.2
83.9
88.4
93.0
93.7
5%
95
95.4
95.5
10%
97.3
25%
99.2
t
cr ip
us 95.2
95.2
96
95.5
95.4
97.9
97.4
96.8
95.9
95.7
98.9
98.3
97.6
96.3
96
100
99.4
98.9
98.2
96.7
96.3
95%
100
99.8
99.5
98.9
97.2
96.7
50%
ed
96.3
pt
M an
95.4
96.6
ce
75%
99.8
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Percentile
24
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 1 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 5 Random Samples from the Standard Normal Distribution
25
I
-2.8
2.07
H
-2.18
2.01
G
-2.38
2.22
F
-2.22
2.65
E
-2.04
1.79
D
-1.6
2.77
C
-2.3
B
-1.95
A
-2.19
2.46
M an
2.34 1.55
-2
0
2
(Min, Max)
ce
pt
ed
-4
t
2.33
cr ip
-1.85
us
J
Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
Samples
Figure 2 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 50 Random Samples from the Standard Normal Distribution
26
4
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 3 – Plots of (Min, Max) of 10 Monte Carlo Simulations from 100 Random Samples from the Standard Normal Distribution
27
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 4 – Box Plot of the Lower and Upper Bounds of Specification using (97.5%, 95%) Onesided Tolerance Interval from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96)
28
Figure 5 – Box Plot of Coverage using 90% Confidence Limits of Percentiles
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Coverage of p = 95%
29
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 6 – Box Plot of the Lower and Upper Bounds of Specification Interval using 90% Confidence Limits of Percentile from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96)
30
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 7 – Box Plot of the Lower and Upper Bounds of Specification Interval from 105 Monte Carlo Simulations on the Standard Normal Distribution with a Targeted Coverage of 95% using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and 90% Confidence Limits of Percentiles (CL of Perc.)
31
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 8 – Box Plot of the Lower and Upper Bounds of Specification from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and 90% Confidence Limits of Percentiles (CL of Perc.)
32
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 9 – Box Plots of the Inflation Rate of Consumer’s Risk from 105 Monte Carlo Simulations Generated on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and Confidence Limits of Percentiles (CL of Perc.)
33
cr ip us M an ed pt ce Ac
Downloaded by [Tulane University] at 19:32 21 January 2015
t
Figure 10 – Box Plot of the Inflation Rate of Manufacturer’s Risk from 105 Monte Carlo Simulations on Standard Normal Distribution with a Targeted Interval of (-1.96, 1.96) using Reference Interval (RI), (97.5%, 95%) One-sided Tolerance Interval (TI) and Confidence Limits of Percentiles (CL of Perc.)
34
