J Forensic Sci, January 2016, Vol. 61, No. 1 doi: 10.1111/1556-4029.12899 Available online at: onlinelibrary.wiley.com

TECHNICAL NOTE CRIMINALISTICS

Kayleigh E. Rowan,1 M.S.; Genevieve A. Wellner,1 M.S.; and Catherine M. Grgicak,1 Ph.D.

Exploring the Impacts of Ordinary Laboratory Alterations During Forensic DNA Processing on Peak Height Variation, Thresholds, and Probability of Dropout*,†

ABSTRACT: Impacts of validation design on DNA signal were explored, and the level of variation introduced by injection, capillary

changes, amplification, and kit lot was surveyed by examining a set of replicate samples ranging in mass from 0.25 to 0.008 ng. The variations in peak height, heterozygous balance, dropout probabilities, and baseline noise were compared using common statistical techniques. Data indicate that amplification is the source of the majority of the variation observed in the peak heights, followed by capillary lots. The use of different amplification kit lots did not introduce variability into the peak heights, heterozygous balance, dropout, or baseline. Thus, if data from case samples run over a significant time period are not available during validation, the validation must be designed to, at a minimum, include the amplification of multiple samples of varying quantity, with known genotype, amplified and run over an extended period of time using multiple pipettes and capillaries.

KEYWORDS: forensic sciences, forensic validation, forensic DNA interpretation, reproducibility, error, repeatability

Low-template mixture interpretation is a challenge in forensic casework (1). In recent years, a significant amount of effort has been devoted to exploring issues associated with complex mixture interpretation. Most notable is the development of the likelihood ratio (LR) approach (2–9), which may take into account a given number of contributors (10–12), probability of dropout (Pr (D)) (13,14), probability of drop-in (Pr(DI)), peak height and stutter proportion (15). Typically, the Pr(D), noise, and stutter is characterized via validation and measured with respect to some value, such as average peak height or mass of input DNA. As proper interpretation relies on validation and calibration, it is of interest to identify and characterize sources of peak height uncertainty that result from natural changes of laboratory equipment or procedures. It is also necessary to confirm that these laboratory alterations do not significantly impact the thresholds or probabilities utilized during interpretation. There are a number of laboratory procedures and/or chemistries which may introduce variability in the DNA signal. For example, it has been suggested that differences in amplification kit batches/ lots may introduce additional peak height variability (16). Although PCR efficiencies are known to decrease with cycle 1 Program in Biomedical Forensic Sciences, Boston University School of Medicine, 72 E. Concord St, Rm R806, Boston, MA 02118. *Supported in part by 2011-DN-BX-K558 and 2012-DN-BX-K050 awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. † The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the author(s) and do not reflect those of the Department of Justice. Received 21 Mar. 2014; and in revised form 12 Dec. 2014; accepted 2 Jan. 2015.

© 2015 American Academy of Forensic Sciences

number, kinetic studies have suggested primer concentration effects are not observed until the amplicon concentration approaches that of the primer concentration (17). Therefore, if the primer concentrations are in excess of the final amplicon concentration, it may not be the case that moderate changes in primer levels between kit batches would have substantial effects on DNA signal. Similarly, polymerase deactivation effects, which become more pronounced at the end stages of thermal cycling, are not expected to affect the final results as long as the active polymerase concentration is above a critical value. As the exact concentrations of the STR amplicons, primers, and active enzyme are unknown during forensic DNA processing, it is difficult to determine whether between-batch variability in peak heights should be the expectation. If between-batch variability is present, then calibration, research, and validation efforts should include samples amplified with multiple kit/polymerase lots. Other natural laboratory modifications which may affect signal consistency include, but are not limited to, changes in capillary lots, injection variation, amplification variation, and quantification consistency. If a significant amount of variability is introduced at every step of the DNA process, then efforts which aim to characterize signal and establish models for casework purposes should be based on studies which mimic the natural changes present during routine casework processing. This may include amplifying multiple samples, from repeated quantifications, using multiple kit lots, capillary lots, injections, etc. Alternatively, if the signal errors associated with some laboratory processes are insignificant to those of the PCR, then it may be possible to establish validation/ calibration protocols which minimize the burden of creating elaborate validation schemes, while still providing accurate determinations of signal and its variability. 177

178

JOURNAL OF FORENSIC SCIENCES

This work represents a study which explores the impacts four laboratory changes have on the variability in DNA signal. The laboratory processes tested were those of injection, capillary, amplification, and kit changes. To test injection variation, the same sample preparation was injected four times on one capillary. An additional sample preparation was also injected four times, but in this instance, the capillary lot was changed before each injection. Amplification variation was measured by amplifying the samples in quadruplicate utilizing one kit lot, while variability introduced by kit lot was explored by then amplifying these samples three additional times with three different kit lots. The effects on peak height, allele dropout, peak height ratios, and baseline noise are presented.

Materials and Methods General Laboratory Protocols All aspects of the study were conducted in compliance with ethical standards set out by the Institutional Review Board. All reagents were purchased from Sigma-Aldrich (Sigma-Aldrich, St. Louis MO) unless stated otherwise. Three single-source samples of known genotype were obtained by extracting DNA using phenol/chloroform extraction and alcohol precipitation. The DNA was dissolved in 50 lL of TE buffer (10 mM Tris, 0.1 mM EDTA, pH 8.0) at 56°C and stored at
20°C until use. DNA quantification was accomplished by Quantifilerâ Duo DNA Quantification Kit (Applied Biosystems, Foster City, CA) according to the manufacturer’s recommended thermal cycling protocol using a 7500 real-time PCR system (RT-PCR, Applied Biosystems) (18) and an external calibration curve; that is, the slope and y-intercept of the external calibrator were not determined on a per-run basis, but rather were determined at least annually by running a set of qPCR standards (50– 0.023 ng/lL) after instrument calibration/maintenance (19). Each of the specimens was amplified in quadruplicate at each of the six target quantities (0.25, 0.125, 0.063, 0.032, 0.016, and 0.008 ng) using Applied Biosystems’ AmpF‘STRâ Identifilerâ PCR Plus Amplification Kit according to the manufacturer’s recommended 29-cycle protocol (20) on an Applied Biosystems’ GeneAmpâ PCR System 9700 using 9600 emulation mode. Positive and negative amplification controls were run and showed expected results (data not shown). Fragment separation was achieved using an Applied Biosystems’ 3130 Genetic Analyzer and a mixture containing appropriate amounts of HiDi (highly deionized) formamide (8.7 lL/sample) (Applied Biosystems, Foster City CA) and GeneScanTM-600 LIZTM Size Standard (0.3 lL/sample) (Applied Biosystems, Foster City CA). A volume of 9 lL of that mixture and 1 lL of sample, negative, or ladder was added to the appropriate wells. The samples were placed on a heating block at 95°C for 3 min and snap-cooled at
20°C for 3 min. The samples were injected for 10 sec using a 3 kV injection potential, and the run was programmed according to the manufacturer’s recommended protocol. The samples were analyzed using GeneMapper IDX v 1.1.1 (Applied Biosystems, Foster City, CA) and an RFU threshold of 1. Validation Design Comparison To test the effect of different capillaries on signal, a single sample preparation was run on the ABI 3130 four times, utilizing a different capillary lot for each injection (Validation 1). To test the effects injections had on signal, a single sample preparation was

subjected to 4, 10 sec injections (Validation 2) on one capillary. To test the process variation associated with the amplification and sample preparation process, the samples were amplified with one kit lot, and each of the amplified products was prepared for fragment analysis and injected once (Validation 3). Lastly, the samples were amplified using the same amplification protocol; however, each amplification was carried out using different kit lots (Validation 4). This resulted in a total of 72 profiles for each validation. Figure 1 shows a schematic of the capillary number, kit number, amplifications, and injections utilized during the course of this study, and the relationships between data sets. Once the samples were analyzed in GeneMapperâ ID-X, the data were exported and all nonallele peaks were removed. The data were sorted by type, according to the validation being studied, sample number, and locus. The between-replicate peak height variability was assessed by determining the variance in peak height for each allele a, for each validation v, which we term var(Ha,v). Var(Ha,v) was then plotted against the average peak height obtained for each allele (APHa,v). We utilize the following equation varðHa;v Þ 2 ¼ Ca;v APHa;v

ð1Þ

2 to determine Ca;v . Consequently, for laboratory processes with 2 . Further, multiple regreslow variability, we expect a small Ca;v sion was performed in MS Excel to identify the laboratory pro2 , as per cess most predictive for changes in Ca;v 2 Ca;v ¼ b0 þ b1 ðCÞ þ b3 ðAÞ þ b4 ðKÞ

ð2Þ

where C is a categorical variable indicating whether different capillaries were utilized. A value of 0 was assigned if the same capillary was used between injections, and a value of 1 was assigned when different capillaries were utilized. Similarly, A and K were assigned values of 1 when the group contained

Cap #1 Cap #2 Kit #1

Cap #3 Inj #1 Cap #4 Validation 1 Cap #5

DNA Extracts (0.25 to 0.008 ng)

Kit #2

Amp #1

Cap #5

Amp #2

Cap #5

Amp #3

Cap #5

Amp #4

Inj #2 Inj#3

Inj #4 Validation 2

Cap #5 Validation 3

Kit #3

Cap #5

Kit #4

Cap #5

Underlined = Validation 4

FIG. 1––A schematic of the laboratory processes used to obtain data from the four validation experiments. Inj, injection; Cap, capillary; Amp, amplification. The dashed line between capillary #5 and injections #1–4 signifies that the data originating from validations 2 and 1 stem from different 3130 sample preparations/plates.

ROWAN ET AL.

multiple amplifications utilizing one kit and multiple amplifications utilizing multiple kits, respectively. The variation in peak heights was also monitored for each allele by calculating the moving range (mR-) of the peak height for each successive run (r), where the moving range is defined as (21,22), mR ¼ jHa;v;r
Ha;v;r
1 j

ð3Þ

The lower control limit (LCL) of 0 and the upper control limit (UCL) of 3.267 times the average mR were determined. The mR- control regions were used to assess whether there were measureable changes in peak variability between validation designs. Within locus reproducibility was also examined and summarized by the heterozygous balance within a locus, Hbl;v ¼

HHMV;l;v HLMV;l;v

ð4Þ

where HHMW,l,v is the peak height of the high molecular weight allele at locus l, and HHMW,l,v is the height of the low molecular weight allele at that locus. The Hb,l,v was plotted against APHl,v. Peak height balance has been previously studied, and it has been suggested that variability in Hb proportionally decreases with APH as per (15,23,24) varðHbl;v Þ ¼

r2l;v APHl;v

.

EXPLORING IMPACT OF DNA LAB ALTERATIONS

179

sis, is the variation originating from instrument effects. Gross instrument changes, such as the replacement of a CCD or laser, may be so significant as to require additional calibration or validation. However, these major changes to equipment are not part of the day-to-day natural variation observed in the laboratory. Instead, routine changes to equipment, such as the replacement of a spent capillary, may also have an effect on DNA signal. For purposes of validation and calibration, laboratories may be interested in understanding the extent to which laboratory alterations impact signal variability. Figure 2a shows the FGA locus of one of the samples injected four times, while Figure 2b shows the electropherograms obtained when the extract is amplified four times using 4 kit lots. This figure shows that changes in peak heights are clearly visible between the groups, suggesting that, as expected, reinjection of a single sample preparation is not a viable validation design as it does not capture the expected long-term variability in signal. By contrast, it is less obvious whether there is a change in baseline noise, or whether we would expect capillary changes to introduce measureable changes in signal. Further, it is unclear whether validation schemes need to incorporate various amplification kit lots, or whether a single kit lot is sufficient. Figure 3 shows the var(Ha,v) plotted against APHa,v for each allele and validations 1–4. It demonstrates that the var(Ha,v)

ð5Þ

Multiple regression of r2l;v against categorical variables C, A, and K, was performed to evaluate how laboratory changes combine to predict the variance of Hbl,v. Further, the dropout probability as a function of present-allele height was assessed for each data set using the STR validator program designed by Hansson et al. (25). The maximum peak height for which the sister allele dropped out was also identified for Validations 1–4. The noise peak heights from the samples were examined by sorting by validation design and then by color channel. All peaks originating from spectral overlap (i.e., pull-up) or -A were manually removed. In order for peaks to be classified as pull-up, the peak in question had to be in the same position (0.3 bases) as the allelic peak in another color channel and have a peak height of 5% or less of the allelic peak. Further, if a peak that fell between two adjacent allelic peaks in a different color had a “plateau-like” shape, then the peak in question was classified as complex pull-up and was removed. A peak was determined to be -A if it was one base shorter (0.3 bases) than an allelic peak. There were no height restrictions for the -A artifact. Histograms for each laboratory alteration (i.e., capillary, injection, kit, and amplification) were created in IGOR PRO v. 6.2 (Wavemetrics, Portland, OR) using a manual binning method such that each bin was one RFU and the bins extended from 1 RFU up to the maximum observed noise peak height. The histograms were then fitted with a log-normal curve (26).

a) 104

94

100 97

99 97

94 97

b)

165 104

94

18

140

34

38

23

Results Uncertainty in DNA signal can come from a variety of sources. Some typical sources include sample, storage, and operator effects. Another main source of variation, and one that may be of particular consequence to forensic DNA analy-

FIG. 2––Representative electropherograms for the FGA locus from a sample which was amplified using 0.031 ng of DNA, as per qPCR results and (a) injected four times on one capillary, (b) amplified four times using four kit lots. The height of each peak is listed above the peak.

180

JOURNAL OF FORENSIC SCIENCES

10000

10000

1000

1000

var(Hi)

100000

var (Hi)

100000

100 10 1

100 10 1

y = 3.5x - 58.4

y = 0.7x - 51.4

0.1

0.1 0

500

1000

1500

0

10000

10000

1000

1000

var(Hi)

100000

var(Hi)

100000

100 10 1

500

1000

1500

100 10 1

y = 42.5x - 525.7 0.1

y = 47.9x - 2678.3

0.1 0

500

1000

1500

0

APH

500

APH

1000

1500

FIG. 3––The variance of the peak height (H) of allele a, between replications for validations 1, 2, 3, and 4. The slope and y-intercept from a least squares regression are also presented.

increases with APHa,v and does so in a proportional manner. It should be noted that the fit was not used to make any conclusion regarding a model for peak height variance, but rather to allow for a comparison between the validation schemes. Therefore, we argue that although direct proportionality may be unlikely to be exactly the case, it represents the data to a reasonable degree and thus can be utilized to assess the impact laboratory changes have on signal. The slopes (shown in Fig. 3) are demonstrative 2 (Eq. 1) obtained for all alleles. This value can of the typical Ca;v be taken to be a measure of reproducibility between replicates and is an indication of the variability expected between valida2 (henceforth referred to as tion schemes. A boxplot of the Ca;v 2 c ) values for Validations 1–4 is shown in Fig. 4 and the med-

250

ian, maximum, and minimum values are provided in Table 1. Further, the results of the multiple regression (Eq. 2) are shown in Table 2. These results demonstrate that, as expected, reinjection of the same sample preparation had minimal effect on peak variance. The median c2 increased by a factor of 10 when new capillaries were introduced into the processing scheme, suggesting a small, but noticeable increase in the peak variance. When comparing the c2 values between Validations 1 and 2 using a ttest, a p-value of 2e-31 suggested peak height variability increased significantly with capillary changes. Although changing of capillaries has some effect on peak height variability, it is dwarfed by the level of imprecision caused by amplification (Table 2). Similarly, when c2 of validations 3 and 4 is com-

2 TABLE 1––Median, maximum, and minimum values for Ca;v , for validations 1, 2, 3, and 4.

200

c2

150

100

Validation

Laboratory Modification

Median

Maximum

Minimum

1 2 3 4

Capillary lot Injection Amplification Kit lot

1 0.1 25 16

23 6 233 180

0 0 0 0

2 TABLE 2––Coefficients and p-values after regression of Ca;v , against variables C, A, and K, which categorically indicate whether different capillary lots, amplifications, or kits were utilized during validation.

50

0

Validaon 1

Validaon 2

Validaon 3

Validaon 4

2 FIG. 4––A boxplot of the Ca;v parameter of Eq. 2 values obtained for each allele for validations 1–4. The box shows the median and the 1st and 3rd quartile. The whiskers represent 1.5 times the interquartile range. Outlier points are also indicated.

Validation (v) 1 3 4

Laboratory Modification (Variable) Capillary lot (C) Amplification (A) Kit lot (K)

Intercept (p-Value) 0.31 (0.94)

bv

Error bv

p-Value

1.8 33.4
8.0

1.3 1.4 1.3

0.18 1e-117 1e-09

32 12 137 189 20 20 148 91 13 4 194 50 5 4 72 32 3 2 59 34 3 3 26 19 (145) (50) (464) (331) (72) (45) (357) (303) (47) (9) (464) (167) (7) (4) (334) (94) (14) (14) (162) (83) (9) (11) (70) (95) 44 15 142 101 22 13 109 93 14 3 142 51 2 1 102 29 4 4 50 25 3 3 22 29 (160) (39) (332) (412) (113) (21) (381) (280) (28) (9) (472) (191) (18) (4) (254) (56) (13) (4) (170) (91) (7) (8) (64) (75) 49 12 102 126 35 6 117 86 9 3 145 59 5 2 78 17 4 1 52 28 2 3 20 23 (120) (27) (406) (458) (72) (88) (466) (396) (60) (10) (446) (209) (23) (20) (217) (94) (12) (7) (166) (87) (13) (13) (90) (81) 37 8 124 140 22 27 143 121 19 3 137 64 7 6 66 29 4 2 51 27 4 4 28 25 (253) (23) (758) (584) (163) (71) (332) (335) (54) (14) (547) (211) (36) (12) (280) (134) (10) (5) (317) (85) (12) (9) (85) (119) 77 7 232 179 50 22 102 102 17 4 168 65 11 4 86 41 3 2 97 26 4 3 26 36 (199) (43) (643) (764) (92) (55) (525) (589) (41) (15) (931) (193) (24) (19) (286) (162) (8) (5) (223) (137) (19) (9) (94) (110) 61 13 197 234 28 17 161 180 13 5 285 59 7 6 88 49 3 2 68 42 6 3 29 34 (153) (44) (772) (445) (79) (40) (463) (493) (67) (15) (462) (247) (34) (10) (344) (195) (11) (10) (180) (145) (10) (6) (140) (114) 47 14 236 136 24 12 142 151 21 5 141 76 10 3 105 60 3 3 55 44 3 2 43 35 (173) (46) (434) (274) (100) (70) (394) (439) (45) (14) (430) (142) (23) (9) (265) (154) (13) (3) (197) (112) (9) (8) (93) (101) 53 14 133 84 30 21 121 134 14 4 132 43 7 3 81 47 4 1 60 34 3 2 28 31 (188) (50) (872) (556) (105) (118) (526) (380) (56) (13) (861) (296) (17) (6) (330) (204) (17) (4) (184) (119) (13) (15) (100) (117) 58 15 267 170 32 36 161 116 17 4 264 87 5 2 101 62 5 1 56 36 4 4 31 36 (210) (43) (955) (966) (89) (62) (478) (599) (93) (14) (976) (343) (12) (6) (542) (252) (15) (8) (334) (156) (5) (4) (86) (106) 64 13 292 296 27 19 146 183 28 4 299 105 4 2 166 77 5 3 102 48 2 1 26 33 (216) (53) (773) (851) (104) (52) (517) (354) (50) (10) (801) (222) (39) (11) (317) (205) (12) (7) (264) (163) (10) (7) (136) (128) 66 16 237 261 32 16 158 108 15 3 245 68 12 3 97 63 4 2 81 50 3 2 42 39 (262) (54) (655) (701) (110) (46) (580) (492) (56) (15) (874) (195) (17) (9) (333) (159) (5) (3) (358) (162) (3) (8) (92) (113) 80 17 201 215 34 14 178 151 17 5 268 60 5 3 102 49 2 1 110 50 1 2 28 35 (207) (51) (359) (476) (114) (65) (405) (207) (27) (6) (527) (117) (18) (4) (219) (122) (6) (1) (209) (114) (1) (3) (65) (50) 64 16 110 146 35 20 124 63 8 2 161 36 6 1 67 37 2 0 64 35 0 1 20 15 (128) (32) (408) (492) (75) (51) (345) (394) (35) (9) (309) (167) (13) (3) (192) (132) (9) (4) (150) (61) (7) (1) (75) (81) 39 10 125 148 23 16 106 121 11 3 95 51 4 1 59 40 3 1 46 19 2 0 23 25 (152) (37) (529) (357) (83) (59) (504) (284) (49) (9) (878) (183) (22) (6) (218) (168) (7) (15) (292) (87) (9) (10) (73) (71) 47 11 162 109 25 18 154 87 15 3 269 56 7 2 67 51 2 5 89 26 3 3 22 22 (195) (58) (677) (749) (69) (50) (412) (529) (53) (11) (765) (423) (25) (9) (462) (180) (17) (7) (222) (107) (14) (5) (75) (94) 0.008

0.016

0.031

0.063

0.125

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 0.25

60 17 207 229 21 15 126 162 16 3 234 129 8 3 142 55 5 2 68 33 4 2 23 29

D5 AM D18 TPOX vWA D19 D2 D16 D13 TH0 D3 CSF D7 D21 Validation No.

D8

181

(106) (39) (448) (618) (66) (66) (482) (299) (42) (12) (632) (162) (17) (14) (236) (105) (11) (6) (192) (110) (9) (9) (85) (60)

EXPLORING IMPACT OF DNA LAB ALTERATIONS

Mass (ng)

mR(UCL)

TABLE 3––Mean moving ranges and UCL (upper control limits) for validations 1, 2, 3, and 4. Values were rounded to the nearest RFU value.

pared, kit-to-kit variability was smaller than that produced when the samples were amplified with one kit. In fact, the regression output suggested that utilizing different kit lots decreases the variability in peak height. More likely, these results are indicative of the fact that kit lots do not introduce additional variability into the system. Analysis into whether changing capillaries or kits introduce an observable change in peak variability can be further examined by utilizing mR- control charts. The mR- charts, traditionally used in process engineering, are used to assess natural process limits and to detect whether there has been an actual process change or whether a special cause has occurred. They can also be used to predict the expected range of outcomes from a process (21, 22, 27) and to compare variability between processes. Table 3 summarizes the mean and UCL for the moving ranges for each locus at every target for each validation type. The LCL is 0 for all cases. When comparing the control ranges of validations 2 and 1, Validation 1 had larger control regions for 86 of 96 loci tested at the various targets. In a similar vein, when examining the mR individually, 37% of the moving ranges from Validation 1 were above the UCL established with data obtained during Validation 2, signifying a measurable process change had occurred when capillaries were changed. In contrast, only 1% of the mR’s from Validation 4 exceed the control regions established during Validation 3. To examine within locus balance, the peak height balance, Hbl,v, (Eq. 4) was evaluated for each heterozygous pair. Figure 5 exhibits the log(Hbl,v) versus APHl,v, which shows that the variability in peak balance increases with a decrease in APH. The funnel shape seen in Fig. 5 is consistent with previous work and has been observed in studies which have examined heterozygous balance of samples amplified with IdentifilerTM, MiniFilerTM, and NGM SElectTM amplification kits (15,28,29). These works have proposed that the variance in heterozygous balance is indirectly proportional to APH as per Eq. 5. Thus, r2 is a constant which can be utilized to assess the level of variability between capillary, kit, amplification, and injection. Table 4 provides the summary statistics, while Table 5 shows the regression output of the r2 values calculated for each locus within a validation. These data indicate that amplification is the one laboratory process that impacts the variance in heterozygous balance. Results of a t-test which were obtained when comparing r2 from different capillaries versus reinjection indicates that changing capillaries does not significantly impact the heterozygous balance between samples (p-value = 0.31). Lastly, the probability of dropout as a function of average height was determined and compared between data sets (Fig. 5). The shaded section marks the 95% confidence interval of the dropout probability. The points at P = 0 represent heterozygotes where the partner allele had not dropped out, while the points at P = 1 represent the loci where one allele from a heterozygous pair had not been detected. The APH at which the probability of dropout is 0.05 was determined to be 169, 177, 182, and 186 for Validations 1–4, respectively, suggesting that changes in capillary and kit lot did not significantly affect the levels of dropout, where only minor changes in the dropout profile were observed between validations (Fig. 6). The noise height distributions of the four types of laboratory validations were compared. Figure 7 shows the noise peak height distributions obtained from samples validated using one or four capillaries and one or four amplification kit lots. The noise distribution for the four kits is offset from the other distributions in the blue and green channels, and the combined

.

FGA

ROWAN ET AL.

JOURNAL OF FORENSIC SCIENCES 2

2

1.5

1.5

1

1

0.5

log(Hbl,2)

log(Hbl,1)

182

0 -0.5

0.5 0 -0.5

-1

-1

-1.5

-1.5

-2

-2 500

1000

1500

2

2

1.5

1.5

1

1

0.5

log(Hbl,4)

log(Hbl,3)

0

0 -0.5

500

0

500

1000

1500

1000

1500

0.5 0 -0.5

-1

-1

-1.5

-1.5

-2

0

-2 0

500

APH

1000

1500

APH

FIG. 5––The logarithm of the heterozygous balance for validations 1, 2, 3, and 4 versus average peak height.

TABLE 4––Median, maximum, and minimum values for r2l;v , for validations 1, 2, 3, and 4.

Validation (v)

Laboratory Modification

Median

Maximum

Minimum

1 2 3 4

Capillary lot Injection Amplification Kit lot

0.08 0.08 47 33

62 25 28,179 3864

0.001 0 0.05 0.14

TABLE 5––Coefficients and p-values of r2l;v against variables C, A, and K, which are categorical variables indicating whether different capillary lots, amplifications, or kits were utilized.

Validation 1 3 4

Laboratory Modification (Variable) Capillary lot (C) Amplification (A) Kit lot (K)

Intercept (p-Value) 6 (0.94)

bv

Error bv

p-Value

79 307
128

127 122 118

0.53 0.01 0.28

capillaries distribution is offset from the combined distributions of the other sample preparation variables in the yellow and red channels. The largest difference in the means of the noise peaks between each validation set is 2 and the spreads nearly overlap for every color channel, suggesting only minor changes in baseline noise occurs due to the introduction of various capillary or kit lots. Discussion As one of the primary challenges from the perspective of laboratories is the energy, cost, and time required to complete an extensive and representative validation study, it is of importance

to determine which laboratory factors or equipment must be modified during validation in order to obtain intermediate measures of reproducibility and ensure whether the interpretation guidelines, thresholds, and probabilities are applicable to evidence samples. There are several documents and publications that provide guidelines or recommendations regarding internal validations. However, most lack detailed information regarding the way in which validation is to be completed in order to ensure the data obtained are representative of the variability expected in the laboratory. For example, SWGDAM recommends that known and nonprobative evidence samples, or mock case samples, be run to determine the sensitivity, stochastic effects, precision, and accuracy, as well as to test mixtures (30). EURACHEM also provides guidelines and states that “studies should. . .be conducted to provide a realistic survey of the number and range of effects operating during normal use of the method, as well as covering the concentration ranges and sample types,” as well as “ensure that the larger effects are varied where possible” (31). Further, with the advent of complex, low-template mixture analysis, and the need to define thresholds and or probabilities that can be applied to evidentiary samples, a detailed understanding of the effect on the uncertainty in the bio-analytical measurement is necessary. The impact of four laboratory alterations on the DNA signal was tested and compared by examining effects on the variability in peak heights, heterozygous balance, baseline noise, and allelic dropout. Changing the capillary lot had a small, but consistent effect on peak heights, suggesting multiple capillary lots should be used when assessing peak height variability. Multiple amplifications are also required to determine the natural variability that is typical of a laboratory. Interestingly, the variability introduced by utilizing multiple kit lots did not have an effect on either the peak height or the heterozygous balances. Like the peak height

ROWAN ET AL.

1.00

1.00

a)

EXPLORING IMPACT OF DNA LAB ALTERATIONS

Drop-out probability, P(D)

0.50

b)

0.50

0.25

0.25

P(dropout/T=177)=0.05

P(dropout/T=169)=0.05

P(dropout>0.05/T=208)0.05/T=198)0.05/T=219)0.05/T=225)2 RFU. Further, the baseline noise distribution for these low-template samples was below the commonly employed analytical thresholds of 50–100 RFU. Conclusion This study sought to identify which, and to what extent, ordinary laboratory changes impacted the variability in the DNA signal. Forensic DNA laboratory processing requires multiple steps, which include ordinary equipment changes. This was accomplished by examining the influence of four validation designs on the peak heights obtained. Data indicate that the uncertainty associated with the amplification has the greatest effect on the reproducibility of the RFU signal, followed by capillary lot, then kit lot and injection. Changes in baseline noise were also examined and results show that this signal was not impacted by kit lot, amplification, capillary lot, or injection. Therefore, if data from case samples of known genotype run over a significant time period are not available during validation, careful consideration as to the methods by which the

validation data set is created is required. The validation must be designed such that the variability in peak heights and dropout rates is representative of casework. One way to make certain a representative data set is generated is to ensure intermediate measurements of reproducibility are incorporated into the validation. This would, at a minimum, include the amplification of multiple samples of varying quantities, with known genotype, amplified, and run over an extended period of time using multiple pipettes and capillaries (31). According to the data presented herein, the inclusion of multiple kit lots may not be necessary to capture the expected variation. It is emphasized that a larger study with large numbers of kit lots, manufactured over long-time periods, would be required to fully demonstrate the truth of the statement above. The impact of only four typical laboratory processes was evaluated. Assessments as to the consequences of multiple instruments, polymer lots, pipettes, etc. may also be of interest to laboratories. Methods by which this may be accomplished were presented. Further, we present control charts as a means to monitor the performance of the forensic laboratory process. Acknowledgments The authors thank Desmond Lun and the anonymous reviewers for helpful suggestions.

184

JOURNAL OF FORENSIC SCIENCES

a)

b)

c)

d)

FIG. 7––The distribution of the noise peak height for the validation studies; capillary lot ( ), injection (□), amplification (○), and kit lots (●), for the (a) blue, (b) green, (c) yellow, and (d) red channels.

References 1. Word C.J. Mixture Interpretation: Why is it Sometimes So Hard? Promega Corporation Web site. http://www.promega.com/resources/profilesin-dna/2011/mixture-interpretation-why-is-it-sometimes-so-hard/ Updated 2011. Accessed August 05, 2015. 2. Brenner C, Fimmers R, Baur M. Likelihood ratios for mixed stains when the number of donors cannot be agreed. Int J Legal Med 1996;109 (4):218–9. 3. Buckleton J, Curran J. A discussion of the merits of random man not excluded and likelihood ratios. Forensic Sci Int Genet 2008;2:343–8. 4. Lohmueller KE, Rudin N. Calculating the weight of evidence in lowtemplate forensic DNA casework. J Forensic Sci 2013;58(S1):S243–9. 5. Gill P, Gusmao L, Haned H, Mayr WR, Morling N, Parson W, et al. DNA commission of the International Society of Forensic Genetics: recommendations on the evaluation of STR typing results that may include drop-out and/or drop-in using probabilistic methods. Forensic Sci Int Genet 2012;6:679–88. 6. Perlin MW, Szabady B. Linear mixture analysis: a mathematical approach to resolving mixed DNA samples. J Forensic Sci 2001;46 (6):1372–8. 7. Ballantyne J, Hanson EK, Perlin MW. DNA mixture genotyping by probabilistic computer interpretation of binomially-sampled laser captured cell populations: combining quantitative data for greater identification information. Sci Justice 2012;53(2):103–14. 8. Perlin MW, Sinelnikov A. An information gap in DNA evidence interpretation. PLoS ONE 2009;4(12):e8327.

9. Evett IW, Weir BS. Interpreting DNA evidence: statistical genetics for forensic scientists. Sunderland, MA: Sinauer Associates, 1998. 10. Egeland T, Dalen I, Mostad PF. Estimating the number of contributors to a DNA profile. Int J Legal Med 2003;117:271–5. 11. Haned H, Pene L, Lobry JR, Dufour A, Pontier D. Estimating the number of contributors to forensic DNA mixtures: does maximum likelihood perform better than maximum allele count? J Forensic Sci 2011;56(1): 23–8. 12. Biedermann A, Bozza S, Konis K, Taroni F. Inference about the number of contributors to a DNA mixture: comparative analyses of a Bayesian network approach and the maximum allele count. Forensic Sci Int Genet 2012;6:689–96. 13. Tvedebrink T, Eriksen PS, Mogensen HS, Morling N. Estimating the probability of allelic drop-out of STR alleles in forensic genetics. Forensic Sci Int Genet 2009;3:222–6. 14. Haned H, Egeland T, Pontier D, Pene L, Gill P. Estimating drop-out probabilities in forensic DNA samples: a simulation approach to evaluate different models. Forensic Sci Int Genet 2011;5:525–31. 15. Bright J-A, Taylor D, Curran JM, Buckleton JS. Developing allelic and stutter peak height models for a continuous method of DNA interpretation. Forensic Sci Int Genet 2013;7:296–304. 16. Puch-Solis R, Rodgers L, Mazumbder A, Pope S, Evett I, Curran J, et al. Evaluating forensic DNA profiles using peak heights, allowing for multiple donors, allelic dropout and stutters. Forensic Sci Int Genet 2013;7:555–63. 17. Gevertz JL, Dunn SM, Roth CM. Mathematical model of real-time PCR kinetics. Biotechnol Bioeng 2005;92(3):346–55.

ROWAN ET AL. 18. Life Technologies Corporation. Quantifiler(R) Duo DNA Quantification Kit user guide. Foster City, CA: Life Technologies Corporation, 2008. 19. Grgicak CM, Urban ZM, Cotton RW. Investigation of reproducibility and error associated with qPCR methods using Quantifiler Duo DNA Quantification Kit. J Forensic Sci 2010;55(5):1331–9. 20. Life Technologies Corporation. AmpFlSTR(R) Identifiler(R) Plus PCR Amplification Kit user’s manual, 1st edn. Applied Biosystems. Foster City, CA: Life Technologies Corporation, 2006. 21. Batson RG, Jeong Y, Fonseca DJ, Ray PS. Control charts for monitoring field failure data. Qual and Reliab Eng Int 2006;22:733–55. 22. Trip A, Wieringa JE. Individuals charts and additional tests for changes in spread. Qual and Reliab Eng Int 2006;22:239–49. 23. Kelly H, Bright J-A, Curran JM, Buckleton J. Modelling heterozygote balance in forensic DNA profiles. Forensic Sci Int Genet 2012;6:729–34. 24. Gilder JR, Inman K, Shields W, Krane DE. Magnitude-dependent variation in peak height balance at heterozygous STR loci. Int J Legal Med 2011;125:87–94. 25. Hansson O, Gill P, Egeland T. STR-validator: an open source platform for validation and process control. Forensic Sci Int Genet 2014;13:154– 66. 26. Bregu J, Conklin D, Coronado E, Terrill M, Cotton RW, Grgicak CM. Analytical thresholds and sensitivity: establishing RFU thresholds for forensic DNA analysis. J Forensic Sci 2012;58:120–9.

.

EXPLORING IMPACT OF DNA LAB ALTERATIONS

185

27. Miller J, Miller J. Statistics for analytical chemistry, 3rd edn. Chichester: Ellis Horwood Limited, 1993. 28. Bright J-A, Turkington J, Buckleton J. Examination of the variability in mixed DNA profile parameters for the Identifiler multiplex. Forensic Sci Int Genet 2010;4:111–4. 29. Bright J-A, Huizing E, Melia L, Buckleton J. Determination of the variables affecting mixed MiniFilerTM DNA profiles. Forensic Sci Int Genet 2011;5:381–5. 30. SWGDAM. Validation guidelines for DNA analysis methods, 2012; http://swgdam.org/SWGDAM_Validation_Guidelines_APPROVED_Dec_2012.pdf (accessed October 2014). 31. EURACHEM/CITAC Guide CT4. Quantifying uncertainty in analytical measurement, 2nd edn, 2000; http://www.citac.cc/QUAM2000-1.pdf. Additional information and reprint requests: Catherine Grgicak, Ph.D. Biomedical Forensic Sciences Boston University School of Medicine 72 E. Concord Street, Room R806B Boston, MA 02118 E-mail: [email protected]