186

Validation of Hemostasis and Coagulation Assays: Recommendations and Guidelines Richard A. Marlar, PhD1,2

Jana N. Gausman, MT(ASCP)1

1 Department of Pathology and Laboratory Medicine, Oklahoma City

VA Medical Center, Oklahoma City, Oklahoma 2 Department of Pathology, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma

James W. Engel, MLT (ASCP)1

Address for correspondence Richard A. Marlar, PhD, Laboratory Services 113, Oklahoma City Veterans Administration Medical Center, 921 13th Street, Oklahoma City, OK 73104 (e-mail: [email protected]).

Abstract

Keywords

► ► ► ► ► ► ►

hemostasis assay coagulation assay test validation specificity sensitivity accuracy precision

The clinical hemostasis laboratory is a complex testing arena which employs numerous coagulation assays and spans several different test methodologies. Adding further complexity, these test results are expressed in a wide variety of unique units (concentration, activity, time, percentage, and ratio). Unfortunately, many of these reference values are derived from a local plasma pool or manufacturer’s standards, as there are few established international standards. These three main issues complicate the validation and performance of the coagulation testing. Before an assay can be introduced into clinical use, both analytical and clinical performance parameters must be validated or verified using the standard validation procedures of the laboratory. This article summarizes the initial evaluation and validation processes of the coagulation laboratory, which sometimes can be difficult concepts to implement. A standardized validation protocol is described in this article and, if used, will help to objectively evaluate the assay performance and determine if it meets acceptable laboratory criteria.

The clinical coagulation laboratory is a complex testing arena that does not fit well into the general molds of hematology (“counting” of particles—red blood cells or platelets) or chemistry with known concentrations of analytes (sodium charge and albumin mass). The hemostasis or coagulation assay inventory spans multiple tests methodologies (from clotting tests to chromogenic and immunologic assays, to specialized tests such as electrophoresis, aggregation, and radioactive-based tests) and results are expressed in a wide variety of units: time, percentage, units, concentration, activity, optical density (OD) units, ratio, and even visual interpretation. Many of these values are based on a local plasma pool or manufacturer’s calibrators, as there are few international standards available. These parameters complicate the development, validation, and performance of methods in the routine coagulation laboratory and those more complex methods used within the “special” coagulation laboratory.

As with most laboratory tests, coagulation tests rely on the concept of “postanalytical decision making” to differentiate patients with “normal” results from those which fall outside the norm and are therefore considered “disease.”1 This concept requires that each assay be accurate and precise, as clinical decisions are being made based on the interpretation of these results. Coagulation test interpretations must also be founded on “evidence-based medicine.” However, evidence-based medicine has only recently been introduced into the field of coagulation, as in the past many coagulation tests relied on tradition to determine the validity of methodology and testing.2,3 Another issue that plagues coagulation testing is the difficulty of obtaining the statistically “correct” number of samples for validation.4,5 This article will focus on the major aspects of the coagulation test validation process and acceptable limits of test validation.

published online February 4, 2014

Copyright © 2014 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.

Issue Theme Quality in Hemostasis and Thrombosis, Part III; Guest Editors, Emmanuel J. Favaloro, PhD, FFSc (RCPA), Giuseppe Lippi, MD, and Mario Plebani, MD.

DOI http://dx.doi.org/ 10.1055/s-0033-1364186. ISSN 0094-6176.

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

Semin Thromb Hemost 2014;40:186–194.

Coagulation Validation Concepts Coagulation tests are used as a pathophysiologic assessment of the presence of a “disease” in a patient or to monitor anticoagulant therapy.1,6 There are numerous tests to accomplish these evaluations starting with “global” or “screening” assays, then narrowing to “specialized” tests.7,8 For each test, there are different methodologies, different reagent manufacturers, and often a “mix and match” approach with different manufacturer’s reagents and instruments that must also be addressed.5,6 Despite these drawbacks, the overall goal of coagulation testing is still to produce accurate and reproducible results.5 Producing accurate results requires specific procedures to guarantee to the clinical provider that the results reflect the pathophysiology of the patient.1,2 To this end, a systematic validation must be undertaken. The level of validation/verification that must be undertaken for any test varies depending on the type of regulatory “approval” that exists for the assay in that geographical locality as a whole or for the various components of the assay (►Table 1).9 An assay need only be “verified” when it has been validated by the manufacturer and approved or cleared by a regulatory agency in that laboratory’s locality.

Verification Verification of a validated and approved assay only requires evidence that the assay perform to the standards presented by the manufacture and are within the stated parameters of the approving entity.9 In essence, this generally means that the laboratory has verified what is stated by the manufacturer in their product insert regarding key assay performances. On the contrary, modification of a validated assay method, such as using one manufacturer’s kit on another manufacturer’s instrument or laboratory developed tests (home-brew) or components, leads to much more stringent testing of the method (termed validation).6,9,10

Marlar et al.

The general and continuing assessment of “good laboratory practices” for clinical coagulation testing falls to accrediting agencies sanctioned by each country. For many coagulation tests, significant problems can be encountered, and may include differences in reagents (e.g., activated partial thromboplastin time, aPTT); generating results in different arbitrary units (e.g., factor VIII inhibitor); tests with multiple methodologies (1-stage, 2-stage, and chromogenic factor VIII assays); and test results based on visual interpretation (platelet aggregation or von Willebrand factor [VWF] multimers). However, before a new method can be introduced into clinical use, both the analytical and clinical performance must be validated under standard operating parameters.6 The use of a standardized validation protocol is essential for objectively evaluating a method’s performance.4,6 The parameters of the validation protocol must be established before performing the validation or verification study, to better define method limits in the clinical laboratory.6

Validation Procedure Validation is defined as the process of proving that a procedure, process, system, equipment, reagents, and method work singly and together as expected to achieve the intended result.4,6 The complete method validation protocol not only assesses the major characteristics of the method but also monitors the assay’s performance over time assuring the same characteristics as initially assigned. After the assay has been deemed valid and the performance characteristics established, an ongoing verification process must be established on a daily basis known as internal quality control (IQC) and also in the long term using IQC and external quality assessment (EQA).11 The validation procedure is used to ensure the generation of high-quality results to make the correct diagnosis. It also ensures the accurate reporting of results. The validation

Table 1 Category of assay reagent groups relative to validation process Assay category

Level of validation

Comments

Government-approved in vitro device (IVD) or assay

Verification

1. Proof that the assay works as the manufacturer specified 2. Reference ranges can be accepted from manufacturer, literature or be determined

Government-approved assay (IVD) with single component modified

Limited validation

1. Proof that substituted reagent works exactly as original reagent 2. Reference ranges can be accepted from manufacturer, literature or be determined

Analytic-specific reagent (ASR) assay

Full validation

1. Complete proof that assay works 2. Reference intervals should be determined

Research use only assay

Full validation

1. Complete proof that assay works 2. Reference intervals should be determined

Laboratory developed test (home-brew)

Full validation

1. Complete proof that assay works 2. Reference intervals should be determined

Changing methodology or manufacturer IVD assay

Verification

1. Comparison studies between the previous and new assay. 2. If not, must validate

Note: This table reflects practice and terminology in use in the United States; in different geographical localities, different terms may be used.

Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

187

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

Hemostasis Validation and Performance

Hemostasis Validation and Performance

Marlar et al.

procedure consists of two main parts, administrative and statistical. All aspects of the assay method must be validated. Nevertheless, the extent of validation varies depending on the known circumstances associated with the assay. To establish the validity and accuracy of the assay, the acceptance criteria must be established before the actual validation process. This is especially true of the statistical components including what statistical parameters must be analyzed, the number of samples to be tested, types of samples evaluated, and comparison with the previous assay (in-house or sent out), along with the level of statistical and clinical acceptability.6

After performing the validation protocol, data are analyzed using predetermined statistical criteria. If results are acceptable, then the conclusions are presented in the validation summary report. If the defined criteria established in the protocol are met or any variations justified, then the method can be considered valid. If the evaluation criteria are not met, then the assay method, the validation protocol, or the acceptance criteria must be reconsidered.4,6 The final validation report, along with all of the data, statistical analyses, and signatures and titles of all participants, are kept readily available for inspection. An example of a validation form is presented in ►Fig. 1A–C.

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

188

Fig. 1 Example of a validation summary form. The form is set up in a computer spreadsheet format. The form can be modified to meet the needs of each assay and requirements of different geographic laboratory locations: (A) Page 1; (B) Page 2; and (C) Page 3. QA, quality assurance; QC, quality control; QM, quality management; LIM, laboratory information manager. Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

Marlar et al.

Fig. 1 (Continued)

Administrative Guidelines The validation process for a new assay or significant changes to an existing assay requires administrative confirmation that the assay fulfills the needs of laboratory and end user. If the assay is already established with only minor changes or if the test is manufacturer validated and regulatory agency approved, then the administrative component can be abbreviated. Nevertheless, if it is a laboratory developed test, a new “research-based” test, or other nonapproved test, then more details are needed in the administrative validation steps. The individual ultimately responsible for test validation is the laboratory director. Most aspects can be delegated, but the laboratory director has the overall responsibility. The coagulation section director or supervisor is responsible for completing the administrative checklist (►Table 2 and ►Fig. 1A–C). In particular, the writing of the Standard Operating Procedure (SOP) document should be done before the assay is set up, but

can be modified after the test is validated to incorporate changes found in the validation process. Training and safety issues are always an important component for any test within the laboratory. The laboratory information manager must be responsible for providing the correct information in the computer (laboratory information system [LIS], hospital information system [HIS], and any middleware) (►Table 2). The quality assessment technologist must assign the test to an inventory log of procedures, the laboratory activity menu, and implementation of EQA programs (►Table 2). These administrative aspects, while they do not reflect test performance, are absolutely paramount to the accurate reporting of test results to the clinical provider.

Statistical Guidelines Before initiating any validation study, whether establishing a new assay or changing reagents or methodology, a wellSeminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

189

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

Hemostasis Validation and Performance

Hemostasis Validation and Performance

Marlar et al.

Fig. 1 (Continued)

planned statistical validation protocol must be developed using sound scientific and clinical criteria.4,6,12,13 The commercial company who supplies the assay may provide validation protocol outlines. The validation study must detail

statistical studies and define acceptance criteria.4,12 The protocol should be performed in a timely manner with the samples, standards, and calibrators. Much of the general information in a validation protocol is described in a variety of documents provided by accrediting and standards organizations.4 The purpose of test method validation is to ensure highquality data for accurate diagnosis of disease.6 The time invested in validating an analytical method on the front end will ultimately provide the necessary diagnostic advantages to the clinical provider in the long term. Validation will demonstrate that the procedure, method, and instrument are acceptable for the overall intended use of the assay. The assay validation incorporates the common statistical parameters into the protocol. The typical validation parameters are briefly discussed below using descriptions from formal definitions but slanted toward coagulation testing.4,13,14 These include accuracy, precision, specificity, linearity, limit of detection (LOD), limit of quantitation (LOQ), correlation, and robustness. A validation process for coagulation assays must be designed to ensure accurate results to support the diagnosis of patients with coagulation abnormalities. The samples, reagents, controls, calibrators, and instruments should be

Table 2 The basic components and responsibilities of a validation study for a new or modified coagulation assay Supervisor responsibilities Confirmation of no existing patents (laboratory developed tests) Written laboratory procedure (CLSI GP2-A5 format) Validation study: accuracy, precision, analytical specificity, interferences, limit of detection, limit of quantification, reportable range Quality control procedure and availability of QC reagents MSDS update sheets Training plan: training method, list of staff requiring training, competency assessment, new-employee training Prepare memo to announce availability of test Cost analysis of test and recommended test charge Laboratory information manager Establish “test definition” Add test to LIS and HIS files Verify report format is acceptable in all systems QA technologist Add document to document inventory log and assign number Add test to specimen collection manual Add test to activity menu Order proficiency testing materials Laboratory director Written approval of test establishment Summary statement with director signature and date of implementation Abbreviations: CLSI, Clinical and Laboratory Standards Institute; HIS, hospital information system; LIS, laboratory information system; MSDS, material safety data sheet; QA, quality assurance; QC, quality control. Note: Not all of the listed components may be needed for every assay. The laboratory director must decide which aspects are relevant. Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

190

carefully selected. Samples for validation must be collected, processed, and stored by established guidelines and identical to laboratory collection and storage procedures.7 The reagent lots and instruments must be those that will be used when the method is implemented.5 The establishment of statistical criteria must be stringent, using the same criteria found in the clinical chemistry section of the laboratory. Not all methodologies, however, can follow these exacting criteria. Only assay methodologies associated with “clinical chemistry”–based methods must be evaluated under the stringent criteria (immunoprecipitation, enzymelinked immunosorbent assay, chromogenic, chemiluminescence, and nephelometric).

Accuracy Accuracy is the closeness of agreement between test value and true value.13 In coagulation testing, this can be one of the most difficult—or even impossible—parameters to determine. In fact, the concept of “true value” does not apply to some coagulation assays (such as results reported as time values).5,7 On top of this, some of the coagulation assays do not have a “gold” standard or even any established true values. This problem is slowly changing as international standards are being developed.15,16 For some standards, accuracy issues still arise due to differences in methodology (clotting assays vs. chromogenic assays), potentially including the calibration process itself.16 The laboratory must make certain that their standards or calibrators are linked to an international primary standard through secondary standards from the manufacturer.5

Precision Precision is the closeness of agreement (degree of variability) among a series of measurements obtained from multiple analyses from a single sample or reference material.5,17 Precision includes within-assay variability (intra-assay) and day-to-day variability (interassay). Intra-assay variability is the precision under the same operating conditions. Interassay reproducibility is the precision of the method when the assay components may be slightly different (different days, different operators, and different reagent vials). Precision is established irrespective of accuracy. From the precision perspective, it is the reproducibility of the results that is important. The precision is usually expressed numerically as the level of “imprecision” and namely as a standard deviation (SD) or more commonly as a coefficient of variation (CV%). Precision evaluation consists of a two pronged assessment: (1) intra-assay variation and (2) interassay variation.5,9,13 Variation for intra-assay assessment is determined by performing the assay on the same patient or control sample within a single run using the same reagents for 10-20 measurements.5,13 The CV should usually be 3 to 6% for clotting, chromogenic, and most immunologic analytes, but never more than 10%. However, for the more complex assays (e.g., platelet aggregation, VWF, and lupus anticoagulant), the imprecision in terms of CV may be 10 to 20% or more. Interassay precision is evaluated by repeating the same samples (usually controls) on the same instrument but with

Marlar et al.

new reagents, different days, different operators, and different environmental conditions for a minimum of 10 runs, typically performed over 10 consecutive working days.5,13 In general, the interassay precision is greater than that observed for the intra-assay precision studies. Usually, the CV for interassay studies is 4 to 10% but not more than 12%. Again for the more complex assays, the precision CV can increase to 20 to 40%, thus often making it very difficult for these assay results to reflect the true state of the patient. The acceptable limits of precision during the validation phase are defined by a variety of factors and will vary among laboratories. No hard and fast rules apply for acceptability of coagulation testing precision. However, the laboratory director must decide the acceptable limits of precision based on publications, manufacturer’s data, and/or published guidelines. At least three samples that span the reportable range (including normal, low abnormal, and high abnormal values) should be used in the precision study. The acceptable levels of precision may be different between normal and abnormal samples, the type of assay, and the reagent–instrument combinations. The precision results should mirror the values reported by the reagent and/or instrument manufacturer. Precision results within the manufacturer’s reported limits are acceptable. If the imprecision value is greater than the manufacturer’s reported values, then the laboratory may still accept the results if they feel their method parameters justify the increased imprecision. This is a laboratory director decision. Care must be taken with very low values when data are represented as CVs, as small changes in magnitude can lead to large changes in CV, as CVs are dependent on the values obtained. For example, a 0.5 magnitude variation in a mean value of 100 U/dL factor (F) VIII will provide a very low CV of 0.5% when testing this normal sample; however, the same 0.5 magnitude variation in a mean value of 1 U/dL FVIII will provide a very high CV of 50% when testing this pathological sample reflective of a severe hemophilia A sample. For such cases, it may be better to identify an acceptable range of values (e.g.,  0.5 U/dL) rather than a CV value.

Specificity Specificity is the ability to unequivocally assess the analyte in a standard specimen in the presence of components that may be expected to be present.5,13 This typically includes such components as the plasma (matrix), degraded or inactive components, as well as potential cross-reacting substances. The method must be able to differentiate similar analytes and interfering substances that could have an effect on the assay value. In commercially available and approved methods, the manufacturer should have performed these evaluations. For laboratory developed tests, the laboratory must demonstrate specificity, a task that may be very difficult.

Limits In the validation of an assay’s performance, two types of “limits” must be evaluated: limit of detection (LOD) and limit of quantitation (LOQ).5,18 The assays with time-based results (prothrombin time and aPTT) do not need to have “limits” Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

191

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

Hemostasis Validation and Performance

Hemostasis Validation and Performance

Marlar et al.

determined as they are global or screening assays without specific analytes to be determined and they do not approach zero. Therefore, no LOD or LOQ is usually determined for time-based assays. The LOD (also known as analytical sensitivity) of a method is the level at which the assay can distinguish a sample without analyte present (blank) from the sample with analyte present; however, the assay may not accurately quantify the amount of analyte.4,5 The LOD is usually defined as 3 SD above the mean of the blank, thus marking the limit above the “noise” of the method. Therefore, the probability of a false positive is minimal (< 1%). The accuracy and precision of the method and preanalytical variables play an important role in determining the LOD. Some coagulation methods have poor LOD due to a high imprecision. The standard protocol for determining the LOD is to measure a zero standard (no analyte present) multiple times (20 replicates) and calculate the SD.18 The upper 3 SD range is considered “noise” and the value at the upper end of the 3 SD is the lower LOD. In coagulation, this LOD is sometimes difficult to ascertain, true “zero” standards that are plasma based may not be available in large enough quantities. Usually, the “zero” standard plasma is an artificially created sample because clinically relevant “zero” samples are not available. It is important to understand the lower LOD of the assay in relation to the clinical use of the assay. A good example is found in hemophilia testing. It is important to clinically distinguish a level of < 1 U/dL from one with 2 U/dL. If the lower LOD is only 3 U/dL, then patients with severe hemophilia (< 1 U/dL with severe bleeding symptoms) cannot be differentiated from patients with moderate hemophilia (2 U/dL with milder bleeding symptoms). The laboratory must decide the LOD that is necessary for each analyte, based on the intended clinical utility of the method. There are several different “detection limits” that must be taken into account in the overall evaluation of the coagulation assay method (instrument LOD, method LOD, reagent LOD, and plasma substrate LOD).18 The instrument LOD and method LOD are the main parameters evaluated for new methods or reagent–instrument systems. This information is usually supplied by the manufacturer, but should be verified by the laboratory before using the assay.

Limits of Quantitation LOQ defines the lowest amount of analyte that is quantifiable in the assay.18 In addition, the LOQ defines the level at which two values can be distinguished with precision and accuracy.4,5 In common practice, the lower LOQ is statistically defined as 5 to 10 SD from the “zero” standard value. Nevertheless, each method must be evaluated independently to determine the lower threshold of LOQ.4,5 As an example, factor VIII can be distinguished at the < 1 U/dL level, whereas factor XII may be distinguished at the 5 U/dL level and VWF activity at the 12 U/dL level. The laboratory in consultation with the clinical staff must determine the clinically relevant lower LOQ for each assay. For clinical purposes, the assay must be able to accurately differentiate between levels leading to medical decisions. Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

Analytical Measurement Range The analytical measurement range (AMR) of an analytical method is the interval between the upper and lower analyte concentration for which the analytical method has demonstrated a suitable level of precision, accuracy, and linearity without pretreatment (i.e., sample dilution).5,18,19 The AMR of each assay is dependent on the assay characteristics, but more importantly on the clinical needs for the specific analyte. The lower end of the linear range is usually more important in coagulation, but may be the most difficult to establish at the clinically relevant level. The laboratory cannot report values beyond the lowest or highest standards of the calibration curve. If the assay method cannot be reported to the level necessary for clinical utility, then modifications of the method or alternative methods must be used to achieve the desired detection level, including curves established for lower ranges, dilutions of standards, or patient samples.4,5 This is termed the reportable range, which can be different from the AMR. Validation studies must be undertaken when sample modification (usually dilution of sample) is performed to extend the reportable range from the AMR. The same requirements for accuracy and precision must be employed to establish an expanded reportable range. Another issue that should be validated is the matrix used for sample dilution, which can typically include sample buffers, specific diluents, plasma or serum samples with undetectable concentration of the analyte, as well as saline or even water.

Linearity The linearity is the ability to obtain results that are directly proportional between the instrument response and the concentration of analyte within a given range.5,20 Linearity acceptance criteria are based on statistical correlation of regression (r2 value and slope). Mathematical transformations of data can help to promote linearity if there is scientific evidence that transformation is appropriate for the method. A good example is the factor assay, which may use semilog or log–log transformations. It is important not to force the line through the origin by calculation, as this may skew the slope in the clinically relevant range. For diagnostic and monitoring assays, the analytical method must have a good proportional relationship between analyte concentration and instrument response. Methods for linearity determination of coagulation quantitative assays have changed over the past decade.5 Linearity computations have evolved from visual assessment of the line to statistical analysis via linear regression.5 Linear regression, however, will not readily define acceptable limits because many of the quantitative coagulation assays may be imprecise and have a poor linear fit. In the future, refined statistical methods including polynomial analysis will be a standard linearity assessment tool.

Robustness The robustness of an analytical procedure is a measure of the assay capacity to remain unaffected by small to moderate variations in method parameters and preanalytical

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

192

Hemostasis Validation and Performance

Continued Performance of Coagulation Assays The final step of the validation process is actually monitoring on a continuous basis the ongoing performance of the assay. To ensure the validity of an assay, consistency and reproducibility over time in conjunction with accurate results are required. This ongoing evaluation of methodology is required

193

not only during each run (i.e., IQC)11 but also during periodic comparison with other laboratories for continual accuracy (i.e., EQA).23 The main aim of an IQC program is to confirm that generated results remain consistent over time.11 EQA is a critical element in the laboratory’s quality program. It evaluates the accuracy of coagulation results by a comparative process.23,24

Standards and Guideline Developing Organizations Several international organizations develop consensus guidelines and standards to establish concise and cost-effective laboratory medicine processes to improve patient testing. Most of these organizations base their guidelines and standards on a consensus document written by experts followed by open and unbiased input from the laboratory community. “Standards” documents are developed through the consensus process to help clearly identify specific and essential requirements for the material, method, or practice of laboratory testing. However, “guideline” documents are developed through the consensus process for the general criteria of the method or practice of laboratory testing used as written or modified upon validation of the modification. One of the most well-known organizations is Clinical Laboratory Standards Institute (CLSI; www.clsi.org/), which is an international nonprofit organization that uses the input of voluntary experts from the laboratory community, manufacturing and government agencies to develop these guidelines and standards in all areas of patient testing and other healthcare issues. Other involved groups are international and national thrombosis and hemostasis societies, hemophilia groups, and specific groups who may tend to focus efforts on specific tests or methodologies (e.g., lupus anticoagulant or solid phase antiphospholipid antibody tests).25

Table 3 The basic statistical components of a validation study for a new or modified coagulation assay Category

Method

Acceptance range

Accuracy

10 runs of sample with known value

10%

Precision

10–20 replicates of samples on same run 10 replicates on different runs

3–6% 4–10%

Specificity

Compare values in various samples with validated assay

R2 > 0.90 Slope ¼ 0.95–1.05

AMR

Lowest and highest values where accuracy and precision are within acceptable range

10%

LOD

Replicates of “zero” samples

> 3 SD

LOQ

Lowest value where accuracy and precision are within acceptable range

10%

Robustness

Identify variables in the method affecting results

10% with reagent variability

Correlation with old method

Correlation of values over AMR

R2 > 0.90 Slope ¼ 0.95–1.05

Abbreviations: AMR, analytical measurement range; LOD, limit of detection; LOQ, limit of quantitation. Note: Not all of the listed components may be needed for every assay. The laboratory director must decide which aspects are relevant. Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

variables.21,22 It provides an indication of assay reliability during normal usage.5,7,21,22 The important analytical parameters include instrument differences, operators, reagent lots, and sample preparation. Robustness is difficult to truly assess and is generally associated with the increasing complexity of the analytical system. The majority of analytical parameters for robustness are determined by the manufacturer and usually approved by a governmental regulatory agency such as the Food and Drug Administration (FDA). However, if the reagents are not designed and evaluated by the manufacturer or used differently than intended, then stated claims of method are not implicitly valid. In laboratory developed tests, research use only, and laboratory introduced changes of methods from reagent manufacturer instructions, the coagulation laboratory must assume responsibility for validating robustness. In addition, the laboratory must evaluate the preanalytical parameters encountered in the clinical setting. Ideally, robustness should be explored during the development of the assay method through the use of a detailed validation protocol. Initially, one must first identify variables in the method that may influence the results. These include storage conditions and processing of the sample, minimum and maximum dilution, and type of diluents and level of interference (hemolysis, turbidity, etc.). This type of method validation will ensure that the system components are robust.

Marlar et al.

Hemostasis Validation and Performance

Marlar et al.

Summary and Conclusion

7 Clinical and Laboratory Standards Institute (CLSI): Collection,

Before initiating any validation study, whether establishing a new assay, changing reagents or instruments, or changing methodology, a well-planned validation procedure must be in place using sound scientific and clinical criteria. The supplier of the reagent and/or instrument may provide validation protocol outlines, or the laboratory may have to develop their own. The major components of a sound validation protocol are presented in ►Tables 2 and 3 and an example of a standardized form is shown in ►Fig. 1A–C. The planned validation studies, including statistics and defined acceptance criteria, must be determined before the start of the validation process by using the literature or manufacturer’s data for acceptance criteria. The protocol must be performed in a timely manner with adequate samples, standards, and calibrators. Much of the general information in validation protocols is described in documents provided by accrediting and standards organization.4,5,13,14,17–20,22 After performing the validation protocol, the data must be analyzed with predetermined statistical criteria to ensure valid conclusions for presentation in the validation summary report. If the defined criteria established in the protocol are met and/or any deviations that might affect overall conclusions are justified, then the method can be considered valid. The final validation report, along with all of the data, statistical analyses, and the signatures of all participants are placed in the official SOP manual, or kept readily available in a separate file or notebook.

8

9

10

11

12

13

14

15 16

17

Acknowledgments This work was supported in part by the Department of Veteran’s Affairs Pathology and Laboratory Services. The authors would like to thank the technologists at the Oklahoma City VA Laboratory.

References

18

19

20

1 Jhang JS, Sireci AN, Kratz A. Postanalysis: medical decision making.

2 3

4

5

6

In: McPherson RA, Pincus MR, eds. Henry’s Clinical Diagnosis and Management by Laboratory Methods. Philadelphia, PA: Saunders; 2011:80–90 Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ 1995;310(6987):1122–1126 Straus SE, Glasziou P, Richardson WS. Evidence-Based Medicine: How to Practice and Teach It. 4th ed. Edinburgh: Churchill Livingstone; 2010 CLSI Evaluation Protocols (EP05 through EP21), the Clinical and Laboratory Standards Institute Web Site: www.clsi.org. Wayne, PA: 2013 Clinical and Laboratory Standards Institute (CLSI): Protocol for the Evaluation, Validation, and Implementation of Coagulometers. CLSI document H57-P. Wayne, PA: CLSI; 2008 Marlar RA. Hemostasis test validation, performance and reference intervals. In: Kitchen S, Olson JD, Preston FE, eds. Quality in Laboratory Hemostasis and Thrombosis. West Sussex, England: Wiley-Blackwell; 2009:9–18

Seminars in Thrombosis & Hemostasis

Vol. 40

No. 2/2014

21

22

23

24

25

Transport, and Processing of Blood Specimens for Testing Plasma-Based Coagulation Assays and Molecular Hemostasis Assays— H21–A5. Wayne, PA: CLSI; 2008 Schmaier AH, Miller JL. Coagulation and fibrinolysis. In: McPherson RA, Pincus MR, eds. Henry’s Clinical Diagnosis and Management by Laboratory Methods. Philadelphia, PA: Saunders; 2011:785–801 Pincus MR, Abraham NZ. Interpreting laboratory results. In: McPherson RA, Pincus MR, eds. Henry’s Clinical Diagnosis and Management by Laboratory Methods. Philadelphia, PA: Saunders; 2011:91–108 Dilts TJ, McPherson RA. Optimizing laboratory workflow and performance. In: McPherson RA, Pincus MR, eds. Henry’s Clinical Diagnosis and Management by Laboratory Methods. Philadelphia, PA: Saunders; 2011:13–23 Kitchen S, Preston FE, Olson JD. Internal quality control in the hemostasis laboratory. In: Kitchen S, Olson JD, Preston FE, eds. Quality in Laboratory Hemostasis and Thrombosis. West Sussex, England: Wiley-Blackwell; 2009:43–50 McPherson RA. Laboratory statistics. In: McPherson RA, Pincus MR, eds. Henry’s Clinical Diagnosis and Management by Laboratory Methods. Philadelphia, PA: Saunders; 2011:109–118 Clinical and Laboratory Standards Institute (CLSI). User Demonstration of Performance for Precision and Accuracy. EP15-A2. Wayne, PA: CLSI; 2006 Clinical and Laboratory Standards Institute (CLSI). Validation of Laboratory Tests When Proficiency Testing is Not Available. GP29A2. Wayne, PA: CLSI; 2008 Hubbard AR. International biological standards for coagulation factors and inhibitors. Semin Thromb Hemost 2007;33(3):283–289 Hubbard AR, Heath AB. Standardization of factor VIII and von Willebrand factor in plasma: calibration of the WHO 5th International Standard (02/150). J Thromb Haemost 2004; 2(8):1380–1384 Clinical and Laboratory Standards Institute (CLSI). Evaluation of Precision Performance of Quantitative Measurement Methods. EP5–A2. Wayne, PA: CLSI; 2004 Clinical and Laboratory Standards Institute (CLSI). Protocols for Determination of Detection and Limits of Quantitation. EP17-A. Wayne, PA: CLSI; 2012 Clinical and Laboratory Standards Institute (CLSI). Statistical Quality Control for Quantitative Measurement Procedures: Principles and Definitions; Approved Guideline-Third Edition C24–A3; 2006 Clinical and Laboratory Standards Institute (CLSI). Evaluation of the Linearity of Quantitative Measurement Procedures. EP6-A. Wayne, PA: CLSI; 2003 Kroll M, Gilstad C, Gochman G, et al , eds. Laboratory Instrument Evaluation, Verification and Maintenance Manual. 5th ed. Northfield, IL: College of American Pathologists; 1999 Clinical and Laboratory Standards Institute (CLSI). Method Comparison and Bias Estimation Using Patient Samples. EP9–A3. Wayne, PA: CLSI; 2013 Preston FE, Kitchen S, Srivastava A. External quality assessment in hemostasis: its importance and significance. In: Kitchen S, Olson JD, Preston FE, eds. Quality in Laboratory Hemostasis and Thrombosis. West Sussex, England: Wiley-Blackwell; 2009:51–62 Preston FE, Lippi G, Favaloro EJ, Jayanardhan GR, Edision ES, Srivastava A. Quality in coagulation and hemostasis testing. Hemophilia 2010;16(Suppl 5):93–99 Favaloro EJ, Plebani M, Lippi G. Regulation in hemostasis and thrombosis: part I-in vitro diagnostics. Semin Thromb Hemost 2013;39(3):235–249

This document was downloaded for personal use only. Unauthorized distribution is strictly prohibited.

194

Copyright of Seminars in Thrombosis & Hemostasis is the property of Thieme Medical Publishing Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Validation of hemostasis and coagulation assays: recommendations and guidelines.

The clinical hemostasis laboratory is a complex testing arena which employs numerous coagulation assays and spans several different test methodologies...
1MB Sizes 2 Downloads 0 Views