Journal of Antimicrobial Chemotherapy (1975) 1, 103-116
Quality control of serum gentamicin assays—experience of national surveys
D. S. Reeves and M. J. Bywater
The increasing use of the aminoglycoside antibiotic gentamicin together with an awareness of the importance of adequate serum levels for therapy and of a possible relationship between those levels and its main side effect, ototoxicity, has led to many laboratories being asked to perform gentamicin serum assays. The relatively low therapeutic ratio of the aminoglycosides antibiotics indicated that accurate assays were essential if they were to realize their full potential in controlling therapy. With this in mind four laboratories, our own among them, exchanged some specimens of sera containing gentamicin in late 1971 and early 1972. The results showed wide divergencies in the estimated concentration of single samples by the participant laboratories. This was clearly a worrying state of affairs and recognizing this, the British Society for Antimicrobial Chemotherapy (B.S.A.C.) sponsored a number of larger surveys. In 1973 this laboratory was invited by the Public Health Laboratory Service (P.H.L.S.) Quality Control Committee to send out specimens for antibiotic assay. The experiences described here are those of the later B.S.A.C. surveys and those under the joint auspices of the P.H.L.S. and B.S.A.C. in 1973 and 1974: For convenience of reference the surveys are designated as follows: January 1973 (B.S.A.C.) —Survey A March 1973 (B.SA.C.) —Survey B June 1973 (P.H.L.S./B.S.A.C.) —Survey C (P.H.L.S. 20th Distribution) February 1974 (P.H.L.S./B.S.A.C.)—Survey D (P.H.L.S. 28th Distribution) Gentamicin was chosen as the antibiotic to be investigated for the reasons given above. Furthermore, it was found to be by far the most frequently assayed antibiotic in a small poll taken among participant laboratories, and it seemed to be a technically suitable drug because of its stability in solution at room temperature. Objectives The circulation of sera containing known concentrations of gentamicin for assay by participant laboratories and the receipt of their results would allow an assessment to be made of the prevailing accuracy of assays. By obtaining methodological details it was hoped to identify any technical factors having a bearing on accuracy. The circulation with the specimens of a standard solution of gentamicin for use in the preparation of working standards was aimed at determining whether laboratories' usual source of standard material was adequate. Finally it was hoped that feedback of results to individual laboratories would help them to identify deficiencies in their performance and correct them. 103
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Department of Medical Microbiology, Division of Pathology, Southmead Hospital, Bristol, BS10 5NB, England
104
D. S. Reeves and M. J. Bywater
Distribution of samples Sera was distributed into individual bottles using a mechanical dispenser. One of the main problems was finding a suitable container. Polystyrene containers were the most durable in the post but tended to leak around the screw cap. Eventually glass containers (approx. 5 ml size) with plastic caps and card liners, sterilized by sub-atmospheric steam with formaldehyde, were found to be the most suitable. Containers were carefully packed in approved postal boxes along with the relevant documents. These consisted of a description of the samples requesting the investigation required (a typical version is shown in Figure 1), an assay report form (Figure 2), and a method report form (Figure 3). The reporting forms (Figure 2 and 3) are shown in their final form, modified in the light of experience. Provided standard material In the later surveys participants were provided with a stock standard solution of gentamicin. \ They were told to assume the potency was 1000 ng/ml of gentamicin base and report their results accordingly. In fact, these solutions were deliberately prepared • Donated by Nicholas Laboratories Ltd. t "Accuiamatic"—Jencons Scientific Ltd. \ Made and ampouled by courtesy of Nicholas Laboratories Ltd.
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Methods and results Preparation of samples The circulated samples were prepared in pooled human serum obtained from healthy donors individually and regularly checked for the presence of serum hepatitis antigen and antibody. All individual samples were also checked for the presence of antibiotics in a highly sensitive plate assay system employing Bacillus subtilis as an indicator organism. Powdered gentamicin sulphate of known potency* was dried at 100°C for 180 min and allowed to cool in a desiccator. An amount exceeding 100 mg was quickly weighed on an Oertling balance capable of weighing accurately to 0-1 mg and transferred completely to a volumetric flask. Sterile Grade A glassware was used throughout. Solution was effected with sterile distilled water made up to the mark. An accurately measured aliquot, usually 10 ml, was pipetted into a second volumetric flask and made up to the mark with pooled human sera. After mixing this solution it was dispensed in 2-2 ml lots into the individual sample bottles for circulation to the participant laboratories. A typical example of this preparation was as follows: 2520 mg of powder of a potency of 635 (Ag/mg and thus equivalent to 160 mg of gentamicin base was weighed out and dissolved in 200 ml of water. 10 ml of this solution was transferred to a 1000 ml volumetric flask and made up to the mark with serum resulting in a final concentration of 8 (J-g/ml. All values for potency in this paper are expressed as gentamicin base. Thus, by appropriate modification of the general method, each sample of gentamicin solution could be accurately prepared by using large volumes, illustrating one of the advantages of having many participant laboratories. A particular problem was maintaining solutions in serum free of contamination. In one circulation there were a few complaints about contaminants, mainly from laboratories employing doubling dilution assay techniques. Subsequent to this all serum was obtained under sterile conditions, all glassware was sterilized prior to use, and the individual samples were prepared by a roller-pump dispenserj using plastic tubes sterilized in a sub-atmospheric steam-formaldehyde injection autoclave.
Quality control of serum gentamicln assays
DISTMBUTIQI - Vttek
105
Ing l 8 / ; / 7 4 .
Antibiotic Sera and solution* issued by Dr. D.S. Rcevfts, Department of Bacteriology, Soutimwad Itospital, We.iUnii-y-on-Trym, B r i s t o l . BS1O 5*8.
Six i H p l u of Mr* (Labelled 9fl, 100, 101, 102, 103 and 104) containing gentamicin ir« tncloMd. Scmplma 99, 100, 101, 102 and J.03 contain no other antibiotics but It should be asauetd that sample I04 comes fro* a patient who i* simultaneously receiving lincomycin. Although some samples «*y Sir* similar assay T i h u they ar« not necessarily Identical. No value exceeds 20 ug/»l or i s leas than 0.5 ug/ml.
Tb« samples sboold be assayed and reported twice usin( your usual —thod i1.
Uainx standards you already hero or would normally prepare on receipt of such apecisMns}
2.
Using standards prepared from the enclosed ampoule (A) of geotamicin eolation. Tou •hould assume the coocentration of this stock standard to be 10OO usy«l of (entamicin base. *l*asa report the results as if tbe assumptioo Is valid, although i t may not be exactly s o .
Tbe results on Form A and details of tbe method oo Form » should be s«nt by f i r s t class mall in an envelop* marked •Confidential" to i Dr. 0.8. Xs*ve« at the above address. Completed forme should be returned not later than 11th March, 1974.
Figure 1. Typical instructions accompanying a set of quality control samples. LAB. 1DEVTITT HO. ( P l e a s e Check)
BSAC QUALITY COimiOL - FEB. 1974 DI5TEIBUTICni AASAT RCFOrt (CDfTAMICUl) STAntAIDS - OWf
STAjnUjtm - A Calculated potency (us/ml) S
Individual IOM als^s *
Clcul.t^l potency (m^ml) 8
99 1OO 101 102 103 104
wm (««/-i>
*
Flease enter tbe unit of Measurement, m.g. diameter in mma, arbitrary u n i t s , etc. each sample and standards - no\ Just wean «one s i t e .
S
Flease glve results to eam declmal p l a c e .
8TAJRURDS OVD
DATE Of ASSAT
Enter f 11 aome s l s e s for
TIKI or AS SAT TO Dete specimens received
Dete of results
A Flease return tbe completed fora. #long with Form B i * l i t y C o n t r o l Fata. 1 9 7 4 . ASSAT KETIWO (GEJCTAHICIjO BPECIKETS 0 - 104 1
BASTC
2
HEDttm
Vertical diffusion Otber (daftue)
Plato diffu aioo Doublina d i l u t i o n In broth Attor Typo VOIOM* per pi*to Di-a^h Type
Pil •1..
Volume/Tube SDOO1>«
Strain Dtraaltr of viable count Surfaco or inoonwratad Stated potancy
4-
STATOARIB
5
PREP. OF
• aat Dllutod
fdiluent
6
METHOD Or
Cut WeiLa
(di^wetor
Cut* "SDinea-
falsa
Source of atatarLal r i n a l Dilueat
(Batch Vo.
)
PH
(else Fixed
7
ORDER OF APPLICATION
S
ftuafcer
COLD PREDIPFTJSIOM
10
ISCUBATIOM
11
KETBOO Or READING
11
HETBOO 07 CALCOLATIOM
of
Standard*
Tea
Art>itrarr
StandartU rirac
Randoai
9
)
(Includin* dllutioneT
Sample*
lo
Baaipio* r i r s t
Timm (hra)
)
TC-P.
Te«p
Tiaw
Callpars (T tYi»e Rolar (T typa Optical Optical an toaated (7 type Other
i 7 varnier
) ) )
O m E * USEFUL DETAILS IXCLUTUKO OOWtEMTS OM PRE5EJCCB OF UKCOWTCTU IV BPIClWlH 1QA
Figure 3. Method report form.
Stability of antibiotics in post Obviously it is important that the concentration of antibiotic in the samples when they reach the recipient laboratories should be that originally made up. Any deterioration would cast doubts on the validity of survey since it would be an indeterminable variable due to different times and conditions of transit. This was checked by posting extra sets of samples either to ourselves or to other laboratories and asking them to return them unopened. In this way we received sets of samples which had been in the post either once or twice. Assay of these samples when using gentamicin alone never showed any detectable deterioration of potency. Mixtures of antibiotics Two antibiotics, carbenicillin and lincomycin (or clindamycin), are frequently administered concurrently to gentamicin and it was therefore appropriate to investigate how they might interfere with gentamicin assays. Carbenicillin. In one circulation (Survey B) participants were sent gentamicin (4-5 |xg/ml)
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
INDICATOR OSCAKJSM
d •Marcial batch Mo. Siie t f plate t Sia« of tube
Quality control of sermn gentamldn assays
107
and carbenicillin (1000 ng/ml) in serum as a mixture prepared immediately prior to dispatch. They were also sent separate solutions of both antibiotics that when mixed in exactly equal proportions gave the same concentration of gentamicin and carbenicillin as in the pre-mixed sample. They were asked to assay both mixtures for gentamicin simultaneously. The results are shown in Table I. Even allowing for the large variances the results seem to show clearly that the presence of carbenicillin in posted samples of gentamicin could lead to a serious underrating of the potency of the gentamicin. This effect is presumably due to the temperature dependant inactivation of gentamicin by carbenicillin (Mclaughlin & Reeves, 1971) during transit. Table I. Influence of carbenicillin on gentamicin assay
Value as made up Mean of assay results (21 values) 2 S.D.
Mixed prior to dispatch
Mixed on receipt
4-5
4-5
2 0 (2-3)* 2-2 (3-2)
4-6 (5-1) 4-2 (4-8)
* Results in brackets are those obtained with working standards prepared from standard material provided.
Lincomycin {clindamycin). In view of the apparent inactivation of gentamicin by carbenicillin, a study was made to see if lincomycin or clindamycin would inactivate gentamicin. Solutions of gentamicin in serum or buffer were prepared to contain concentrations of lincomycin or clindamycin typically found in clinical therapy. After incubation at three temperatures (4°, 20° and 37°C) and for various lengths of time, up to 3 days, samples were assayed. Gentamicin was assayed by a microbiological plate assay using a Klebsiella spp. as indicator organism. This assay was unaffected by lincomycin or clindamycin at a concentration many times higher than those used in the experiment. Lincomycin and clindamycin were assayed by bio-autography after separation from gentamicin by high voltage electrophoresis in agar. The results showed that there was no detectable inactivation of any of the antibiotics either when present alone or in combination (Holt & Reeves, unpublished data). Collection and analysis of results Those participant laboratories involved through the P.H.L.S. returned their report and method forms to Mr W. B. Fletcher at Central Public Health Laboratory, Colindale, London, NW9 5HT. The remaining laboratories were members of the B.S.A.C. surveys and returned their forms to this laboratory. All the results were collated and analysed by a team including Mr Fletcher, Mr Alan Perkins (biometrician, lately of Nicholas Laboratories Ltd) and the authors. Since laboratories were identified by code numbers known only to the organizers confidentiality was maintained. Analyses were usually done manually, including inter-survey comparisons. The results of Survey D were stored on tape files in a Hewlett-Packard 98 30-A programmable calculator. The statistical parameters for each laboratory were then calculated automatically and the data stored for comparisons with future surveys. The primary statistic calculated for each individual assay result was the percentage
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Assayed potencies of gentamicin (ng/ml)
108
D. S. Reeves and M. J. Bywater
Feedback of results to participants
Shortly after the date set for the receipt of reports from the participants, a circular was issued giving the true values of the samples and the potency of the issued standard. A typical circular is shown in Figure 4. Later a more detailed report was issued giving the results and their statistical for breakdown for each laboratory, and many of the type of tables shown in this paper. This report also contained detailed comments on the results. MAC QCALITT CONTROL, p p . 1974 CPTAJUCIH ATOAT
TIM MI-« eontainad tha following eoocentratla (axpraaaad a* • • n t M l c l n basa)lt» 100 101 101 101 104
6.
of (ootmlels
5
) ) i 1 1. i i >
- 10. 5 1.
-
«. 6. 5
)
99, 103 and LO4 ca•a fro. tb. aa«. « U S-pl~ 104 bad aolld ollndai• j c U addad to | l n a final lAataljr 5 «*/•!.
of approx-
Staodaj-d 'A' contained 1050 o«/al of (aataadcU u d IO, If uaad aa dlractad, tha aaaay r«*ulta afcoald bars b«ao it laaa ( l . a . 6.2, 10.0, 1.4, 1.1, t . l , and 6.1 rwp.rtl-r.lr, to tw> almlflcaot fi^uraa).
Figure 4. Circular giving true results of sera to participant laboratories
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
error from the "true" result The "true" result was taken as the value of the test sample as made up, making an allowance where necessary for the potency of a supplied standard solution. In Surveys C and D six samples were issued, each to be assayed twice. The mean percentage error and the standard deviation (S.D.) of the percentage errors were calculated for each set of six results. The statistic of the modulus of the mean percentage error (|mean|)+2 S.D. was used to grade laboratories performances on the basis of the results using their own standards. Because the statistic |mean|+2s.D. is fundamental to the analyses given in the results, an illustration of its derivation from a hypothetical set of results follows: on a set of six sera of true concentrations of 6-5, 10-5, 2-5, 3-25, 6-5 and 6-5 ug/ml of gentamicin a laboratory reported values of 6-2, 11-6, 30, 30, 6-5 and 6-5 ug/ml, respectively. The percentage error of each result was calculated by (true-reported/true) x 100 giving values of —5, +10, +20, —8, 0 and 0%, respectively. The mean of these values was taken as the "mean percentage error", and their standard deviation was also calculated. The final statistic taken as an assessment of performance on the basis of the set of results was the modulus of the mean percentage error (i.e. the mean without its sign) plus 2 S.D. The percentage errors of each set of results were assumed to fall into a normal distribution. This assumption was found to be best fulfilled by converting the data to a geometric scale (e.g. to logarithms), but the arithmetic data was shown to fit a normal distribution well enough for the purposes of the analysis. In Surveys A and B the statistics were in fact also calculated on the basis of logarithmically converted reported results and this form of analysis was repeated in Survey D. No significant information accrued additional to that already known on the basis of arithmetic analysis of results. This form of analysis was therefore not pursued.
Quality control of serum gentamidn assays
109
Results on individual specimens The overall results for 20 specimens assayed for gentamicin in 4 surveys are shown in Table II. The statistics were calculated in a similar manner to that for sets of assays from individual laboratories, but based instead on all the reported results for each sample. By
A A A A A
6
B
O
s s s
o o o o o
7
B
8
B
67
C
68
C
69
C
70
C
71
C
72
C
99
D
s u
op op
s
o
s s s s s s
p
o op op op op p
o p
100
D
101
D
102
D
103
D
104
D
1
s s s s s
op op op op op
— — — — —
— — — — — — —
- 5-8 - 20 - 6-7
+ + + + +
46 56 46 67 50
-
50 48 55 46 54
53-8 50-0 62-7
- 4-2
24 24 28 37 29
450-0 4500
22 20 22 20 18 19
- 4-4 - 4-4 + 2-2 + 6-7 - 91 -13-6
26 23 35 45 60 64
+ + + + + +
64 56 122 167 117 122
-
49 51 38 36 78 78
56-4 50-4 72-2 96-7 1291 141-6
4-5 4-1 5-5 50 5-5 50 110 100 4-5 4-1 5-5 50
79 76 80 77 79 77 81 77 80 76 79 77
- 6-7 - 4-9 -12-7 - 20 -14-5 -14-5 - 6-4 -111 - 6-8 -110 - 40
62 62 69 71 66 74 55 46 69 62 51 48
+ 360 + 267 +409 + 334 + 355 + 300 + 300 + 210 +433 + 322 + 278 + 198
- 93 - 98 -100 -100 - 91 - 90 - 88 - 93 - 89 - 83 - 85 - 80
130-7 128-9 150-7 1440 146-5 1480 124-5 98-4 1491 130-8 1130 1000
6-5 6-2
86 87 87 88 85 86 87 87 87 88 87 86
+ + + + + +
30 27 34 32 38 45 34 41 29 35 31 36
+ 115 + 102 + 90 + 100 + 160 + 200 + 131 + 158 + 85 + 158 + 146 + 158
-
11-2 2-4 2-3
10-5
3-25 — L L
Maximum Minimum |Mean| result result + 2 S.D.
19 19 19 19 19
100 2-5 2-4 —
errors from true result
S.D
4-5 4-5 90 90
p
/o
|Mean
120 4-9 — — —
0/
No. < resul
S S S
Othe r drugs81
Stanc mate
1 2 3 4 5
True* result (|ig/) ml
31 6-5 6-2 6-5 6-2
00
0-0
90 0-9 3-7 9-9 6-3 1-5 5-3 2-6 4-7 2-7 91 2-2
66 73 66 70 80 71 85 74 66 77 69 68
740
62-2
690
54-9 71-7 73-9 82-3 91-5 73-3 83-6 62-7 72-7 711
74-2
S=samples in pooled human serum; U=samples in pooled human urine, adjusted to pH 5-5. • O=working standards, as prepared for routine assays; P=working standards prepared from provided standard material. • L=sample contained clindamycin, 5 |ig/ml. • Potency of samples as made up, making allowance for potency of provided standard material where necessary.
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
1
1 11 Spcci
Samr. ;no.
Table IL Overall results for individual specimens as assayed by participant laboratories
110
D. S. Reeves and M. J. Bywater
showing the overall performance of participant laboratories for each specimen it is possible to obtain an estimate of the typical performance of the laboratories. Naturally, this form of analysis does not identify the quality of performance of individual laboratories.
Table IEL Numbers of sets of results falling into categories of quality on basis of assays made with the laboratorys' own standards -
N
„ Survey
' ^era
No. of laboratories
A B
5 2
19 20
C D
6 6
81 92
Number (and %) of results in each category . . ^ misleading 1(5%) 10(53%) 8(42%) Insufficient serum samples for meaningful analysis 17(21%) 29(36%) 35(43%) 17(18%) 29(32%) 46(50%)
In Surveys C and D comparison was made between the gradings obtained in this manner and gradings obtained by assay against the standard gentamicin solution provided (Table IV). It can be seen that there was a strong tendency for laboratories to reproduce their quality of performance whichever standards were used. In particular a movement from "Highly misleading" to "Good" or vice versa on changing standards was very uncommon. Quality of results related to basic method used Sufficient laboratories participated in Surveys C and D to make meaningful analysis of methodology possible (Table V). As might be expected from a theoretical examination
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Quality of performance by individual laboratories As explained above, a statistic (|mean|+2 S.D. of percentage errors) was used to grade the performance of individual laboratories for each set of specimens. On the basis of the requirements for accuracy of gentamicin assays in the clinical context for controlling dosage the laboratory results were graded as follows: 'mean + 2 S.D. 50% — "Highly misleading" mean All three gradings may be illustrated by taking the results of laboratories assaying a hypothetical sample containing 20ug/ml of gentamicin. Assuming that the mean percentage error is negligible, then a "Good" set of results would be in the range 1-5 to 2-5 ug/ml about 19 out of 20 times, a "Poor" set in the range 10 to 30 ug/ml, and a "Highly misleading" set in the range < 1 0 - > 3 0ug/ml. In every case, the remaining 1 result out of 20 would be expected to fall outside the quoted ranges. In view of the fact that there appears to be a crucial clinical distinction between pre-dose levels of 1-5 and 3-0 ug/ml (Nordstrom, et al., 1973), we did not feel our gradings to be unduly severe. Table III shows the proportions of sets of results falling into each of these categories for Surveys A-D. It was thought that to grade the sets of results on the basis of assays against the working standards which laboratories would normally use for thenassays gave the fairest representation of their likely performance of clinical assays.
Quality control of serum gentamidn assays
111
of the method, doubling dilutions in broth gave the worst results overall as it is impossible to attain consistently an accuracy of ±25% with a method which provides answers as discontinuous variables in steps of
Quality control of serum gentamicin assays—experience of national surveys
D. S. Reeves and M. J. Bywater
The increasing use of the aminoglycoside antibiotic gentamicin together with an awareness of the importance of adequate serum levels for therapy and of a possible relationship between those levels and its main side effect, ototoxicity, has led to many laboratories being asked to perform gentamicin serum assays. The relatively low therapeutic ratio of the aminoglycosides antibiotics indicated that accurate assays were essential if they were to realize their full potential in controlling therapy. With this in mind four laboratories, our own among them, exchanged some specimens of sera containing gentamicin in late 1971 and early 1972. The results showed wide divergencies in the estimated concentration of single samples by the participant laboratories. This was clearly a worrying state of affairs and recognizing this, the British Society for Antimicrobial Chemotherapy (B.S.A.C.) sponsored a number of larger surveys. In 1973 this laboratory was invited by the Public Health Laboratory Service (P.H.L.S.) Quality Control Committee to send out specimens for antibiotic assay. The experiences described here are those of the later B.S.A.C. surveys and those under the joint auspices of the P.H.L.S. and B.S.A.C. in 1973 and 1974: For convenience of reference the surveys are designated as follows: January 1973 (B.S.A.C.) —Survey A March 1973 (B.SA.C.) —Survey B June 1973 (P.H.L.S./B.S.A.C.) —Survey C (P.H.L.S. 20th Distribution) February 1974 (P.H.L.S./B.S.A.C.)—Survey D (P.H.L.S. 28th Distribution) Gentamicin was chosen as the antibiotic to be investigated for the reasons given above. Furthermore, it was found to be by far the most frequently assayed antibiotic in a small poll taken among participant laboratories, and it seemed to be a technically suitable drug because of its stability in solution at room temperature. Objectives The circulation of sera containing known concentrations of gentamicin for assay by participant laboratories and the receipt of their results would allow an assessment to be made of the prevailing accuracy of assays. By obtaining methodological details it was hoped to identify any technical factors having a bearing on accuracy. The circulation with the specimens of a standard solution of gentamicin for use in the preparation of working standards was aimed at determining whether laboratories' usual source of standard material was adequate. Finally it was hoped that feedback of results to individual laboratories would help them to identify deficiencies in their performance and correct them. 103
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Department of Medical Microbiology, Division of Pathology, Southmead Hospital, Bristol, BS10 5NB, England
104
D. S. Reeves and M. J. Bywater
Distribution of samples Sera was distributed into individual bottles using a mechanical dispenser. One of the main problems was finding a suitable container. Polystyrene containers were the most durable in the post but tended to leak around the screw cap. Eventually glass containers (approx. 5 ml size) with plastic caps and card liners, sterilized by sub-atmospheric steam with formaldehyde, were found to be the most suitable. Containers were carefully packed in approved postal boxes along with the relevant documents. These consisted of a description of the samples requesting the investigation required (a typical version is shown in Figure 1), an assay report form (Figure 2), and a method report form (Figure 3). The reporting forms (Figure 2 and 3) are shown in their final form, modified in the light of experience. Provided standard material In the later surveys participants were provided with a stock standard solution of gentamicin. \ They were told to assume the potency was 1000 ng/ml of gentamicin base and report their results accordingly. In fact, these solutions were deliberately prepared • Donated by Nicholas Laboratories Ltd. t "Accuiamatic"—Jencons Scientific Ltd. \ Made and ampouled by courtesy of Nicholas Laboratories Ltd.
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Methods and results Preparation of samples The circulated samples were prepared in pooled human serum obtained from healthy donors individually and regularly checked for the presence of serum hepatitis antigen and antibody. All individual samples were also checked for the presence of antibiotics in a highly sensitive plate assay system employing Bacillus subtilis as an indicator organism. Powdered gentamicin sulphate of known potency* was dried at 100°C for 180 min and allowed to cool in a desiccator. An amount exceeding 100 mg was quickly weighed on an Oertling balance capable of weighing accurately to 0-1 mg and transferred completely to a volumetric flask. Sterile Grade A glassware was used throughout. Solution was effected with sterile distilled water made up to the mark. An accurately measured aliquot, usually 10 ml, was pipetted into a second volumetric flask and made up to the mark with pooled human sera. After mixing this solution it was dispensed in 2-2 ml lots into the individual sample bottles for circulation to the participant laboratories. A typical example of this preparation was as follows: 2520 mg of powder of a potency of 635 (Ag/mg and thus equivalent to 160 mg of gentamicin base was weighed out and dissolved in 200 ml of water. 10 ml of this solution was transferred to a 1000 ml volumetric flask and made up to the mark with serum resulting in a final concentration of 8 (J-g/ml. All values for potency in this paper are expressed as gentamicin base. Thus, by appropriate modification of the general method, each sample of gentamicin solution could be accurately prepared by using large volumes, illustrating one of the advantages of having many participant laboratories. A particular problem was maintaining solutions in serum free of contamination. In one circulation there were a few complaints about contaminants, mainly from laboratories employing doubling dilution assay techniques. Subsequent to this all serum was obtained under sterile conditions, all glassware was sterilized prior to use, and the individual samples were prepared by a roller-pump dispenserj using plastic tubes sterilized in a sub-atmospheric steam-formaldehyde injection autoclave.
Quality control of serum gentamicln assays
DISTMBUTIQI - Vttek
105
Ing l 8 / ; / 7 4 .
Antibiotic Sera and solution* issued by Dr. D.S. Rcevfts, Department of Bacteriology, Soutimwad Itospital, We.iUnii-y-on-Trym, B r i s t o l . BS1O 5*8.
Six i H p l u of Mr* (Labelled 9fl, 100, 101, 102, 103 and 104) containing gentamicin ir« tncloMd. Scmplma 99, 100, 101, 102 and J.03 contain no other antibiotics but It should be asauetd that sample I04 comes fro* a patient who i* simultaneously receiving lincomycin. Although some samples «*y Sir* similar assay T i h u they ar« not necessarily Identical. No value exceeds 20 ug/»l or i s leas than 0.5 ug/ml.
Tb« samples sboold be assayed and reported twice usin( your usual —thod i1.
Uainx standards you already hero or would normally prepare on receipt of such apecisMns}
2.
Using standards prepared from the enclosed ampoule (A) of geotamicin eolation. Tou •hould assume the coocentration of this stock standard to be 10OO usy«l of (entamicin base. *l*asa report the results as if tbe assumptioo Is valid, although i t may not be exactly s o .
Tbe results on Form A and details of tbe method oo Form » should be s«nt by f i r s t class mall in an envelop* marked •Confidential" to i Dr. 0.8. Xs*ve« at the above address. Completed forme should be returned not later than 11th March, 1974.
Figure 1. Typical instructions accompanying a set of quality control samples. LAB. 1DEVTITT HO. ( P l e a s e Check)
BSAC QUALITY COimiOL - FEB. 1974 DI5TEIBUTICni AASAT RCFOrt (CDfTAMICUl) STAntAIDS - OWf
STAjnUjtm - A Calculated potency (us/ml) S
Individual IOM als^s *
Clcul.t^l potency (m^ml) 8
99 1OO 101 102 103 104
wm (««/-i>
*
Flease enter tbe unit of Measurement, m.g. diameter in mma, arbitrary u n i t s , etc. each sample and standards - no\ Just wean «one s i t e .
S
Flease glve results to eam declmal p l a c e .
8TAJRURDS OVD
DATE Of ASSAT
Enter f 11 aome s l s e s for
TIKI or AS SAT TO Dete specimens received
Dete of results
A Flease return tbe completed fora. #long with Form B i * l i t y C o n t r o l Fata. 1 9 7 4 . ASSAT KETIWO (GEJCTAHICIjO BPECIKETS 0 - 104 1
BASTC
2
HEDttm
Vertical diffusion Otber (daftue)
Plato diffu aioo Doublina d i l u t i o n In broth Attor Typo VOIOM* per pi*to Di-a^h Type
Pil •1..
Volume/Tube SDOO1>«
Strain Dtraaltr of viable count Surfaco or inoonwratad Stated potancy
4-
STATOARIB
5
PREP. OF
• aat Dllutod
fdiluent
6
METHOD Or
Cut WeiLa
(di^wetor
Cut* "SDinea-
falsa
Source of atatarLal r i n a l Dilueat
(Batch Vo.
)
PH
(else Fixed
7
ORDER OF APPLICATION
S
ftuafcer
COLD PREDIPFTJSIOM
10
ISCUBATIOM
11
KETBOO Or READING
11
HETBOO 07 CALCOLATIOM
of
Standard*
Tea
Art>itrarr
StandartU rirac
Randoai
9
)
(Includin* dllutioneT
Sample*
lo
Baaipio* r i r s t
Timm (hra)
)
TC-P.
Te«p
Tiaw
Callpars (T tYi»e Rolar (T typa Optical Optical an toaated (7 type Other
i 7 varnier
) ) )
O m E * USEFUL DETAILS IXCLUTUKO OOWtEMTS OM PRE5EJCCB OF UKCOWTCTU IV BPIClWlH 1QA
Figure 3. Method report form.
Stability of antibiotics in post Obviously it is important that the concentration of antibiotic in the samples when they reach the recipient laboratories should be that originally made up. Any deterioration would cast doubts on the validity of survey since it would be an indeterminable variable due to different times and conditions of transit. This was checked by posting extra sets of samples either to ourselves or to other laboratories and asking them to return them unopened. In this way we received sets of samples which had been in the post either once or twice. Assay of these samples when using gentamicin alone never showed any detectable deterioration of potency. Mixtures of antibiotics Two antibiotics, carbenicillin and lincomycin (or clindamycin), are frequently administered concurrently to gentamicin and it was therefore appropriate to investigate how they might interfere with gentamicin assays. Carbenicillin. In one circulation (Survey B) participants were sent gentamicin (4-5 |xg/ml)
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
INDICATOR OSCAKJSM
d •Marcial batch Mo. Siie t f plate t Sia« of tube
Quality control of sermn gentamldn assays
107
and carbenicillin (1000 ng/ml) in serum as a mixture prepared immediately prior to dispatch. They were also sent separate solutions of both antibiotics that when mixed in exactly equal proportions gave the same concentration of gentamicin and carbenicillin as in the pre-mixed sample. They were asked to assay both mixtures for gentamicin simultaneously. The results are shown in Table I. Even allowing for the large variances the results seem to show clearly that the presence of carbenicillin in posted samples of gentamicin could lead to a serious underrating of the potency of the gentamicin. This effect is presumably due to the temperature dependant inactivation of gentamicin by carbenicillin (Mclaughlin & Reeves, 1971) during transit. Table I. Influence of carbenicillin on gentamicin assay
Value as made up Mean of assay results (21 values) 2 S.D.
Mixed prior to dispatch
Mixed on receipt
4-5
4-5
2 0 (2-3)* 2-2 (3-2)
4-6 (5-1) 4-2 (4-8)
* Results in brackets are those obtained with working standards prepared from standard material provided.
Lincomycin {clindamycin). In view of the apparent inactivation of gentamicin by carbenicillin, a study was made to see if lincomycin or clindamycin would inactivate gentamicin. Solutions of gentamicin in serum or buffer were prepared to contain concentrations of lincomycin or clindamycin typically found in clinical therapy. After incubation at three temperatures (4°, 20° and 37°C) and for various lengths of time, up to 3 days, samples were assayed. Gentamicin was assayed by a microbiological plate assay using a Klebsiella spp. as indicator organism. This assay was unaffected by lincomycin or clindamycin at a concentration many times higher than those used in the experiment. Lincomycin and clindamycin were assayed by bio-autography after separation from gentamicin by high voltage electrophoresis in agar. The results showed that there was no detectable inactivation of any of the antibiotics either when present alone or in combination (Holt & Reeves, unpublished data). Collection and analysis of results Those participant laboratories involved through the P.H.L.S. returned their report and method forms to Mr W. B. Fletcher at Central Public Health Laboratory, Colindale, London, NW9 5HT. The remaining laboratories were members of the B.S.A.C. surveys and returned their forms to this laboratory. All the results were collated and analysed by a team including Mr Fletcher, Mr Alan Perkins (biometrician, lately of Nicholas Laboratories Ltd) and the authors. Since laboratories were identified by code numbers known only to the organizers confidentiality was maintained. Analyses were usually done manually, including inter-survey comparisons. The results of Survey D were stored on tape files in a Hewlett-Packard 98 30-A programmable calculator. The statistical parameters for each laboratory were then calculated automatically and the data stored for comparisons with future surveys. The primary statistic calculated for each individual assay result was the percentage
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Assayed potencies of gentamicin (ng/ml)
108
D. S. Reeves and M. J. Bywater
Feedback of results to participants
Shortly after the date set for the receipt of reports from the participants, a circular was issued giving the true values of the samples and the potency of the issued standard. A typical circular is shown in Figure 4. Later a more detailed report was issued giving the results and their statistical for breakdown for each laboratory, and many of the type of tables shown in this paper. This report also contained detailed comments on the results. MAC QCALITT CONTROL, p p . 1974 CPTAJUCIH ATOAT
TIM MI-« eontainad tha following eoocentratla (axpraaaad a* • • n t M l c l n basa)lt» 100 101 101 101 104
6.
of (ootmlels
5
) ) i 1 1. i i >
- 10. 5 1.
-
«. 6. 5
)
99, 103 and LO4 ca•a fro. tb. aa«. « U S-pl~ 104 bad aolld ollndai• j c U addad to | l n a final lAataljr 5 «*/•!.
of approx-
Staodaj-d 'A' contained 1050 o«/al of (aataadcU u d IO, If uaad aa dlractad, tha aaaay r«*ulta afcoald bars b«ao it laaa ( l . a . 6.2, 10.0, 1.4, 1.1, t . l , and 6.1 rwp.rtl-r.lr, to tw> almlflcaot fi^uraa).
Figure 4. Circular giving true results of sera to participant laboratories
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
error from the "true" result The "true" result was taken as the value of the test sample as made up, making an allowance where necessary for the potency of a supplied standard solution. In Surveys C and D six samples were issued, each to be assayed twice. The mean percentage error and the standard deviation (S.D.) of the percentage errors were calculated for each set of six results. The statistic of the modulus of the mean percentage error (|mean|)+2 S.D. was used to grade laboratories performances on the basis of the results using their own standards. Because the statistic |mean|+2s.D. is fundamental to the analyses given in the results, an illustration of its derivation from a hypothetical set of results follows: on a set of six sera of true concentrations of 6-5, 10-5, 2-5, 3-25, 6-5 and 6-5 ug/ml of gentamicin a laboratory reported values of 6-2, 11-6, 30, 30, 6-5 and 6-5 ug/ml, respectively. The percentage error of each result was calculated by (true-reported/true) x 100 giving values of —5, +10, +20, —8, 0 and 0%, respectively. The mean of these values was taken as the "mean percentage error", and their standard deviation was also calculated. The final statistic taken as an assessment of performance on the basis of the set of results was the modulus of the mean percentage error (i.e. the mean without its sign) plus 2 S.D. The percentage errors of each set of results were assumed to fall into a normal distribution. This assumption was found to be best fulfilled by converting the data to a geometric scale (e.g. to logarithms), but the arithmetic data was shown to fit a normal distribution well enough for the purposes of the analysis. In Surveys A and B the statistics were in fact also calculated on the basis of logarithmically converted reported results and this form of analysis was repeated in Survey D. No significant information accrued additional to that already known on the basis of arithmetic analysis of results. This form of analysis was therefore not pursued.
Quality control of serum gentamidn assays
109
Results on individual specimens The overall results for 20 specimens assayed for gentamicin in 4 surveys are shown in Table II. The statistics were calculated in a similar manner to that for sets of assays from individual laboratories, but based instead on all the reported results for each sample. By
A A A A A
6
B
O
s s s
o o o o o
7
B
8
B
67
C
68
C
69
C
70
C
71
C
72
C
99
D
s u
op op
s
o
s s s s s s
p
o op op op op p
o p
100
D
101
D
102
D
103
D
104
D
1
s s s s s
op op op op op
— — — — —
— — — — — — —
- 5-8 - 20 - 6-7
+ + + + +
46 56 46 67 50
-
50 48 55 46 54
53-8 50-0 62-7
- 4-2
24 24 28 37 29
450-0 4500
22 20 22 20 18 19
- 4-4 - 4-4 + 2-2 + 6-7 - 91 -13-6
26 23 35 45 60 64
+ + + + + +
64 56 122 167 117 122
-
49 51 38 36 78 78
56-4 50-4 72-2 96-7 1291 141-6
4-5 4-1 5-5 50 5-5 50 110 100 4-5 4-1 5-5 50
79 76 80 77 79 77 81 77 80 76 79 77
- 6-7 - 4-9 -12-7 - 20 -14-5 -14-5 - 6-4 -111 - 6-8 -110 - 40
62 62 69 71 66 74 55 46 69 62 51 48
+ 360 + 267 +409 + 334 + 355 + 300 + 300 + 210 +433 + 322 + 278 + 198
- 93 - 98 -100 -100 - 91 - 90 - 88 - 93 - 89 - 83 - 85 - 80
130-7 128-9 150-7 1440 146-5 1480 124-5 98-4 1491 130-8 1130 1000
6-5 6-2
86 87 87 88 85 86 87 87 87 88 87 86
+ + + + + +
30 27 34 32 38 45 34 41 29 35 31 36
+ 115 + 102 + 90 + 100 + 160 + 200 + 131 + 158 + 85 + 158 + 146 + 158
-
11-2 2-4 2-3
10-5
3-25 — L L
Maximum Minimum |Mean| result result + 2 S.D.
19 19 19 19 19
100 2-5 2-4 —
errors from true result
S.D
4-5 4-5 90 90
p
/o
|Mean
120 4-9 — — —
0/
No. < resul
S S S
Othe r drugs81
Stanc mate
1 2 3 4 5
True* result (|ig/) ml
31 6-5 6-2 6-5 6-2
00
0-0
90 0-9 3-7 9-9 6-3 1-5 5-3 2-6 4-7 2-7 91 2-2
66 73 66 70 80 71 85 74 66 77 69 68
740
62-2
690
54-9 71-7 73-9 82-3 91-5 73-3 83-6 62-7 72-7 711
74-2
S=samples in pooled human serum; U=samples in pooled human urine, adjusted to pH 5-5. • O=working standards, as prepared for routine assays; P=working standards prepared from provided standard material. • L=sample contained clindamycin, 5 |ig/ml. • Potency of samples as made up, making allowance for potency of provided standard material where necessary.
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
1
1 11 Spcci
Samr. ;no.
Table IL Overall results for individual specimens as assayed by participant laboratories
110
D. S. Reeves and M. J. Bywater
showing the overall performance of participant laboratories for each specimen it is possible to obtain an estimate of the typical performance of the laboratories. Naturally, this form of analysis does not identify the quality of performance of individual laboratories.
Table IEL Numbers of sets of results falling into categories of quality on basis of assays made with the laboratorys' own standards -
N
„ Survey
' ^era
No. of laboratories
A B
5 2
19 20
C D
6 6
81 92
Number (and %) of results in each category . . ^ misleading 1(5%) 10(53%) 8(42%) Insufficient serum samples for meaningful analysis 17(21%) 29(36%) 35(43%) 17(18%) 29(32%) 46(50%)
In Surveys C and D comparison was made between the gradings obtained in this manner and gradings obtained by assay against the standard gentamicin solution provided (Table IV). It can be seen that there was a strong tendency for laboratories to reproduce their quality of performance whichever standards were used. In particular a movement from "Highly misleading" to "Good" or vice versa on changing standards was very uncommon. Quality of results related to basic method used Sufficient laboratories participated in Surveys C and D to make meaningful analysis of methodology possible (Table V). As might be expected from a theoretical examination
Downloaded from http://jac.oxfordjournals.org/ at Simon Fraser University on June 4, 2015
Quality of performance by individual laboratories As explained above, a statistic (|mean|+2 S.D. of percentage errors) was used to grade the performance of individual laboratories for each set of specimens. On the basis of the requirements for accuracy of gentamicin assays in the clinical context for controlling dosage the laboratory results were graded as follows: 'mean + 2 S.D. 50% — "Highly misleading" mean All three gradings may be illustrated by taking the results of laboratories assaying a hypothetical sample containing 20ug/ml of gentamicin. Assuming that the mean percentage error is negligible, then a "Good" set of results would be in the range 1-5 to 2-5 ug/ml about 19 out of 20 times, a "Poor" set in the range 10 to 30 ug/ml, and a "Highly misleading" set in the range < 1 0 - > 3 0ug/ml. In every case, the remaining 1 result out of 20 would be expected to fall outside the quoted ranges. In view of the fact that there appears to be a crucial clinical distinction between pre-dose levels of 1-5 and 3-0 ug/ml (Nordstrom, et al., 1973), we did not feel our gradings to be unduly severe. Table III shows the proportions of sets of results falling into each of these categories for Surveys A-D. It was thought that to grade the sets of results on the basis of assays against the working standards which laboratories would normally use for thenassays gave the fairest representation of their likely performance of clinical assays.
Quality control of serum gentamidn assays
111
of the method, doubling dilutions in broth gave the worst results overall as it is impossible to attain consistently an accuracy of ±25% with a method which provides answers as discontinuous variables in steps of
