Novel Approach for Assessing Performance of PCR Cyclers Used for Diagnostic Testing
http://www.100md.com
微生物临床杂志 2005年第6期
Institute for Milk Hygiene, Milk Technology and Food Science
Institute for Medical Physics and Biostatistics, Veterinary University, Veterinrplatz 1, 1210 Vienna, Austria
Danish Institute for Food and Veterinary Research, Bülowsvej 27, DK-1790 Copenhagen, Denmark
Congen Biotechnologie, Robert-Roessler Strasse 10, 13125 Berlin, Germany
ABSTRACT
As part of a large international project for validation and standardization of PCR, the influence of thermocyclers on PCR was tested. Six brand-new, Peltier technology-driven 96-well thermocyclers were subjected to a novel and stringent in-tube (not block) physical testing. The temperature was directly monitored in PCR tubes containing 50 μl of distilled water at 13 different block positions. The certified temperature accuracy of the measurement system was ±0.3°C. Finally, the results of the physical testing were compared to those of an amplification efficiency study running an in-house PCR assay. The cyclers did not perform within the manufacturer's specification. Premature timing, under- and overshooting, and spatial variation of heat transfer were found to be the critical factors. The physical testing allowed us to distinguish accurate from less-accurate (2/6) cyclers. The lack of thermal homogeneities became most evident at the denaturation level during the first 15 s. At the time point zero, the accurate cyclers showed temperature deviations of 0.5 to 1.5°C, whereas less-accurate cyclers failed to reach the set temperature by 13 to 20°C. Consequently, the two less-accurate cyclers could not gain positive PCR results by running an in-house PCR assay. However, by modifying the original temperature protocol by increasing the denaturation temperature and time, the amplification efficiency of these two cyclers could be improved significantly. The results have implication for laboratories using diagnostic PCR testing.
INTRODUCTION
The breakthrough of PCR has been primarily due to the development of recombinant, thermostable DNA polymerases (15) and the development of thermocyclers. The cycler enables the automation of this repetitive thermal process. It should guarantee temperature uniformity for all samples and ensure a rapid heat transfer from the heating block to the in-tube sample liquid.
Although a convincing method, PCR needs further standardization, since a large variety of parameters can influence the outcome of PCR amplification (7). PCR protocols generated by one lab are hardly reproducible by another (1, 2, 20). Even under identical assay conditions, it was shown that there are difficulties in obtaining reproducible results (16).
One of the reasons might be the influence of the thermocycler on amplification efficiency. Despite the striking importance of PCR, the literature on thermocyclers is scarce. Some studies were published on the first generation of cycler models (5, 10, 12, 14, 17). Others determined the amplification efficiency but not the physical properties of thermocyclers (7, 16). According to an interlaboratory study, inappropriate cyclers that produced the most nonrepeatable data were older than 4 years, had no heated lid, and were run under block control (16). The physical functionality of the suspected cyclers, however, was not determined.
In a recent study, brand-new thermocyclers were subjected to a physical testing (19). The evaluation of their temperature profiles allowed the distinguishing of accurate from less-accurate cyclers. Premature timing was seen to be the main problem. In the case of one cycler, the sample temperatures were even 20°C below the set temperature. However, the consequences of these shortcomings on the PCR results were not determined.
It was the goal of this study to compare the physical properties of the latest generation of brand-new thermocylers, to demonstrate the impact of physical inaccuracy on amplification efficiency, and to point out the critical factors in the reliability of PCR performance in order to pave the way for preparation of an international standard.
The present work was initiated within a European project dealing with standardization of noncommercial PCR assays for the detection of major foodborne pathogens (www.pcr.dk).
MATERIALS AND METHODS
Terminology. The following expressions are repeatedly used. (i) Block uniformity (BU) refers to the temperature homogeneity within the 96-well block and is given by the manufacturer (Table 1). (ii) In-tube temperature (ITT) is the temperature measured in the PCR tubes at 13 different block positions. (iii) Set temperature (Tset°C) corresponds to the programmed temperature. (iv) Tcycler°C is the temperature which is indicated on the cycler's display. (iv) Maximum temperature (Tmax°C) describes the maximum value, minimum temperature (Tmin°C) the minimum value, and mean temperature (Tmean°C) the mean value of the 13 ITTs. (v) Sample uniformity (SU) refers to the temperature homogeneity of the 13 ITTs (Fig. 1); it describes the difference between Tmax and Tmin. (vi) Deviation (Dv) describes the difference between the ITT and Tset (Dvhigh = Tmax – Tset; Dvmean = Tmean – Tset; Dvlow = Tmin – Tset). The deviation indicates the tendency for overshooting (higher than programmed temperature; positive Dv values) or undershooting (lower than programmed temperature; negative Dv values). (vii) t0s (time point zero) describes the beginning of the onset of timing. It is the time point when Tcycler°C equals the set temperature. (viii) txs is the time point of measurement after x seconds from the onset of timing. (ix) Nonamplification means the absence of a detectable PCR product.
Instruments and programming. Brand-new thermocyclers (lettered A to F) from six major suppliers were selected for this performance study. Since the 96-well instrument is the most-used model, this block format was chosen for the experiments. Out of three options to control the block's temperature (block control, in-sample probe control, and calculated temperature control), the calculated temperature control mode was chosen if available (Table 1). All experiments were performed by setting the lid temperature to 105°C and using the maximum heating rate available.
Evaluation of physical performance. The temperature measurement followed the technical procedure of a recently published thermocycler performance study (19). The temperature was measured in 0.2-ml PCR tubes (MicroAmp, Applied Biosystems, Foster City, CA) containing 50 μl of distilled water. Fast-response microthermocouples of type T (RS Components GmbH, Gmünd, Austria) and the data logger 263A Data Bucket (Fluke Cooperation, Everett, WA) enabled the data collection. Temperatures were measured at 13 different block positions (Fig. 1). The temperature relative unit was calibrated in accordance with the law concerning legal metrology, which documents the traceability to national standards. The certified temperature accuracy and reproducibility were ±0.3°C and ± 0.1°C, respectively.
Each thermocycler was programmed to perform four static temperature protocols. The temperature protocols included a prerun phase at 26°C for 60 s followed by the heating of the cycler block to a set temperature of either 95°C, 72°C, 55°C, or 39°C. This temperature was then held for 300 s.
Evaluation of the amplification efficiency. A recently developed PCR assay was used to test the amplification efficiency (6). PCR was performed in a 50-μl reaction mixture containing 0.6 U of AmpliTaqGold polymerase (Applied Biosystems, Foster City, CA). A batch of master mix was prepared, and aliquots of this batch were used for all the experiments. Four nanograms of DNA of L. monocytogenes strain EGD (4) served as a template. The DNA concentration was measured with a fluorescence photometer (Hoefer DyNA Quant 200; Pharmacia Biotech, San Francisco, CA). Samples were loaded at four corner positions (A1, A12, H1, and H12) and one central position (D4). In addition, the control reactions were amplified using an in-house cycler model (GeneAmp 2400; Applied Biosystems, Foster City, CA). The PCR amplicons were separated in a 1.5% agarose gel using ethidium bromide staining. The intensity of the DNA band in the gel was assessed optically. Each temperature protocol comprised an initial denaturation step at 94°C for 600 s and a final extension step at 72°C for 300 s. The cycling conditions as published were as follows: 94°C/15 s, 68°C/30 s, and 72°C/60 s (for 45 cycles; PCR Liststand). The protocol was also used with some modifications as follows (i) PCR Listmod1 (96°C/15 s, 68°C/30 s, and 72°C/60 s), (ii) PCR Listmod2 (94°C/60 s, 68°C/30 s, 72°C/60 s), and (iii) PCR Listmod3 (96°C/60 s, 68°C/30 s, and 72°C/60 s).
RESULTS
Physical functionality of six thermocyclers. (i) Spatial influence: SU versus cycler-specific BU. The thermocyclers had 300 s to establish temperature uniformity. At time point t300s, the SU was calculated and set into relation with the block uniformity as given by the manufacturer for each cycler model (Table 1).
Out of the six cyclers tested, five cyclers showed an SU which exceeded the BU irrespective of the temperature value chosen. Only cycler A did not exceed the specified BU limit of ±0.5°C (Fig. 2).
In all cycler models there was a tendency for overshooting when lower block temperatures (55°C and 39°C) were used. All cyclers showed positive Dv values. The Dvhigh value was highest in all machines at 39°C (Fig. 2). Overshooting became most clear in cyclers D and E, showing a Dvhigh of +1.8°C and +1.4°C, respectively. On the contrary, Dvlow became negative in some cyclers at higher block temperatures. This phenomenon was most obvious in cycler F, resulting in a Dvlow of –1.6°C at 95°C.
Irregularities of the SU followed the loading pattern (spatial distribution) of the samples. PCR tubes loaded into the inner positions usually revealed the Tmax°C values, whereas samples loaded into the outer positions revealed the Tmin°C value. This was most evident in cyclers C and F (data not shown).
(ii) Temporal influence: determination of the SU and deviation of ITTs at time points t0s, t15s, t30s, and t60s after onset of timing. Generally, the dependence of thermal variations on timing became most evident using the 95°C protocol. Therefore, the data are shown in more detail (Fig. 3). At time point zero (t0s), the SU varied from 1.5°C (cycler A) to 7.9°C (cycler F) and 10.7°C (cyler C). After 60 s (t60s), the SU was narrowed down to 0.9°C, 1.6°C, and 2.3°C in cyclers A, C, and F, respectively (Fig. 3).
Cycler A most accurately met the set temperature, 95°C, at time point zero (t0s), showing the most balanced deviation (Dvlow and Dvhigh, –0.4 and +1.1). On the contrary, the Dvlow in cyclers C and F failed to reach the set temperature by –19.7°C and –12.8°C at t0s, respectively. These differences were narrowed down to –1.8°C and –2.2°C at t60s. From these data it was expected that fast cycling protocols, in general, especially those showing a short denaturation phase, have a higher risk of nonamplification.
Comparison of the cycler’s amplification efficiency using an in-house Listeria PCR assay. An in-house Listeria PCR assay (PCR ListStand) was used to test the amplification efficiency. Cyclers A, B, D, and E attained good results, while cyclers C and F showed no amplification in the five positions tested (Fig. 4).
Upon shifting the denaturation temperature to 96°C (PCR Listmod1), the amplification efficiency of cycler C but not of cycler F was improved. Cycler C amplified the product at four positions, except for position H12, whereas cycler F did not produce a visible band. Only by expanding the denaturation time from 15 s to 60 s (PCR Listmod2) was it possible to achieve consistently good amplification for all cyclers at all five positions (Fig. 4). Upon shifting the denaturation temperature to 96°C and expanding the time from 15 s to 60 s (PCR Listmod3), all five positions in cycler C and F showed clear bands, but the amplification efficiency of cycler E dramatically decreased. Cycler E showed no amplification in all five positions tested (data not shown).
DISCUSSION
Soon after the introduction of the first commercial instrument in 1987 (Perkin-Elmer Cetus DNA thermal cycler), other suppliers offered similar devices. Today the thermocycler market is dominated by a few major manufacturers. Thermocyclers of the previous generations differ widely from the contemporary models. Developments of heating/cooling systems (circulating water versus Peltier technology), internal temperature control options (block control versus in-sample probe control and calculated temperature control), and evaporation control concepts (oil overlay versus heated lid) have been changing the technical architecture of the cyclers completely (15, 16). Accurate thermocyclers are a prerequisite for advanced applications such as cycling sequencing, in situ PCR, and real-time quantitative PCR in sample formats of up to 384 block positions. All these techniques rely on (i) an efficient heating and cooling system, (ii) an efficient heat transfer from the block to the sample liquid, and (iii) a sophisticated steering algorithm which allows the reproducibility of a given set temperature profile as stringent as possible.
The performance studies on thermal cyclers published so far lack relevance (5, 8, 14, 21). On the one hand, the studies refer to the older generation of thermocyclers, or processors from only one or two manufacturers were tested (5, 8, 21). Interlaboratory trials, on the other hand, compared only the amplification efficiency of used cyclers of various makes and models (16). The physical functionality of the machines was not determined.
This study focused on brand-new cyclers of the last generation. By embedding a fast-response microthermocouple inside the PCR tube, it was possible to determine the temperature uniformity within the sample block and to study the thermodynamic process from the block via the tube to the PCR sample in more detail. If available, the calculated temperature control mode, which is known to bring the samples more easily to the programmed temperatures (16), and each maximum heating rate had been chosen. By performing a PCR assay (PCR ListStand), the amplification efficiency of each model was determined.
Both physical testing using a static protocol and PCR testing allowed us to distinguish accurate (A, B, D, and E) from less-accurate (C and F) cyclers. This was also shown using a dynamic three-level protocol in a recent study (19). The most obvious irregularities of the machines were premature timing, followed by under- and overshooting and spatial irregularities of heat transfer. These results indicated that a difference in the cycler performance is not only a matter of abrasion after years of use, as suggested by previous studies (7, 16). It is an outcome of the variable technical architecture of the cyclers, which differs in the number, size, and quality of the Peltier elements, the location and number of the thermosensors steering the Peltier elements, and the underlying algorithm. A difference in the ITT from the block temperature might also result from poor heat transfer from the block to the PCR sample. In particular, the models which were notable for high heating and cooling rates seemed to be "susceptible" to premature timing.
The short denaturation step of PCR ListStand was seen to be the critical point. The cyclers C and F failed to amplify the target DNA. The physical testing of cyclers C and F showed a significant tendency for undershooting and premature timing. Usually the optimum denaturation temperature is between 91 and 94°C (11). Referring to the PCR ListStand protocol (94°C/15 s, 68°C/30 s, and 72°C/60 s), cylers C and F performed the effective temperature level only for 5 s (33%) or 7 s (47%), respectively (data not shown). Consequently, we concluded that the lack of amplification of cyclers C and F was most likely caused by the insufficient melting of the template DNA. This hypothesis could be proved by modifying the original denaturation step by increasing the temperature (PCR Listmod1), time (PCR Listmod2) or both (PCR Listmod3). The amplification efficiencies of cyclers C and F could be improved significantly. The best results could be obtained by expanding the denaturation time from 15 to 60 s (PCR Listmod2). This was seen as a proof that premature timing was the main reason for the lack of amplification of cyclers C and F.
By increasing and expanding the denaturation step up to 96°C and 60 s (PCR Listmod3), the tendency of cycler E for overshooting became most evident. Its amplification efficiency decreased dramatically, perhaps due to polymerase inactivation or deoxynucleoside triphosphate breakdowns (3, 22, 23, 24).
In conclusion, this is the first time that the pivotal influence of the thermocycler on PCR efficiency has been studied in detail. It is well known that PCR protocols need to be optimized for different instruments and that a cycling program that works on one brand of instrument may not necessarily work on another. It is the physical testing of cyclers that helps to explain why an adoption of the original temperature protocol leads to an amplification in one cycler and lack of amplification in another. The performance of various modifications of the original PCR assay protocol (PCR Listmod1 to PCR Listmod3) demonstrated that PCRs run with inaccurate machines do not necessarily result in lack of amplification. However, an extended experimental expenditure is required to match a PCR protocol with the technical idiosyncrasies of such cyclers.
Each PCR user should have an effective and simple tool to test the performance of in-house cyclers. Several cycler companies developed end-user-friendly temperature validation systems, such as the Temperature Verification system (Applied Biosystems, Foster City, CA) or the Temperature Validation system (Eppendorf Netheler-Hinz GmbH, Hamburg, Germany). These single-sensor test systems allow static temperature measurement only within the block wells. Our study could show that static temperature procedures, on condition that they are done in-tube, are appropriate to detect some of the main weak points of thermocyclers (19). Another approach could be a cycler performance test using a standardized temperature-sensitive PCR system, such as SureCycle (Congen, Berlin, Germany), which was recently developed and introduced into the market (9). Cycler performance testing should regularly be carried out and become a part of any accreditation of an expert or an end-user lab devoted to PCR diagnostics. New PCR strategies should demonstrate the robustness of a temperature protocol, and validation of PCR protocols must include the testing of a PCR assay on several thermocyclers.
ACKNOWLEDGMENTS
This work was supported by EU grant no. QLK1-CT-1999-00226.
We thank Kurt Wimmer (Institute for Medical Physics and Biostatistics, Vienna, Austria) for his assistance in construction of the temperature measurement unit, Jose A. Vazquez-Boland (Grupo de Pathogenesis Molecular Bacteriana, Universidad de Leon) for the provision of L. monocytogenes strain EGD, and finally, Martin D’Agostino and Nigel Cook (Defra CSL) for carefully reading the manuscript.
REFERENCES
Bootman, J. S., and P. A. Kitchin. 1992. An international collaborative study to assess a set of reference reagents for HIV-1 PCR. J. Virol. Methods 37:23-41.
Defer, C., H. Agut, A. Garbarg-Chenon, M. Moncany, F. Morinet, D. Vignon, M. Mariotti, and J. J. Lefrere. 1992. Multicenter quality control of polymerase chain reaction for detection of HIV DNA. AIDS 6:659-663.
Gelfand, D. H. 1989. Taq DNA polymerase, p. 17-22. In H. A. Erlich (ed.), PCR technology: principles and applications for DNA amplifications. Stockton Press, New York, N.Y.
Glaser, P., L. Frangeul, C. Buchrieser, C. Rusniok, A. Amend, F. Baquero, et al. 2001. Comparative genomics of Listeria species. Science 294:849-852.
He, Q., M. K. Viljanen, and J. Mertsola. 1994. Effects of thermocyclers and primers on the reproducibility of banding patterns in randomly amplified polymorphic DNA analysis. Mol. Cell Probes 8:155-159.
Hein, I., D. Klein, A. Lehner, A. Bubert, E. Brandl, and M. Wagner. 2001. Detection and quantification of the iap gene of Listeria monocytogenes and Listeria innocua by a new real-time quantitative PCR assay. Res. Microbiol. 152:37-46.
Hübner, P., E. Studer, D. Hfliger, M. Stadler, C. Wolf, and M. Looser. 1999. Detection of genetically modified organisms in food: critical points for quality assurance. Accredit. Qual. Assur. 4:292-298.
Kopp, D. W., C. A. Sansieri, and T. E. Mifflin. 1994. Routine monitoring of temperatures inside thermal cyclers blocks for quality control. Clin. Chem. 40:2117-2119.
Kuhn, M., S. Mergemeier, D. Schoder, and M. Wagner. 2002. Funktioniert der Thermocycler richtig LaborPraxis 4:80-83.
MacPherson, J. M., P. E. Eckstein, G. J. Scoles, and A. A. Gajadhar. 1993. Variability of the random amplified polymorphic DNA assay among thermal cyclers, and effects of primer and DNA concentration. Mol. Cell Probes 7:293-299.
Mai, M., R. Grabs, R. D. Barnes, P. Vafiadis, and C. Polychronakos. 1998. Shortened PCR cycles in a conventional thermal cycler. BioTechniques 25:208-210.
Penner, G. A., A. Bush, R. Wise, W. Kim, L. Domier, K. Kasha, A. Laroche, G. Scoles, S. J. Molnar, and G. Fedak. 1993. Reproducibility of random amplified polymorphic DNA (RAPD) analysis among laboratories. PCR Methods Appl. 2:341-345.
Reference deleted.
Rezendez-Perez, D., and H. A. Barrera-Saldana. 1990. Thermocycler temperature variation invalidates PCR results. BioTechniques 9:286-292.
Saiki, R. K., D. H. Gelfand, S. Stoffel, S. J. Scharf, R. Higuchi, G. T. Horn, K. B. Mullis, and H. A. Erlich. 1988. Primer-directed enzymatic amplification of DNA with a thermostable DNA polymerase. Science 239:487-491.
Saunders, G. C., J. Dukes, H. C. Parkes, and J. H. Cornett. 2001. Interlaboratory study in thermal cycler performance in controlled PCR and random amplified polymorphic DNA analyses. Clin. Chem. 47:47-55.
Schauberger, G., and M. Wagner. 1998. Qualittskontrolle von PCR Thermocyclern: überprüfung der rumlichen und zeitlichen Temperaturverteilung. Biomed. Engin. 43:216-218.
Reference deleted.
Schoder, D., A. Schmalwieser, G. Schauberger, M. Kuhn, J. Hoorfar, and M. Wagner. 2003. Physical characteristics of six new thermocyclers. Clin. Chem. 49:960-963.
Sheppard, H. W., M. S. Ascher, M. P. Busch, P. R. Sohmer, M. Stanley, M. C. Luce, J. A. Chimera, R. Madej, G. C. Rodgers, C. Lynch, et al. 1991. A multicenter proficiency trial of gene amplification (PCR) for the detection of HIV-1. J. Acquir. Immun. Defic. Syndr. 4:277-283.
Tweed, G. P., J. Whitney, and P. L. Bloch. 1991. Temperature cycler evaluation: what do you need to know BioTechniques 10:526-532.
Wittwer, C. T., and D. J. Garling. 1991. Rapid cycle DNA amplification: time and temperature optimization. BioTechniques 10:76-83.
Wittwer, C. T., G. C. Fillmore, and D. R. Hillyard. 1989. Automated polymerase chain reaction in capillary tubes with hot air. Nucleic Acids Res. 17:4353-4357.
Yu, K., and K. P. Pauls. 1992. Optimization of the PCR program for RAPD analysis. Nucleic Acids Res. 20:2606.(D. Schoder, A. Schmalwies)
Institute for Medical Physics and Biostatistics, Veterinary University, Veterinrplatz 1, 1210 Vienna, Austria
Danish Institute for Food and Veterinary Research, Bülowsvej 27, DK-1790 Copenhagen, Denmark
Congen Biotechnologie, Robert-Roessler Strasse 10, 13125 Berlin, Germany
ABSTRACT
As part of a large international project for validation and standardization of PCR, the influence of thermocyclers on PCR was tested. Six brand-new, Peltier technology-driven 96-well thermocyclers were subjected to a novel and stringent in-tube (not block) physical testing. The temperature was directly monitored in PCR tubes containing 50 μl of distilled water at 13 different block positions. The certified temperature accuracy of the measurement system was ±0.3°C. Finally, the results of the physical testing were compared to those of an amplification efficiency study running an in-house PCR assay. The cyclers did not perform within the manufacturer's specification. Premature timing, under- and overshooting, and spatial variation of heat transfer were found to be the critical factors. The physical testing allowed us to distinguish accurate from less-accurate (2/6) cyclers. The lack of thermal homogeneities became most evident at the denaturation level during the first 15 s. At the time point zero, the accurate cyclers showed temperature deviations of 0.5 to 1.5°C, whereas less-accurate cyclers failed to reach the set temperature by 13 to 20°C. Consequently, the two less-accurate cyclers could not gain positive PCR results by running an in-house PCR assay. However, by modifying the original temperature protocol by increasing the denaturation temperature and time, the amplification efficiency of these two cyclers could be improved significantly. The results have implication for laboratories using diagnostic PCR testing.
INTRODUCTION
The breakthrough of PCR has been primarily due to the development of recombinant, thermostable DNA polymerases (15) and the development of thermocyclers. The cycler enables the automation of this repetitive thermal process. It should guarantee temperature uniformity for all samples and ensure a rapid heat transfer from the heating block to the in-tube sample liquid.
Although a convincing method, PCR needs further standardization, since a large variety of parameters can influence the outcome of PCR amplification (7). PCR protocols generated by one lab are hardly reproducible by another (1, 2, 20). Even under identical assay conditions, it was shown that there are difficulties in obtaining reproducible results (16).
One of the reasons might be the influence of the thermocycler on amplification efficiency. Despite the striking importance of PCR, the literature on thermocyclers is scarce. Some studies were published on the first generation of cycler models (5, 10, 12, 14, 17). Others determined the amplification efficiency but not the physical properties of thermocyclers (7, 16). According to an interlaboratory study, inappropriate cyclers that produced the most nonrepeatable data were older than 4 years, had no heated lid, and were run under block control (16). The physical functionality of the suspected cyclers, however, was not determined.
In a recent study, brand-new thermocyclers were subjected to a physical testing (19). The evaluation of their temperature profiles allowed the distinguishing of accurate from less-accurate cyclers. Premature timing was seen to be the main problem. In the case of one cycler, the sample temperatures were even 20°C below the set temperature. However, the consequences of these shortcomings on the PCR results were not determined.
It was the goal of this study to compare the physical properties of the latest generation of brand-new thermocylers, to demonstrate the impact of physical inaccuracy on amplification efficiency, and to point out the critical factors in the reliability of PCR performance in order to pave the way for preparation of an international standard.
The present work was initiated within a European project dealing with standardization of noncommercial PCR assays for the detection of major foodborne pathogens (www.pcr.dk).
MATERIALS AND METHODS
Terminology. The following expressions are repeatedly used. (i) Block uniformity (BU) refers to the temperature homogeneity within the 96-well block and is given by the manufacturer (Table 1). (ii) In-tube temperature (ITT) is the temperature measured in the PCR tubes at 13 different block positions. (iii) Set temperature (Tset°C) corresponds to the programmed temperature. (iv) Tcycler°C is the temperature which is indicated on the cycler's display. (iv) Maximum temperature (Tmax°C) describes the maximum value, minimum temperature (Tmin°C) the minimum value, and mean temperature (Tmean°C) the mean value of the 13 ITTs. (v) Sample uniformity (SU) refers to the temperature homogeneity of the 13 ITTs (Fig. 1); it describes the difference between Tmax and Tmin. (vi) Deviation (Dv) describes the difference between the ITT and Tset (Dvhigh = Tmax – Tset; Dvmean = Tmean – Tset; Dvlow = Tmin – Tset). The deviation indicates the tendency for overshooting (higher than programmed temperature; positive Dv values) or undershooting (lower than programmed temperature; negative Dv values). (vii) t0s (time point zero) describes the beginning of the onset of timing. It is the time point when Tcycler°C equals the set temperature. (viii) txs is the time point of measurement after x seconds from the onset of timing. (ix) Nonamplification means the absence of a detectable PCR product.
Instruments and programming. Brand-new thermocyclers (lettered A to F) from six major suppliers were selected for this performance study. Since the 96-well instrument is the most-used model, this block format was chosen for the experiments. Out of three options to control the block's temperature (block control, in-sample probe control, and calculated temperature control), the calculated temperature control mode was chosen if available (Table 1). All experiments were performed by setting the lid temperature to 105°C and using the maximum heating rate available.
Evaluation of physical performance. The temperature measurement followed the technical procedure of a recently published thermocycler performance study (19). The temperature was measured in 0.2-ml PCR tubes (MicroAmp, Applied Biosystems, Foster City, CA) containing 50 μl of distilled water. Fast-response microthermocouples of type T (RS Components GmbH, Gmünd, Austria) and the data logger 263A Data Bucket (Fluke Cooperation, Everett, WA) enabled the data collection. Temperatures were measured at 13 different block positions (Fig. 1). The temperature relative unit was calibrated in accordance with the law concerning legal metrology, which documents the traceability to national standards. The certified temperature accuracy and reproducibility were ±0.3°C and ± 0.1°C, respectively.
Each thermocycler was programmed to perform four static temperature protocols. The temperature protocols included a prerun phase at 26°C for 60 s followed by the heating of the cycler block to a set temperature of either 95°C, 72°C, 55°C, or 39°C. This temperature was then held for 300 s.
Evaluation of the amplification efficiency. A recently developed PCR assay was used to test the amplification efficiency (6). PCR was performed in a 50-μl reaction mixture containing 0.6 U of AmpliTaqGold polymerase (Applied Biosystems, Foster City, CA). A batch of master mix was prepared, and aliquots of this batch were used for all the experiments. Four nanograms of DNA of L. monocytogenes strain EGD (4) served as a template. The DNA concentration was measured with a fluorescence photometer (Hoefer DyNA Quant 200; Pharmacia Biotech, San Francisco, CA). Samples were loaded at four corner positions (A1, A12, H1, and H12) and one central position (D4). In addition, the control reactions were amplified using an in-house cycler model (GeneAmp 2400; Applied Biosystems, Foster City, CA). The PCR amplicons were separated in a 1.5% agarose gel using ethidium bromide staining. The intensity of the DNA band in the gel was assessed optically. Each temperature protocol comprised an initial denaturation step at 94°C for 600 s and a final extension step at 72°C for 300 s. The cycling conditions as published were as follows: 94°C/15 s, 68°C/30 s, and 72°C/60 s (for 45 cycles; PCR Liststand). The protocol was also used with some modifications as follows (i) PCR Listmod1 (96°C/15 s, 68°C/30 s, and 72°C/60 s), (ii) PCR Listmod2 (94°C/60 s, 68°C/30 s, 72°C/60 s), and (iii) PCR Listmod3 (96°C/60 s, 68°C/30 s, and 72°C/60 s).
RESULTS
Physical functionality of six thermocyclers. (i) Spatial influence: SU versus cycler-specific BU. The thermocyclers had 300 s to establish temperature uniformity. At time point t300s, the SU was calculated and set into relation with the block uniformity as given by the manufacturer for each cycler model (Table 1).
Out of the six cyclers tested, five cyclers showed an SU which exceeded the BU irrespective of the temperature value chosen. Only cycler A did not exceed the specified BU limit of ±0.5°C (Fig. 2).
In all cycler models there was a tendency for overshooting when lower block temperatures (55°C and 39°C) were used. All cyclers showed positive Dv values. The Dvhigh value was highest in all machines at 39°C (Fig. 2). Overshooting became most clear in cyclers D and E, showing a Dvhigh of +1.8°C and +1.4°C, respectively. On the contrary, Dvlow became negative in some cyclers at higher block temperatures. This phenomenon was most obvious in cycler F, resulting in a Dvlow of –1.6°C at 95°C.
Irregularities of the SU followed the loading pattern (spatial distribution) of the samples. PCR tubes loaded into the inner positions usually revealed the Tmax°C values, whereas samples loaded into the outer positions revealed the Tmin°C value. This was most evident in cyclers C and F (data not shown).
(ii) Temporal influence: determination of the SU and deviation of ITTs at time points t0s, t15s, t30s, and t60s after onset of timing. Generally, the dependence of thermal variations on timing became most evident using the 95°C protocol. Therefore, the data are shown in more detail (Fig. 3). At time point zero (t0s), the SU varied from 1.5°C (cycler A) to 7.9°C (cycler F) and 10.7°C (cyler C). After 60 s (t60s), the SU was narrowed down to 0.9°C, 1.6°C, and 2.3°C in cyclers A, C, and F, respectively (Fig. 3).
Cycler A most accurately met the set temperature, 95°C, at time point zero (t0s), showing the most balanced deviation (Dvlow and Dvhigh, –0.4 and +1.1). On the contrary, the Dvlow in cyclers C and F failed to reach the set temperature by –19.7°C and –12.8°C at t0s, respectively. These differences were narrowed down to –1.8°C and –2.2°C at t60s. From these data it was expected that fast cycling protocols, in general, especially those showing a short denaturation phase, have a higher risk of nonamplification.
Comparison of the cycler’s amplification efficiency using an in-house Listeria PCR assay. An in-house Listeria PCR assay (PCR ListStand) was used to test the amplification efficiency. Cyclers A, B, D, and E attained good results, while cyclers C and F showed no amplification in the five positions tested (Fig. 4).
Upon shifting the denaturation temperature to 96°C (PCR Listmod1), the amplification efficiency of cycler C but not of cycler F was improved. Cycler C amplified the product at four positions, except for position H12, whereas cycler F did not produce a visible band. Only by expanding the denaturation time from 15 s to 60 s (PCR Listmod2) was it possible to achieve consistently good amplification for all cyclers at all five positions (Fig. 4). Upon shifting the denaturation temperature to 96°C and expanding the time from 15 s to 60 s (PCR Listmod3), all five positions in cycler C and F showed clear bands, but the amplification efficiency of cycler E dramatically decreased. Cycler E showed no amplification in all five positions tested (data not shown).
DISCUSSION
Soon after the introduction of the first commercial instrument in 1987 (Perkin-Elmer Cetus DNA thermal cycler), other suppliers offered similar devices. Today the thermocycler market is dominated by a few major manufacturers. Thermocyclers of the previous generations differ widely from the contemporary models. Developments of heating/cooling systems (circulating water versus Peltier technology), internal temperature control options (block control versus in-sample probe control and calculated temperature control), and evaporation control concepts (oil overlay versus heated lid) have been changing the technical architecture of the cyclers completely (15, 16). Accurate thermocyclers are a prerequisite for advanced applications such as cycling sequencing, in situ PCR, and real-time quantitative PCR in sample formats of up to 384 block positions. All these techniques rely on (i) an efficient heating and cooling system, (ii) an efficient heat transfer from the block to the sample liquid, and (iii) a sophisticated steering algorithm which allows the reproducibility of a given set temperature profile as stringent as possible.
The performance studies on thermal cyclers published so far lack relevance (5, 8, 14, 21). On the one hand, the studies refer to the older generation of thermocyclers, or processors from only one or two manufacturers were tested (5, 8, 21). Interlaboratory trials, on the other hand, compared only the amplification efficiency of used cyclers of various makes and models (16). The physical functionality of the machines was not determined.
This study focused on brand-new cyclers of the last generation. By embedding a fast-response microthermocouple inside the PCR tube, it was possible to determine the temperature uniformity within the sample block and to study the thermodynamic process from the block via the tube to the PCR sample in more detail. If available, the calculated temperature control mode, which is known to bring the samples more easily to the programmed temperatures (16), and each maximum heating rate had been chosen. By performing a PCR assay (PCR ListStand), the amplification efficiency of each model was determined.
Both physical testing using a static protocol and PCR testing allowed us to distinguish accurate (A, B, D, and E) from less-accurate (C and F) cyclers. This was also shown using a dynamic three-level protocol in a recent study (19). The most obvious irregularities of the machines were premature timing, followed by under- and overshooting and spatial irregularities of heat transfer. These results indicated that a difference in the cycler performance is not only a matter of abrasion after years of use, as suggested by previous studies (7, 16). It is an outcome of the variable technical architecture of the cyclers, which differs in the number, size, and quality of the Peltier elements, the location and number of the thermosensors steering the Peltier elements, and the underlying algorithm. A difference in the ITT from the block temperature might also result from poor heat transfer from the block to the PCR sample. In particular, the models which were notable for high heating and cooling rates seemed to be "susceptible" to premature timing.
The short denaturation step of PCR ListStand was seen to be the critical point. The cyclers C and F failed to amplify the target DNA. The physical testing of cyclers C and F showed a significant tendency for undershooting and premature timing. Usually the optimum denaturation temperature is between 91 and 94°C (11). Referring to the PCR ListStand protocol (94°C/15 s, 68°C/30 s, and 72°C/60 s), cylers C and F performed the effective temperature level only for 5 s (33%) or 7 s (47%), respectively (data not shown). Consequently, we concluded that the lack of amplification of cyclers C and F was most likely caused by the insufficient melting of the template DNA. This hypothesis could be proved by modifying the original denaturation step by increasing the temperature (PCR Listmod1), time (PCR Listmod2) or both (PCR Listmod3). The amplification efficiencies of cyclers C and F could be improved significantly. The best results could be obtained by expanding the denaturation time from 15 to 60 s (PCR Listmod2). This was seen as a proof that premature timing was the main reason for the lack of amplification of cyclers C and F.
By increasing and expanding the denaturation step up to 96°C and 60 s (PCR Listmod3), the tendency of cycler E for overshooting became most evident. Its amplification efficiency decreased dramatically, perhaps due to polymerase inactivation or deoxynucleoside triphosphate breakdowns (3, 22, 23, 24).
In conclusion, this is the first time that the pivotal influence of the thermocycler on PCR efficiency has been studied in detail. It is well known that PCR protocols need to be optimized for different instruments and that a cycling program that works on one brand of instrument may not necessarily work on another. It is the physical testing of cyclers that helps to explain why an adoption of the original temperature protocol leads to an amplification in one cycler and lack of amplification in another. The performance of various modifications of the original PCR assay protocol (PCR Listmod1 to PCR Listmod3) demonstrated that PCRs run with inaccurate machines do not necessarily result in lack of amplification. However, an extended experimental expenditure is required to match a PCR protocol with the technical idiosyncrasies of such cyclers.
Each PCR user should have an effective and simple tool to test the performance of in-house cyclers. Several cycler companies developed end-user-friendly temperature validation systems, such as the Temperature Verification system (Applied Biosystems, Foster City, CA) or the Temperature Validation system (Eppendorf Netheler-Hinz GmbH, Hamburg, Germany). These single-sensor test systems allow static temperature measurement only within the block wells. Our study could show that static temperature procedures, on condition that they are done in-tube, are appropriate to detect some of the main weak points of thermocyclers (19). Another approach could be a cycler performance test using a standardized temperature-sensitive PCR system, such as SureCycle (Congen, Berlin, Germany), which was recently developed and introduced into the market (9). Cycler performance testing should regularly be carried out and become a part of any accreditation of an expert or an end-user lab devoted to PCR diagnostics. New PCR strategies should demonstrate the robustness of a temperature protocol, and validation of PCR protocols must include the testing of a PCR assay on several thermocyclers.
ACKNOWLEDGMENTS
This work was supported by EU grant no. QLK1-CT-1999-00226.
We thank Kurt Wimmer (Institute for Medical Physics and Biostatistics, Vienna, Austria) for his assistance in construction of the temperature measurement unit, Jose A. Vazquez-Boland (Grupo de Pathogenesis Molecular Bacteriana, Universidad de Leon) for the provision of L. monocytogenes strain EGD, and finally, Martin D’Agostino and Nigel Cook (Defra CSL) for carefully reading the manuscript.
REFERENCES
Bootman, J. S., and P. A. Kitchin. 1992. An international collaborative study to assess a set of reference reagents for HIV-1 PCR. J. Virol. Methods 37:23-41.
Defer, C., H. Agut, A. Garbarg-Chenon, M. Moncany, F. Morinet, D. Vignon, M. Mariotti, and J. J. Lefrere. 1992. Multicenter quality control of polymerase chain reaction for detection of HIV DNA. AIDS 6:659-663.
Gelfand, D. H. 1989. Taq DNA polymerase, p. 17-22. In H. A. Erlich (ed.), PCR technology: principles and applications for DNA amplifications. Stockton Press, New York, N.Y.
Glaser, P., L. Frangeul, C. Buchrieser, C. Rusniok, A. Amend, F. Baquero, et al. 2001. Comparative genomics of Listeria species. Science 294:849-852.
He, Q., M. K. Viljanen, and J. Mertsola. 1994. Effects of thermocyclers and primers on the reproducibility of banding patterns in randomly amplified polymorphic DNA analysis. Mol. Cell Probes 8:155-159.
Hein, I., D. Klein, A. Lehner, A. Bubert, E. Brandl, and M. Wagner. 2001. Detection and quantification of the iap gene of Listeria monocytogenes and Listeria innocua by a new real-time quantitative PCR assay. Res. Microbiol. 152:37-46.
Hübner, P., E. Studer, D. Hfliger, M. Stadler, C. Wolf, and M. Looser. 1999. Detection of genetically modified organisms in food: critical points for quality assurance. Accredit. Qual. Assur. 4:292-298.
Kopp, D. W., C. A. Sansieri, and T. E. Mifflin. 1994. Routine monitoring of temperatures inside thermal cyclers blocks for quality control. Clin. Chem. 40:2117-2119.
Kuhn, M., S. Mergemeier, D. Schoder, and M. Wagner. 2002. Funktioniert der Thermocycler richtig LaborPraxis 4:80-83.
MacPherson, J. M., P. E. Eckstein, G. J. Scoles, and A. A. Gajadhar. 1993. Variability of the random amplified polymorphic DNA assay among thermal cyclers, and effects of primer and DNA concentration. Mol. Cell Probes 7:293-299.
Mai, M., R. Grabs, R. D. Barnes, P. Vafiadis, and C. Polychronakos. 1998. Shortened PCR cycles in a conventional thermal cycler. BioTechniques 25:208-210.
Penner, G. A., A. Bush, R. Wise, W. Kim, L. Domier, K. Kasha, A. Laroche, G. Scoles, S. J. Molnar, and G. Fedak. 1993. Reproducibility of random amplified polymorphic DNA (RAPD) analysis among laboratories. PCR Methods Appl. 2:341-345.
Reference deleted.
Rezendez-Perez, D., and H. A. Barrera-Saldana. 1990. Thermocycler temperature variation invalidates PCR results. BioTechniques 9:286-292.
Saiki, R. K., D. H. Gelfand, S. Stoffel, S. J. Scharf, R. Higuchi, G. T. Horn, K. B. Mullis, and H. A. Erlich. 1988. Primer-directed enzymatic amplification of DNA with a thermostable DNA polymerase. Science 239:487-491.
Saunders, G. C., J. Dukes, H. C. Parkes, and J. H. Cornett. 2001. Interlaboratory study in thermal cycler performance in controlled PCR and random amplified polymorphic DNA analyses. Clin. Chem. 47:47-55.
Schauberger, G., and M. Wagner. 1998. Qualittskontrolle von PCR Thermocyclern: überprüfung der rumlichen und zeitlichen Temperaturverteilung. Biomed. Engin. 43:216-218.
Reference deleted.
Schoder, D., A. Schmalwieser, G. Schauberger, M. Kuhn, J. Hoorfar, and M. Wagner. 2003. Physical characteristics of six new thermocyclers. Clin. Chem. 49:960-963.
Sheppard, H. W., M. S. Ascher, M. P. Busch, P. R. Sohmer, M. Stanley, M. C. Luce, J. A. Chimera, R. Madej, G. C. Rodgers, C. Lynch, et al. 1991. A multicenter proficiency trial of gene amplification (PCR) for the detection of HIV-1. J. Acquir. Immun. Defic. Syndr. 4:277-283.
Tweed, G. P., J. Whitney, and P. L. Bloch. 1991. Temperature cycler evaluation: what do you need to know BioTechniques 10:526-532.
Wittwer, C. T., and D. J. Garling. 1991. Rapid cycle DNA amplification: time and temperature optimization. BioTechniques 10:76-83.
Wittwer, C. T., G. C. Fillmore, and D. R. Hillyard. 1989. Automated polymerase chain reaction in capillary tubes with hot air. Nucleic Acids Res. 17:4353-4357.
Yu, K., and K. P. Pauls. 1992. Optimization of the PCR program for RAPD analysis. Nucleic Acids Res. 20:2606.(D. Schoder, A. Schmalwies)