Ultramicroscopy 46 (1992) 207-227 North-Holland

Automated microscopy for electron tomography A.J. Koster, H. Chen, J.W. Sedat and D.A. Agard Department of Biochemistry and Biophysics and the Howard Hughes Medical Institute, University of California at San Francisco, San Francisco, CA 94143-0448, USA Received at Editorial Office 20 May 1992

Instrumentation and methodology for the automatic collection of tomographic tilt series data for the three-dimensional reconstruction of single particles is described. The system consists of a Philips EM 430 TEM, with a Gatan 673 cooled slow-scan CCD camera and a Philips C400 microscope computer control unit attached. The procedure for data collection includes direct digital recording of the images on the CCD camera and the automatic measurement and correction of (a) image shifts resulting from tilting the specimen, (b) variation of defocus and (c) the eucentric height position of the specimen. Experiments are described illustrating the possibilities and limitations of automatic data collection. Data collection at a magnification of 30k shows that the exposure time of the specimen to the beam is reduced by a factor of 10-100 compared to manual operation of the TEM.

1. Introduction

Three-dimensional electron microscopic imaging has become an important addition to the variety of methods available for research on biological structures. Non-crystalline samples can be examined by high resolution electron tomography which requires that projection data be collected over a large range of specimen tilts. Practical limitations of tomography are set by the large number of micrographs to be processed, and by the required (and tedious) recentering and refocusing of the object during data collection. In general, since all the required tilt data must be collected from a single specimen, the cumulated electron dose can be extremely high. Thus for beam-sensitive specimens such as frozen-hydrated samples, high-resolution reconstructions are precluded. With automated electron tomography a number of these problems can be overcome [1]. First of all, the images are recorded directly in digital

format, using a cooled slow-scan CCD camera. Furthermore, with automatic tracking and correction for image shift and focus variation (during data collection), a pre-aligned dataset is obtained, with every image recorded under well defined imaging conditions. A savings of up to a 100-fold in electron dose can be achieved compared to manual operation. When automatic data collection is done under low-dose conditions, possibly in combination with spot scan imaging [2], threedimensional reconstruction of dose-sensitive material becomes feasible. An exciting possibility is that this may open the way to routine tomography of frozen-hydrated samples. In general, reliable interpretation of a series of images is only possible when the series is taken under well defined imaging conditions. In particular, accurate and known settings of defocus and precise corrections for astigmatism and beam tilt misalignment are essential. Consequently, one important part of an automated electron tomographic approach is autotuning: the capability to

0304-3991/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved

2(18

A.J. Kosteret a L / Automated microscopyfor eh'ctron tomography

automatically measure and set the imaging parameters for the transmission electron microscope (TEM). Several papers on autotuning of a TEM have been published [3-5]. Three different methods have been proposed, based on the evaluation of digital diffractograms, image contrast and beamtilt-induced image shifts. An overview of these methods can be found in ref. [6]. The beam-tiltinduced image shift method was initially introduced by LePoole [7], and was further developed by van dcr Mast [8], Nomura and Isamozwa [9] and Koster et al. [10-12]. Recently, the method was extended to the automatic correction of astigmatism and beam tilt misalignment (Koster and de Ruijter [13]). The beam-tilt-induced image shift method is suitable for operation over a large magnification range for both amplitude- and (weak) phase-contrast objects. Also, compared to the other two methods, the beam-tilt method is dose-efficient and suitable to be embedded in a larger automated microscopy system, as the method performs reliably for a large range of specimen types. During the last few years, several (partly commercially available) systems have been described which incorporate features of the autotuning methods mentioned above [31-33]. The TEM parameters essential for electron tomography are the specimen tilt angle, defocus and astigmatism, illumination conditions (spotsize, intensity, beam position, beam tilt), the specimen position ( X - Y ) , and the specimen height (Z). Each of these TEM parameters needs to be calibrated before it can be controlled by the computer. Thus, together with the increase in complexity of control needed for control in tomography, the complexity of calibrating these controls also increases. Consequently, automatic calibration should be part of the automated microscope system. In this paper the present developments in TEM automation at UCSF for the application to electron tomography are discussed. The theoretical aspects of autotuning, and other techniques related to automated electron tomography, are described in section 2. Practical and instrumental aspects of implementation are evaluated in section 3. Section 4 describes our present instrumen-

tal setup, together with experimental data illustrating the possibilities and restrictions of automatic data collection. In section 5, future possibilities for automated electron tomography are discussed.

2. Theory of autotuning 2.1. Correction of defocus, astigmatisrn and re&alignment

The autotuning method is based on the effects of beam tilt on electron microscope images. Relatively coarse image detail ( > 1-10 nm) is mainly due to amplitude contrast, as is described in ref. [14]. The amplitude of the incident electron wave is altered by position-dependent transparency, resulting directly in image contrast. The effect of beam tilt on large image detail can be simply described in terms of geometric optics. Formation of finer image detail is, however, more complicated and can only be properly understood using propagation of electron waves through the specimen and the remaining optical system of thc microscope. The main contrast mechanism is then phase contrast in which the phase of the incident electron wave is altered by position-dependent transparency. With weak phase objects, image contrast arises from objective lens aberrations; the most important being the defocus, axial astigmatism and spherical aberration. A simple linear image formation theory of fine image detail is only possible for weak scattering objects, such as thin amorphous films; the effect of beam tilt on fine image detail of weak scattering objects has been published [15]. For thick non-crystalline specimens image interpretation and object reconstruction is more complicated due to combined effects of multiple scattering and energy loss of electrons within the specimen [16,17]. In the case of crystalline specimens, image formation is also not straightforward because of dynamical (multiple) scattering. Nevertheless, exit surface wave functions can be derived from known structures using numerical simulations and the image intensities can be analytically derived from these known functions. This analytical ex-

2{19

A.J. Koster et al. / Automated microscopy for electron tomography

pression includes all microscope parameters, so that the effects of beam tilt on images of crystalline material can be characterized. Work on the reconstruction of the exit wave function using holography or a through-focus series can be found in refs. [18,19]. The smallest structure details detectable in an image are dependent on the electron optical magnification and on the characteristics of the primary detection element of the image pickup device. At low magnification, fine image detail is cut off by the modulation transfer function of the camera, whereas at high magnification the effective field of view is small, thereby excluding amplitude contrast from contributing much to the recorded image. In the remaining part of the paper, we discuss an autotuning method using image detail as formed by the amplitude- and weak phase-contrast mechanism.

Illumination

Induced Beam Tilt

Beam tilt coils

defocus

Specimen

I

Object Plane

Objective lens

C)

y

([~

J Image displacementJ ~ :,',',',',',','.'.',',',',',',',',',',','.1

ImageShift Coils Image Plane CCDCamera

Fig. 1. The principle of the aulomatic focusing method: image displacement, resulting from an induced beam tilt, is linearly related to the amotmt of defocus.

2.1.1. Beam-tilt-induced displacement 2.1.1.1. Amplitude contrast

From basic geometric optics it can be derived that an induced beam tilt results in an image displacement with respect to the image without beam tilt. A simplified representation of beam-tilt-induced displacement is given in fig. 1. In the absence of beam misalignment and astigmatism the displacement is simply given by:

d = M(C~/3 2 - D)/3,

(1)

where the displacement d is in the direction of the beam tilt /3, D is defocus, C~ is the constant of spherical aberration, and M denotes the magnification. This expression can be extended to include the effects of astigmatism and beam misalignment. Because astigmatism is merely a direction-dependent defocus we obtain an expression where the displacement vector d is given by: d= [-It +ml 2 + (D-½A)](t

+ A [ ( t + m ) "a]a,

matism, A denoting the difference between the maximum and minimum defocus directions and a the direction of minimum defocus, t denotes the induced beam tilt, m is misalignment. All parameters in eq. (2) are given in reduced units Sch and G1, with 1 Sch = ( C s A ) I/2 and 1 GI = (CsA3) I/4, A being the wavelength of the electrons. Some comments should be made on relationship (2) between image displacement and beam tilt. The parameters m, D, A and a can only be estimated if more than one image displacement is measured. From (2) it is clear that the direction of the induced beam tilt t is, in general, not the same as the direction of image displacement d. These are equal only if no astigmatism and misalignment are present (so there is only defocus). Since the direction of image displacements is important, two-dimensional image displacement measurements are necessary (section 2.3).

2.1.1.2. Phase contrast

+ m) (2)

where D denotes defocus, positive D being described as underfocus, A and a define axial astig-

The image for weak phase objects can be derived using the phase-contrast transfer function (PCTF) of the objective lens. The PCTF is a complex function which defines the strength and phase of transfer for every spatial frequency passing through the objective lens.

210

A.J. Koster et al. / Automated microscopy for electron tomography

Image contrast can be written, again in reduced units, as [20]:

C(x; O) =F-1[F(k; 0 ) ~ ( k ) ] ,

(3)

where x denotes a position (in G1) on the specimen and k denotes frequency (in GI-1). C(x; O) denotes image contrast, F(k; O) the phase-contrast transfer function and ~(k) the Fourier transform of the phase of the exit wave function r/(x). 0 is a vector of parameters which include defocus, astigmatism and misalignment, rt(k) is directly related to the projected potential of the specimen. The image displacement is given by the derivative of the phase of the PCTF with respect to the spatial frequency k. The displacement induced by a beam tilt t can be written as [21]: d= [-Ikl2-]t+m]

2+(D-½A)](t+m)

+ A [ ( t + m ) "a]a.

(4)

Eq. (4) is almost identical to the result (2) derived directly from geometric optics except for the additional term - ] k l 2, which implicates a dispersive (frequency-dependent) image displacement in the case of weak phase objects. Note that relation (4) is similar to (2) after applying a lowpass Fourier filter to the images which are used to measure image displacement d in such a way that I k[ 2 is negligible compared to the other terms in eq. (4).

2.1.2. Correction of misalignment, defocus and astigmatism The direct application of (2) for autotuning is possible only if the relationship between the computer controls (in arbitrary control units or "ticks") and the actual physical value of the T E M parameters (in Sch and G1, or in nm and mrad) is properly calibrated. This calibration includes sensitivity ratios of the controls with respect to the actual amount and orientation of beam tilt and defocus/astigmatism. The calibrations depend upon the magnification, which alters not only sensitivities, but also introduces image rotation. Several methods can be used for calibration of the controls [22].

After the controls have been calibrated, the beam-tilt misalignment can be corrected (corresponding to coma-free alignment) by using three images. One image is formed without using induced beam tilt and a pair of images is formed with equal, but opposite induced tilt angles _+t. The difference in the image displacements relative to the image formed without beam tilt is a measure of the misalignment. It follows from eq. (2) that a difference between the displacements d c is given by

dc= - ( m . t ) t - Z t Z m ,

(5)

where t is induced and known. Note that the estimation of misalignment m is independent of the defocus or astigmatism. During the alignment procedure, the displacements are scaled to specimen level, the actual misalignment m is estimated and a correcting beam tilt opposite to m is induced. The misalignment m is directly solved from the two linear equations in m given in (5). The rotation alignment procedure measures the image displacement d r following a defocus change (D 1 - D 2) given by

dr = ( D , - D z ) m ,

(6)

where (DI - D 2) is induced and known. From (6) it follows that the alignment is independent of the actual amount of defocus and astigmatism. Rotation alignment can be performed in similar fashion to coma-free alignment, by measuring the displacement dr, estimating the misalignment m and inducing a correcting beam tilt opposite to the estimated misalignment. The misalignment is calculated from the two linear equations given in (6). Not included in this model of image formation are the second-order effects of image rotation and the change in magnification, due to changes in objective lens current. Defocus and astigmatism are estimated and corrected after the coma-free alignment process. The relationship between the image displacement d and the defocus and astigmatism for an induced beam tilt t follows from eq. (2) and is given by

df=[-tZ+(D-½A)]t+A(t.a)a,

(7)

A.Z Koster et a L / Automated microscopy for electron tomography

where t is induced and known. Note that the relationship between the image displacement and the defocus or astigmatism is linear for beam tilts t smaller than ( D - 0 . 5 A ) x/2. For larger beam tilts, the term t 2 has to be included within the correction algorithm. The estimator of defocus and astigmatism is derived from eq. (2). If we ignore the squared tilt term, this equation can be rewritten as:

(8)

d = or,

where the 2 × 2 matrix P is given by:

P =

D+A a Ab

A~ ) D -A a '

(9)

and where astigmatism is written in terms of the variables A a = 0.5A cos(2~p) and A b = 0.5A sin(2q~), q~ denotes the direction of minimum defocus. Since the incident beam previously has been aligned on the coma-free axis, the effect of beam tilt is symmetric with respect to zero tilt. Because of this symmetry, it is sufficient to measure only three images: one image formed without induced tilt, and the other two images formed with beam tilt equal in magnitude but with 7r/2 rad difference in azimuthal angle. Measurement of the two displacements dl and d 2 between the images formed with and without beam tilt result in four linear equations defined by (8). The parameters describing the defocus and astigmatism are estimated by calculating the least-squares solution of this overdetermined set of four equations with respect to D, A a and A.

2.1.3. Autotuning as part of automated microscopy The performance of the autotuning method as described in the previous section with eqs. (5)-(9) is highly dependent on the validity of the calibration of the controls. Unfortunately, for several reasons these calibrations will vary with time. The variation might be due to long-term (temperature) drift effects within the microscope itself, but also due to more abrupt changes in the instrumental setup like a camera rotation (e.g. after servicing the camera). Consequently, to have optimal performance of the autotuning method it is

211

required that the controls be frequently calibrated, which takes time, effort and expertise. Fortunately, the calibration of the microscope controls can also be automated when we formulate the autotuning method somewhat differently (as described in ref. [13]). For example, we can rewrite eq. (7) as a linear equation with respect to the induced beam tilt t

df= ( fF + aA + bB)t,

(10)

where df is an image displacement in pixels, induced by a beam tilt t = (c, d) T, which is defined by the values of the outputs c and d to the controls of the beam tilt coils, f denotes the objective lens control output, a and b denote the outputs controlling the stigmator coils, a - d are in "ticks" which correspond to the smallest step in the output of the computer for the control of the electron microscope. F, A and /3 are 2 x 2 calibration matrices defining the relation between the image positions and the values of the control outputs. The columns of the calibration matrices are calculated from two beam-tilt-induced displacements. For example, the first column of F is determined by measurement of two displacements d I and d2, induced by beam tilts (t, 0), ( - t , 0), with defoci f l and f2, respectively. So:

F(o 1

]

dz - dl 2(f2-fl)t"

(11)

The second column of F, and the columns of A and /3, are measured in a similar fashion. Measurement of the calibration matrices A, B and F requires a total of 24 images. For the relation between beam tilt and misalignment a similar linear expression can be obtained using a 2 × 2 calibration matrix C [13]. The columns of matrix C are calculated from four beam-tilt-induced displacement differences. Each displacement difference is calculated from three images. Measurement of the calibration matrix C requires a total of 12 images. Calibration has to be executed only once and can be implemented fully automatically. The calibration matrices take into account all possible linear relations between the controls and the actual T E M parameter values, including sensitivi-

212

A.J. Koster et al. / Automated microscopy fi~r electron tomography

ties, orientations and also linear geometric distortions which occur in the projector lenses of the T E M or in the image pickup device. When the calibration matrices (F, A, B and C) are known, rotation alignment can be measured from one defocus-induced displacement (two images). Defocus and astigmatism can be measured from two beam-tilt-induced displacements (four images). Alignment on a coma-free axis can be achieved using three images.

2.2. Automatic tomography and spot scanning For electron tomography, the dataset required for object reconstruction consists of a series of images taken at different specimen tilt angles [23]. The resolvable structural details of the reconstruction are determined by the range and increment of specimen tilt (typically + 7 0 ° and 1.25 °, respectively). Consequently, the number of specimen projections (images) to be taken and processed is quite large (typically a hundred). Also, for high-resolution reconstructions, the imaging conditions at each image must be both known and maintained constant (especially defocus, astigmatism and the illumination conditions),

together with the electron dose per image. A recent description of an approach to automatic electron tomography for low-dose electron microscopy (including autotuning), can be found in ref. [I]. One of the problems of tilting the specimen stage (either manually or in a computerized way) is that the object of interest will generally not stay centered on the image pickup device (camera). Due to mechanical limitations of the specimen stage, the tilt axis will not be exactly coincident with the optical axis, and, consequently, when the specimen is tilted, the object of interest will shift. The larger the amount of specimen tilt, the larger the amount of image shift. Especially at high tilt angles ( > 6(1°), the operator is forced to correct for these image shifts. In addition to the image shifts, the defocus will also change when the specimen is tilted, and hence the operator must also correct the focus during data collection. The operator can use either the electron optical controls for keeping the object centered and focused (image shift deflection coils and objective lens current, respectively), or the specimen stage controls ( X - Y - Z controls) (see fig. 2). The electron optical control is fast and convenient, accu-

Illumination

¢D

I Correctwit

(E)

-- ---

X,Y,ZNovement ~

Beam ti, c0i/

Beam t,lt c0,,s

Specimen

~'~

Objectwe lens

I~

I~

Image Shirt Coils

Specimen

T __ ~

Object Plane

Object Plane

Correct with objective lens current

Correct with image shirt coils

ObJective lens

I~

~

~ ( ~

Image Shirt Coils

Neasure Image Displacement (a)

Image Plane ;....f,,,,,....,,,,,,,,,,, .................

CCD Camera

.... (b)

;,',',','.','.',','.'.','.',','.'.',',','.,

Image Plane CCD Camera

Fig. 2. (7orrection for the image shift and defocus change when the specimen is tilted at a non-eucentric height. The correction can be done (left) by movement of the specimen stage, or (right) by adjusting the electron optics.

A.J. Koster et al. / Automated microscopy ]'or electron tomography

rate and reproducible compared to control of the mechanical stage. When the amount of defocus, or image shift, to be corrected is large, the allowable control range of the electron optics will limit the quality of the three-dimensional reconstruction. A change of objective lens current introduces a change in the magnification, and image orientation (rotation). A change in objective lens current may also effect the reconstruction because the illuminating beam intensity will change with focus when the beam is slightly divergent (or convergent). Finally, to keep the illuminating beam in the same position relative to the image pickup device in spite of using the image shift coils (beneath the objective), the beam shift coils (above the objective) should also be used. However, when the electron optically induced image shift a n d / o r focus variation is very large, image distortions will be introduced. Therefore, for optimal resolution and quality of the reconstruction, these controls must be used carefully near their limits. For tomography of large structures using high tilt angles (where a large amount of image shift correction and re-focusing is almost inevitable during data collection), it may be preferable to control the specimen stage in X - Y position, and Z-height, and not the electron optical image shift deflection coils and objective lens current. Alternatively, control of the X - Y - Z stage positioning, in combination with the electron optical controls, will give an optimal compromise between the range of control, and the speed and accuracy of control. An overview of the T E M parameters to be controlled for automated electron tomography is given in table 1. Spot scan imaging has a number of favorable properties compared to flat beam imaging. A recent description of the application of spot scan imaging as part of an automatic data collection system can be found in ref. [2]. With spot scanning, the specimen is not illuminated with one broad illuminating beam (flat beam), but is scanned with a small illuminating spot (preferably under coherent imaging conditions for HREM). With spot scan imaging, the specimen damage is reduced, and the resolution of the image is increased. Most spot scanning results are obtained

213

Table 1 TEM parameters to be controlled for automated tomography TEM parameter Beam tilt Defocus Astigmatism Specimen tilt Image shift Beam shift Specimen X - Y shift Specimen Z shift

Controlled by Coils above objective lens Objective lens current Stigmator coils Specimen stage Coils beneath objective lens Coils above objective lens Specimen stage Specimen stage

by controlling the beam scan with an external computer, and recording the image (as an array of spots) on an electron micrograph. Nowadays, with high-quality CCD cameras available, it is feasible to record digital images directly, which increases both data collection throughput and accuracy since it is not necessary to digitize every micrograph before processing. Also, spot scanning, in combination with dynamic focus control (to adjust the defocus to match the tilt of the specimen) and autotuning, makes automated high-resolution electron tomography of radiation-sensitive structures possible. In this case, the required automation becomes more complex, since the field of view of a CCD camera is small compared to an electron micrograph and the number of resolvable pixels are limited: the computer needs to control not only the beam deflection coils for the scanning of the spot, but also the image shift coils to keep the illuminated area of interest centered on the CCD. Summarizing, the number of T E M parameters to be controlled and calibrated for automated electron tomography and spot scan imaging in combination with dynamic focusing is quite large, especially when it is combined with autotuning. We next describe an approach for controlling and calibrating these controls, applicable to work with a variety of specimens and imaging conditions. 2.2.1. A u t o m a t i c tracking o f image shift

Next, we will illustrate in more detail the approach for automatic tracking of the image shift, as well as for the automatic setting of the Z-height of the specimen to ensure eucentric tilting. A

214

A.J. Koster et al. / Automated microscopy for electron tomography

similar approach can be taken for the control and calibration of the beam shift, beam deflection, and specimen stage X - Y movement. The control of the specimen stage Z-height control for automatic tracking of focus is done in a similar way as for the objective lens. The relation between an induced change in activation of the image shift coils and the resulting image shift is given by

where d S is the image displacement (pixels), ,.q a 2 × 2 matrix describing the orientation and proportionality between the image displacement d s and the induced change in activation of the image shift coils (s x, sy) in arbitrary control units (ticks). When, during data collection, the image is shifted over an amount d (e.g. due to tilting the stage, or due to specimen drift), the image shift coils should be activated with an amount

(s:)= S~

Calibration of the image shift control is done by measuring the matrix ,3. First, the displacement d~ is measured after activation of the image shift coils with (s, O) (x-coil). Secondly, a displacement d 2 is measured after activating of the image shift coils with (0, s) (y-coil). The matrix S is then given by:

l[d~. d2x) S= 71G, '

(14)

d2y ] .

Prior to data collection, the matrix S is measured, ensuring accurate image positioning by activating the image shift coils with an amount (s x, sy) after measuring image displacement d following eq. (14). The matrix S contains all the information related to the magnification, the sensitivity of each coil, the orientation of the coils to each other, and to the orientation of the coils to the coordinate system of the camera.

2.2.2. Automatic adjustment of Z-height for eucentric tilting For electron tomography it is important that the Z-height of the specimen tilt stage is adjusted for eucentric tilting. At the correct height, the object of interest is positioned simultaneously on the electron optical and specimen stage tilt axes. When these requirements are fulfilled, the object stays centered on the image pickup device during the tilt series. Therefore, an automatic and accurate setting of the correct Z-height of the specimen stage prior to the automatic electron tomography data collection procedure is desirable, especially for dose-sensitive objects. The method for automatic setting of the Zheight described is similar to the way the operator of the TEM would do it by hand and visually. First, the stage is tilted (say to - 5 °) and an image is recorded. Then, the stage is tilted to the opposite angle ( + 5°), and a second image is recorded. The amount of displacement between the two images is measured (dl). Next the Z-height is changed with an amount 6z, from z 1 to z 2, and another displacement is measured (d2). From simple geometric considerations, it follows that the amount of image displacement varies linearly with the Z-height. Eucentric tilting corresponds to the Z-height where the image displacement will be zero, and can be found from the series of image displacements, using

di=azi+~,

(15)

where d i is the measured image displacement at specimen height z i, with i = 1,..'., N where N is the number of measured image displacements. From the N measurements, the parameters a and /3 can be estimated. The eucentric height z 0 is given by

z o = -[3/a.

(16)

It is clear that the accuracy obtainable in setting the eucentric height is limited by the mechanical reproducibility in setting the Z-height and specimen tilt angle.

2.3. Measurement of image displacements A displacement between two images can be measured using the cross correlation function

A.J. Koster et al. / Automated microscopy for electron tomography

(CCF). The CCF' C(x) between two images I~(x) and I2(x) can be evaluated using the discrete Fourier transform:

C ( x ) = T '{F,F~},

(17)

where F 1 and F 2 are the Fourier transforms of II and 12, respectively, T -J the inverse Fourier transform, * the complex conjugate and x the (discrete) image coordinates in pixels. The Fourier transforms are efficiently calculated with the fast Fourier transform (FFT) algorithm. Because the FFT assumes that the data is part of a repeating array, it is necessary to properly treat the borders of the image to avoid edge artifacts. We linearly taper the data at the edges down to the mean image intensity beginning at a distance of 40 pixels from each edge. In the array processor, it is most efficient to calculate FFTs that are powers of two in size. For non-power-of-two images, the tapered data are padded out to the next powerof-two size using the mean intensity. This approach for generating a correlation function gives rise to a very broad peak (essentially the self-convolution of the image). An alternative approach which results in substantially sharper peaks, providing substantially improved accuracy, is based on the phase-only cross-correlation function where the amplitude components have been removed [24]:

C(x)_=T 1( FIF ) (IF, IZlF212+a)'/z "

(18)

The a is used much like in a Wien filter to limit the contribution of terms having small amplitudes. For computational efficiency, the denominator is constructed using the squared amplitudes, so that only a single square root calculation need be performed. An initial estimate for the displacement is obtained from the position of the maximum of the CCF. The estimation is refined by a leastsquares fit of a two-dimensional second-order polynomial through the maximum of the CCF and its eight surrounding pixels.

215

3. Aspects of implementation Automated microscopy involves image acquisition, analysis and microscope control. For automated electron microscopy, the system consists of three parts: the TEM, the image pickup system, and the computer for the analysis and microscope control. Next, considerations important for implementing autotuning are described, followed by the more general requirements for automated microscopy.

3.1. Autotuning In general, the autotuning process consists of three steps; beam tilt alignment, correction of astigmatism/defocus, and setting the defocus to the desired value. This procedure may vary depending on the type of specimen, the allowable electron dose, the desired alignment accuracy and the quality and stability of the TEM and image pickup device. In the case of insufficient initial alignment, the procedure for auto-alignment is robust when it performs the adjustments in three stages; select a specimen area with sufficient image contrast at low magnification, perform a first iteration towards accurate alignment and perform a final, accurate, adjustment at high magnification using the specimen area of interest. Normally the alignment is done at high magnifications ( ~ 500 000), the correction of astigmatism at medium-high magnifications ( ~ 100000). Defocus correction is done at all magnifications. The accuracy for autotuning depends highly on the instrumentation, the type of specimen and the imaging conditions. Fig. 3 gives an indication of the reproducibility in correcting the defocus, using a TV camera or CCD camera as image pickup system, at several magnifications, for untilted thin high-contrast specimens (as described in ref. [13]. An indication of the speed of autotuning as obtained by several image processing systems coupled to a TEM is given in fig. 4. A practical limitation to the accuracy of autotuning (and calibration of the controls) is specimen drift. When specimen drift is large, the relation between the image displacement and the induced change in the settings of the TEM is

A.J. Koster et al. / Automated microscopy for electron tornography

216

obscured by the drift. Also, limitations result from the hysteresis of the objective lens which complicates a reproducible focusing. Furthermore, the amount of beam tilt to be induced for autotuning must be adapted to the electron optical magnification and to the amount of defocus. At high magnifications, or at large defocus values, the beam tilt to be induced for autotuning should be sufficiently small to ensure that the resulting displacement is smaller than the field of view of the camera. Furthermore, the size of the objective aperture may limit the allowable beam tilt (especially when the position of the beam tilt pivot point is not exactly within the specimen plane). Autotuning will be less robust in its performance during data collection for electron tomography and spot scan imaging as compared to fiat field images of untilted specimens for several reasons. When the specimen is tilted over a large angle, the optical thickness of the object increases, and the correlation peak indicating the amount of image shift is broadened and lowered due to the increased amount of scattering contrast. Especially for low-contrast objects in thick and tilted sections, special means must be taken to extract from the images used for autotuning,

A u t o f o c u s using a T V c a m e r a 300

Automatic

measurement

defocus

15 []

10

TEM Control

[]

Computations

[]

Delays

[]

Readout CCD

[]

Exposure correction

E

°~

[..

,:i:i:::i, 1

2

3

4

Instrumental set-up Fig. 4. Timing of automatic lk~cusing with different types of instrumentation. The set-ups 1-4 are described in ref. [13], and use a TV camera as image pickup device. Set-up 5 is described in section 4-, and is equipped with a cooled slow scan CCD camera. From the diagrams it can be concluded that for all these types of instrumentation the exposure time of the specimen is small compared with the total amount of time needed to correct for the defocus (TV: 0.32 s, CCD: 0.2 s). The time needed for TEM control differs between set-ups (1, 2, 4) and (4, 5). In the set-ups (1, 2, 4) the microscope is controlled serially (Philips CM interface), which is slow compared with the parallel control within the set-ups (3, 4) (JEM 4000 EX non-commercial interface, Philips C400 interface to the Philips EM series). In all set-ups the time needed for the computation of the cross correlation function is not limiting (in spite of the different types of array processors used, ranging in speed from 10-80 Mflops). Note the large amount of time required fl)r the readout of the CCD camera compared to the systems using a TV camera.

E 200 o~

L,

1O0

B Ella lab

r~

[]

!

i

i

i

|

lOO

200

3o0

400

soo

600

Magnification Fig. 3. Measured reproducibility in correcting the defocus and astigmatism with different types of specimen and at different magnifications [13]. The measurements are done with a TV camera as image pickup system.

the image content related to the defocus level of the object of interest. A reason for a decrease in robustness of autotuning in combination with spot scanning, is that the field of view of the CCD camera is not large compared to the size of the illuminating spot. Due to this small field of view, the pivot point for beam-tilt must coincide quite accurately with the object plane. When the pivot point does not coincide with the object plane, then the position of the illuminating spot will shift off the CCD camera when a beam tilt is induced.

A.J. Koster et al. / Automated microscopy for electron tomography

3.2. Camera Using a digital image pickup device, such as a CCD- or video camera, a discrete, digital electron microscopical image is obtained. Depending on the application for which the digital image will be used, the desired specifications of the digital image pickup device differ. The camera attached to the microscope ti'ansforms the electron intensity distribution to a digital image stored in the computer. This transform consists of several steps. The camera influences the performance of automated microscopy considerably because it has a finite resolution and sensitivity. Furthermore, the camera can introduce geometric distortions and intensity non-linearities. The resolution limits the choice of electron optical magnification since the camera can only detect image detail up to some given spatial frequency. An overview of the performance of the T E M / c a m e r a interfaces can be found in refs. [25-27]. For real-time control purposes, it is important to have new T E M images available quickly, containing the information required for control. For instance, when the aim is to measure and correct specimen drift, it is important to be able to measure image displacements fast (at least once a second). For control purposes a video camera has favorable features; new T E M images are available at a frequency of 2 5 - 3 0 / s . The distortion, linearity, dynamic range (256 gray values), usable image size (300 x 300 pixels), and sensitivity are good enough to measure with reasonable accuracy image displacements (about 0.2 pixels) under "normal" operating conditions (with a dose > 10 e / , ~ 2) where an average of 8 or 16 TV frames is sufficient to attain an acceptable noise reduction for calculating image shifts. The practical limitation of the image displacement m e a s u r e m e n t procedure is about 0.2 pixel. In practice, an increase of the signal-to-noise ratio, achieved by an increase in the electron dose rate, does not result in better accuracy because of other limitations. A more detailed discussion of noise in T V images can be found elsewhere [28]. The shading pattern (loosely defined as the image recorded when the camera is illuminated

217

with a flat homogeneous beam) of a TV-camera system frustrates the estimation of image displacements between low-contrast images. Due to this shading pattern, with low-contrast images the cross correlation image will show two peaks: one (s~rong) peak near the origin resulting from the shading pattern, and a second (less strong) peak resulting from the shifted low-contrast object. The strong peak near the origin in the cross correlation images is an important cause for the poor robustness of autotuning systems using video cameras because it obscures the displacement measurement of the (low-contrast) objects. Therefore, pre-processing of the TV-images (before calculation of the cross correlation), by eliminating the effects due to the shading pattern from the image is crucial for the performance of autotuning. This correction can consist of a division of the image with an image obtained with the camera illuminated with a flat illuminating beam to correct for gain variations, or one can correct simultaneously for gain and offset. However, in practice, the shading pattern of a TV camera varies considerably with time and is difficult to correct for. For radiation-sensitive, low-contrast objects, quantitative image interpretation and measurements, and three-dimensional reconstruction purposes, a cooled slow-scan CCD camera is much more suitable than a video camera. Cooled slow scan CCD cameras have much better linearity, sensitivity, dynamic range, noise characteristics and time-stability specifications, compared with a video camera. When the CCD camera characteristics are carefully measured, the linearity and dynamic range are superior to photographic plates. For on-line T E M control applications the slow-scan CCD cameras have one drawback: the readout speed. The readout speed (50-500 kpixels/s) limits the image rate to typically 0.2-2 i m a g e s / s (depending on the image size, the dynamic range per pixel and specific hardware characteristics), and is less than video rate (25-30 images/s).

3.3. Image processing Automated microscopy requires a computer system with some specific hardware require-

218

A.J. Koster et al. / Automated microscopy for electron tomography

merits. The system must include a camera interface (frame grabber for TV systems or a dedicated interface to a CCD camera system), and a T E M / c o m p u t e r interface. The latter can consist of a simple serial port for microscopes equipped with digital control or of a digital-to-analog converter board for direct lens control. It is essential to have the results of displacement measurements available quite fast, to be able to complete the autotuning procedures within

reasonable time. This requires addition of dedicated hardware, for example an array processor to speed up the Fourier transform calculations. To take advantage of the high performance of such boards it should be possible to transfer data from camera to this board (and preferably also to the display) at high speed, requiring high bus bandwidth and D M A capabilities. A very helpful aid of the (manual and visually assisted) correction of defocus and astigmatism is

Fig. 5. Correction for defocus and astigmatism on a specimen of graphatized carbon with gold particles on a thin carbon film using a TV camera at an electron-optical magnification of 88000×[13]. U p p e r left: the image prior to correction (astigmatism and defocus). Lower left: the image after correction (0.8/xm underfocus). On the right: digital diffractograms of the amorphous image section.

A.J. Koster et al. / Automated microscopy for electron tomography

the real-time display of digital diffractograms, together with the original image, on a display monitor at the TEM (Fig. 5). The operator adjusts the stigmator and defocus knobs until the displayed diffractogram is nicely circular. For high-resolution electron microscopy it is important that prior to the correction of astigmatism using this technique, the misalignment is corrected [29,30]. To be able to display those diffractograms at a high rate, special hardware and processor boards are available (Fourier board, array processor, video-image integrator). The basic computer system of choice merely functions as a host for the accelerator board, and its main function is to provide the user interface. The user interface should be designed in such a way that the operator is able to influence continuously the status of TEM and image processor. For this reason the accessibility and user-friendliness of the software is of importance. The most convenient way to control the computer is by using commands invoked with a mouse a n d / o r function keys. For more complex data collection procedures a (macro-) command language is appropriate for controlling the image collection, microscope control and on-line real-time image processing. For data collection of a large number of large images, a fast and large mass storage device, or a fast connection to another (host) computer, is important. For complex imaging procedures, a multi-tasking operating system may be useful for the simultaneous control of the microscope, the camera, and the image processor.

3.4. Microscope control For automated electron tomography, spot scanning and autotuning, the TEM must have facilities for external control of the beam tilt/shift coils, objective lens stigmator, objective lens focus, specimen tilt angle, and image shift coils. In addition, it is convenient when the communication between TEM and computer is two-way, so that the TEM is able to give specific information on its status when prompted by the external computer. During the last few years computer-micro-

219

scope interfacing has been improved considerably by the manufactures of the electron microscopes. Consequently, the use of complex imaging methods is no longer hampered by the computer-microscope interface. Nevertheless, data collection procedure implementations differ for every instrumental set-up and each type of specimen, and, ideally, one should be able to adapt each data collection procedure easily. Most suitable for the exchange of data collection procedures, the existence of a uniformly accepted standard command language would be the convenient. Therefore, to be able to exchange procedures for data collection easily, a microscope command language (MCL) is proposed and described briefly in the appendix. An extensive description is available by contacting the companies given in refs. [31-32], and is described in ref. [34].

4. Automatic electron tomography implementations

4.1. Automatic tomography of large-scale chromatin structural domains within chromosomes To study the structural organization of chromatin structural domains in chromosomes throughout the cell-cycle, we use both light and electron microscopy. Traditional light microscopy methods are limited not only by resolution, but more significantly by out-of-focus blurring from thick specimens which obscures even global changes in chromosome architecture. On the other hand, electron microscopy methods are limited not only by the inherent potential for artifacts resulting from fixation and embedding, but also by the lack of a chromatin-specific, electrondense stain. To overcome these technical limitations, we take advantage of new technology to address this issue. Using a combination of threedimensional, optical sectioning microscopy [35,36], conventional and intermediate voltage electron microscopy, we have come to evidence that a large fraction of the 30 nm diameter chromatin fibers are folded into large-scale domains measuring of the order of about 130 nm.

A.J. Koster et al. / Automated microscopy Jor electron tomography

220

Presently, we use intermediate voltage electron tomography to study the arrangement of this fiber at the several phases of the cell cycle. The microscope is equipped with a slow scan CCD camera for direct digital imaging, and a T E M / c o m p u t e r interface.

computer controlling the specimen stage), data collection typically requires 3 - 4 h. One of the present limitations in using electron tomography to study chromosome structure is the shrinkage of the plastic-embedded specimens due to the electron irradiation during data collection. To minimize the shrinkage (less than 1%) during data collection, the specimens are pre-irradiated for about 1 h. With automatic electron tomography we can decrease the amount of electron irradiation of the specimen considerably, and hope to increase the throughput and quality of the collected data. Even though it may still be necessary to pre-irradiate the specimens to stabilize them, the dramatically reduced electron dose should greatly improve the self-consistency of the data set. Presently, we are in the process extending the automation of our electron tomography approach, and have now included automatic control

4.2. Instrumental set-up

The chromatin fiber of interest is a dominant component of telophase chromosomes which because of their complexity must be embedded in Epon or Lowacryl and cut into 0.3-0.5 /zm thick sections. The aim is to resolve details of about 5 nm to allow us to fully trace the three-dimensional paths of these fibers. Data is collected typically at magnifications between 10k and 20k, with a specimen tilt range of _+70 °, using 1.25 ° tilt steps. Using a semi-automatic procedure (the CCD camera for direct digital imaging, and a

Philips EM 430 TEN

,Q 8

-t

QO

Philips C400 TEN/Computer Interface

[

VAX-VNS NET

t I I

I

DEC microVAX Ill

Parallax Display System

8

IllllllllllW

Gatan 676 CCD Camera

I

Pisotometrics Controller

Peripherals

, Mercury 20 MFlops Array Processor

Fig. 6. The instrumental set-up at UCSF. A Philips EM 430 (300 kV, LaBt,) is controlled with a Philips C400 T E M / c o m p u t e r interface. The C400 controls most of the functions of the EM 430, including the specimen stage position ( x - y - z ) and specimen tilt. A Gatan 676 cooled slow scan CCD camera is attached to the microscope as image pickup device. The digital images are transferred from the CCD camera to a MicroVAX llI computer, using a Photometrics controller unit. The MicroVAX Ill is part of a V A X - V M S network and has access to several peripherals, including a Mercury 20 MFlops array processor and several display systems. An IBM PC/386 is used as an intelligent terminal of the MicroVAX lIl, or as stand-alone control unit of the C400.

A.J. Koster et al. / Automated microscopy for electron tomography

of the eucentric height (section 2.2.2), together with pre-alignment of the images by automatic tracking and control of the specimen X - Y controls (section 2.2.1), and automatic tracking of the focus (section 2.1.3). By developing an automated electron tomography system, we have reduced the dose needed for electron tomography considerably (10-100 fold), increasing the reproducibility, accuracy and throughput of three-dimensional reconstructions. We are currently in the process of implementing the automatic correction of defocus using the Z-height control of the specimen stage in combination with the control of the objective lens current, of the objective astigmatism, and of the microscope misalignment for high resolution data collection. The instrumental set-up at UCSF is given in fig. 6. The set-up consists of a Philips EM430 (300 kV, LAB6), with a Philips C400 microscope/ computer interface, a Gatan 676 cooled ( - 3 5 ° C ) slow scan CCD camera connected to a Photometrics controller (Thompson CCD chip 1024 x 1024 elements of 19 X 19 /~2) with a readout speed of 200 kHz (12 bit), and a microVAX III computer with a Mercury 20 Mflops array processor. Using the C400, the microVAX controls serially all the required currents of coils and lenses in the TEM, including the specimen stage ( X - Y - Z , and tilt). The Mercury array processor is used to determine the image displacements using cross correlation functions. The electron intensity distribution of the image is transferred to a light intensity distribution with a thin single-crystal YAG scintillator (20 ~m) coupled to the CCD camera chip via glass fiber optics. Due to the limited modulation transfer function of the scintillator at 300 kV, the number of independent picture elements is somewhat less than the number of picture elements of the CCD chip. Typically, images are acquired with one pixel being the sum of 4 (2 x 2) pixels (5122 pixel elements).

4.3. Results 4.3.1. Correcting the CCD image for inhomogeneous noise and sensitiuity Crucial for the quality of the three-dimensional reconstruction is that the recorded CCD

221

images must be of high quality. The quality of the CCD images is ensured only when the gain and offset of each pixel in the image are accurately known. To suppress statistical effects in measuring these characteristics, the mean value of the dark current for each individual pixel element is estimated from a number of images (typically 4), and the gain is estimated for a range of CCD exposure times (typically 5-10 exposure times between 0.2 and 2 s). Furthermore, during the measurement of these characteristics, so called " b a d " pixels are identified. Bad pixels are those individual pixels which have gain and linear correlation coefficient values outside a pre-defined range compared to the average over the image. Bad pixels are due to defects in the CCD chip itself, or due to irregular features of the scintillator on top of the CCD (for example dirt particles). The characteristics of each pixel are stored in a "correction file". During data collection, the "raw" image recorded by the CCD camera is corrected by the correction file as the data is being read in from the CCD camera.

4.3.2. Correcting the CCD image for X-ray eL~ents Besides correction for the inhomogeneous distribution of the gain and offset level of each pixel, another image correction is required during data collection. When high-energy electrons hit the scintillator, X-rays are produced and detected by the CCD chip. The X-ray events are often, but not always readily, visible as brighter spots in the image (especially in noisy electron microscopic images). The number of X-ray events in an image is dependent on the high tension, and the specific CCD camera design (in particular of the choice and thickness of the scintillator and length of the fiber optic element). In our set-up (300 kV), typically between the 10 and 100 X-ray events are detected per 1 s exposure. We have developed three approaches for correcting these (randomly distributed) X-ray events: (a) median filtration, (b) detection and correction based on neighbor pixels, and (c) detection and correction using two images. A 3 x 3 or 5 x 5 median filter will effectively remove the X-ray events but at the cost of distorting the rest of the data. By contrast, a detection scheme based on

222

A.J. Koster et al. / Automated microscopy for electron tomography

testing if the ratio of intensity in the central pixel is greater than a constant times the average of its 8 neighbors will only affect selected pixels. In this case, a bad pixel is then replaced by the average surrounding value. However, since X-ray events can spread over several pixels, this a p p r o a c h often leaves remnants of the event still intact. Alt h o u g h these two approaches have the advantage of requiring only a single image, they can never be as accurate as the third method. F u r t h e r m o r e , neither m e t h o d will work reliably on electron diffraction data. In the optimal scheme to correct X-ray events, we rapidly record two images one after the other ("double take"), align the two images to correct for drift, c o m p a r e the two images, remove the intensities due to the X-ray event, and combine the two images into one. The algorithm for detecting and correcting for the X-ray events is rather complex, and it is beyond the scope of the paper to describe it in detail. However, the basic idea of the corrective algorithm is as follows. If the intensity distribution of a small cluster of pixels in the one image is considerably different from the corresponding cluster of pixels in the other image, we assume an X-ray event is detected. The X-ray event in the one image is removed using the pixel values of the o t h e r image. The final image used for display and further processing is the average of the two images. This algorithm has proved to be exceptionally reliable in removing the X-ray events from the images and diffraction patterns.

Image

s h i f t as f u n c t i o n

of

25 2o 15

25 20

~

stage Z-height

x

15-

co

.x

5-

5

0-

~0

-5,~

¢.-

-lo -

--io

-15 -

- -15

-20 -

- -20

-2S -2go0

X

.~

!

!

-2800

-2700

Z-height

~,

-25 -2600

(ticks)

Fig. 7. Determination of the eucentric height position. Given are the x - y coordinates of the image displacements between two images when the stage is tilted over + 5° at a magnification of 13.6k (1 pixel = 2.1 nm, 1 tick = 0.46 jzm). The eucentric Z-height is near the - 2800 ticks position. o p e r a t o r is able to find the correct Z-height in 1 min, then the specimen is continuously exposed for 60 s while the o p e r a t o r is looking at the screen and rocking the specimen tilt angle. O f course, the setting to the eucentric Z-height position is only accurate when the backlash of the Setting

Eucentric

height

300

200

4.3.3. A u t o m a t i c Z-height setting f o r eucentric tilting Prior to data collection for the t o m o g r a p h y series, the stage can be set automatically to its eucentric height, as described in section 2.2.2. Fig. 7 illustrates the linear relation between a change in the setting of the Z-height, and the resulting image displacement w h e n the specimen stage is tilted over 5 ° at a magnification of 13.6k. Fig. 8 gives the a m o u n t of time required for measuring the eucentric Z-height position. T h e automatic system will find the eucentric height in about 1.5 min, but with an exposure time of only 0.8 s. As a comparison, if we assume that the

1O0

Automatic Manual Fig. 8. Timing of the automatic setting of the eucentric height. The exposure time of the specimen for acquiring eight images

(for the measurement of four image displacements at four different Z-height positions), is small (0.8 s) compared to the total time needed for the procedure. The readout time of the 8 CCD images is 36 s, the computation of the cross correlation functions 7.2 s, and tilting the specimen four times over +5 ° 160 s.

A.J. Koster et aL / Automated microscopy for electron tomography

2004

Shift corrections

during

tilt s e r i e s

175 1 ¢.-

150t

=% 1oo_ I

[]

I~ mrn []

[] []

02° o711

[]

•l~aj.

[]

(1) ¢

~ D~

[]

:tb"

(2) m

~ . ~ e . ~ . ~ .,~,. , ! ~ ~-,,~,_ ,-'. ,-. ~ . ,

-80-70.60-50-4030-20-10 0 10 20 30 40 50 60 70 80

Stage tilt (o) Fig. 9. Required image shift corrections during data collection. The specimen is tilted over a range of _+70° with 2 ° increments. As is described in the text, the correction is done in three steps. Steps (1) and (2) are done using the image shift coils of the microscope. Step (3) is done within the computer; matching the last image recorded with one pixel accuracy to the last image stored.

Z-motor positioning and specimen tilt is sufficiently small to justify the (small) increments ( + 5 °) in specimen tilt. In our configuration, the backlash and error in the tilt system is extremely small and the encoder steps are ~ 0.03 °. By contrast, although the Z-motor steps are only 3 nm, the backlash is non-negligible, and must be taken into account to produce accurate and reliable Z motions.

4.3.4. Automatic pre-alignment of images during data collection Given in fig. 9 is an example of the performance for automatically tracking the image X - Y shift induced by the specimen tilt. After setting the specimen stage to be only crudely eucentric (in this case set manually) image displacements of typically 100 pixels result from tilt changes of 2 °, at a magnification of 30k (1 pixel = 0.93 nm), over the specimen tilt range of _+70 °. The alignment of the images is done at 2 ° increments, and in three steps of correction. In this description, the images used for the reconstruction are written to disk (in the "double take" mode; twice an exposure of 1 s), and the images used for the correc-

223

tive actions are single takes (exposures of 0.1 s) and are not stored. In the first step (after the increment of 2°), an image is .taken, and the image shift coils are activated to match this image to the last image that was stored on disk, following the method described in section 2.2.2. Because of imperfect calibration of the image shift coil sensitivity (due to drift during the calibration, for example), this first step may not be sufficiently accurate to precisely match the images. In fig. 9 the first step is seen to correct for the shift between the actual image and the reference with about 5 - 1 0 % accuracy. Therefore, a second corrective step is done, repeating step 1 and again requiring a 0.1 s exposure, but now the shifts are corrected with an accuracy of 0.5-10 nm. Finally, in the third step, the image is recorded and stored to disk, after having digitally aligned it to the last stored image with + 0.5 pixel accuracy. This image will be used as the new reference. The use of the last digital alignment step prevents systematic errors in image shift from slowly accumulating. Such errors give rise to a drift of the image across the field. Following this corrective scheme, the prealignment of the images is done with a _+0.5 pixel accuracy through the whole series. Typically, we record data at magnifications between the 10k and 30k, and t r a c k / c o r r e c t image shifts at 1.25-2 ° intervals. Fig. 10 gives the (exposure) time needed for the automatic tracking of the X - Y shift. The exposure time is small (2.2 s) compared to the total time (32 s) required for each tilted image. As a comparison, if the operator would correct manually for the image shift, the specimen is exposed while the operator looks at the screen. In practice, the manual correction may take 30 s plus 2 s required for actually recording the image as a double take. Therefore, considering the pronounced decrease in irradiation damage, the automatic system is greatly preferred to manual control.

4.3.5. Automatic determination of defocus Automatic measurement of defocus is done at pre-defined specimen tilt increments. The defocus is measured using basically the method de-

A.J. Koster et al. / Automated microscopy,/or electron tomography

224

Automatic

correction

image

shift

Automatic

40

tracking of defocus

30

2

~"



~

25-

o

20-

.

3O "~ --2

~a

20'

,,,a

o~

15

.~_

.

e-

- -4

:

10

10 ~a

--,6

5

0

0

2 °

[]

Delay

[]

TEM Control

[]

ReadoutCCD

[]

Computations

~

Exposure

Fig. 10. Timing of the automatic correction of the image shift. The exposures of the specimen needed for the reconstruction (2 s) and for the correction (0.2 s) are small compared to the total time, Most of the time required for the automatic correction is needed tor the readout of the CCD camera (four images, 18 s), for the computations of image displacements (four correlations, 7.2 s), and for tilting the specimen stage (5 s).

scribed in section 2.1.1. Instead of measuring the defocus together with the astigmatism, only the defocus is measure4. The measurement of defocus requires only two images (exposure of 0.1 s), formed with opposite beam tilt. Fig. 11 gives as example the accuracy and range for automatic tracking of the defocus on these types of structures as a function of the specimen tilt (_+ 50°). It is clear that the defocus needed to be adjusted over a large range (indicating that the stage is set poorly to its eucentric height). Although it was possible to correct for changes in specimen height by adjusting defocus, the large change in the objective lens current during the series will influence the quality of the reconstruction due to variations in magnification, image rotation, and illumination conditions. 4.3.6 Automatic data collection and specimen expoSll?'L"

The exposure-time required for the automatic data collection procedure is illustrated in fig. 12. The test object (a negatively stained particle) was tilted over _+70 ° and images were recorded at a

I

-40

"

!

I

-20

0

Specimen

Tilt stage

[]

'

-60

tilt increment



tilt

I

I

20

40

-8

60

(o)

Fig. 11. Automatic tracking of the defocus. Shown is the amount of refl)cusing performed manually by the operator (black dots), in order to keep the object at the same defocus during the tilt series. The amount of defocus is indicated by the amount of image displacement (the 25 pixels correspond in this case to 9.3 # m underfocus). It can be concluded that the operator did not set the eucentric height accurately, considering the amount of refl)cusing (8/~m).

Timing data collection +70 ° , 2 ° increment 200

° m

E

100

Automatic

Manual

Fig. 12. Timing of the automatic data collection procedure compared to manual operation. After every 2° specimen tilt increment, the image shifts induced by the change in tilt are corrected and the defocus is measured. The exposure time of the specimen is for the automatic system 3.3 min, considerably less than the 2.3 h needed for manual control (assuming that the operator needs 2 min to correct for image shifts and change in defocus at 2 ° increments through the whole series). The total time needed for automatic data collection is 1.2 h. Most of the time is needed for the readout of the CCD camera during the tilt series (73%), and for the computation of the cross correlation functions (16%).

A.J. Koster et al. / Automated microscopy for electron tomography

magnification of 30k. The image collection was done at 2 ° increments, as well as the image shift correction and the defocus measurements. The exposure time for the automatic system is about 3 min, considerably less than the 2.3 h needed for manual control (assuming that the operator will need 2 rain to correct for image shifts and change in defocus at 2 ° increments throughout the series). The total time needed for automatic data collection is 1.2 h. Most of the time is needed for the readout of the CCD camera during the tilt series (73%), and for the calculation of the cross correlation functions (16%). With the improved CCD readout rates now possible ( 1 - 4 M pixels/s) and faster F F T calculations, it should be possible to significantly speed the automated data collection process.

5. Conclusion

Methods and instrumentation are described for automated electron microscopy, in particular for autoalignment of T E M and automated electron tomography. Also, experiments are described illustrating the possibilities and limitations of these methods in practice. Using a computer-controllable TEM, a highquality CCD camera and a computer system for both the image processing and microscope control, the data collection process for electron tomography has been fully automated. The automatic procedure includes (a) autotuning of the T E M (measurement and correction of defocus, astigmatism and alignment), and (b) automatic image shift and focus correction and (c) determination of the eucentric height of the specimen stage. For automated electron tomography, a large number of T E M parameters need to be controlled; the most important being the defocus, beam shift, beam tilt, image shift, specimen tilt, and specimen height. The methods described are self-calibrating, which makes extensive calibration of the microscope controls unnecessary. It is shown that with automated electron tomography, the exposure time of the specimen to the electron beam is reduced by factors of 10-100

225

compared to the manual operation of the microscope. Also, data collection at a magnification of 30k, reduced the exposure time of the specimen from 2.3 h to 3.3 rain, with a specimen tilted over a _+70° tilt range with 2 ° tilt intervals. Limiting factors to the data collection time are the readout time of the CCD camera, the motor controlling the specimen tilt angle, and the calculation of cross correlation functions. With further automation and improvement of automated electron tomography procedures and instrumentation, it will be possible to reconstruct with high resolution, large, radiation-sensitive objects with only a minimum of manual operation. The adoption of a standardized microscope command language by the electron microscopic community, such as the M C L briefly described in this paper, would facilitate the exchange of implemented procedures related to automated microscopy. Important instrumental limitations are the CCD readout rate, the relative small number of resolvable pixel elements of CCD cameras attached to intermediate voltage electron microscopes ( ~ 5122), and the intrinsic backlash of the specimen stage Z-height control.

Acknowledgments

This research was supported by funds from the Howard Hughes Medical Institute and by N I H grants GM-25101 (J.W.S.) and GM-31627 (D.A.A.). We are thankful to M.B. Braunfeld for the fruitful discussions and experimental assistance. Part of the results presented were obtained at the Max Planck Institute of Biochemistry in Martinsried, Germany, as part of a research project supported by TVIPS G m b H , Germany, and funded by B r i t e / E u r a m project 3322. We are thankful to Professor W. Baumeister, Dr. D. Typke, K. Dierksen (MPI), and to H.R. Tietz and Dr. J. Chalcroft (TVIPS) for the collaboration and provision of laboratory facilities.

Appendix MCL - microscope command language

As an example of a microscope independent command language, a brief description of the

226

A.Z Koster et al. / Automated microscopy for electron tomography

microscope command language MCL [31,32] is given. The language is available for several types of electron microscopes. It is assumed that the MCL commands are embedded in a larger command language environment to perform the required image processing steps during data collection. Examples of such languages are TCL [31], S E M P E R [32] and Digital Micrograph [33]. The microscope command language (MCL) consists of command names as short as possible, but long enough to ensure that is purpose is evident for the electron microscopist. Also, the command arguments and options are suitable for programming purposes. The MCL commands give a useful and general environment for a variety of electron microscopical applications. The MCL incorporates a database of microscope-related calibrations and unit conversion functions which can be used to control the microscope not only in "ticks" (arbitrary microscope control units), but also in standardized MKS units such as meters (m), radians (rad), and Glasers (G1) and Scherzers (Sch). The commands are designed to get values from the microscope (the actual settings of a TEM parameter), set values of the microscope (absolute setting), and change values of the microscope (relative settings). Furthermore, commands are available to s t o r e / r e t r i e v e data of the TEM in flies administered by the external cornputer. The first part of the command indication, a command for the gun contains the the condenser lens contains the the objective lens contains the the specimen stage contains the the diffraction lens contains the the projection lens contains the the exposure contains the

is the SUBJECT string string string string string string string

"Gun", "Con", "Obj", "Spec", "Difr", "Proj", "Expo".

Then, a more SPECIFIC indication of the subject follows, a command for the deflection coils contains the string "Tilt" or "Shift", the stigmator coils contains the string "Stig", the pit.,ot points contains the string "Pivot", the sensitivity contains the string "Step".

The commands related to the several EM MODES are distinguished by the final word of the command, (default mode is the bright field imaging mode); a command for the dark field ends with the string " D F " , the diffraction ends with the string "DI", the low dose ends with the string " L D " , the nano-probe ends with the string " N P " . Finally the F U N C T I O N is indicated by an option, and available are /S to set a value, /G to get a value from the microscope, /C to change a value, /B to set, get or change a Boolean value, /E to get an error code message number, / D B to use the database for conversion of data. When the MCL-database is to be used, this is indicated in the command line with option / D B . The following possibilities are available for the conversion of input and output data: / D B = ( s i n ]p[d it) s SI units (meters, radians), n normalized units (Scherzers, Glasers), t ticks units (default), p polar coordinates, d angles in degrees. Those commands which are not applicable for most types of microscopes, and only valid for certain microscope types, start with a specific special character: Philips start with "P", Jeol start with "J", Zeiss start with "Z", Hitachi start with " H " . Examples of command names: G u n T i l t / C 28 38 Change the Gun Tilt setting with 28 38 (ticks), ObjShift/S 355 88 Set the illuminating beam at position 355 88 (ticks, absolute), ObjTilt/C - 5 - 5 Change the beam tilt with an amount - 5 5 (ticks), -

A.J. Koster et al. / Automated microscopy for electron tomography SpecTiltl/C/DB PKnob/C

DF

DifrFocStep/S

= d 2.5 C h a n g e t h e s p e c i m e n tilt w i t h 2,5 ° , Philips command, push knob Dark Field, 2 S e t t h e d i f f r a c t i o n foc u s s t e p t o 2.

References [1] K. Dierksen, D. Typke, R. Hegerl, A.J. Koster and W. Baumeister, Ultramicroscopy 40 (1992) 71. [2] K. Downing, Ultramicroscopy, submitted. [3] S.J. Erasmus and K.C.A. Smith, J. Microscopy 127 (1982) 185. [4] W.O. Saxton, D.J. Smith, M.A. O'Keefe, G.J. Wood and W.M. Stobbs, J. Microscopy 130 (1983) 187. [5] D.J. Smith, W.O. Saxton, M.A. O'Keefe, G.J. Wood and W.M. Stobbs, Ultramicroscopy 11 (1983) 263. [6] M. Chang and W.O. Saxton, Inst. Phys. Conf. Ser. 93 (1988) 59. [7] J.B. LePoole, Philips Tech. Rev. 2 (1947) 33. [8] K.D. van der Mast, in: Proc. 8th Eur. Congr. on Electron Microscopy, Budapest, 1984, Vol. 1, p. 3. [9] S. Nomura and S. Isamozwa, J. Microscopy 36 (1987) 157. [10] A.J. Koster, A. van den Bos and K.D. van der Mast, Ultramicroscopy 21 (1987) 209. [11] A.J. Koster, A. van den Bos and K.D. van der Mast, in: Proc. 6th Pfefferkorn Conf. on Image and Signal Processing, Niagara Falls, NY, 1987, Eds. P.W. Hawkes, F.P. Ottensmeyer, A. Rosenfeld and W.O. Saxton (Scanning Microscopy, AMF O'Hare, 1987) p. 83. [12] A.J. Koster W.J. de Ruijter, A. van den Bos and K.D. van der Mast, Ultramicroscopy 27 (1989) 251. [13] A.J. Koster and W.J. de Ruijter, Ultramicroscopy 40 (1992) 89. [14] D.L. Misell, Image Analysis, Enhancement and Interpretation (North-Holland, Amsterdam, 1978). [15] R.H. Wade and W.J. Jenkins, Optik 50 (1978) 1. [16] L. Reimer, Transmission Electron Microscopy (Springer, Berlin, 1984).

227

[17] L. Reimer and M. Ross-Messemer, J. Microscopy 155 (1988) 169. [18] D. van Dyck and M. Op de Beeck, in: Proc. 12th Int. Congr. on Electron Microscopy, Seattle, 1990, Vol. 1, p. 26. [19] H. Lichte, UItramicroscopy 20 (1986) 293. [20] W.O. Saxton, Computer Techniques for Image Processing in Electron Microscopy (Academic Press, New York, 1978). [21] A.J. Koster and A.F. de Jong, Ultramicroscopy 38 (1991) 235. [22] J.H. Spence, Experimental High-Resolution Electron Microscopy (Oxford University Press, New York, 1988). [23] W. Hoppe and R. Hegerl, in: Computer Processing of Electron Microscope Images, Topics in Current Physics, Vol. 13, Ed. P.W. Hawkes (Springer, Berlin, 1980). [24] P.J Shaw, D.A. Agard, Y. Hiraoka and J.W. Sedat, Biophys. J. 55 (1989) 101. [25] S. Kujawa and D. Krahl, Ultramicroscopy 46 (1992) 395. [26] W.J. de Ruijter and J.K. Weiss, in: Proc. 10th Pfefferkorn Conf. on Image and Signal Processing, Cambridge, 1991, in press. [27] I. Daberkov, K.H. Herrmann, Libin Liu and W.D. Rau, Ultramicroscopy 38 (1991) 215. [28] K.H. Herrmann and D. Krahl, J. Microscopy 127 (1982) 17. [29] F. Zemlin, K. Weiss, P. Schiske, W. Kunath and K.H. Herrmann, Ultramicroscopy 3 (1978) 49. [30] F. Zemlin, Ultramicroscopy 4 (1979) 241. [31] TVIPS GmbH, Herbststrasse 7, D-8035 Gauting, Germany, (89) 850-6567. [32] Synoptics Ltd., 15 The Innovation Centre, Cambridge Science Park, Milton Road, Cambridge, CB4 4BH, UK, (0223) 322267. [33] Gatan Inc., 6678 Owens Drive, Pleasanton, CA 94588, USA, (510) 463-0200. [34] W.O. Saxton and A.J. Koster, in preparation. [35] A.S. Belmont, J.W. Sedat and D.A. Agard, J. Cell Biol. 105 (1987) 77. [36] A.S. Belmont, M.B. Braunfeld, J.W. Sedat and D.A. Agard, Chomosoma (Berlin) 98 (1989) 129.

Automated microscopy for electron tomography.

Instrumentation and methodology for the automatic collection of tomographic tilt series data for the three-dimensional reconstruction of single partic...
2MB Sizes 0 Downloads 0 Views