MICROSCOPY RESEARCH AND TECHNIQUE 21:283-291 (1992)
An Image Processing/Stereological Analysis System for Transmission Electron Microscopy MAX C. POOLE Department of Anatomy and Cell Biology, School of Medicine, East Carolina University, Greenuille, North Carolina 27858-4354
KEY WORDS
Computer imaging, Cell ultrastructure, Morphometry
ABSTRACT This study examines the feasibility of combining computer image digitization, image enhancement, and point counting stereological techniques to quantify video images from transmission electron microscopes (TEM). The essential hardware consists of an IBM PC/AT, a Matrox imaging board, a digitizing tablet, a high resolution black and white monitor, and a portable mass storage device. In addition a video camera must be mounted to the TEM. The software is written in three modules which have numerous routines for image acquisition, enhancement, and quantification. Quantification is achieved by selecting an electronic lattice and superimposing it on the cell image. A cursor is moved on the lattice (via the digitizing tablet) and the points are entered into a spreadsheet. One of the major limitations of the system was the reduced resolution inherent in the current hardware. However, sampling experiments showed that one could compensate for the reduced resolution by increasing the magnification of the digitized images, and the stereological values from digitized images compared favorably to those from electron micrographs. Furthermore, the system proved advantageous by eliminating the usual darkroom work, and in enhancing low contrast tissue. In spite of several hardware limitations, the concept of quantifying computer digitized TEM images appears promising. o 1992 Wiley-Liss, Inc. mounting to the 35 mm camera port just above the INTRODUCTION Morphometric and stereological studies of cell ultra- viewing chamber. The image is formed on a YAG (ytstructure are usually done using photographic prints, trium, aluminum, garnet) crystal scintillator and is or film which is projected onto a viewing screen. In both transmitted through the crystal to a mirror below. The cases the images are often quantified by point counting image is then reflected to the port where the video techniques using acetate counting lattices overlaid camera and its optics (Nikon 150 mm lens) are onto the images. With the advent of computer digitized mounted. The camera is a Dage-MTI (Dage-MTICorp., imaging and enhancements, it was thought that this Michigan City, IN) 68 series video camera. It possesses technology could be combined with point counting ste- a Newvicon selected grade 1 imaging tube which proreological techniques to quantify the images for trans- vides excellent low light level detection with adequate mission electron microscopes (TEM). Such a system contrast. Dage-MTI notes that this tube can detect could expedite analysis by bypassing the darkroom, light at .005 lux but its “usability” for imaging is apand also provide a means of enhancing low contrast proximately 10 x greater or 0.05 lux (personal communication). In spite of this light sensitivity, the TEM images. The objective of this study was to ascertain the fea- must usually be operated at 80-90 kvolts in order to sibility of combining computer digitization, image en- get the brightest and best image using this system. The hancement, and stereological methods for analyzing camera is capable of 900 lines resolution, and is and quantifying cell ultrastructure. Furthermore, in equipped with auto-black and auto-gain controls which order to assemble an efficient but low cost system, an suppress stray background light while providing crisp effort was made to use only computer and imaging images. The camera has dual video outputs with one hardware which were microcomputer compatible. video monitor being attached directly to the camera, and the second output connecting the camera to the Therefore, the study considers how the imaging taxes video digitizing board in the computer. This camera the processing and memory capabilities of the microsystem has been used successfully in other microscopic computer configuration. systems (Miller, 1984). MATERIALS AND METHODS Hardware TEMNideo Interface. The output of a transmission electron microscope is a visual image. Therefore, a closed circuit TV imaging system designed for TEM Received December 15, 1990; accepted in revised form January 14, 1991. (Ernest Fullam, Latham, NY) was used as the means of Address reprint requests to Max C. Poole, Department of Anatomy and Cell converting the image to a video signal (Fig. 1).The Biology, School of Medicine, East Carolina University, Greenville, NC 27858system adapts easily t o the JEOL 1200 EX TEM by 4354.
0 1992 WILEY-LISS, INC.
284
M.C. POOLE
standard of 525 lined60 Hertz video scanning rates, the image is visible as 512 points horizontal by 480 lines vertical on the monitor. The image memory is expressed a t 8 bitslpixel which allows 256 intensity levels per pixel (i.e., 256 levels of grey). Another feature of the board is the 1024 x 1024 memory buffer which can be divided into quadrants, thus allowing four different images to be stored and recalled on command. Finally, the board comes with a utility software package which accesses the image control and enhancement features of the board. A Panasonic WV 5410 black and white monitor with a resolution of 750 lines is attached to the PIP 1024. This monitor provides excellent resolution for the analysis. Image Storage. Due to the memory size of the image (262,144 bytes) it is not feasible to store the images on 360 Kbyte disks or even 1.2 Mbyte disks usually used in microcomputers. Also, due to the number of I I images required for a stereological analysis it is not I advisable to fill the hard disk of the computer with I Panasonic I images. Therefore, we store the images on portable 20 I Matrox Monitor I PIP 1 0 2 4 Mbyte Iomega Bernoulli disks (Iomega Corp., Roy, I UT). Images are stored on one Bernoulli unit a t the I TEM, and the disks are removed and inserted into a second Bernoulli unit a t the IBM PC/AT for enhanceIBM 2 0 MB ment and analysis. Therefore, due to its mass storage -Summasketch and portability, we have found the Bernoulli system to PC/AT Bernoulli Data Tablet be very efficient for our imaging system. Digitizing Tablet. The last component of our hardware is a Summasketch Plus digitizing tablet (SummaFig. 1. Schematic showing the hardware configuration of the im- graphics Corp., Fairfield, CT). The tablet has a resoluage processing/stereological analysis system. Video images from the tion of 40 lines per mm, and is used with the IBM transmission electron microscope are digitized using the Matrox PIP PC/AT to circumscribe and highlight cells or organelles 1024 imaging board in an IBM PC and then stored on Bernoulli disks. The disks are carried to the IBM PClAT for enhancement and stereo- on the monitor. It is also used for optional area measurements, but more importantly the tablet is used for logical analysis. moving the cursor on the counting grids superimposed over the video image. This aspect will be described in Computer HardwarelDigitizing Board. The sys- more detail in the software discussion. tem (Fig. 1) is designed around a n IBM PC/AT microSoftware computer with a 20 megabyte hard disk and 640 kilobyte internal memory (RAM). The central processing The software was developed over a period of three unit is the Intel 80286 chip which provides adequate years, although some portions are still being debugged processing speed for the large arrays of data generated and redesigned. Most of the software is written in C, in image digitization. For convenience, a n older IBM although some hardware drivers are in assembly lanPC with 256 RAM is used at the transmission electron guage. Basically, the package is written in three modmicroscope to grab and store the images prior to anal- ules: image acquisition, image analysis, and data analysis. The images are subsequently taken to the labora- ysis. All modules operate from a series of menus listing tory for enhancement and analysis using the IBM PC/ the features available for the module (Fig. 2). The first AT. Since the TEM is a shared instrument, this two modules were completed first since a critical part of procedure does not tie up the TEM due to analysis at the project was to determine the feasibility of performthe scope, and it minimizes the moving of the computer ing stereological analyses on video digitized images. components which would be necessary if the second Both of these modules utilize image manipulation and IBM PC were not available. However, the processor of possess a variety of image utilities which take advanthe IBM PC (Intel 8088) is entirely too slow for use in tage of the 1024 x 1024 memory buffer on the PIP the system other than as a n image grabber. 1024 board. These utilities allow the operator to store, The image digitizing/processing board is the PIP- recall, and view several enhanced images in the differ1024 from Matrox Electronics Systems, Ltd. (Dorval, ent quadrants for comparison. Quebec, Canada). The board converts the video signal Image Acquisition. This module has routines for from analogue to digital and stores it in a frame buffer digitizing and storing the video images on the Berin 1/30 second. The digitized resolution of the image is noulli disk (Fig. 3). Once stored, the disk is taken to the 512 x 512 pixels which requires 262,144 bytes of mem- laboratory where image enhancements can be perory in order to store the image. Using the American formed using the same module. Since electron micro-
-
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
Image
Image
Acquisition
Analysis
Acquire
Recall
Enhance
Image
Image
Image
285
Calculations
Image
r Data File Setup
Pixel
Grey Level
Grid
Averaging
Modifications
Selection
Utilities
E Calculations
4 Organelle
Calculations
Image
Counting
Sharpening
Data Entry
Convolutions
Edge Detection Fig. 2. Schematic showing the software configuration of the system. There are three modules which integrate image acquisition, image enhancement, and point counting routines.
graphs usually have low contrast when compared to many other images, the enhancements are usually necessary prior to quantification. Although many routines were tried, we have included those enhancement routines which were most useful for the micrographs. These include drawing a histogram defining the grey scale of the image, enhancing the contrast of the micrograph by setting ranges of the histogram to black, etc., and using various 3 x 3 convolutions to alter the grey value of the pixels (Pratt, 1978). Many of the more common convolutions are already preset or can be defined by the user (MacRae, 1987). The convolutions scan each pixel and redefine the grey value of a target pixel using a formula which incorporates mathematical constants and the values of neighboring pixels (Fig. 4c). We have found that the image sharpening convolution illustrated in Figure 4 was most useful for electron micrographs. Image Analysis. This module possesses the same image enhancement and manipulation utilities found in the first module. However, it also has routines for point counting stereology and for using the digitizing tablet. It is usually very beneficial to outline the cell with the digitizing tablet prior to point counting. One of the more difficult aspects of the project was devising an efficient means of point counting the video images. A method was developed which allows one to
superimpose electronic counting lattices onto the image (Fig. 5). The lattices are 1 cm and 2 cm square grids and also 1 cm and 2 cm Weibel multipurpose grids (Weibel, 1979). They may be in either black or white, but the lattice cannot be removed from the image without removing the image. The operator moves a cursor freely along the x axis test line of the grid by using the puck of the digitizing tablet. By pressing key 1 on the puck a count is registered, and a white symbol appears on the counting lattice denoting where the count was made. After evaluating a test line, the cursor is moved to either the test line above or below by pressing either of two keys on the puck. Therefore the vertical movement of the cursor is locked into the grid, while the horizontal movement is free along the test line. Another feature of this module is the data file. Before point counting, the operator must establish a data file by defining the project name, the animal number, the tissue block number, and the EM grid number from which the images were obtained. The images are then recalled separately and counted using the routines described earlier. The acquisition of the data is displayed in a spreadsheet format. After the images are counted, the data are stored on floppy disk until calculations are made. The morphometric counts are taken from the spreadsheet and entered manually into a program for calculation of the stereological parameters.
286
M.C. POOLE
Fig. 3. A Electron micrograph of the image illustrated in the subsequent figures. This cell was chosen because of its lack of contrast. x 7,400. B: Digitized image of the same cell using the camera controls to provide some contrast. Although the resolution is obvi-
ously reduced, the aesthetic qualities of the digitized images are better when viewed on a monitor than depicted in these photographs of a monitor screen. x 7,500. C: Histogram of the grey scale of the digitized image.
Data Analysis. The last module of the software provides stereological formulae for calculating the numerical density, the cell volume, and organelle volumes and surface areas. As an example of a well-known counting method, we include the Weibel and Gomez (1962) equation in which the numerical density of a cell (Nv), and ultimately the volume of an "average" cell (Weibel and Bolender, 19731, can be derived if the size distribution K and shape (p) of a cell reference structure (e.g., nuclei) can be determined:
where N A is the number of nuclear profiles per unit area, and Vv is the volume density of the nuclei. A detailed explanation of our use of these principles is given in Poole an Kornegay (1985).Other methods using disectors, etc., could also be included in the software module.
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
IF]
287
IMAGE SHARPENING CONVOLUTION I
-1
-1
,
-1
PA
PB PC
\/ PD ,PE,PF
C Fig. 4. A. Digitized image following a grey level modification and a sharpening convolution. Although the image is still not the quality of electron micrographs, the organelle membranes are crisper than in the original digitized image. X 7,500. B: A modified grey scale histogram of the enhanced image. C: Schematic showing the principle of using a 3 x 3 convolution, and the kernel used in enhancing this image.
Calibration of the System Since the system uses a series of video devices, we have found that the best way to calibrate the system is to digitize a n image of a n electron microscopic calibra-
tion grid and calculate the final magnification on the image analysis monitor. From these data one can determine how much the image was enlarged by the system. Typically our system enlarges the image 3.1 x over the magnification of the original image. However
288
M.C. POOLE
Fig. 5. A Digitized image with an electronic square counting lattice superimposed onto the image.
x 8,100. B Digitized image with an electronic Weibel counting lattice superimposed onto the image. x 8,100.
289
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
TABLE I . The effect of increasing magnification on the organelle volume densities (V, organelle, cytoplasm) of digitized images I J P T S I L S Plectron microprauhs o f the same cells' Organelles Digitized Electron Magnification analyzed images micrographs x 24,000
x 36,000
x 75,000
Difference
(a,)
Mitoc. RER Golgi S. gran.
.03 .03 .05 .04
.06 .06 .10 .05
50 50 50 20 Mean 42.5
Mitoc. RER Golgi S. gran.
.06
.09
.07
.07
.11 .04
.14 .05
33 0 21 20 Mean 18.5
Mitoc. RER Golgi S. gran.
.ll
.10
.07
.07
.12 .04
.14 .04
TABLE 2 . Comparison of organelle volume densities (V, organelle, cytoplasm) using digitized images versus electron micrographs of the same cells'
9
0 14 0 Mean 5.8
'Each value represents the accumulative counts of 20 cells. Used 1 cm square counting lattice.
this can vary with the size of the monitor. Basically the calibration is not any different than is done in analyzing images from photographs. The characteristics of the image analysis monitor must also be considered. For example, unless you are using a flat screen monitor, there will be some peripheral image distortion. Fortunately, since the counting grid is electronically superimposed over the image and therefore subject to the same distortions, this effect has little influence on the data. However, if one uses a clear acetate grid overlaid onto the monitor screen for quantifying the image there will be some error in the values due to the peripheral distortion. Another possible source of error is in the display of the grids on the monitor. For example, the 2 cm square grid is actually a 2.1 cm square lattice when displayed on our monitor due to the characteristics of the monitor. Finally, the operator must be aware of the horizontal and vertical adjustment of the monitor. Often it is not easy to recognize the image distortion of an electron micrograph due to an improperly adjusted monitor. The monitor can be checked by digitizing the image of a video test pattern and periodically adjusting the monitor to the test pattern.
Tissue Preparation A pituitary gland was removed from a sixty day old female rat which had been killed using ether vapor, and the anterior pituitary was prepared for electron microscopy. The gland was dissected into eight pieces which were fixed in 2.5% glutaraldehyde/2% paraformaldehyde in 0.05 M phosphate buffer followed by a secondary fix in 1%phosphate buffered OsO,. The tissue was stained en bloc with 0.5% uranyl acetate and dehydrated in increasing concentrations of ethanol and embedded in araldite 502 plastic. Following ultramicrotomy, the sections were stained with lead citrate and viewed with a JEOL 1200 EX electron microscope. Only mammotrophs were analyzed in this study.
Organelle Mitochondria Rough endoplasmic reticulum Golgi complex Secretory granules 'Mean
2
S.E.M. N
=
From electron micrographs (final mag = x 23,000)
From digitized images (final mag = x 36,000)
.06 f ,014 ,118 f ,028 .08 f ,004 ,042 f ,006
,056 f ,016 ,122 f ,021 ,103 f ,008 .06 f .01
15 cells analyzed. Used 1 cm counting lattice.
SAMPLING EXPERIMENTS Resolution Effects Perhaps the greatest liability of the system is the loss of resolution of the video image versus the electron micrograph. It was thought that by increasing the magnification of the digitized images one could compensate for the effect that the lost resolution has on the stereological analysis. Therefore cells were both photographed and digitized, and enlarged to final magnifications of x 24,000, x 36,000, and x 75,000. The mitochondria, rough endoplasmic reticulum, Golgi complex, and secretory granules were counted using acetate overlays on the electon micrographs and the electronic grids on the digitized images. The volume densities (Vv) of the organelles were compared at the different magnifications. Special care was taken to count the same regions of the photographs and the digitized images. As shown in Table 1,the volume densities of the organelles in both the digitized images and electron micrographs increased as the magnification (and resolution) increased. This resolution effect has been noted earlier by Paumgartner et al. (1981). It was also interesting that the differences in the volume densities of the digitized images and electron micrographs decreased as the magnification (and resolution) was increased. This would suggest that the digitized images should be analyzed at higher magnifications to compensate for the decreased resolution inherent in the digitization of the images. Comparison of Organelle Volume Densities Using Micrographs Versus Digitized Images In an effort to validate the stereological values acquired from the video images, a study was performed comparing digitized and photographic images. Cells were both photographed and digitized, and enlarged to final magnifications of x 23,000 for the electron micrographs and ~ 3 6 , 0 0 0for the digitized images. These magnifications were chosen because the data from Table 1 also showed that the volume densities from digitized images at higher magnifications were comparable to those from electron micrographs a t lower magnifications. The volume densities (Vv) of the mitochondria, rough endoplasmic reticulum, Golgi complex, and secretory granules were compared using the two methods. The micrographs were quantified using acetate counting grids overlaid onto the micrograph, and the digitized images were quantified using the electronic lattices in the system. It was found (Table 2) that there
290
M.C. POOLE
was no significant difference in the volume densities of the same organelles when quantified using the two methods.
DISCUSSION Currently there are few image analysis systems which address the problems of transmission electron microscopy. Most systems are directed toward light microscopy where the analysis is usually done in real time with no storage of the image. However, real time analysis is not practical for TEM where the tissue cannot withstand the prolonged exposure to the electron beam while being analyzed. Furthermore due to the time required for stereological analyses, it is not practical to use the TEM for the analyses since the TEM is often shared by other investigators. Therefore this system solves these problems by electronically digitizing and storing the cellular images, which are subsequently quantified a t another station. One of the greatest advantages of this approach is the time saved by circumventing the darkroom. Many cell images are usually required for a stereological analysis and the time savings could be substantial if one routinely quantifies the images from photographic prints. While the time savings may be less if one routinely quantifies film which has been contact printed from negatives, the imaging system is still less labor intensive. Since tissue processed for immunocytochemistry, etc., often possesses low contrast, the enhancement capabilities are another advantage of the system. The major disadvantages of the system appear to be hardware related, but even these are being reduced by the introduction of new generations of hardware. The most apparent and critical limitation is that of image resolution. Currently this limitation appears to be largely due to the image digitizinglprocessing board which provides a 512 x 512 resolution. Although we have shown that one can compensate for the resolution by digitizing and quantifying higher magnification images, the image lacks the crispness and the aesthetics of a photograph. Fortunately new image digitizinglprocessing boards are being introduced for microcomputers which have resolutions of approximately 1024 x 1024 pixels. While this fourfold increase in board resolution should provide an aesthetically pleasing image and be more than adequate for analysis, other hardware limitations will become more apparent such as the resolutions of the camera, video monitors, and even the YAG scintillation crystal. Currently these hardware limitations are not as pronounced as that of the image digitizing resolution. Another hardware limitation is the processing speed. The IBM PC with the Intel 8088 processor may take ten minutes to complete a routine convolution. Therefore the processor should be at least a 80286 and preferrably a 80386 to dramatically reduce the processing time. For example, a routine 3 x 3 convolution will require at least 2.3 million math operations for a 512 x 512 image. Due to the slow processing speed of even the IBM PCIAT we encountered another problem when frame averaging multiple images during the grab. Frame averaging routines are useful in sharpening the image by reducing the video noise, and Squire et al.
(1986) noted that they often give a result comparable to fast Fourier transformations (FFT). However, after a series of preliminary experiments we decided not to incorporate frame averaging or FFTs into the software due to their processing time. Please note, however, that our board was one of the first manufactured by Matrox for IBM PCs, and that the newer image digitizing/processing boards usually have on-board processors which can quickly handle frame averaging routines. The issue of processing speed will become even more critical as the resolution of image digitizinglprocessing boards approaches 1024 x 1024 or approximately 1 million bytes per image. However, perhaps this will not be a great liability since the trend among imaging board manufacturers seems to be toward providing on-board processors. Finally, a third consideration is the mass storage required for multiple images. As mentioned previously a 512 x 512 image requires 262,144 bytes of memory, and while the Matrox PIP 1024 has a memory buffer which allows us to store and view images without using the computer memory, the long-term storage of these images is a different matter. In our system this was solved by using portable Bernoulli 20 MByte disks. However, as the image resolution improves and the resulting image memory size increases with new image digitizing boards, mass storage will become an even greater problem. The primary objective of this study was to ascertain the feasibility of morphometrically quantifying transmission electron microscopic images which were acquired by electronic digitization. Although there are limitations, many of these are disappearing as hardware is evolving. The concept is attractive due to its time-saving features which allow morphometric analyses immediately after digitizing the image, and also its image enhancement capabilities. In summary this study shows that the concept is feasible, and practical to a large extent. As new hardware is developed, this method of acquiring and quantifying TEM images appears very promising.
ACKNOWLEDGMENTS A special thanks is given to W. Daniel Kornegay I11 for his skilled programming and creative software design of this system. A second thanks is given to Dr. Carl R. Morgan for the financial support to try something different.
REFERENCES MacRae, D. (1987) Convolution hardware accelerates imaging tasks. ESD: The Electronic System Design Magazine, 17:58-60. Miller, M. (1984) Specialized video operations: Low-light pickup and high resolution detail mark advances in cameras for medicine. Video Systems, 10:32-47. Paumgartner, D., Losa, G., and Weibel, E.R. (1981) Resolution effect on the stereological estimation of surface and volume and its interpretation in terms of fractal dimensions. J. Microsc., 121:51-63. Poole, M.C., and Kornegay, W.D., I11 (1985): A microcomputer approach to point-counting stereology. In: The Microcomputer in Cell and Neurobiology Research. R.R. Mize, ed. Elsevier Publishing, New York, pp. 217-232. Pratt, W.K. (1978) Image enhancement and restoration. In: Digital Image Processing. W.K. Pratt, ed. Wiley and Sons, New York, pp. 307-468.
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES Squire, J.M., Luther, P.K., and Agnew, G.D. (1986) Averaging of periodic images using a microcomputer. J . Microsc., 142:289-300. Weibel, E.R. (1979) Point counting methods. In: Stereological Methods. E.R. Weibel, ed. Academic Press, New York, pp. 101-161. Weibel, E.R., and Bolender, R.P. (1973) Stereological techniques for
291
electron microscopy. In: Principles and Techniques of Electron Microscopy. Vol. 3. M.A. Hyatt, ed. Van Nostrand Reinhold, New York, pp. 239-296. Weibel, E.R., and Gomez, D.M.(1962) A principle for counting tissue structures on random sections. J . Appl. Physiol., 17:343-348.
An Image Processing/Stereological Analysis System for Transmission Electron Microscopy MAX C. POOLE Department of Anatomy and Cell Biology, School of Medicine, East Carolina University, Greenuille, North Carolina 27858-4354
KEY WORDS
Computer imaging, Cell ultrastructure, Morphometry
ABSTRACT This study examines the feasibility of combining computer image digitization, image enhancement, and point counting stereological techniques to quantify video images from transmission electron microscopes (TEM). The essential hardware consists of an IBM PC/AT, a Matrox imaging board, a digitizing tablet, a high resolution black and white monitor, and a portable mass storage device. In addition a video camera must be mounted to the TEM. The software is written in three modules which have numerous routines for image acquisition, enhancement, and quantification. Quantification is achieved by selecting an electronic lattice and superimposing it on the cell image. A cursor is moved on the lattice (via the digitizing tablet) and the points are entered into a spreadsheet. One of the major limitations of the system was the reduced resolution inherent in the current hardware. However, sampling experiments showed that one could compensate for the reduced resolution by increasing the magnification of the digitized images, and the stereological values from digitized images compared favorably to those from electron micrographs. Furthermore, the system proved advantageous by eliminating the usual darkroom work, and in enhancing low contrast tissue. In spite of several hardware limitations, the concept of quantifying computer digitized TEM images appears promising. o 1992 Wiley-Liss, Inc. mounting to the 35 mm camera port just above the INTRODUCTION Morphometric and stereological studies of cell ultra- viewing chamber. The image is formed on a YAG (ytstructure are usually done using photographic prints, trium, aluminum, garnet) crystal scintillator and is or film which is projected onto a viewing screen. In both transmitted through the crystal to a mirror below. The cases the images are often quantified by point counting image is then reflected to the port where the video techniques using acetate counting lattices overlaid camera and its optics (Nikon 150 mm lens) are onto the images. With the advent of computer digitized mounted. The camera is a Dage-MTI (Dage-MTICorp., imaging and enhancements, it was thought that this Michigan City, IN) 68 series video camera. It possesses technology could be combined with point counting ste- a Newvicon selected grade 1 imaging tube which proreological techniques to quantify the images for trans- vides excellent low light level detection with adequate mission electron microscopes (TEM). Such a system contrast. Dage-MTI notes that this tube can detect could expedite analysis by bypassing the darkroom, light at .005 lux but its “usability” for imaging is apand also provide a means of enhancing low contrast proximately 10 x greater or 0.05 lux (personal communication). In spite of this light sensitivity, the TEM images. The objective of this study was to ascertain the fea- must usually be operated at 80-90 kvolts in order to sibility of combining computer digitization, image en- get the brightest and best image using this system. The hancement, and stereological methods for analyzing camera is capable of 900 lines resolution, and is and quantifying cell ultrastructure. Furthermore, in equipped with auto-black and auto-gain controls which order to assemble an efficient but low cost system, an suppress stray background light while providing crisp effort was made to use only computer and imaging images. The camera has dual video outputs with one hardware which were microcomputer compatible. video monitor being attached directly to the camera, and the second output connecting the camera to the Therefore, the study considers how the imaging taxes video digitizing board in the computer. This camera the processing and memory capabilities of the microsystem has been used successfully in other microscopic computer configuration. systems (Miller, 1984). MATERIALS AND METHODS Hardware TEMNideo Interface. The output of a transmission electron microscope is a visual image. Therefore, a closed circuit TV imaging system designed for TEM Received December 15, 1990; accepted in revised form January 14, 1991. (Ernest Fullam, Latham, NY) was used as the means of Address reprint requests to Max C. Poole, Department of Anatomy and Cell converting the image to a video signal (Fig. 1).The Biology, School of Medicine, East Carolina University, Greenville, NC 27858system adapts easily t o the JEOL 1200 EX TEM by 4354.
0 1992 WILEY-LISS, INC.
284
M.C. POOLE
standard of 525 lined60 Hertz video scanning rates, the image is visible as 512 points horizontal by 480 lines vertical on the monitor. The image memory is expressed a t 8 bitslpixel which allows 256 intensity levels per pixel (i.e., 256 levels of grey). Another feature of the board is the 1024 x 1024 memory buffer which can be divided into quadrants, thus allowing four different images to be stored and recalled on command. Finally, the board comes with a utility software package which accesses the image control and enhancement features of the board. A Panasonic WV 5410 black and white monitor with a resolution of 750 lines is attached to the PIP 1024. This monitor provides excellent resolution for the analysis. Image Storage. Due to the memory size of the image (262,144 bytes) it is not feasible to store the images on 360 Kbyte disks or even 1.2 Mbyte disks usually used in microcomputers. Also, due to the number of I I images required for a stereological analysis it is not I advisable to fill the hard disk of the computer with I Panasonic I images. Therefore, we store the images on portable 20 I Matrox Monitor I PIP 1 0 2 4 Mbyte Iomega Bernoulli disks (Iomega Corp., Roy, I UT). Images are stored on one Bernoulli unit a t the I TEM, and the disks are removed and inserted into a second Bernoulli unit a t the IBM PC/AT for enhanceIBM 2 0 MB ment and analysis. Therefore, due to its mass storage -Summasketch and portability, we have found the Bernoulli system to PC/AT Bernoulli Data Tablet be very efficient for our imaging system. Digitizing Tablet. The last component of our hardware is a Summasketch Plus digitizing tablet (SummaFig. 1. Schematic showing the hardware configuration of the im- graphics Corp., Fairfield, CT). The tablet has a resoluage processing/stereological analysis system. Video images from the tion of 40 lines per mm, and is used with the IBM transmission electron microscope are digitized using the Matrox PIP PC/AT to circumscribe and highlight cells or organelles 1024 imaging board in an IBM PC and then stored on Bernoulli disks. The disks are carried to the IBM PClAT for enhancement and stereo- on the monitor. It is also used for optional area measurements, but more importantly the tablet is used for logical analysis. moving the cursor on the counting grids superimposed over the video image. This aspect will be described in Computer HardwarelDigitizing Board. The sys- more detail in the software discussion. tem (Fig. 1) is designed around a n IBM PC/AT microSoftware computer with a 20 megabyte hard disk and 640 kilobyte internal memory (RAM). The central processing The software was developed over a period of three unit is the Intel 80286 chip which provides adequate years, although some portions are still being debugged processing speed for the large arrays of data generated and redesigned. Most of the software is written in C, in image digitization. For convenience, a n older IBM although some hardware drivers are in assembly lanPC with 256 RAM is used at the transmission electron guage. Basically, the package is written in three modmicroscope to grab and store the images prior to anal- ules: image acquisition, image analysis, and data analysis. The images are subsequently taken to the labora- ysis. All modules operate from a series of menus listing tory for enhancement and analysis using the IBM PC/ the features available for the module (Fig. 2). The first AT. Since the TEM is a shared instrument, this two modules were completed first since a critical part of procedure does not tie up the TEM due to analysis at the project was to determine the feasibility of performthe scope, and it minimizes the moving of the computer ing stereological analyses on video digitized images. components which would be necessary if the second Both of these modules utilize image manipulation and IBM PC were not available. However, the processor of possess a variety of image utilities which take advanthe IBM PC (Intel 8088) is entirely too slow for use in tage of the 1024 x 1024 memory buffer on the PIP the system other than as a n image grabber. 1024 board. These utilities allow the operator to store, The image digitizing/processing board is the PIP- recall, and view several enhanced images in the differ1024 from Matrox Electronics Systems, Ltd. (Dorval, ent quadrants for comparison. Quebec, Canada). The board converts the video signal Image Acquisition. This module has routines for from analogue to digital and stores it in a frame buffer digitizing and storing the video images on the Berin 1/30 second. The digitized resolution of the image is noulli disk (Fig. 3). Once stored, the disk is taken to the 512 x 512 pixels which requires 262,144 bytes of mem- laboratory where image enhancements can be perory in order to store the image. Using the American formed using the same module. Since electron micro-
-
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
Image
Image
Acquisition
Analysis
Acquire
Recall
Enhance
Image
Image
Image
285
Calculations
Image
r Data File Setup
Pixel
Grey Level
Grid
Averaging
Modifications
Selection
Utilities
E Calculations
4 Organelle
Calculations
Image
Counting
Sharpening
Data Entry
Convolutions
Edge Detection Fig. 2. Schematic showing the software configuration of the system. There are three modules which integrate image acquisition, image enhancement, and point counting routines.
graphs usually have low contrast when compared to many other images, the enhancements are usually necessary prior to quantification. Although many routines were tried, we have included those enhancement routines which were most useful for the micrographs. These include drawing a histogram defining the grey scale of the image, enhancing the contrast of the micrograph by setting ranges of the histogram to black, etc., and using various 3 x 3 convolutions to alter the grey value of the pixels (Pratt, 1978). Many of the more common convolutions are already preset or can be defined by the user (MacRae, 1987). The convolutions scan each pixel and redefine the grey value of a target pixel using a formula which incorporates mathematical constants and the values of neighboring pixels (Fig. 4c). We have found that the image sharpening convolution illustrated in Figure 4 was most useful for electron micrographs. Image Analysis. This module possesses the same image enhancement and manipulation utilities found in the first module. However, it also has routines for point counting stereology and for using the digitizing tablet. It is usually very beneficial to outline the cell with the digitizing tablet prior to point counting. One of the more difficult aspects of the project was devising an efficient means of point counting the video images. A method was developed which allows one to
superimpose electronic counting lattices onto the image (Fig. 5). The lattices are 1 cm and 2 cm square grids and also 1 cm and 2 cm Weibel multipurpose grids (Weibel, 1979). They may be in either black or white, but the lattice cannot be removed from the image without removing the image. The operator moves a cursor freely along the x axis test line of the grid by using the puck of the digitizing tablet. By pressing key 1 on the puck a count is registered, and a white symbol appears on the counting lattice denoting where the count was made. After evaluating a test line, the cursor is moved to either the test line above or below by pressing either of two keys on the puck. Therefore the vertical movement of the cursor is locked into the grid, while the horizontal movement is free along the test line. Another feature of this module is the data file. Before point counting, the operator must establish a data file by defining the project name, the animal number, the tissue block number, and the EM grid number from which the images were obtained. The images are then recalled separately and counted using the routines described earlier. The acquisition of the data is displayed in a spreadsheet format. After the images are counted, the data are stored on floppy disk until calculations are made. The morphometric counts are taken from the spreadsheet and entered manually into a program for calculation of the stereological parameters.
286
M.C. POOLE
Fig. 3. A Electron micrograph of the image illustrated in the subsequent figures. This cell was chosen because of its lack of contrast. x 7,400. B: Digitized image of the same cell using the camera controls to provide some contrast. Although the resolution is obvi-
ously reduced, the aesthetic qualities of the digitized images are better when viewed on a monitor than depicted in these photographs of a monitor screen. x 7,500. C: Histogram of the grey scale of the digitized image.
Data Analysis. The last module of the software provides stereological formulae for calculating the numerical density, the cell volume, and organelle volumes and surface areas. As an example of a well-known counting method, we include the Weibel and Gomez (1962) equation in which the numerical density of a cell (Nv), and ultimately the volume of an "average" cell (Weibel and Bolender, 19731, can be derived if the size distribution K and shape (p) of a cell reference structure (e.g., nuclei) can be determined:
where N A is the number of nuclear profiles per unit area, and Vv is the volume density of the nuclei. A detailed explanation of our use of these principles is given in Poole an Kornegay (1985).Other methods using disectors, etc., could also be included in the software module.
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
IF]
287
IMAGE SHARPENING CONVOLUTION I
-1
-1
,
-1
PA
PB PC
\/ PD ,PE,PF
C Fig. 4. A. Digitized image following a grey level modification and a sharpening convolution. Although the image is still not the quality of electron micrographs, the organelle membranes are crisper than in the original digitized image. X 7,500. B: A modified grey scale histogram of the enhanced image. C: Schematic showing the principle of using a 3 x 3 convolution, and the kernel used in enhancing this image.
Calibration of the System Since the system uses a series of video devices, we have found that the best way to calibrate the system is to digitize a n image of a n electron microscopic calibra-
tion grid and calculate the final magnification on the image analysis monitor. From these data one can determine how much the image was enlarged by the system. Typically our system enlarges the image 3.1 x over the magnification of the original image. However
288
M.C. POOLE
Fig. 5. A Digitized image with an electronic square counting lattice superimposed onto the image.
x 8,100. B Digitized image with an electronic Weibel counting lattice superimposed onto the image. x 8,100.
289
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES
TABLE I . The effect of increasing magnification on the organelle volume densities (V, organelle, cytoplasm) of digitized images I J P T S I L S Plectron microprauhs o f the same cells' Organelles Digitized Electron Magnification analyzed images micrographs x 24,000
x 36,000
x 75,000
Difference
(a,)
Mitoc. RER Golgi S. gran.
.03 .03 .05 .04
.06 .06 .10 .05
50 50 50 20 Mean 42.5
Mitoc. RER Golgi S. gran.
.06
.09
.07
.07
.11 .04
.14 .05
33 0 21 20 Mean 18.5
Mitoc. RER Golgi S. gran.
.ll
.10
.07
.07
.12 .04
.14 .04
TABLE 2 . Comparison of organelle volume densities (V, organelle, cytoplasm) using digitized images versus electron micrographs of the same cells'
9
0 14 0 Mean 5.8
'Each value represents the accumulative counts of 20 cells. Used 1 cm square counting lattice.
this can vary with the size of the monitor. Basically the calibration is not any different than is done in analyzing images from photographs. The characteristics of the image analysis monitor must also be considered. For example, unless you are using a flat screen monitor, there will be some peripheral image distortion. Fortunately, since the counting grid is electronically superimposed over the image and therefore subject to the same distortions, this effect has little influence on the data. However, if one uses a clear acetate grid overlaid onto the monitor screen for quantifying the image there will be some error in the values due to the peripheral distortion. Another possible source of error is in the display of the grids on the monitor. For example, the 2 cm square grid is actually a 2.1 cm square lattice when displayed on our monitor due to the characteristics of the monitor. Finally, the operator must be aware of the horizontal and vertical adjustment of the monitor. Often it is not easy to recognize the image distortion of an electron micrograph due to an improperly adjusted monitor. The monitor can be checked by digitizing the image of a video test pattern and periodically adjusting the monitor to the test pattern.
Tissue Preparation A pituitary gland was removed from a sixty day old female rat which had been killed using ether vapor, and the anterior pituitary was prepared for electron microscopy. The gland was dissected into eight pieces which were fixed in 2.5% glutaraldehyde/2% paraformaldehyde in 0.05 M phosphate buffer followed by a secondary fix in 1%phosphate buffered OsO,. The tissue was stained en bloc with 0.5% uranyl acetate and dehydrated in increasing concentrations of ethanol and embedded in araldite 502 plastic. Following ultramicrotomy, the sections were stained with lead citrate and viewed with a JEOL 1200 EX electron microscope. Only mammotrophs were analyzed in this study.
Organelle Mitochondria Rough endoplasmic reticulum Golgi complex Secretory granules 'Mean
2
S.E.M. N
=
From electron micrographs (final mag = x 23,000)
From digitized images (final mag = x 36,000)
.06 f ,014 ,118 f ,028 .08 f ,004 ,042 f ,006
,056 f ,016 ,122 f ,021 ,103 f ,008 .06 f .01
15 cells analyzed. Used 1 cm counting lattice.
SAMPLING EXPERIMENTS Resolution Effects Perhaps the greatest liability of the system is the loss of resolution of the video image versus the electron micrograph. It was thought that by increasing the magnification of the digitized images one could compensate for the effect that the lost resolution has on the stereological analysis. Therefore cells were both photographed and digitized, and enlarged to final magnifications of x 24,000, x 36,000, and x 75,000. The mitochondria, rough endoplasmic reticulum, Golgi complex, and secretory granules were counted using acetate overlays on the electon micrographs and the electronic grids on the digitized images. The volume densities (Vv) of the organelles were compared at the different magnifications. Special care was taken to count the same regions of the photographs and the digitized images. As shown in Table 1,the volume densities of the organelles in both the digitized images and electron micrographs increased as the magnification (and resolution) increased. This resolution effect has been noted earlier by Paumgartner et al. (1981). It was also interesting that the differences in the volume densities of the digitized images and electron micrographs decreased as the magnification (and resolution) was increased. This would suggest that the digitized images should be analyzed at higher magnifications to compensate for the decreased resolution inherent in the digitization of the images. Comparison of Organelle Volume Densities Using Micrographs Versus Digitized Images In an effort to validate the stereological values acquired from the video images, a study was performed comparing digitized and photographic images. Cells were both photographed and digitized, and enlarged to final magnifications of x 23,000 for the electron micrographs and ~ 3 6 , 0 0 0for the digitized images. These magnifications were chosen because the data from Table 1 also showed that the volume densities from digitized images at higher magnifications were comparable to those from electron micrographs a t lower magnifications. The volume densities (Vv) of the mitochondria, rough endoplasmic reticulum, Golgi complex, and secretory granules were compared using the two methods. The micrographs were quantified using acetate counting grids overlaid onto the micrograph, and the digitized images were quantified using the electronic lattices in the system. It was found (Table 2) that there
290
M.C. POOLE
was no significant difference in the volume densities of the same organelles when quantified using the two methods.
DISCUSSION Currently there are few image analysis systems which address the problems of transmission electron microscopy. Most systems are directed toward light microscopy where the analysis is usually done in real time with no storage of the image. However, real time analysis is not practical for TEM where the tissue cannot withstand the prolonged exposure to the electron beam while being analyzed. Furthermore due to the time required for stereological analyses, it is not practical to use the TEM for the analyses since the TEM is often shared by other investigators. Therefore this system solves these problems by electronically digitizing and storing the cellular images, which are subsequently quantified a t another station. One of the greatest advantages of this approach is the time saved by circumventing the darkroom. Many cell images are usually required for a stereological analysis and the time savings could be substantial if one routinely quantifies the images from photographic prints. While the time savings may be less if one routinely quantifies film which has been contact printed from negatives, the imaging system is still less labor intensive. Since tissue processed for immunocytochemistry, etc., often possesses low contrast, the enhancement capabilities are another advantage of the system. The major disadvantages of the system appear to be hardware related, but even these are being reduced by the introduction of new generations of hardware. The most apparent and critical limitation is that of image resolution. Currently this limitation appears to be largely due to the image digitizinglprocessing board which provides a 512 x 512 resolution. Although we have shown that one can compensate for the resolution by digitizing and quantifying higher magnification images, the image lacks the crispness and the aesthetics of a photograph. Fortunately new image digitizinglprocessing boards are being introduced for microcomputers which have resolutions of approximately 1024 x 1024 pixels. While this fourfold increase in board resolution should provide an aesthetically pleasing image and be more than adequate for analysis, other hardware limitations will become more apparent such as the resolutions of the camera, video monitors, and even the YAG scintillation crystal. Currently these hardware limitations are not as pronounced as that of the image digitizing resolution. Another hardware limitation is the processing speed. The IBM PC with the Intel 8088 processor may take ten minutes to complete a routine convolution. Therefore the processor should be at least a 80286 and preferrably a 80386 to dramatically reduce the processing time. For example, a routine 3 x 3 convolution will require at least 2.3 million math operations for a 512 x 512 image. Due to the slow processing speed of even the IBM PCIAT we encountered another problem when frame averaging multiple images during the grab. Frame averaging routines are useful in sharpening the image by reducing the video noise, and Squire et al.
(1986) noted that they often give a result comparable to fast Fourier transformations (FFT). However, after a series of preliminary experiments we decided not to incorporate frame averaging or FFTs into the software due to their processing time. Please note, however, that our board was one of the first manufactured by Matrox for IBM PCs, and that the newer image digitizing/processing boards usually have on-board processors which can quickly handle frame averaging routines. The issue of processing speed will become even more critical as the resolution of image digitizinglprocessing boards approaches 1024 x 1024 or approximately 1 million bytes per image. However, perhaps this will not be a great liability since the trend among imaging board manufacturers seems to be toward providing on-board processors. Finally, a third consideration is the mass storage required for multiple images. As mentioned previously a 512 x 512 image requires 262,144 bytes of memory, and while the Matrox PIP 1024 has a memory buffer which allows us to store and view images without using the computer memory, the long-term storage of these images is a different matter. In our system this was solved by using portable Bernoulli 20 MByte disks. However, as the image resolution improves and the resulting image memory size increases with new image digitizing boards, mass storage will become an even greater problem. The primary objective of this study was to ascertain the feasibility of morphometrically quantifying transmission electron microscopic images which were acquired by electronic digitization. Although there are limitations, many of these are disappearing as hardware is evolving. The concept is attractive due to its time-saving features which allow morphometric analyses immediately after digitizing the image, and also its image enhancement capabilities. In summary this study shows that the concept is feasible, and practical to a large extent. As new hardware is developed, this method of acquiring and quantifying TEM images appears very promising.
ACKNOWLEDGMENTS A special thanks is given to W. Daniel Kornegay I11 for his skilled programming and creative software design of this system. A second thanks is given to Dr. Carl R. Morgan for the financial support to try something different.
REFERENCES MacRae, D. (1987) Convolution hardware accelerates imaging tasks. ESD: The Electronic System Design Magazine, 17:58-60. Miller, M. (1984) Specialized video operations: Low-light pickup and high resolution detail mark advances in cameras for medicine. Video Systems, 10:32-47. Paumgartner, D., Losa, G., and Weibel, E.R. (1981) Resolution effect on the stereological estimation of surface and volume and its interpretation in terms of fractal dimensions. J. Microsc., 121:51-63. Poole, M.C., and Kornegay, W.D., I11 (1985): A microcomputer approach to point-counting stereology. In: The Microcomputer in Cell and Neurobiology Research. R.R. Mize, ed. Elsevier Publishing, New York, pp. 217-232. Pratt, W.K. (1978) Image enhancement and restoration. In: Digital Image Processing. W.K. Pratt, ed. Wiley and Sons, New York, pp. 307-468.
STEREOLOGICAL SYSTEM FOR TEM VIDEO IMAGES Squire, J.M., Luther, P.K., and Agnew, G.D. (1986) Averaging of periodic images using a microcomputer. J . Microsc., 142:289-300. Weibel, E.R. (1979) Point counting methods. In: Stereological Methods. E.R. Weibel, ed. Academic Press, New York, pp. 101-161. Weibel, E.R., and Bolender, R.P. (1973) Stereological techniques for
291
electron microscopy. In: Principles and Techniques of Electron Microscopy. Vol. 3. M.A. Hyatt, ed. Van Nostrand Reinhold, New York, pp. 239-296. Weibel, E.R., and Gomez, D.M.(1962) A principle for counting tissue structures on random sections. J . Appl. Physiol., 17:343-348.
