1991, The British Journal of Radiology, 64, 1030-1035

Registration of MR and CT images for skull base surgery using point-like anatomical features By D. L G. Hill, MSc, D. J. Hawkes, PhD, J. E. Crossman, BSc, *M. J. Gleeson, FRCS, T. C. S. Cox, FRCR, E. E. C. M. L Bracey, *A. J. Strong, FRCS and P. Graves, DCR Division of Radiological Sciences and 'Department of Surgery, UMDS, Guy's Hospital, London SE1 9RT, UK (Received February 1991 and in revised form May 1991) Keywords: Multimodal, Registration, Image processing, Surgery planning, Synergistic imaging

Abstract. We have developed a registration technique for combining magnetic resonance imaging (MRI) and computed tomography (CT) images of the skull base for use in surgical planning. The technique is based on user identification of point-like landmarks visible in both modalities. The combination of images involves a small amount of expert interaction, is relatively quick and preliminary evaluation indicates that it is accurate to within 1.5 mm. Registered or fused images can be viewed either on an image processing workstation, or fused images can be printed onto conventional film for convenience in clinical use. We present one patient in order to demonstrate the technique's indications and advantages.

In skull base surgery, the information provided by magnetic resonance imaging (MRI) and X-ray computed tomography (CT) is complementary. The soft tissue pathology and nervous tissue are better visualized by MRI, while the bony structures of the temporal bone, including the middle ear, cochlea and internal auditory meatus, are better visualized by CT. Patients undergoing skull base surgery at this institution have thin slice MRI and CT scans for surgical planning purposes. Conventionally these are printed onto films and displayed side by side on a light box using the expertise of the diagnostician or surgeon to mentally fuse the images. We consider this method of viewing the images to be suboptimal and, therefore, hypothesized that considerable benefit could be gained by combining both sets of data. There are two stages to achieving such a combined display. The first is to register the images to determine the geometric transformation relating the coordinate systems of the two 3D data sets, and the second involves fusing the images, by generating a single image from the two registered originals. Various techniques have been proposed for registering images from different tomographic modalities. External markers attached to the patient's skin or to a thermoplastic face mask have been used to register nuclear medicine images to MR or CT images (Evans et al, 1988; Fleming, 1989; Wilson & Mountz, 1989; Hawkes et al, 1990; Meltzer et al, 1990; Rizzo et al, 1990). These methods do not provide sufficient accuracy for registering MR and CT to each other because of movement of the markers. Skin markers are most accurate when left in position for both scans, but this is inconvenient if not impossible when scans are performed on different days and/or at different institutions. Stereotactic frames have been used for registering MR images with CT images (Peters etal, 1986; Schad etal, 1030

1987; Henri et al, 1989; Zhang et al, 1990), but the invasiveness of a stereotactic frame limits its use to stereotactic procedures and, in any case, stereotactic frames are often too large to fit in the head coils of MR scanners. A more flexible approach to image combination is to use anatomical features visible in each modality rather than using external markers or frames to achieve registration. We propose that the use of anatomical features for registration has the potential for greater registration accuracy than external markers and requires less interference with normal radiographic procedures. Various anatomical features have been proposed for registering MR and CT images. The features can comprise points (Chen et al, 1985; Boesecke et al, 1990), centroids of segmented regions (Bartoo & Hanson, 1989) or surfaces (Gamboa-Aldeco et al, 1986; Pelizzari et al, 1989). Since slice planes will, in general, be different for the MR and CT images, it is essential that the registration is in 3D. In our application only a small portion of the head is imaged (an axial distance of a few cm centred on the temporal bone). Pelizzari et al (1989) have reported using the skin surface to register MR, CT and positron emission tomography (PET) images, but unfortunately we have found that in our application the area of skin surface sampled by both imaging modalities is rather small and hence skin surface fitting is an ill posed problem. More fundamentally, the skin surface, especially around the ears and base of the skull, is distorted differently and non-reproducibly by the different head restraining devices used in MR and CT scanners. Like Chen and Boesecke (Chen etal, 1985; Boesecke etal, 1990) we use point landmarks to achieve registration. Three linearly independent 3D points are required to determine uniquely the 3D coordinate system of each imaging modality if voxel dimensions are known. However, we propose using a large number of points to The British Journal of Radiology, November 1991

Registration of MR and CT images for skull base surgery

improve registration reliability and accuracy, as Evans et al (1989) have reported for registering MR and PET data. The skull base region has several features visible on both MR and CT images, but the accuracy with which individual features can be located is not necessarily sufficient for registration. By using additional points and a least squares algorithm for registration, it is possible to achieve a registration accuracy greater than the accuracy with which each point is located, assuming the location error is random. In this paper we present the application of this technique to one patient. The patient had an acoustic neuroma on the right side, which was surgically removed using a translabyrinthine approach. Materials and methods Acquisition MR and CT images of the skull base are acquired on an outpatient basis at different institutions on different days. MR images are acquired with 256 x 256 pixels per slice, and a slice thickness of between 2 mm and 5 mm (Philips Gyroscan SI5). The pulse sequence used is selected according to the normal clinical protocol. CT images are acquired with 512x512 pixels per slice and a slice thickness of between 1.5 mm and 3 mm (GE Medical Systems CT 9800). Plan views from the first set of images to be acquired can be used as an approximate aid to positioning the slices for the second modality. This is not essential for registration, but it helps ensure that the same volume of the patient is imaged in both modalities and it assists subsequent location of the point-like landmarks if the images from the different modalities are approximately co-planar. The images are transferred onto a network of SUN workstations for further processing. Each CT slice is translated anteriorly or posteriorly to correct for gantry angle and hence produce an orthogonal data set, but no distortion correction was necessary on the MR images. We have measured the geometric distortion of our MR scanner (Philips Gyroscan SI5) to be a maximum voxel displacement of 2.3 mm for transverse slices at the periphery of the head coils field of view, with an average integral non-linearity of less than 0-8 mm (Price, 1990). This is within the registration accuracy required for our application. Criteria for selection of point-like landmarks The following are examples, but not an exhaustive list, of point-like structures which may be used for registration. A point anatomical structure (e.g. the apical turn of the cochlea). The intersection of two linear structures (e.g. blood vessel bifurcation or confluence). A particular topographic feature on a surface (e.g. an identifiable part of a sulcus or gyrus). The intersection of a surface structure with a linear structure (e.g. where a nerve passes through a foramen). The intersection of three surface structures. Vol. 64, No. 767

The 3D coordinate of each feature must be identified in each data set. Registration The unregistered MR and CT images are displayed as sequences of 2D slices on two SUN workstations placed side by side. Two workstations are used because a single SUN workstation does not allow a minimum of three slices from each modality to be displayed at full resolution (512x512 pixels per slice for CT). The user then identifies the 3D coordinates of equivalent point-like anatomical features in both modalities using a mouse and cursor. Careful scrutiny of the hard copy films of the images which are also displayed on a light box adjacent to the SUN workstation assists in this. The 3D coordinates of the selected points are used to determine the coordinate transformation relating the MR and CT images. This is achieved using a least squares fitting algorithm (Arun et al, 1987) which we have applied previously to external marker based registration of MR and single photon emission computed tomography (SPECT) images (Hawkes et al, 1990; Hill et al, 1990). Details of this algorithm are given in the appendix. The registration algorithm determines the three translations and three rotations of a rigid body transformation. The scaling (i.e. the voxel dimension for each modality) was determined by separate calibration as part of the routine quality assurance programme of each scanner, but could easily be determined from the location of the corresponding features. The registration algorithm calculates the registration error for each of the point landmarks and the root mean square (RMS) error for all points. The calculated registration transformation was used to transform the MR image data set to the coordinate system of the CT images using trilinear interpolation. The study presented in this paper was on a 57-yearold woman with a suspected right acoustic neuroma, who had conventional MR and CT investigations as detailed in Table I. Six point-like anatomical structures were located in both modalities. These points are listed in Table II. Three independent observers located the six point-like landmarks in the MR image twice to give an indication of the precision of landmark location. Display The registered data set comprises an array of voxels, each of which contains two attributes: one from the CT Table I. Image acquisition details MR images Image dimensions 256 x 256 x 14 Acquisition time 15 min with saddle head coil Voxel size 0.9 x 0.9 x 2.0 mm Post-gadolinium DTPA enhanced Tt -weighted spin-echo: echo time (TE) 30 ms, repetition time (TR) 450 ms CT images Image dimensions 512x512x21 Voxel size 0.48 x 0.48 x 1.5 mm No contrast

1031

D. L. G. Hill et al

image, one from the MR image. Both these modalities provide information primarily on patient anatomy, so the colour overlay typically used to display images of anatomy (MR or CT) registered with images of function (SPECT or PET) (Hawkes et al, 1990) would tend to be confusing. To overcome this we have developed two alternative types of display. The first involves adjacent display of equivalent MR and CT slices selected from the registered image volume, with a linked cursor identifying equivalent regions in the two modalities. The second method involves generating a single grey scale distribution from the registered data volume by fusing the data with a logical operation at each voxel. For example we chose: IF CT Hounsfield Number corresponds to bone DISPLAY CT attribute ELSE DISPLAY MR attribute. These fused images are printed onto film to enable them to be viewed on conventional light boxes. Results

Accuracy Each of the three independent observers were able to relocate the six anatomical features to within 1 pixel in each dimension of the MR image. This corresponds to a distance of 0.9 mm in the slice plane and 2 mm in the perpendicular direction. The RMS registration error for the point-like landmarks is 1.43 mm. The registration accuracy will be better than this since a least square technique is used to

Table II. Point like structures used to register MR and CT images 1) Tip of the alar cartilage (visible in both modalities). 2) The most inferior junction of the anterior wall of the sphenoid sinus with the sphenoid intersinus septum (visible in both modalities). 3) The modiolus of the left cochlea (visible in both modalities). 4) The modiolus of the right cochlea (visible in both modalities). 5) The termination of the naso-lacrimal duct in the maxillary sinus (visible in both modalities). 6) The confluence of the superior sagittal sinus and the transverse sinuses (visible in MRI) and the internal occipital protruberence (visible in CT).

derive the transformation matrix and, as more points are identified than are necessary to solve the transformation, there will be some averaging of errors. Images Figure 1 shows adjacent display of the MR and CT components of a single slice at the level of the internal auditory meatus taken from the registered image volume. Figure 2 shows the fused display of this same slice, with the CT value displayed in pixels where the Hounsfield number corresponds to bone and the MR value is displayed in the remaining pixels. Figure 3 shows this same fused display of a slice higher in the data set.

Figure 1. Adjacent display of a single slice at the level of the internal auditory meatus taken from registered MR and CT volume images. The linked cursor indicates the position in both modalities of the centre of an acoustic neuroma.

1032

The British Journal of Radiology, November 1991

Registration of MR and CT images for skull base surgery

Figure 2. Single fused display of the MR and CT components of the images illustrated in Fig. 1. In the fused display, bone information from the CT image is displayed with soft tissue information from the MR image.

Discussion

We have developed a technique for registering MR and CT images which is entirely retrospective and does not interfere in any way with normal image acquisition. The technique requires a user, who is familiar with MR and CT appearances of the anatomy of the skull base, to identify between six and 12 point-like features. This process takes an experienced user less than half an hour. Once this is done, the calculation of the registration transformation and the interpolation of the image data takes a few minutes, but requires no user interaction. The RMS and maximum error calculated by the registration program give an indication of the accuracy of registration. Only three points are required to determine the rigid body transformation of data from one modality to the other. We used six points and a "least squares" technique to calculate the best estimate of the transformation, which assumes that the errors on the location of each point is independent, random and approximately the same. The best points to choose for accurate calculation of the rigid body transformation would be around the periphery of the field of view. However, because the location error due to distortion in the MR image is highest at the periphery, it is more appropriate to select points evenly distributed around the site of interest. Vol. 64, No. 767

Figure 3. Fused display of a slice 9 mm more craniad than that shown in Fig. 2.

As we have greater a priori confidence in the location of some of the features than others, it would be appropriate to use this a priori confidence to weight the least squares calculation. This is being implemented. We have used a simple thresholding technique on the CT data to define bone in the fused images. Therefore, the partial volume effect will lead to aliasing in the final display, which becomes more severe with increasing CT and/or MR slice thickness. We are investigating the use of colour displays with appropriate look up tables to produce the most effective display. For skull base surgery, registration and fused display clearly provide the surgeon and radiologist with better information on the precise geometric relationship between the soft tissue structures seen on MR and the bony structures seen on CT. Most lesions within the temporal bone are diagnosed by MR, e.g. acoustic neuromas and dermoids and cholesterol cysts. Nevertheless, CT is still required to establish the optimal surgical approach and to delineate the bony anatomy that will be encountered in a transtemporal resection. The fused image can be displayed as a single grey level distribution and, therefore, can be printed onto conventional film using a laser imager or matrix camera. This is convenient, provides a familiar medium for display and enables the registered images to be used in the operating theatre. In this paper, we have described the technique and presented results for a single patient. This technique has 1033

D. L. G. Hill et al

so far been applied to six patients with various skull base pathologies and the results of a full clinical evaluation of the method will be presented in due course.

where V is obtained from V by setting the column corresponding to the most singular value of M to 0. Having calculated R, T can be found from T=p-Rp

Acknowledgments We are grateful to the Leverhulme Trust for funding D. L. G. Hill and D. J. Hawkes. We acknowledge many of our colleagues for their help, cooperation and support in this work. In particular, we thank Ken Allen, Medical Physicist at King's College Hospital, London, Gay Coombes, CT radiographer at the Maudsley Hospital, London, and Beth Nichols, MR radiographer at Guy's Hospital London. We also acknowledge useful discussions with Dr Alan Evans of the Neurolmaging Lab., Montreal Neurological Institute, McGill University, Canada. Appendix. Least squares fitting of two 3 D point sets The rigid body transformation relating two 3D image data sets can be determined by finding the least squares registration transformation relating an arbitrary number of approximately equivalent 3D points identified in the image data sets. To achieve this, we use an algorithm based on singular value decomposition (SVD) derived from Arun et al (1987). The two 3D points sets {pj and {pj} are related by (1)

p\ =

where each of p{ is a 3D point represented as a column vector, R is a 3D rotation matrix, T is a translation vector, and A^ is a noise vector arising as a result of errors in locating equivalent 3D points in the two image data sets. Finding the registration transformation relating {pj and {p|} involves determining values of R and T which minimize

= I \\p[-{RPi+T)\\2

(2)

where n is the number of points in the point sets. As described in Arun et al (1987), R and T can be decoupled by finding the centroids of each point set and subtracting these centroids from each point set to give {gj and {q[}. In this way, the rotation transformation is the value of R which minimizes (3) The matrix R can be calculated by first determining the matrix M given by

(7)

In our implementation, the SVD of M and matrix determinants were calculated using functions from the Numerical Recipes in C library (Press et al, 1988). References ARUN,

K.

S., HUANG,

T.

S. &

BLOSTEIN

S.

D.,

1987.

Least-squares fitting of two 3D point sets. IEEE Transactions of Pattern Analysis and Machine Intelligence, PAMI-9, 698-700. BARTOO, G. T. & HANSON, W. A., 1989. Multimodality image

registration using centroid mapping. Proceedings of IEEE 11th International Conference of the Engineering in Medicine and Biology Society, 550-551. BOESECKE, R., BRUCKNER, T. & ENDE, G., 1990. Landmark

based correlation of medical images. Physics in Medicine and Biology, 35, 121-126. CHEN, G. T. Y., KESSLER, M. & PITLUCK, S., 1985. Structure

transfer between sets of three dimensional medical imaging data. Proceedings of Computer Graphics, 171-177. EVANS, A. C , BEIL, C , MARRETT, S., THOMPSON, C. J. &

HAKIM, A., 1988. Anatomical-functional correlation using an adjustable MRI-based region of interest atlas with positron emission tomography. Journal of Cerebral Blood Flow and Metabolism, 8, 513-529. EVANS, A., MARRET, S., COLLINS, L. & PETERS, T., 1989.

Anatomical-functional correlative analysis of the human brain using three dimensional imaging systems. Proceedings of Society of Photo-optical Instrumentation Engineers, 1092, 264-274. FLEMING, J. S., 1989. A technique for using CT images in attenuation correction and quantification in SPECT. Nuclear Medicine Communications, 10, 83-97. GAMBOA-ALDECO, A., FELLINGHAM, L. L. & CHEN, G. T. Y.,

1986. Correlation of 3D surfaces from multiple modalities in medical imaging. Proceedings of Society of Photo-optical Instrumentation Engineers, 626, 467-473. HAWKES, D. J., HILL, D. L. G., LEHMANN, E. D., ROBINSON, G. P., MAISEY, M. N. & COLCHESTER, A. C. F., 1990.

Preliminary work on the interpretation of SPECT images with the aid of registered MRI images and an MR derived neuro-anatomical atlas. In 3D Imaging in Medicine: Algorithms, Systems, Applications, ed. by K. H. Hoehne, H. Fuchs & S. M. Pizer (Springer-Verlag, Berlin), pp. 241-252. HENRI, C , COLLINS, L., PETERS, T., EVANS, A. & MARRET, S.,

(4) where the superscript t represents matrix transpose. Using SVD, the real matrix M can be decomposed into a column orthogonal matrix U, a diagonal square matrix with nonnegative entries W, and the transpose of a square orthogonal matrix V, such that M equals the product UWV. Provided the determinant of (VU*) is + 1 , then R= VU'

(5)

If the determinant is — 1, this solution is a reflection rather than a rotation. As the physics of the system constrains the transformation to be a rotation, the correct rotation can be found from

= V'U' 1034

(6)

1989. Three-dimensional interactive display of medical images for stereotactic neurosurgery planning. Proceedings of Society of Photo-optical Instrumentation Engineers, 1092, 67-74. HILL, D. L. G., HAWKES, D. J., LEHMANN, E. D., CROSSMAN, J. E., ROBINSON, L., BIRD, C. F. & MAISEY, M. N., 1990.

Registered high resolution images in the interpretation of radionuclide scans. Proceedings of IEEE 12th International Conference of the Engineering in Medicine and Biology Society, 143-144. MELTZER, C. C , BRYAN, R. N., HOLCOMB, H. H., KIMBALL, A. W., MAYBERG, H. S., SADZOT, B., LEAN, J. P., WAGNER, H.

N. & FROST, J. J., 1990. Anatomical localisation for PET using M R imaging. Journal of Computer Assisted Tomography, 14, 418-426. PELIZZARI,

C.

A.,

CHEN,

G.

T.

Y.,

SPELBRING,

D.

R.,

WEICHSELBAUM, R. R. & CHEN, C.-T., 1989. Accurate three The British Journal of Radiology, November 1991

Registration of MR and CT images for skull base surgery dimensional registration of CT, PET and/or MR images of the brain. Journal of Computer Assisted Tomography, 13, 20-26. PETERS, T., CLARK, J. A., OLIVIER, A., MARCHAND, E. P., MAWKO, G., DIEUMEGARDE, M., MURESAN, L. V. & ETHIER,

Proceedings of IEEE 12th International Conference of the Engineering in Medicine and Biology Society, 145-146. SCHAD, L. R., BOESECKE, R., SCHLEGEL, W . , HARTMANN, G . H . , STURM, V., STRAUSS, L. G. & LORENZ, W. J., 1987. Three

dimensional image correlation of CT, MR and PET studies in radiotherapy planning of brain tumours. Journal of R., 1986. Integrated stereotaxic imaging with CT, MR Computer Assisted Tomography, 11, 948-954. imaging and digital subtraction angiography. Radiology, WILSON, M. W. & MOUNTZ, J. M., 1989. A reference system for 161, 821-826. neuroanatomical localization on functional reconstructed PRESS, W. H., FLANNERY, B. P., TEUKOLSKY, S. A. & cerebral images. Journal of Computer Assisted Tomography, VETTERLING, W. T., 1988. Numerical Recipes in C. 13, 174-178. (Cambridge University Press, Cambridge, UK). ZHANG, J., LEVESQUE, M. F., WILSON, C. L., HARPER, R. M., PRICE, D., 1990. Magnetic Resonance Imaging Brain ENGEL, J., LUFKIN, R. & BEHNKE, E. J., 1990. Multimodality Morphometry: An Investigation into its Accuracy. MSc Thesis imaging of brain structures for stereotactic surgery. (University of Surrey, Guildford, Surrey). Radiology, 175, 435-441. Rizzo, G., GILARDI, M. C , BETTINARDI, V., CERUTTI, S. & FAZIO, F., 1990. Integration of multimodal medical images.

Vol. 64, No. 767

1035

Registration of MR and CT images for skull base surgery using point-like anatomical features.

We have developed a registration technique for combining magnetic resonance imaging (MRI) and computed tomography (CT) images of the skull base for us...
1MB Sizes 0 Downloads 0 Views