Segmentation of Magnetic Resonance Images Using an Artificial Neural Network D.W. Piraino, M.D.', S.C. Amartur M.S.2, B.J. Richmond, M.D.!, J.P. Schils, M.D.', J.M. Thome, D.O.1, and P.B. Weber, M.D.1, 'Department of Radiology, Cleveland Clinic Foundation and 'Department of Radiology, Case Western Reserve University network is made of an input layer, a Kohonen layer, and a competition layer (Figure 1).
ABSTRACT Signal intensities from intermediate and 72 weighted spin echo images of the brain were used as inputs into an artificial neural network (ANN). The signal intensities were used to train the network to recognize anatomically-impoitant segments. The ANN was a self-organizing map (SOM) neural network which develops a continuous topographical map of the signal intensities within the two images. The neural network segmented images demonstrated good correlation with white matter, gray matter, and cerebral spinal fluid (CSF) spaces. This technique was rated better than manual thresholding of the intermediate images, but not as good as manual thresholding of the 72 weighted images.
NPUT LAYER
KOHONEN LAYER 6 x 6 grid
*TITION LAYER
Figure 1 SOM Network [4]
INTRODUCTION Accurate segmentation of soft tissue structures is required prior to volume calculation and 3-D
surface display [1]. Segmentation by hand outlining is tedious and time consuming. Simple thresholding methods often are not adequate to distinguish soft tissue structures [1]. Automated methods to segment soft tissue structures would be useful prior to 3-D surface display and in calculating volumes of normal and abnormal structures [2].
The input layer distributes the inputs to all nodes of the Kohonen layer. Each Kohonen node calculates the distance between its weight vector and the input vector. The distance is adjusted by a conscience mechanism and output to the competition layer. The competition layer determines the winning node in the Kohonen layer. The winning Kohonen node and its neighbors adjust their weight vectors to be closer to the input vector as determined by a cooling factor [4]. If the input vectors are chosen randomly, the Kohonen node weights will expand to cover the probability distribution of the input values. For this study, the Kohonen layer had 25 nodes.
ANNs have been trained to classify structures within many different types of images [3]. Application of ANNs to the process of segmentation in medical imaging has been limited. ANNs are information processing structures which are trained to perform different information processing tasks. It is the hypothesis of this paper that ANNs can be trained to segment magnetic resonance (MR) images into anatomically-important segments with minimal operator intervention. This paper presents the results from a feasibility study in using an ANN to segment MR images of the head.
Signal intensities from intermediate and T2 weighted images were used as inputs into the SOM neural network. Random points in the images were used to train the network. A segmented image was then reconstructed by processing the signal intensities through the trained SOM neural network, and assigning a classification to the points in the image on the basis of the winning node in the network. Image values were assigned to each classification according a threshold technique in order to produce the final image.
MATERIALS AND METHODS A SOM is an ANN which develops a continuous topographical map of its inputs [3]. It is trained in an unsupervised manner and therefore does not need human intervention to be trained. The 0195-4210/91/$5.00 C 1992 AMIA, Inc.
470
Figure 3a Intermediate threshold segmentation
Figure 2a Original intermediate image
Figure 3b T2 weighted density threshold segmentation
Figure 2b Original T2 weighted image
Three sets of segmented images were made. The first set used intermediate and T2 signal intensities (Intermediate & T2 & ANN) only to train the network and to produce the segmented image. The other two sets of images were segmented interactively using thresholding of the original intermediate (Intermediate & Threshold) and T2 weighted (T2 & Threshold) images for comparison. One example showing the original images, the threshold images, and the ANN segmented images are demonstrated in Figures 2 and 3.
Figure 3c SOM network segmentation The segmented images were evaluated by three radiologists for how well the images differentiated white matter, gray matter, and cerebral spinal fluid (CSF) spaces. Each radiologist was asked how well the segmented image differentiated white matter, gray matter, and CSF spaces on a scale from 1-5 with 1 representing poor segmentation, and 5 representing excellent image segmentation. The radiologists had no knowledge of which images were segmented by which technique.
In this feasibility study 8 different image sets of the brain were used. The ANN was constructed using ExploreNet 3000 (HNC San Diego, CA) and the ANN was trained independently for each image set. Images were displayed on a SUN 370 for evaluation.
RESULTS The average ratings over the three observers and the 8 different image sets for each type of segmented image are shown in Table 1:
471
Image Type
White Matter
Grey Matter
CSF
Average
Intermediate & Threshold T2 & Threshold Intermediate & T2 & ANN
2.50 3.59 3.06
2.42 2.62 2.83
1.67 3.67 3.67
2.20 3.29 3.05
Table 1 The average rating over the 3 observers and the 8 image sets for each type of segmented image
Since ANNs are able to process multiple inputs, the type of inputs that provide the best image segmentation also needs to be explored. For example, differences between image sets, averages from small regions, and standard deviations from small regions are several different inputs into an ANN that could be used to improve the segmentation process.
The images produced with interactive thresholding of T2 weighted images were rated slightly higher than those produced by the ANN. These threshold images required interactive thresholding techniques with a radiologist to produce the segmented images. The segmented images produced by the ANN using intermediate images and T2 weighted images were rated higher than the threshold segmented intermediate images.
Accurate automated segmentation of medical images is needed for three dimensional image display and for quantitative analysis of medical images. Further research in ANN segmentation of medical images is needed to determine their usefulness in this area.
DISCUSSION
This feasibility study demonstrated that an ANN can be trained to segment MR images of the brain with minimal human intervention. The images segmented by the ANN were considered of similar accuracy in differentiating gray matter, white matter, and CSF as compared to segmented images produced by interactive thresholding of the original T2 images. The ANN was considered better than the threshold segmentation of the intermediate image in all tissue categories. It should be noted that white matter and CSF demonstrate very similar signal intensities on intermediate images and therefore thresholding of this type of image would be expected to have poor differentiation of CSF and white matter.
1.
This preliminary study demonstrated that an ANN can be used in automated segmentation of images. Further study into the types of ANNs that best perform segmentation of medical images is needed to improve their performance in this area. Back propagation and counter propagation neural networks could also be used to segment medical images. These networks use supervised training methods and therefore would require designation of a training set prior to segmentation. Also the specific implementation of a specific ANN can change its performance and more research in this area is also needed.
472
REFERENCES Fan RT, Trivedi SS, Fellingham LL, et al. Soft Tissue Segmentation and 3-D Display From Computerized Tomography and Magnetic Resonance Imaging, SPIE - Med Imaging, 767, 494, 1987.
2.
Cline HE, Lorensen WE, Kikinis R, Jolesz R. Three-Dimensional Segmentation of MR Images of the Head Using Probability and Connectivity. J Comput Assist Tomogr, 14(6),1037. 1990.
3.
Hecht-Nielsen R. Neurocomputing. New York: Addison-Wesley Publishing, 1990.
4.
HNC Inc. ExploreNet 3000 Manual. San Diego: HNC Inc., 1990.
ABSTRACT Signal intensities from intermediate and 72 weighted spin echo images of the brain were used as inputs into an artificial neural network (ANN). The signal intensities were used to train the network to recognize anatomically-impoitant segments. The ANN was a self-organizing map (SOM) neural network which develops a continuous topographical map of the signal intensities within the two images. The neural network segmented images demonstrated good correlation with white matter, gray matter, and cerebral spinal fluid (CSF) spaces. This technique was rated better than manual thresholding of the intermediate images, but not as good as manual thresholding of the 72 weighted images.
NPUT LAYER
KOHONEN LAYER 6 x 6 grid
*TITION LAYER
Figure 1 SOM Network [4]
INTRODUCTION Accurate segmentation of soft tissue structures is required prior to volume calculation and 3-D
surface display [1]. Segmentation by hand outlining is tedious and time consuming. Simple thresholding methods often are not adequate to distinguish soft tissue structures [1]. Automated methods to segment soft tissue structures would be useful prior to 3-D surface display and in calculating volumes of normal and abnormal structures [2].
The input layer distributes the inputs to all nodes of the Kohonen layer. Each Kohonen node calculates the distance between its weight vector and the input vector. The distance is adjusted by a conscience mechanism and output to the competition layer. The competition layer determines the winning node in the Kohonen layer. The winning Kohonen node and its neighbors adjust their weight vectors to be closer to the input vector as determined by a cooling factor [4]. If the input vectors are chosen randomly, the Kohonen node weights will expand to cover the probability distribution of the input values. For this study, the Kohonen layer had 25 nodes.
ANNs have been trained to classify structures within many different types of images [3]. Application of ANNs to the process of segmentation in medical imaging has been limited. ANNs are information processing structures which are trained to perform different information processing tasks. It is the hypothesis of this paper that ANNs can be trained to segment magnetic resonance (MR) images into anatomically-important segments with minimal operator intervention. This paper presents the results from a feasibility study in using an ANN to segment MR images of the head.
Signal intensities from intermediate and T2 weighted images were used as inputs into the SOM neural network. Random points in the images were used to train the network. A segmented image was then reconstructed by processing the signal intensities through the trained SOM neural network, and assigning a classification to the points in the image on the basis of the winning node in the network. Image values were assigned to each classification according a threshold technique in order to produce the final image.
MATERIALS AND METHODS A SOM is an ANN which develops a continuous topographical map of its inputs [3]. It is trained in an unsupervised manner and therefore does not need human intervention to be trained. The 0195-4210/91/$5.00 C 1992 AMIA, Inc.
470
Figure 3a Intermediate threshold segmentation
Figure 2a Original intermediate image
Figure 3b T2 weighted density threshold segmentation
Figure 2b Original T2 weighted image
Three sets of segmented images were made. The first set used intermediate and T2 signal intensities (Intermediate & T2 & ANN) only to train the network and to produce the segmented image. The other two sets of images were segmented interactively using thresholding of the original intermediate (Intermediate & Threshold) and T2 weighted (T2 & Threshold) images for comparison. One example showing the original images, the threshold images, and the ANN segmented images are demonstrated in Figures 2 and 3.
Figure 3c SOM network segmentation The segmented images were evaluated by three radiologists for how well the images differentiated white matter, gray matter, and cerebral spinal fluid (CSF) spaces. Each radiologist was asked how well the segmented image differentiated white matter, gray matter, and CSF spaces on a scale from 1-5 with 1 representing poor segmentation, and 5 representing excellent image segmentation. The radiologists had no knowledge of which images were segmented by which technique.
In this feasibility study 8 different image sets of the brain were used. The ANN was constructed using ExploreNet 3000 (HNC San Diego, CA) and the ANN was trained independently for each image set. Images were displayed on a SUN 370 for evaluation.
RESULTS The average ratings over the three observers and the 8 different image sets for each type of segmented image are shown in Table 1:
471
Image Type
White Matter
Grey Matter
CSF
Average
Intermediate & Threshold T2 & Threshold Intermediate & T2 & ANN
2.50 3.59 3.06
2.42 2.62 2.83
1.67 3.67 3.67
2.20 3.29 3.05
Table 1 The average rating over the 3 observers and the 8 image sets for each type of segmented image
Since ANNs are able to process multiple inputs, the type of inputs that provide the best image segmentation also needs to be explored. For example, differences between image sets, averages from small regions, and standard deviations from small regions are several different inputs into an ANN that could be used to improve the segmentation process.
The images produced with interactive thresholding of T2 weighted images were rated slightly higher than those produced by the ANN. These threshold images required interactive thresholding techniques with a radiologist to produce the segmented images. The segmented images produced by the ANN using intermediate images and T2 weighted images were rated higher than the threshold segmented intermediate images.
Accurate automated segmentation of medical images is needed for three dimensional image display and for quantitative analysis of medical images. Further research in ANN segmentation of medical images is needed to determine their usefulness in this area.
DISCUSSION
This feasibility study demonstrated that an ANN can be trained to segment MR images of the brain with minimal human intervention. The images segmented by the ANN were considered of similar accuracy in differentiating gray matter, white matter, and CSF as compared to segmented images produced by interactive thresholding of the original T2 images. The ANN was considered better than the threshold segmentation of the intermediate image in all tissue categories. It should be noted that white matter and CSF demonstrate very similar signal intensities on intermediate images and therefore thresholding of this type of image would be expected to have poor differentiation of CSF and white matter.
1.
This preliminary study demonstrated that an ANN can be used in automated segmentation of images. Further study into the types of ANNs that best perform segmentation of medical images is needed to improve their performance in this area. Back propagation and counter propagation neural networks could also be used to segment medical images. These networks use supervised training methods and therefore would require designation of a training set prior to segmentation. Also the specific implementation of a specific ANN can change its performance and more research in this area is also needed.
472
REFERENCES Fan RT, Trivedi SS, Fellingham LL, et al. Soft Tissue Segmentation and 3-D Display From Computerized Tomography and Magnetic Resonance Imaging, SPIE - Med Imaging, 767, 494, 1987.
2.
Cline HE, Lorensen WE, Kikinis R, Jolesz R. Three-Dimensional Segmentation of MR Images of the Head Using Probability and Connectivity. J Comput Assist Tomogr, 14(6),1037. 1990.
3.
Hecht-Nielsen R. Neurocomputing. New York: Addison-Wesley Publishing, 1990.
4.
HNC Inc. ExploreNet 3000 Manual. San Diego: HNC Inc., 1990.
