Bio-Medical Materials and Engineering 24 (2014) 1125–1131 DOI 10.3233/BME-130912 IOS Press
1125
Cytoplasm segmentation on cervical cell images using graph cut-based approach Ling Zhanga, b, c, Hui Kongd, Chien Ting China, b, c , Tianfu Wanga, b, c* and Siping Chena,b,c* a
Department of Biomedical Engineering, Shenzhen University, Shenzhen 518060, China National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Shenzhen 518060, China c Guangdong Key Laboratory of Biomedical Information Detection and Ultrasound Imaging, Shenzhen 518060, China d Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139 USA b
Abstract. This paper proposes a method to segment the cytoplasm in cervical cell images using graph cut-based algorithm. First, the A* channel in CIE LAB color space is extracted for contrast enhancement. Then, in order to effectively extract cytoplasm boundaries when image histograms present non-bimodal distribution, Otsu multiple thresholding is performed on the contrast enhanced image to generate initial segments, based on which the segments are refined by the multi-way graph cut method. We use 21 cervical cell images with non-ideal imaging condition to evaluate cytoplasm segmentation performance. The proposed method achieved a 93% accuracy which outperformed state-of-the-art works. Keywords: Cervical cell, cytoplasm segmentation, A* channel, graph cut
1. Introduction Screening by cytology is the commonest approach to prevent cervical cancer at a pre-cancerous stage. However, it is known that screening of cervical cytology slides is very labor intensive and demands that the cytotechnologist be capable of high levels of concentration for extended periods. Automation-assisted reading (AAR) techniques have the potential to increase productivity and reduce screening errors. However, a large, prospective randomized trial found that the performance of AAR system was not good enough for cervical cancer screening [1]. It was demonstrated that certain biologically relevant morphological changes can be captured only in case when cells are well segmented [2]. Hence, cervical cell segmentation has attracted more and more research interests recently. Generally, a cell image segmentation algorithm should capture both the cytoplasm and nuclei. Many of the previous literatures focused on the nucleus segmentation. Actually, as demonstrated in [3], the most informative feature for discrimination between health and ab-
*
Corresponding author. E-mail: [email protected]. Corresponding author. E-mail: [email protected].
*
0959-2989/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
1126
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
normal cervical cells is nucleus/cytoplasm (N/C) ratio. Therefore, accurate segmentation of cytoplasm is very important. In recent year, the majority of cytoplasm segmentation used one or integration of the following techniques: K-means [4], [5], edge detection [6], thresholding [7], [8] and active contours [5], [9]. Most of these works are designed for images of isolated cells, especially for those in the Herlev data set [10]. For segmentation in images containing multiple cells, thresholding [7] [8] [11] and level set [9] techniques have been used. However, thresholding may lead to inaccurate results in images where inconsistent staining and/or illumination and overlapping cells are present, and the level set is computation-intensive and tends to locate some local extrema. The graph cut (GC) approach [12] which is highly attractive in nucleus segmentation recently, has not found its use in cervical cytoplasm segmentation. In this paper, using the multi-way GC on the A* channel enhanced images to obtain effective cytoplasm segmentation is proposed when image histograms present non-bimodal distribution. 2. Methods Given a cervical cell image, a multi-way segmentation is applied to separate the cytoplasm from the background. Specifically, the A* channel in CIE LAB color space is used for preprocessing. Then, initial segments are generated automatically by using Otsu’s multiple thresholding algorithm [13] on the preprocessed image, based on which the segments are refined by the multi-way GC method. In the rest of this section, we introduce the details of cytoplasm segmentation. 2.1. Preprocessing In actual cervical cell images, the poor contrast, non-uniform staining and noise will likely hinder cell segmentation. To enhance the contrast, we extract the A* channel in the CIE LAB color space. The reason of selecting the A* channel is as follows: In hematoxylin and eosin (H&E) staining, cell regions are colored with tones of red and background regions remain colorless. This inspires us to use color to discriminate cells from background. The A* channel represents change between red and green and is able to embody this difference. Hence, in the A* channel image, cells are obviously brighter than the background. To further enhance contrast, the A* channel image is stretched linearly from their original range [Imin, Imax] to the range [0, 255]. A common and effective technique in handling noise is the median filter. It was demonstrated by Tsai et al. [4] that median filter can eliminate both impulse and Gaussian noise in cervical smear images. In our work, a 5×5 median filter is applied to the contrast enhanced images to discard noise. Fig. 1(b) shows the result of the above preprocessing. 2.2. Multi-Way Segmentation Although the difference between cell and background is significantly enhanced by preprocessing, not all the image histograms present bimodal distribution due to the complexity of our images which include inhomogeneous illumination, non-uniform staining and the presence of inflammation cells and debris. Therefore, a single threshold cannot successfully separate the cervical cell from the background. For example, some cytoplasm with brighter intensity tends to be classified into the background class. With this in mind, we propose to use the multi-way GC approach. The output image contains four classes. The class which has the highest mean intensity is the background. The other three
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1127
classes which contain cytoplasm, nuclei, inflammation cells and debris et al. are integrated as the foreground (cell region). Our approach is summarized below:
Fig. 1. (a) An original color image. (b) Contrast enhancement and noise removal. (c) 3D Otsu. (d) Multi-way GC. (e) Foreground merging and morphological opening. (f) The obtained cytoplasm boundary.
Given an image with intensity values from Imin to Imax, we compute three optimal thresholds th1*, th2* and th3* with Imin th1* th2* th3* Imax, by applying 3D Otsu algorithm [13]. Then, the mean intensity values, c1, c2, c3 and c4, of the four classes (C1, C2, C3 and C4) are computed, where C1 = [Imin, … , th1*], C2 = [th1* + 1, … , th2*], C3 = [th2* + 1, … , th3*], and C4 = [th3* + 1, … , Imax]. With c1, c2, c3 and c4, we construct a four-terminal (labels) graph. This graph contains two types of vertices: pixels and labels. Each pixel has an n-links to its neighbors, and is also connected to all labels by t-links. According to this graph, the Potts model energy function Ep(f) is constructed as equation (1). Boykov et al. [12] had demonstrated that the minimization of Ep(f) can resolve the multi-way cut problem,
EP ( f ) = ¦ D p ( f p ) + p∈P
¦ω
{ p ,q }∈N
{ p ,q }
⋅T ( f p ≠ fq ) (1)
where f denotes the pixel label, p indexes pixel, N is the set of adjacent pixels, {p, q} represents a pair of pixels, and the function T(.) is a constant (set as 0.1 in our work through empirical evaluation) when the condition inside the parentheses is true and 0 otherwise. The first term of Eq. (1) is the data term, which is determined by the connection energy t-links between each pixel and each terminal of the graph. In our work, Dp(fp) is assigned as (cp - Ip)2, where Ip is the intensity value of pixel p. The second term in Eq. (1) is the pixel continuity term, which is determined by the connection energy n-links between neighborhood pixels in the multiple-terminal graph, as defined by the following equation [12],
°2 K if | I p − I q |≤ 5 °¯K if | I p − I q |> 5
ω{ p,q} = ω ( I p − I q ) = ®
where K is the Potts model parameter, Ip and Iq are two neighboring pixel values.
(2)
1128
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
Finally, by using an implementation of the -expansion and fast max-flow/min-cut algorithms introduced in [12], the energy function of Eq. (1) is optimized approximately, and the pixel category label f is obtained. Consequently, we can re-divide each pixel to its new class. Finally, to split some cytoplasm with slight overlapping, the segmented binary image (after merging the three foreground subclasses) is processed by morphological opening with a 5×5 square structuring element. Actually, there exists some other method to replace the 3D Otsu thresolding, such as K-means clustering. We choose Otsu thresholding because it is more efficient and shows better segmentation results in our evaluation (please refer to the result section for more details). In addition, Otsu’s method is one of the best thresholding methods for general images [14]. Fig. 1(c)-(e) show the examples of cytoplasm segmentation. By examining the center part of Fig. 1(f), it can be seen that the segmentation is not affected by the bright illumination and dirt. However, we do not attempt to split the overlapping cytoplasm in this work. This is because that reliable delineation of the cytoplasm boundary for each cell is unrealistic even for human expert in the presence of heavily overlapping cells. 3. Experiments 3.1. Clinical Data Collection All images used in this study were acquired using an Olympus BX41 microscope equipped with 20X objective (Olympus America, Inc., Central Valley, PA), Jenoptik ProgRes CF Color 1.4 Megapixel Camera (Jenoptik Optical Systems Inc., Jena, Germany), and MS300 motorized stage (NJRGB Inc., Nanjing, China). Image specifications were 24 bit RGB channels with resolution of 1360×1024 pixels. The data set included 51 cervical cell images from 21 cervical slides, which were collected from the Department of Pathology, People's Hospital of Nanshan District, Shenzhen, China, in 2010. All slides are prepared using manual liquid-based cytology (MLBC) technique and are stained with H&E. Our training and test set consisted of 30 and 21 images, respectively. The ground truths used for evaluations of cytoplasm segmentation were obtained by manual delineation by a pathologist. 3.2. Quantitative Assessment Methodologies The evaluation of cytoplasm segmentation was based on comparison with two other cytoplasm segmentation methods [8], [11] using mutual overlap metric [15]: Acc = 2|RGT RSeg| / (|RGT| + |RSeg|), where RGT denotes the ground truth region, RSeg is the segmented region, and |.| is the number of pixels in a certain region. 3.3. Results Our methods were implemented using C++. We run our C++ release software on a 64-bit Windows PC, which has a 2.66GHz quad-core CPU and a 4GB RAM. In our method, there is only one major parameter (cluster number in the multi-way segmentation) that we tuned on training images. We set it to 3, 4, 5, and 6, respectively. Correspondingly, the cytoplasm segmentation accuracy Acc was 0.89, 0.93, 0.93, and 0.91, respectively. Considering the tradeoff between accuracy and computational complexity, we chose 4 as the cluster number. Figure 2 gives three examples of our cytoplasm segmentation approach on cervical cell images, demonstrating that our method produced promising segmentations of cytoplasm in the presence of
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1129
overlap cells (Fig. 2(a)), poor contrast and non-uniform staining (Fig. 2(b)). We also can see that our algorithm can successfully extract cytoplasm boundaries when confronted with a large amount of inflammatory cells. Compared to the first two examples, Fig. 2(c) shows a relatively poor segmentation in which a very bright red speck exists in the upper part, obscuring boundaries and decreasing contrast of cells. Actually, the performance of a majority of our H&E stained images resembles the first two examples in Fig. 2.
Fig. 2. Examples of the proposed cytoplasm segmentation method applied to three cervical cell images. (a) Heavily overlap cells. (b) Poor contrast and non-uniform staining. (c) Bright red speck.
Fig. 3. Comparison of cytoplasm segmentation results with (a) manual delineation, (b) Gençtav et al.’s method [8], (c) Hu et al.’s method [11] and (d) our method.
Figure 3 shows two example segmentation results by the proposed method and comparison with the manual delineation, [8], and [11], where the first to the third columns are results by manual delineation, [8] and [11], respectively, and the fourth column corresponds to our propose method. The results of our method look much better than those two compared methods in terms of segmentation accuracy. Figure 4 shows the mean and standard deviation of the cytoplasm segmentation accuracy achieved by the compared algorithms along with Otsu thresholding and K-means. The average segmentation accuracies with respect to the mean and standard deviation of all the algorithms are listed in Table 1. In Gençtav et al.’s [8] method, the only parameter is the radius of the disk of the black top-hat algo-
1130
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
rithm. We found that this method had difficulties in obtaining consistently satisfactory results with the same set of this parameter for different images. So we used a radius of 210 pixels as set in Ref. [8]. The results obtained by Otsu and K-means are better than Ref. [8] and [11]. This is because our proposed preprocessing is applied on the test image before applying Otsu and K-means. Note that Otsu performs better than K-means especially in terms of the standard deviation. Furthermore, we can see that 3% improvement (from 90% to 93%) is achieved after applying GC refinement on the Otsu segmentation. Our cytoplasm segmentation method takes 0.3 seconds per image on average.
Fig. 4. Comparison of cytoplasm segmentation performance with [8], [11], Otsu, and K-means. Table 1 Comparison of cytoplasm segmentation performance with respect to the mean and standard deviation of the segmentation accuracy Mean Std.
ours 0.93 0.03
[8] 0.64 0.26
[11] 0.76 0.18
Otsu 0.90 0.04
K-means 0.83 0.17
4. Conclusion This paper focus on the automated segmentation of cervical cytoplasm from cytology images with non-ideal imaging conditions, as existing techniques often designed for nuclei. The proposed GCbased segmentation method along with our preprocessing approach allow for cytoplasm delineation when image histograms present non-bimodal distribution. However, current segmentation method cannot delineate the cytoplasm boundary for each cell, and may fail to delineate the cytoplasm bound-
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1131
ary in red regions caused by intensive illumination. In our future work, we plan to use the proposed method combined with our proposed nucleus segmentation method [16] for finding of abnormal cells from cervical cytology slides based on classification of segmentation results. 5. Acknowledgement We would like to thank Dr. Shaoxiong Liu from the Pathology Department at the People's Hospital of Nanshan District for providing the ground truth delineation. References [1]
[2] [3]
[4] [5] [6] [7]
[8] [9] [10] [11] [12] [13] [14] [15] [16]
H. C. Kitchener, R. Blanks, G. Dunn, L. Gunn, M. Desai, R. Albrow, J. Mather, D. N. Rana, H. Cubie, C. Moore, R. Legood, A. Gray and S. Moss, Automation-assisted versus manual reading of cervical cytology (MAVARIC): a randomised controlled trial, The Lancet Oncology 12 (2011), 56–64. J. Rittscher, Characterization of biological process through automated image analysis, Annual Review of Biomedical Engineering 12 (2010), 315–344. Y. Marinakis, G. Dounias and J. Jantzen, Pap smear diagnosis using a hybrid intelligent scheme focusing on genetic algorithm based feature selection and nearest neighbour classification, Computers in Biology and Medicine 39 (2009), 69–78. M. H. Tsai, Y. K. Chan, Z. Z. Lin, S. F. Yang-Mao and P. C. Huang, Nucleus and cytoplast contour detector of cervical smear image, Pattern Recognition Letters 29 (2008), 1441–1453. K. Li, Z. Lu, W. Liu and J. Yin, Cytoplasm and nucleus segmentation in cervical smear images using Radiating GVF Snake, Pattern Recognition 45 (2012), 1255–1264. S. F. Yang-Mao, Y. K. Chan and Y. P. Chu, Edge enhancement nucleus and cytoplast contour detector of cervical smear images, IEEE Transactions on Systems Man and Cybernetics-Part B: Cybernetics 38 (2008), 353–366. C. Bergmei, M. G. Silvente, J. E. Lopez-Cuervo and J. M. Benitez, Segmentation of cervical cell nuclei in highresolution microscopic images: A new algorithm and a web-based software framework, Computer Methods and Programs in Biomedicine 107 (2012), 497–512. A. Gençtav, S. Aksoy and S. Önder, Unsupervised segmentation and classification of cervical cell images, Pattern Recognition 45 (2012), 4151–4168. N. M. Harandi, S. Sadri, N. A. Moghaddam and R. Amirfattahi, An automated method for segmentation of epithelial cervical cells in images of ThinPrep, Journal of Medical Systems 34 (2010), 1043–1058. J. Jantzen, J. Norup, G. Dounias and B. Bjerregaard, Pap-smear benchmark data for pattern classification, in Proceedings of the Nature-Inspired Smart Information Systems 1nd Annual Symposium (2005). M. Hu, X. Ping, and Y. Ding, Automated cell nucleus segmentation using improved snake, in International Conference on Image Processing (2004), 2737–2740. Y. Boykov, O. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2001), 1222–1239. N. A. Otsu, Threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics 9 (1979), 62–66. P. K. Sahoo, S. Soltani, A. K. C. Wong and Y. Chen, A survey of thresholding techniques, Computer Vision Graphics Image Processing 41 (1988), 233–260. M. Sonka and J. M. Fitzpatrick. Handbook of medical imaging, SPIE Press, Washington, 2000. L. Zhang, J. Li, S. Chen, T. Wang, S. Jiang and S. Liu, Segmentation of abnormal cervical nuclei using an adaptive and local approach, Journal of Image and Graphics, accepted.
Copyright of Bio-Medical Materials & Engineering is the property of IOS Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
1125
Cytoplasm segmentation on cervical cell images using graph cut-based approach Ling Zhanga, b, c, Hui Kongd, Chien Ting China, b, c , Tianfu Wanga, b, c* and Siping Chena,b,c* a
Department of Biomedical Engineering, Shenzhen University, Shenzhen 518060, China National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Shenzhen 518060, China c Guangdong Key Laboratory of Biomedical Information Detection and Ultrasound Imaging, Shenzhen 518060, China d Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139 USA b
Abstract. This paper proposes a method to segment the cytoplasm in cervical cell images using graph cut-based algorithm. First, the A* channel in CIE LAB color space is extracted for contrast enhancement. Then, in order to effectively extract cytoplasm boundaries when image histograms present non-bimodal distribution, Otsu multiple thresholding is performed on the contrast enhanced image to generate initial segments, based on which the segments are refined by the multi-way graph cut method. We use 21 cervical cell images with non-ideal imaging condition to evaluate cytoplasm segmentation performance. The proposed method achieved a 93% accuracy which outperformed state-of-the-art works. Keywords: Cervical cell, cytoplasm segmentation, A* channel, graph cut
1. Introduction Screening by cytology is the commonest approach to prevent cervical cancer at a pre-cancerous stage. However, it is known that screening of cervical cytology slides is very labor intensive and demands that the cytotechnologist be capable of high levels of concentration for extended periods. Automation-assisted reading (AAR) techniques have the potential to increase productivity and reduce screening errors. However, a large, prospective randomized trial found that the performance of AAR system was not good enough for cervical cancer screening [1]. It was demonstrated that certain biologically relevant morphological changes can be captured only in case when cells are well segmented [2]. Hence, cervical cell segmentation has attracted more and more research interests recently. Generally, a cell image segmentation algorithm should capture both the cytoplasm and nuclei. Many of the previous literatures focused on the nucleus segmentation. Actually, as demonstrated in [3], the most informative feature for discrimination between health and ab-
*
Corresponding author. E-mail: [email protected]. Corresponding author. E-mail: [email protected].
*
0959-2989/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
1126
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
normal cervical cells is nucleus/cytoplasm (N/C) ratio. Therefore, accurate segmentation of cytoplasm is very important. In recent year, the majority of cytoplasm segmentation used one or integration of the following techniques: K-means [4], [5], edge detection [6], thresholding [7], [8] and active contours [5], [9]. Most of these works are designed for images of isolated cells, especially for those in the Herlev data set [10]. For segmentation in images containing multiple cells, thresholding [7] [8] [11] and level set [9] techniques have been used. However, thresholding may lead to inaccurate results in images where inconsistent staining and/or illumination and overlapping cells are present, and the level set is computation-intensive and tends to locate some local extrema. The graph cut (GC) approach [12] which is highly attractive in nucleus segmentation recently, has not found its use in cervical cytoplasm segmentation. In this paper, using the multi-way GC on the A* channel enhanced images to obtain effective cytoplasm segmentation is proposed when image histograms present non-bimodal distribution. 2. Methods Given a cervical cell image, a multi-way segmentation is applied to separate the cytoplasm from the background. Specifically, the A* channel in CIE LAB color space is used for preprocessing. Then, initial segments are generated automatically by using Otsu’s multiple thresholding algorithm [13] on the preprocessed image, based on which the segments are refined by the multi-way GC method. In the rest of this section, we introduce the details of cytoplasm segmentation. 2.1. Preprocessing In actual cervical cell images, the poor contrast, non-uniform staining and noise will likely hinder cell segmentation. To enhance the contrast, we extract the A* channel in the CIE LAB color space. The reason of selecting the A* channel is as follows: In hematoxylin and eosin (H&E) staining, cell regions are colored with tones of red and background regions remain colorless. This inspires us to use color to discriminate cells from background. The A* channel represents change between red and green and is able to embody this difference. Hence, in the A* channel image, cells are obviously brighter than the background. To further enhance contrast, the A* channel image is stretched linearly from their original range [Imin, Imax] to the range [0, 255]. A common and effective technique in handling noise is the median filter. It was demonstrated by Tsai et al. [4] that median filter can eliminate both impulse and Gaussian noise in cervical smear images. In our work, a 5×5 median filter is applied to the contrast enhanced images to discard noise. Fig. 1(b) shows the result of the above preprocessing. 2.2. Multi-Way Segmentation Although the difference between cell and background is significantly enhanced by preprocessing, not all the image histograms present bimodal distribution due to the complexity of our images which include inhomogeneous illumination, non-uniform staining and the presence of inflammation cells and debris. Therefore, a single threshold cannot successfully separate the cervical cell from the background. For example, some cytoplasm with brighter intensity tends to be classified into the background class. With this in mind, we propose to use the multi-way GC approach. The output image contains four classes. The class which has the highest mean intensity is the background. The other three
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1127
classes which contain cytoplasm, nuclei, inflammation cells and debris et al. are integrated as the foreground (cell region). Our approach is summarized below:
Fig. 1. (a) An original color image. (b) Contrast enhancement and noise removal. (c) 3D Otsu. (d) Multi-way GC. (e) Foreground merging and morphological opening. (f) The obtained cytoplasm boundary.
Given an image with intensity values from Imin to Imax, we compute three optimal thresholds th1*, th2* and th3* with Imin th1* th2* th3* Imax, by applying 3D Otsu algorithm [13]. Then, the mean intensity values, c1, c2, c3 and c4, of the four classes (C1, C2, C3 and C4) are computed, where C1 = [Imin, … , th1*], C2 = [th1* + 1, … , th2*], C3 = [th2* + 1, … , th3*], and C4 = [th3* + 1, … , Imax]. With c1, c2, c3 and c4, we construct a four-terminal (labels) graph. This graph contains two types of vertices: pixels and labels. Each pixel has an n-links to its neighbors, and is also connected to all labels by t-links. According to this graph, the Potts model energy function Ep(f) is constructed as equation (1). Boykov et al. [12] had demonstrated that the minimization of Ep(f) can resolve the multi-way cut problem,
EP ( f ) = ¦ D p ( f p ) + p∈P
¦ω
{ p ,q }∈N
{ p ,q }
⋅T ( f p ≠ fq ) (1)
where f denotes the pixel label, p indexes pixel, N is the set of adjacent pixels, {p, q} represents a pair of pixels, and the function T(.) is a constant (set as 0.1 in our work through empirical evaluation) when the condition inside the parentheses is true and 0 otherwise. The first term of Eq. (1) is the data term, which is determined by the connection energy t-links between each pixel and each terminal of the graph. In our work, Dp(fp) is assigned as (cp - Ip)2, where Ip is the intensity value of pixel p. The second term in Eq. (1) is the pixel continuity term, which is determined by the connection energy n-links between neighborhood pixels in the multiple-terminal graph, as defined by the following equation [12],
°2 K if | I p − I q |≤ 5 °¯K if | I p − I q |> 5
ω{ p,q} = ω ( I p − I q ) = ®
where K is the Potts model parameter, Ip and Iq are two neighboring pixel values.
(2)
1128
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
Finally, by using an implementation of the -expansion and fast max-flow/min-cut algorithms introduced in [12], the energy function of Eq. (1) is optimized approximately, and the pixel category label f is obtained. Consequently, we can re-divide each pixel to its new class. Finally, to split some cytoplasm with slight overlapping, the segmented binary image (after merging the three foreground subclasses) is processed by morphological opening with a 5×5 square structuring element. Actually, there exists some other method to replace the 3D Otsu thresolding, such as K-means clustering. We choose Otsu thresholding because it is more efficient and shows better segmentation results in our evaluation (please refer to the result section for more details). In addition, Otsu’s method is one of the best thresholding methods for general images [14]. Fig. 1(c)-(e) show the examples of cytoplasm segmentation. By examining the center part of Fig. 1(f), it can be seen that the segmentation is not affected by the bright illumination and dirt. However, we do not attempt to split the overlapping cytoplasm in this work. This is because that reliable delineation of the cytoplasm boundary for each cell is unrealistic even for human expert in the presence of heavily overlapping cells. 3. Experiments 3.1. Clinical Data Collection All images used in this study were acquired using an Olympus BX41 microscope equipped with 20X objective (Olympus America, Inc., Central Valley, PA), Jenoptik ProgRes CF Color 1.4 Megapixel Camera (Jenoptik Optical Systems Inc., Jena, Germany), and MS300 motorized stage (NJRGB Inc., Nanjing, China). Image specifications were 24 bit RGB channels with resolution of 1360×1024 pixels. The data set included 51 cervical cell images from 21 cervical slides, which were collected from the Department of Pathology, People's Hospital of Nanshan District, Shenzhen, China, in 2010. All slides are prepared using manual liquid-based cytology (MLBC) technique and are stained with H&E. Our training and test set consisted of 30 and 21 images, respectively. The ground truths used for evaluations of cytoplasm segmentation were obtained by manual delineation by a pathologist. 3.2. Quantitative Assessment Methodologies The evaluation of cytoplasm segmentation was based on comparison with two other cytoplasm segmentation methods [8], [11] using mutual overlap metric [15]: Acc = 2|RGT RSeg| / (|RGT| + |RSeg|), where RGT denotes the ground truth region, RSeg is the segmented region, and |.| is the number of pixels in a certain region. 3.3. Results Our methods were implemented using C++. We run our C++ release software on a 64-bit Windows PC, which has a 2.66GHz quad-core CPU and a 4GB RAM. In our method, there is only one major parameter (cluster number in the multi-way segmentation) that we tuned on training images. We set it to 3, 4, 5, and 6, respectively. Correspondingly, the cytoplasm segmentation accuracy Acc was 0.89, 0.93, 0.93, and 0.91, respectively. Considering the tradeoff between accuracy and computational complexity, we chose 4 as the cluster number. Figure 2 gives three examples of our cytoplasm segmentation approach on cervical cell images, demonstrating that our method produced promising segmentations of cytoplasm in the presence of
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1129
overlap cells (Fig. 2(a)), poor contrast and non-uniform staining (Fig. 2(b)). We also can see that our algorithm can successfully extract cytoplasm boundaries when confronted with a large amount of inflammatory cells. Compared to the first two examples, Fig. 2(c) shows a relatively poor segmentation in which a very bright red speck exists in the upper part, obscuring boundaries and decreasing contrast of cells. Actually, the performance of a majority of our H&E stained images resembles the first two examples in Fig. 2.
Fig. 2. Examples of the proposed cytoplasm segmentation method applied to three cervical cell images. (a) Heavily overlap cells. (b) Poor contrast and non-uniform staining. (c) Bright red speck.
Fig. 3. Comparison of cytoplasm segmentation results with (a) manual delineation, (b) Gençtav et al.’s method [8], (c) Hu et al.’s method [11] and (d) our method.
Figure 3 shows two example segmentation results by the proposed method and comparison with the manual delineation, [8], and [11], where the first to the third columns are results by manual delineation, [8] and [11], respectively, and the fourth column corresponds to our propose method. The results of our method look much better than those two compared methods in terms of segmentation accuracy. Figure 4 shows the mean and standard deviation of the cytoplasm segmentation accuracy achieved by the compared algorithms along with Otsu thresholding and K-means. The average segmentation accuracies with respect to the mean and standard deviation of all the algorithms are listed in Table 1. In Gençtav et al.’s [8] method, the only parameter is the radius of the disk of the black top-hat algo-
1130
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
rithm. We found that this method had difficulties in obtaining consistently satisfactory results with the same set of this parameter for different images. So we used a radius of 210 pixels as set in Ref. [8]. The results obtained by Otsu and K-means are better than Ref. [8] and [11]. This is because our proposed preprocessing is applied on the test image before applying Otsu and K-means. Note that Otsu performs better than K-means especially in terms of the standard deviation. Furthermore, we can see that 3% improvement (from 90% to 93%) is achieved after applying GC refinement on the Otsu segmentation. Our cytoplasm segmentation method takes 0.3 seconds per image on average.
Fig. 4. Comparison of cytoplasm segmentation performance with [8], [11], Otsu, and K-means. Table 1 Comparison of cytoplasm segmentation performance with respect to the mean and standard deviation of the segmentation accuracy Mean Std.
ours 0.93 0.03
[8] 0.64 0.26
[11] 0.76 0.18
Otsu 0.90 0.04
K-means 0.83 0.17
4. Conclusion This paper focus on the automated segmentation of cervical cytoplasm from cytology images with non-ideal imaging conditions, as existing techniques often designed for nuclei. The proposed GCbased segmentation method along with our preprocessing approach allow for cytoplasm delineation when image histograms present non-bimodal distribution. However, current segmentation method cannot delineate the cytoplasm boundary for each cell, and may fail to delineate the cytoplasm bound-
L. Zhang et al. / Cytoplasm segmentation on cervical cell images using graph cut-based approach
1131
ary in red regions caused by intensive illumination. In our future work, we plan to use the proposed method combined with our proposed nucleus segmentation method [16] for finding of abnormal cells from cervical cytology slides based on classification of segmentation results. 5. Acknowledgement We would like to thank Dr. Shaoxiong Liu from the Pathology Department at the People's Hospital of Nanshan District for providing the ground truth delineation. References [1]
[2] [3]
[4] [5] [6] [7]
[8] [9] [10] [11] [12] [13] [14] [15] [16]
H. C. Kitchener, R. Blanks, G. Dunn, L. Gunn, M. Desai, R. Albrow, J. Mather, D. N. Rana, H. Cubie, C. Moore, R. Legood, A. Gray and S. Moss, Automation-assisted versus manual reading of cervical cytology (MAVARIC): a randomised controlled trial, The Lancet Oncology 12 (2011), 56–64. J. Rittscher, Characterization of biological process through automated image analysis, Annual Review of Biomedical Engineering 12 (2010), 315–344. Y. Marinakis, G. Dounias and J. Jantzen, Pap smear diagnosis using a hybrid intelligent scheme focusing on genetic algorithm based feature selection and nearest neighbour classification, Computers in Biology and Medicine 39 (2009), 69–78. M. H. Tsai, Y. K. Chan, Z. Z. Lin, S. F. Yang-Mao and P. C. Huang, Nucleus and cytoplast contour detector of cervical smear image, Pattern Recognition Letters 29 (2008), 1441–1453. K. Li, Z. Lu, W. Liu and J. Yin, Cytoplasm and nucleus segmentation in cervical smear images using Radiating GVF Snake, Pattern Recognition 45 (2012), 1255–1264. S. F. Yang-Mao, Y. K. Chan and Y. P. Chu, Edge enhancement nucleus and cytoplast contour detector of cervical smear images, IEEE Transactions on Systems Man and Cybernetics-Part B: Cybernetics 38 (2008), 353–366. C. Bergmei, M. G. Silvente, J. E. Lopez-Cuervo and J. M. Benitez, Segmentation of cervical cell nuclei in highresolution microscopic images: A new algorithm and a web-based software framework, Computer Methods and Programs in Biomedicine 107 (2012), 497–512. A. Gençtav, S. Aksoy and S. Önder, Unsupervised segmentation and classification of cervical cell images, Pattern Recognition 45 (2012), 4151–4168. N. M. Harandi, S. Sadri, N. A. Moghaddam and R. Amirfattahi, An automated method for segmentation of epithelial cervical cells in images of ThinPrep, Journal of Medical Systems 34 (2010), 1043–1058. J. Jantzen, J. Norup, G. Dounias and B. Bjerregaard, Pap-smear benchmark data for pattern classification, in Proceedings of the Nature-Inspired Smart Information Systems 1nd Annual Symposium (2005). M. Hu, X. Ping, and Y. Ding, Automated cell nucleus segmentation using improved snake, in International Conference on Image Processing (2004), 2737–2740. Y. Boykov, O. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2001), 1222–1239. N. A. Otsu, Threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics 9 (1979), 62–66. P. K. Sahoo, S. Soltani, A. K. C. Wong and Y. Chen, A survey of thresholding techniques, Computer Vision Graphics Image Processing 41 (1988), 233–260. M. Sonka and J. M. Fitzpatrick. Handbook of medical imaging, SPIE Press, Washington, 2000. L. Zhang, J. Li, S. Chen, T. Wang, S. Jiang and S. Liu, Segmentation of abnormal cervical nuclei using an adaptive and local approach, Journal of Image and Graphics, accepted.
Copyright of Bio-Medical Materials & Engineering is the property of IOS Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
