Bio-Medical Materials and Engineering 24 (2014) 1209–1216 DOI 10.3233/BME-130922 IOS Press
1209
Texture analysis and classification of ultrasound liver images1 Shuang Gaoa, Yuhua Peng a, Huizhi Guo a, Weifeng Liu a, Tianxin Gaoa, Yuanqing Xu a* and Xiaoying Tanga* a
School of Life Science, Beijing Institute of Technology, Beijing, 100081, China
Abstract. Ultrasound as a noninvasive imaging technique is widely used to diagnose liver diseases. Texture analysis and classification of ultrasound liver images have become an important research topic across the world. In this study, GLGCM (Gray Level Gradient Co-Occurrence Matrix) was implemented for texture analysis of ultrasound liver images first, followed by the use of GLCM (Gray Level Co-occurrence Matrix) at the second stage. Twenty two features were obtained using the two methods, andseven most powerful features were selected for classification using BP (Back Propagation) neural network. Fibrosis was divided into five stages (S0-S4) in this study. The classification accuracies of S0-S4 were 100%, 90%, 70%, 90% and 100%, respectively. Keywords: liver fibrosis, ultrasonic image, texture features analysis, texture features extraction, artificial neural network
1. Introduction Liver fibrosis is the pathological characteristic in most types of chronic liver diseases. More than 500, 000 people die of liver cancer every year1. Since the 1990s, many studies have been conducted on texture analysis of ultrasound liver images. Wu Chung-Ming used a texture feature set based on spatial gray-level dependence matrices, the Fourier power spectrum, the gray-level difference statistics, and the Laws’ texture energy measures, to extract features2. Ebara calculated the relationship between the diameter of the regenerative nodule and the coarse score (CS) to classify cirrhotic liver and chronic hepatitis3. Mojsilovic proposed an approach of texture description based on nonseparable wavelet decomposition4. Ahmadian proposed a method for texture classification based on Gabor wavelet, which was applied to classify ultrasonic liver images into three types: normal liver, liver hepatitis and cirrhosis5. Azaid applied the gray level histogram to extract texture features to distinguish normal, cirrhosis and liver cancer6. Li Guo-kuan used GLCM to extract texture features to distinguish fatty liver from normal liver7. Fujita extracted features based on Gabor transform to distinguish normal liver from fibrosis8. At present, studies on texture features of ultrasound liver fibrosis images mainly focus on histogram, GLCM, gray stroke length, Fourier power spectrum or wavelet transform, and the classifications are mainly based on Bayesian classifier, neural networks9 and support vector machines10.
1 *
The study is supported by National Natural Science Foundation of China (No. 81271568) Corresponding author: Yuanqing Xu and Xiaoying Tang. E-mail: [email protected] or [email protected]
0959-2989/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
1210
S. Gao et al. / Texture analysis and classification of ultrasound liver images
According to the Guidelines of Prevention and Treatment of Viral Hepatitis of Chinese Medical Association (2001), fibrosis is divided into five stages (S0-S4), which is very meaningful for the early diagnosis, precaution and treatment of liver cirrhosis. S0 means no fibrosis;S1 is fibrosis of portal tracts and their around; S2 is bridging fibrosis, mostly resulting from bridging necrosis; S3 is wide bridging fibrosis which separates and destroys hepatic lobule, but no cirrhosis; and S4 is early cirrhosis with extensive destruction of liver parenchyma and fibrosis proliferation11. GLGCM is a powerful tool for texture analysis and is used in both segmentation of blood vessels and intelligent identification of wallpaper labeling 121314. However, GLGCM is seldom applied in texture analysis of ultrasound liver images. In this study, GLGCM was initially introduced to extract texture features of ultrasound liver images, followed by the adoption of GLCM as the tool at the second stage. Finally, BP neural network was chosen as the classifier to classify S0, S1, S2, S3 and S4. 2. Materials All ultrasound images were obtained from a local municipal hospital by an expert radiologist. Thirty seven patients were involved in this study, with an average age of 31.4 years old. All the patients were verified histologically: S0: 4 patients, S1:16 patients, S2: 8 patients, S3: 5 patients, and S4: 4 patients. The image acquisition system was HITACHI-HIVISION 900 with 3.5MHz transducer frequency. All experimental images were ultrasound liver images of right lobe. The gain, focus, depth range, and time gain compensation (TGC) had been adjusted by the radiologist to obtain the perceptually best backscattered ultrasound image before data collection and remained unchanged until the last case was scanned. There were 300 images from the 37 cases, whose sizes were all 1024 x 768 pixels. Selecting regions of interest (ROI) from ultrasonic images was under the guidance of liver disease doctors. The ROI selection in the experiment followed the following principles: 1) The ROIs from each image were the same area of the liver; 2) The ROIs were rectangular and excluded areas containing blood vessels and ribs acoustic shadow; and 3) The size of the ROIs was fixed as 64 × 64 pixels. 3. Methods 3.1. Gray Level Gradient Co-Occurrence Matrix The definition of GLGCM is as follows15. Suppose an image to be analyzed is a square whose di-
mension is N × N , and the gray level appearing in each pixel is quantized to N g levels.
{ f (i , j ); i, j = 0,1, 2,3,..., N − 1} is introduced to represent the image. The definition of gray level matrix is as Eq. (1):
F (i, j ) = [ f (i, j ) L f N g ] + 1 L
(1)
where f is the defined gray level. Firstly, a gradient matrix is calculated by the gradient operator, which is represented by { g (i , j ); i , j = 0,1, 2,3,..., N − 1} .The new gradient matrix is defined as Eq. (2):
S. Gao et al. / Texture analysis and classification of ultrasound liver images
1211
G (i, j ) = [ g (i, j ) L g g max ] + 1
(2)
L g where g is the defined gray level, and max is the gray level of g (i , j ) . Then the definition of GLGCM is as Eq. (3): {H ( x, y ); x = 0,1,2,3,..., L f − 1; y = 0,1,2,3,..., L g − 1}
(3)
The definition of H ( x , y ) is the number of {(i , j ) F (i , j ) = x , G (i , j ) = y ; i , j = 0,1, 2, 3,..., N − 1} , that is to say, H ( x , y ) is the number of pixels whose gray level is x and gradient level is y. The definition of normalization of GLGCM is as Eq. (4): ∧
H ( x, y) = L
H ( x, y) f
−1Lg −1
¦¦ H ( x, y) i =0 j =0
(4)
Based on the GLGCM, 15 texture features can be defined as shown in Table 1: Table 1 The definitions of texture features from GLGCM Feature name Small Gradient Advantage
Definition L g −1 L f −1 ∧
T1 =
¦ ¦ H ( x , y ) /( y + 1) y=0 x=0
L f −1 L g −1 ∧
¦ ¦ H ( x, y ) x=0 y=0
Big Gradient Advantage
L g −1 L f −1 ∧
¦¦
2
Explanation Small Gradient Advantage and Big Gradient Advantage measure gray change. A homogeneous image will result in high Small Gradient Advantage value and low Big Gradient Advantage value.
H ( x, y ) y 2
y=0 x=0
T2 =
L f −1 L g −1 ∧
¦¦
H ( x, y )
x=0 y=0
Gray Level Distribution Inhomogeneity
L f −1 L g −1 ∧
¦ [ ¦ H ( x , y )] x=0
T3 =
2
y =0
L f −1 L g −1 ∧
¦ ¦ H ( x, y ) x =0 y =0
Gradient Level Distribution Inhomogeneity
L g −1 L f −1 ∧
¦ [ ¦ H ( x , y )] y =0
T4 =
2
x=0
L f −1 L g −1 ∧
¦ ¦ H ( x, y )
Gray Level Distribution Inhomogeneity measures gray level change. If the gray of an image only focuses on some gray levels, Gray Level Distribution Inhomogeneity value will become large. Gradient Level Distribution Inhomogeneity measures texture homogeneity of the image. For an image with homogeneous texture, Gradient Level Distribution Inhomogeneity is low.
x =0 y =0
Energy Gray Level Average Gradient Level Average
T5 =
T6 = T7 =
L f −1 L g −1
¦¦
∧
H 2 ( x, y)
When the image is homogeneous, the Energy will have high values.
x=0 y=0
L f −1
Lg −1
¦ x[ ¦ H ( x, y)] x =0
y =0
Lg −1
L f −1 ∧
y =0
x =0
¦ y[ ¦ H ( x, y)]
Gray Level Average measures gray magnitude distribution. A dark image has low Gray Level Average. Gradient Level Average measures the distribution of gradient magnitudes. A high value of GM indicates an image with large texture
1212
S. Gao et al. / Texture analysis and classification of ultrasound liver images
Gray Level Mean Square Error
L f −1
Lg −1
T8 = { ¦ ( x − T6 ) 2 [ ¦ H ( x, y )]}1 2 x =0
Gradient Level Mean Square Error Correlation Gray Level Entropy Gradient Level Entropy
∧
y =0
L g −1
L
f
−1 ∧
T 9 = { ¦ ( y − T 7 ) 2 [ ¦ H ( x , y )]} 1 y=0
L
T 10 = −
f
2
x=0
−1 L g −1
¦ ¦ x=0
y=0
∧
L g −1 ∧
L f −1 L g −1 ∧
T 11 = − ¦
¦ H ( x , y ) log ¦ H ( x , y ) y=0
x=0 y=0
L f −1 L g −1 ∧
T12 = − ¦
¦ H ( x , y ) log
L f −1 L g −1 ∧
T 13 = − ¦
Gradient Level Mean Square Error measures gradient heterogeneity.
( x − T 6 )( y − T 7 ) H ( x , y )
x=0 y =0
Mixture Entropy
grooves depth. Gray Level Mean Square Error measures gray heterogeneity.
¦
L f −1 ∧
¦ H ( x, y ) x=0
∧
No matter Gray Level Entropy, Gradient Level Entropy or Mixture Entropy, they all measure the amount of information in the image, also known as randomness of the image texture. A homogeneous image will result in a lower entropy value.
H ( x , y ) log H ( x , y )
x=0 y=0
Inertia Inverse Difference Moment
T 14 =
T15 =
L f −1 L g −1
¦¦
∧
( x, y) 2 H ( x, y )
x=0 y=0
L f −1 L g −1
¦
x =0
∧
H ( x, y ) ¦ 1 + (x − y) 2 y=0
3.2. Gray Level Co-occurrence Matrix The definition of GLCM is as follows16. Suppose the image to be analyzed is a rectangle whose di-
N mension is M × N , and the gray level appearing in each pixel is quantized to g levels. Let
Lx = {1, 2,3,..., M } and Ly = {1, 2,3,..., N } be the spatial domains, and G = {1, 2,3,..., N g } be the set
of
Ng
quantized gray levels. The image I can be represented as a function which assigns some gray
L ×L ;I : L ×L → G
x y x . level in G to each pixel or pair of coordinates in y In order to extract the spatial relationship among pixels, gray level co-occurrence matrices are constructed to record the relative frequencies P (i , j , d , θ ) with two pixels in the image, one with gray lev-
el
i (( x1 , y1 )) and the other with gray level j (( x2 , y2 )) , in direction θ and distance d . When θ = 0 °, 45 °, 90 °, 135 °, starting from the ox axis counterclockwise, the calculation of Co-
occurrence matrix element is defined as Eq. (5):
P(i, j, d ,0) = #{((k , l ),(m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx )
| k − m = 0,| l − n |= d , I (k , l ) = i, I (m, n) = j} P(i, j , d , 45ο ) = #{((k , l ), (m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) | (k − m = d , l − n = −d )or (k − m = −d , l − n = d ) I (k , l ) = i, I (m, n) = j} P (i, j , d ,90ο ) = #{((k , l ), ( m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) || k − m |= d , l − n = 0, I (k , l ) = i, I (m, n) = j}
S. Gao et al. / Texture analysis and classification of ultrasound liver images
1213
P(i, j , d ,135ο ) = #{((k , l ), (m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) | (k − m = d , l − n = d )or (k − m = −d , l − n = −d ), I (k , l ) = i, I (m, n) = j} (5) where # denotes the number of elements in the set. The definition of normalization of GLCM is as Eq. (6):
P (i , j ) = P (i , j , d , θ ) / R
(6)
When θ = 0, R = ( M − 1) × N , When θ = 90 , R = M × ( N − 1) , and when θ = 45o or θ = 135 o , R = ( M − 1) × ( N − 1) . Based on the GLCM, seven texture features can be defined as shown in Table 2: o
Table 2 The definitions of texture features from GLCM Feature name Mean
Definition m =
Explanation
N g −1 N g −1
¦ ¦ i=0
i ⋅ P (i, j )
i=0
Angular Second Moment
f 1 = ASM = ¦ ¦ {P (i, j )}
Contrast
f 2 = ¦¦ (i − j ) 2 P(i, j )
i
i
Correlation
Entropy
f3 =
N g −1 N g −1
¦¦ i=o
ijP (i, j ) − u1u2
j =0
σ1 σ 2
f 4 = − ¦¦ P (i, j ) log P (i, j ) j
f 5 = ¦ ¦ (i − m) 2 P (i, j ) i
Inverse Differential Moment
j
j
i
Variance
2
f6 =
j
N g −1 N g −1
¦ i =0
Angular Second Moment, a measure of homogeneity, is also known as energy. When the image is homogeneous, the Angular Second Moment will have high values. Contrast is a measure of the local variations presented in an image, and it measures clarity and texture grooves depth of the image. Correlation measures the similarity between elements of the row and column of GLCM which reflects the correlation of the gray in the local area of image. Higher values can be obtained for similar gray-level regions. Entropy measures the amount of information in the image, also known as randomness of the image texture. A homogeneous image will result in a lower entropy value.
P(i, j ) ¦ 2 j =0 1 + (i − j )
Inverse Differential Moment reflects the texture homogeneity of the image, and it measures local texture change. When images are homogeneous, the value of IDM is large.
4. Results 4.1. Texture feature extraction results Twenty two texture features have been obtained from GLCM and GLGCM. It is necessary to select the most powerful features to build our quantitative tissue characterization system. In GLGCM, there are more obvious changes in Gray Level Distribution Inhomogeneity, Energy and Gray Level Mean-Square Error. The results are presented in Figure 1. As shown in Figure 1, as fibrosis grade increases Gray Level Distribution Inhomogeneity and Energy become smaller, while the value of Gray Level Mean-Square Error becomes larger, and this is consistent with the pathological features of liver fibrosis and visual sense of fibrosis ultrasound images.
1214
S. Gao et al. / Texture analysis and classification of ultrasound liver images
The reason behind the feature value change is that fibrosis causes liver ultrasound images to have more gray levels instead of some specific gray levels, and also aggrandize gradient11. Therefore according to the definitions in Table 1, as fibrosis increases the value of Gray Level Distribution Inhomogeneity becomes smaller, while Gray Level Mean-Square Error becomes larger. The change in Energy is similar to the ASM in GLCM
Fig.1. Features of GLGCM
In GLCM, there are more obvious changes in Variance, ASM (energy), Contrast and Correlation in o different grades (S0-S4) of fibrosis when θ =135 d = 5. More detailed results are shown in Figure 2.
Fig.2. Features of GLCM
As shown in Figure 2, the values of Angular Second Moment and Correlation become lower as fibrosis grade increases, while the values of Variance and Contrast become higher. The reason behind the feature value change is that reflection and scattering of ultrasound beam increase in liver fibrosis areas2, which can lead to ultrasound image inhomogeneity and gray level difference. The texture of
1215
S. Gao et al. / Texture analysis and classification of ultrasound liver images
fibrosis liver becomes deeper, which also leads to more clarity in ultrasound images, so according to the definitions in Table 2,the values of Angular Second Moment and Correlation become lower as the degree of fibrosis increases, while the values of Contrast and Variance are higher with the increase of fibrosis degree. The changes of all these values are consistent with texture changes of liver fibrosis. In addition, there is more texture in diagonal direction as fibrosis increases, especially at 135o. 4.2. Classification results BP (Back Propagation) neural network, a multilayer feed-forward neural network, was selected as the classifier in this study with input layer, output layer and one hidden layer. The output of the neural network was set with five nodes corresponding to S0, S1, S2, S3 and S4. The input layer of the neural network had seven nodes, corresponding to the seven texture features discussed in the previous section of this paper (3.2).Five images of each stage, 25 in all, were chosen as the training samples. The hidden layer was set with 16 nodes finally on the basis of the training results using different nodes. Ten images of each stage (50 in all) were selected as the test samples. The result of the classification is shown in Table 3. Table 3 Result of classification by BP neural network S0 S1 S2 S3 S4
1
2 S1
3
4 S2
5 S2
6
7 S1
8 S3
9
10
The classification accuracies of S0-S4 are 100%, 90%, 70%, 90% and 100%, respectively. As can be seen from the classification results, there is not cross-phase recognition. Classification accuracies of S0 and S4 are higher. In the wrong recognition of S1, S2 and S3, S2 tends to be recognized as S1 or S3. This is consistent with clinicalsymptoms.S1, S2and S3 are all fibrosis, so symptoms in patients of S1, S2 and S3 identical to each other. This is particularly the case with S2, which is in the middle stage of fibrosis, making it harder to be classified. 5. Discussion and conclusions In this study ultrasonic liver images of five stages (S0-S4) are used for texture analysis, and GLGCM and GLCM are adopted as description methods for texture features, which is followed by the use of BP neural network the classifier to achieve good classification results. The findings of this study indicate that more detailed division is important for early diagnosis of liver diseases; however, this can also make the analysis and classification more difficult. As shown in Figure 1 and Figure2, S0 and S4 have significant difference from other stages of all features, while S1-S3, despite their obvious differences from S0 and S4, are easily confused among themselves, which maybe the reason why classification accuracies of S1-S3 are lower, especiallyS2, which is the middle stage of fibrosis, is more difficult to distinguish from both S1 and S3. With GLGCM, more texture features are extracted, so it has advantages in extraction of texture features of ultrasonic liver images. In future research, more effective texture features of GLGCM need to
1216
S. Gao et al. / Texture analysis and classification of ultrasound liver images
be proved and more experiments need to be conducted to improve the classification, so that S0-S4 could be classified more accurately. References [1] [2] [3]
[4] [5] [6] [7]
[8] [9] [10]
[11] [12] [13] [14] [15] [16]
Golfieri R, Marini E, Bazzocchi A. Small (
1209
Texture analysis and classification of ultrasound liver images1 Shuang Gaoa, Yuhua Peng a, Huizhi Guo a, Weifeng Liu a, Tianxin Gaoa, Yuanqing Xu a* and Xiaoying Tanga* a
School of Life Science, Beijing Institute of Technology, Beijing, 100081, China
Abstract. Ultrasound as a noninvasive imaging technique is widely used to diagnose liver diseases. Texture analysis and classification of ultrasound liver images have become an important research topic across the world. In this study, GLGCM (Gray Level Gradient Co-Occurrence Matrix) was implemented for texture analysis of ultrasound liver images first, followed by the use of GLCM (Gray Level Co-occurrence Matrix) at the second stage. Twenty two features were obtained using the two methods, andseven most powerful features were selected for classification using BP (Back Propagation) neural network. Fibrosis was divided into five stages (S0-S4) in this study. The classification accuracies of S0-S4 were 100%, 90%, 70%, 90% and 100%, respectively. Keywords: liver fibrosis, ultrasonic image, texture features analysis, texture features extraction, artificial neural network
1. Introduction Liver fibrosis is the pathological characteristic in most types of chronic liver diseases. More than 500, 000 people die of liver cancer every year1. Since the 1990s, many studies have been conducted on texture analysis of ultrasound liver images. Wu Chung-Ming used a texture feature set based on spatial gray-level dependence matrices, the Fourier power spectrum, the gray-level difference statistics, and the Laws’ texture energy measures, to extract features2. Ebara calculated the relationship between the diameter of the regenerative nodule and the coarse score (CS) to classify cirrhotic liver and chronic hepatitis3. Mojsilovic proposed an approach of texture description based on nonseparable wavelet decomposition4. Ahmadian proposed a method for texture classification based on Gabor wavelet, which was applied to classify ultrasonic liver images into three types: normal liver, liver hepatitis and cirrhosis5. Azaid applied the gray level histogram to extract texture features to distinguish normal, cirrhosis and liver cancer6. Li Guo-kuan used GLCM to extract texture features to distinguish fatty liver from normal liver7. Fujita extracted features based on Gabor transform to distinguish normal liver from fibrosis8. At present, studies on texture features of ultrasound liver fibrosis images mainly focus on histogram, GLCM, gray stroke length, Fourier power spectrum or wavelet transform, and the classifications are mainly based on Bayesian classifier, neural networks9 and support vector machines10.
1 *
The study is supported by National Natural Science Foundation of China (No. 81271568) Corresponding author: Yuanqing Xu and Xiaoying Tang. E-mail: [email protected] or [email protected]
0959-2989/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
1210
S. Gao et al. / Texture analysis and classification of ultrasound liver images
According to the Guidelines of Prevention and Treatment of Viral Hepatitis of Chinese Medical Association (2001), fibrosis is divided into five stages (S0-S4), which is very meaningful for the early diagnosis, precaution and treatment of liver cirrhosis. S0 means no fibrosis;S1 is fibrosis of portal tracts and their around; S2 is bridging fibrosis, mostly resulting from bridging necrosis; S3 is wide bridging fibrosis which separates and destroys hepatic lobule, but no cirrhosis; and S4 is early cirrhosis with extensive destruction of liver parenchyma and fibrosis proliferation11. GLGCM is a powerful tool for texture analysis and is used in both segmentation of blood vessels and intelligent identification of wallpaper labeling 121314. However, GLGCM is seldom applied in texture analysis of ultrasound liver images. In this study, GLGCM was initially introduced to extract texture features of ultrasound liver images, followed by the adoption of GLCM as the tool at the second stage. Finally, BP neural network was chosen as the classifier to classify S0, S1, S2, S3 and S4. 2. Materials All ultrasound images were obtained from a local municipal hospital by an expert radiologist. Thirty seven patients were involved in this study, with an average age of 31.4 years old. All the patients were verified histologically: S0: 4 patients, S1:16 patients, S2: 8 patients, S3: 5 patients, and S4: 4 patients. The image acquisition system was HITACHI-HIVISION 900 with 3.5MHz transducer frequency. All experimental images were ultrasound liver images of right lobe. The gain, focus, depth range, and time gain compensation (TGC) had been adjusted by the radiologist to obtain the perceptually best backscattered ultrasound image before data collection and remained unchanged until the last case was scanned. There were 300 images from the 37 cases, whose sizes were all 1024 x 768 pixels. Selecting regions of interest (ROI) from ultrasonic images was under the guidance of liver disease doctors. The ROI selection in the experiment followed the following principles: 1) The ROIs from each image were the same area of the liver; 2) The ROIs were rectangular and excluded areas containing blood vessels and ribs acoustic shadow; and 3) The size of the ROIs was fixed as 64 × 64 pixels. 3. Methods 3.1. Gray Level Gradient Co-Occurrence Matrix The definition of GLGCM is as follows15. Suppose an image to be analyzed is a square whose di-
mension is N × N , and the gray level appearing in each pixel is quantized to N g levels.
{ f (i , j ); i, j = 0,1, 2,3,..., N − 1} is introduced to represent the image. The definition of gray level matrix is as Eq. (1):
F (i, j ) = [ f (i, j ) L f N g ] + 1 L
(1)
where f is the defined gray level. Firstly, a gradient matrix is calculated by the gradient operator, which is represented by { g (i , j ); i , j = 0,1, 2,3,..., N − 1} .The new gradient matrix is defined as Eq. (2):
S. Gao et al. / Texture analysis and classification of ultrasound liver images
1211
G (i, j ) = [ g (i, j ) L g g max ] + 1
(2)
L g where g is the defined gray level, and max is the gray level of g (i , j ) . Then the definition of GLGCM is as Eq. (3): {H ( x, y ); x = 0,1,2,3,..., L f − 1; y = 0,1,2,3,..., L g − 1}
(3)
The definition of H ( x , y ) is the number of {(i , j ) F (i , j ) = x , G (i , j ) = y ; i , j = 0,1, 2, 3,..., N − 1} , that is to say, H ( x , y ) is the number of pixels whose gray level is x and gradient level is y. The definition of normalization of GLGCM is as Eq. (4): ∧
H ( x, y) = L
H ( x, y) f
−1Lg −1
¦¦ H ( x, y) i =0 j =0
(4)
Based on the GLGCM, 15 texture features can be defined as shown in Table 1: Table 1 The definitions of texture features from GLGCM Feature name Small Gradient Advantage
Definition L g −1 L f −1 ∧
T1 =
¦ ¦ H ( x , y ) /( y + 1) y=0 x=0
L f −1 L g −1 ∧
¦ ¦ H ( x, y ) x=0 y=0
Big Gradient Advantage
L g −1 L f −1 ∧
¦¦
2
Explanation Small Gradient Advantage and Big Gradient Advantage measure gray change. A homogeneous image will result in high Small Gradient Advantage value and low Big Gradient Advantage value.
H ( x, y ) y 2
y=0 x=0
T2 =
L f −1 L g −1 ∧
¦¦
H ( x, y )
x=0 y=0
Gray Level Distribution Inhomogeneity
L f −1 L g −1 ∧
¦ [ ¦ H ( x , y )] x=0
T3 =
2
y =0
L f −1 L g −1 ∧
¦ ¦ H ( x, y ) x =0 y =0
Gradient Level Distribution Inhomogeneity
L g −1 L f −1 ∧
¦ [ ¦ H ( x , y )] y =0
T4 =
2
x=0
L f −1 L g −1 ∧
¦ ¦ H ( x, y )
Gray Level Distribution Inhomogeneity measures gray level change. If the gray of an image only focuses on some gray levels, Gray Level Distribution Inhomogeneity value will become large. Gradient Level Distribution Inhomogeneity measures texture homogeneity of the image. For an image with homogeneous texture, Gradient Level Distribution Inhomogeneity is low.
x =0 y =0
Energy Gray Level Average Gradient Level Average
T5 =
T6 = T7 =
L f −1 L g −1
¦¦
∧
H 2 ( x, y)
When the image is homogeneous, the Energy will have high values.
x=0 y=0
L f −1
Lg −1
¦ x[ ¦ H ( x, y)] x =0
y =0
Lg −1
L f −1 ∧
y =0
x =0
¦ y[ ¦ H ( x, y)]
Gray Level Average measures gray magnitude distribution. A dark image has low Gray Level Average. Gradient Level Average measures the distribution of gradient magnitudes. A high value of GM indicates an image with large texture
1212
S. Gao et al. / Texture analysis and classification of ultrasound liver images
Gray Level Mean Square Error
L f −1
Lg −1
T8 = { ¦ ( x − T6 ) 2 [ ¦ H ( x, y )]}1 2 x =0
Gradient Level Mean Square Error Correlation Gray Level Entropy Gradient Level Entropy
∧
y =0
L g −1
L
f
−1 ∧
T 9 = { ¦ ( y − T 7 ) 2 [ ¦ H ( x , y )]} 1 y=0
L
T 10 = −
f
2
x=0
−1 L g −1
¦ ¦ x=0
y=0
∧
L g −1 ∧
L f −1 L g −1 ∧
T 11 = − ¦
¦ H ( x , y ) log ¦ H ( x , y ) y=0
x=0 y=0
L f −1 L g −1 ∧
T12 = − ¦
¦ H ( x , y ) log
L f −1 L g −1 ∧
T 13 = − ¦
Gradient Level Mean Square Error measures gradient heterogeneity.
( x − T 6 )( y − T 7 ) H ( x , y )
x=0 y =0
Mixture Entropy
grooves depth. Gray Level Mean Square Error measures gray heterogeneity.
¦
L f −1 ∧
¦ H ( x, y ) x=0
∧
No matter Gray Level Entropy, Gradient Level Entropy or Mixture Entropy, they all measure the amount of information in the image, also known as randomness of the image texture. A homogeneous image will result in a lower entropy value.
H ( x , y ) log H ( x , y )
x=0 y=0
Inertia Inverse Difference Moment
T 14 =
T15 =
L f −1 L g −1
¦¦
∧
( x, y) 2 H ( x, y )
x=0 y=0
L f −1 L g −1
¦
x =0
∧
H ( x, y ) ¦ 1 + (x − y) 2 y=0
3.2. Gray Level Co-occurrence Matrix The definition of GLCM is as follows16. Suppose the image to be analyzed is a rectangle whose di-
N mension is M × N , and the gray level appearing in each pixel is quantized to g levels. Let
Lx = {1, 2,3,..., M } and Ly = {1, 2,3,..., N } be the spatial domains, and G = {1, 2,3,..., N g } be the set
of
Ng
quantized gray levels. The image I can be represented as a function which assigns some gray
L ×L ;I : L ×L → G
x y x . level in G to each pixel or pair of coordinates in y In order to extract the spatial relationship among pixels, gray level co-occurrence matrices are constructed to record the relative frequencies P (i , j , d , θ ) with two pixels in the image, one with gray lev-
el
i (( x1 , y1 )) and the other with gray level j (( x2 , y2 )) , in direction θ and distance d . When θ = 0 °, 45 °, 90 °, 135 °, starting from the ox axis counterclockwise, the calculation of Co-
occurrence matrix element is defined as Eq. (5):
P(i, j, d ,0) = #{((k , l ),(m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx )
| k − m = 0,| l − n |= d , I (k , l ) = i, I (m, n) = j} P(i, j , d , 45ο ) = #{((k , l ), (m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) | (k − m = d , l − n = −d )or (k − m = −d , l − n = d ) I (k , l ) = i, I (m, n) = j} P (i, j , d ,90ο ) = #{((k , l ), ( m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) || k − m |= d , l − n = 0, I (k , l ) = i, I (m, n) = j}
S. Gao et al. / Texture analysis and classification of ultrasound liver images
1213
P(i, j , d ,135ο ) = #{((k , l ), (m, n)) ∈ ( Ly × Lx˅× ( Ly × Lx ) | (k − m = d , l − n = d )or (k − m = −d , l − n = −d ), I (k , l ) = i, I (m, n) = j} (5) where # denotes the number of elements in the set. The definition of normalization of GLCM is as Eq. (6):
P (i , j ) = P (i , j , d , θ ) / R
(6)
When θ = 0, R = ( M − 1) × N , When θ = 90 , R = M × ( N − 1) , and when θ = 45o or θ = 135 o , R = ( M − 1) × ( N − 1) . Based on the GLCM, seven texture features can be defined as shown in Table 2: o
Table 2 The definitions of texture features from GLCM Feature name Mean
Definition m =
Explanation
N g −1 N g −1
¦ ¦ i=0
i ⋅ P (i, j )
i=0
Angular Second Moment
f 1 = ASM = ¦ ¦ {P (i, j )}
Contrast
f 2 = ¦¦ (i − j ) 2 P(i, j )
i
i
Correlation
Entropy
f3 =
N g −1 N g −1
¦¦ i=o
ijP (i, j ) − u1u2
j =0
σ1 σ 2
f 4 = − ¦¦ P (i, j ) log P (i, j ) j
f 5 = ¦ ¦ (i − m) 2 P (i, j ) i
Inverse Differential Moment
j
j
i
Variance
2
f6 =
j
N g −1 N g −1
¦ i =0
Angular Second Moment, a measure of homogeneity, is also known as energy. When the image is homogeneous, the Angular Second Moment will have high values. Contrast is a measure of the local variations presented in an image, and it measures clarity and texture grooves depth of the image. Correlation measures the similarity between elements of the row and column of GLCM which reflects the correlation of the gray in the local area of image. Higher values can be obtained for similar gray-level regions. Entropy measures the amount of information in the image, also known as randomness of the image texture. A homogeneous image will result in a lower entropy value.
P(i, j ) ¦ 2 j =0 1 + (i − j )
Inverse Differential Moment reflects the texture homogeneity of the image, and it measures local texture change. When images are homogeneous, the value of IDM is large.
4. Results 4.1. Texture feature extraction results Twenty two texture features have been obtained from GLCM and GLGCM. It is necessary to select the most powerful features to build our quantitative tissue characterization system. In GLGCM, there are more obvious changes in Gray Level Distribution Inhomogeneity, Energy and Gray Level Mean-Square Error. The results are presented in Figure 1. As shown in Figure 1, as fibrosis grade increases Gray Level Distribution Inhomogeneity and Energy become smaller, while the value of Gray Level Mean-Square Error becomes larger, and this is consistent with the pathological features of liver fibrosis and visual sense of fibrosis ultrasound images.
1214
S. Gao et al. / Texture analysis and classification of ultrasound liver images
The reason behind the feature value change is that fibrosis causes liver ultrasound images to have more gray levels instead of some specific gray levels, and also aggrandize gradient11. Therefore according to the definitions in Table 1, as fibrosis increases the value of Gray Level Distribution Inhomogeneity becomes smaller, while Gray Level Mean-Square Error becomes larger. The change in Energy is similar to the ASM in GLCM
Fig.1. Features of GLGCM
In GLCM, there are more obvious changes in Variance, ASM (energy), Contrast and Correlation in o different grades (S0-S4) of fibrosis when θ =135 d = 5. More detailed results are shown in Figure 2.
Fig.2. Features of GLCM
As shown in Figure 2, the values of Angular Second Moment and Correlation become lower as fibrosis grade increases, while the values of Variance and Contrast become higher. The reason behind the feature value change is that reflection and scattering of ultrasound beam increase in liver fibrosis areas2, which can lead to ultrasound image inhomogeneity and gray level difference. The texture of
1215
S. Gao et al. / Texture analysis and classification of ultrasound liver images
fibrosis liver becomes deeper, which also leads to more clarity in ultrasound images, so according to the definitions in Table 2,the values of Angular Second Moment and Correlation become lower as the degree of fibrosis increases, while the values of Contrast and Variance are higher with the increase of fibrosis degree. The changes of all these values are consistent with texture changes of liver fibrosis. In addition, there is more texture in diagonal direction as fibrosis increases, especially at 135o. 4.2. Classification results BP (Back Propagation) neural network, a multilayer feed-forward neural network, was selected as the classifier in this study with input layer, output layer and one hidden layer. The output of the neural network was set with five nodes corresponding to S0, S1, S2, S3 and S4. The input layer of the neural network had seven nodes, corresponding to the seven texture features discussed in the previous section of this paper (3.2).Five images of each stage, 25 in all, were chosen as the training samples. The hidden layer was set with 16 nodes finally on the basis of the training results using different nodes. Ten images of each stage (50 in all) were selected as the test samples. The result of the classification is shown in Table 3. Table 3 Result of classification by BP neural network S0 S1 S2 S3 S4
1
2 S1
3
4 S2
5 S2
6
7 S1
8 S3
9
10
The classification accuracies of S0-S4 are 100%, 90%, 70%, 90% and 100%, respectively. As can be seen from the classification results, there is not cross-phase recognition. Classification accuracies of S0 and S4 are higher. In the wrong recognition of S1, S2 and S3, S2 tends to be recognized as S1 or S3. This is consistent with clinicalsymptoms.S1, S2and S3 are all fibrosis, so symptoms in patients of S1, S2 and S3 identical to each other. This is particularly the case with S2, which is in the middle stage of fibrosis, making it harder to be classified. 5. Discussion and conclusions In this study ultrasonic liver images of five stages (S0-S4) are used for texture analysis, and GLGCM and GLCM are adopted as description methods for texture features, which is followed by the use of BP neural network the classifier to achieve good classification results. The findings of this study indicate that more detailed division is important for early diagnosis of liver diseases; however, this can also make the analysis and classification more difficult. As shown in Figure 1 and Figure2, S0 and S4 have significant difference from other stages of all features, while S1-S3, despite their obvious differences from S0 and S4, are easily confused among themselves, which maybe the reason why classification accuracies of S1-S3 are lower, especiallyS2, which is the middle stage of fibrosis, is more difficult to distinguish from both S1 and S3. With GLGCM, more texture features are extracted, so it has advantages in extraction of texture features of ultrasonic liver images. In future research, more effective texture features of GLGCM need to
1216
S. Gao et al. / Texture analysis and classification of ultrasound liver images
be proved and more experiments need to be conducted to improve the classification, so that S0-S4 could be classified more accurately. References [1] [2] [3]
[4] [5] [6] [7]
[8] [9] [10]
[11] [12] [13] [14] [15] [16]
Golfieri R, Marini E, Bazzocchi A. Small (
