IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
2575
Super-Resolution Image Reconstruction for Ultrasonic Nondestructive Evaluation Shanglei Li and Tsuchin Philip Chu Abstract—Ultrasonic testing is one of the most successful nondestructive evaluation (NDE) techniques for the inspection of carbon-fiber-reinforced polymer (CFRP) materials. This paper discusses the application of the iterative backprojection (IBP) super-resolution image reconstruction technique to carbon epoxy laminates with simulated defects to obtain high-resolution images for NDE. Super-resolution image reconstruction is an approach used to overcome the inherent resolution limitations of an existing ultrasonic system. It can greatly improve the image quality and allow more detailed inspection of the region of interest (ROI) with high resolution, improving defect evaluation and accuracy. First, three artificially simulated delamination defects in a CFRP panel were considered to evaluate and validate the application of the IBP method. The results of the validation indicate that both the contrast-tonoise ratio (CNR) and the peak signal-to-noise ratio (PSNR) value of the super-resolution result are better than the bicubic interpolation method. Then, the IBP method was applied to the low-resolution ultrasonic C-scan image sequence with subpixel displacement of two types of defects (delamination and porosity) which were obtained by the micro-scanning imaging technique. The result demonstrated that super-resolution images achieved better visual quality with an improved image resolution compared with raw C-scan images.
I. Introduction
W
ith the extensive use of carbon composite material in the aviation industry, ultrasonic testing (UT) has become one of the proven nondestructive evaluation (NDE) techniques, featuring large scale, high speed, and noncontact testing capabilities [1]–[4]. However, UT Cscan contrast and detection of defects in carbon composites is limited by the inhomogeneous density and the anisotropic properties of the composite material. Moreover, the C-scan image resolution depends on the physical characteristics of the ultrasound transducer (sensor); thus, increasing the resolution by UT sensor modification may not be a viable option. In addition, ultrasonic image quality can be affected by the speckle noise produced by the interference of backscattered signals [5], [6]. This makes it difficult for ultrasonic C-scan inspection to qualitatively and quantitatively evaluate defects’ size, shape, and type. A key NDE goal is to obtain images having
Manuscript received May 17, 2013; accepted September 10, 2013. S. Li is with the Department of Electrical and Computer Engineering, Southern Illinois University Carbondale, Carbondale, IL (e-mail: [email protected]). T. P. Chu is with the Department of Mechanical Engineering and Energy Process, Southern Illinois University Carbondale, Carbondale, IL. DOI http://dx.doi.org/10.1109/TUFFC.2013.2856
0885–3010/$25.00
the best possible spatiotemporal resolution [7]. A superresolution image has particular value in defect diagnosis. It has the potential to easily distinguish defects to enhance cracks, inclusions, voids, delaminations, ablations, and other flaws. The super-resolution image reconstruction technique is an approach to overcome the inherent resolution limitations of an existing ultrasonic system. It attempts to solve the problem with software (a mathematical approach) rather than hardware. Super-resolution reconstruction can greatly improve the image quality and also provide a closer inspection of the ROI with high resolution [8], improving defect evaluation and accuracy. This paper presents an implementation of the iterative backprojection (IBP) method by fusing a set of low-resolution ultrasound images to reconstruct a super-resolution image. For this, a micro-scanning imaging technique is utilized to obtain the low-resolution image set with sub-pixel displacement. A super-resolution ultrasound image with a pixel size of 0.0635 × 0.0635 mm was reconstructed from nine 0.01 × 0.01 in low-resolution images which were obtained from the immersion UT C-scan system. Then, the reconstructed super-resolution result was compared with the bicubic interpolation result of the low-resolution image (to match the super-resolution resolution, which is 4 times larger). The paper is organized as follows: the paper starts with a brief introduction of the basic superresolution theory, IBP algorithms for super-resolution image reconstruction, micro-scanning techniques, computer simulation, and the experimental setup. They are followed by the experimental results and discussions. Finally, conclusions are provided at the end.
II. Super-Resolution Image Reconstruction A. Super-Resolution Theory The super-resolution image reconstruction is a digital signal processing approach which uses several frequency domain aliasing, blurring, and additive noise lower resolution images to obtain high-frequency information and more pixel values to overcome the inherent resolution limitations of the existing imaging system. The concept of super-resolution reconstruction was based on a sequential image first introduced by Tsai and Huang [9]. They implemented the frequency-domain-based approach to reconstruct one improved resolution image from several blurred, down-sampled, and noisy images. After that, var-
© 2013 IEEE
2576
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
l times in both x and y directions; thus, N1 × N2 = lM1 × lM2. The IBP scheme can be expressed as: f n +1(n 1, n 2) = f n(n 1, n 2) +
∑
(g k(m 1, m 2) − g kn(m 1, m 2))
m 1,m 2 m 1,m 2 ∈ γk BP
× h (m 1, m 2; n 1, n 2), (1)
Fig. 1. Basic premise for super-resolution image reconstruction.
ious studies have reported on the use of super-resolution and related topics [10]–[17]. The basic premise of increasing the spatial resolution in super-resolution techniques is to obtain sub-pixel information from multiple low-resolution images [18], as shown in Fig. 1. The circles, triangles, and squares in Fig. 1 indicate sub-pixels from multiple low-resolution images. If these low-resolution images have known sub-pixel shifts from each other and the aliasing is presented correctly, then each image contains unique sub-pixel information which cannot be obtained from the other low-resolution images. In this case, the new information contained in each low-resolution image can be exploited to obtain a high-resolution image.
where f n(n 1, n 2) is the estimated high-resolution image after n iterations; g kn is the kth simulated low-resolution image after n iterations; γ k m 1,m 2 is the location set of lowresolution image influenced by f (n1, n2); gk(m1, m2) − g kn(m 1, m 2) is the backprojection (simulated) error; and hBP(m1, m2; n1, n2) is the backprojection kernel, which determines the contribution of the simulated error to the estimated high-resolution image f n(n 1, n 2). The flowchart diagram of the IBP algorithm is shown in Fig. 3; LR and HR stand for low resolution and high resolution. The backprojection procedure is mainly divided into two steps: 1) The estimated error e kn(m 1, m 2) resolution is interpolated from M1 × M2 to N1 × N2 (expanded l times in both x- and y-directions) by applying the bilinear interpolation method. Accordingly, the E kn(n 1, n 2) is
B. IBP Algorithm Irani and Peleg [11], [12] presented the IBP algorithm in 1990. The IBP approach is similar to the back-projection used in tomography. The IBP algorithm is used to compare several low-resolution images, and decide how to superimpose them so that the pixels would match correctly in the spatial domain and therefore produce the high-resolution image. In this approach, a set of simulated low-resolution frames is generated, and then compared with the observed low-resolution image sequence. The high-resolution image is estimated by backprojecting the error (difference) between the simulated low-resolution images and the observed low-resolution images. The process is repeated iteratively until some stopping criteria are met, such as the minimization of the energy of the error, or the maximum number of allowed iterations. The schematic diagram of the IBP method is shown in Fig. 2. Let gk(m1, m2) be the observed low-resolution image sequence, with the resolution of M1 × M2. The desired resolution N1 × N2 of high-resolution image f(n1, n2) expands
Fig. 2. A pictorial example of the iterative backprojection method [19].
li and chu: super-resolution image reconstruction for ultrasonic nde
2577
Fig. 3. Flowchart diagram of the iterative backprojection method.
obtained because the pixel displacement has (x, y)k = (lx0, ly0)k. 2) The E kn is projected (superimposed) to the current estimated high-resolution image f n based on the displacement (x, y)k accordingly. Because the displacement (xk, yk) = (x, y)k are decimal numbers, each point in E kn will overlay 4 pixels in f n, correspondingly. Let [xk] and [yk] be the integer parts of xk and yk, then x k′ = xk − [xk] and y k′ = yk − [yk] are the decimal parts of xk and yk, respectively. As depicted in Fig. 4, the 4 f n pixels overlaid by E kn(n 1, n 2) are: f n(n 1 + [xk], n2 + [yk]), f n(n 1 + [xk] + 1, n2 + [yk]), f n(n 1 + [xk], n2 + [yk] + 1), f n(n 1 + [xk] + 1, n2 + [yk] +1).
f n(n 1 + [x k ], n 2 + [y k ]) 1 = f n(n 1 + [x k ], n 2 + [y k ]) + E kn(n 1,n 2)(1 − x k′ )(1 − y k′ ) K f n(n 1 + [x k ] + 1, n 2 + [y k ]) 1 = f n(n 1 + [x k ] + 1, n 2 + [y k ]) + E kn(n 1,n 2)x k′(1 − y k′ ) K n f (n 1 + [x k ], n 2 + [y k ] + 1) 1 = f n(n 1 + [x k ], n 2 + [y k ] + 1) + E kn(n 1,n 2)(1 − x k′ )y k′ K f n(n 1 + [x k ] + 1, n 2 + [y k ] + 1) 1 = f n(n 1 + [x k ] + 1, n 2 + [y k ] + 1) + E kn(n 1,n 2)x k′ y k′ , K (2)
2578
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
Fig. 4. A schematic diagram of estimated error backprojection.
where K is the number of low-resolution images. After steps 1 and 2, the new estimated high-resolution image f n +1 is updated by backprojecting the difference [estimated error e kn(m 1, m 2)] between the observed low-resolution image sequence gk(m1, m2) and the estimated lowresolution image sequence g kn(m 1, m 2) to the current estimated high-resolution image f n. The estimated iteration error en between the updated estimated low-resolution image sequence g kn and the observed low-resolution image sequence gk can be expressed as
en =
1 1 K M1 × M1 ×
K −1
∑K =0 ∑m ,m 1
2
(3) (g kn(m 1, m 2) − g kn(m 1, m 2))2 .
The estimated iteration error en is utilized as a criterion for evaluating the quality of the reconstructed highresolution image. If en is smaller than the set error threshold, indicating estimated low-resolution images and observed low-resolution images are similar, which means the quality of the estimated high-resolution image f n is satisfied, iteration ends. Otherwise, iteration will continue until the maximum number of allowed iterations is reached. In this study, the threshold is set as 1.5 × 10−3, and the maximum iteration number is set at 40 to prevent an infinite loop. III. Micro-Scanning Method Micro-scanning is a method to increase the resolution of desired images. In this study, for the optimum performance of the UT image sub-pixel displacement and the computational complexity, a low-resolution image sequence was implemented using the 3 × 3 (9-step) micro-scanning method, as shown in Fig. 5. The black frame indicates the
vol. 60, no. 12,
December
2013
original image, which has 4 pixels (black squares) at each corner, and the red frames are the micro-scanning images with sub-pixel (red squares) displacement. In this study, the UT system has the minimum increment of 0.254 mm on both x- and y-axes, which means the UT image has the minimum dot pitch of 0.254 mm. To break the 0.254 mm dot pitch down to a smaller level and to make the 3 × 3 scan more efficient, the sub-pixel displacement is chosen at 0.0762 mm. Thus, in a 16-pixel image (black frame), besides the original 4 pixels (black) at the corners, the other 12 grid spaces will be filled with pixels (red) obtained from the 3 × 3 micro-scanning, as shown in Fig. 5. As a result, sub-pixels unrevealed from the original scan are now obtained. These sub-pixel displacement images then can be utilized to reconstruct a super-resolution image with the IBP method. During the micro-scanning process, real (measured) pixel values are captured in each scan; no artificial pixel value is generated in a low-resolution image sequence. To keep consistency in the 3 × 3 micro-scanning, no scan parameters were changed other than the required physical displacement between each of the nine scans. Therefore, when these displacements are known within sub-pixel accuracy and combined with the super-resolution algorithm, high-resolution image reconstruction is possible, as illustrated in Fig. 1. IV. Computer Simulation To first evaluate and validate the application of superresolution UT image reconstruction with the IBP algorithm, three artificially simulated UT images were considered. These are cross-sectional images of 3-D ultrasonic data of delaminations in a carbon epoxy laminate. Details on the method used to generate these images were previously described in [19]. The test images are 456 × 432, 464 × 396, and 508 × 492 pixels, and are plotted in 8-bit grayscale images (256 gray levels), as shown in Fig. 6. To simulate the low-resolution image sets obtained by micro-scanning techniques, each defect image (shown in Fig. 6) was subsampled to 16 low-resolution images with resolutions 114 × 108, 116 × 99, and 127 × 123 pixels for defects I, II, and III, respectively. Considering the image quality of the super-resolution reconstruction and the computational complexity, the best balance was found using 9 low-resolution images for the super-resolution image reconstruction. Thus, for this simulation, 9 of the 16 subsampled low-resolution images for each defect were used for super-resolution image reconstruction. For each defect, one of the 16 generated subsampled images is shown in Fig. 7. These subsampled images are scaled to the same size as the simulated UT images only for the purposes of displaying and comparing detail. Then, the IBP method was applied to the subsampled low-resolution images. Fig. 8 shows the super-resolution images reconstructed from 9 low-resolution images for each type of defect, as described earlier. The super-resolu-
li and chu: super-resolution image reconstruction for ultrasonic nde
2579
Fig. 5. Sub-pixel displacement with 3 × 3 micro-scanning method.
tion image resolution is 456 × 432, 464 × 396, and 508 × 492 pixels for defects I, II, and III, respectively. For subjective evaluation, the resolution of super-resolution images (as shown in Fig. 8) is 4 times the individual subsampled images (shown in Fig. 7). Pixel blocking effects are observed from images in Fig. 7 because of the low spatial resolution. In contrast, super-resolution images (shown in Fig. 8) provide almost the same image quality as the simulated UT images (shown in Fig. 6). The defect edges in the super-resolution images are smoother and clearer than the subsampled images. In addition to the perceived image quality with human visual system (HVS), for objective evaluation, peak signal-to-noise ratio (PSNR) and contrast-to-noise ratio (CNR) are employed for quantitative assessment. The super-resolution and bicubic interpolation results are compared with the original simulated ultrasonic image results to test the reconstructed image quality and robustness of the IBP method. The PSNR is given as
MAX 2I , (4) PSNR = 10 ⋅ log 10 MSE
where MAXI is the maximum possible pixel value of the image. In this study, all pixels are represented using 8-bit gray levels; here, MAXI is 255. MSE is the mean squared error between two compared images. The CNR is given as
CNR =
Si − So σ i2 + σ o2
, (5)
where Si and So are the mean values inside and outside the ROI, respectively; σi and σo are the standard deviations inside and outside the ROI, respectively From Table I, it can be verified that the CNR of the super-resolution image is higher than that of the bicubic interpolation image. Further, note that the PSNR value of super-resolution results has an average increase of 5.71 dB compared with the bicubic interpolation method. The IBP
Fig. 6. Simulated ultrasonic testing delamination images: (a) Defect I, 456 × 432 pixels; (b) Defect II, 464 × 396 pixels; and (c) Defect III, 508 × 492 pixels.
2580
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
Fig. 7. One of the 16 generated subsampled images for each defect: (a) Defect I, 114 × 108 pixels; (b) Defect II, 116 × 99 pixels; and (c) Defect III, 127 × 123 pixels.
method provides better performance in PSNR, indicating the quality of the reconstructed super-resolution image is close to the original simulated UT image, which can verify that the IBP method is robust and reliable. Because the applied method provides good performance in both CNR and PSNR, it can be seen that the proposed superresolution reconstruction method is effective in resolution enhancement. V. Experimental Setup A. Instrument The immersion ultrasonic testing system with associated instrumentation used to inspect the specimens is shown in Fig. 9(a). A 5-MHz dual-element Panametrics transducer (Panametrics, Waltham, MA) with a 50.8 mm focal length was utilized in a pulse–echo mode for the inspection. The standoff distance between the transducer and the panel was set to 2 in and the scan was conducted at an increment of 0.01 in. As depicted in Fig. 9(b), an X-Y table with microscanning technique was utilized to obtain the sub-pixel
displacement. The step size of the X-Y table is 1/1000 inches. B. Specimens To verify the application of super-resolution image reconstruction for ultrasonic imaging, two CFRP panels with predefined internal defects, i.e., delamination and porosity were considered. Specimen A, a 102 × 257 × 4.45 mm CFRP panel, consists of delamination at three different locations. The delamination defects in specimen A are artificially simulated by impacting the panel with an external object of known energy. These defects are difficult to recognize by visual inspection, but have severely progressed within the panel. The ultrasound image and defect map of the specimen are shown in Fig. 10. Specimen B is a carbon epoxy laminate with simulated porosity defects, as shown in Fig. 11. This panel has dimensions of 200 × 300 × 3.36 mm. The specimen has five porosity defects of varied diameters (15 mm and 30 mm). These defects are artificially created by controlling the pressure during the curing process. The range of porosity is from 1% to 5%.
Fig. 8. Reconstructed super-resolution image from 9 low-resolution images: (a) Defect I, 456 × 432 pixels; (b) Defect II, 464 × 396 pixels; and (c) Defect III, 508 × 492 pixels.
li and chu: super-resolution image reconstruction for ultrasonic nde
2581
TABLE I. Experimental Comparison of Contrast-to-Noise Ratio (CNR) and the Peak Signal-to-Noise Ratio (PSNR). Region of interest Defect I Defect II Defect III
Bicubic interpolation IBP super-resolution result Bicubic interpolation IBP super-resolution result Bicubic interpolation IBP super-resolution result
The raw C-scan results of specimen A and B are plotted in Figs. 10(a) and 11(a), respectively, where defect areas are represented by dark shades of gray, caused by a significant drop in pulse–echo signal amplitude. VI. Results and Discussion The super-resolution IBP algorithm, as described earlier, was applied to the ultrasound image sequence (C-scan back wall amplitude data) obtained from 3 × 3 microscanning method to verify the versatility and stability of the super-resolution image reconstruction. The proposed method was implemented in Matlab R2012a (The MathWorks Inc., Natick, MA). The ultrasonic C-scan image of specimen A has 361 × 961 pixels. To reduce the computation complexity and processing time, only defect area images are utilized for super-resolution image reconstruction. The C-scan amplitude data of ROIs are normalized and plotted in 8-bit grayscale images (256 gray levels), as shown in Fig. 12. Images in Fig. 12 are obtained by the micro-scanning method with the displacement of 0.0762 mm, approximately 1/3 pixel of the raw low-resolution image. Because of the limitation of space constraints, only 5 of 9 images are shown for each defect. The resolution of defects 1,
CNR (dB)
PSNR (dB)
0.95 1.10 0.59 0.61 1.85 1.97
19.44 23.95 18.36 24.27 24.82 29.80
2, and 3 are 108 × 111, 97 × 99, and 140 × 119 pixels, respectively. Before the reconstruction process, a 3 × 3 Wiener filter was applied on low-resolution images to reduce the noise. For subjective evaluation, the super-resolution outputs of ROIs are shown in Fig. 13. Images (a), (d), and (g) are the original low-resolution images (reference g0) of each ROI. To better compare with super-resolution results, these images are magnified 4 times with the bicubic interpolation method. Corresponding to the interpolated low-resolution images (a), (d), and (g) on the left side, super-resolution results are presented in (c), (f), and (i). Detailed comparisons of both low-resolution and superresolution images for each defect are amplified and presented in (b), (e), and (h). From the results obtained in Fig. 13, the comparison indicates that super-resolution images (c), (f), and (i) provide better visual quality. The super-resolution method is able to detect the defects with more fidelity by enhancing the defect outline compared with the low-resolution C-scan images. For defect 1, as shown in Fig. 13(c), the defect outline in super-resolution images is more distinct, allowing post-processing work to be easier. For defect 2, as depicted in Fig. 13(f), the super-resolution method is able to recover the defect missed in the low-resolution image. For defect 3, color images are used to reconstruct the
Fig. 9. (a) Immersion ultrasonic testing system and (b) micro-scanning setup.
2582
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
Fig. 10. (a) Ultrasound image and (b) defect map of specimen A. All dimensions are in millimeters.
Fig. 11. (a) Ultrasound image and (b) defect map of specimen B. All dimensions are in millimeters.
Fig. 12. Low-resolution (LR) images of regions of interest (ROIs) in specimen A.
vol. 60, no. 12,
December
2013
li and chu: super-resolution image reconstruction for ultrasonic nde
2583
Fig. 13. Comparison of low-resolution and super-resolution images (SR): (a) 4× magnification of defect 1, (b) detail comparison of defect 1, and (c) SR image of defect 1; (d) 4× magnification of defect 2, (e) detail comparison of defect 2, and (f) SR image of defect 2; (g) 4× magnification of defect 3, (h) detail comparison of defect 3, and (i) SR image of defect 3.
super-resolution image. The upper images in (h) indicate that the missed delamination areas (red spot) in the lowresolution image are successfully recovered in super-resolution images. Similarly, the lower images in (h) show the enhanced delamination area which failed to be detected in low-resolution image (g). For objective evaluation, PSNR and CNR are used for quantitative assessment. From Table II, it can be verified that the CNR of the super-resolution image is greater than the bicubic interpolation image. Further, note that the PSNR values of the 3 defects are close to each other, which indicates that the quality of the reconstructed super-resolution image is robust and reliable. Because the proposed method provides good performance in both CNR and PSNR, we can observe that the proposed super-resolution reconstruction method is effective in resolution enhancement.
To demonstrate the feasibility of implementing the IBP method on the porosity specimen, ultrasonic C-scan images of specimen B defect 2 was considered. The image in Fig. 14(a) is one of the 9 images obtained by the micro-scanning method with the displacement of 0.0762 mm (approximately 1/3 pixel of the UT image). The resolution of the obtained defect 2 UT image is 180 × 180 pixels. As for specimen A, a 3 × 3 Wiener filter was applied to specimen B C-scan images to reduce the noise. From the super-resolution result obtained in Fig. 14(b), comparisons indicate that the super-resolution image shows better visual quality. The reconstructed superresolution image’s resolution is increased to 720 × 720 pixels, and it has a more distinctly recognizable defect outline than the low-resolution C-scan image. The Canny edge detector was utilized for the objective evaluation of porosity UT images, see Fig. 15. The same
TABLE II. Specimen A Experimental Comparison of Contrast-to-Noise Ratio (CNR) and the Peak Signal-to-Noise Ratio (PSNR). Region of interest Defect 1 Defect 2 Defect 3
Bicubic interpolation method IBP method Bicubic interpolation method IBP method Bicubic interpolation method IBP method
CNR (dB)
PSNR (dB)
15.40 18.53 10.47 12.18 5.89 6.01
22.91 24.49 24.01
2584
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
than the original images while maintaining the same scanning time; or an image with the same resolution can be achieved in a quarter of the original scanning time. Therefore, the super-resolution method has particular value in NDE defect diagnosis. Acknowledgments
Fig. 14. Comparison of (a) one of the micro-scan ultrasonic testing images and (b) the resonstructed super-resolution image.
parameters (threshold low: 10, threshold high: 20, and smooth filter size: 17) used in the detector were applied to both the low-resolution and super-resolution images. VII. Conclusions High-resolution UT images of ROIs are valuable for defect size, shape, and location measurements. In this study, a super-resolution image reconstruction methodology based on IBP and micro-scanning was applied to improve the resolution of UT C-scan images. The experimental results obtained have demonstrated the applied method is capable of providing better visual quality than the raw UT images. The resolution of the super-resolution result was increased 4 times over the raw C-scan image. The super-resolution method greatly improved image quality and also allowed for more detailed inspection of the ROIs with high resolution. Super-resolution image reconstruction can also be utilized to overcome the inherent limitations of the existing ultrasonic C-scan system. Because the C-scan time is proportional to the desired resolution, to double the resolution in both x- and y-directions, 4 times the original scanning time may be required. If a C-scan system was equipped with multiple ultrasound transducers with known displacement from each other, the system can offer a super-resolution result resolution 4 times larger
Fig. 15. Canny edge detector result comparison of (a) low-resolution micro-scan ultrasonic testing image and (b) the corresponding superresolution image.
The authors thank the Center for Advanced Friction Studies (CAFS), Southern Illinois University Carbondale, Carbondale, IL, directed by Dr. P. Fillip, for partially supporting this project. We also thank Mr. A. Poudel and Mr. C. McGee for providing their assistance in performing the testing. References [1] K. H. Im, D. K. Hsu, and I. Y. Yang, “Inspection of inhomogeneities in carbon/phenolic matrix composite materials using NDE techniques,” Key Eng. Mater., vol. 270, no. 1, pp. 1799–1805, 2004. [2] J. H. Lee, S. W. Choi, K. S. Kim, and J. H. Park, “Nondestructive characterisation of carbon/carbon brake disks using ultrasonics,” in 10th Asia-Pacific Conf. Non-Destructive Testing, 2001, pp. 1109–1112. [3] A. Ruosi, “Nondestructive detection of damage in carbon fiber composites by SQUID magnetometry,” Phys. Status Solidi C, vol. 2, no. 5, pp. 1533–1555, 2005. [4] D. E. Bray and D. McBride, Nondestructive Testing Techniques, New York, NY: Wiley, 1992. [5] H. Zhang, M. Wan, J. Wan, and X. Qin, “Super-resolution reconstruction of deformable tissue from temporal sequence of ultrasound images,” in Int. Conf. Artificial Intelligence and Computational Intelligence, 2010, vol. 1, pp. 337–342. [6] D. Kouame and M. Ploquin, ” Super-resolution in medical imaging : An illustrative approach through ultrasound,” in IEEE Int. Symp. Biomedical Imaging: From Nano to Macro, 2009, pp. 249–252. [7] G. Clark and J. Jackson, “Super-resolution algorithms for ultrasonic nondestructive evaluation imagery,” J. Acoust. Soc. Am., vol. 120, no. 5, p. 3140, 2006. [8] Y. Dai, B. Wang, and D. Liu, “A fast and robust super resolution method for intima reconstruction in medical ultrasound,” in 3rd Int. Conf. Bioinformatics and Biomedical Engineering, 2009, pp. 1–4. [9] R. Y. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vision Image Process., vol. 1, no. 2, pp. 101–106, 1984. [10] H. Stark and P. Oskoui, “High resolution image recovery from image plane arrays, using convex projections,” J. Opt. Soc. Am., vol. 6, no. 11, pp. 1715–1726, 1989. [11] M. Irani and S. Peleg, “Super resolution from image sequences,” in Proc. 10th Int. Conf. Pattern Recognition, 1990, vol. 112, pp. 115–120. [12] M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP Graph. Models Image Process., vol. 53, no. 3, pp. 231–239, 1991. [13] R. R. Schultz and R. L. Stevenson, “Improved definition video frame enhancement,” in Int. Conf. Acoustics, Speech, and Signal Processing Conf. Proc., 1995, vol. 4, pp. 2169–2172. [14] R. R. Schultz and R. L. Stevenson, “Video resolution enhancement,” in Proc. Conf. Image and Video Processing III, 1995, pp. 23–34. [15] R. R. Schultz and R. L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process., vol. 5, no. 6, pp. 996–1011, 1996. [16] R. R. Schultz and R. L. Stevenson, “Motion-compensated scan conversion of interlaced video sequences,” in Proc. Conf. Image and Video Processing IV, 1996, pp. 107–118. [17] M. K. Ng and N. K. Bose, “Mathamatical analysis of super-resolution methodology,” IEEE Signal Process. Mag., vol. 20, no. 3, pp. 62–74, 2003.
li and chu: super-resolution image reconstruction for ultrasonic nde [18] S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21–36, 2003. [19] S. Li and T. P. Chu, “Ultrasonic 3D reconstruction of CFRP panel delamination,” in Proc. ASNT Fall Conf., 2012.
Shanglei Li received the B.S. degree in electrical engineering from Changchun University of Science and Technology, Changchun, P. R. China, in 2006, and the M.S. degree in electrical engineering from California State University, Fullerton, in 2009. He is currently a Ph.D. candidate in the Electrical and Computer Engineering Department, Southern Illinois University, Carbondale. He has been working with Dr. Tsuchin Chu at the Intelligent Measurement & Evaluation Laboratory (IMEL) and conducting research work in nondestructive evaluation (NDE) since 2010. His research focuses on the development of an intelligent NDE expert system for carbon-based composites used for aircraft and aerospace applications by using ultrasonic (conventional, immersion, and air-coupled) and infrared thermography methods. His main tasks are NDE data analysis and image processing utilizing fuzzy logic, artificial neural networks (ANNs), signal processing, superresolution image reconstruction, and other intelligent algorithms. Mr. Li is highly motivated in doing NDE-related research work and possesses the capability to carry out his research work independently.
2585
Tsuchin Philip Chu received the B.E. degree in mechanical engineering from the National Cheng Kung University, Taiwan, R.O.C., in 1974; the M.S. degree in mechanical engineering from Aubrn University, Auburn, AL, in 1980; and the Ph.D. degree in mechanical engineering from the University of South Carolina, Columbia, SC, in 1982. He is currently a senior Professor in the Department of Mechanical Engineering and Energy Processes at Southern Illinois University, Carbondale (SIU). He has conducted research for more than 30 years in nondestructive evaluation (NDE), biomedical engineering, experimental mechanics, computer-aided design, manufacturing, and engineering (CAD/ CAM/CAE), finite element analysis (FEA), sensors, and instrumentation. He is a pioneer in the area of digital image correlation (DIC) and at the cutting edge of research in NDE and biomechanics. He was a faculty member of Polytechnic University in New York before he joined SIU in 1990. He has more than 80 peer-reviewed journal publications and conference proceedings and more than $2 million in grants from NASA, Boeing, the US Air Force, IBM, the Illinois Clean Coal Institute, etc. He developed the Intelligent Measurement and Evaluation Lab (IMEL), which houses state-of-the-art equipment including a DIC system and an infrared (IR) thermography system, as well as immersion, contact, and air-coupled ultrasonic C-scan systems. He has advised more than 40 graduate students. He is currently serving as the council director of the ASNT (American Society of Nondestructive Testing) and the ASNT St. Louis Section Board of Directors. He is also an associate editor for the professional journal Experimental Techniques. Dr. Chu is a co-founder of Clipius Technologies, a think-tank company producing intellectual property in the areas of defense, aerospace, and biomedical devices.
vol. 60, no. 12,
December
2013
2575
Super-Resolution Image Reconstruction for Ultrasonic Nondestructive Evaluation Shanglei Li and Tsuchin Philip Chu Abstract—Ultrasonic testing is one of the most successful nondestructive evaluation (NDE) techniques for the inspection of carbon-fiber-reinforced polymer (CFRP) materials. This paper discusses the application of the iterative backprojection (IBP) super-resolution image reconstruction technique to carbon epoxy laminates with simulated defects to obtain high-resolution images for NDE. Super-resolution image reconstruction is an approach used to overcome the inherent resolution limitations of an existing ultrasonic system. It can greatly improve the image quality and allow more detailed inspection of the region of interest (ROI) with high resolution, improving defect evaluation and accuracy. First, three artificially simulated delamination defects in a CFRP panel were considered to evaluate and validate the application of the IBP method. The results of the validation indicate that both the contrast-tonoise ratio (CNR) and the peak signal-to-noise ratio (PSNR) value of the super-resolution result are better than the bicubic interpolation method. Then, the IBP method was applied to the low-resolution ultrasonic C-scan image sequence with subpixel displacement of two types of defects (delamination and porosity) which were obtained by the micro-scanning imaging technique. The result demonstrated that super-resolution images achieved better visual quality with an improved image resolution compared with raw C-scan images.
I. Introduction
W
ith the extensive use of carbon composite material in the aviation industry, ultrasonic testing (UT) has become one of the proven nondestructive evaluation (NDE) techniques, featuring large scale, high speed, and noncontact testing capabilities [1]–[4]. However, UT Cscan contrast and detection of defects in carbon composites is limited by the inhomogeneous density and the anisotropic properties of the composite material. Moreover, the C-scan image resolution depends on the physical characteristics of the ultrasound transducer (sensor); thus, increasing the resolution by UT sensor modification may not be a viable option. In addition, ultrasonic image quality can be affected by the speckle noise produced by the interference of backscattered signals [5], [6]. This makes it difficult for ultrasonic C-scan inspection to qualitatively and quantitatively evaluate defects’ size, shape, and type. A key NDE goal is to obtain images having
Manuscript received May 17, 2013; accepted September 10, 2013. S. Li is with the Department of Electrical and Computer Engineering, Southern Illinois University Carbondale, Carbondale, IL (e-mail: [email protected]). T. P. Chu is with the Department of Mechanical Engineering and Energy Process, Southern Illinois University Carbondale, Carbondale, IL. DOI http://dx.doi.org/10.1109/TUFFC.2013.2856
0885–3010/$25.00
the best possible spatiotemporal resolution [7]. A superresolution image has particular value in defect diagnosis. It has the potential to easily distinguish defects to enhance cracks, inclusions, voids, delaminations, ablations, and other flaws. The super-resolution image reconstruction technique is an approach to overcome the inherent resolution limitations of an existing ultrasonic system. It attempts to solve the problem with software (a mathematical approach) rather than hardware. Super-resolution reconstruction can greatly improve the image quality and also provide a closer inspection of the ROI with high resolution [8], improving defect evaluation and accuracy. This paper presents an implementation of the iterative backprojection (IBP) method by fusing a set of low-resolution ultrasound images to reconstruct a super-resolution image. For this, a micro-scanning imaging technique is utilized to obtain the low-resolution image set with sub-pixel displacement. A super-resolution ultrasound image with a pixel size of 0.0635 × 0.0635 mm was reconstructed from nine 0.01 × 0.01 in low-resolution images which were obtained from the immersion UT C-scan system. Then, the reconstructed super-resolution result was compared with the bicubic interpolation result of the low-resolution image (to match the super-resolution resolution, which is 4 times larger). The paper is organized as follows: the paper starts with a brief introduction of the basic superresolution theory, IBP algorithms for super-resolution image reconstruction, micro-scanning techniques, computer simulation, and the experimental setup. They are followed by the experimental results and discussions. Finally, conclusions are provided at the end.
II. Super-Resolution Image Reconstruction A. Super-Resolution Theory The super-resolution image reconstruction is a digital signal processing approach which uses several frequency domain aliasing, blurring, and additive noise lower resolution images to obtain high-frequency information and more pixel values to overcome the inherent resolution limitations of the existing imaging system. The concept of super-resolution reconstruction was based on a sequential image first introduced by Tsai and Huang [9]. They implemented the frequency-domain-based approach to reconstruct one improved resolution image from several blurred, down-sampled, and noisy images. After that, var-
© 2013 IEEE
2576
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
l times in both x and y directions; thus, N1 × N2 = lM1 × lM2. The IBP scheme can be expressed as: f n +1(n 1, n 2) = f n(n 1, n 2) +
∑
(g k(m 1, m 2) − g kn(m 1, m 2))
m 1,m 2 m 1,m 2 ∈ γk BP
× h (m 1, m 2; n 1, n 2), (1)
Fig. 1. Basic premise for super-resolution image reconstruction.
ious studies have reported on the use of super-resolution and related topics [10]–[17]. The basic premise of increasing the spatial resolution in super-resolution techniques is to obtain sub-pixel information from multiple low-resolution images [18], as shown in Fig. 1. The circles, triangles, and squares in Fig. 1 indicate sub-pixels from multiple low-resolution images. If these low-resolution images have known sub-pixel shifts from each other and the aliasing is presented correctly, then each image contains unique sub-pixel information which cannot be obtained from the other low-resolution images. In this case, the new information contained in each low-resolution image can be exploited to obtain a high-resolution image.
where f n(n 1, n 2) is the estimated high-resolution image after n iterations; g kn is the kth simulated low-resolution image after n iterations; γ k m 1,m 2 is the location set of lowresolution image influenced by f (n1, n2); gk(m1, m2) − g kn(m 1, m 2) is the backprojection (simulated) error; and hBP(m1, m2; n1, n2) is the backprojection kernel, which determines the contribution of the simulated error to the estimated high-resolution image f n(n 1, n 2). The flowchart diagram of the IBP algorithm is shown in Fig. 3; LR and HR stand for low resolution and high resolution. The backprojection procedure is mainly divided into two steps: 1) The estimated error e kn(m 1, m 2) resolution is interpolated from M1 × M2 to N1 × N2 (expanded l times in both x- and y-directions) by applying the bilinear interpolation method. Accordingly, the E kn(n 1, n 2) is
B. IBP Algorithm Irani and Peleg [11], [12] presented the IBP algorithm in 1990. The IBP approach is similar to the back-projection used in tomography. The IBP algorithm is used to compare several low-resolution images, and decide how to superimpose them so that the pixels would match correctly in the spatial domain and therefore produce the high-resolution image. In this approach, a set of simulated low-resolution frames is generated, and then compared with the observed low-resolution image sequence. The high-resolution image is estimated by backprojecting the error (difference) between the simulated low-resolution images and the observed low-resolution images. The process is repeated iteratively until some stopping criteria are met, such as the minimization of the energy of the error, or the maximum number of allowed iterations. The schematic diagram of the IBP method is shown in Fig. 2. Let gk(m1, m2) be the observed low-resolution image sequence, with the resolution of M1 × M2. The desired resolution N1 × N2 of high-resolution image f(n1, n2) expands
Fig. 2. A pictorial example of the iterative backprojection method [19].
li and chu: super-resolution image reconstruction for ultrasonic nde
2577
Fig. 3. Flowchart diagram of the iterative backprojection method.
obtained because the pixel displacement has (x, y)k = (lx0, ly0)k. 2) The E kn is projected (superimposed) to the current estimated high-resolution image f n based on the displacement (x, y)k accordingly. Because the displacement (xk, yk) = (x, y)k are decimal numbers, each point in E kn will overlay 4 pixels in f n, correspondingly. Let [xk] and [yk] be the integer parts of xk and yk, then x k′ = xk − [xk] and y k′ = yk − [yk] are the decimal parts of xk and yk, respectively. As depicted in Fig. 4, the 4 f n pixels overlaid by E kn(n 1, n 2) are: f n(n 1 + [xk], n2 + [yk]), f n(n 1 + [xk] + 1, n2 + [yk]), f n(n 1 + [xk], n2 + [yk] + 1), f n(n 1 + [xk] + 1, n2 + [yk] +1).
f n(n 1 + [x k ], n 2 + [y k ]) 1 = f n(n 1 + [x k ], n 2 + [y k ]) + E kn(n 1,n 2)(1 − x k′ )(1 − y k′ ) K f n(n 1 + [x k ] + 1, n 2 + [y k ]) 1 = f n(n 1 + [x k ] + 1, n 2 + [y k ]) + E kn(n 1,n 2)x k′(1 − y k′ ) K n f (n 1 + [x k ], n 2 + [y k ] + 1) 1 = f n(n 1 + [x k ], n 2 + [y k ] + 1) + E kn(n 1,n 2)(1 − x k′ )y k′ K f n(n 1 + [x k ] + 1, n 2 + [y k ] + 1) 1 = f n(n 1 + [x k ] + 1, n 2 + [y k ] + 1) + E kn(n 1,n 2)x k′ y k′ , K (2)
2578
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
Fig. 4. A schematic diagram of estimated error backprojection.
where K is the number of low-resolution images. After steps 1 and 2, the new estimated high-resolution image f n +1 is updated by backprojecting the difference [estimated error e kn(m 1, m 2)] between the observed low-resolution image sequence gk(m1, m2) and the estimated lowresolution image sequence g kn(m 1, m 2) to the current estimated high-resolution image f n. The estimated iteration error en between the updated estimated low-resolution image sequence g kn and the observed low-resolution image sequence gk can be expressed as
en =
1 1 K M1 × M1 ×
K −1
∑K =0 ∑m ,m 1
2
(3) (g kn(m 1, m 2) − g kn(m 1, m 2))2 .
The estimated iteration error en is utilized as a criterion for evaluating the quality of the reconstructed highresolution image. If en is smaller than the set error threshold, indicating estimated low-resolution images and observed low-resolution images are similar, which means the quality of the estimated high-resolution image f n is satisfied, iteration ends. Otherwise, iteration will continue until the maximum number of allowed iterations is reached. In this study, the threshold is set as 1.5 × 10−3, and the maximum iteration number is set at 40 to prevent an infinite loop. III. Micro-Scanning Method Micro-scanning is a method to increase the resolution of desired images. In this study, for the optimum performance of the UT image sub-pixel displacement and the computational complexity, a low-resolution image sequence was implemented using the 3 × 3 (9-step) micro-scanning method, as shown in Fig. 5. The black frame indicates the
vol. 60, no. 12,
December
2013
original image, which has 4 pixels (black squares) at each corner, and the red frames are the micro-scanning images with sub-pixel (red squares) displacement. In this study, the UT system has the minimum increment of 0.254 mm on both x- and y-axes, which means the UT image has the minimum dot pitch of 0.254 mm. To break the 0.254 mm dot pitch down to a smaller level and to make the 3 × 3 scan more efficient, the sub-pixel displacement is chosen at 0.0762 mm. Thus, in a 16-pixel image (black frame), besides the original 4 pixels (black) at the corners, the other 12 grid spaces will be filled with pixels (red) obtained from the 3 × 3 micro-scanning, as shown in Fig. 5. As a result, sub-pixels unrevealed from the original scan are now obtained. These sub-pixel displacement images then can be utilized to reconstruct a super-resolution image with the IBP method. During the micro-scanning process, real (measured) pixel values are captured in each scan; no artificial pixel value is generated in a low-resolution image sequence. To keep consistency in the 3 × 3 micro-scanning, no scan parameters were changed other than the required physical displacement between each of the nine scans. Therefore, when these displacements are known within sub-pixel accuracy and combined with the super-resolution algorithm, high-resolution image reconstruction is possible, as illustrated in Fig. 1. IV. Computer Simulation To first evaluate and validate the application of superresolution UT image reconstruction with the IBP algorithm, three artificially simulated UT images were considered. These are cross-sectional images of 3-D ultrasonic data of delaminations in a carbon epoxy laminate. Details on the method used to generate these images were previously described in [19]. The test images are 456 × 432, 464 × 396, and 508 × 492 pixels, and are plotted in 8-bit grayscale images (256 gray levels), as shown in Fig. 6. To simulate the low-resolution image sets obtained by micro-scanning techniques, each defect image (shown in Fig. 6) was subsampled to 16 low-resolution images with resolutions 114 × 108, 116 × 99, and 127 × 123 pixels for defects I, II, and III, respectively. Considering the image quality of the super-resolution reconstruction and the computational complexity, the best balance was found using 9 low-resolution images for the super-resolution image reconstruction. Thus, for this simulation, 9 of the 16 subsampled low-resolution images for each defect were used for super-resolution image reconstruction. For each defect, one of the 16 generated subsampled images is shown in Fig. 7. These subsampled images are scaled to the same size as the simulated UT images only for the purposes of displaying and comparing detail. Then, the IBP method was applied to the subsampled low-resolution images. Fig. 8 shows the super-resolution images reconstructed from 9 low-resolution images for each type of defect, as described earlier. The super-resolu-
li and chu: super-resolution image reconstruction for ultrasonic nde
2579
Fig. 5. Sub-pixel displacement with 3 × 3 micro-scanning method.
tion image resolution is 456 × 432, 464 × 396, and 508 × 492 pixels for defects I, II, and III, respectively. For subjective evaluation, the resolution of super-resolution images (as shown in Fig. 8) is 4 times the individual subsampled images (shown in Fig. 7). Pixel blocking effects are observed from images in Fig. 7 because of the low spatial resolution. In contrast, super-resolution images (shown in Fig. 8) provide almost the same image quality as the simulated UT images (shown in Fig. 6). The defect edges in the super-resolution images are smoother and clearer than the subsampled images. In addition to the perceived image quality with human visual system (HVS), for objective evaluation, peak signal-to-noise ratio (PSNR) and contrast-to-noise ratio (CNR) are employed for quantitative assessment. The super-resolution and bicubic interpolation results are compared with the original simulated ultrasonic image results to test the reconstructed image quality and robustness of the IBP method. The PSNR is given as
MAX 2I , (4) PSNR = 10 ⋅ log 10 MSE
where MAXI is the maximum possible pixel value of the image. In this study, all pixels are represented using 8-bit gray levels; here, MAXI is 255. MSE is the mean squared error between two compared images. The CNR is given as
CNR =
Si − So σ i2 + σ o2
, (5)
where Si and So are the mean values inside and outside the ROI, respectively; σi and σo are the standard deviations inside and outside the ROI, respectively From Table I, it can be verified that the CNR of the super-resolution image is higher than that of the bicubic interpolation image. Further, note that the PSNR value of super-resolution results has an average increase of 5.71 dB compared with the bicubic interpolation method. The IBP
Fig. 6. Simulated ultrasonic testing delamination images: (a) Defect I, 456 × 432 pixels; (b) Defect II, 464 × 396 pixels; and (c) Defect III, 508 × 492 pixels.
2580
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
Fig. 7. One of the 16 generated subsampled images for each defect: (a) Defect I, 114 × 108 pixels; (b) Defect II, 116 × 99 pixels; and (c) Defect III, 127 × 123 pixels.
method provides better performance in PSNR, indicating the quality of the reconstructed super-resolution image is close to the original simulated UT image, which can verify that the IBP method is robust and reliable. Because the applied method provides good performance in both CNR and PSNR, it can be seen that the proposed superresolution reconstruction method is effective in resolution enhancement. V. Experimental Setup A. Instrument The immersion ultrasonic testing system with associated instrumentation used to inspect the specimens is shown in Fig. 9(a). A 5-MHz dual-element Panametrics transducer (Panametrics, Waltham, MA) with a 50.8 mm focal length was utilized in a pulse–echo mode for the inspection. The standoff distance between the transducer and the panel was set to 2 in and the scan was conducted at an increment of 0.01 in. As depicted in Fig. 9(b), an X-Y table with microscanning technique was utilized to obtain the sub-pixel
displacement. The step size of the X-Y table is 1/1000 inches. B. Specimens To verify the application of super-resolution image reconstruction for ultrasonic imaging, two CFRP panels with predefined internal defects, i.e., delamination and porosity were considered. Specimen A, a 102 × 257 × 4.45 mm CFRP panel, consists of delamination at three different locations. The delamination defects in specimen A are artificially simulated by impacting the panel with an external object of known energy. These defects are difficult to recognize by visual inspection, but have severely progressed within the panel. The ultrasound image and defect map of the specimen are shown in Fig. 10. Specimen B is a carbon epoxy laminate with simulated porosity defects, as shown in Fig. 11. This panel has dimensions of 200 × 300 × 3.36 mm. The specimen has five porosity defects of varied diameters (15 mm and 30 mm). These defects are artificially created by controlling the pressure during the curing process. The range of porosity is from 1% to 5%.
Fig. 8. Reconstructed super-resolution image from 9 low-resolution images: (a) Defect I, 456 × 432 pixels; (b) Defect II, 464 × 396 pixels; and (c) Defect III, 508 × 492 pixels.
li and chu: super-resolution image reconstruction for ultrasonic nde
2581
TABLE I. Experimental Comparison of Contrast-to-Noise Ratio (CNR) and the Peak Signal-to-Noise Ratio (PSNR). Region of interest Defect I Defect II Defect III
Bicubic interpolation IBP super-resolution result Bicubic interpolation IBP super-resolution result Bicubic interpolation IBP super-resolution result
The raw C-scan results of specimen A and B are plotted in Figs. 10(a) and 11(a), respectively, where defect areas are represented by dark shades of gray, caused by a significant drop in pulse–echo signal amplitude. VI. Results and Discussion The super-resolution IBP algorithm, as described earlier, was applied to the ultrasound image sequence (C-scan back wall amplitude data) obtained from 3 × 3 microscanning method to verify the versatility and stability of the super-resolution image reconstruction. The proposed method was implemented in Matlab R2012a (The MathWorks Inc., Natick, MA). The ultrasonic C-scan image of specimen A has 361 × 961 pixels. To reduce the computation complexity and processing time, only defect area images are utilized for super-resolution image reconstruction. The C-scan amplitude data of ROIs are normalized and plotted in 8-bit grayscale images (256 gray levels), as shown in Fig. 12. Images in Fig. 12 are obtained by the micro-scanning method with the displacement of 0.0762 mm, approximately 1/3 pixel of the raw low-resolution image. Because of the limitation of space constraints, only 5 of 9 images are shown for each defect. The resolution of defects 1,
CNR (dB)
PSNR (dB)
0.95 1.10 0.59 0.61 1.85 1.97
19.44 23.95 18.36 24.27 24.82 29.80
2, and 3 are 108 × 111, 97 × 99, and 140 × 119 pixels, respectively. Before the reconstruction process, a 3 × 3 Wiener filter was applied on low-resolution images to reduce the noise. For subjective evaluation, the super-resolution outputs of ROIs are shown in Fig. 13. Images (a), (d), and (g) are the original low-resolution images (reference g0) of each ROI. To better compare with super-resolution results, these images are magnified 4 times with the bicubic interpolation method. Corresponding to the interpolated low-resolution images (a), (d), and (g) on the left side, super-resolution results are presented in (c), (f), and (i). Detailed comparisons of both low-resolution and superresolution images for each defect are amplified and presented in (b), (e), and (h). From the results obtained in Fig. 13, the comparison indicates that super-resolution images (c), (f), and (i) provide better visual quality. The super-resolution method is able to detect the defects with more fidelity by enhancing the defect outline compared with the low-resolution C-scan images. For defect 1, as shown in Fig. 13(c), the defect outline in super-resolution images is more distinct, allowing post-processing work to be easier. For defect 2, as depicted in Fig. 13(f), the super-resolution method is able to recover the defect missed in the low-resolution image. For defect 3, color images are used to reconstruct the
Fig. 9. (a) Immersion ultrasonic testing system and (b) micro-scanning setup.
2582
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
Fig. 10. (a) Ultrasound image and (b) defect map of specimen A. All dimensions are in millimeters.
Fig. 11. (a) Ultrasound image and (b) defect map of specimen B. All dimensions are in millimeters.
Fig. 12. Low-resolution (LR) images of regions of interest (ROIs) in specimen A.
vol. 60, no. 12,
December
2013
li and chu: super-resolution image reconstruction for ultrasonic nde
2583
Fig. 13. Comparison of low-resolution and super-resolution images (SR): (a) 4× magnification of defect 1, (b) detail comparison of defect 1, and (c) SR image of defect 1; (d) 4× magnification of defect 2, (e) detail comparison of defect 2, and (f) SR image of defect 2; (g) 4× magnification of defect 3, (h) detail comparison of defect 3, and (i) SR image of defect 3.
super-resolution image. The upper images in (h) indicate that the missed delamination areas (red spot) in the lowresolution image are successfully recovered in super-resolution images. Similarly, the lower images in (h) show the enhanced delamination area which failed to be detected in low-resolution image (g). For objective evaluation, PSNR and CNR are used for quantitative assessment. From Table II, it can be verified that the CNR of the super-resolution image is greater than the bicubic interpolation image. Further, note that the PSNR values of the 3 defects are close to each other, which indicates that the quality of the reconstructed super-resolution image is robust and reliable. Because the proposed method provides good performance in both CNR and PSNR, we can observe that the proposed super-resolution reconstruction method is effective in resolution enhancement.
To demonstrate the feasibility of implementing the IBP method on the porosity specimen, ultrasonic C-scan images of specimen B defect 2 was considered. The image in Fig. 14(a) is one of the 9 images obtained by the micro-scanning method with the displacement of 0.0762 mm (approximately 1/3 pixel of the UT image). The resolution of the obtained defect 2 UT image is 180 × 180 pixels. As for specimen A, a 3 × 3 Wiener filter was applied to specimen B C-scan images to reduce the noise. From the super-resolution result obtained in Fig. 14(b), comparisons indicate that the super-resolution image shows better visual quality. The reconstructed superresolution image’s resolution is increased to 720 × 720 pixels, and it has a more distinctly recognizable defect outline than the low-resolution C-scan image. The Canny edge detector was utilized for the objective evaluation of porosity UT images, see Fig. 15. The same
TABLE II. Specimen A Experimental Comparison of Contrast-to-Noise Ratio (CNR) and the Peak Signal-to-Noise Ratio (PSNR). Region of interest Defect 1 Defect 2 Defect 3
Bicubic interpolation method IBP method Bicubic interpolation method IBP method Bicubic interpolation method IBP method
CNR (dB)
PSNR (dB)
15.40 18.53 10.47 12.18 5.89 6.01
22.91 24.49 24.01
2584
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control ,
vol. 60, no. 12,
December
2013
than the original images while maintaining the same scanning time; or an image with the same resolution can be achieved in a quarter of the original scanning time. Therefore, the super-resolution method has particular value in NDE defect diagnosis. Acknowledgments
Fig. 14. Comparison of (a) one of the micro-scan ultrasonic testing images and (b) the resonstructed super-resolution image.
parameters (threshold low: 10, threshold high: 20, and smooth filter size: 17) used in the detector were applied to both the low-resolution and super-resolution images. VII. Conclusions High-resolution UT images of ROIs are valuable for defect size, shape, and location measurements. In this study, a super-resolution image reconstruction methodology based on IBP and micro-scanning was applied to improve the resolution of UT C-scan images. The experimental results obtained have demonstrated the applied method is capable of providing better visual quality than the raw UT images. The resolution of the super-resolution result was increased 4 times over the raw C-scan image. The super-resolution method greatly improved image quality and also allowed for more detailed inspection of the ROIs with high resolution. Super-resolution image reconstruction can also be utilized to overcome the inherent limitations of the existing ultrasonic C-scan system. Because the C-scan time is proportional to the desired resolution, to double the resolution in both x- and y-directions, 4 times the original scanning time may be required. If a C-scan system was equipped with multiple ultrasound transducers with known displacement from each other, the system can offer a super-resolution result resolution 4 times larger
Fig. 15. Canny edge detector result comparison of (a) low-resolution micro-scan ultrasonic testing image and (b) the corresponding superresolution image.
The authors thank the Center for Advanced Friction Studies (CAFS), Southern Illinois University Carbondale, Carbondale, IL, directed by Dr. P. Fillip, for partially supporting this project. We also thank Mr. A. Poudel and Mr. C. McGee for providing their assistance in performing the testing. References [1] K. H. Im, D. K. Hsu, and I. Y. Yang, “Inspection of inhomogeneities in carbon/phenolic matrix composite materials using NDE techniques,” Key Eng. Mater., vol. 270, no. 1, pp. 1799–1805, 2004. [2] J. H. Lee, S. W. Choi, K. S. Kim, and J. H. Park, “Nondestructive characterisation of carbon/carbon brake disks using ultrasonics,” in 10th Asia-Pacific Conf. Non-Destructive Testing, 2001, pp. 1109–1112. [3] A. Ruosi, “Nondestructive detection of damage in carbon fiber composites by SQUID magnetometry,” Phys. Status Solidi C, vol. 2, no. 5, pp. 1533–1555, 2005. [4] D. E. Bray and D. McBride, Nondestructive Testing Techniques, New York, NY: Wiley, 1992. [5] H. Zhang, M. Wan, J. Wan, and X. Qin, “Super-resolution reconstruction of deformable tissue from temporal sequence of ultrasound images,” in Int. Conf. Artificial Intelligence and Computational Intelligence, 2010, vol. 1, pp. 337–342. [6] D. Kouame and M. Ploquin, ” Super-resolution in medical imaging : An illustrative approach through ultrasound,” in IEEE Int. Symp. Biomedical Imaging: From Nano to Macro, 2009, pp. 249–252. [7] G. Clark and J. Jackson, “Super-resolution algorithms for ultrasonic nondestructive evaluation imagery,” J. Acoust. Soc. Am., vol. 120, no. 5, p. 3140, 2006. [8] Y. Dai, B. Wang, and D. Liu, “A fast and robust super resolution method for intima reconstruction in medical ultrasound,” in 3rd Int. Conf. Bioinformatics and Biomedical Engineering, 2009, pp. 1–4. [9] R. Y. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vision Image Process., vol. 1, no. 2, pp. 101–106, 1984. [10] H. Stark and P. Oskoui, “High resolution image recovery from image plane arrays, using convex projections,” J. Opt. Soc. Am., vol. 6, no. 11, pp. 1715–1726, 1989. [11] M. Irani and S. Peleg, “Super resolution from image sequences,” in Proc. 10th Int. Conf. Pattern Recognition, 1990, vol. 112, pp. 115–120. [12] M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP Graph. Models Image Process., vol. 53, no. 3, pp. 231–239, 1991. [13] R. R. Schultz and R. L. Stevenson, “Improved definition video frame enhancement,” in Int. Conf. Acoustics, Speech, and Signal Processing Conf. Proc., 1995, vol. 4, pp. 2169–2172. [14] R. R. Schultz and R. L. Stevenson, “Video resolution enhancement,” in Proc. Conf. Image and Video Processing III, 1995, pp. 23–34. [15] R. R. Schultz and R. L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process., vol. 5, no. 6, pp. 996–1011, 1996. [16] R. R. Schultz and R. L. Stevenson, “Motion-compensated scan conversion of interlaced video sequences,” in Proc. Conf. Image and Video Processing IV, 1996, pp. 107–118. [17] M. K. Ng and N. K. Bose, “Mathamatical analysis of super-resolution methodology,” IEEE Signal Process. Mag., vol. 20, no. 3, pp. 62–74, 2003.
li and chu: super-resolution image reconstruction for ultrasonic nde [18] S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21–36, 2003. [19] S. Li and T. P. Chu, “Ultrasonic 3D reconstruction of CFRP panel delamination,” in Proc. ASNT Fall Conf., 2012.
Shanglei Li received the B.S. degree in electrical engineering from Changchun University of Science and Technology, Changchun, P. R. China, in 2006, and the M.S. degree in electrical engineering from California State University, Fullerton, in 2009. He is currently a Ph.D. candidate in the Electrical and Computer Engineering Department, Southern Illinois University, Carbondale. He has been working with Dr. Tsuchin Chu at the Intelligent Measurement & Evaluation Laboratory (IMEL) and conducting research work in nondestructive evaluation (NDE) since 2010. His research focuses on the development of an intelligent NDE expert system for carbon-based composites used for aircraft and aerospace applications by using ultrasonic (conventional, immersion, and air-coupled) and infrared thermography methods. His main tasks are NDE data analysis and image processing utilizing fuzzy logic, artificial neural networks (ANNs), signal processing, superresolution image reconstruction, and other intelligent algorithms. Mr. Li is highly motivated in doing NDE-related research work and possesses the capability to carry out his research work independently.
2585
Tsuchin Philip Chu received the B.E. degree in mechanical engineering from the National Cheng Kung University, Taiwan, R.O.C., in 1974; the M.S. degree in mechanical engineering from Aubrn University, Auburn, AL, in 1980; and the Ph.D. degree in mechanical engineering from the University of South Carolina, Columbia, SC, in 1982. He is currently a senior Professor in the Department of Mechanical Engineering and Energy Processes at Southern Illinois University, Carbondale (SIU). He has conducted research for more than 30 years in nondestructive evaluation (NDE), biomedical engineering, experimental mechanics, computer-aided design, manufacturing, and engineering (CAD/ CAM/CAE), finite element analysis (FEA), sensors, and instrumentation. He is a pioneer in the area of digital image correlation (DIC) and at the cutting edge of research in NDE and biomechanics. He was a faculty member of Polytechnic University in New York before he joined SIU in 1990. He has more than 80 peer-reviewed journal publications and conference proceedings and more than $2 million in grants from NASA, Boeing, the US Air Force, IBM, the Illinois Clean Coal Institute, etc. He developed the Intelligent Measurement and Evaluation Lab (IMEL), which houses state-of-the-art equipment including a DIC system and an infrared (IR) thermography system, as well as immersion, contact, and air-coupled ultrasonic C-scan systems. He has advised more than 40 graduate students. He is currently serving as the council director of the ASNT (American Society of Nondestructive Testing) and the ASNT St. Louis Section Board of Directors. He is also an associate editor for the professional journal Experimental Techniques. Dr. Chu is a co-founder of Clipius Technologies, a think-tank company producing intellectual property in the areas of defense, aerospace, and biomedical devices.
