1034
Vol. 55, No. 5 / February 10 2016 / Applied Optics
Research Article
Flexible calibration method for an inner surface detector based on circle structured light YE ZHU,1 YONGGANG GU,2,* YI JIN,2
AND
CHAO ZHAI2
1
Department of Precise Machinery and Precise Instrumentation, University of Science and Technology of China, Hefei 230027, China Experiment Center of Engineering and Material Science, University of Science and Technology of China, Hefei 230027, China *Corresponding author: [email protected]
2
Received 3 November 2015; revised 25 December 2015; accepted 26 December 2015; posted 6 January 2016 (Doc. ID 252938); published 8 February 2016
A new calibration method for an inner surface detector based on circle structured light is proposed in this study. Compared with existing methods, this technique is more flexible and practical and only requires a blank planar board and an additional camera, which is precalibrated under the detector’s coordinate system. The board is observed by the detector and the additional camera at a few (at least two) different orientations, which need not be known. The mathematical model of this method considers different alignment errors, which are disregarded in existing methods; therefore, precise assembly is not required. The binocular intersection algorithm is used to calculate the coordinates of the calibration points. The measurement system calibrated by this method performs well in the field test in which the maximum relative error of the measured values is less than 0.18%. The experimental result indicates that this method is highly accurate and can be easily applied in inner surface detection. © 2016 Optical Society of America OCIS codes: (040.0040) Detectors; (120.6650) Surface measurements, figure; (150.1488) Calibration; (330.1400) Vision - binocular and stereopsis. http://dx.doi.org/10.1364/AO.55.001034
1. INTRODUCTION Small-diameter tubes that require high accuracy, such as artillery tubes, flow field control pipelines, and heat exchanger lines, are widely used in the defense industry, aerospace engineering, and the chemical industry. The working condition of these tubes is directly determined by the machining precision and defects of their inner surface. Therefore, a reliable method for inner surface detection is important in product detection and fault diagnosis. Researchers have proposed several feasible methods. One of which is the circle structured light detection method, which, unlike other techniques, has significant advantages, such as compact structure, small diameter, noncontact, and full-field 3D reconstruction ability [1–3]. A typical construction diagram of the detector is shown in Fig. 1. The laser beam is refracted by the conical lens (the conical lens can be replaced by a conical reflection in some systems) and transformed into a conic surface, which is called circle structured light. The conic surface is intersected by the inner surface to create a close-curving light stripe, which can be captured by the CCD camera. The coordinates of the points on the stripe depends on the spatial parameters of the conic surface; therefore, the calibration of conical surface is the key to highprecision measurement. 1559-128X/16/051034-06$15/0$15.00 © 2016 Optical Society of America
Figure 2 shows the beam path of the circle structured light generation, where d 0 is the diameter of the laser beam, t is the width of the circle structured light, and α is the semiapex angle. Researchers have developed several circle structured light calibration methods. Zhang used a thin thread to intersect the light conic surface to create bright points whose coordinates were measured by two electric theodolites; these points were fit to the conic surface. [4] This method has long been used for linearly structured light calibration, but its operational complexity is apparent. Zhang also developed a calibration method for structured light detectors using a planar target of unknown orientations [5–7]. This method is easy to operate but is only appropriate for linear structured light. Zhuang calibrated the circle structured light detector by measuring a standard ring gauge and by comparing the result with the reference value [8,9]. This method requires strict coincidence between the axis of the structured light and the axis of the ring gauge, which is extremely difficult to implement. Albertazzi improved the circle structured light detector by applying fringe projection to reconstruct a section of the inner surface without moving the detector [10]. He also calibrated the detector by measuring a reference cylinder that requires precise displacement along a
Research Article
Vol. 55, No. 5 / February 10 2016 / Applied Optics
X u − C x1 ; F1 Z
Fig. 1. Construction diagram of a circle structured light detector.
v − C y1 Y ; F1 Z
1035
(1)
where F 1 is the focal length, and C x1 C y1 is the principal point. A wide-angle lens is necessary to ensure that the inner surface is completely captured, but it causes a distortion problem. A fourth-order radial distortion model and a second-order tangential distortion model as Eq. (2) are adopted to describe and correct the distortion: 8 u ud > 2 4 > 1 k 1 r k2 r d < vd v : (2) > 2k 3 uv k 4 r 2 2u2 > : d k3 r 2 2v 2 2k4 uv All the image coordinates used below are the corrected image points. Some researchers have assumed that the conic axis strictly overlaps with the Z label in the measurement system. Therefore, the conic surface is described as Eq. (3): z − Δz 2 cos2 α; x 2 y 2 z − Δz 2
Fig. 2. Beam path diagram of the circle structured light generation.
fixed axis. Wang used a radial concentric circle planar pattern to calibrate the detector based on the principle of invariant cross ratio; the planar pattern requires precise movement along the axis of the circle structured light [11,12]. Wakayama obtained the parameters of the detector through accurate assembly and did not take assembly errors into consideration, although they are inevitable and result in system errors [13–15]. We developed a flexible circle structured light calibration method based on binocular vision. This method only requires the detector, along with an additional camera, to observe a blank planar board (assumed to be a perfectly plane) at a few (at least two) different unknown orientations. The additional camera is precalibrated under the detector’s coordinate system. This method also considers different assembly errors of the conical lens, such as axis misalignment and vertex deviation; therefore, the detector does not require precise fabrication.
where 0 0 Δz are the coordinates of the conic node, and α is the semi-apex angle. Nevertheless, assembly errors such as axis misalignment and conic node deviation are inevitable; thus, system errors occur in the previously mentioned measurement model. Conversely, our model considers assembly errors. The parameters are shown in Fig. 3. C is the conic node and l is the conic axis. The blue segment l 1 is parallel to l and intersects the z axis at an included angle φ, which is also the space angle between the conic axis l and the z axis. l 2 is the projection of conic axis l on the x-o-y plane, intersecting with the y axis at an included angle γ. It is worth mentioning that the refracted beam is still assumed to be an ideal conic surface in our model, which means that the beam requires it to be parallel to the axis of the conical lens. Equation (4) is the expression of the conic surface S, where Δx , Δy , Δz , φ, γ, and α are the parameters to be calibrated: x − Δx sin φ sin γ y − Δy sin φ cos γ z − Δz cos φ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi cos α: x − Δx 2 y − Δy 2 z − Δz 2 (4)
2. MATHEMATICAL MODEL OF THE CIRCLE STRUCTURED LIGHT DETECTOR Two cameras are used in our calibration experiment: the detection camera denoted as camera 1 (Fig. 1) and the camera only used for calibration denoted as camera 2. Camera 1 is part of the detector and included in the measurement model. The camera coordinate system of camera 1 is regarded as the local coordinate system O-X Y Z ; the origin of the coordinates is set at the optic center, and the z axis is along the optic axis. The relationship between the 3D object point PX ; Y ; Z and its image projection pu; v can be modeled by the pinhole imaging model and described by the following imaging ray equation:
(3)
Fig. 3. Diagram of the conic surface expression.
1036
Vol. 55, No. 5 / February 10 2016 / Applied Optics
As previously mentioned, the point under test P belongs to both imaging ray l and conic surface S. Therefore, the coordinates of point P can be calculated using generalized optical triangulation, which involves solving Eq. (5) that consists of Eqs. (1) and (4) if the corresponding parameters are determined: 8 v−C y u−C x y x < z Fx ; z Fy : (5) φ sin γy−Δy sin φ cos γz−Δz cos φ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi : x−Δx sin p cos α 2 2 2 x−Δx y−Δy z−Δz
The parameters of Eq. (1) are calibrated through Zhang’s camera calibration method [16], and the parameters of Eq. (4) are calibrated by the proposed method. The calibration process is explained in the succeeding paragraphs. 3. CALIBRATION ALGORITHM A blank planar board is intersected with the conic surface in an arbitrary orientation and position. The intersection line (a close-curving light stripe) is a spatial ellipse called E i , as described by Eq. (6): ( ai x bi y c i z 1 0 x−Δx sin φ sin γy−Δy sin φ cos γz−Δz cos φ (6) pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi cos α : 2 2 2 x−Δx y−Δy z−Δz
E i is captured by the two cameras. According to the principles of projective geometry, an ellipse corresponds to another ellipse in the central projection. Therefore, the images that correlate with E i called e i1 (in camera 1) and e i2 (in camera 2) are also ellipses, as described by Eq. (7), in which the coordinates belong to the image coordinate system: e i1 :ai1 u21 bi1 v 21 c i1 u1 d i1 v 1 1 0 : (7) e i2 :ai2 u22 bi2 v 22 c i2 u2 d i2 v 2 1 0 Based on the centers of the projections captured by the cameras, the center of ellipse E i can be calculated using the binocular stereo reconstruction algorithm. The planar board intersects with the structured light at several different orientations and positions to create several intersecting lines (spatial ellipses). All the ellipse centers are on the conic axis. Therefore, the conic axis can be estimated through linear fitting based on the sufficient number of ellipse centers. Equation (8) is the fitted expression of the conic axis L: x t 1 z m1 : (8) y t 2 z m2 The reliable fitting points ensure an accurate estimation, and their reliability is guaranteed by the subpixel precision of ellipse fitting. Surface points are necessary to determine a complete conic surface expression. Given the focal length f 1 of camera 1, the spatial expression of e i1 in the world coordinate system is generated as follows: ai1 x 2 bi1 y 2 c i1 x d i1 y 1 0 : (9) zf1 An inclined ellipse conic surface Ai is constructed with the optical center (also the origin) O as the vertex and e i1 as the directrix. Equation (10) is the expression of Ai :
Research Article ai1 f 1 x 2 bi1 f 1 y2 z c i1 x d i1 y 0: z z f1
(10)
Clearly, E i is the intersecting line of Ai and S. Therefore, the points of E i are the surface points required to determine the conic surface S. E i also correlates with e i2 on the image plane of camera 2. This correlation implies that each image point p of e i2 correlates with a point P that belongs to E i . Therefore, P can be determined by the intersection of its imaging ray l p with Ai , as shown in Fig. 4. Equation (11) shows the expression of l p , the imaging ray of image point p u2 v 2 : v2 −C y2 u2 −C x2 y 2 x2 z2 f 2 ; z2 f 2 ; (11) x y z T R x 2 y 2 z 2 T T where R T is the coordinate conversion from camera 2 to camera 1. All parameters with the subscript “2” belong to the coordinate system of camera 2. Equations (10) and (11) are simultaneously solved to determine surface points. Generally, a pair of crossing points P 0 and P 0 0 exist, but only one of them is the desired surface point. Therefore, an algorithm must be used to distinguish the points. The spatial angle between two optical axes in our calibration system is less than 90°. Two image points correlating with a common object remain in the same general direction relative to their ellipse center point. This condition helps to determine the wrong point, with the corresponding image points in camera 1 being in a different direction relative to the ellipse center from p. A set of surface points fP ik jk 1 ∼ ng is obtained by the intersection calculation between n different imaging ray l pi of image points on e i2 and the ellipse conic surface Ai . Furthermore, the noncoplanar point set fP ik jk 1 ∼ n; i 1 ∼ mg is obtained when m different spatial ellipses are created by moving the planar board. Conic surface fitting can be conducted. As previously mentioned, the conic axis is already determined to be reliable as Eq. (8). The expression of the conic surface is transformed into Eq. (12): AX c − t 1 Δz − m1 BY c − t 2 Δz − m2 CZ c − Δz pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi cos α; X c − t 1 Δz − m1 2 Y c − t 2 Δz − m2 2 Z c − Δz 2 (12) where all the parameters except Δz and α are known because the conic axis is determined. A, B, and C are newly introduced parameters helping to simplify the formula, and they are defined in Eq. (13):
Fig. 4. Diagram of the intersection between Ai and l p .
Research Article (
Vol. 55, No. 5 / February 10 2016 / Applied Optics
A sin φ sin γ B sin φ cos γ : C cos φ
(13)
Equation (13) is the residual percentage sum of squares defined in this study, and it is selected as the object function to be minimized: 2 n X m X d ik −1 ; (14) Qα; Δz r ik k1 i1 where d ik is the distance between P ik and L, and r ik is the conic radius coplanar with P ik . The optimal solution of α; Δz is obtained by solving Eq. (14): ∂Q 0; ∂α
∂Q 0: ∂Δz
(15)
The circle structured light calibration is completed.
4. EXPERIMENT A. Calibration
As shown in Fig. 5, the experimental system is set up on an optical table. The system consists of a conical lens, a blank planar board, a laser, and two high-precision monochrome CCD cameras (with a resolution of 1024 × 1024 and focal distance of 16 mm). Camera 1 is used for detection; camera 2 is used for calibration. The cameras are connected to the PC by a dualchannel image acquisition card. The blank planar board is made by spraying white matte paint on a normal glass board; the roughness of the sprayed glass board is about Ra1.5. The calibration procedure is summarized as follows: 1. Calibrate the two cameras using Zhang’s algorithm [16]. The external parameters of camera 2 must be transformed into the image coordinate system of camera 1. 2. Block the circle structured light with the blank planar board in an arbitrary orientation to create a light stripe, which will be captured by the two cameras. 3. Change the orientation and position of the plate and repeat step 2 five times. 4. Process the data using the algorithm in Section 2.
Fig. 5. Physical map of the calibration experimental system.
1037
Fig. 6. Images of the light stripes captured in the experiment.
The captured images of five elliptical light stripes from the two cameras are presented in Fig. 6. As mentioned in Section 3, the expression of e i1 is required to determine Ai . Therefore, these light stripes must be fitted into the ellipse before the intersection calculation. We adopt the least-squares fitting method proposed by Gander [17]. As shown in Fig. 7, the red curve is the fitted ellipse and the blue cross is the ellipse center. It is worth noting that the light stripe would be created with different width at different orientation, especially during the inner surface detection. To avoid the inconsistency of coordinate extraction caused by width difference, outer edge points are selected to conduct intersection calculation during calibration and measurement. The minimum width of 4 pixels and enough contrast are needed to ensure effective edge detection. The centers of the fitted ellipses are used to determine the axis of the structured light L through binocular stereo reconstruction algorithm. A sufficient number of fitted ellipse edge points of e i2 are selected for intersection calculation. The same amount of surface points is determined and used to conduct conic surface fitting using Eq. (13). The fitting result is presented in Fig. 8 and the parameters used in Eq. (4) are presented in Table 1. B. Measurement
Actual measurement experiments must be conducted to evaluate the accuracy of the calibration procedure. As shown in Fig. 9, the experimental system is composed of the entire detector with the calibration from Section 4.A, a precise rotating platform, a precise linear translation plate, and a bearing with
Fig. 7. Ellipse fitting result of the light stripe.
1038
Research Article
Vol. 55, No. 5 / February 10 2016 / Applied Optics
Fig. 10. Captured images in the experiment.
Fig. 8. Conic surface fitting result.
Table 1.
Parameters of Circle Structured Light
Parameters Values Parameters Values
Δx 7.658 mm φ 2.351°
Δy 5.422 mm γ 3.768°
Δz 193.867 mm α 27.633°
an internal diameter of 80 mm, which acts as a small-diameter tube to be detected. In this experiment, the internal diameter of the bearing is measured by our system and compared with the reference value. The inner surface of the bearing is an ideal cylinder; therefore, the reference value can be easily obtained through a 3D coordinate measurement machine and cylinder fitting. For generality, the internal diameter of the bearing is measured under different inclined angles between the axis of the bearing and the axis of the structured light. The inclined angle, which ranges from −6° to 6°, is changed by rotating the precise rotating platform every 0.5°. In each inclined angle, the bearing is detected at three known positions provided by the precise linear translation plate. The inner surface points measured at the three positions are used in the cylinder fitting. Figure 10 shows some of the captured images in the experiment. Based on the calibration result in Section 4.A and the captured images above, the 3D coordinates of the inner surface
Fig. 11. Measured values at different inclined angles.
points can be calculated using the optical triangle method described in Eq. (5). The inner surface points are fitted to a cylindrical surface. The diameters of the fitted cylindrical surfaces are considered the measured values of the bearing being tested, the reference value is 80.001 mm, which is provided by a Daisy8106-type three-coordinate measuring machine with an indication error of 2 μm in the 80 mm range. Figure 11 presents the measured values at different inclined angles. The maximum relative error is 0.18%, and the average relative error is 0.12%. The standard error of all 25 data points is 2.2 μm, which indicates good stability of the system. These results demonstrate that the proposed method is reliable and accurate. It is noteworthy that all the measured values are smaller than the reference value of 80.001 mm; thus, there is a small negative systematic error in the measurement results. Here is the analysis of causes of this systematic error: (1) during camera calibration, the scale factor might be underestimated due to the calibration board’s thermal deformation; (2) the lens distortion is radial nonuniform (generally greater near the border of the field of view) and is estimated using calibration points uniformly distributed in the whole field of view, while the measured points are mainly distributed in the boundary area. Thus, there might be small errors in the calibrated distortion model. 5. CONCLUSION
Fig. 9. Physical map of the measurement experimental system.
A new mathematical model for an inner surface detector that considers the error of misalignment of the conical lens was proposed in this study. Described by this model, the detector is able to conduct accurate measurement without the conical lens precisely aligning with the camera. A flexible calibration
Research Article method was also described. Results of the calibration experiments and measurement experiments indicate that the model and method are reliable and accurate. Only a blank free-moving planar board and an additional calibrated camera are required to conduct the calibration. The calibration was conducted without any difficulty. Therefore, the proposed method is flexible and easy to perform. Funding. National Natural Science Foundation of China (NSFC) (U1331201); The Fundamental Research Funds for the Central Universities. REFERENCES 1. E. Hong, R. Katz, B. Hufnagel, and J. Agapiou, “Optical method for inspecting surface defects inside a small bore,” Meas. Sci. Technol. 21, 015704 (2010). 2. O. Duran, K. Althoefer, and L. D. Seneviratne, “State of the art in sensor technologies for sewer inspection,” IEEE Sens. J. 2, 73–81 (2002). 3. O. Duran, K. Althoefer, and L. D. Seneviratne, “Pipe inspection using a laser-based transducer and automated analysis techniques,” IEEE/ ASME Trans. Mechatron. 8, 401–409 (2003). 4. G. Zhang, J. He, and X. Li, “3D vision inspection for internal surface based on circle structured light,” Sens. Actuators A 122, 68–75 (2005). 5. F. Zhou and G. Zhang, “Complete calibration of a structured light stripe vision sensor through planar target of unknown orientations,” Image Vision Comput. 23, 59–67 (2005). 6. G. Zhang and Z. Wei, “A novel calibration approach to structured light 3D vision inspection,” Opt. Laser Technol. 34, 373–380 (2002).
Vol. 55, No. 5 / February 10 2016 / Applied Optics
1039
7. F. Zhou, G. Zhang, and J. Jiang, “Constructing feature points for calibrating a structured light vision sensor by viewing a plane from unknown orientations,” Opt. Lasers Eng. 43, 1056–1070 (2005). 8. B. H. Zhuang, W. Zhang, and D. Y. Cui, “Noncontact laser sensor for pipe inner wall inspection,” Opt. Eng. 37, 1643–1647 (1998). 9. W. W. Zhang and B. H. Zhuang, “Non-contact laser inspection for the inner wall surface of a pipe,” Meas. Sci. Technol. 9, 1380–1387 (1998). 10. A. G. Albertazzi, Jr., A. C. Hofmann, A. V. Fantin, and J. Santos, “Photogrammetric endoscope for measurement of inner cylindrical surfaces using fringe projection,” Appl. Opt. 47, 3868–3876 (2008). 11. Y. Wang, R. Zhang, and Y. Zhang, “Constructing method of calibration feature points used for circle structure light vision sensor,” J. Appl. Opt. 5, 013 (2012). 12. W. Ying and Z. Yuan, “A calibration method for circle structure light based on coplanar reference object,” Infrared Laser Eng. 42, 174– 178 (2013). 13. T. Wakayama, H. Takano, and T. Yoshizawa, “Development of a compact inner profile measuring instrument,” Proc. SPIE 6762, 67620D (2007). 14. T. Yoshizawa and T. Wakayama, “Development of an inner profile measurement instrument using a ring beam device,” Proc. SPIE 7855, 78550B (2010). 15. T. Wakayama, K. Machi, and T. Yoshizawa, “Small size probe for inner profile measurement of pipes using optical fiber ring beam device,” Proc. SPIE 8563, 85630L (2012). 16. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000). 17. W. Gander, G. H. Golub, and R. Strebel, “Least-squares fitting of circles and ellipses,” BIT Numer. Math. 34, 558–578 (1994).
Vol. 55, No. 5 / February 10 2016 / Applied Optics
Research Article
Flexible calibration method for an inner surface detector based on circle structured light YE ZHU,1 YONGGANG GU,2,* YI JIN,2
AND
CHAO ZHAI2
1
Department of Precise Machinery and Precise Instrumentation, University of Science and Technology of China, Hefei 230027, China Experiment Center of Engineering and Material Science, University of Science and Technology of China, Hefei 230027, China *Corresponding author: [email protected]
2
Received 3 November 2015; revised 25 December 2015; accepted 26 December 2015; posted 6 January 2016 (Doc. ID 252938); published 8 February 2016
A new calibration method for an inner surface detector based on circle structured light is proposed in this study. Compared with existing methods, this technique is more flexible and practical and only requires a blank planar board and an additional camera, which is precalibrated under the detector’s coordinate system. The board is observed by the detector and the additional camera at a few (at least two) different orientations, which need not be known. The mathematical model of this method considers different alignment errors, which are disregarded in existing methods; therefore, precise assembly is not required. The binocular intersection algorithm is used to calculate the coordinates of the calibration points. The measurement system calibrated by this method performs well in the field test in which the maximum relative error of the measured values is less than 0.18%. The experimental result indicates that this method is highly accurate and can be easily applied in inner surface detection. © 2016 Optical Society of America OCIS codes: (040.0040) Detectors; (120.6650) Surface measurements, figure; (150.1488) Calibration; (330.1400) Vision - binocular and stereopsis. http://dx.doi.org/10.1364/AO.55.001034
1. INTRODUCTION Small-diameter tubes that require high accuracy, such as artillery tubes, flow field control pipelines, and heat exchanger lines, are widely used in the defense industry, aerospace engineering, and the chemical industry. The working condition of these tubes is directly determined by the machining precision and defects of their inner surface. Therefore, a reliable method for inner surface detection is important in product detection and fault diagnosis. Researchers have proposed several feasible methods. One of which is the circle structured light detection method, which, unlike other techniques, has significant advantages, such as compact structure, small diameter, noncontact, and full-field 3D reconstruction ability [1–3]. A typical construction diagram of the detector is shown in Fig. 1. The laser beam is refracted by the conical lens (the conical lens can be replaced by a conical reflection in some systems) and transformed into a conic surface, which is called circle structured light. The conic surface is intersected by the inner surface to create a close-curving light stripe, which can be captured by the CCD camera. The coordinates of the points on the stripe depends on the spatial parameters of the conic surface; therefore, the calibration of conical surface is the key to highprecision measurement. 1559-128X/16/051034-06$15/0$15.00 © 2016 Optical Society of America
Figure 2 shows the beam path of the circle structured light generation, where d 0 is the diameter of the laser beam, t is the width of the circle structured light, and α is the semiapex angle. Researchers have developed several circle structured light calibration methods. Zhang used a thin thread to intersect the light conic surface to create bright points whose coordinates were measured by two electric theodolites; these points were fit to the conic surface. [4] This method has long been used for linearly structured light calibration, but its operational complexity is apparent. Zhang also developed a calibration method for structured light detectors using a planar target of unknown orientations [5–7]. This method is easy to operate but is only appropriate for linear structured light. Zhuang calibrated the circle structured light detector by measuring a standard ring gauge and by comparing the result with the reference value [8,9]. This method requires strict coincidence between the axis of the structured light and the axis of the ring gauge, which is extremely difficult to implement. Albertazzi improved the circle structured light detector by applying fringe projection to reconstruct a section of the inner surface without moving the detector [10]. He also calibrated the detector by measuring a reference cylinder that requires precise displacement along a
Research Article
Vol. 55, No. 5 / February 10 2016 / Applied Optics
X u − C x1 ; F1 Z
Fig. 1. Construction diagram of a circle structured light detector.
v − C y1 Y ; F1 Z
1035
(1)
where F 1 is the focal length, and C x1 C y1 is the principal point. A wide-angle lens is necessary to ensure that the inner surface is completely captured, but it causes a distortion problem. A fourth-order radial distortion model and a second-order tangential distortion model as Eq. (2) are adopted to describe and correct the distortion: 8 u ud > 2 4 > 1 k 1 r k2 r d < vd v : (2) > 2k 3 uv k 4 r 2 2u2 > : d k3 r 2 2v 2 2k4 uv All the image coordinates used below are the corrected image points. Some researchers have assumed that the conic axis strictly overlaps with the Z label in the measurement system. Therefore, the conic surface is described as Eq. (3): z − Δz 2 cos2 α; x 2 y 2 z − Δz 2
Fig. 2. Beam path diagram of the circle structured light generation.
fixed axis. Wang used a radial concentric circle planar pattern to calibrate the detector based on the principle of invariant cross ratio; the planar pattern requires precise movement along the axis of the circle structured light [11,12]. Wakayama obtained the parameters of the detector through accurate assembly and did not take assembly errors into consideration, although they are inevitable and result in system errors [13–15]. We developed a flexible circle structured light calibration method based on binocular vision. This method only requires the detector, along with an additional camera, to observe a blank planar board (assumed to be a perfectly plane) at a few (at least two) different unknown orientations. The additional camera is precalibrated under the detector’s coordinate system. This method also considers different assembly errors of the conical lens, such as axis misalignment and vertex deviation; therefore, the detector does not require precise fabrication.
where 0 0 Δz are the coordinates of the conic node, and α is the semi-apex angle. Nevertheless, assembly errors such as axis misalignment and conic node deviation are inevitable; thus, system errors occur in the previously mentioned measurement model. Conversely, our model considers assembly errors. The parameters are shown in Fig. 3. C is the conic node and l is the conic axis. The blue segment l 1 is parallel to l and intersects the z axis at an included angle φ, which is also the space angle between the conic axis l and the z axis. l 2 is the projection of conic axis l on the x-o-y plane, intersecting with the y axis at an included angle γ. It is worth mentioning that the refracted beam is still assumed to be an ideal conic surface in our model, which means that the beam requires it to be parallel to the axis of the conical lens. Equation (4) is the expression of the conic surface S, where Δx , Δy , Δz , φ, γ, and α are the parameters to be calibrated: x − Δx sin φ sin γ y − Δy sin φ cos γ z − Δz cos φ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi cos α: x − Δx 2 y − Δy 2 z − Δz 2 (4)
2. MATHEMATICAL MODEL OF THE CIRCLE STRUCTURED LIGHT DETECTOR Two cameras are used in our calibration experiment: the detection camera denoted as camera 1 (Fig. 1) and the camera only used for calibration denoted as camera 2. Camera 1 is part of the detector and included in the measurement model. The camera coordinate system of camera 1 is regarded as the local coordinate system O-X Y Z ; the origin of the coordinates is set at the optic center, and the z axis is along the optic axis. The relationship between the 3D object point PX ; Y ; Z and its image projection pu; v can be modeled by the pinhole imaging model and described by the following imaging ray equation:
(3)
Fig. 3. Diagram of the conic surface expression.
1036
Vol. 55, No. 5 / February 10 2016 / Applied Optics
As previously mentioned, the point under test P belongs to both imaging ray l and conic surface S. Therefore, the coordinates of point P can be calculated using generalized optical triangulation, which involves solving Eq. (5) that consists of Eqs. (1) and (4) if the corresponding parameters are determined: 8 v−C y u−C x y x < z Fx ; z Fy : (5) φ sin γy−Δy sin φ cos γz−Δz cos φ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi : x−Δx sin p cos α 2 2 2 x−Δx y−Δy z−Δz
The parameters of Eq. (1) are calibrated through Zhang’s camera calibration method [16], and the parameters of Eq. (4) are calibrated by the proposed method. The calibration process is explained in the succeeding paragraphs. 3. CALIBRATION ALGORITHM A blank planar board is intersected with the conic surface in an arbitrary orientation and position. The intersection line (a close-curving light stripe) is a spatial ellipse called E i , as described by Eq. (6): ( ai x bi y c i z 1 0 x−Δx sin φ sin γy−Δy sin φ cos γz−Δz cos φ (6) pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi cos α : 2 2 2 x−Δx y−Δy z−Δz
E i is captured by the two cameras. According to the principles of projective geometry, an ellipse corresponds to another ellipse in the central projection. Therefore, the images that correlate with E i called e i1 (in camera 1) and e i2 (in camera 2) are also ellipses, as described by Eq. (7), in which the coordinates belong to the image coordinate system: e i1 :ai1 u21 bi1 v 21 c i1 u1 d i1 v 1 1 0 : (7) e i2 :ai2 u22 bi2 v 22 c i2 u2 d i2 v 2 1 0 Based on the centers of the projections captured by the cameras, the center of ellipse E i can be calculated using the binocular stereo reconstruction algorithm. The planar board intersects with the structured light at several different orientations and positions to create several intersecting lines (spatial ellipses). All the ellipse centers are on the conic axis. Therefore, the conic axis can be estimated through linear fitting based on the sufficient number of ellipse centers. Equation (8) is the fitted expression of the conic axis L: x t 1 z m1 : (8) y t 2 z m2 The reliable fitting points ensure an accurate estimation, and their reliability is guaranteed by the subpixel precision of ellipse fitting. Surface points are necessary to determine a complete conic surface expression. Given the focal length f 1 of camera 1, the spatial expression of e i1 in the world coordinate system is generated as follows: ai1 x 2 bi1 y 2 c i1 x d i1 y 1 0 : (9) zf1 An inclined ellipse conic surface Ai is constructed with the optical center (also the origin) O as the vertex and e i1 as the directrix. Equation (10) is the expression of Ai :
Research Article ai1 f 1 x 2 bi1 f 1 y2 z c i1 x d i1 y 0: z z f1
(10)
Clearly, E i is the intersecting line of Ai and S. Therefore, the points of E i are the surface points required to determine the conic surface S. E i also correlates with e i2 on the image plane of camera 2. This correlation implies that each image point p of e i2 correlates with a point P that belongs to E i . Therefore, P can be determined by the intersection of its imaging ray l p with Ai , as shown in Fig. 4. Equation (11) shows the expression of l p , the imaging ray of image point p u2 v 2 : v2 −C y2 u2 −C x2 y 2 x2 z2 f 2 ; z2 f 2 ; (11) x y z T R x 2 y 2 z 2 T T where R T is the coordinate conversion from camera 2 to camera 1. All parameters with the subscript “2” belong to the coordinate system of camera 2. Equations (10) and (11) are simultaneously solved to determine surface points. Generally, a pair of crossing points P 0 and P 0 0 exist, but only one of them is the desired surface point. Therefore, an algorithm must be used to distinguish the points. The spatial angle between two optical axes in our calibration system is less than 90°. Two image points correlating with a common object remain in the same general direction relative to their ellipse center point. This condition helps to determine the wrong point, with the corresponding image points in camera 1 being in a different direction relative to the ellipse center from p. A set of surface points fP ik jk 1 ∼ ng is obtained by the intersection calculation between n different imaging ray l pi of image points on e i2 and the ellipse conic surface Ai . Furthermore, the noncoplanar point set fP ik jk 1 ∼ n; i 1 ∼ mg is obtained when m different spatial ellipses are created by moving the planar board. Conic surface fitting can be conducted. As previously mentioned, the conic axis is already determined to be reliable as Eq. (8). The expression of the conic surface is transformed into Eq. (12): AX c − t 1 Δz − m1 BY c − t 2 Δz − m2 CZ c − Δz pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi cos α; X c − t 1 Δz − m1 2 Y c − t 2 Δz − m2 2 Z c − Δz 2 (12) where all the parameters except Δz and α are known because the conic axis is determined. A, B, and C are newly introduced parameters helping to simplify the formula, and they are defined in Eq. (13):
Fig. 4. Diagram of the intersection between Ai and l p .
Research Article (
Vol. 55, No. 5 / February 10 2016 / Applied Optics
A sin φ sin γ B sin φ cos γ : C cos φ
(13)
Equation (13) is the residual percentage sum of squares defined in this study, and it is selected as the object function to be minimized: 2 n X m X d ik −1 ; (14) Qα; Δz r ik k1 i1 where d ik is the distance between P ik and L, and r ik is the conic radius coplanar with P ik . The optimal solution of α; Δz is obtained by solving Eq. (14): ∂Q 0; ∂α
∂Q 0: ∂Δz
(15)
The circle structured light calibration is completed.
4. EXPERIMENT A. Calibration
As shown in Fig. 5, the experimental system is set up on an optical table. The system consists of a conical lens, a blank planar board, a laser, and two high-precision monochrome CCD cameras (with a resolution of 1024 × 1024 and focal distance of 16 mm). Camera 1 is used for detection; camera 2 is used for calibration. The cameras are connected to the PC by a dualchannel image acquisition card. The blank planar board is made by spraying white matte paint on a normal glass board; the roughness of the sprayed glass board is about Ra1.5. The calibration procedure is summarized as follows: 1. Calibrate the two cameras using Zhang’s algorithm [16]. The external parameters of camera 2 must be transformed into the image coordinate system of camera 1. 2. Block the circle structured light with the blank planar board in an arbitrary orientation to create a light stripe, which will be captured by the two cameras. 3. Change the orientation and position of the plate and repeat step 2 five times. 4. Process the data using the algorithm in Section 2.
Fig. 5. Physical map of the calibration experimental system.
1037
Fig. 6. Images of the light stripes captured in the experiment.
The captured images of five elliptical light stripes from the two cameras are presented in Fig. 6. As mentioned in Section 3, the expression of e i1 is required to determine Ai . Therefore, these light stripes must be fitted into the ellipse before the intersection calculation. We adopt the least-squares fitting method proposed by Gander [17]. As shown in Fig. 7, the red curve is the fitted ellipse and the blue cross is the ellipse center. It is worth noting that the light stripe would be created with different width at different orientation, especially during the inner surface detection. To avoid the inconsistency of coordinate extraction caused by width difference, outer edge points are selected to conduct intersection calculation during calibration and measurement. The minimum width of 4 pixels and enough contrast are needed to ensure effective edge detection. The centers of the fitted ellipses are used to determine the axis of the structured light L through binocular stereo reconstruction algorithm. A sufficient number of fitted ellipse edge points of e i2 are selected for intersection calculation. The same amount of surface points is determined and used to conduct conic surface fitting using Eq. (13). The fitting result is presented in Fig. 8 and the parameters used in Eq. (4) are presented in Table 1. B. Measurement
Actual measurement experiments must be conducted to evaluate the accuracy of the calibration procedure. As shown in Fig. 9, the experimental system is composed of the entire detector with the calibration from Section 4.A, a precise rotating platform, a precise linear translation plate, and a bearing with
Fig. 7. Ellipse fitting result of the light stripe.
1038
Research Article
Vol. 55, No. 5 / February 10 2016 / Applied Optics
Fig. 10. Captured images in the experiment.
Fig. 8. Conic surface fitting result.
Table 1.
Parameters of Circle Structured Light
Parameters Values Parameters Values
Δx 7.658 mm φ 2.351°
Δy 5.422 mm γ 3.768°
Δz 193.867 mm α 27.633°
an internal diameter of 80 mm, which acts as a small-diameter tube to be detected. In this experiment, the internal diameter of the bearing is measured by our system and compared with the reference value. The inner surface of the bearing is an ideal cylinder; therefore, the reference value can be easily obtained through a 3D coordinate measurement machine and cylinder fitting. For generality, the internal diameter of the bearing is measured under different inclined angles between the axis of the bearing and the axis of the structured light. The inclined angle, which ranges from −6° to 6°, is changed by rotating the precise rotating platform every 0.5°. In each inclined angle, the bearing is detected at three known positions provided by the precise linear translation plate. The inner surface points measured at the three positions are used in the cylinder fitting. Figure 10 shows some of the captured images in the experiment. Based on the calibration result in Section 4.A and the captured images above, the 3D coordinates of the inner surface
Fig. 11. Measured values at different inclined angles.
points can be calculated using the optical triangle method described in Eq. (5). The inner surface points are fitted to a cylindrical surface. The diameters of the fitted cylindrical surfaces are considered the measured values of the bearing being tested, the reference value is 80.001 mm, which is provided by a Daisy8106-type three-coordinate measuring machine with an indication error of 2 μm in the 80 mm range. Figure 11 presents the measured values at different inclined angles. The maximum relative error is 0.18%, and the average relative error is 0.12%. The standard error of all 25 data points is 2.2 μm, which indicates good stability of the system. These results demonstrate that the proposed method is reliable and accurate. It is noteworthy that all the measured values are smaller than the reference value of 80.001 mm; thus, there is a small negative systematic error in the measurement results. Here is the analysis of causes of this systematic error: (1) during camera calibration, the scale factor might be underestimated due to the calibration board’s thermal deformation; (2) the lens distortion is radial nonuniform (generally greater near the border of the field of view) and is estimated using calibration points uniformly distributed in the whole field of view, while the measured points are mainly distributed in the boundary area. Thus, there might be small errors in the calibrated distortion model. 5. CONCLUSION
Fig. 9. Physical map of the measurement experimental system.
A new mathematical model for an inner surface detector that considers the error of misalignment of the conical lens was proposed in this study. Described by this model, the detector is able to conduct accurate measurement without the conical lens precisely aligning with the camera. A flexible calibration
Research Article method was also described. Results of the calibration experiments and measurement experiments indicate that the model and method are reliable and accurate. Only a blank free-moving planar board and an additional calibrated camera are required to conduct the calibration. The calibration was conducted without any difficulty. Therefore, the proposed method is flexible and easy to perform. Funding. National Natural Science Foundation of China (NSFC) (U1331201); The Fundamental Research Funds for the Central Universities. REFERENCES 1. E. Hong, R. Katz, B. Hufnagel, and J. Agapiou, “Optical method for inspecting surface defects inside a small bore,” Meas. Sci. Technol. 21, 015704 (2010). 2. O. Duran, K. Althoefer, and L. D. Seneviratne, “State of the art in sensor technologies for sewer inspection,” IEEE Sens. J. 2, 73–81 (2002). 3. O. Duran, K. Althoefer, and L. D. Seneviratne, “Pipe inspection using a laser-based transducer and automated analysis techniques,” IEEE/ ASME Trans. Mechatron. 8, 401–409 (2003). 4. G. Zhang, J. He, and X. Li, “3D vision inspection for internal surface based on circle structured light,” Sens. Actuators A 122, 68–75 (2005). 5. F. Zhou and G. Zhang, “Complete calibration of a structured light stripe vision sensor through planar target of unknown orientations,” Image Vision Comput. 23, 59–67 (2005). 6. G. Zhang and Z. Wei, “A novel calibration approach to structured light 3D vision inspection,” Opt. Laser Technol. 34, 373–380 (2002).
Vol. 55, No. 5 / February 10 2016 / Applied Optics
1039
7. F. Zhou, G. Zhang, and J. Jiang, “Constructing feature points for calibrating a structured light vision sensor by viewing a plane from unknown orientations,” Opt. Lasers Eng. 43, 1056–1070 (2005). 8. B. H. Zhuang, W. Zhang, and D. Y. Cui, “Noncontact laser sensor for pipe inner wall inspection,” Opt. Eng. 37, 1643–1647 (1998). 9. W. W. Zhang and B. H. Zhuang, “Non-contact laser inspection for the inner wall surface of a pipe,” Meas. Sci. Technol. 9, 1380–1387 (1998). 10. A. G. Albertazzi, Jr., A. C. Hofmann, A. V. Fantin, and J. Santos, “Photogrammetric endoscope for measurement of inner cylindrical surfaces using fringe projection,” Appl. Opt. 47, 3868–3876 (2008). 11. Y. Wang, R. Zhang, and Y. Zhang, “Constructing method of calibration feature points used for circle structure light vision sensor,” J. Appl. Opt. 5, 013 (2012). 12. W. Ying and Z. Yuan, “A calibration method for circle structure light based on coplanar reference object,” Infrared Laser Eng. 42, 174– 178 (2013). 13. T. Wakayama, H. Takano, and T. Yoshizawa, “Development of a compact inner profile measuring instrument,” Proc. SPIE 6762, 67620D (2007). 14. T. Yoshizawa and T. Wakayama, “Development of an inner profile measurement instrument using a ring beam device,” Proc. SPIE 7855, 78550B (2010). 15. T. Wakayama, K. Machi, and T. Yoshizawa, “Small size probe for inner profile measurement of pipes using optical fiber ring beam device,” Proc. SPIE 8563, 85630L (2012). 16. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000). 17. W. Gander, G. H. Golub, and R. Strebel, “Least-squares fitting of circles and ellipses,” BIT Numer. Math. 34, 558–578 (1994).
