Autofocus using adaptive prediction approximation combined search for the fluorescence microscope in second-generation DNA sequencing system Hancong Xu, Jinfeng Liu, Yang Li, Yan Yin, Chenxu Zhu, and Hua Lu* State Key Laboratory of Bioelectronics, School of Biological Science and Medical Engineering, Southeast University, No. 2 Si Pai Lou, Nanjing 210096, China *Corresponding author: [email protected] Received 7 March 2014; revised 20 May 2014; accepted 24 May 2014; posted 29 May 2014 (Doc. ID 206972); published 9 July 2014
Autofocus is an important technique for high-speed image acquisition in the second-generation DNA sequencing system, and this paper studies the passive focus algorithm for the system, which consists of two parts: focus measurement (FM) and focus search (FS). Based on the properties of DNA chips’ images, we choose the normalized variance as the FM algorithm and develop a new robust FS named adaptive prediction approximation combined search (APACS). APACS utilizes golden section search (GSS) to approximate the focus position and engages the curve-fitting search (CFS) to predict the position simultaneously in every step of GSS. When the difference between consecutive predictions meets the set precision, the search finishes. Otherwise, it ends as GSS. In APACS, we also propose an estimation method, named the combination of centroid estimation and overdetermined equations estimation by least squares solution, to calculate the initial vector for the nonlinear equations in APACS prediction, which reduces the iterations and accelerates the search. The simulation and measured results demonstrate that APACS not only maintains the stability but also reduces the focus time compared with GSS and CFS, which indicates APACS is a robust and fast FS for the fluorescence microscope in a sequencing system. © 2014 Optical Society of America OCIS codes: (170.2520) Fluorescence microscopy; (170.3890) Medical optics instrumentation; (170.3880) Medical and biological imaging. http://dx.doi.org/10.1364/AO.53.004509
1. Introduction
Today, optical microscopes are widely used in biomedical engineering, medicine, hylology, industrial production, etc. They bring the micro-world to our view and help to observe micro-objects, such as cells and viruses, which cannot be seen with naked eyes. One microscope, termed a fluorescence microscope, uses fluorescence to generate images, and is widely applied in the second-generation DNA sequencing system to obtain images of sequencing chips for future analysis. In order to guarantee the image quality and reduce acquisition time, autofocusing 1559-128X/14/204509-10$15.00/0 © 2014 Optical Society of America
must be realized with the fluorescence microscope. However, compared with the traditional autofocus methods, it is challenging to achieve an accurate focus within a short time in the system. In principle, there are two strategies in an autofocus process: active autofocus and passive autofocus. Active autofocus mainly uses different sensors or measurement tools to determine the distance between the lens and the object, while passive autofocus relies on the image information only, taking advantage of image clarity to detect the focus position [1–5]. Generally, microscopes are not equipped with active focus parts and it is very expensive to integrate such a part into the microscope. Thus the passive focus is more adaptive to the sequencing system. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4509
However, traditional passive focus algorithms are inefficient and time-consuming, so we developed a fast and robust one to meet the requirement of the fluorescence microscope in the sequencing systems. The passive autofocus algorithm consists of two components: focus measurement (FM) and focus search (FS) [1]. Theoretically microscopes can be simplified as a lens model and if the view position in the image space is away from the focal plane, any viewed images turn out to be blurred [6]. Passive focus takes advantage of this and regards the clarity of captured images at different Z positions as the criterion of focus level, which is called the FM [1,7–9]. Then an FS procedure is applied to iterate through the candidates of FM values to select the focus position [1,4,5]. Our study is mainly concerned with developing a new FS algorithm. Meanwhile, a simple selection of FM algorithms is also taken into consideration. 2. Material and Experimental Platform
Pink SPHERO Fluorescent Carboxyl Magnetic Particles with a diameter of 1 μm (produced by the Spherotech company) are used as the specimen in our experiment. Maximum absorption for these beads occurs at a wavelength of approximately 560 nm and emission at 580 nm. We use the mercury lamp as the excitation light source and apply a Cool SNAP HQ2 CCD (produced by the Photometrics) to capture images, which has 1392H × 1040V pixels on a 8.98H × 6.71V mm sensor. The microscope is Nikon TE-2000-E, and the minimal movement step length of the Z axis is 50 nm. Also, the objective lens magnification is 20. A computer is utilized to control the microscope’s Z movement, CCD photographing, and algorithm implementation in our platform, with an Intel(R) Pentium(R)D CPU at operating frequency 2.6 GHz. Further, the available memory is 3.37 GB, and the operating system is Windows XP Professional. The sample images captured by the system at different Z positions from the bottom up are shown in Fig. 1.
Fig. 1. Sample images captured by the microscope system at different Z positions from the bottom up. The exposure time is 300 ms, and the bar in each image is 20 μm. As there exists depth of field in the system, (c) and (d) are two images in focus, while the other four are not. 4510
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
3. Focus Measurement
FM is actually a function of Z position, and each FM value is generated from a captured image at a certain Z position by FM algorithms, which have been widely studied in the previous literature, and no less than 15 FM algorithms have been proposed. FMs calculated by different algorithms are supposed to meet at least three standards: unimodality, accuracy, and reproducibility. Other properties are also important, such as calculating speed and noise immunity [10]. Sobel-Ten [3,6,11], normalized variance [12,13], and autocorrelation [3,10,14–16] are the most commonly used FM algorithms, which meet all three standards. Sun et al. compared 18 algorithms including the above three and found normalized variance is the best one according to its performance in different applications such as bright field, phase contrast, or differential interference contrast (DIC) [17]. Santos et al. found that autocorrelation is the optimal FM for fluorescence microscopy applications [16]. In our experiment the three FMs mentioned above are tested for our specimen, and their function graphs are shown in Fig. 2, which indicates that all three FMs are well-functioned. However, normalized variance is outstanding in this application, because it is easy to see that its peak width, namely full width at half-maximum (FWHM), among the three is the largest, and its monotony interval on each side of the peak is also the longest. Thus normalized variance confers an extended range near the focus position, which is more robust and more suitable for the curve-fitting strategy in our FS
Fig. 2. FM curve of normalized variance, Sobel-Ten, and autocorrelation in 100 μm interval. Each curve consists of an FM value at 1001 positions along the Z axis, and the step size is 0.1 μm. The Y axis represents the normalized FM value, and the peaks’ Z position indicates the focus position. As we can see, the three FMs are all well-functioned. The peak width at half height of normalized variance among the three ones is the largest, and its monotony interval on each side of the peak is also the longest, making it more suitable for the curve-fitting strategy in our focus search APACS.
algorithm, adaptive prediction approximation combined search (APACS), mentioned next. The detailed reason is given in Section 6, and normalized variance is chosen as the FM algorithm in this paper. Its formula is shown in Eq. (1) [12,13]: F var
XX
1 M N f x; y
x
f x; y − f x; y2 :
(1)
y
In Eq. (1), f x; y is the gray level at pixel x; y in the image, and f x; y is the average of f x; y. M and N represent the image’s row and column number, respectively, while F var stands for the FM value of the captured image. 4. Focus Search A.
FS Algorithm Review
In passive focus the ideal FM is a unimodal function, of which the peak’s Z position indicates the focus position, and then FS is applied to locate the peak. The most commonly used FS algorithms proposed in the previous literature are global search [18,19], hillclimbing search [3,6,20–22], binary search [23,24], curve-fitting search (CFS) [25], Fibonacci search (FibS) [23,26], and golden section search (GSS) [27], among which CFS is the fastest in theory. However, frequent errors of CFS are a problem. CFS utilizes some points in the FM curve to do the fitting and get its analytic formula, from which the focus position is calculated. Before fitting, we must decide the function type by which FM can be approximated, such as the quadratic function [28], Gaussian function [28], and Lorentz function [29], which have been proposed and proved to be effective in some cases. The formulas and the calculated focus positions are shown in Eqs. (2)–(4), respectively: Fz az2 bz c;
Gz a × e−
z−b2 c
;
a ; b z − c2
b the focus position∶ − ; 2a
(2)
the focus position∶ b;
(3)
When conducting least squares fitting, the quadratic function is the worst one because of its large fitting residual. Meanwhile Lorenz’s computational complexity is less than that of Gaussian, and it is the most suitable curve used in CFS. FibS and GSS are more reliable than CFS. They are two similar searching algorithms and share the same strategy to approximate the focus position by comparing the FM value of two points within the searching range to cut down part of the interval where the focus position is not located. Therefore, the final result of FibS and GSS is the interval including the focus position rather than a definite position value. Before applying FibS, the iterations of the search should be set based on the required accuracy. Theoretically, FibS is the optimum search and demands the fewest iterations, but when applied in autofocus, it is less accurate as the microscope cannot move to the exact position calculated by FibS. Therefore, more iterations are required to meet the set precision. Compared with FibS, GSS is an improved method, which does not finish until the length of the target interval meets the set precision. If the initial interval is [a, b], in theory, the target interval will be 0.618n × b − a after an n-step search, which is almost as large as that of FibS. The detailed process of GSS is shown in Fig. 3(a), and GSS uses two golden section points as the compared points in each interval [a, b], which are defined in Eq. (5): p 5−1 ; Gold ratio 2 G1 a 1 − Gold ratio × b − a; G2 a Gold ratio × b − a:
(5)
However, on our platform none of these methods meet the requirement of the sequencing system. For instance, CFS is not robust enough and GSS is not fast enough. Therefore, we developed a new one named APACS, which combines the merits of GSS and CFS, resulting in a faster speed with high stability.
(4)
B. Proposed Algorithm: Adaptive Prediction Approximation Combined Search
The three above functions all have three parameters, and at least three points in the FM curve, each resulting in an equation, are required to conduct the fitting. We can calculate all three parameters simply by three equations, or by least squares fitting with more than three points. The direct calculation by three equations is fast but inaccurate because of random noise when collecting images. On the contrary, the least squares fitting is more reliable, and its accuracy greatly depends on the number of points used for fitting, which are highly related to the searching time.
In GSS, we compare the FM value of the interval’s golden section points and obtain a shorter interval in which the focus position is still located. So in every step only two new points are calculated and engaged, while the other acquired points in the previous steps are abandoned, which also indicate the focus position to some extent. If we can make the most of the abandoned points when employing GSS, we can increase the convergence rate. Furthermore, we notice that the acquired points can be used for CFS, so we put forward a new FS algorithm named APACS based on CFS and GSS, where CFS is in fact a prediction search strategy and GSS is an approximation one.
Lz
the focus position∶ c:
10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4511
Fig. 3. Flowcharts of GSS and APACS. (a) Flowchart of GSS. The ak ; bk denotes the searched range after k steps. G1k and G2k denote the golden section points of the interval ak ; bk . f G1k and f G2k represent the FM value of the two golden section points. AR denotes the accuracy requirement, and k denotes the searching steps. (b) Flowchart of APACS. The variable denoted here is same as (a) except for focus_now and focus_pre, which represent the two consecutive predictions of APACS. The parts highlighted in red are some important steps that are different from GSS.
Besides, among the two ways in CFS, least squares fitting is chosen with a Lorenz function. APACS engages all the acquired points to do CFS in every step of GSS to predict the focus position. If the focus position is not within the interval, it is a wrong prediction and of no use, which usually happens at the beginning of the focus. As the search goes on and more points are obtained, the fitting deviation decreases and the CFS prediction slowly approaches the focus position. If the absolute difference between the two consecutive predictions is less than the set precision, APACS finishes successfully. However, if the above situation does not occur, unfortunately, the focus procedure finishes, acting like GSS. It guarantees the focus precision but actually turns out to be a little slower than GSS because of the vast computation in least squares fitting. After testing it is manifested that such a case only happens occasionally and APACS is faster than 4512
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
GSS in most instances. The detailed process is shown in Fig. 3(b). C.
Initial Vector for Fitting in APACS
The curve-fitting prediction in APACS brings forth another problem that least squares CFS is supposed to solve the nonlinear equations, which will appear in the fitting process. Nonlinear equations are usually solved by iteration, such as Jacob iteration, and the iteration process starts with an initial vector for the unknown parameters. Thus an improper initial vector can lead to an increase of iterations or even a wrong iteration result in the end. So we proposed some estimation algorithms to get the best initial vector based on the points acquired in every step of APACS. As we can see in Eq. (4), parameter c represents the focus position, while a and b are the two other related parameters when calculating parameter c.
However, it is quite difficult to estimate all three unknown parameters at the same time. So we plan to estimate the c value first and use the estimation of c to estimate a and b in every step of APACS. The known condition for the initial vector is all the points we have obtained in the process of APACS, stated as the set R shown in Eq. (6). The number of R elements is increasing after the Z axis moves to a new place and acquires a new point in every step of APACS.
yield, as shown in Eq. (10). The equations can be transformed to formula (11) by least squares conversion [30], which is actually two linear equations with two unknowns. Therefore, we can calculate the final result shown in Eq. (12). OEELSS engages all the acquired points to make the estimation, which is supposed to be more reliable than the direct calculation and will contribute to reducing the iterations in APACS prediction:
acquired after n steps in APACS;
2. Estimation of a and b No empirical value is provided for parameters a and b, and some indirect methods are applied for estimating these two parameters. At first, we regard c¯ as the actual focus position by approximation. Then we transform the Lorenz function to Eq. (8), where parameters a and b are in a linear relationship, and thus we can calculate them by substituting two points (z1 , Lz1 ) and (z2 , Lz2 ) in set R for z and Lz in Eq. (8). The result is shown in Eq. (9): a − Lz × b Lz × z − c¯ 2 ; K 1 Lz2 − K 2 Lz1 ; Lz2 − Lz1
b
1 61 6 6 .. 4.
1 ≤ i ≤ ng: (6)
1. Estimation of c Parameter c can be estimated by an empirical value such as the initial Z position, but deviation of the estimation is not stable. Thus we introduce a new dynamic method named centroid estimation (CE). In CE, we regard the FM f z as the weight distribution of a rod, and a larger f z indicates that position z is closer to the centroid of the rod because of FM’s unimodality. Obviously the focus position with the biggest FM value is just near the centroid. Or, in other words, we can use the centroid position to estimate the focus position. In APACS, only a few points are obtained, and we use these points to estimate the centroid so as to reckon the focus position c, as shown in Eq. (7): n X f zi : (7) c¯ zi × P n i1 f zi i1
a
3 3 2 −Lz1 K1 6 K2 7 −Lz2 7 7 a 7 6 6 .. 7; 7 .. 5 b 4 . 5 .
2
R fzi ; f zi j All the points i
K1 − K2 ; Lz2 − Lz1
(8)
(9)
where K 1 Lz1 × z1 − c¯ 2 , K 2 Lz2 × z2 − c¯ 2 . The above method makes use of only two points to achieve the estimation and can be easily affected by noise. In order to reduce the deviation, we introduce a new calculating method named overdetermined equations estimation by least squares solution (OEELSS). First we substitute all the points in set R for z and Lz in Eq. (9), and thus overdetermined equations
1
(10)
Kn
−Lzn
where K i Lzi × zi − c¯ 2 ;
P −P ni1 Lzi a n 2 − i1 Lzi b i1 Lzi Pn P i1 K i ; − ni1 K i × Lzi Pn n
(11)
M 1 × N 22 − M 2 × N 12 ; N 11 × N 22 − N 21 × N 12 M 1 × N 21 − M 2 × N 11 b ; N 21 × N 12 − N 11 × N 22 P n, N 12 N 21 − ni1 Lzi , N 22 a
where N 11 P n 2 i1 Lzi , M1
n X i1
K i;
M2 −
n X
K i × Lzi :
(12)
i1
5. Results
In modern microscopes the objective parfocal distance is fixed at 45 mm. Thus no matter what kind of objective lens, the focus plane is ideally positioned at a given position when the specimen on the slide is placed correctly. However, the slide surface is not absolutely flat, and the specimen on the glass slide is of finite thickness. Therefore, the microscope is supposed to be adjusted in a small range along the Z axis to get the sample in focus. We assume the microscope begins searching at a start position along the Z axis, which is near the focus plane, and all the searches only deal with the 50 μm interval where the start position is just the mid-point. Besides, our simulation also deals with the 50 μm interval in most instances. In our experiment, APACS and some other typical FS algorithms including CFS and GSS were simulated in MATLAB, and the results between different methods were compared. After the simulation, we implemented the GSS and APACS on our platform to test the focus speed and reliability. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4513
Fig. 4. Deviation of c¯ estimated by CE. The X axis shows the number of points of R generated by GSS, which are used for CE. And as the point quantity increases, the deviation decreases, because the points gathered during the GSS routine are actually approaching the focus position, which helps to stabilize the CE.
A.
Simulation Result
Before simulation, we need to generate some FMs of different areas on the sequencing chip. In practice, we explored five areas separately, and for each area, a 100 μm interval containing the focus position along the Z axis was investigated. We moved the microscope Z axis to go through the interval by 0.1 μm per step from the bottom up, and at each step an image was captured. Altogether, a total of 1001 images were obtained in each area, with which we generated the FM curve. Our simulation is mainly based on these curves. 1. Simulation of Estimation Methods As for CE, we calculated the average deviation of c¯ in every step of GSS, which is the difference between the calculated c¯ and the real focus position. The result is shown in Fig. 4, from which we can see c¯ is approaching the actual focus position as the number of GSS increases, and CE is useful and steady. Then we studied the direct calculation and OEELSS, namely the two methods estimating parameters a and b. As we do not know the exact Table 1.
values of a and b, first we use the curve-fitting method with all the 1001 points in the FM curve to approximate the values of a and b for each focus area and regard these values as the final results of the two parameters. Then we compare these two values with the estimation results from the two methods mentioned above and calculate the deviations. Furthermore, we also evaluate the two methods by comparing the computing time for fitting in GSS, as the less the time is, the better the estimation should be. Besides, we changed the searching interval from [5,55] to [45,95], where the focus position was always near 50 μm, and conducted the GSS with least squares fitting in every step. Parameter c was estimated in CE, while parameters a and b were calculated in the two ways mentioned above. The total running time and the estimation deviations of parameters a and b were recorded for each focus area. The final results for five sample locations are shown in Table 1, and it is easy to find that the average computing time for OEELSS is 19.858 s, far less than the direct calculation’s 87.066 s. The estimation deviations of parameters a and b using OEELSS are all much less than those using direct calculation in all the five focus areas. It indicates that OEELSS reduces the estimation deviations significantly, and we can consequently conclude that OEELSS is much more reliable. 2. Simulation of CFS and APACS In order to test our algorithm, we compared APACS with the three CFS strategies mentioned above, which were direct CFS, three-point least squares CFS, and five-point least squares CFS. The used points were equably selected along the interval, and the initial vector ([a, b, c]) for iteration was fixed [1,1, the midpoint of the searched interval]. As we did in the simulation of estimation methods, we changed the searching range and explored different focus areas to ensure the validity. Table 2 shows the results. When we set the accuracy requirement to 1 μm, APACS has the highest accuracy of 97.073% among the four simulated methods, and all the CFS approaches simulated are of poor performance (less than 75.000%). Thus APACS is much more stable than CFS methods. As for the mean value and the standard deviation of the focus bias, which we think can represent the
Computing Time for Fitting by Different Estimating Methods
Direct Calculation Focus Area No. 1 No. 2 No. 3 No. 4 No. 5 Average
4514
OEELSS
Direct Calculation (s)
OEELSS (s)
Deviation of a
Deviation of b
Deviation of a
Deviation of b
96.752 82.534 88.386 80.335 87.321 87.066
24.261 20.300 23.984 17.556 13.191 19.858
282.017 586.404 116.378 178.861 234.019 279.536
394.820 640.828 180.549 226.347 290.627 346.634
94.393 187.542 46.748 36.228 53.306 83.643
165.262 138.214 91.1979 58.878 79.784 106.667
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
Table 2.
Direct CFS Average bias (μm) Standard deviation of bias (μm) Number of in-focus searches Accuracy rate
CFS and APACS
Three-Point Least Squares CFS
399.672 787.252 69.000 33.659%
average accuracy and stability, respectively, to some extent, APACS’s results are much less than those of other CFS methods. We believe it is primarily due to the fitting points selection and initial vector for fitting. As mentioned above, the traditional CFS methods usually utilize FM points equably selected along the searched interval, and the number of points is fixed for each CFS algorithm, which is believed to be not adaptive enough. Furthermore, the initial vector for three-point least squares CFS and fivepoint least squares CFS is not wise enough and may lead to a wrong iteration result. The detailed data indicate that the deviations of CFS methods are quite large, sometimes reaching hundreds of micrometers among those focus failures in the simulation, which makes the mean value and standard deviation of focus bias very large. On the contrary, APACS’s fitting points are varying with the searching process, and APACS provides an estimation algorithm (CE and OEELSS) to calculate a better initial vector for fitting, which can stabilize the search and increase the accuracy. Based on the simulation results, we then analyze the specific deviation of APACS. Figure 5(a) shows the specific deviation distribution of the five focus areas, and Fig. 5(b) shows the average focus bias and the standard deviation of the bias, from which we can see that it is more precise when the focus position is next to the middle of the searched interval, and on the contrary when the focus position is located in either end of the interval, it is less accurate and larger focus bias and deviation occur, which, however, are still acceptable according to the set precision and depth of field. 3. Simulation of GSS and APACS Generally speaking, if the microscope’s total movements are reduced during the search, the focus time is also reduced. So in simulation we compared GSS and APACS in terms of the movement and focus deviation. Table 3 gives us the final results of the five areas. The movements of APACS at each focus area are much less than those of GSS, while the deviations are almost the same, which indicates that APACS is both steady and fast. Figure 6 shows the average deviation distribution of the five sample areas. APACS may lead to a slightly larger deviation occasionally compared with GSS, but the average deviations of both FS algorithms are acceptable.
930.653 989.304 119.000 58.049%
Five-Point Least Squares CFS 579.065 926.305 145.000 70.732%
APACS 0.416 0.178 199.000 97.073%
B. Measured Results on Microscope
We implemented GSS and APACS on our microscope platform, and both algorithms dealt with the 50 μm
Fig. 5. Deviation distribution of APACS. (a) This is the specific distribution of the deviation. The X axis represents the sample area, and the Y axis represents the mid-point position of the searched interval, while the Z axis represents the deviation. The actual focus position of each sample is near 50 μm. We can see that it is more precise when the focus position is next to the middle of the searched interval, and on the contrary when the focus position is located in either end of the interval, it is less accurate and larger focus bias and deviation occur, which, however, are still acceptable according to the set precision and depth of field. (b) This is the average distribution of the deviation, and the errorbar shows the standard deviation of the focus bias at each interval. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4515
Table 3.
GSS and APACS (Simulation)
Focus Area avg. numa avg. dev (μm)b avg. num avg. dev (μm) avg. num avg. dev (μm) avg. num avg. dev (μm) avg. num avg. dev (μm)
No. 1 No. 2 No. 3 No. 4 No. 5
GSS
APACS
10.561 0.520 10.390 0.576 10.585 0.492 10.634 0.473 10.561 0.476
5.317 0.381 5.366 0.592 5.049 0.274 4.902 0.386 5.024 0.447
a avg. num demonstrates the average number of movements of the Z axis. b avg. dev demonstrates the average deviation of each focus trial.
Table 4 illustrates the comparison of the two FSs at different start positions in terms of the running time, number of movements, and deviation, and each result is the average value of five focus areas. In this table, we can see that GSS is just a little more accurate than APACS, and both focus deviations are within the depth of field. However, APACS reduces the average focus time by 30.097% and cuts the average number of movements by 40.889%. Among the 45 focus trials in the experiment, APACS is faster than GSS in 40 trials with high accuracy, and the remaining five trials, in which GSS was faster, occurred when the focus start position was far away from the actual focus position. In these five trials, the average focus times of APACS and GSS are 15.475 and 15.006 s, while the average numbers of movements are 8.200 and 10.000, respectively, which indicates that the focus time of APACS, with fewer movements and more curve-fitting computation time, is only about 3.125% more than that of GSS. When the focus position is near the start position, APACS can find the proper focus plane with much higher efficiency than GSS, which agrees with the simulation results. Besides, we believe that the focus time will keep shrinking if the calculating speed and Z moving speed of our platform are increased. 6. Discussion
Fig. 6. Average deviation distribution. When the focus position is relatively too far away from the mid-point of the interval, APACS may lead to a larger deviation compared with GSS. But the deviations are still within the depth of field.
interval. The actual focus positions were within the searched intervals, and we applied two FSs at different start positions with five randomly selected focus areas on the sequencing chips. Table 4.
GSS and APACS (On-the-Spot Survey)
Running Time a
In terms of FM selection for our specimen and search strategy, normalized variance is superior to the other two algorithms. In practice, the approximation of FM is reconstructed by the curve-fitting method in APACS from the FM value at sampled Z positions, and the peak position of this function is predicted to be the desired focus. If the sampled positions are located in the peak area of the fitted focus curve, the predicted value will be close to the real situation. However, when the data used for curve fitting are positioned on the flatness region of the curve, the prediction will probably be an error. Although in general, the choice of the FM should bring a focus curve with a narrow peak [16], in order to speed up FS with reduced sampling positions, in
Number of Movements
Deviation
Focus Start Position
GSS (s)
APACS (s)
GSS
APACS
GSS (μm)
APACS (μm)
No. 1 (20 μm) No. 2 (15 μm) No. 3 (10 μm) No. 45 μm) No. 5 (0 μm) No. 6 (−5 μm) No. 7 (−10 μm) No. 8 (−15 μm) No. 9 (−20 μm) Average
15.322 15.222 15.419 15.181 15.053 15.116 15.680 15.347 15.085 15.269
14.869 11.219 11.378 7.828 7.119 7.066 9.431 12.850 14.303 10.674
10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000
8.200 6.400 6.200 4.400 4.000 4.000 5.400 7.200 7.400 5.911
0.130 0.250 0.260 0.370 0.130 0.180 0.500 0.180 0.470 0.274
0.840 0.150 0.370 0.720 0.130 0.880 0.220 0.360 0.790 0.496
a In this column, the contents in the brackets demonstrate the distance between the focus start position and the real focus position. “+” means the start position is above the focus position along the Z axis, while “−” means the opposite situation.
4516
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
this particular situation, a reconsideration of FM selection is needed. Instead of choosing a FM whose curve has a narrow peak, one with a wide FWHM is preferred. When the search APACS starts from arbitrary position with fewer sampled positions, it is more likely that some of the collected data will lie within the wide peak, which will result in a better focus prediction. However, if the sampled data are fitted to a curve with narrow peak, the risk that these data locate outside a narrow peak will increase. And if that happens, the peak finding procedure will probably fail. Generally, normalized variance is of the largest FWHM, while autocorrelation has the narrowest peak. Hence, we suggest normalized variance as the most appropriate FM. As for FS, we find that the combination of CE and OEELSS is an effective estimation strategy to calculate the initial vector for APACS from the above results, which contributes to reducing both the iterations and the focus time. Besides, the proposed APACS is more stable than different CFS methods according to the accuracy rates comparison in simulation (shown in Table 2), and the average focus bias and standard deviation of bias results also support this point. The stability of APACS is almost the same as that of GSS from the results in Section 5.A.3. When it comes to focus time and efficiency, APACS behaves better than GSS in most cases. Nevertheless when the focus position is not close to our start position, the procedure becomes a bit more difficult and APACS’s prediction may encounter a relatively bigger deviation or longer processing time occasionally. 7. Conclusion
In this paper we first analyzed FM for the magnetic beads sample and chose the normalized variance as the FM algorithm for the passive focus. Then we proposed a new fast FS algorithm named APACS, based on GSS and least squares CFS so as to improve the speed of the passive focus. Also a new estimation strategy, the combination of CE and OEELSS, was put forward to calculate the initial vector for APACS prediction. This estimation method is actually an important part of APACS and helps to accelerate the focus. Then we did some simulation and on-the-spot survey with CFS, GSS, and our APACS. In simulation APACS was superior to all the traditional CFSs in stability and accuracy. Besides, APACS needed only a few more searching steps than any CFS in most cases. Compared with GSS, the simulation indicates that APACS not only maintains the stability and accuracy but also reduces the searching iterations. In the on-the-spot survey, APACS still shows strong robustness. The average focus time was 30.097% less than that of GSS, and the average searching steps were 40.889% less. Meanwhile the average deviation was only 0.496 μm. In the 45 trials, the biases were within the depth of field, and the captured images were clear enough for future research.
After analyzing all the results, we finally conclude that APACS is very suitable for the DNA sequencing platform with fluorescent magnetic beads. This work was financially supported by the National Science Foundation of China (grant no. 30600133). References 1. L. Shih, “Autofocus survey: a comparison of algorithms,” Proc. SPIE 6502, 65020B (2007). 2. Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing in computer microscopy: selecting the optimal focus algorithm,” Microsc. Res. Tech. 65, 139–149 (2004). 3. J. F. Schlag, A. C. Sanderson, C. P. Neuman, and F. C. Wimberly, Implementation of Automatic Focusing Algorithms for a Computer Vision System with Camera Control (Citeseer, 1983). 4. N. Kehtarnavaz and H. J. Oh, “Development and real-time implementation of a rule-based auto-focus algorithm,” RealTime Imaging 9, 197–203 (2003). 5. C.-Y. Chen, R.-C. Hwang, and Y.-J. Chen, “A passive auto-focus camera control system,” Appl. Soft Comput. 10, 296–303 (2010). 6. H.-L. Shen, Z.-H. Zheng, W. Wang, X. Du, S.-J. Shao, and J. H. Xin, “Autofocus for multispectral camera using focus symmetry,” Appl. Opt. 51, 2616–2623 (2012). 7. M. Gamadia, V. Peddigari, N. Kehtarnavaz, S.-Y. Lee, and G. Cook, “Real-time implementation of autofocus on the TI DSC processor,” Proc. SPIE 5297, 10–18 (2004). 8. R. Redondo, G. Bueno, J. C. Valdiviezo, R. Nava, G. Cristobal, O. Deniz, M. Garcia-Rojo, J. Salido, M. del Milagro Fernandez , J. Vidal, and B. Escalante-Ramirez, “Autofocus evaluation for brightfield microscopy pathology,” J. Biomed. Opt. 17, 036008 (2012). 9. O. Osibote, R. Dendere, S. Krishnan, and T. Douglas, “Automated focusing in bright-field microscopy for tuberculosis detection,” J. Microsc. 240, 155–163 (2010). 10. F. C. Groen, I. T. Young, and G. Ligthart, “A comparison of different focus functions for use in autofocus algorithms,” Cytometry 6, 81–91 (1985). 11. J. M. Tenenbaum, “Accommodation in computer vision,” Ph.D. thesis (Stanford University, 1970). 12. T. Yeo, S. Ong, and R. Sinniah, “Autofocusing for tissue microscopy,” in Image and Vision Computing (1993), pp. 629–639. 13. M. A. Bueno-Ibarra and L. Acho, “Fast autofocus algorithm for automated microscopes,” Opt. Eng. 44, 063601 (2005). 14. J. H. Price and D. A. Gough, “Comparison of phase-contrast and fluorescence digital autofocus for scanning microscopy,” Cytometry 16, 283–297 (1994). 15. D. Vollath, “Automatic focusing by correlative methods,” J. Microsc. 147, 279–288 (1987). 16. A. Santos, C. Ortiz de Solorzano, J. J. Vaquero, J. Pena, N. Malpica, and F. Del Pozo, “Evaluation of autofocus functions in molecular cytogenetic analysis,” J. Microsc. 188, 264–272 (1997). 17. Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing algorithm selection in computer microscopy,” in 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2005), pp. 70–76. 18. Y. Yao, B. Abidi, N. Doggaz, and M. Abidi, “Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images,” in Defense and Security Symposium (International Society for Optics and Photonics, 2006), p. 62460G. 19. M. Rahman and N. Kehtarnavaz, “Real-time face-priority auto focus for digital and cell-phone cameras,” IEEE Trans. Consum. Electron. 54, 1506–1513 (2008). 20. J. He, R. Zhou, and Z. Hong, “Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera,” IEEE Trans. Consum. Electron. 49, 257–262 (2003). 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4517
21. K. Ooi, K. Izumi, M. Nozaki, and I. Takeda, “An advanced autofocus system for video camera using quasi condition reasoning,” IEEE Trans. Consum. Electron. 36, 526–530 (1990). 22. R. A. Jarvis, “Focus optimization criteria for computer image processing,” Microscope 24, 163–180 (1976). 23. E. P. Krotkov, Active Computer Vision by Cooperative Focus and Stereo (Springer-Verlag, 1989). 24. J. Baina and J. Dublet, “Automatic focus and iris control for video cameras,” in Fifth International Conference on Image Processing and its Applications (IET, 1995), pp. 232–235. 25. M. Subbarao and J.-K. Tyan, “Selecting the optimal focus measure for autofocusing and depth-from-focus,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 864–870 (1998).
4518
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
26. E. Krotkov, “Focusing,” Int. J. Comput. Vis. 1, 223–237 (1956). 27. D. L. Marks, A. L. Oldenburg, J. J. Reynolds, and S. A. Boppart, “Autofocus algorithm for dispersion correction in optical coherence tomography,” Appl. Opt. 42, 3038–3046 (2003). 28. Q. Wu, F. Merchant, and K. Castleman, Microscope Image Processing (Elsevier, 2010). 29. S. Yazdanfar, K. B. Kenny, K. Tasimi, A. D. Corwin, E. L. Dixon, and R. J. Filkins, “Simple and robust image-based autofocusing for digital microscopy,” Opt. Express 16, 8670–8677 (2008). 30. H. Anton, Elementary Linear Algebra (Wiley, 2010).
Autofocus is an important technique for high-speed image acquisition in the second-generation DNA sequencing system, and this paper studies the passive focus algorithm for the system, which consists of two parts: focus measurement (FM) and focus search (FS). Based on the properties of DNA chips’ images, we choose the normalized variance as the FM algorithm and develop a new robust FS named adaptive prediction approximation combined search (APACS). APACS utilizes golden section search (GSS) to approximate the focus position and engages the curve-fitting search (CFS) to predict the position simultaneously in every step of GSS. When the difference between consecutive predictions meets the set precision, the search finishes. Otherwise, it ends as GSS. In APACS, we also propose an estimation method, named the combination of centroid estimation and overdetermined equations estimation by least squares solution, to calculate the initial vector for the nonlinear equations in APACS prediction, which reduces the iterations and accelerates the search. The simulation and measured results demonstrate that APACS not only maintains the stability but also reduces the focus time compared with GSS and CFS, which indicates APACS is a robust and fast FS for the fluorescence microscope in a sequencing system. © 2014 Optical Society of America OCIS codes: (170.2520) Fluorescence microscopy; (170.3890) Medical optics instrumentation; (170.3880) Medical and biological imaging. http://dx.doi.org/10.1364/AO.53.004509
1. Introduction
Today, optical microscopes are widely used in biomedical engineering, medicine, hylology, industrial production, etc. They bring the micro-world to our view and help to observe micro-objects, such as cells and viruses, which cannot be seen with naked eyes. One microscope, termed a fluorescence microscope, uses fluorescence to generate images, and is widely applied in the second-generation DNA sequencing system to obtain images of sequencing chips for future analysis. In order to guarantee the image quality and reduce acquisition time, autofocusing 1559-128X/14/204509-10$15.00/0 © 2014 Optical Society of America
must be realized with the fluorescence microscope. However, compared with the traditional autofocus methods, it is challenging to achieve an accurate focus within a short time in the system. In principle, there are two strategies in an autofocus process: active autofocus and passive autofocus. Active autofocus mainly uses different sensors or measurement tools to determine the distance between the lens and the object, while passive autofocus relies on the image information only, taking advantage of image clarity to detect the focus position [1–5]. Generally, microscopes are not equipped with active focus parts and it is very expensive to integrate such a part into the microscope. Thus the passive focus is more adaptive to the sequencing system. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4509
However, traditional passive focus algorithms are inefficient and time-consuming, so we developed a fast and robust one to meet the requirement of the fluorescence microscope in the sequencing systems. The passive autofocus algorithm consists of two components: focus measurement (FM) and focus search (FS) [1]. Theoretically microscopes can be simplified as a lens model and if the view position in the image space is away from the focal plane, any viewed images turn out to be blurred [6]. Passive focus takes advantage of this and regards the clarity of captured images at different Z positions as the criterion of focus level, which is called the FM [1,7–9]. Then an FS procedure is applied to iterate through the candidates of FM values to select the focus position [1,4,5]. Our study is mainly concerned with developing a new FS algorithm. Meanwhile, a simple selection of FM algorithms is also taken into consideration. 2. Material and Experimental Platform
Pink SPHERO Fluorescent Carboxyl Magnetic Particles with a diameter of 1 μm (produced by the Spherotech company) are used as the specimen in our experiment. Maximum absorption for these beads occurs at a wavelength of approximately 560 nm and emission at 580 nm. We use the mercury lamp as the excitation light source and apply a Cool SNAP HQ2 CCD (produced by the Photometrics) to capture images, which has 1392H × 1040V pixels on a 8.98H × 6.71V mm sensor. The microscope is Nikon TE-2000-E, and the minimal movement step length of the Z axis is 50 nm. Also, the objective lens magnification is 20. A computer is utilized to control the microscope’s Z movement, CCD photographing, and algorithm implementation in our platform, with an Intel(R) Pentium(R)D CPU at operating frequency 2.6 GHz. Further, the available memory is 3.37 GB, and the operating system is Windows XP Professional. The sample images captured by the system at different Z positions from the bottom up are shown in Fig. 1.
Fig. 1. Sample images captured by the microscope system at different Z positions from the bottom up. The exposure time is 300 ms, and the bar in each image is 20 μm. As there exists depth of field in the system, (c) and (d) are two images in focus, while the other four are not. 4510
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
3. Focus Measurement
FM is actually a function of Z position, and each FM value is generated from a captured image at a certain Z position by FM algorithms, which have been widely studied in the previous literature, and no less than 15 FM algorithms have been proposed. FMs calculated by different algorithms are supposed to meet at least three standards: unimodality, accuracy, and reproducibility. Other properties are also important, such as calculating speed and noise immunity [10]. Sobel-Ten [3,6,11], normalized variance [12,13], and autocorrelation [3,10,14–16] are the most commonly used FM algorithms, which meet all three standards. Sun et al. compared 18 algorithms including the above three and found normalized variance is the best one according to its performance in different applications such as bright field, phase contrast, or differential interference contrast (DIC) [17]. Santos et al. found that autocorrelation is the optimal FM for fluorescence microscopy applications [16]. In our experiment the three FMs mentioned above are tested for our specimen, and their function graphs are shown in Fig. 2, which indicates that all three FMs are well-functioned. However, normalized variance is outstanding in this application, because it is easy to see that its peak width, namely full width at half-maximum (FWHM), among the three is the largest, and its monotony interval on each side of the peak is also the longest. Thus normalized variance confers an extended range near the focus position, which is more robust and more suitable for the curve-fitting strategy in our FS
Fig. 2. FM curve of normalized variance, Sobel-Ten, and autocorrelation in 100 μm interval. Each curve consists of an FM value at 1001 positions along the Z axis, and the step size is 0.1 μm. The Y axis represents the normalized FM value, and the peaks’ Z position indicates the focus position. As we can see, the three FMs are all well-functioned. The peak width at half height of normalized variance among the three ones is the largest, and its monotony interval on each side of the peak is also the longest, making it more suitable for the curve-fitting strategy in our focus search APACS.
algorithm, adaptive prediction approximation combined search (APACS), mentioned next. The detailed reason is given in Section 6, and normalized variance is chosen as the FM algorithm in this paper. Its formula is shown in Eq. (1) [12,13]: F var
XX
1 M N f x; y
x
f x; y − f x; y2 :
(1)
y
In Eq. (1), f x; y is the gray level at pixel x; y in the image, and f x; y is the average of f x; y. M and N represent the image’s row and column number, respectively, while F var stands for the FM value of the captured image. 4. Focus Search A.
FS Algorithm Review
In passive focus the ideal FM is a unimodal function, of which the peak’s Z position indicates the focus position, and then FS is applied to locate the peak. The most commonly used FS algorithms proposed in the previous literature are global search [18,19], hillclimbing search [3,6,20–22], binary search [23,24], curve-fitting search (CFS) [25], Fibonacci search (FibS) [23,26], and golden section search (GSS) [27], among which CFS is the fastest in theory. However, frequent errors of CFS are a problem. CFS utilizes some points in the FM curve to do the fitting and get its analytic formula, from which the focus position is calculated. Before fitting, we must decide the function type by which FM can be approximated, such as the quadratic function [28], Gaussian function [28], and Lorentz function [29], which have been proposed and proved to be effective in some cases. The formulas and the calculated focus positions are shown in Eqs. (2)–(4), respectively: Fz az2 bz c;
Gz a × e−
z−b2 c
;
a ; b z − c2
b the focus position∶ − ; 2a
(2)
the focus position∶ b;
(3)
When conducting least squares fitting, the quadratic function is the worst one because of its large fitting residual. Meanwhile Lorenz’s computational complexity is less than that of Gaussian, and it is the most suitable curve used in CFS. FibS and GSS are more reliable than CFS. They are two similar searching algorithms and share the same strategy to approximate the focus position by comparing the FM value of two points within the searching range to cut down part of the interval where the focus position is not located. Therefore, the final result of FibS and GSS is the interval including the focus position rather than a definite position value. Before applying FibS, the iterations of the search should be set based on the required accuracy. Theoretically, FibS is the optimum search and demands the fewest iterations, but when applied in autofocus, it is less accurate as the microscope cannot move to the exact position calculated by FibS. Therefore, more iterations are required to meet the set precision. Compared with FibS, GSS is an improved method, which does not finish until the length of the target interval meets the set precision. If the initial interval is [a, b], in theory, the target interval will be 0.618n × b − a after an n-step search, which is almost as large as that of FibS. The detailed process of GSS is shown in Fig. 3(a), and GSS uses two golden section points as the compared points in each interval [a, b], which are defined in Eq. (5): p 5−1 ; Gold ratio 2 G1 a 1 − Gold ratio × b − a; G2 a Gold ratio × b − a:
(5)
However, on our platform none of these methods meet the requirement of the sequencing system. For instance, CFS is not robust enough and GSS is not fast enough. Therefore, we developed a new one named APACS, which combines the merits of GSS and CFS, resulting in a faster speed with high stability.
(4)
B. Proposed Algorithm: Adaptive Prediction Approximation Combined Search
The three above functions all have three parameters, and at least three points in the FM curve, each resulting in an equation, are required to conduct the fitting. We can calculate all three parameters simply by three equations, or by least squares fitting with more than three points. The direct calculation by three equations is fast but inaccurate because of random noise when collecting images. On the contrary, the least squares fitting is more reliable, and its accuracy greatly depends on the number of points used for fitting, which are highly related to the searching time.
In GSS, we compare the FM value of the interval’s golden section points and obtain a shorter interval in which the focus position is still located. So in every step only two new points are calculated and engaged, while the other acquired points in the previous steps are abandoned, which also indicate the focus position to some extent. If we can make the most of the abandoned points when employing GSS, we can increase the convergence rate. Furthermore, we notice that the acquired points can be used for CFS, so we put forward a new FS algorithm named APACS based on CFS and GSS, where CFS is in fact a prediction search strategy and GSS is an approximation one.
Lz
the focus position∶ c:
10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4511
Fig. 3. Flowcharts of GSS and APACS. (a) Flowchart of GSS. The ak ; bk denotes the searched range after k steps. G1k and G2k denote the golden section points of the interval ak ; bk . f G1k and f G2k represent the FM value of the two golden section points. AR denotes the accuracy requirement, and k denotes the searching steps. (b) Flowchart of APACS. The variable denoted here is same as (a) except for focus_now and focus_pre, which represent the two consecutive predictions of APACS. The parts highlighted in red are some important steps that are different from GSS.
Besides, among the two ways in CFS, least squares fitting is chosen with a Lorenz function. APACS engages all the acquired points to do CFS in every step of GSS to predict the focus position. If the focus position is not within the interval, it is a wrong prediction and of no use, which usually happens at the beginning of the focus. As the search goes on and more points are obtained, the fitting deviation decreases and the CFS prediction slowly approaches the focus position. If the absolute difference between the two consecutive predictions is less than the set precision, APACS finishes successfully. However, if the above situation does not occur, unfortunately, the focus procedure finishes, acting like GSS. It guarantees the focus precision but actually turns out to be a little slower than GSS because of the vast computation in least squares fitting. After testing it is manifested that such a case only happens occasionally and APACS is faster than 4512
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
GSS in most instances. The detailed process is shown in Fig. 3(b). C.
Initial Vector for Fitting in APACS
The curve-fitting prediction in APACS brings forth another problem that least squares CFS is supposed to solve the nonlinear equations, which will appear in the fitting process. Nonlinear equations are usually solved by iteration, such as Jacob iteration, and the iteration process starts with an initial vector for the unknown parameters. Thus an improper initial vector can lead to an increase of iterations or even a wrong iteration result in the end. So we proposed some estimation algorithms to get the best initial vector based on the points acquired in every step of APACS. As we can see in Eq. (4), parameter c represents the focus position, while a and b are the two other related parameters when calculating parameter c.
However, it is quite difficult to estimate all three unknown parameters at the same time. So we plan to estimate the c value first and use the estimation of c to estimate a and b in every step of APACS. The known condition for the initial vector is all the points we have obtained in the process of APACS, stated as the set R shown in Eq. (6). The number of R elements is increasing after the Z axis moves to a new place and acquires a new point in every step of APACS.
yield, as shown in Eq. (10). The equations can be transformed to formula (11) by least squares conversion [30], which is actually two linear equations with two unknowns. Therefore, we can calculate the final result shown in Eq. (12). OEELSS engages all the acquired points to make the estimation, which is supposed to be more reliable than the direct calculation and will contribute to reducing the iterations in APACS prediction:
acquired after n steps in APACS;
2. Estimation of a and b No empirical value is provided for parameters a and b, and some indirect methods are applied for estimating these two parameters. At first, we regard c¯ as the actual focus position by approximation. Then we transform the Lorenz function to Eq. (8), where parameters a and b are in a linear relationship, and thus we can calculate them by substituting two points (z1 , Lz1 ) and (z2 , Lz2 ) in set R for z and Lz in Eq. (8). The result is shown in Eq. (9): a − Lz × b Lz × z − c¯ 2 ; K 1 Lz2 − K 2 Lz1 ; Lz2 − Lz1
b
1 61 6 6 .. 4.
1 ≤ i ≤ ng: (6)
1. Estimation of c Parameter c can be estimated by an empirical value such as the initial Z position, but deviation of the estimation is not stable. Thus we introduce a new dynamic method named centroid estimation (CE). In CE, we regard the FM f z as the weight distribution of a rod, and a larger f z indicates that position z is closer to the centroid of the rod because of FM’s unimodality. Obviously the focus position with the biggest FM value is just near the centroid. Or, in other words, we can use the centroid position to estimate the focus position. In APACS, only a few points are obtained, and we use these points to estimate the centroid so as to reckon the focus position c, as shown in Eq. (7): n X f zi : (7) c¯ zi × P n i1 f zi i1
a
3 3 2 −Lz1 K1 6 K2 7 −Lz2 7 7 a 7 6 6 .. 7; 7 .. 5 b 4 . 5 .
2
R fzi ; f zi j All the points i
K1 − K2 ; Lz2 − Lz1
(8)
(9)
where K 1 Lz1 × z1 − c¯ 2 , K 2 Lz2 × z2 − c¯ 2 . The above method makes use of only two points to achieve the estimation and can be easily affected by noise. In order to reduce the deviation, we introduce a new calculating method named overdetermined equations estimation by least squares solution (OEELSS). First we substitute all the points in set R for z and Lz in Eq. (9), and thus overdetermined equations
1
(10)
Kn
−Lzn
where K i Lzi × zi − c¯ 2 ;
P −P ni1 Lzi a n 2 − i1 Lzi b i1 Lzi Pn P i1 K i ; − ni1 K i × Lzi Pn n
(11)
M 1 × N 22 − M 2 × N 12 ; N 11 × N 22 − N 21 × N 12 M 1 × N 21 − M 2 × N 11 b ; N 21 × N 12 − N 11 × N 22 P n, N 12 N 21 − ni1 Lzi , N 22 a
where N 11 P n 2 i1 Lzi , M1
n X i1
K i;
M2 −
n X
K i × Lzi :
(12)
i1
5. Results
In modern microscopes the objective parfocal distance is fixed at 45 mm. Thus no matter what kind of objective lens, the focus plane is ideally positioned at a given position when the specimen on the slide is placed correctly. However, the slide surface is not absolutely flat, and the specimen on the glass slide is of finite thickness. Therefore, the microscope is supposed to be adjusted in a small range along the Z axis to get the sample in focus. We assume the microscope begins searching at a start position along the Z axis, which is near the focus plane, and all the searches only deal with the 50 μm interval where the start position is just the mid-point. Besides, our simulation also deals with the 50 μm interval in most instances. In our experiment, APACS and some other typical FS algorithms including CFS and GSS were simulated in MATLAB, and the results between different methods were compared. After the simulation, we implemented the GSS and APACS on our platform to test the focus speed and reliability. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4513
Fig. 4. Deviation of c¯ estimated by CE. The X axis shows the number of points of R generated by GSS, which are used for CE. And as the point quantity increases, the deviation decreases, because the points gathered during the GSS routine are actually approaching the focus position, which helps to stabilize the CE.
A.
Simulation Result
Before simulation, we need to generate some FMs of different areas on the sequencing chip. In practice, we explored five areas separately, and for each area, a 100 μm interval containing the focus position along the Z axis was investigated. We moved the microscope Z axis to go through the interval by 0.1 μm per step from the bottom up, and at each step an image was captured. Altogether, a total of 1001 images were obtained in each area, with which we generated the FM curve. Our simulation is mainly based on these curves. 1. Simulation of Estimation Methods As for CE, we calculated the average deviation of c¯ in every step of GSS, which is the difference between the calculated c¯ and the real focus position. The result is shown in Fig. 4, from which we can see c¯ is approaching the actual focus position as the number of GSS increases, and CE is useful and steady. Then we studied the direct calculation and OEELSS, namely the two methods estimating parameters a and b. As we do not know the exact Table 1.
values of a and b, first we use the curve-fitting method with all the 1001 points in the FM curve to approximate the values of a and b for each focus area and regard these values as the final results of the two parameters. Then we compare these two values with the estimation results from the two methods mentioned above and calculate the deviations. Furthermore, we also evaluate the two methods by comparing the computing time for fitting in GSS, as the less the time is, the better the estimation should be. Besides, we changed the searching interval from [5,55] to [45,95], where the focus position was always near 50 μm, and conducted the GSS with least squares fitting in every step. Parameter c was estimated in CE, while parameters a and b were calculated in the two ways mentioned above. The total running time and the estimation deviations of parameters a and b were recorded for each focus area. The final results for five sample locations are shown in Table 1, and it is easy to find that the average computing time for OEELSS is 19.858 s, far less than the direct calculation’s 87.066 s. The estimation deviations of parameters a and b using OEELSS are all much less than those using direct calculation in all the five focus areas. It indicates that OEELSS reduces the estimation deviations significantly, and we can consequently conclude that OEELSS is much more reliable. 2. Simulation of CFS and APACS In order to test our algorithm, we compared APACS with the three CFS strategies mentioned above, which were direct CFS, three-point least squares CFS, and five-point least squares CFS. The used points were equably selected along the interval, and the initial vector ([a, b, c]) for iteration was fixed [1,1, the midpoint of the searched interval]. As we did in the simulation of estimation methods, we changed the searching range and explored different focus areas to ensure the validity. Table 2 shows the results. When we set the accuracy requirement to 1 μm, APACS has the highest accuracy of 97.073% among the four simulated methods, and all the CFS approaches simulated are of poor performance (less than 75.000%). Thus APACS is much more stable than CFS methods. As for the mean value and the standard deviation of the focus bias, which we think can represent the
Computing Time for Fitting by Different Estimating Methods
Direct Calculation Focus Area No. 1 No. 2 No. 3 No. 4 No. 5 Average
4514
OEELSS
Direct Calculation (s)
OEELSS (s)
Deviation of a
Deviation of b
Deviation of a
Deviation of b
96.752 82.534 88.386 80.335 87.321 87.066
24.261 20.300 23.984 17.556 13.191 19.858
282.017 586.404 116.378 178.861 234.019 279.536
394.820 640.828 180.549 226.347 290.627 346.634
94.393 187.542 46.748 36.228 53.306 83.643
165.262 138.214 91.1979 58.878 79.784 106.667
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
Table 2.
Direct CFS Average bias (μm) Standard deviation of bias (μm) Number of in-focus searches Accuracy rate
CFS and APACS
Three-Point Least Squares CFS
399.672 787.252 69.000 33.659%
average accuracy and stability, respectively, to some extent, APACS’s results are much less than those of other CFS methods. We believe it is primarily due to the fitting points selection and initial vector for fitting. As mentioned above, the traditional CFS methods usually utilize FM points equably selected along the searched interval, and the number of points is fixed for each CFS algorithm, which is believed to be not adaptive enough. Furthermore, the initial vector for three-point least squares CFS and fivepoint least squares CFS is not wise enough and may lead to a wrong iteration result. The detailed data indicate that the deviations of CFS methods are quite large, sometimes reaching hundreds of micrometers among those focus failures in the simulation, which makes the mean value and standard deviation of focus bias very large. On the contrary, APACS’s fitting points are varying with the searching process, and APACS provides an estimation algorithm (CE and OEELSS) to calculate a better initial vector for fitting, which can stabilize the search and increase the accuracy. Based on the simulation results, we then analyze the specific deviation of APACS. Figure 5(a) shows the specific deviation distribution of the five focus areas, and Fig. 5(b) shows the average focus bias and the standard deviation of the bias, from which we can see that it is more precise when the focus position is next to the middle of the searched interval, and on the contrary when the focus position is located in either end of the interval, it is less accurate and larger focus bias and deviation occur, which, however, are still acceptable according to the set precision and depth of field. 3. Simulation of GSS and APACS Generally speaking, if the microscope’s total movements are reduced during the search, the focus time is also reduced. So in simulation we compared GSS and APACS in terms of the movement and focus deviation. Table 3 gives us the final results of the five areas. The movements of APACS at each focus area are much less than those of GSS, while the deviations are almost the same, which indicates that APACS is both steady and fast. Figure 6 shows the average deviation distribution of the five sample areas. APACS may lead to a slightly larger deviation occasionally compared with GSS, but the average deviations of both FS algorithms are acceptable.
930.653 989.304 119.000 58.049%
Five-Point Least Squares CFS 579.065 926.305 145.000 70.732%
APACS 0.416 0.178 199.000 97.073%
B. Measured Results on Microscope
We implemented GSS and APACS on our microscope platform, and both algorithms dealt with the 50 μm
Fig. 5. Deviation distribution of APACS. (a) This is the specific distribution of the deviation. The X axis represents the sample area, and the Y axis represents the mid-point position of the searched interval, while the Z axis represents the deviation. The actual focus position of each sample is near 50 μm. We can see that it is more precise when the focus position is next to the middle of the searched interval, and on the contrary when the focus position is located in either end of the interval, it is less accurate and larger focus bias and deviation occur, which, however, are still acceptable according to the set precision and depth of field. (b) This is the average distribution of the deviation, and the errorbar shows the standard deviation of the focus bias at each interval. 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4515
Table 3.
GSS and APACS (Simulation)
Focus Area avg. numa avg. dev (μm)b avg. num avg. dev (μm) avg. num avg. dev (μm) avg. num avg. dev (μm) avg. num avg. dev (μm)
No. 1 No. 2 No. 3 No. 4 No. 5
GSS
APACS
10.561 0.520 10.390 0.576 10.585 0.492 10.634 0.473 10.561 0.476
5.317 0.381 5.366 0.592 5.049 0.274 4.902 0.386 5.024 0.447
a avg. num demonstrates the average number of movements of the Z axis. b avg. dev demonstrates the average deviation of each focus trial.
Table 4 illustrates the comparison of the two FSs at different start positions in terms of the running time, number of movements, and deviation, and each result is the average value of five focus areas. In this table, we can see that GSS is just a little more accurate than APACS, and both focus deviations are within the depth of field. However, APACS reduces the average focus time by 30.097% and cuts the average number of movements by 40.889%. Among the 45 focus trials in the experiment, APACS is faster than GSS in 40 trials with high accuracy, and the remaining five trials, in which GSS was faster, occurred when the focus start position was far away from the actual focus position. In these five trials, the average focus times of APACS and GSS are 15.475 and 15.006 s, while the average numbers of movements are 8.200 and 10.000, respectively, which indicates that the focus time of APACS, with fewer movements and more curve-fitting computation time, is only about 3.125% more than that of GSS. When the focus position is near the start position, APACS can find the proper focus plane with much higher efficiency than GSS, which agrees with the simulation results. Besides, we believe that the focus time will keep shrinking if the calculating speed and Z moving speed of our platform are increased. 6. Discussion
Fig. 6. Average deviation distribution. When the focus position is relatively too far away from the mid-point of the interval, APACS may lead to a larger deviation compared with GSS. But the deviations are still within the depth of field.
interval. The actual focus positions were within the searched intervals, and we applied two FSs at different start positions with five randomly selected focus areas on the sequencing chips. Table 4.
GSS and APACS (On-the-Spot Survey)
Running Time a
In terms of FM selection for our specimen and search strategy, normalized variance is superior to the other two algorithms. In practice, the approximation of FM is reconstructed by the curve-fitting method in APACS from the FM value at sampled Z positions, and the peak position of this function is predicted to be the desired focus. If the sampled positions are located in the peak area of the fitted focus curve, the predicted value will be close to the real situation. However, when the data used for curve fitting are positioned on the flatness region of the curve, the prediction will probably be an error. Although in general, the choice of the FM should bring a focus curve with a narrow peak [16], in order to speed up FS with reduced sampling positions, in
Number of Movements
Deviation
Focus Start Position
GSS (s)
APACS (s)
GSS
APACS
GSS (μm)
APACS (μm)
No. 1 (20 μm) No. 2 (15 μm) No. 3 (10 μm) No. 45 μm) No. 5 (0 μm) No. 6 (−5 μm) No. 7 (−10 μm) No. 8 (−15 μm) No. 9 (−20 μm) Average
15.322 15.222 15.419 15.181 15.053 15.116 15.680 15.347 15.085 15.269
14.869 11.219 11.378 7.828 7.119 7.066 9.431 12.850 14.303 10.674
10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000
8.200 6.400 6.200 4.400 4.000 4.000 5.400 7.200 7.400 5.911
0.130 0.250 0.260 0.370 0.130 0.180 0.500 0.180 0.470 0.274
0.840 0.150 0.370 0.720 0.130 0.880 0.220 0.360 0.790 0.496
a In this column, the contents in the brackets demonstrate the distance between the focus start position and the real focus position. “+” means the start position is above the focus position along the Z axis, while “−” means the opposite situation.
4516
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
this particular situation, a reconsideration of FM selection is needed. Instead of choosing a FM whose curve has a narrow peak, one with a wide FWHM is preferred. When the search APACS starts from arbitrary position with fewer sampled positions, it is more likely that some of the collected data will lie within the wide peak, which will result in a better focus prediction. However, if the sampled data are fitted to a curve with narrow peak, the risk that these data locate outside a narrow peak will increase. And if that happens, the peak finding procedure will probably fail. Generally, normalized variance is of the largest FWHM, while autocorrelation has the narrowest peak. Hence, we suggest normalized variance as the most appropriate FM. As for FS, we find that the combination of CE and OEELSS is an effective estimation strategy to calculate the initial vector for APACS from the above results, which contributes to reducing both the iterations and the focus time. Besides, the proposed APACS is more stable than different CFS methods according to the accuracy rates comparison in simulation (shown in Table 2), and the average focus bias and standard deviation of bias results also support this point. The stability of APACS is almost the same as that of GSS from the results in Section 5.A.3. When it comes to focus time and efficiency, APACS behaves better than GSS in most cases. Nevertheless when the focus position is not close to our start position, the procedure becomes a bit more difficult and APACS’s prediction may encounter a relatively bigger deviation or longer processing time occasionally. 7. Conclusion
In this paper we first analyzed FM for the magnetic beads sample and chose the normalized variance as the FM algorithm for the passive focus. Then we proposed a new fast FS algorithm named APACS, based on GSS and least squares CFS so as to improve the speed of the passive focus. Also a new estimation strategy, the combination of CE and OEELSS, was put forward to calculate the initial vector for APACS prediction. This estimation method is actually an important part of APACS and helps to accelerate the focus. Then we did some simulation and on-the-spot survey with CFS, GSS, and our APACS. In simulation APACS was superior to all the traditional CFSs in stability and accuracy. Besides, APACS needed only a few more searching steps than any CFS in most cases. Compared with GSS, the simulation indicates that APACS not only maintains the stability and accuracy but also reduces the searching iterations. In the on-the-spot survey, APACS still shows strong robustness. The average focus time was 30.097% less than that of GSS, and the average searching steps were 40.889% less. Meanwhile the average deviation was only 0.496 μm. In the 45 trials, the biases were within the depth of field, and the captured images were clear enough for future research.
After analyzing all the results, we finally conclude that APACS is very suitable for the DNA sequencing platform with fluorescent magnetic beads. This work was financially supported by the National Science Foundation of China (grant no. 30600133). References 1. L. Shih, “Autofocus survey: a comparison of algorithms,” Proc. SPIE 6502, 65020B (2007). 2. Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing in computer microscopy: selecting the optimal focus algorithm,” Microsc. Res. Tech. 65, 139–149 (2004). 3. J. F. Schlag, A. C. Sanderson, C. P. Neuman, and F. C. Wimberly, Implementation of Automatic Focusing Algorithms for a Computer Vision System with Camera Control (Citeseer, 1983). 4. N. Kehtarnavaz and H. J. Oh, “Development and real-time implementation of a rule-based auto-focus algorithm,” RealTime Imaging 9, 197–203 (2003). 5. C.-Y. Chen, R.-C. Hwang, and Y.-J. Chen, “A passive auto-focus camera control system,” Appl. Soft Comput. 10, 296–303 (2010). 6. H.-L. Shen, Z.-H. Zheng, W. Wang, X. Du, S.-J. Shao, and J. H. Xin, “Autofocus for multispectral camera using focus symmetry,” Appl. Opt. 51, 2616–2623 (2012). 7. M. Gamadia, V. Peddigari, N. Kehtarnavaz, S.-Y. Lee, and G. Cook, “Real-time implementation of autofocus on the TI DSC processor,” Proc. SPIE 5297, 10–18 (2004). 8. R. Redondo, G. Bueno, J. C. Valdiviezo, R. Nava, G. Cristobal, O. Deniz, M. Garcia-Rojo, J. Salido, M. del Milagro Fernandez , J. Vidal, and B. Escalante-Ramirez, “Autofocus evaluation for brightfield microscopy pathology,” J. Biomed. Opt. 17, 036008 (2012). 9. O. Osibote, R. Dendere, S. Krishnan, and T. Douglas, “Automated focusing in bright-field microscopy for tuberculosis detection,” J. Microsc. 240, 155–163 (2010). 10. F. C. Groen, I. T. Young, and G. Ligthart, “A comparison of different focus functions for use in autofocus algorithms,” Cytometry 6, 81–91 (1985). 11. J. M. Tenenbaum, “Accommodation in computer vision,” Ph.D. thesis (Stanford University, 1970). 12. T. Yeo, S. Ong, and R. Sinniah, “Autofocusing for tissue microscopy,” in Image and Vision Computing (1993), pp. 629–639. 13. M. A. Bueno-Ibarra and L. Acho, “Fast autofocus algorithm for automated microscopes,” Opt. Eng. 44, 063601 (2005). 14. J. H. Price and D. A. Gough, “Comparison of phase-contrast and fluorescence digital autofocus for scanning microscopy,” Cytometry 16, 283–297 (1994). 15. D. Vollath, “Automatic focusing by correlative methods,” J. Microsc. 147, 279–288 (1987). 16. A. Santos, C. Ortiz de Solorzano, J. J. Vaquero, J. Pena, N. Malpica, and F. Del Pozo, “Evaluation of autofocus functions in molecular cytogenetic analysis,” J. Microsc. 188, 264–272 (1997). 17. Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing algorithm selection in computer microscopy,” in 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2005), pp. 70–76. 18. Y. Yao, B. Abidi, N. Doggaz, and M. Abidi, “Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images,” in Defense and Security Symposium (International Society for Optics and Photonics, 2006), p. 62460G. 19. M. Rahman and N. Kehtarnavaz, “Real-time face-priority auto focus for digital and cell-phone cameras,” IEEE Trans. Consum. Electron. 54, 1506–1513 (2008). 20. J. He, R. Zhou, and Z. Hong, “Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera,” IEEE Trans. Consum. Electron. 49, 257–262 (2003). 10 July 2014 / Vol. 53, No. 20 / APPLIED OPTICS
4517
21. K. Ooi, K. Izumi, M. Nozaki, and I. Takeda, “An advanced autofocus system for video camera using quasi condition reasoning,” IEEE Trans. Consum. Electron. 36, 526–530 (1990). 22. R. A. Jarvis, “Focus optimization criteria for computer image processing,” Microscope 24, 163–180 (1976). 23. E. P. Krotkov, Active Computer Vision by Cooperative Focus and Stereo (Springer-Verlag, 1989). 24. J. Baina and J. Dublet, “Automatic focus and iris control for video cameras,” in Fifth International Conference on Image Processing and its Applications (IET, 1995), pp. 232–235. 25. M. Subbarao and J.-K. Tyan, “Selecting the optimal focus measure for autofocusing and depth-from-focus,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 864–870 (1998).
4518
APPLIED OPTICS / Vol. 53, No. 20 / 10 July 2014
26. E. Krotkov, “Focusing,” Int. J. Comput. Vis. 1, 223–237 (1956). 27. D. L. Marks, A. L. Oldenburg, J. J. Reynolds, and S. A. Boppart, “Autofocus algorithm for dispersion correction in optical coherence tomography,” Appl. Opt. 42, 3038–3046 (2003). 28. Q. Wu, F. Merchant, and K. Castleman, Microscope Image Processing (Elsevier, 2010). 29. S. Yazdanfar, K. B. Kenny, K. Tasimi, A. D. Corwin, E. L. Dixon, and R. J. Filkins, “Simple and robust image-based autofocusing for digital microscopy,” Opt. Express 16, 8670–8677 (2008). 30. H. Anton, Elementary Linear Algebra (Wiley, 2010).
