IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

1459

Low-Level Feature Extraction for Edge Detection Using Genetic Programming Wenlong Fu, Student Member, IEEE, Mark Johnston, Member, IEEE, and Mengjie Zhang, Senior Member, IEEE

Abstract—Edge detection is a subjective task. Traditionally, a moving window approach is used, but the window size in edge detection is a tradeoff between localization accuracy and noise rejection. An automatic technique for searching a discriminated pixel’s neighbors to construct new edge detectors is appealing to satisfy different tasks. In this paper, we propose a genetic programming (GP) system to automatically search pixels (a discriminated pixel and its neighbors) to construct new lowlevel subjective edge detectors for detecting edges in natural images, and analyze the pixels selected by the GP edge detectors. Automatically searching pixels avoids the problem of blurring edges from a large window and noise influence from a small window. Linear and second-order filters are constructed from the pixels with high occurrences in these GP edge detectors. The experiment results show that the proposed GP system has good performance. A comparison between the filters with the pixels selected by GP and all pixels in a fixed window indicates that the set of pixels selected by GP is compact but sufficiently rich to construct good edge detectors. Index Terms—Edge detection, feature extraction, genetic programming.

I. Introduction

E

DGE DETECTION is an important problem in image processing and image understanding. It is widely applied in computer vision and image analysis, such as object recognition [1], object tracking [2], and image retrieval [3]. A simple description of the content of an image can be captured by its edges since different objects have different shapes. Edges simplify the presentation of an original image, but keep some important content. There are many approaches to edge detection [4]–[6]. These approaches mainly focus on differentiation [5], [7], statistics [8], mathematical morphology [9], [10], machine learning (ML) [11], phase congruency and local energy [12], [13],

Manuscript received February 12, 2013; revised June 27, 2013; accepted October 10, 2013. Date of publication October 31, 2013; date of current version July 15, 2014. This work was supported in part by the Marsden Fund of New Zealand Government under Grant VUW0806, administrated by the Royal Society of New Zealand, and in part by the University Research Fund, Victoria University of Wellington, under Grant 203936/3337. This paper was recommended by Associate Editor B. Zhang. W. Fu and M. Johnston are with the School of Mathematics, Statistics and Operation Research, Victoria University of Wellington, Wellington 6140, New Zealand (e-mail: [email protected]; [email protected]). M. Zhang is with the Victoria University of Wellington, Wellington 6140, New Zealand (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TCYB.2013.2286611

multiresolution [14], a combination of local features [15], and optimization techniques based on groups of pixels [16]. These approaches can generally be combined with preprocessing techniques, such as Gaussian filtering [5], [6]. Also, an edge detection technique can be further developed to detect contours. For example, the differentiation technique has been used for contour detection based on receptive field models [17], [18]. Almost all approaches extract features with all pixels (neighbors of a discriminated pixel) in a local area, and most of these pixels are similar to each other, e.g., similar intensities, but there is not much work in reducing redundancies among these pixels. Here, features in edge detection are functions of raw pixel values in an image relative to a local point and are used in the process of classifying pixels as edge points or nonedge points. Since high-level features are usually constructed from lowlevel features and low-level features are normally suitable for real-time applications [19], low-level edge features are important for edge detection. The low-level feature approaches based on raw pixels generally choose all pixels in a local area, but there are redundancies (e.g., the same intensity) existing in these pixels. Almost all existing feature selection methods are based on predefined features, but they have no ability to find new features [6]. For the low-level features based on raw pixels, pixels in a fixed area are chosen [11], but pixels not in the area are not considered as candidates to construct features. A small area may not include some pixels which are useful to detect edges, but a larger area will increase the computational cost. When manually selecting a moving window to construct edge features, the window size is hard to set. Window size is a tradeoff between noise rejection (would like a large window) and edge localization (would like a small window). In order to address the window size problem, it is necessary to automatically select single pixels to construct edge filters. In most of the existing edge detection techniques, neighbors with the same distance to their discriminated pixel are considered as the equal contribution to construct edge features [6], [7]. There are two problems in selecting a compact and sufficiently rich set of pixels for constructing an edge detector. First, the number of the possible subsets in a window increases exponentially as the window size increases. The second problem is whether neighbors with the same distance to their discriminated pixel are randomly selected. In order to address the two problems, an analysis of the pixels selected by an automatic technique is required.

c 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. 2168-2267  See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

1460

Fig. 1.

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

Typical GP algorithm description.

In order to automatically select pixels to construct edge detectors, genetic programming (GP) is employed. GP is a population-based evolutionary algorithm, aimed at finding computer programs for automatically solving problems [20]. Similar to other evolutionary algorithms, GP is inspired by biological evolution to generate biased candidate solutions with the mutation and crossover operations, and evaluates individuals with fitness functions. However, the encoding scheme (chromosome representation) in GP is different from other evolutionary algorithms; it is flexible and can be considered as a program to directly perform a user-defined task. A GP system includes a terminal set (all possible input variables) and a function set (function operators). A typical GP algorithm is outlined in Fig. 1. The replication, crossover, and mutation operations are used to generate a new population to replace the current population. There are two main reasons for employing GP for edge detection. First, GP can use flexible representations to create programs. In this paper, programs evolved by GP are expected to automatically search pixels to extract edge features. Therefore, a moving window is not employed in GP, which avoids manually setting a size for the window. Second, GP has weak prior knowledge requirements, marking it suitable for extracting low-level edge features from raw single pixels. Applications of GP in image processing and image analysis are popular in recent years [21], but there has not been much research using GP for edge detection. A. Goals The overall goal of this paper is to investigate automatically searching pixels (a discriminated pixel and its neighbors without window limitation) to construct features in edge detection using GP, and analyze the pixels selected by evolved GP edge detectors via using these selected pixels to construct linear and second-order filters. Our previous work [22] shows that GP can automatically choose pixels to construct good edge detectors. Therefore, we utilize GP to conduct a further investigation on automatically finding pixels to automatically construct lowlevel subjective edge detectors. A set of raw pixels extracted from GP edge detectors are employed to construct linear and second-order filters with a fixed threshold constant (rather than

a threshold ratio for the sake of fast response and without memory buffer requirement). In order to check whether the selected set of raw pixels is better than a set of raw pixels in a window, linear and second-order filters are constructed from the same set of raw pixels. Specifically, we investigate the following research objectives: 1) how to use GP to efficiently obtain a set of raw pixels to construct features for edge detection; 2) what characteristics (e.g., relative pixel positions) exist in a set of raw pixels selected by GP; 3) whether a set of raw pixels from the GP edge detectors is better to construct linear and second-order filters than using all pixels in a fixed window; 4) whether these estimated filters are better than common existing low-level edge detectors, such as the Laplacian and Sobel edge detectors; 5) whether the GP edge detectors are better than the above estimated filters. B. Organization In the remainder of the paper, Section II briefly describes relevant background on edge detection and GP for edge detection. Section III gives details of the new GP approach to edge detection. Section IV proposes an optimization model for constructing features based on filter techniques. Section V provides the results of the experiments. Section VI supplies further discussions. Finally, Section VII draws conclusions and suggests future work directions.

II. Related work GP has been used for feature construction [23], [24] and feature extraction [25]–[27]. Features constructed by GP represent rich information and improve classification accuracy [23], [24]. The purpose of feature extraction is to reduce redundancies in data and transform data into a reduced representation set of features [26]. In image processing, there are wide applications using GP [21], [28]–[31]. The existing methods for feature extraction in edge detection are roughly divided into two categories as follows. A. Approximation Based on Desired Features In order to obtain good edge features based on 1-D step response, Harris and Buxton [32] utilized GP to approximate responses on 1-D edge signals and then employed the evolved programs to extract edge features based on a fixed window. Different from designing 1-D signals, features similar to existing edge features are also evolved by GP. The objective is to fit the outputs of the existing feature extraction methods. GP was used to approximate the outputs from the Sobel detector [33], [34] and the Canny detector [35]. Approximating existing edge detectors or desired signal responses can be considered as regression problems. However, the desired outputs are dependent on the human knowledge for understanding edge responses. Features extracted based on desired outputs (edge responses) can only represent the known (limited) edge information.

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

1461

B. Evaluations Based on Ground Truth Edge detection has been considered as a special binary classification problem when edge detectors discriminate each pixel as an edge point or a nonedge point [22], [36]. The outputs of edge detectors are usually evaluated based on detection accuracy after thresholding. The simplest way to construct edge detectors is based on raw pixels. Pixels nearby a discriminated pixel as inputs are used to construct a low-level edge detector. Zhang and Rockett [11] used 13 × 13 windows as edge patterns to extract features via multiobjective GP, with objectives of Bayesian error, classification error, and the number of nodes. In order to find good edge detectors, existing image operators are used to construct GP edge detectors. In [37], morphological operators erosion and dilation were used as the terminal set to evolve edge detectors to classify pixels as edge points or nonedge points. To detect boundaries, some image operators were employed to combine a boundary detector by GP [38]. Also, GP has also been used to design detectors for some specific tasks. In [39], a 64-bit digital circuit for detecting edges was evolved by GP based on artificial images. In [40], programs were designed to find edge pixels in one image. These programs start at a pixel and then walk to find edge points. These GP edge detectors are dependent on a fixed window to extract features. However, the window size is a tradeoff between noise rejection and edge localization [5]. Pixels selected from a fixed 13 × 13 window to construct GP edge detectors are nearby the discriminated pixel [11], which indicates that not all pixels in a window are necessary to be used in an edge detector. In our previous work [22], [41], [42], we employed GP to choose pixels (not from a window) to construct edge detectors by a shifting function so that the space for candidate sets of raw pixels is as large as possible. If a pixel is not close to the discriminated pixel, it would be a low probability to select the pixel to construct edge detectors in our system. Without a window limitation, there are more sets of pixels to construct edge detectors. However, the size of a suitable set of pixels selected by GP to construct edge detectors has not been investigated. If only a few pixels are selected as a suitable set, whether these selected pixels can be used to construct good edge detectors is not clear. This paper is to further investigate these selected pixels. III. GP for Edge Detection This section describes a GP system for evolving edge detectors based on full individual images, rather than a set of pixels from a moving window. A. Sets of Terminals and Functions In order to choose pixels from an image to construct lowlevel edge detectors, rather than selecting pixels in a small window, the selection operation is based on a whole image. Therefore, a whole image x is used as input. Taking into account coefficients in an edge detector, constants would be

Fig. 2. 2 × 2 Robert detector constructed with GP. Nodes functions sqrt and square.

√

and SQ are

required. The terminal set contains an individual image x and random constants rnd in the range of [−10, 10]. To construct low-level edge detectors, basic operators {+, −, ∗, ÷, abs, sqrt, square} are usually used to construct edge detectors [7], so they are utilized in the function set. Note that the square root function sqrt is protected, which produces a result of 0 for negative inputs, and ÷ is also protected, producing a result of 1 for a 0 divisor. A search operator is also included in the function set for selecting pixels to construct features. Here, a shifting function sn,m is developed from the four directional shifting functions used in [36]. Function sn,m means that its single input will shift n columns and m rows. If n is negative, the input shifts to the left, otherwise shifts to the right. If m is negative, it means shifting up, otherwise down. Its input can be an original image x or the output after shifting x. Additionally, if the argument of the function sn,m is rnd, the coefficient will be multiplied by 2n+m so that more different coefficients are generated. Some traditional edge detectors can be expressed by this terminal set and the function set. For example, the 2 × 2 window Robert detector [7] can be represented the GP edge detector GERobert , which is defined by (1). In order to give the pixels used in the Robert detector, functions s1,1 , s1,0 and s0,1 are used to select the pixels nearby the discriminated pixel. Fig. 2 presents a tree representation of the 2×2 window Robert filter based on whole image x  GERobert = (x − s1,1 (x))2 + (s1,0 (x) − s0,1 (x))2 . (1) The search operator implicitly finds the neighbors in GERobert . The three neighbors used in GERobert can be implicitly expressed by s1,1 (x), s1,0 (x) and s0,1 (x). Also, some other existing low-level edge detectors can be represented by the GP system. Taking the 3 × 3 Sobel edge detector as another example, its horizontal derivative GEx,Sobel is shown in (2). Search operators s1,1 (x), s−1,1 (x), s1,−1 (x), and s−1,−1 (x) implicitly find the four diagonal neighbors in the 3 × 3 Sobel edge detector GEx,Sobel = s1,1 (x) − s−1,1 (x) + 2s1,0 (x) −2s−1,0 (x) + s1,−1 (x) − s−1,−1 (x).

(2)

1462

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

the interest of the analysis of a set of pixels selected from GP, we build linear and nonlinear (second-order) filters with these selected pixels to perform edge detection. We design these filters with only one pass of the images so that they give response as fast as possible and no extended memory is required, which is good for applying to process sequence images and hardware design for the implementation of the filter with fast response speed. A. Filters

Fig. 3.

Process of finding pixels to construct edge filters.

B. Fitness Function We treat the edge detection task as a binary classification task (with the edge points as the main class) in the evolutionary training process. For the output of a program, we use 0 as the threshold (greater than 0 as edge points, otherwise as nonedge points). Since only low-level edge detectors are evolved in this paper and search operators are employed to find pixels to construct GP detectors (namely, this paper only focuses on the way to extract features for edge detection), the output is directly evaluated without postprocessing, following [43]. The F -measure [15] [see (3)] is employed to evaluate the accuracy of a detector for images, where α is a weighting factor, from 0 to 1, recall is the number of pixels on the edges correctly detected as a proportion of the total number of pixels on the edges, and precision is the number of pixels on the edges correctly detected as a proportion of the total number of pixels detected as edges. We use F as the fitness function, and treat precision and recall equally important with α = 0.5; and higher fitness indicates better performance recall ∗ precision F= . (3) α ∗ recall + (1 − α) ∗ precision C. Process Flow Fig. 3 shows the process flow to find neighbors of a discriminated pixel. After analyzing a whole image x and shifting function sn,m in the evolved edge detectors, a set of neighbors for a discriminated pixel will be obtained. In the neighborhood part of Fig. 3, empty cells from the 5×5 window are considered to be redundant for constructing edge filters.

IV. Feature Construction Based on Filter Techniques To analyze the pixels selected from the GP edge detectors, a technique is required to use these pixels to construct edge detectors to validate whether the set of the selected pixels is good for edge detection. If the search operator sn,m can be effectively used to construct edge detectors, it would be expected that the pixels implicitly selected by sn,m should be better than all pixels in a window to construct edge detectors. Filter techniques are popular for extracting edge features when fast response speed in real-time applications is required. In

The linear filter LF is defined in (4), where, xi is a discriminated pixel from image x, xi,j are the pixels (neighbors, Setnb including xi ) around the discriminated pixel, βj is the coefficient for xi,j for all i, which needs to be optimized, and b is the intercept, being similar to a threshold value. Note that j is a 2-D position index for these pixels in (4), and b can be considered as β0 . For example, j = (0, −1) indicates the left neighbor of the discriminated pixel i. In this paper, we only use pixel intensities as the values of xi,j  LF (xi ) = βj xi,j + b. (4) j∈Setnb

A linear filter in [44] includes βj for the pixels in the 3 × 3 window (Setnb = {(0, 0), (−1, 0), (−1, −1), (0, −1), (1, −1), (1, 0), (1, 1), (0, 1), (−1, 1)}). The sum of βj must be 0 in the filter, and b must be 0. The threshold for the filter needs to be solved by an adaptive algorithm, which takes some computational cost. In addition, the training image is only based on a binary image. However, images used in (4) are based on grayscale levels, being more complicated than binary images. In order to rapidly detect edges in this paper, the fixed threshold 0 is used for LF (xi ) discriminating xi as an edge point when LF (xi ) > 0. With regard to approximating other existing edge detectors, a second-order filter SF is defined in (5), where βj,l is the coefficient for the combined factor xi,j xi,l , and Setj is the subset of pixels from the first pixel (position) in Setnb to the pixel at j. Based on the limitation of only one pass over an image, the existing edge detectors, such as the Sobel detector can be expressed by the equation without using a square root SF (xi ) =



βj xi,j +

j∈Setnb

 

βj,l xi,j xi,l + b.

(5)

j∈Setnb l∈Setj

From (4) and (5), we can use different sets of pixels (Setnb ) to construct edge filters. The pixels from 3 × 3 and 5 × 5 windows are used to compare with the pixels selected from GP. Note that two types of sets of pixels are used to optimize the linear and second-order filters. The first is based on the a fixed area, such as the 3 × 3 and 5 × 5 windows. The other is based on a set of pixels which might be scattered around a discriminated pixel, and these pixels are selected by the GP edge detectors with high occurrences. B. Parameter Estimation To estimate the parameters in the linear and second-order filters, a hybrid particle swarm optimization with differential evolution algorithm (PSO-DE) [45] is employed. PSO has

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

1463

been employed for image processing [46], and there has been some work in edge detection [47], [48]. The detection accuracy based on F is used as the objective function in PSO-DE. Therefore, the estimated parameters are based on the maximum value F . The objective function F only presents the response signs (edge points or not) of a filter, and cannot indicate the response magnitudes. When the parameters βj or βj,l change by a small amount, F might not be changed. To clearly indicate the changes of the estimated parameters, a correlation ratio variant [49] is used as a new objective function for searching optimal parameters. The correlation ratio variant FCR is defined in (6), and it is based on the correlation ratio measure. Note that μ0 and μ1 are the means of the outputs oi (oi = LF (xi ) or oi = SF (xi )) from nonedge points and edge points, μ is the means of all outputs oi , N0 and N1 are the number of the nonedge points and the number of the edge points, X is the set of pixels across all training images, and Th(μ0 , μ1 ) [see (7)) is used to indicate whether the means of two categories (edge points and nonedge points) are on different sides of threshold 0, where c = 0 for nonedge points and c = 1 for edge points    1  Nc (μc − μ)2   1 c=0 Th(μ0 , μ1 )  Fcr =  + (6) 2 2 2 (oi − μ) i∈X  1 if μ0 ≤ 0 Th(μ0 , μ1 ) = 0 otherwise.

and

μ1 > 0

(7)

V. Experiment Results A. Experiment Design 1) Image Dataset: A subjective edge detector learned by a supervised approach is dependent on the desired outputs. To investigate the pixels selected by GP edge detectors, the Berkeley Segmentation Dataset (BSD) [15] is employed, which is popular for edge detection and image segmentation [6]. BSD consists of natural images (of size 481 × 321 pixels) with ground truth provided. The main reason to choose this image dataset is that the ground truth images are combined from five to ten human observations so that edges in these images have fairness of judgement. The aim of evolving edge detectors is to fit the human observations as much as possible. Since each human observation is different from the others, the subjective outputs are difficult to approximate by an edge detector. The BSD image dataset is mainly used for contour and boundary detection. Since edge detection can be applied to contour and boundary detection by adding postprocessing techniques, edge detectors can be evaluated from the ground truth for boundary detection [50]. The ground truth in the BSD image dataset includes different human observations, so edges in these images are rich, which is suitable as the ground truth for edge detection. From observation of the BSD training image set, some redundancies exist in the dataset. Therefore, we only select six images from the BSD training set as the training dataset

Fig. 4. Training images from BSD dataset and their ground truths. (a) 207056. (b) 23080. (c) 105019. (d) 105053. (e) 113044. (f) 216053.

because of the rich edge contents existing in these six images. Fig. 4 shows the training images and the ground truth. To present the human observations, brighter edges mean a larger number of people have marked them as edges. In order to reduce the computation time, we randomly sample five different subimages of size 41×41 from each image as training subimages. A restriction for the sampling operation is that all subimages must contain some true edge points. 2) Parameter Settings: The parameter values for GP are: population size 500; maximum generation 200; maximum depth (of a program) 10; and probabilities for mutation 0.15, crossover 0.80 and elitism 0.05. In order to improve the probability to choose sn,m in a new node, s−1,0 , s0,−1 , s1,0 and s0,1 (four subfunctions of sn,m ) are added into the function set. The GP experiment is repeated for 30 independent runs. The parameters for the hybrid PSO-DE approach are: population size 36; maximum generation 500; the domain for the estimated parameters from −20 to 20; and the domain for the velocity from −5 to 5; and the others are the same with the settings in [45]. These values were chosen based on common setting [45] and empirical search. B. Results The BSD dataset contains 100 test images. We evaluate the test performance based on F with all pixels across the whole 100 test images, not with each image separately. For fair comparison, we compared all results with the 3 × 3 Laplacian edge detector (a linear edge detector) and the 3 × 3 Sobel edge detector (approximately considered as a secondorder edge detector). After normalizing the results from both edge detectors, the maximum F based on the test images k with 51 different thresholds ( 51 , k = 0, 1, . . . , 50) are used as the baseline performance for both edge detectors. Since the Laplacian and Sobel edge detectors use fixed parameters, there is only one F value (the maximum) for each edge detector. The results from the GP edge detectors and the edge filters optimized by the hybrid PSO-DE approach are independent. 1) GP Edge Detectors: Table I presents the average (and standard deviation) of the overall test performance F of GP edge detectors on BSD test images, and the maximum F values of the Laplacian and Sobel edge detectors. The

1464

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

Fig. 5.

Four example test images from the BSD dataset detected by the best GP, Sobel and Laplacian edge detectors.

TABLE I

TABLE II

Test Performance on 100 BSD Test Images for GP Edge Detectors and the 3 × 3 Laplacian and Sobel Detectors

Test Performance (F , Recall and Precision) for GP, the Sobel and Laplacian Edge Detectors on Four Images

one-sample t-test with significance level of 0.05 is used to compare GP with the Laplacian and Sobel edge detectors. The results from GP are significantly higher than the others. The averages (and standard deviations) of recall and precision are also shown in the table. Note that the average of F values of the GP edge detectors is not equal to the F value calculated by the averages of recall and precision of the GP edge detectors, since their relationship is not linear from the definition. The offset distance from a predicted edge point to a true edge point is not allowed in F and there are no postprocessing operations, so the precisions of these detection results are very low. First of all, from the comparison of F values among the GP edge detectors, and the Laplacian and Sobel detectors, we can see that the GP edge detectors are remarkably better than the Laplacian edge detector, and also significantly better than the Sobel edge detector. Second, comparing recall with the Laplacian and Sobel detectors, GP edge detectors obviously improve the ability of finding edge points, and the improvement is more than 0.11. Third, the GP edge detectors have similar precision to the Sobel edge detector, but have higher precision than the Laplacian edge detector. These results suggest that the GP edge detectors are significantly better than the Sobel and Laplacian edge detectors. 2) Visual Results of the Best GP Edge Detector: To present the detected images visually, four example images from the BSD test image dataset are chosen (shown in Fig. 5). Fig. 5 gives the detected binary images from the best GP edge detector, and the Sobel and Laplacian edge detectors with the best threshold value for each detector (based on the maximum F ). The threshold of the Sobel detector for the maximum F is

0.1765, and is 0.03922 for the Laplacian detector. From Fig. 5, the Laplacian detector is sensitive to noise and has poor ability to find edges of the right part in image 45096 [Fig. 5 (a)]. Compared with the Sobel detector, the GP edge detector has similar detection results for the four images, but the GP edge detector has better ability to detect distinct edges, e.g., the edges in the right part in the image 45096 and image 106024. For image 253055, the edges of the clouds are not clear and most observers do not mark them as edges. The GP edge detector matches most of the observations, and does not give responses on the boundaries of the clouds. However, the GP edge detector fails to detect the right and bottom boundaries of the pyramid in image 299086 because the edges are not distinct. The Sobel edge detector has slightly better detection for these edges than the GP edge detector, but is affected by the sand texture noise. From the results, we can see that GP has good ability to find edges from the clear discontinuities, and suppresses the noise but possibly lost edges that are difficult to distinguish. Table II gives F , recall and precision for the GP edge detector, the Sobel and Laplacian edge detectors on each of the four test images. From the view of F , the GP edge detector has the best detection results for all four images. The GP edge

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

Fig. 6. Neighbors (more than 14 occurrences) selected from 30 GP edge detectors. Note that numbers are the occurrences for the neighbors.

detector misses some edges in image 299086, but its recall is still reasonably high, compared with recalls of the Sobel and Laplacian edge detectors. The reason is that the GP edge detector finds almost all of the other true edge points. 3) Pixels Selected by GP: To investigate the pixels selected by GP, Fig. 6 reveals that pixels existing in the 30 GP edge detectors with more than 14 occurrences. The window presents pixels (neighbors) selected around the center (discriminated) pixel based on sn,m . The number of occurrences of the selected neighbors are calculated based on the final terminal x. If a pixel only exists in a few GP edge detectors, it will be considered as giving limited contribution for detecting edges. The threshold for the pixel occurrences in these edge detectors is chosen a half of 30 (runs), namely, at least 15. Note that a pixel might have several occurrences in a GP edge detector. All pixels with at least 15 occurrences are located in the 5 × 5 window. Pixels with less than 14 occurrences do not give their occurrences, and a few pixels are located outside of the 5 × 5 window. Considering the center pixel is the original position [j = (0, 0)], a 2-D position index (j) in the window contains the horizontal direction (right for the positive) and the vertical direction (up for the positive). The greatest number occurrence (47) is from the discriminated pixel [j = (0, 0)], and the second greatest number occurrence (36) is from the right neighbor [j = (1, 0)]. The average number of occurrences of the discriminated pixel itself in the 30 GP edge detectors is more than 1.5 in each tree-based expression. Another interesting observation existing in these selected pixels is that the pixels with high occurrences are diagonal pairs, such as (1, 0) with (0, 1), (2, 0) with (0, 2), (0, −1) with (−1, 0), (1, 1) with (−1, −1). Diagonal pairs have been investigated by applying diagonal derivatives for constructing edge features in our previous work [51], which shows that the edge features from the diagonal derivatives are better than the edge features from the horizontal and vertical derivatives. We use Set1 (for Setnb ) to indicate the nine pixels in Fig. 6 (including the discriminated pixel itself) or filters constructed by the nine pixels. Taking into account the similarity in the pixels around the same direction, a merging operation is used to reduce the number of selected pixels and find more rich pixels around a discriminated pixel in these GP edge detectors. In a function node of a tree-based program, the pixels selected from its subtree will be merged together if they are in the same direction. For instance, (1, 1) and (1, 0) are considered as the same direction (the direction only based on n or m, here n

1465

Fig. 7.

Example of the merging operation on a simple GP edge detector.

Fig. 8. Pixels (more than 14 occurrences) selected from 30 GP edge detectors with a merging operation. Note that numbers are the occurrences for the neighbors. TABLE III Test Performance (F ) on 100 BSD Test Images For Linear Filters Estimated By PSO Based on F and FCR

is chosen), and only (1, 1) or (1, 0) is a selected pixel. Fig. 7 shows a simple example tree to extract neighbors by using this merging operation. In s1,0 (x) + s1,1 (x), the node + merges {(1, 0)} from s1,0 (x) and {(1, 1)} from s1,1 (x) to {(1, 0)}. The root node − has {(2, 2), (1, 2)} as the positions for the relevant selected neighbors. This merging operation is not equal to the simple combination of the final selected pixels, because it is based on function nodes, not final terminals. Fig. 8 gives the selected pixels with more than 14 occurrences. First, there are only six pixels selected, so the number of pixels is reduced by 3. Second, the number of the selected pixels connected to the discriminated pixel is only three, but these pixels are not necessarily connected to each other. Third, two selected pixels in the positions (2, 3) and (3, 2) are not located in the 5 × 5 window. Lastly, these pixels have no diagonal pairs, compared with Set1. Using this merging operation, the selected pixels are scattered into a local area. We use Set2 (for Setnb ) to indicate these six pixels or filters constructed by these six pixels. 4) Optimized Linear Filters: Table III gives the averages and standard deviations of F values of the linear filters constructed by all pixels in 3 × 3 and 5 × 5 windows, Set1 and Set2 based on (4) with F and FCR . For Setnb , 3 × 3 indicates filters constructed with all pixels in a 3 × 3 window, and 5 × 5

1466

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

TABLE V Test Performance (F ) on 100 BSD Test Images for Second-Order Filters Estimated by PSO Based on F and FCR

Fig. 9. Test performance (F ) averaged over the 100 BSD test images for linear filters estimated by PSO-DE based on F and FCR . TABLE IV Post Hoc Comparisons (F, FCR ) Among Linear Filters, GP Edge Detectors, the Laplacian Detector, and the Sobel Detector

Fig. 10. Test performance (F ) averaged over the 100 BSD test images for second-order filters estimated by PSO-DE based on F and FCR .

for filters constructed with all pixels in a 5 × 5 window. Fig. 9 shows the performance of these linear filters using the means and standard deviations of their F values. From Table III and Fig. 9, the test performance for the parameters of these linear filters estimated by the objective function F is better than the objective function FCR , but the linear filters estimated by FCR have more stable performances than the filters estimated by F . Table IV gives the comparisons of the test performance F among the estimated linear filters, the GP edge detectors and the Laplacian and Sobel edge detectors based on multiple onesample t-tests with overall significance level 0.05 and p-value adjustment using Holm’s method [52] over all comparisons. Here, ↓ indicates that the edge detector from the first column is significantly worse than the edge detector from the first row, ↑ for the significantly worse indication (the first column to the first row), and − for no significant difference. In each pair, the first comparison is based on the objective function F , and the second is based on the objective function FCR . There are six interesting comparisons from Table IV. First, all estimated linear filters are significantly better than the 3 × 3 Laplacian edge detector. It seems that linear filters optimized by the hybrid PSO-DE based on ground truth are better to detect edges than the Laplacian edge detector. Second, the linear filters based on the 5 × 5 window have the best detection performance based on the average of F values for the 100 test images in the four estimated linear filters with the objective function F , but the linear filters constructed by Set1 have very close test performance to the 5 × 5 window linear filters according to the averages of F values in Table III. Third, all the estimated linear filters by the objective function FCR have close test performance, but the linear filters constructed by Set1 are the significantly better than the other estimated linear filters. Fourth, the estimated

linear filters cannot compete with the Sobel and GP edge detectors. In general, the GP edge detectors are nonlinear, so it seems that the nonlinear edge detectors are better than the linear edge detectors. Fifth, the estimated linear filters from Set1 have better test performance than the estimated linear filters from Set2. Finally, the edge detectors constructed by Set1 are significantly better than the estimated 3 × 3 window linear edge detectors, and there are no significant differences between the linear edge detectors constructed by Set2 and the estimated 3 × 3 window linear edge detectors. Although Set1 and Set2 are not as good as the estimated 5 × 5 window edge detectors by the objective function F , these estimated linear edge detectors show that the pixels selected by GP contain rich information for constructing edge features based on the overall comparisons. 5) Optimized Second-Order Filters: Table V gives the averages and standard deviations of F values of the secondorder filters constructed by all pixels in 3 × 3 and 5 × 5 windows, Set1 and Set2 based on (5) with F and FCR . Fig. 10 shows the error-bars of these second-order filters based on the test performance using F . Different from the estimated linear filters, these second-order filters estimated by the objective functions F and FCR have close test performance (the average of values F ). The second-order filters constructed with all pixels in Set2 have the highest average of F values. However, the standard deviations of the test performance of the secondorder filters are higher than the linear filters. Table VI gives the comparisons of the test performance F among the estimated second-order filters, the GP edge detectors, and the Laplacian and Sobel edge detectors. There are several interesting comparisons in Table VI: first, the estimated second-order filters constructed with Set2 have the best test performance among the estimated second-order filters with the objective function F or FCR ; second, the estimated

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

1467

TABLE VI Post Hoc Comparisons (F, FCR ) Among Second-Order Filters, GP Edge Detectors, the Laplacian Detector, and the Sobel Detector

Fig. 11. Four example detected images by a second-order filter constructed with Set2. (a) 45096. (b) 106024. (c) 253055. (d) 299086. TABLE VII

second-order filters constructed with Set1 can compete with the Sobel edge detector; third, the estimated second-order filters constructed with Set2 based on the objective function F or FCR are significantly better than the Sobel detector and the estimated 5 × 5 window second-order filters; fourth, the estimated second-order filters constructed with Set1 or Set2 are significantly better than the estimated 3×3 window secondorder filters; and lastly, the estimated second-order filters with Set2 based on the objective function F can compete with the GP edge detectors. From the comparisons, it seems that the pixels selected by GP with high occurrences are compact but rich for constructing low-level edge detectors. Although the estimated 5 × 5 window second-order filters are significantly better the estimated 3 × 3 window second-order filters, the estimated second-order filters with Set1 can compete with the estimated 5 × 5 window second-order filters. Since Set1 is included in the 5×5 window, it shows that there are redundancies existing in that area for constructing edge detectors. The number of pixels in Set1 is the same as the number of pixels in the 3 × 3 window. It seems that pixels in Set1 are better than the pixels in 3 × 3 window for constructing edge detectors. Although the number of pixels in Set2 is less than the number of pixels in the other filters, the estimated second-order edge detectors based on Set2 outperform the others, except for the GP edge detectors. Therefore, if the redundancy in the same direction is considered, the pixels selected by GP are more efficient and richer for constructing second-order filters than the pixels in Set1. In addition, these pixels in Set2 are not totally located in the 5 × 5 window. From the comparisons between GP and the estimated (linear and second-order) filters with moving (3 × 3 and 5 × 5) windows, GP has better detection results. A potential reason is that GP implicitly searches pixels to directly construct edge features. However, edge filters constructed by pixels in an optimization method using a moving window need a fixed model, such as a second-order edge filter. It is possible that a fixed model has limited ability for edge detection. The flexible programs in GP make the evolved programs automatically fit the ground truth of the dataset. Also, a method using a moving window employs all pixels in a fixed window. However, GP automatically identifies a compact but sufficiently rich set of pixels to construct edge detectors. The estimated filters with the pixels selected by GP show that GP effectively selects pixels to construct edge filters.

Test Performance (F , Recall and Precision) on Four Images From an Estimated Second-Order Filter

C. Estimated Parameters for Second-Order Filter To present the best second-order filter constructed by Set2 with F , the xi,j indices (j for pixel positions) in (5) have Setnb = {(−2, −1), (−1, −1), (0, 1), (1, −1), (2, 3), (3, 2)}. The estimated parameters (βj by the position order of neighbors in the Setnb ) with four-decimal-point precision are {−3.5443, −3.0891, −4.8875, −18.1688, −4.0319, −18.5054} and the estimated parameters of the combined factors [βj,l by the position order in the (5)] are {−2.5557, 20.0000, −3.9357, −4.2809, 1.4129, −5.8742, −3.6624, 0.9682, −9.7323, −9.9464, 5.1471, 13.1088, −7.9164, −7.4543, −9.7803, 11.6844, −9.9062, 6.3426, 13.1130, 1.7990, 0.2317}. The overall test performance F is 0.2574. Fig. 11 shows the four corresponding detected images in Fig. 5. These detected images are similar to the detected results by the GP edge detector and the Sobel edge detector in Fig. 5. However, compared with the Sobel edge detector, the secondorder filter here finds more edge points in images 45096 and 106024. For image 253055, the second-order filter does not give responses on the boundaries of the clouds, which is the same as the GP edge detector. For image 299086, the secondorder filter detects the some part of the edge of the right boundary of the pyramid, and rejects more noise than the Sobel edge detector. Table VII gives the test performances F , recall and precision of the second-order filter for the four images. Compared with the Sobel edge detector and the GP edge detector for the four images (based on Table II), the second-order filter has highest recall. However, precision for the second-order filter is lower than precision for the GP edge detector. Overall, the detection results from the second-order filter are approximately similar to the detection results from the GP edge detector. It seems that the pixels selected by GP with high occurrences have compact but rich information for constructing edge features (second-order filters) when the merging operation is employed. The estimated second-order filters have similar detection to the GP edge detectors.

1468

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

VI. Further Discussions This section gives the reasons for choosing the baselines, and discusses interesting phenomena occurring in this experiment, mainly focusing on the pixels used to construct low-level edge detectors. A. Baselines for the Comparisons The Laplacian and Sobel edge detectors are selected as the baselines from the existing edge detectors. There are two reasons to only choose these two edge detectors. First, the two detectors are popular low-level edge detectors and do not use preprocessing and postprocessing. Since preprocessing and postprocessing techniques can routinely be effectively applied to the outputs of both edge detectors, this paper only focuses on the outputs themselves. The other lowlevel edge feature extraction methods, such as the Canny detector, can be considered as a combination of methods for extracting edge features and techniques for preprocessing and postprocessing. A Canny detector can be approximated by the combination of a Gaussian filter (preprocessing), a Sobel detector (feature extraction), and a nonmaximum suppression (postprocessing) [53]. Also, the postprocessing techniques in the Canny edge detector can be combined with the GP edge detectors. Second, the investigation is on low-level edge detectors constructed by raw pixels. For fair comparison, only edge detectors constructed by raw pixels are selected. The Laplacian edge detector is a linear filter based on raw pixels, and the Sobel edge detector can be approximated as a second-order edge detector based on raw pixels. To validate the choice of a compact but rich set of pixels, the linear and second-order filters are employed. Therefore, these estimated filters based on the selected pixels are fairly compared with existing linear and second-order filters. There are other edge detectors similar to the Sobel edge detector [54]. Therefore, only the Sobel edge detector is chosen as the baseline for the existing second-order edge detector. The selected pixels from GP edge detectors are different from the pixels used in the Laplacian and Sobel edge detectors. In order to check whether the pixels extracted from GP edge detectors are good to construct edge detectors, the pixels in the 3 × 3 and 5 × 5 windows are used to construct filters with the same approach. The filters based on the 3×3 and 5×5 windows are used as baselines for the comparisons because they are optimized by the same approach. When a machine learning algorithm is normally used for constructing low-level edge detectors and the ground truth is provided, the parameters of the constructed edge filters with raw single pixels are searched based on the training data and a objective function. Generally, to construct edge filters, all pixels in the 3 × 3 window are considered [44], [55]. Since some of the pixels selected from GP are not in the 3 × 3 window, the 5 × 5 window is also used. B. Objective Functions F and FCR 1) Robustness: To estimate the parameters of a linear or second-order filter, PSO-DE is employed. Our previous work [45] shows that this method is effective. However, the

standard deviations of the test performance for all estimated filters from the objective function F are higher than the standard deviations of the test performance for the filters from the objective function FCR . The reason is that F is only based on the sign of the response. A slight change of a parameter in the filter affects the response magnitudes, but the sign of the response still stays the same. Therefore, the same F value in the training stage allows different values for the estimated parameters of a linear or second-order filter. A change of estimated parameters for a filter based on FCR brings a difference in FCR because FCR includes the information of the response magnitudes of the filter. The final solutions for these estimated parameters are almost the same. The training data (randomly sampled for the six images in Fig. 4) is not fixed in 30 runs, so the difference for the estimated parameters still exists. 2) Accuracy: From the test performance of the estimated linear and second-order filters, the fitness function F is better than FCR . Since FCR indicates linear dispersion [56], when the two classes are difficult to separate by a threshold, FCR prefers the response magnitudes with a small range, but loses some accuracy. However, when the filter has some ability to distinguish edge points from nonedge points, the accuracy in FCR becomes very important. That is why the second-order filters estimated by FCR have the similar detection performance to the second-order filters estimated by F . In summary, F is good for estimating the continuous parameters if only the detection accuracy is considered. FCR has more robust performance, and it is not bad for estimating the parameters of a second-order filter, but it is not good at estimating the parameters of a linear filter due to low accuracy. C. Linear and Second-Order Filters The estimated second-order filters are better than the estimated linear filters, but are more complicated. The estimated linear filters are better than the Laplacian edge detector, but are worse than the Sobel edge detector. A potential reason is that the linear relationship is not enough to suppress noise in the natural images. Another reason is that the estimated filters use a fixed threshold 0, but the performance for the Sobel edge detector is the maximum F on the test images from a set of thresholds after normalizing the detection results. If the Sobel edge detector directly uses a fixed threshold based on training images, without normalizing, the performance might decrease a bit. The best 5 × 5 window linear filter and the best linear filter constructed with all pixels in Set1 still have close test performance to the Sobel edge detector. D. Number of Pixels Used to Construct Edge Detectors The performance of the estimated filters has no obvious relationship with the size of a set of pixels for constructing them. Although the estimated 5 × 5 window filters are better than the estimated 3 × 3 window filters based on the construction of a linear filter or a second-order filter, the filters constructed by all pixels in Set2 are significantly better than the 3 × 3 window filters based on the linear construction or the secondorder construction. A 3 × 3 window filter contains nine pixels, but a filter using Set2 only has six pixels.

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

Fig. 12. Convergence for the estimated second-order filters with fitness function F .

Only considering the estimated linear filters, we can see that a set of pixels with a large size is good to construct linear filters, but there might be some relationships among these pixels. The estimated 5 × 5 window linear filters have 25 pixels and have the best test performance. However, the filters constructed with all pixels in Set1 are better than the 3 × 3 window linear filters. Since Set1 contains pixels (with diagonal pairs) from the 5 × 5 window, the linear filters constructed with Set1 are better than the linear filters constructed with Set2, and they are close to the 5 × 5 window constructed linear filters. Although Set2 contains useful pixels but the linear relationship is not enough to detect pixels as edge points or not. Therefore, when extracting neighbors Setnb from GP edge detectors to construct linear edge detectors, the redundancy in a direction (do not using the merging operation) should be kept. Since a second-order filter has a stronger ability to disperse two classes than a linear filter, the constructed second-order filters have better test performance than the latter. The 5 × 5 window second-order filters are better than the 3 × 3 window second-order filters, which indicates all pixels in the 5 × 5 window are better to construct edge filters than all pixels in the 3 × 3 window. The second-order filters constructed with Set2 are better than the 5 × 5 window second-order filters, which indicates the pixels from Set2 are more useful to construct edge detectors than the pixels in the 5 × 5 window. From the results based on Set1 and Set2, a filter constructed with Set2 contains fewer number of pixels, but provides better test performances. This suggests that the directional redundancy in the pixels for constructing second-order filters can be removed. In addition, two pixels in Set2 are not located in the 5 × 5 window; it seems that the pixels with a short distance are also helpful for discriminating pixels. The number of the parameters in a second-order filter is now analyzed to determine whether it affects the final solution. For a second-order filter, the number of parameters is nx (nx + 1) nx + , 2 where nx is the number of pixels to construct the filter, and the intercept b is not considered. Therefore, a 3 × 3 window second-order filter has 54 estimated parameters, 299 estimated parameters for a 5 × 5 window second-order filter, 54 estimated parameters for a second-order filter with Set1,

1469

and 27 estimated parameters for a second-order filter with Set2. Fig. 12 shows the convergence for the four estimated second-order filters with the fitness function F . Here, the vertical axis is the average of F values of the best solutions in each generation. From Fig. 12, the values of the objective functions from the these estimated filters converge to a constant or are very close to a constant. Since the hybrid PSO-DE has good ability to find good solutions, the number of these estimated parameters does not strongly affect the final solution. However, in the training stage, the estimated filters with Set1 are slightly better than Set2. Whether there is overfitting existing in the estimated filters with Set1 will be further work. In summary, based on two different strategies to select pixels from GP edge detectors to construct low-level edge features, Set1 and Set2 indicate that GP automatically selects compact but rich pixels for the edge detector construction. If the limited ability of dispersion for a linear filter is considered, pixels in the GP edge detectors should be directly chosen to construct linear edge features. To construct more complicated edge detectors based on raw pixels, the redundancy in the same direction should be removed and pixels with a limited distance are possibly useful to construct edge detectors. Therefore, it is suggested that the pixels in the GP edge detectors used to construct complicated filters should be chosen after using the proposed merging operation. E. Pixels Selected by GP Since the merging operation is employed in the function node, two different sn,m functions with the same direction in a subtree will be merged as one. In each function node, its children are considered as terminals, which is very different from the selection based on the real terminal (image x). In the merging operation, a terminal is not only x, but also a subtree containing sn,m is also considered as terminal. This is why Set1 and Set2 are different. The result for merging the Set1 is not equal to Set2. Set1 is only dependent on the original terminal x, but Set2 is related to the function nodes and the original terminal x. After the merging operations, only a few pixels are selected and all these pixels are based on different directions or distances to the discriminated pixels. Therefore, the set of pixels is rich and has few redundancies. It is a potential reason that Set2 only including six pixels can construct very good edge detectors, which are better than the estimated 5 × 5 window second-order filters, and can compete with the GP edge detectors. A direct merging operation based on Set1 is not investigated because the set of pixels after the merging operation is approximated as a subset of Set1. F. Computational Cost The training time for evolving GP edge detectors is around 720 s on a single machine with CPU 3.1 GHz. The times for estimating the parameters of a linear filter in each run are around 30 s for Set1 and the 3 × 3 window, 19 s for Set2, and 84 s for the 5 × 5 window. The times for estimating the parameters of a second-order filter in each run are around 450 s for Set1 and the 3 × 3 window, 204 s for Set2, and

1470

Fig. 13.

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

Example results detected by GP after postprocessing.

2230 s for the 5 × 5 window. When we use a 5 × 5 window second-order filter, the time for the parameter estimation is much longer than constructing a GP edge detector. Since the number of parameters in the estimated 5 × 5 window second-order filter is large, there is heavy computational cost for estimating a second-order filter using all pixels from the window. Therefore, effectively selecting pixels to construct filters can rapidly reduce computational cost in the training stage. The test time for the GP edge detectors, the estimated filters using pixels in Set1 and Set2 are less than 0.05 s, which usually satisfies a real-time application. G. Postprocessing Fig. 13 shows two example images detected by GP after postprocessing by the operations of thinning edges [57] and removing shortly connected edge points. Compared with the original results detected by GP, the detected edges after performing the postprocessing operations mainly focus on the object boundaries. The thick responses in GP edge detectors are easily thinned by the thinning operation. In the two example edges, we only remove shortly connected edges. In order to obtain smooth boundary detection results, more postprocessing techniques can be employed, such as linking broken edges. Therefore, the GP edge detectors have the ability to apply to boundary detection. VII. Conclusion This goals of this paper were to investigate whether GP can automatically find a suitable set of pixels to construct lowlevel subjective edge detectors, and valid whether the set of pixels is good to construct edge detectors (with the linear and second-order filters). Via finding neighborhood from the GP edge detectors for each discriminated pixel and utilizing the selected pixels with high occurrences to construct linear and second-order filters, the goals were achieved. The experiment results on a benchmark dataset (the BSD image dataset) show that the proposed GP system has good performance to search pixels and construct edge detectors. Based on the analysis of the pixels existing in the evolved edge detectors, rich pixels were selected from these GP edge detectors and were used to construct linear and second-order filters. Based on the comparisons of the test performance on these filters, and the Laplacian and Sobel edge detectors, also the 3 × 3 and 5 × 5 window filters, they show the set of pixels obtained from GP with high occurrences has a small size, and it is very efficient to construct low-level subjective edge detectors.

Fig. 14.

Horizontal derivative of the 3 × 3 Sobel edge detector.

Two different strategies were used to search a suitable set of pixels from the GP edge detectors to construct edge filters. When the pixels are directly searched from the terminal x based on the sn,m function, the set of pixels obviously contains diagonal pairs, which contains rich information for finding edges. When the redundancy from the same direction is removed and the pixels are selected by a merging operation based on sn,m function, the set of pixels has smaller size than the former set and the pixels in the set around the discriminated pixel are not connected. The test performance on the second-order filters using these pixels shows the set of pixels is very rich and efficient to construct edge detectors. The objective function based on the F-measure method gives good solutions for the estimated filters, although it is not continuous for the estimated parameters of a filter. The continuous objective function based on correlation ratio has very good stability for the optimal filters, but low performance for optimizing linear filters. This suggests that the objective function should be chosen correlation ratio to optimize a complicated filter if the stability requirement is very strict, otherwise, the F-measure method should always be a good choice, compared with correlation ratio. For future work, we will further investigate the structures of evolved edge detectors so that we can find new feature extraction methods. Also, we will utilize the evolved edge detectors as low-level features to construct composite features (such as, using ensemble techniques) to detect edges, and compare them with other popular edge detectors based on middle-level or high-level features. Appendix Sobel Edge Detector Constructed by GP The 3×3 Sobel edge detector GESobel is given by (9) based on the GP system using sn,m . GESobel contains the horizontal derivative GEx,Sobel (2) and the vertical derivative GEy,Sobel (8). Fig. 14 shows the horizontal derivative GEx,Sobel . Since the whole tree of the 3 × 3 Sobel edge detector is a bit large, only the horizontal derivative GEx,Sobel is represented. Note that search operators can be combined together. For example, s−1,1 (x) is equal to s−1,0 (s0,1 (x)) and s−2,1 (s1,0 (x)) GEy,Sobel = s1,1 (x) − s1,−1 (x) + 2s0,1 (x) − 2s0,−1 (x) + s−1,1 (x) − s−1,−1 (x)  2 2 GESobel = GEx,Soble . + GEy,Soble

(8) (9)

FU et al.: LOW-LEVEL FEATURE EXTRACTION FOR EDGE DETECTION

References [1] C. Olson and D. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. Image Process., vol. 6, no. 1, pp. 103–113, Jan. 1997. [2] J. Sullivan and S. Carlsson, “Recognizing and tracking human action,” in Proc. 7th Eur. Conf. Comput. Vision I, 2002, pp. 629–644. [3] M. Kokare, P. K. Biswas, and B. N. Chatterji, “Edge based features for content based image retrieval,” Pattern Recognit., vol. 36, no. 11, pp. 2649–2661, 2003. [4] S. Ando, “Image field categorization and edge/corner detection from gradient covariance,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 2, pp. 179–190, Feb. 2000. [5] M. Basu, “Gaussian-based edge-detection methods: A survey,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 3, pp. 252–260, Aug. 2002. [6] G. Papari and N. Petkov, “Edge and line oriented contour detection: State of the art,” Image Vision Comput., vol. 29, nos. 2–3, pp. 79–103, 2011. [7] L. Ganesan and P. Bhattacharyya, “Edge detection in untextured and textured images: A common computational framework,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 27, no. 5, pp. 823–834, Sep. 1997. [8] S. Verzakov, P. Pacl´ik, and R. Duin, “Edge detection in hyperspectral imaging: Multivariate statistical approaches,” in Proc. Joint IAPR Int. Workshops Structural, Syntactic Stat. Pattern Recognit., vol. 4109, no. 10. 2006, pp. 551–5591. [9] Y.-Q. Zhao, W.-H. Gui, Z.-C. Chen, J.-T. Tang, and L.-Y. Li, “Medical images edge detection based on mathematical morphology,” in Proc. 27th Ann. Int. Conf. Eng. Med. Biol. Soc., 2005, pp. 6492–6495. [10] S. Liu, X. Wang, X. Qiu, and Y. He, “Mathematical morphology edge detection algorithm of remote sensing image with high resolution,” in Proc. 1st Int. Conf. Inf. Sci. Eng., 2009, pp. 1323–1326. [11] Y. Zhang and P. I. Rockett, “Evolving optimal feature extraction using multi-objective genetic programming: A methodology and preliminary study on edge detection,” in Proc. Conf. Genetic Evol. Comput., 2005, pp. 795–802. [12] D. Boukerroui, J. A. Noble, and M. Brady, “On the choice of band-pass quadrature filters,” J. Math. Imag. Vision, vol. 21, no. 1, pp. 53–80, 2004. [13] Z. Xiao and Z. Hou, “Phase based feature detector consistent with human visual system characteristics,” Pattern Recognit. Lett., vol. 25, no. 10, pp. 1115–1121, 2004. [14] B. Schunck, “Edge detection with Gaussian filters at multiple scales,” in Proc. IEEE Workshop Comput. Vision, Representation Control, 1987, pp. 208–210. [15] D. Martin, C. Fowlkes, and J. Malik, “Learning to detect natural image boundaries using local brightness, color, and texture cues,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 5, pp. 530–549, May 2004. [16] S. Wang, T. Kubota, J. M. Siskind, and J. Wang, “Salient closed boundary extraction with ratio contour,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 546–561, Apr. 2005. [17] C. Grigorescu, N. Petkov, and M. Westenberg, “Contour detection based on nonclassical receptive field inhibition,” IEEE Trans. Image Process., vol. 12, no. 7, pp. 729–739, Jul. 2003. [18] G. Papari and N. Petkov, “An improved model for surround suppression by steerable filters and multilevel inhibition with application to contour detection,” Pattern Recognit., vol. 44, no. 9, pp. 1999–2007, 2011. [19] M. Donoser, H. Riemenschneider, and H. Bischof, “Linked edges as stable region boundaries,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., Jun. 2010, pp. 1665–1672. [20] J. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA, USA: MIT Press, 1992. [21] R. Poli, W. B. Langdon, and N. F. McPhee. (2008). A Field Guide to Genetic Programming [Online]. Available: http://www.gp-fieldguide.org.uk [22] W. Fu, M. Johnston, and M. Zhang, “Genetic programming for edge detection: A global approach,” in Proc. IEEE Congr. Evol. Comput., Jun. 2011, pp. 254–261. [23] K. Krawiec, “Genetic programming-based construction of features for machine learning and knowledge discovery tasks,” Genetic Program. Evol. Mach., vol. 3, no. 4, pp. 329–343, 2002. [24] K. Neshatian, M. Zhang, and P. Andreae, “A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming,” IEEE Trans. Evol. Comput., vol. 16, no. 5, pp. 645–661, Oct. 2012. [25] K. Neshatian and M. Zhang, “Unsupervised elimination of redundant features using genetic programming,” in Proc. 22nd Aust. Joint Conf. Artif. Intell., 2009, pp. 432–442.

1471

[26] Y. Zhang and P. I. Rockett, “A generic multi-dimensional feature extraction method using multiobjective genetic programming,” Evol. Comput., vol. 17, no. 1, pp. 89–115, 2009. [27] D. Atkins, K. Neshatian, and M. Zhang, “A domain independent genetic programming approach to automatic feature extraction for image classification,” in Proc. IEEE Congr. Evol. Comput., Jun. 2011, pp. 238–245. [28] N. R. Harvey, S. P. Brumby, S. Perkins, J. Theiler, J. J. Szymanski, J. J. Bloch, et al., “Image feature extraction: GENIE vs conventional supervised classification techniques,” IEEE Trans. Geosci. Remote Sens., vol. 40, no. 2, pp. 393–404, Feb. 2002. [29] M. Zhang, V. B. Ciesielski, and P. Andreae, “A domain-independent window approach to multiclass object detection using genetic programming,” EURASIP J. Adv. Signal Process., vol. 2003, no. 8, pp. 841–459, 2003. [30] W. Smart and M. Zhang, “Classification strategies for image classification in genetic programming,” in Proc. Image Vision Comput. Conf., 2003, pp. 402–407. [31] M. Zhang and W. Smart, “Genetic programming with gradient descent search for multiclass object classification,” in Proc. 7th Eur. Conf. Genetic Program., 2004, pp. 399–408. [32] C. Harris and B. Buxton, “Evolving edge detectors with genetic programming,” in Proc. 1st Annu. Conf. Genetic Program., 1996, pp. 309–314. [33] G. Hollingworth, S. Smith, and A. Tyrrell, “Design of highly parallel edge detection nodes using evolutionary techniques,” in Proc. 7th Euromicro Workshop Parallel Distrib. Process., 1999, pp. 35–42. [34] S. Harding and W. Banzhaf, “Genetic programming on GPUs for image processing,” Int. J. High Performance Syst. Arch., vol. 1, no. 4, pp. 231–240, 2008. [35] M. Ebner, “On the edge detectors for robot vision using genetic programming,” in Proc. Horst-Michael Groβ, Workshop SOAVE, 1997, pp. 127–134. [36] R. Poli, “Genetic programming for image analysis,” in Proc. 1st Annu. Conf. Genetic Program., 1996, pp. 363–368. [37] J. Wang and Y. Tan, “A novel genetic programming based morphological image analysis algorithm,” in Proc. 12th Annu. Conf. Genetic Evol. Comput., 2010, pp. 979–980. [38] I. Kadar, O. Ben-Shahar, and M. Sipper, “Evolution of a local boundary detector for natural images via genetic programming and texture cues,” in Proc. 11th Annu. Conf. Genetic Evol. Comput., 2009, pp. 1887–1888. [39] T. Golonek, D. Grzechca, and J. Rutkowski, “Application of genetic programming to edge detector design,” in Proc. Int. Symp. Circuits Syst., 2006, pp. 4683–4686. [40] E. Bolis, C. Zerbi, P. Collet, J. Louchet, and E. Lutton, “A GP artificial ant for image processing: Preliminary experiments with EASEA,” in Proc. 4th Eur. Conf. Genetic Program., 2001, pp. 246–255. [41] W. Fu, M. Johnston, and M. Zhang, “Genetic programming for edge detection based on accuracy of each training image,” in Proc. 24th Aust. Joint Conf. Artif. Intell., 2011, pp. 301–310. [42] W. Fu, M. Johnston, and M. Zhang, “Soft edge maps from edge detectors evolved by genetic programming,” in Proc. IEEE Congr. Evol. Comput., Jun. 2012, pp. 24–31. [43] R. Moreno, D. Puig, C. Julia, and M. Garcia, “A new methodology for evaluation of edge detectors,” in Proc. 16th IEEE Int. Conf. Image Process., Nov. 2009, pp. 2157–2160. [44] M. Alipoor, S. Imandoost, and J. Haddadnia, “Designing edge detection filters using particle swarm optimization,” in Proc. ICEE, 2010, pp. 548–552. [45] W. Fu, M. Johnston, and M. Zhang, “A hybrid particle swarm optimisation with differential evolution approach to image segmentation,” in Proc. Int. Conf. EvoApplicat., 2011, pp. 173–182. [46] A. Mohemmed, Z. Mengjie, and M. Johnston, “Particle swarm optimization based AdaBoost for face detection,” in Proc. IEEE Congr. Evol. Comput., May 2009, pp. 2494–2501. [47] M. Setayesh, M. Zhang, and M. Johnston, “A new homogeneity-based approach to edge detection using PSO,” in Proc. Int. Conf. Image Vision Comput., 2009, pp. 231–236. [48] M. Setayesh, M. Johnston, and M. Zhang, “Edge and corner extraction using particle swarm optimisation,” in Proc. Aust. Conf. Artif. Intell., 2010, pp. 323–333. [49] U. Bhowan, M. Johnston, and M. Zhang, “Developing new fitness functions in genetic programming for classification with unbalanced data,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 406–421, Apr. 2012.

1472

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 44, NO. 8, AUGUST 2014

[50] W. Song, G. Feng, and L. Tiecheng, “Evaluating edge detection through boundary detection,” EURASIP J. Appl. Signal Process., vol. 2006, no. 1, pp. 1–15, 2006. [51] W. Fu, M. Johnston, and M. Zhang, “Analysis of diagonal derivatives in edge detectors evolved by genetic programming,” in Proc. 26th Int. Conf. Image Vision Comput., 2011, pp. 345–350. [52] S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian J. Stat., vol. 6, no. 2, pp. 65–70, 1979. [53] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 8, no. 6, pp. 679–698, Nov. 1986. [54] P. Arbel´aez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898–916, May 2011. [55] H. G´omez-Moreno, S. Maldonado-Basc´on, and F. L´opez-Ferreras, “Edge detection in noisy images using the support vector machines,” in Proc. 6th Int. Work-Conf. Artif. Natural Neural Netw. Connectionist Models Neurons, Learn. Processes Artif. Intell. I, 2001, pp. 685–692. [56] R. A. Fisher, Statistical Methods for Research Workers, 14th ed. Edinburgh, U.K.: Oliver & Boyd, 1970. [57] L. Lam, S.-W. Lee, and C. Suen, “Thinning methodologies: A comprehensive survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 9, pp. 869–885, Sep. 1992.

Wenlong Fu (M’10) received the B.Sc. and M.Sc. degrees from the School of Electronic Engineering, Xidian University, China, in 2006 and 2009, respectively. Since 2010, he has been pursuing the Ph.D. degree at Victoria University of Wellington, Wellington, New Zealand. His current research interests include machine learning, evolutionary computation, particularly genetic programming, and evolutionary vision.

Mark Johnston (M’10) received the B.Sc. degree in mathematics and computer science and the Ph.D. degree in operations research from Massey University, Palmerston North, New Zealand. He is currently a Senior Lecturer with the Victoria University of Wellington, Wellington, New Zealand, where he teaches various operations research courses. His current research interests include primarily combinatorial optimization and evolutionary computation, with a particular interest in scheduling models, genetic programming, and multiobjective optimization.

Mengjie Zhang (SM’10) received the B.E. and M.E. degrees from the Artificial Intelligence Research Center, Agricultural University of Hebei, Baoding, China, in 1989 and 1992, respectively, and the Ph.D. degree in computer science from RMIT University, Melbourne, Australia, in 2000. Since 2000, he has been with the Victoria University of Wellington, Wellington, New Zealand. He is currently a Professor of computer science and heads the Evolutionary Computation Research Group. His current research interests include evolutionary computation, genetic programming, and particle swarm optimization with application areas of image analysis, multiobjective optimization, classification with unbalanced data, and feature selection and dimension reduction for classification with high dimensions. Dr. Zhang is a member of the IEEE Computer Society, the IEEE CI Society, and the IEEE SMC Society. He is also a member of the IEEE CIS Evolutionary Computation Technical Committee, a member of the IEEE CIS Intelligent Systems Applications Technical Committee, a Vice Chair of the IEEE CIS Task Force on Evolutionary Computer Vision and Image Processing, and a Committee Member of the IEEE New Zealand Central Section. He is a member of the ACM and the ACM SIGEVO Group. He has been serving as an Associate Editor or Editorial Board Member for five international journals.