Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
Contents lists available at ScienceDirect
Colloids and Surfaces B: Biointerfaces journal homepage: www.elsevier.com/locate/colsurfb
Pattern recognition for identification of lysozyme droplet solution chemistry Heather Meloy Gorr ∗ , Ziye Xiong, John A. Barnard Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA, USA
a r t i c l e
i n f o
Article history: Received 25 June 2013 Received in revised form 20 October 2013 Accepted 3 November 2013 Available online 12 November 2013 Keywords: Droplet evaporation Sessile drop Image pattern classification
a b s t r a c t Pattern formation during evaporation of a colloidal sessile droplet is a phenomenon relevant to a wide variety of scientific disciplines. The patterns remaining on the substrate are indicative of the transport mechanisms and phase transitions occurring during evaporation and may reflect the solution chemistry of the fluid [1–18]. Pattern formation during evaporation of droplets of biofluids has also been examined and these complex patterns may reflect the health of the patient [23–31]. Automatic detection of variations in the fluid composition based on these deposit patterns could lead to rapid screening for diagnostic or quality control purposes. In this study, a pattern recognition algorithm is presented to differentiate between deposits containing various solution compositions. The deposits studied are from droplets of simplified, model biological fluids of aqueous lysozyme and NaCl solutions. For the solution concentrations examined here, the deposit patterns are dependent upon the initial solution composition. Deposit images are represented by extracting features using the Gabor wavelet, similar to the method used for iris recognition. Two popular pattern recognition algorithms are used to classify the deposits. The k-means clustering algorithm is used to test if incremental changes in solution concentration result in reproducible and statistically interpretable variations in the deposit patterns. The k-nearest neighbor algorithm is also used to classify the deposit images by solution concentration based on a set of training images for each class. Here, we demonstrate that the deposit patterns may act as a “fingerprint” for identification of solution chemistry. The results of this study are very promising, with classification accuracies of 90–97.5%. © 2013 Elsevier B.V. All rights reserved.
1. Introduction The self-organization of solute particles during evaporation of colloidal sessile droplets has attracted the attention of researchers from a variety of disciplines over the past few decades. Interesting solute patterns are formed on the substrate, which are a result of the evaporative flux as well as the multi-scale transport mechanisms and phase transitions occurring within the droplet. Typically, a “coffee ring” of solute is formed at the periphery, due to radial capillary flows generated during evaporation [1–3]. In addition to ring-like structures, a variety of other patterns have been observed including spherical cap-like deposits [4], pillarlike structures [5], and deposits containing crystalline features [6,7]. These patterns may be influenced by capillary flows [8,9], Marangoni convection [10,11], contact line depinning [12,13], advection–diffusion of particles [14], nanoparticle concentration [15], particle–particle interactions [16], surface functional groups [17], and particle–substrate interactions [10,18]. Self-organization
∗ Corresponding author. Tel.: +1 412 651 5196. E-mail addresses: [email protected], [email protected] (H.M. Gorr). 0927-7765/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.colsurfb.2013.11.005
during evaporation is applicable to a wide range of technologies including 3-D [5] and inkjet printing [19,20], carbon nanotube production [21], and DNA micro-array printing [22]. In addition to the numerous industrial applications, recent studies have examined pattern formation during evaporation of human biological fluids for medical screening and diagnostic purposes. Researchers have observed the patterns remaining after evaporation of blood serum [23,24], whole blood [25–27], tears [28–30], and other biofluids [31] and found that these patterns may reflect the health of a patient. The deposit patterns exhibit unique and distinctive morphological characteristics which may act as markers for various pathological processes. Automated identification of these patterns may therefore play a role in developing rapid disease detection. Recent studies have demonstrated success using image analysis [23,27] and pattern recognition techniques [32] in classifying solute patterns from various fluids. For example, Zeid et al. [27] studied the effect of the droplet evaporation rate on formation of patterns in drops of whole blood. They used a segmentation method for automatic extraction and characterization of the patterns, to aid in analysis of the morphological characteristics of the patterns. In addition, Kim et al. [32] successfully employed pattern
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
recognition in identifying a variety of fluids such as popular soft drinks, wines, and colloidal systems with organic and inorganic solute. By testing several machine learning algorithms, they found that the deposit patterns could be classified based on the various combinations of liquid and substrate chemistry. These algorithms included k-nearest neighbor classification as well as k-means, average linking, and spectral clustering algorithms. The multicomponent flows and accompanying phase transitions can be difficult to characterize in biofluid systems, but they are broadly similar to those of aqueous, multi-component colloidal suspensions of biological relevance. Recently, we studied pattern formation during sessile droplet evaporation of dilute aqueous solutions of lysozyme [4,33] and lysozyme + NaCl [34]. Lysozyme is a well-studied globular protein found in human biofluids, including tears and saliva. We examined the morphology of dried droplet patterns with diameters ranging from 1 m to 2 mm using optical microscopy and atomic force microscopy (AFM). The deposit shapes depended sensitively on the characteristic time and length scales of the system. With the addition of NaCl in solution, the patterns showed distinct regions including an amorphous peripheral ring as well as a central region containing crystalline and dendritic structures. These distinctive patterns were highly sensitive to the initial solution composition of lysozyme and NaCl. Therefore, these patterns may act as a “fingerprint” which reflects the solution chemistry of the droplet. In this study, a pattern recognition algorithm is presented to differentiate between deposit patterns from lysozyme solution droplets with varying concentrations of NaCl. The deposit images are represented by a feature vector, determined by applying the 2D Gabor function to the images, similar to the method of feature extraction used in iris recognition [35]. Two of the most popular machine learning algorithms are employed to support the main ideas of this study. An unsupervised learning algorithm (kmeans clustering) is used to demonstrate that incremental changes in solution concentration result in reproducible and statistically interpretable variations in the deposit patterns. Furthermore, a supervised learning algorithm (k-nearest neighbor classification) is used to classify the deposit images by solution concentration based on a set of training images. 2. Experimental methods 2.1. Lysozyme solutions Lysozyme is a globular protein found in high concentration in human mucosal secretions, including tears [28,36,37] and saliva [38]. Lysozyme is roughly ellipsoidal in shape with approximate dimensions of 3.0 nm × 3.0 nm × 4.5 nm, molecular mass of 14 kDa, and carries a net positive charge at physiological pH [39]. This wellstudied protein has been the subject of many protein crystallization [40–43] and self-assembly [44–49] experiments. Concentrated lysozyme and sodium chloride stock solutions were prepared using high purity lysozyme powder (Sigma–Aldrich, L6876) and sodium chloride (NaCl). As-received powders were dissolved in deionized water (Millipore, 18.2 M cm) at 35 ◦ C to concentrations of 2.0 wt% and stored at 2 ◦ C. These stock solutions were diluted with DI water to create the desired concentrations. In this study, the solutions contained 1.0 wt% lysozyme (ϕL ) with NaCl concentrations (ϕN ) of 0.10, 0.50, and 1.0 wt%. Solutions were brought to room temperature prior to deposition and used within one week. 2.2. Substrates and deposition Silicon wafer substrates were cleaned in ultrasonic baths (Fisher Scientific, FS20) of alternating isopropyl alcohol (IPA), acetone, and
171
Fig. 1. Representative deposits from droplets containing 1.00 wt% lysozyme and NaCl in concentrations of ϕN = 0, 0.10, 0.50, and 1.00 wt% (D ∼ 100 m).
DI water, and blown dry with compressed air. A drop-on-demand printer (MicroFab Technologies, Inc., JetLab4® ) was used to produce micro-arrays of droplets. The arrays consisted of droplets in a 5 × 5 grid with diameters D ∼ 100 m and spacing of ∼100 m. This spacing was chosen to minimize overlap in the evaporation fields of neighboring droplets. The droplets evaporated under ambient conditions in our climate-controlled laboratory with T ∼ 25 ◦ C and relative humidity of ∼30%. 2.3. Image acquisition and preprocessing The deposits were observed after evaporation in ambient conditions using a digital reflection optical microscope (Keyence, VHX-600). The colors were adjusted by setting the white balance and the images were collected in “image enhancement” mode, which automatically adjusts the contrast in the image. The entire array of deposits was captured and the individual deposits were cropped from the array image. These images were cropped to a region slightly larger than the deposit to minimize the background. The deposit images were rescaled to 150 × 150 pixels, which was similar to the dimensions of the cropped deposit images. The cropped and scaled deposit images were then converted to grayscale intensity images. Representative deposit images for the concentrations considered in this study are collected in Fig. 1. It should be noted that deposits with irregular shapes, likely due to local substrate heterogeneities, were not used in this experiment. Software was developed using MATLAB® to process the images as described above, select training and test images at random, perform the image feature extraction, and classify the deposit images (Statistics Toolbox® ) [50]. The algorithm and its implementation are described in the following sections. 3. Algorithm 3.1. Image feature extraction In order to classify the images, specific image information must be obtained and used as a representative feature vector. Wavelet transforms have been used for feature extraction in a number of pattern recognition applications [32,51,52], due to the robust and informative nature of the wavelet. The wavelet represents the spatial variation of a signal and the energy of the signal can be represented by a few expansion coefficients. These coefficients contain information about the signal at various scales and can be used to represent texture when applied to an image.
172
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
y) from Eq. (3), we start by fixing m = 1 and changing the orientation by incrementing n from 1 to 8. Then, the computation is performed for m = 2 and incrementing n from 1 to 8, and so on. The response, v, is a collection of coefficients which is a vector of length (m*n × 1). These are complex values and for computational simplicity, the magnitude of the complex vector, |v|, is taken as the feature vector in this experiment. This vector is used to represent each image for classification. 3.2. Unsupervised learning
Fig. 2. Three dimensional surface plot of the 2D Gabor function.
In this experiment, the Gabor function was used to extract textural information from the deposit images. This method is used in iris recognition [35] and other image processing applications [53,54], as the Gabor function is used to model receptive field profiles of individual neurons in the visual cortex [53,55,56]. Consider the wavelet −2m (x , y ), where x and y contain dilations in the size m (x, y) = 2 of the wavelet by 2m and rotations in angle by : x = 2−m [x sin() − cos()]
(1)
y = 2−m [x sin() + cos()]
The response, v, to an arbitrary image I(x, y) is the product of the image intensity distribution and the receptive field function g(x, y) [55]:
v=
(2)
I(x, y)g(x, y) dx dy ˝
where ˝ is the set of all image points. The receptive field function is modeled with the 2D Gabor function [57]:
g(x, y) = exp
−
x 2 + 2y 2 2 2
2x
exp i
+ϑ
(3)
parameterized as in Refs. [53,56]. The 2D Gabor function in Eq. (3) is a Gaussian kernel modulated by a plane wave [57]. A 3-dimensional surface plot of the Gabor function, as parameterized in this experiment, is given in Fig. 2. The values of the parameters , ˛, , ϑ, m, n used in this experiment are collected in Table 1. The eccentricity of the receptive field ellipse, or the spatial aspect ratio, , is taken as = 0.5. The size of the receptive field is given by ˛ = 0.56 , which is the standard deviation of the Gaussian factor, and the wavelength, = 2, is given in pixels [56]. The phase offset is described by ϑ. When ϑ = 0 and , the function g(x, y) is symmetric and when ϑ = ±/2, the function, g(x, y) is antisymmetric. The variables m and n appear in x and y in Eq. (1). The dilation of the wavelet is described by 2m , where m is the number of “scales.” The angle, , describes the orientation to the normal (x -axis) with = n (/8), where n = 0, 1, 2, 3, . . ., 8 is the number of orientations. In this experiment, n = 8 orientations and the number of scales, m, is varied for comparison. To determine g(x, Table 1 Parameters used in the Gabor function. Parameter
Value
˛ ϑ n m
0.56 0.5 2 8 6, 8, 10, 12
In this experiment, unsupervised learning was used to determine if the deposit patterns were unique enough to be automatically grouped by class, or initial NaCl concentration. In unsupervised learning techniques, the data is not identified by class a priori. The feature vectors are grouped into classes by some measure of similarity (distance, connectivity, etc.). The popular k-means clustering algorithm is applied in this study with k = 4 clusters, representing four solution concentrations. The k-means algorithm generally consists of two steps: assign samples to the nearest cluster mean and calculate the new cluster centroids [58]. This process is repeated until no new cluster assignments are made. In this study, Euclidean distance was used as the distance metric and the four cluster means were initialized to random seeds. The clustering was repeated five times with new initial centroids for each cluster. The accepted clustering had the minimum sum of the point-to-centroid distances [50]. 3.3. Supervised learning In classification, a training set is represented statistically for each class and each new sample is compared to the training data and classified accordingly. In this experiment, the k-nearest neighbor (k-NN) algorithm was used for classification. The robust, non-parametric nature of k-NN makes it a natural fit for computer vision applications and has been used in image pattern recognition applications [32]. In k-NN classification, each vector is classified according to a majority vote of the k closest feature vectors. In the training portion of the k-NN algorithm, a subset of feature vectors and associated class labels are stored as the training set. Unlabeled “test” feature vectors are then compared and assigned according to the class of the nearest neighbors. In this experiment, the decision is based on Euclidean distance and k = 1. In other words, the test image is predicted to belong to the class of the closest training image, determined by minimizing the Euclidean distance between feature vectors. 4. Results 4.1. Pattern formation Arrays of deposits from all four concentrations were examined and captured with a digital optical microscope. Several deposits from each concentration were also captured at higher magnification. Within each array, the deposit patterns had similar characteristics. A small number of deposits had irregular shapes, due to local substrate heterogeneities, and these deposits were excluded from the analysis. There were no additional notable variations in the patterns due to environmental factors. Five deposit images from each of the solution concentrations are shown in Fig. 1. A representative deposit from each concentration is illustrated in more detail in Fig. 3. The morphologies of the deposits observed were consistent with our previous observations [4,33,34]. The deposits consisted of an amorphous, lysozyme-rich, peripheral ring and a central region
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
173
Fig. 4. Line plot of mean coefficient vectors for deposits containing 1.00 wt% lysozyme and the four concentrations of NaCl, indicated in the legend. The x-axis represents the index of the vector and the y-axis represents the values of the coefficients.
Fig. 3. Dried deposits from solutions containing 1.00 wt% lysozyme and NaCl in concentrations ϕN = (a) 0, (b) 0.10, (c) 0.50, and (d) 1.00 wt% (D ∼ 100 m).
containing crystallites in the deposits with NaCl (Fig. 3b–d). The formation of these patterns is described in detail in Refs. [4,33,34] and summarized here. Upon deposition, lysozyme molecules adsorb to the substrate, pinning the droplet. The contact angle of the droplet decreases during evaporation and radial flows are generated which transport unadsorbed lysozyme molecules to the ring. Protein molecules accumulate in the perimeter ring, undergoing a phase transition which proceeds inward toward the center of the drop. The remaining liquid in the central region depins and recedes until evaporation is complete. With the addition of NaCl in solution, the ionic strength of the remaining solution increases, driving formation of hierarchical protein structures. In the final stages of evaporation, the remaining liquid percolates through the solid deposit and is accompanied by extensive cracking in the perimeter ring and the formation of crystallites in the central region of the deposit for solutions containing NaCl. With increasing NaCl in solution, more crystallites are present in the central region of the deposit and the width of the peripheral ring decreases. This is attributed to the NaCl screening the positive repulsive charges and therefore leads to the formation of lysozyme aggregates at an earlier time during evaporation [34].
4.2. Feature extraction The deposit images were cropped from the original array image, rescaled to 150 × 150 pixels, and converted to grayscale. The Gabor function in Eq. (3) was determined with scales of m = 6, 8, 10, and 12 and the parameters listed in Table 1. This was applied to each processed image, as described in Section 3, resulting in individual feature vectors containing the coefficients, |v|. Each vector had a size of (m*n × 1) coefficients. Plots of mean coefficients with m = 10 for each concentration are collected in Fig. 4. Fig. 4 shows line plots of the mean coefficient vectors, estimated from 20 random images from each concentration. In this case, m = 10 scales, resulting in a vector of 80 coefficients. These line plots illustrate that the distributions of coefficients for all classes have similar overall shapes, but the values of the coefficients generally vary between the classes.
To further examine the association between the mean coefficients from the various classes, the sample (linear) correlation was determined from the data shown in Fig. 4. The correlation coefficients between the various classes ranged from 0.612 to 0.929. The largest correlation was between the vectors for deposits with 0 and 0.10 wt% NaCl and the lowest correlation occurred between 0 and 1.00 wt% NaCl. The vector representing the 0.50 wt% class was most strongly correlated with the 0.10 wt% vector (0.911). In comparison, there was less correlation between the vector from the 1.00 wt% deposits and the other classes, with correlation coefficients ranging from 0.612 to 0.693. The 1.00 wt% vector was most strongly correlated with the 0.50 wt% vector and least correlated with the vector representing deposits with no NaCl. 4.3. Unsupervised learning algorithm The k-means clustering algorithm was used, as described in Section 3, to group the deposit images into their respective classes, without a priori information. The initial cluster means for the four classes were initialized to random seeds and 20 deposit images for each class were chosen at random. k-means clustering was performed on the feature vectors for these images. The success of the k-means algorithm was measured by the “purity” of the clustering. To determine the purity, each cluster was labeled as the class which occurred most frequently in the cluster. Then the number of correctly assigned vectors in every cluster was divided by the total number of vectors to find the accuracy of the clustering [59]. The k-means clustering test was repeated 5 times, with a new set of random images and different initial cluster seeds. The mean accuracy results for the k-means clustering algorithm are presented in Fig. 5. The mean accuracy for the k-means clustering algorithm was 64%, 76%, 64.3%, and 76.3% for scales of 6, 8, 10, and 12, respectively. Fig. 5 also illustrates the mean accuracy by class with 58.75%, 87.75%, 57%, and 77% for ϕN = 0, 0.10, 0.50, and 1.00 wt%, respectively. Deposits from the droplets containing no NaCl were often grouped with deposits containing ϕN = 0.10 and vice versa. In addition, deposits with ϕN = 0.50 and 1.00 wt% were frequently grouped together. 4.4. Supervised learning algorithm To test the supervised learning algorithm, ten training images from each class were chosen at random to create the k-NN classification model. Ten more images from each class were chosen at random from the remaining images as test images. The feature vector from each test image was assigned to the class of the nearest
174
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
Fig. 5. Clustering accuracy by class (NaCl content) of feature vectors, determined with 6, 8, 10, and 12 scales.
Fig. 6. Mean accuracy results of the k-nearest neighbor (k-NN) classification by class (NaCl content) for feature vectors determined with 6, 8, 10, and 12 scales.
training image by Euclidean distance. The success of the supervised learning algorithm was measured by determining the number of correct classifications for each class and finding the accuracy. The classification was repeated five times, with different sets of training and test images selected at random. The results of the mean k-NN classification accuracy for each class with m = 6, 8, 10, and 12 are collected in Fig. 6. Overall, the k-NN algorithm produced mean accuracies of 90.5%, 94.5%, 97.5%, and 95% for scales of 6, 8, 10, and 12, respectively. Generally, as the number of scales increased, the accuracy improved in this case. The k-NN algorithm was most successful with 10 and 12 scales. By class, the deposits with no NaCl had the highest accuracy results, with 99.5% of the deposits classified correctly. Of the deposits that were misclassified, a majority of deposits with ϕN = 0.10 wt% were misclassified as the deposits with ϕN = 0.00 wt%. A majority of the misclassified deposits containing ϕN = 0.50 and 1.00 wt% were also often mistaken as one another. 5. Discussion Based on the results of this study and our previous observations [4,33,34], the deposit patterns are affected by the initial NaCl solution concentration. The deposits have similar characteristics within the same concentration (Fig. 1) and are clearly influenced by the initial concentration of NaCl (Figs. 1 and 3). More crystallites are present in deposits with higher NaCl concentrations and the width of the peripheral ring decreases as the NaCl content increases. As discussed in our previous findings [31], this can be explained by the formation of aggregates at earlier stages of evaporation. Upon deposition, the positively charged lysozyme molecules are
attracted to the negatively charged SiO2 substrate. The chemistry of the liquid in the drying droplet changes during evaporation as lysozyme molecules leave solution by adsorbing to the substrate or contributing to the gel at the peripheral ring. The ionic strength of the remaining solution increases and the NaCl screens the positive, repulsive charges. This drives liquid–liquid phase separation [51], forming clusters and fractal aggregates of lysozyme [47]. These are formed at an earlier time with increasing NaCl concentration, resulting in more crystallites in deposits with higher initial NaCl content. To further examine these deposit patterns, the images were preprocessed and the Gabor wavelet was applied, resulting in a vector of coefficients, representing each image. The unsupervised learning (k-means clustering) algorithm was implemented to examine whether the feature vectors could be naturally grouped by class (NaCl concentration). The k-means clustering algorithm was applied to the vectors, resulting in clusters with overall accuracies of 64%, 76%, 64.3%, and 76.3% for scales of m = 6, 8, 10, and 12, respectively. This suggests that while there are distinct differences between the patterns, there were also a number of feature vectors that were grouped incorrectly. In the unsupervised learning study, the deposits containing 0 and 0.10 wt% NaCl were frequently grouped together. The same was true between the deposits containing 0.50 and 1.00 wt% NaCl. These incorrect groupings are likely due to similarities between these deposit patterns. For example, the deposits containing 0.50 and 1.00 wt% have crystallites throughout the entire deposit and the deposits with 0 and 0.10 wt% have a central region without crystallites and more features toward the periphery of the deposit. These visual similarities are reflected in the distribution of coefficients in Fig. 4 and will therefore affect the quality of the clustering. In addition, the largest correlation between mean feature vectors occurred for deposits with no NaCl and 0.10 wt% NaCl. This correlation may result in the incorrect clustering of deposits from these two classes. Furthermore, the 1.00 wt% vector was most strongly correlated with the 0.50 wt% vector, likely resulting in the frequent grouping of these two classes. Additional image preprocessing or feature selection may highlight the important features from each concentration and improve the accuracy of the clustering algorithm. The accuracy improved significantly with the supervised learning algorithm, which incorporated training data. Images were chosen at random to train the k-nearest neighbor classifier and random test images were classified accordingly. The mean accuracy for the supervised learning algorithm was 90.5%, 94.5%, 97.5%, and 95% with scales of 6, 8, 10, and 12, respectively. The improvement in accuracy over the unsupervised learning algorithm can likely be attributed to the addition of training data. This notable improvement suggests that while the patterns in this study may be too strongly correlated to cluster, these patterns may be classified based on a training set. The number of scales of the Gabor function was varied in this experiment for comparison. As the number of scales increased, the accuracy of the k-NN algorithm improved in general, although an interpretable relationship was not explored in this study. This general improvement is likely a result of the multi-resolution nature of the wavelet. As the scaling parameter, m, changes, the shape of the wavelet changes resulting in a representation of detail or resolution. There was no obvious relationship in the accuracy of the k-means clustering algorithm and the number of scales. The next steps for improvement of the algorithm would include a more sophisticated optimization of the number of scales. The implementation of this software in MATLAB® is robust, as is computation of the wavelet by nature. Each step of processing images, determining coefficients, developing a training set, and
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
classification or clustering took only seconds to process on a home PC. 6. Conclusions In this study, a pattern recognition algorithm is presented in an attempt to classify drop deposit images based on incremental changes in solution concentration. Deposit images were cropped to a region slightly larger than the deposit, rescaled to 150 × 150 pixels, and converted to grayscale. The 2D Gabor function was used to extract feature information from these processed deposit images. The number of scales (resolution) was varied, using scales of 6, 8, 10, and 12. Two different algorithms for classification were tested: unsupervised and supervised learning. The unsupervised learning algorithm was based on the k-means clustering algorithm, grouping the samples into clusters depending on the Euclidean distance to the nearest feature vectors. The unsupervised learning algorithm achieved overall accuracies of 64%, 76%, 64.3%, and 76.3% for scales of 6, 8, 10, and 12, respectively. With the addition of training images in the supervised learning algorithm, the accuracy was greatly improved. The mean accuracy for the k-nearest neighbor algorithm was 90.5%, 94.5%, 97.5%, and 95% for Gabor wavelet scales of 6, 8, 10, and 12, respectively. This study demonstrates that incremental changes in solution concentration result in reproducible and statistically interpretable variations in the self-assembled patterns that develop during sessile droplet evaporation of simplified model biological fluids. Furthermore, the patterns remaining after evaporation of these simplified, model biofluids may act as a “fingerprint” for identifying the nature of the fluid. The overall results of this study are promising and indicate that automated screening and disease detection could be realized with improved data sets and by including a larger library of images. In addition, the robust and scalable nature of this algorithm makes this relevant to a number of problems and can be easily optimized for various applications such as quality control and automatic detection of product contamination, for example. The next steps in development of this software would include a larger database of images and considering a greater number of concentrations. The accuracy of these algorithms may also be improved by additional image preprocessing and using images with higher resolution. In this study, the deposit images were cropped from the array image, however, capturing an individual deposit would provide images with higher resolution. Finally, additional classification algorithms should also be tested as the experiment is scaled up. Acknowledgements The authors acknowledge the facilities, scientific, and technical assistance of the Materials Micro-Characterization Laboratory of the Department of Mechanical Engineering and Materials Science, Swanson School of Engineering, University of Pittsburgh. References [1] R.D. Deegan, O. Bakajin, T.F. Dupont, G. Huber, S.R. Nagel, T.A. Witten, Nature 389 (1997) 827–829. [2] R.D. Deegan, Phys. Rev. E 61 (2000) 475–485. [3] R.D. Deegan, O. Bakajin, T.F. Dupont, G. Huber, S.R. Nagel, T.A. Witten, Phys. Rev. E 62 (2000) 756–762. [4] H.M. Gorr, J.M. Zueger, J.A. Barnard, J. Phys. Chem. B (2012). [5] K.A. Baldwin, M. Granjard, D.I. Willmer, K. Sefiane, D.J. Fairhurst, Soft Matter 7 (2011).
[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
[25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59]
175
G. Chen, G.J. Mohamed, Eur. Phys. J. E 33 (2010) 19–26. D. Kaya, V.A. Belyi, M. Muthukumar, J. Chem. Phys. 113 (2010). Y.O. Popov, Phys. Rev. E 71 (2005) 36313. H. Hu, R.G. Larson, J. Phys. Chem. B 106 (2002) 1334–1344. R. Bhardwaj, X. Fang, P. Somasundaran, D. Attinger, Langmuir 26 (2010) 7833–7842. H. Hu, R.G. Larson, J. Phys. Chem. B 110 (2006) 7090–7094. J.R. Moffat, K. Sefiane, M.E.R. Shanahan, J. Phys. Chem. B 113 (2009) 8860–8866. A.S. Sangani, C. Lu, K. Su, J.A. Schwarz, Phys. Rev. E 80 (2009). A.G. Marin, H. Gelderblom, D. Lohse, J.H. Snoeijer, Phys. Rev. Lett. 107 (2011). D. Brutin, Colloids Surf. A 429 (2013) 112–120. P.J. Yunker, T. Still, M.A. Lohr, A.G. Yodh, Nature 476 (2011) 308–311. F. Carle, D. Brutin, Langmuir 29 (2013) 9962–9966. L.V. Andreeva, A.V. Koshkin, P.V. Lebedev-Stepanov, A.N. Petrov, M.V. Alfimov, Colloids Surf. A 300 (2007) 300–306. J. Park, J. Moon, Langmuir 22 (2006) 3506–3513. J.X. Zhou, J.Y.H. Fuh, H.T. Loh, Y.S. Wong, Y.S. Ng, J.J. Gray, S.J. Chua, Int. J. Adv. Manuf. Technol. 48 (2010) 243–250. W.R. Small, C.D. Walton, J. Loos, M.I.H. Panhuis, J. Phys. Chem. B 110 (2006) 13029–13036. V. Dugas, J. Broutin, E. Souteyrand, Langmuir 21 (2005) 9130–9136. A.A. Killeen, N. Ossina, R.C. McGlennen, S. Minnerath, J. Borgos, V. Alexandrov, A. Sarvazyan, Mol. Diag. Ther. 10 (2006) 371–380. T.A. Yakhno, V.G. Yakhno, A.G. Sanin, O.A. Sanina, A.S. Pelyushenko, N.A. Egorova, I.G. Terentiev, S.V. Smetanina, O.V. Korochkina, E.V. Yashukova, IEEE Eng. Med. Biol. 24 (2005) 96–104. D. Brutin, B. Sobac, B. Loquet, J. Sampol, J. Fluid Mech. 667 (2011) 85–95. B. Sobac, D. Brutin, Phys. Rev. E 84 (2011) 011603. B.W. Zeid, J. Vicente, D. Brutin, Colloids Surf. A (2013) 139–146. M.T. Kuo, C.C. Lin, H.Y. Liu, H.C. Chang, Invest. Ophthalmol. Vis. Sci. 52 (2011) 4942–4950. L. Ravazzoni, C. Ghini, A. Macri, M. Rolando, Invest. Ophthalmol. Vis. Sci. 38 (1997) s153. E.I. Pearce, A. Tomlinson, Ophthalmic Physiol. Opt. 20 (2000) 306–313. V.N. Shabalin, S.N. Shatokhina, Singapore Med. J. 48 (2007) 440–446. N. Kim, Z. Li, C. Hurth, F. Zenhausern, S.F. Chang, D. Attinger, Anal. Methods 4 (2012) 50–57. H.M. Gorr, J.Z. Zueger, J.A. Barnard, Langmuir (2012). H.M. Gorr, J.M. Zueger, D.R. McAdams, J.A. Barnard, Colloids Surf. B 103 (2013) 59–66. J. Daugman, U.S. Patent No. 5,291,560: Biometric Personal Identification System Based on Iris Analysis, 1994. D. Ronen, E. Eylan, A. Romano, R. Stein, M. Modan, Invest. Ophthalmol. Vis. Sci. 14 (1975) 479–484. R.J. Fullard, C. Snyder, Invest. Ophthalmol. Vis. Sci. 31 (1990) 1119–1126. C.-K. Yeh, M.W.J. Dodds, P. Zuo, D.A. Johnson, Arch. Oral Biol. 42 (1997) 25–31. C.C.F. Blake, D.F. Koenig, G.A. Mair, A.C.T. North, D.C. Phillips, V.R. Sarma, Nature 206 (1965) 757–761. M.L. Pusey, R.S. Snyder, R. Naumann, J. Biol. Chem. 261 (1986) 6524–6529. M. Muschol, F. Rosenberger, J. Chem. Phys. 107 (1997) 1953–1962. Y. Georgalis, P. Umbach, D.M. Soumpasis, W. Saenger, J. Am. Chem. Soc. 120 (1998) 5539–5548. M.C.R. Heijna, M.J. Theelen, W.J.P. van Enckevort, E. Vlieg, J. Phys. Chem. B 111 (2007) 1567–1573. A. Stradner, H. Sedgwick, F. Cardinaux, W.C.K. Poon, S.U. Egelhaaf, P. Schurtenberger, Nature 432 (2004) 492–495. F. Cardinaux, A. Stradner, P. Schurtenberger, F. Sciortino, E. Zaccarelli, Europhys. Lett. 77 (2007). S.P. Rozhkov, A.S. Goryunov, Biophys. Chem. 151 (2010) 22–28. F. Cardinaux, E. Zaccarelli, A. Stradner, S. Bucciarelli, B. Farago, S. Egelhaaf, F. Sciortino, P. Schurtenberger, J. Phys. Chem. B 115 (2011) 7227–7237. E. Blomberg, P.M. Claesson, J.C. Froberg, R.D. Tilton, Langmuir 10 (1994) 2325–2334. S.M. Daly, T.M. Przybycien, R.D. Tilton, Colloids Surf. B 57 (2007) 81–88. MATLAB, Statistics Toolbox, Release 2013a, The MathWorks, Inc., Natick, MA, USA, 2013. S. Fong, J. Biomed. Biotechnol. 2012 (2012) 1–12. R.S.H. Istepanian, A. Sungoor, J. Nebel, Conf. Proc. IEEE Eng. Med. Biol. Soc. (2007) 4629–4632. N. Petkov, Future Generation Comput. Syst. 11 (1995) 451–465. J.A. Montoya Zegarra, N.J. Leite, R. da Silva Torres, J. Comput. Appl. Math. 227 (2009) 294–307. J. Daugman, J. Opt. Soc. Am. A 2 (1985) 1160–1169. N. Petkov, P. Kruizinga, Biol. Cybern. 76 (1997) 83–96. J. Daugman, Int. J. Wavelets Multiresolution Inform. Process. 1 (2003) 1–17. W. Snyder, H. Qi, Machine Vision, Cambridge University Press, New York, 2004. W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Conditional Entropy and Mutual Information in: Numerical Recipes: The Art of Scientific Computing, 3rd ed., Cambridge University Press, New York, 2007 (Section 14.7.3).
Contents lists available at ScienceDirect
Colloids and Surfaces B: Biointerfaces journal homepage: www.elsevier.com/locate/colsurfb
Pattern recognition for identification of lysozyme droplet solution chemistry Heather Meloy Gorr ∗ , Ziye Xiong, John A. Barnard Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA, USA
a r t i c l e
i n f o
Article history: Received 25 June 2013 Received in revised form 20 October 2013 Accepted 3 November 2013 Available online 12 November 2013 Keywords: Droplet evaporation Sessile drop Image pattern classification
a b s t r a c t Pattern formation during evaporation of a colloidal sessile droplet is a phenomenon relevant to a wide variety of scientific disciplines. The patterns remaining on the substrate are indicative of the transport mechanisms and phase transitions occurring during evaporation and may reflect the solution chemistry of the fluid [1–18]. Pattern formation during evaporation of droplets of biofluids has also been examined and these complex patterns may reflect the health of the patient [23–31]. Automatic detection of variations in the fluid composition based on these deposit patterns could lead to rapid screening for diagnostic or quality control purposes. In this study, a pattern recognition algorithm is presented to differentiate between deposits containing various solution compositions. The deposits studied are from droplets of simplified, model biological fluids of aqueous lysozyme and NaCl solutions. For the solution concentrations examined here, the deposit patterns are dependent upon the initial solution composition. Deposit images are represented by extracting features using the Gabor wavelet, similar to the method used for iris recognition. Two popular pattern recognition algorithms are used to classify the deposits. The k-means clustering algorithm is used to test if incremental changes in solution concentration result in reproducible and statistically interpretable variations in the deposit patterns. The k-nearest neighbor algorithm is also used to classify the deposit images by solution concentration based on a set of training images for each class. Here, we demonstrate that the deposit patterns may act as a “fingerprint” for identification of solution chemistry. The results of this study are very promising, with classification accuracies of 90–97.5%. © 2013 Elsevier B.V. All rights reserved.
1. Introduction The self-organization of solute particles during evaporation of colloidal sessile droplets has attracted the attention of researchers from a variety of disciplines over the past few decades. Interesting solute patterns are formed on the substrate, which are a result of the evaporative flux as well as the multi-scale transport mechanisms and phase transitions occurring within the droplet. Typically, a “coffee ring” of solute is formed at the periphery, due to radial capillary flows generated during evaporation [1–3]. In addition to ring-like structures, a variety of other patterns have been observed including spherical cap-like deposits [4], pillarlike structures [5], and deposits containing crystalline features [6,7]. These patterns may be influenced by capillary flows [8,9], Marangoni convection [10,11], contact line depinning [12,13], advection–diffusion of particles [14], nanoparticle concentration [15], particle–particle interactions [16], surface functional groups [17], and particle–substrate interactions [10,18]. Self-organization
∗ Corresponding author. Tel.: +1 412 651 5196. E-mail addresses: [email protected], [email protected] (H.M. Gorr). 0927-7765/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.colsurfb.2013.11.005
during evaporation is applicable to a wide range of technologies including 3-D [5] and inkjet printing [19,20], carbon nanotube production [21], and DNA micro-array printing [22]. In addition to the numerous industrial applications, recent studies have examined pattern formation during evaporation of human biological fluids for medical screening and diagnostic purposes. Researchers have observed the patterns remaining after evaporation of blood serum [23,24], whole blood [25–27], tears [28–30], and other biofluids [31] and found that these patterns may reflect the health of a patient. The deposit patterns exhibit unique and distinctive morphological characteristics which may act as markers for various pathological processes. Automated identification of these patterns may therefore play a role in developing rapid disease detection. Recent studies have demonstrated success using image analysis [23,27] and pattern recognition techniques [32] in classifying solute patterns from various fluids. For example, Zeid et al. [27] studied the effect of the droplet evaporation rate on formation of patterns in drops of whole blood. They used a segmentation method for automatic extraction and characterization of the patterns, to aid in analysis of the morphological characteristics of the patterns. In addition, Kim et al. [32] successfully employed pattern
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
recognition in identifying a variety of fluids such as popular soft drinks, wines, and colloidal systems with organic and inorganic solute. By testing several machine learning algorithms, they found that the deposit patterns could be classified based on the various combinations of liquid and substrate chemistry. These algorithms included k-nearest neighbor classification as well as k-means, average linking, and spectral clustering algorithms. The multicomponent flows and accompanying phase transitions can be difficult to characterize in biofluid systems, but they are broadly similar to those of aqueous, multi-component colloidal suspensions of biological relevance. Recently, we studied pattern formation during sessile droplet evaporation of dilute aqueous solutions of lysozyme [4,33] and lysozyme + NaCl [34]. Lysozyme is a well-studied globular protein found in human biofluids, including tears and saliva. We examined the morphology of dried droplet patterns with diameters ranging from 1 m to 2 mm using optical microscopy and atomic force microscopy (AFM). The deposit shapes depended sensitively on the characteristic time and length scales of the system. With the addition of NaCl in solution, the patterns showed distinct regions including an amorphous peripheral ring as well as a central region containing crystalline and dendritic structures. These distinctive patterns were highly sensitive to the initial solution composition of lysozyme and NaCl. Therefore, these patterns may act as a “fingerprint” which reflects the solution chemistry of the droplet. In this study, a pattern recognition algorithm is presented to differentiate between deposit patterns from lysozyme solution droplets with varying concentrations of NaCl. The deposit images are represented by a feature vector, determined by applying the 2D Gabor function to the images, similar to the method of feature extraction used in iris recognition [35]. Two of the most popular machine learning algorithms are employed to support the main ideas of this study. An unsupervised learning algorithm (kmeans clustering) is used to demonstrate that incremental changes in solution concentration result in reproducible and statistically interpretable variations in the deposit patterns. Furthermore, a supervised learning algorithm (k-nearest neighbor classification) is used to classify the deposit images by solution concentration based on a set of training images. 2. Experimental methods 2.1. Lysozyme solutions Lysozyme is a globular protein found in high concentration in human mucosal secretions, including tears [28,36,37] and saliva [38]. Lysozyme is roughly ellipsoidal in shape with approximate dimensions of 3.0 nm × 3.0 nm × 4.5 nm, molecular mass of 14 kDa, and carries a net positive charge at physiological pH [39]. This wellstudied protein has been the subject of many protein crystallization [40–43] and self-assembly [44–49] experiments. Concentrated lysozyme and sodium chloride stock solutions were prepared using high purity lysozyme powder (Sigma–Aldrich, L6876) and sodium chloride (NaCl). As-received powders were dissolved in deionized water (Millipore, 18.2 M cm) at 35 ◦ C to concentrations of 2.0 wt% and stored at 2 ◦ C. These stock solutions were diluted with DI water to create the desired concentrations. In this study, the solutions contained 1.0 wt% lysozyme (ϕL ) with NaCl concentrations (ϕN ) of 0.10, 0.50, and 1.0 wt%. Solutions were brought to room temperature prior to deposition and used within one week. 2.2. Substrates and deposition Silicon wafer substrates were cleaned in ultrasonic baths (Fisher Scientific, FS20) of alternating isopropyl alcohol (IPA), acetone, and
171
Fig. 1. Representative deposits from droplets containing 1.00 wt% lysozyme and NaCl in concentrations of ϕN = 0, 0.10, 0.50, and 1.00 wt% (D ∼ 100 m).
DI water, and blown dry with compressed air. A drop-on-demand printer (MicroFab Technologies, Inc., JetLab4® ) was used to produce micro-arrays of droplets. The arrays consisted of droplets in a 5 × 5 grid with diameters D ∼ 100 m and spacing of ∼100 m. This spacing was chosen to minimize overlap in the evaporation fields of neighboring droplets. The droplets evaporated under ambient conditions in our climate-controlled laboratory with T ∼ 25 ◦ C and relative humidity of ∼30%. 2.3. Image acquisition and preprocessing The deposits were observed after evaporation in ambient conditions using a digital reflection optical microscope (Keyence, VHX-600). The colors were adjusted by setting the white balance and the images were collected in “image enhancement” mode, which automatically adjusts the contrast in the image. The entire array of deposits was captured and the individual deposits were cropped from the array image. These images were cropped to a region slightly larger than the deposit to minimize the background. The deposit images were rescaled to 150 × 150 pixels, which was similar to the dimensions of the cropped deposit images. The cropped and scaled deposit images were then converted to grayscale intensity images. Representative deposit images for the concentrations considered in this study are collected in Fig. 1. It should be noted that deposits with irregular shapes, likely due to local substrate heterogeneities, were not used in this experiment. Software was developed using MATLAB® to process the images as described above, select training and test images at random, perform the image feature extraction, and classify the deposit images (Statistics Toolbox® ) [50]. The algorithm and its implementation are described in the following sections. 3. Algorithm 3.1. Image feature extraction In order to classify the images, specific image information must be obtained and used as a representative feature vector. Wavelet transforms have been used for feature extraction in a number of pattern recognition applications [32,51,52], due to the robust and informative nature of the wavelet. The wavelet represents the spatial variation of a signal and the energy of the signal can be represented by a few expansion coefficients. These coefficients contain information about the signal at various scales and can be used to represent texture when applied to an image.
172
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
y) from Eq. (3), we start by fixing m = 1 and changing the orientation by incrementing n from 1 to 8. Then, the computation is performed for m = 2 and incrementing n from 1 to 8, and so on. The response, v, is a collection of coefficients which is a vector of length (m*n × 1). These are complex values and for computational simplicity, the magnitude of the complex vector, |v|, is taken as the feature vector in this experiment. This vector is used to represent each image for classification. 3.2. Unsupervised learning
Fig. 2. Three dimensional surface plot of the 2D Gabor function.
In this experiment, the Gabor function was used to extract textural information from the deposit images. This method is used in iris recognition [35] and other image processing applications [53,54], as the Gabor function is used to model receptive field profiles of individual neurons in the visual cortex [53,55,56]. Consider the wavelet −2m (x , y ), where x and y contain dilations in the size m (x, y) = 2 of the wavelet by 2m and rotations in angle by : x = 2−m [x sin() − cos()]
(1)
y = 2−m [x sin() + cos()]
The response, v, to an arbitrary image I(x, y) is the product of the image intensity distribution and the receptive field function g(x, y) [55]:
v=
(2)
I(x, y)g(x, y) dx dy ˝
where ˝ is the set of all image points. The receptive field function is modeled with the 2D Gabor function [57]:
g(x, y) = exp
−
x 2 + 2y 2 2 2
2x
exp i
+ϑ
(3)
parameterized as in Refs. [53,56]. The 2D Gabor function in Eq. (3) is a Gaussian kernel modulated by a plane wave [57]. A 3-dimensional surface plot of the Gabor function, as parameterized in this experiment, is given in Fig. 2. The values of the parameters , ˛, , ϑ, m, n used in this experiment are collected in Table 1. The eccentricity of the receptive field ellipse, or the spatial aspect ratio, , is taken as = 0.5. The size of the receptive field is given by ˛ = 0.56 , which is the standard deviation of the Gaussian factor, and the wavelength, = 2, is given in pixels [56]. The phase offset is described by ϑ. When ϑ = 0 and , the function g(x, y) is symmetric and when ϑ = ±/2, the function, g(x, y) is antisymmetric. The variables m and n appear in x and y in Eq. (1). The dilation of the wavelet is described by 2m , where m is the number of “scales.” The angle, , describes the orientation to the normal (x -axis) with = n (/8), where n = 0, 1, 2, 3, . . ., 8 is the number of orientations. In this experiment, n = 8 orientations and the number of scales, m, is varied for comparison. To determine g(x, Table 1 Parameters used in the Gabor function. Parameter
Value
˛ ϑ n m
0.56 0.5 2 8 6, 8, 10, 12
In this experiment, unsupervised learning was used to determine if the deposit patterns were unique enough to be automatically grouped by class, or initial NaCl concentration. In unsupervised learning techniques, the data is not identified by class a priori. The feature vectors are grouped into classes by some measure of similarity (distance, connectivity, etc.). The popular k-means clustering algorithm is applied in this study with k = 4 clusters, representing four solution concentrations. The k-means algorithm generally consists of two steps: assign samples to the nearest cluster mean and calculate the new cluster centroids [58]. This process is repeated until no new cluster assignments are made. In this study, Euclidean distance was used as the distance metric and the four cluster means were initialized to random seeds. The clustering was repeated five times with new initial centroids for each cluster. The accepted clustering had the minimum sum of the point-to-centroid distances [50]. 3.3. Supervised learning In classification, a training set is represented statistically for each class and each new sample is compared to the training data and classified accordingly. In this experiment, the k-nearest neighbor (k-NN) algorithm was used for classification. The robust, non-parametric nature of k-NN makes it a natural fit for computer vision applications and has been used in image pattern recognition applications [32]. In k-NN classification, each vector is classified according to a majority vote of the k closest feature vectors. In the training portion of the k-NN algorithm, a subset of feature vectors and associated class labels are stored as the training set. Unlabeled “test” feature vectors are then compared and assigned according to the class of the nearest neighbors. In this experiment, the decision is based on Euclidean distance and k = 1. In other words, the test image is predicted to belong to the class of the closest training image, determined by minimizing the Euclidean distance between feature vectors. 4. Results 4.1. Pattern formation Arrays of deposits from all four concentrations were examined and captured with a digital optical microscope. Several deposits from each concentration were also captured at higher magnification. Within each array, the deposit patterns had similar characteristics. A small number of deposits had irregular shapes, due to local substrate heterogeneities, and these deposits were excluded from the analysis. There were no additional notable variations in the patterns due to environmental factors. Five deposit images from each of the solution concentrations are shown in Fig. 1. A representative deposit from each concentration is illustrated in more detail in Fig. 3. The morphologies of the deposits observed were consistent with our previous observations [4,33,34]. The deposits consisted of an amorphous, lysozyme-rich, peripheral ring and a central region
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
173
Fig. 4. Line plot of mean coefficient vectors for deposits containing 1.00 wt% lysozyme and the four concentrations of NaCl, indicated in the legend. The x-axis represents the index of the vector and the y-axis represents the values of the coefficients.
Fig. 3. Dried deposits from solutions containing 1.00 wt% lysozyme and NaCl in concentrations ϕN = (a) 0, (b) 0.10, (c) 0.50, and (d) 1.00 wt% (D ∼ 100 m).
containing crystallites in the deposits with NaCl (Fig. 3b–d). The formation of these patterns is described in detail in Refs. [4,33,34] and summarized here. Upon deposition, lysozyme molecules adsorb to the substrate, pinning the droplet. The contact angle of the droplet decreases during evaporation and radial flows are generated which transport unadsorbed lysozyme molecules to the ring. Protein molecules accumulate in the perimeter ring, undergoing a phase transition which proceeds inward toward the center of the drop. The remaining liquid in the central region depins and recedes until evaporation is complete. With the addition of NaCl in solution, the ionic strength of the remaining solution increases, driving formation of hierarchical protein structures. In the final stages of evaporation, the remaining liquid percolates through the solid deposit and is accompanied by extensive cracking in the perimeter ring and the formation of crystallites in the central region of the deposit for solutions containing NaCl. With increasing NaCl in solution, more crystallites are present in the central region of the deposit and the width of the peripheral ring decreases. This is attributed to the NaCl screening the positive repulsive charges and therefore leads to the formation of lysozyme aggregates at an earlier time during evaporation [34].
4.2. Feature extraction The deposit images were cropped from the original array image, rescaled to 150 × 150 pixels, and converted to grayscale. The Gabor function in Eq. (3) was determined with scales of m = 6, 8, 10, and 12 and the parameters listed in Table 1. This was applied to each processed image, as described in Section 3, resulting in individual feature vectors containing the coefficients, |v|. Each vector had a size of (m*n × 1) coefficients. Plots of mean coefficients with m = 10 for each concentration are collected in Fig. 4. Fig. 4 shows line plots of the mean coefficient vectors, estimated from 20 random images from each concentration. In this case, m = 10 scales, resulting in a vector of 80 coefficients. These line plots illustrate that the distributions of coefficients for all classes have similar overall shapes, but the values of the coefficients generally vary between the classes.
To further examine the association between the mean coefficients from the various classes, the sample (linear) correlation was determined from the data shown in Fig. 4. The correlation coefficients between the various classes ranged from 0.612 to 0.929. The largest correlation was between the vectors for deposits with 0 and 0.10 wt% NaCl and the lowest correlation occurred between 0 and 1.00 wt% NaCl. The vector representing the 0.50 wt% class was most strongly correlated with the 0.10 wt% vector (0.911). In comparison, there was less correlation between the vector from the 1.00 wt% deposits and the other classes, with correlation coefficients ranging from 0.612 to 0.693. The 1.00 wt% vector was most strongly correlated with the 0.50 wt% vector and least correlated with the vector representing deposits with no NaCl. 4.3. Unsupervised learning algorithm The k-means clustering algorithm was used, as described in Section 3, to group the deposit images into their respective classes, without a priori information. The initial cluster means for the four classes were initialized to random seeds and 20 deposit images for each class were chosen at random. k-means clustering was performed on the feature vectors for these images. The success of the k-means algorithm was measured by the “purity” of the clustering. To determine the purity, each cluster was labeled as the class which occurred most frequently in the cluster. Then the number of correctly assigned vectors in every cluster was divided by the total number of vectors to find the accuracy of the clustering [59]. The k-means clustering test was repeated 5 times, with a new set of random images and different initial cluster seeds. The mean accuracy results for the k-means clustering algorithm are presented in Fig. 5. The mean accuracy for the k-means clustering algorithm was 64%, 76%, 64.3%, and 76.3% for scales of 6, 8, 10, and 12, respectively. Fig. 5 also illustrates the mean accuracy by class with 58.75%, 87.75%, 57%, and 77% for ϕN = 0, 0.10, 0.50, and 1.00 wt%, respectively. Deposits from the droplets containing no NaCl were often grouped with deposits containing ϕN = 0.10 and vice versa. In addition, deposits with ϕN = 0.50 and 1.00 wt% were frequently grouped together. 4.4. Supervised learning algorithm To test the supervised learning algorithm, ten training images from each class were chosen at random to create the k-NN classification model. Ten more images from each class were chosen at random from the remaining images as test images. The feature vector from each test image was assigned to the class of the nearest
174
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
Fig. 5. Clustering accuracy by class (NaCl content) of feature vectors, determined with 6, 8, 10, and 12 scales.
Fig. 6. Mean accuracy results of the k-nearest neighbor (k-NN) classification by class (NaCl content) for feature vectors determined with 6, 8, 10, and 12 scales.
training image by Euclidean distance. The success of the supervised learning algorithm was measured by determining the number of correct classifications for each class and finding the accuracy. The classification was repeated five times, with different sets of training and test images selected at random. The results of the mean k-NN classification accuracy for each class with m = 6, 8, 10, and 12 are collected in Fig. 6. Overall, the k-NN algorithm produced mean accuracies of 90.5%, 94.5%, 97.5%, and 95% for scales of 6, 8, 10, and 12, respectively. Generally, as the number of scales increased, the accuracy improved in this case. The k-NN algorithm was most successful with 10 and 12 scales. By class, the deposits with no NaCl had the highest accuracy results, with 99.5% of the deposits classified correctly. Of the deposits that were misclassified, a majority of deposits with ϕN = 0.10 wt% were misclassified as the deposits with ϕN = 0.00 wt%. A majority of the misclassified deposits containing ϕN = 0.50 and 1.00 wt% were also often mistaken as one another. 5. Discussion Based on the results of this study and our previous observations [4,33,34], the deposit patterns are affected by the initial NaCl solution concentration. The deposits have similar characteristics within the same concentration (Fig. 1) and are clearly influenced by the initial concentration of NaCl (Figs. 1 and 3). More crystallites are present in deposits with higher NaCl concentrations and the width of the peripheral ring decreases as the NaCl content increases. As discussed in our previous findings [31], this can be explained by the formation of aggregates at earlier stages of evaporation. Upon deposition, the positively charged lysozyme molecules are
attracted to the negatively charged SiO2 substrate. The chemistry of the liquid in the drying droplet changes during evaporation as lysozyme molecules leave solution by adsorbing to the substrate or contributing to the gel at the peripheral ring. The ionic strength of the remaining solution increases and the NaCl screens the positive, repulsive charges. This drives liquid–liquid phase separation [51], forming clusters and fractal aggregates of lysozyme [47]. These are formed at an earlier time with increasing NaCl concentration, resulting in more crystallites in deposits with higher initial NaCl content. To further examine these deposit patterns, the images were preprocessed and the Gabor wavelet was applied, resulting in a vector of coefficients, representing each image. The unsupervised learning (k-means clustering) algorithm was implemented to examine whether the feature vectors could be naturally grouped by class (NaCl concentration). The k-means clustering algorithm was applied to the vectors, resulting in clusters with overall accuracies of 64%, 76%, 64.3%, and 76.3% for scales of m = 6, 8, 10, and 12, respectively. This suggests that while there are distinct differences between the patterns, there were also a number of feature vectors that were grouped incorrectly. In the unsupervised learning study, the deposits containing 0 and 0.10 wt% NaCl were frequently grouped together. The same was true between the deposits containing 0.50 and 1.00 wt% NaCl. These incorrect groupings are likely due to similarities between these deposit patterns. For example, the deposits containing 0.50 and 1.00 wt% have crystallites throughout the entire deposit and the deposits with 0 and 0.10 wt% have a central region without crystallites and more features toward the periphery of the deposit. These visual similarities are reflected in the distribution of coefficients in Fig. 4 and will therefore affect the quality of the clustering. In addition, the largest correlation between mean feature vectors occurred for deposits with no NaCl and 0.10 wt% NaCl. This correlation may result in the incorrect clustering of deposits from these two classes. Furthermore, the 1.00 wt% vector was most strongly correlated with the 0.50 wt% vector, likely resulting in the frequent grouping of these two classes. Additional image preprocessing or feature selection may highlight the important features from each concentration and improve the accuracy of the clustering algorithm. The accuracy improved significantly with the supervised learning algorithm, which incorporated training data. Images were chosen at random to train the k-nearest neighbor classifier and random test images were classified accordingly. The mean accuracy for the supervised learning algorithm was 90.5%, 94.5%, 97.5%, and 95% with scales of 6, 8, 10, and 12, respectively. The improvement in accuracy over the unsupervised learning algorithm can likely be attributed to the addition of training data. This notable improvement suggests that while the patterns in this study may be too strongly correlated to cluster, these patterns may be classified based on a training set. The number of scales of the Gabor function was varied in this experiment for comparison. As the number of scales increased, the accuracy of the k-NN algorithm improved in general, although an interpretable relationship was not explored in this study. This general improvement is likely a result of the multi-resolution nature of the wavelet. As the scaling parameter, m, changes, the shape of the wavelet changes resulting in a representation of detail or resolution. There was no obvious relationship in the accuracy of the k-means clustering algorithm and the number of scales. The next steps for improvement of the algorithm would include a more sophisticated optimization of the number of scales. The implementation of this software in MATLAB® is robust, as is computation of the wavelet by nature. Each step of processing images, determining coefficients, developing a training set, and
H.M. Gorr et al. / Colloids and Surfaces B: Biointerfaces 115 (2014) 170–175
classification or clustering took only seconds to process on a home PC. 6. Conclusions In this study, a pattern recognition algorithm is presented in an attempt to classify drop deposit images based on incremental changes in solution concentration. Deposit images were cropped to a region slightly larger than the deposit, rescaled to 150 × 150 pixels, and converted to grayscale. The 2D Gabor function was used to extract feature information from these processed deposit images. The number of scales (resolution) was varied, using scales of 6, 8, 10, and 12. Two different algorithms for classification were tested: unsupervised and supervised learning. The unsupervised learning algorithm was based on the k-means clustering algorithm, grouping the samples into clusters depending on the Euclidean distance to the nearest feature vectors. The unsupervised learning algorithm achieved overall accuracies of 64%, 76%, 64.3%, and 76.3% for scales of 6, 8, 10, and 12, respectively. With the addition of training images in the supervised learning algorithm, the accuracy was greatly improved. The mean accuracy for the k-nearest neighbor algorithm was 90.5%, 94.5%, 97.5%, and 95% for Gabor wavelet scales of 6, 8, 10, and 12, respectively. This study demonstrates that incremental changes in solution concentration result in reproducible and statistically interpretable variations in the self-assembled patterns that develop during sessile droplet evaporation of simplified model biological fluids. Furthermore, the patterns remaining after evaporation of these simplified, model biofluids may act as a “fingerprint” for identifying the nature of the fluid. The overall results of this study are promising and indicate that automated screening and disease detection could be realized with improved data sets and by including a larger library of images. In addition, the robust and scalable nature of this algorithm makes this relevant to a number of problems and can be easily optimized for various applications such as quality control and automatic detection of product contamination, for example. The next steps in development of this software would include a larger database of images and considering a greater number of concentrations. The accuracy of these algorithms may also be improved by additional image preprocessing and using images with higher resolution. In this study, the deposit images were cropped from the array image, however, capturing an individual deposit would provide images with higher resolution. Finally, additional classification algorithms should also be tested as the experiment is scaled up. Acknowledgements The authors acknowledge the facilities, scientific, and technical assistance of the Materials Micro-Characterization Laboratory of the Department of Mechanical Engineering and Materials Science, Swanson School of Engineering, University of Pittsburgh. References [1] R.D. Deegan, O. Bakajin, T.F. Dupont, G. Huber, S.R. Nagel, T.A. Witten, Nature 389 (1997) 827–829. [2] R.D. Deegan, Phys. Rev. E 61 (2000) 475–485. [3] R.D. Deegan, O. Bakajin, T.F. Dupont, G. Huber, S.R. Nagel, T.A. Witten, Phys. Rev. E 62 (2000) 756–762. [4] H.M. Gorr, J.M. Zueger, J.A. Barnard, J. Phys. Chem. B (2012). [5] K.A. Baldwin, M. Granjard, D.I. Willmer, K. Sefiane, D.J. Fairhurst, Soft Matter 7 (2011).
[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]
[25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59]
175
G. Chen, G.J. Mohamed, Eur. Phys. J. E 33 (2010) 19–26. D. Kaya, V.A. Belyi, M. Muthukumar, J. Chem. Phys. 113 (2010). Y.O. Popov, Phys. Rev. E 71 (2005) 36313. H. Hu, R.G. Larson, J. Phys. Chem. B 106 (2002) 1334–1344. R. Bhardwaj, X. Fang, P. Somasundaran, D. Attinger, Langmuir 26 (2010) 7833–7842. H. Hu, R.G. Larson, J. Phys. Chem. B 110 (2006) 7090–7094. J.R. Moffat, K. Sefiane, M.E.R. Shanahan, J. Phys. Chem. B 113 (2009) 8860–8866. A.S. Sangani, C. Lu, K. Su, J.A. Schwarz, Phys. Rev. E 80 (2009). A.G. Marin, H. Gelderblom, D. Lohse, J.H. Snoeijer, Phys. Rev. Lett. 107 (2011). D. Brutin, Colloids Surf. A 429 (2013) 112–120. P.J. Yunker, T. Still, M.A. Lohr, A.G. Yodh, Nature 476 (2011) 308–311. F. Carle, D. Brutin, Langmuir 29 (2013) 9962–9966. L.V. Andreeva, A.V. Koshkin, P.V. Lebedev-Stepanov, A.N. Petrov, M.V. Alfimov, Colloids Surf. A 300 (2007) 300–306. J. Park, J. Moon, Langmuir 22 (2006) 3506–3513. J.X. Zhou, J.Y.H. Fuh, H.T. Loh, Y.S. Wong, Y.S. Ng, J.J. Gray, S.J. Chua, Int. J. Adv. Manuf. Technol. 48 (2010) 243–250. W.R. Small, C.D. Walton, J. Loos, M.I.H. Panhuis, J. Phys. Chem. B 110 (2006) 13029–13036. V. Dugas, J. Broutin, E. Souteyrand, Langmuir 21 (2005) 9130–9136. A.A. Killeen, N. Ossina, R.C. McGlennen, S. Minnerath, J. Borgos, V. Alexandrov, A. Sarvazyan, Mol. Diag. Ther. 10 (2006) 371–380. T.A. Yakhno, V.G. Yakhno, A.G. Sanin, O.A. Sanina, A.S. Pelyushenko, N.A. Egorova, I.G. Terentiev, S.V. Smetanina, O.V. Korochkina, E.V. Yashukova, IEEE Eng. Med. Biol. 24 (2005) 96–104. D. Brutin, B. Sobac, B. Loquet, J. Sampol, J. Fluid Mech. 667 (2011) 85–95. B. Sobac, D. Brutin, Phys. Rev. E 84 (2011) 011603. B.W. Zeid, J. Vicente, D. Brutin, Colloids Surf. A (2013) 139–146. M.T. Kuo, C.C. Lin, H.Y. Liu, H.C. Chang, Invest. Ophthalmol. Vis. Sci. 52 (2011) 4942–4950. L. Ravazzoni, C. Ghini, A. Macri, M. Rolando, Invest. Ophthalmol. Vis. Sci. 38 (1997) s153. E.I. Pearce, A. Tomlinson, Ophthalmic Physiol. Opt. 20 (2000) 306–313. V.N. Shabalin, S.N. Shatokhina, Singapore Med. J. 48 (2007) 440–446. N. Kim, Z. Li, C. Hurth, F. Zenhausern, S.F. Chang, D. Attinger, Anal. Methods 4 (2012) 50–57. H.M. Gorr, J.Z. Zueger, J.A. Barnard, Langmuir (2012). H.M. Gorr, J.M. Zueger, D.R. McAdams, J.A. Barnard, Colloids Surf. B 103 (2013) 59–66. J. Daugman, U.S. Patent No. 5,291,560: Biometric Personal Identification System Based on Iris Analysis, 1994. D. Ronen, E. Eylan, A. Romano, R. Stein, M. Modan, Invest. Ophthalmol. Vis. Sci. 14 (1975) 479–484. R.J. Fullard, C. Snyder, Invest. Ophthalmol. Vis. Sci. 31 (1990) 1119–1126. C.-K. Yeh, M.W.J. Dodds, P. Zuo, D.A. Johnson, Arch. Oral Biol. 42 (1997) 25–31. C.C.F. Blake, D.F. Koenig, G.A. Mair, A.C.T. North, D.C. Phillips, V.R. Sarma, Nature 206 (1965) 757–761. M.L. Pusey, R.S. Snyder, R. Naumann, J. Biol. Chem. 261 (1986) 6524–6529. M. Muschol, F. Rosenberger, J. Chem. Phys. 107 (1997) 1953–1962. Y. Georgalis, P. Umbach, D.M. Soumpasis, W. Saenger, J. Am. Chem. Soc. 120 (1998) 5539–5548. M.C.R. Heijna, M.J. Theelen, W.J.P. van Enckevort, E. Vlieg, J. Phys. Chem. B 111 (2007) 1567–1573. A. Stradner, H. Sedgwick, F. Cardinaux, W.C.K. Poon, S.U. Egelhaaf, P. Schurtenberger, Nature 432 (2004) 492–495. F. Cardinaux, A. Stradner, P. Schurtenberger, F. Sciortino, E. Zaccarelli, Europhys. Lett. 77 (2007). S.P. Rozhkov, A.S. Goryunov, Biophys. Chem. 151 (2010) 22–28. F. Cardinaux, E. Zaccarelli, A. Stradner, S. Bucciarelli, B. Farago, S. Egelhaaf, F. Sciortino, P. Schurtenberger, J. Phys. Chem. B 115 (2011) 7227–7237. E. Blomberg, P.M. Claesson, J.C. Froberg, R.D. Tilton, Langmuir 10 (1994) 2325–2334. S.M. Daly, T.M. Przybycien, R.D. Tilton, Colloids Surf. B 57 (2007) 81–88. MATLAB, Statistics Toolbox, Release 2013a, The MathWorks, Inc., Natick, MA, USA, 2013. S. Fong, J. Biomed. Biotechnol. 2012 (2012) 1–12. R.S.H. Istepanian, A. Sungoor, J. Nebel, Conf. Proc. IEEE Eng. Med. Biol. Soc. (2007) 4629–4632. N. Petkov, Future Generation Comput. Syst. 11 (1995) 451–465. J.A. Montoya Zegarra, N.J. Leite, R. da Silva Torres, J. Comput. Appl. Math. 227 (2009) 294–307. J. Daugman, J. Opt. Soc. Am. A 2 (1985) 1160–1169. N. Petkov, P. Kruizinga, Biol. Cybern. 76 (1997) 83–96. J. Daugman, Int. J. Wavelets Multiresolution Inform. Process. 1 (2003) 1–17. W. Snyder, H. Qi, Machine Vision, Cambridge University Press, New York, 2004. W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Conditional Entropy and Mutual Information in: Numerical Recipes: The Art of Scientific Computing, 3rd ed., Cambridge University Press, New York, 2007 (Section 14.7.3).
