Bioresource Technology xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Bioresource Technology journal homepage: www.elsevier.com/locate/biortech

Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process Anna Witek-Krowiak a, Katarzyna Chojnacka b,⇑, Daria Podstawczyk a, Anna Dawiec a, Karol Pokomeda a a b

Division of Chemical Engineering, Department of Chemistry, Wrocław University of Technology, Norwida 4/6, 50-373 Wrocław, Poland Institute of Inorganic Technology and Mineral Fertilizers, Department of Chemistry, Wrocław University of Technology, Smoluchowskiego 25, 50-372 Wrocław, Poland

h i g h l i g h t s  Theoretical background of optimization methods (RSM and ANN).  Design of experiments as a tool for improving efficiency of the processes.  Optimization of biosorption process using response surface methodology (RSM).  Optimization of biosorption process using artificial neural networks (ANN).

a r t i c l e

i n f o

Article history: Available online xxxx Keywords: Response surface methodology Artificial neural networks Optimization Biosorption Design of experiments

a b s t r a c t A review on the application of response surface methodology (RSM) and artificial neural networks (ANN) in biosorption modelling and optimization is presented. The theoretical background of the discussed methods with the application procedure is explained. The paper describes most frequently used experimental designs, concerning their limitations and typical applications. The paper also presents ways to determine the accuracy and the significance of model fitting for both methodologies described herein. Furthermore, recent references on biosorption modelling and optimization with the use of RSM and the ANN approach are shown. Special attention was paid to the selection of factors and responses, as well as to statistical analysis of the modelling results. Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction Biosorption is a complex process (Witek-Krowiak and Reddy, 2013; Chojnacka, 2010) dependent on many factors whose total effect has a direct impact on the process performance. Therefore it is important to select an appropriate mathematical model for predicting and optimizing the system. The optimization and modelling of biosorption is still in the stage of research. Commonly used models employed to describe equilibrium and kinetic studies, for instance the Langmuir (Chojnacka, 2007), Freundlich (Witek-Krowiak et al., 2011) or pseudo-second-order models (Witek-Krowiak, 2012), may be found insufficient in determining the connection between the factors and evaluating their impact on the biosorption process. Optimization is a method for determining the best solution in terms of certain quality criteria (such as process efficiency) and leads to improving the performance of the designed system or process. The optimization of the biosorption process aims at finding the specific conditions (environmental and/or design parameters) ⇑ Corresponding author. Tel.: +48 71 3202435; fax: +48 71 3203469.

at which the process would give the best possible response (best efficiency or uptake). Usually experiments are carried out in such a way that one factor is being applied and analyzed whereas the others remain unaffected. This procedure is called one variable at time (OVAT). A method is time consuming (the researcher has to screen all variables independently) and requires a large number of experiments, which leads to high costs of study. Additionally, OVAT does not include the interactions between the selected parameters. Multivariate statistics techniques allow a significant reduction in the number of experiments, and the description of the impact of the independent variables (individually or in combination) in the process (Amini et al., 2009). This contributes to the development and optimization of the operating system, significantly decreasing the cost of experiments. Response surface methodology (RSM) and the artificial neural network (ANN) are among the most popularly used methods in research on biosorption literature. RSM and ANN provide an alternative for systems where the mathematical relationship between the parameters and the responses is unknown. Both are powerful data modelling tools, which are able to capture and represent complex nonlinear relationships between independent variables and responses of the system.

E-mail address: [email protected] (K. Chojnacka). 0960-8524/$ - see front matter Ó 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.biortech.2014.01.021

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

2

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

1.1. Response surface methodology (RSM) Response surface methodology is a set of mathematical techniques that describe the relation between several independent variables and one or more responses. This method was developed by Box and Wilson (1951) and since then it has been widely used as a technique for designing experiments. The RSM method is based on the fit of mathematical models (linear, square polynomial functions and others) to the experimental results generated from the designed experiment and the verification of the model obtained by means of statistical techniques. The design of experiment (DOE) is a fundamental tool in the field of engineering. This technique can be used especially for improving efficiency of the processes. The basic idea of DOE is to diversify all significant parameters simultaneously over a set of designed experiments and then to combine the results through a mathematical model. Afterwards, this model can be gradually used for optimization, predictions or interpretation. This leads to improving process performance, reducing the number of variables in the process by taking into account only most significant factors, and also to reducing operation costs and experimental time (Montgomery and Runger, 2003; Ghorbani et al., 2008). The optimization by means of RSM approach could be divided into six stages (Fig. 1): (1) selection of independent variables and possible responses, (2) selection of experimental design strategy, (3) execution of experiments and obtaining results, (4) fitting the model equation to experimental data, (5) obtaining response graphs and verification of the model (ANOVA), (6) determination of optimal conditions. 1.1.1. Selection of independent variables and possible response The first and most important step in the whole procedure is to select the most important variables and their ranges from the collection of possible candidates. The batch biosorption process is

usually influenced by several environmental factors (Table 1), like pH, temperature, solute and sorbent concentration, time of the process, agitation speed. If biosorption is carried out in the column mode, important variables include bed height, liquid velocity and particles diameter. Sometimes researchers have searched for optimal condition for biosorbent preparation, selecting reaction conditions as crucial factors (Ahmad and Alrozi, 2010; Mao et al., 2011). Among a multitude of parameters it is important to select the most important, whose effect on the process is most significant. For this purpose, screening experiments should be applied. As an example, the Plackett–Burman (PB) design, two-level full or fractional factorial designs give an opportunity to identify those variables that have a major effect on the output data. An another type of experimental design used for the selection of the significant factors is the Minimum Run Equireplicated Resolution V Design (Cao et al., 2010). The biggest advantage of this design is that it combines great reduction of required experiments and high resolution. Which means, that it takes into consideration 2 and 3-level interactions. In the next stage the determination of the levels of the parameters should be made. A properly selected range of variables increases the chance of response optimum identification. 1.1.2. Selection of experimental design strategy The next crucial step is the design of an experiment with the selection of the points where the response should be estimated. Several design methods have been applied for biosorption optimization, the most popular being the central composite design (CCD), Box–Behnken design (BB), Doehlert Matrix (D), as well as Plackett– Burman (PB) design, full or fractional factorial designs for optimizations with many variables (Fig. 2). Researchers can easily get access to the software that provides simple and clear use of these methods. The most popular programs for biosorption studies are Design Expert (Stat-Ease, Inc.), Minitab (Minitab Inc.), Statistica (StatSoft), JMP (SAS) and Matlab (MathWorks).

Fig. 1. Design of experiment in RSM methodology.

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

3

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx Table 1 Optimization of biosorption using RSM. Sorbent

Sorbate

Method

Input variables

Response

Software

References

Modified rice husks Immobilized Bacillus brevis

Au(III) Cr(VI), Ni(II), Zn(II)

CCD BB

pH, X, C0 pH, T, C0

q %R

Xu et al. (2013) Kumar et al. (2009)

Cashew nut shale

As(III)

CCD

Vg, Vl, h, d

%R

Buckwheat hulls Aeromonas hydrophila Black cumin Sugarcane bagasse

Au(III) Pb(II) Pb(II) Ni(II)

CCD CCD CCD CCF

pH, X, C0 Q, h pH, mX, T pH, X, x

q %R q %R

Pycnoporus sanguineus

Pb(II)

CCF

pH, X, C0

%R

Oryza sativa L. husk

Pb(II)

CCFD

pH, X, T, C0

q

Trametes versicolor

Cu(II)

PB, CCD

pH, T, C0

q

Carbons Modified with Egg Shell Wastes Modified Helianthus annuus

Zn(II)

Taguchi

q

Cr(VI)

BB

Carbon type acid treatment, X(Ca), T pH, X, C0

Minitab Design Expert Design Expert Minitab Minitab Minitab Design Expert Design Expert Design Expert Design Expert –

Modified orange peel

Cu(II)

CCD

pH, X, C0

%R

Almond shell

Cu(II)

FFD

X, C0, Vl

q, %R

Design Expert Design Expert –

Robinia tree leaves

Pb(II)

pH, X, C0

q, %R

Minitab

Ulmus carpinifolia tree leaves, modified Microwaved olive stone AC

Ti(I)

BB, CCD, D CCD

pH, X, C0

%R

Minitab

CCD

PR, tR, IR

Rice polish

Cu(II) Cd(II) Ni(II) Pb(II) Fe(II) Zn(II) As(III) As(V)

CCD

Design Expert Matlab

Wheat bran

Se(IV) Se(VI)

CCD

q

Minitab

Padinasp., algae

UO2 (NO3)2

q

Soybean meal

Cr(III)

BoxWilson BB

Batch mode: pH, C0, T, X column mode: h, C0, Q Batch: pH, X, C0, T column mode: h, Q, C0 pH, t, C0, T

%R, OSAC yield q

Design Expert Matlab

Pseudomonas sp.

Acid Black 172

(PEI)-coated biomass of Corynebacteriumglutamicum Chitosan beads Rice husk

Heavy metal ions

%R

Dora et al. (2013) Yin et al. (2013) Hasan et al. (2010) Bingöl et al. (2012) Garg et al. (2008) Azila et al. (2008) Zulkali et al. (2006) Sahan et al. (2010) Guijarro-Aldaco et al. (2011) Jain et al. (2011) Ghosh et al. (2013) De Hoces et al. (2013) Zolgharnein et al. (2013) Zolgharnein et al. (2010) Alslaibi et al. (2013) Ranjan et al. (2011) Ranjan and Hasan (2010) Khani (2011)

T, C0, X

q

Witek-Krowiak et al. (2013)

PB

pH, T, Fe3+ conc., H2PO4 conc.

%R

Reactive Red 4

CCD

CPEI, CGA

q

Reactive Black 5

PB

pH, t, C0, x

%R

FFD

pH, X, C0

q

Dead yeast cells

Everdirect Orange-3GL Direct Blue-67 Malachite green

PB BB

PB: pH, t, X, C0, x, T BB: pH, X, C0

%R

Chitosan hollow fibers

Reactive Blue 19

CCD

pH, X, t, C0

q

Treated bagasse

Methylene Blue

CCD

h, C0, Vl

Minitab

Walnut shell

Maxillion Red GRL

Taguchi DOE

t, X, C0

%R, COD reduction %R

Mirmohseni et al., 2012 Low et al., 2013

Minitab

Deniz (2012)

Spirulina sp. Oil palm emty fruit bunch AC

Phenol 2,4,6-trichlorophenol

CCD CCD

pH, X TA, tA, IR

%R q AC yield

Coconut husk AC

2,4,6-trichlorophenol

CCD

TA, tA, IR

q AC yield

Statistica Design expert Design expert

Dotto et al. (2013) Hameed et al. (2009) Tan et al. (2008)

Dyes Design Expert Design Expert Design Expert – Design Expert Minitab

Du et al. (2010) Mao et al. (2011) Ong and Seou (2013) Safa and Bhatti (2011) Singh et al., 2012

Organics

Nomenclature: AC-activated carbon; X-sorbent dosage; C0-initial concentration; T-temperature; TA-activation temperature; Q-flow rate; Vl-liquid velocity; Vg-gas velocity; PR-power of radiation; tR- radiation time; IR-impregnation ratio; %R-percentage removal; q-uptake; h-bed high; t-contact time; tA-activation time; d-particle size; x-agitation rate.

1.1.2.1. Full factorial design (FFD). A common experimental design is the full factorial design, where all input parameters are set at two levels. FFD includes all possible combinations of variables with multiple levels. The full factorial design allows to determine the main and low-order interaction effects with great flexibility and efficiency. However (Khatree and Rao, 2003), the application of this

design may pose greater problems with fitting second- or higherorder polynomial models. The second-order model can significantly improve the optimization process, especially in the case of three level factorial designs, by estimate higher-order interactions between factors. For this purpose, Box and Wilson (1951) have developed a central composite design (CCD).

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

4

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

Fig. 2. Basic model designs used in RSM.

1.1.2.2. Central composite design (CCD). The central composite design yields as much information as the 3n full factorial design, however this methodology requires a smaller number of experimental runs than FFD. Additionally CCD provides high quality predictions of linear and quadratic interaction effects of parameters affecting the process. The CCD contains the full factorial or fractional factorial design at two levels (2n), center points (cp), which corresponds to the middle level of the factors, and axial points (2n), which in turn depends on specific properties desired for the design and the number of parameters related (Myers and Montgomery, 2002). Depending upon where the axial points are located, the CCD can be divided into three types: CCC (circumscribed central composite), CCI (inscribed central composite) and CCF (facecentered composite). In the selection of the right type of CCD it is the most important to compare the region of operability with the region of interest. 1.1.2.3. Box–Behnken design (BB). Box and Behnken (1960) developed a 3-level incomplete factorial design as an alternative to the labor extensive full factorial design. To accurately describe linear, quadratic and interaction effects, second order polynomial has to be used in the modelling. Box and Behnken created this design to minimize the number of experiments, specifically in quadratic model fitting. Experiment matrices are built by means of two level factorial designs (+1, 1) with incomplete block designs. The final matrix is completed with several replications of the central point, what improves precision. There are no experimental points in this design, where all factors have extreme values. This feature might be beneficial in experiments where undesired phenomena might occur in extreme conditions. The BB is slightly more labor efficient than the CCD and much more labor efficient than the FFD. The BB has only two significant restrictions: the number of experimental factors has to be equal or higher than three and the BB should not be used for fitting other equations than second order polynomial. 1.1.2.4. Doehlert design (D). The Doehlert Matrix or the Uniform Shell Design is an experimental design method created on the basis of a simplex. In the first step, a k-dimensional regular simplex is created, which has one apex in the central point (Doehlert, 1970). In the next step, the simplex points are subtracted from each other yielding the Doehlert Matrix as a result. The greatest advantage of this type of design is its flexibility. The Doehlert Matrix is fully sequential. Due to the simplex-based architecture, the

k-factor D can be upgraded to (k+1)-factor by adding a few experimental points. Another feature of the Uniform Shell Design is the unequal number of experimental levels. In sequential modelling more levels can be applied to the most significant factor. 1.1.2.5. Plackett–Burman design (PB). The Plackett–Burman design has been developed as a short-cut method for determining main factor effects for multiple factor systems (Plackett and Burman, 1946). This design requires only N = k + 1 experiments. This type of design is called ‘‘saturated design’’ because the number of experiments is equal to the number of parameters in the first order RSM model, and the degree of freedom of such a design is equal to zero. A high degree of the experiment number reduction imposes some modelling constrains. Due to design saturation, it is impossible to use a second order polynomial and this design gives no information on interaction effects. Secondly, due to the specific methodology of the experimental matrix design, the number of experiments must be a multiple of 4. However this restriction can be avoided, when dummy factors are used. Dummy variables can be later used in standard deviation calculation. 1.1.3. Fitting the model equation to experimental data and obtaining response graphs Model fitting in RSM consists of two steps: experimental data coding and regression. RSM models operate on coded input values like +1, 0 and 1 instead of actual factor values (Daneshi et al., 2010). The results of experiments are transformed into coded values using the general equation:

xi ¼

xi  xCo Dxi

ð1Þ

In the next step, coded experimental data are fitted to a selected model using least square procedures. For the most common sets of experimental designs and mathematical models, equations for model coefficients are available in literature. Those functions are also incorporated in most of statistical software. The response can be shown as a graph, either as three-dimensional or as a contour plot. This graphical interpretation allows for the observation of the curvature of the response surface. The graphical representation of modelling results is the fastest way of modelling if the optimal response is within experimental boundaries.

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

1.1.4. Verification of the model (ANOVA) The analysis of variance (ANOVA) is a set of statistical methods and mathematical functions used to identify significant factors in a multi-significant model (Iversen and Norpoth, 1987). The main purpose of ANOVA is the identification of important factors and the determination which is the most significant and also if the experiment results are meaningful. The basic assumption of ANOVA is that every measured value is a function of three components.

Y i ¼ ai þ bi þ ei

ð2Þ

The first component is the overall mean value. The second component represents the effect of measured factors on the system response. The third component is the residual error. This parameter is a function of the measurement inadequacy and the influence of parameters, which were not included in design of the experiment. In a well-designed experiment, residual errors of system responses should be consistent with normal distribution. Transforming this model, the following equation for the residual error can be obtained.

ei ¼ Y i  ai  bi

ð3Þ

Further transformation provides equation for the sum of squares for the residual errors.

RSS ¼

X

ðY i  ai  bi Þ2

ð4Þ

Assuming that the minimal value of the sum of squares for the residual error gives the most accurate model fit, RSM polynomial parameters can be obtained by finding RSS minimum. However, the analysis of the variance provides information not only on accuracy of the fit but also on its significance. In the process of model fitting, the sums of squares are calculated for each factor and for residuals. In the next step, mean squares are calculated by dividing the sum of the square values by the number of degrees of freedom connected, related to the examined parameter. The coefficient F-value is calculated with the quotient of the coefficient mean square and residual mean square. On the basis of the calculated F-value, the coefficient degree of freedom and the residual degree of freedom, p-value and minimal significant F-value can be acquired from statistical tables. Usually, if p-value of the factor is lower than 0.05, the factor is considered as significant. However, if the conducted studies require extreme levels of precision, lower p-values might be set as a p-value significance threshold. 1.1.5. Determination of optimal conditions In the case of the one response model, the optimization can be done using calculus. The first derivative is calculated and all zeros within the experimental range are identified. In the next step, the second derivative value is calculated to determine the possibility of saddle point existence. In case of both, the first and second derivatives equal zero, the existence of the local extreme should be confronted with the model graphical presentation. When multiple responses are treated, the simultaneous optimization of two or more values might be required. The use of desirability function allows determining the most suitable conditions for two or more system responses (Amini and Younesi, 2009). Typically, the desirability function is defined with Eq. (5):

D¼

Ym

d i¼1 i

1=m

ð5Þ

where di stands for desirability of i-th response. The response desirability value ranges from di = 0 for inacceptable value to di = 1 for the single response maximum. This equation can be modified by introducing weights. To optimize the desirability function, numerical methods are usually applied.

5

1.2. Artificial neural network (ANN) It is the linear modelling that over the years has been a commonly used technique to describe a variety of mathematical objects and processes. However, in some cases, where there is no basis for the linear approximation of processes, linear models are not sufficient and their application may lead to formulating unfair opinions about the total lack of mathematical possibilities to describe these systems. In such cases, to solve these difficult issues, the reference to models formed by neural networks may be the fastest and the most convenient solution to the problem. Below, the general procedures (Fig. 3) for the application of ANN in the prediction of the process performance are presented. Once neural networks have been chosen to solve a specific problem, the data necessary for learning in terms of inputs and outputs should be collected. Unfortunately, there is no unique method for determining the size of the data to the appropriate network training. The essence of the problem is lack of knowledge what the set of inputs is, in fact, useful for the effective determination of the required output value. At the first attempt at creating a network to solve a specific issue, the user should take into account all the variables that, in his opinion, may be relevant, because then the collection will be progressively reduced and significant variables will be identified. In the case of biosorption, the responses of the system (output) are usually either the percentage removal of sorbate or the sorption capacity of the sorbent. The most common way of reducing the dimension of the input signals is a Principal Components Analysis (PCA). The aim of the method is to reduce the large number of random variables into a smaller set, which is obtained by assuming that certain groups of the random variables represent the variability of the same factors, which means that the random variables are dependent on each other. The PCA has been successfully used as an effective procedure for the determination of input parameters in biosorption of lead(II) ions by Antep pistachio (Pistacia vera L.) shells (Yetilmezsoy and Demirel, 2008). Another approach used by researchers to solve the signal dimension size problem was Sensitivity Analysis (SA). The purpose of the SA of neural network is to determine the cause-and-effect relationship between the inputs and the effect of modelling (outputs). Regardless of the type, each neural network processes only numeric data belonging to a specific range, which implies the need for pre-processing of the input data followed by an algorithmic transformation and interpretation of the output received from the network (postprocessing). The next step of the ANN design is an appropriate selection of the network architecture. A key issue in choosing an appropriate network structure is the presence of feedback. The simplest networks have a feed forward structure. Such a network is characterized by a greater stability, however a structure with feedback may be used to perform more complicated calculations. Although feed forward ANN are less complicated and seem to be less accurate than recurrent networks, they are suitable for modelling the relationship between the set of input data and one or more response or output variables, especially in the case of biosorption (Yang et al., 2011; Masood et al., 2012). Although there are many types of neural networks that differ in structure and operation, currently the most popular network architecture is associated with the concept of multi-layered perception. To completely define the layered network, it is sufficient to indicate the number of layers and the number of neurons in each layer. Three layer models with one hidden layer are commonly applied in the prediction of the performance of many processes. The next step is to determine how many neurons will be present in each of hidden layers. Gómez et al. (2009) collected four approaches (Weka, Neuralware, Barron’s and Masters’ methods)

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

6

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

Fig. 3. Scheme of ANN procedure.

commonly used in choosing an adequate number of hidden neurons. Too many neurons can extend the time necessary for training the network, while too few may not be sufficient to train it at all. Unfortunately, there is no unique method for determining the optimal number of hidden neurons. The selection of the number of neurons in the hidden layer is often the result of empirical tests coupled with trial and error. Kardam et al. (2012) investigated different artificial neural networks with varying number of neurons in the hidden layers and they selected the best. After the number of layers and the number of neurons in each layer have been specified, the values of weights and thresholds

of all neurons should be selected. These parameters are chosen in a manner that minimizes the error of the network that is performed by the learning algorithm. The best known example of a neural network learning algorithm is the backpropagation (BP) method. In the BP method, the gradient vector of the error surface represents the fastest increasing direction of the error function, while the direction of weight change is exactly opposite to the direction that is determined by the gradient. Except for the BP algorithm, recently more sophisticated techniques such as the Levenberg–Marquardt (LM) and the conjugate gradient method have been used to optimize nonlinear error

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

functions. The Levenberg–Marquardt algorithm, which was independently developed by Levenberg (1944) and Marquardt (1963), is suitable for training small-scaled problems. The conjugate gradient method is useful for optimizing (minimization) nonlinear functions of many variables because it does not require the storage of any matrices. The comparison of both methods showed that the LM is much faster than the conjugate gradient method for performance prediction. The selection of an appropriate learning method is an essential part of modelling by ANN, because a successful training of network is associated with a continuous improvement of the network by minimizing the error function that is performed by training algorithm. The last step of an artificial neural network design is the verification and validation of the prediction model on the basis of error function. The multivariable functional approximation by ANN is evaluated by using a continuous error metric such as the mean absolute error (MAE), the mean squared error (MSE) or the root mean squared error (RMSE). Firstly, all errors are summed, and then normalized error values generated by the ANN model are calculated. Although the evaluation is a critical aspect of any trained neural network, there is no other specific methodology for model validation. Usually, researchers use only one performance measure, and therefore it is difficult to compare their results and techniques. However, in the case of biosorption some researchers measure the prediction performance of ANN models by more than one error function (Aghav et al., 2011), which may be helpful in the evaluation of the predictive performance of these networks and in their comparison to other similar ANN models. The effect of the neural network design, training and validation is a nonlinear predictive model, which may lead to optimization and the prediction of process efficiency and in turn scaling up of the system.

1.3. RSM and ANN limitations Both described methodologies, RSM and ANN, utilize the standardized equations which have no methodological background. The most basic implication of this fact is that the calculated model can only be used within the experimental range and cannot be used for extrapolation. The ANN methodology requires large number of experiments is required for training. Currently, there is no method to determine the minimal number of experiments for the ANN training, therefore this methodology is troublesome while designing the experiments.

2. Optimization of biosorption process using response surface methodology (RSM) Response surface methodology approach has been successfully applied for identification of significant factors, modelling and optimization of various biosorbent-sorbate sets (Table 1). RSM connected with variance analysis provides complete information on model accuracy and significance. This section presents RSM studies related to the optimization of the biosorption process that have been published in the last few years. As it was mentioned above, biosorption is affected by several operating variables, and thus, since it is impossible to run a full one-at-a-time optimization for all of those factors, there emerged a demand for methodology that identifies elements significant for the process, and reduces the number of experiments needed for process modelling and optimization. Response surface methodology along with the design of experiments fulfills all of those objectives, provides complete information on model accuracy and significance.

7

2.1. Heavy metal ions Heavy metal ions are the most popular group of solutes in biosorption literature. The majority of experiments are based on batch studies in a selected range of independent variables. Yin et al. (2013) applied RSM with the central composite design to determine how Au(III) biosorption on organophosphonic acid functionalized spent buckwheat hulls (OPA-BH) is influenced by pH, biosorbent dosage and initial ion concentration. The Au(III) uptake (mg/g) was the response of the second order polynomial equation used for modelling and optimization. The biosorption of Au(III) with organomultiphosphonic acid modified rice husks has also been modelled and optimized with application of RSM by Xu et al. (2013) with the same group of independent variables. This study was supplemented with FT-IR analysis of biosorbent surface and Au(III) adsorption selectivity studies in the presence of Hg(II), Cu(II), Co(II), Zn(II) and Cd(II) ions. The biosorption of Cd(II) on Ulmus tree leaves has been optimized with the use of a full factorial design to screen the main variables while Doehlert design was used to create a model which relates Cd(II) percentage removal with significant factors, and to find the optimal biosorption conditions (Zolgharnein et al., 2010). The modelling using the Doehlert design and the modified third order polynomial gave the coefficient of determination 0.99 and p-value below 0.001. Moreover, the ANOVA results for all model coefficients show p-value below 0.001, thus all coefficients used in the modified polynomial were statistically significant. Zolgharnein et al. (2013) have applied three different experimental designs (BB, CCD, D) for modelling and optimization of Pb(II) biosorption on Robinia tree leaves. In the initial step (with the use of FFD) pH, sorbent dosage and initial ion concentration have been determined as significant factors for the biosorption process. In the main part of the study, the influence of selected parameters on ions removal and the maximum uptake was examined. For the optimization, the function of desirability has been created, which combined the ion removal with the maximum uptake. Out of the examined designs, the CCD showed the highest accuracy, however the authors pointed out that this was case dependent and should not be considered as a rule. Sahan et al. (2010) reported the optimization with the use of the Plackett–Burman design and the central composite design for Cu(II) removal from aqueous solutions by Trametesversicolor. In the first part of their study they determined the most effective medium factors from five natural parameters (pH, temperature, initial Cu(II) concentration, contact time, agitation speed). In the second part of the study, the effect of three selected parameters (pH, temperature, initial copper ion concentration) on response (q) with the use of the central composite design has been investigated. Under the obtained optimal conditions, the maximum quantity of removed of Cu(II) ion was 39.87 mg/g (R2 = 0.87). In the majority of the research works, related to the optimization of heavy metals biosorption process, Design Expert Software was mostly used for graphical and regression analyses and analyses of the variance (ANOVA) of the obtained data. There are only few articles where the authors estimated the optimal conditions for predicted maximum response by MATLAB. Witek-Krowiak et al. (2013) worked on the statistical optimization of Cr(III) biosorption on soybean meal by RSM with the use of MATLAB. The optimum conditions for Cr(III) removal were modelled by the three-variable-three-level Box–Behnken design. The effect of three independent factors, which included the biosorbent dosage, the initial Cr(III) concentration and temperature, was studied for the maximum uptake of Cr(III) by soybean meal, which was 61.07 mg/g. Kumar et al. (2009) also optimized the biosorption process using the Box–Behnken design. They evaluated the effects of three independent variables, pH, temperature and the initial concentration of metal ions on biosorption. The optimal conditions

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

8

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

for the maximum removal of heavy metals have been determined separately for Cr(VI), Ni(II) and Zn(II). Immobilized bacterial strain Bacillus brevis was used as biosorbent. On the basis of the Box–Behnken approach they confirmed that the chosen variables had a significant effect on the biosorption of all of three heavy metals. Azila et al. (2008) employed in their research the faced central composite design (CCF) to establish the optimum condition for Pb(II) removal by immobilized cells of Pycnoporussanguineus. According to the CCF, there were conducted twenty experiments with six star points (a = 1) and six replicates at central points. To determine the optimum conditions, the researchers identified the factors with the greatest influence on the response. Garg et al. (2008) worked on the effect of three independent variables (adsorbent dose, pH, agitation speed) on the removal of Ni(II) by agricultural waste biomass and Sugarcane bagasse, also using the faced central composite design in RSM. However, experiments were conducted according to the central composite design matrix with a range of coded factors: 1.682,1, 0, +1, +1.682. Ghosh et al. (2013) presented the ability to remove Cu(II) ions by chemically modified orange peel. They evaluated the effect of three independent parameters, the initial pH, copper concentration, and contact time on the percentage removal of copper, applying the central composite design. They also used the artificial neural network (ANN) for process modelling in order to compare these two optimization methodologies. In the case of CCD the optimum removal of Cu(II) was 93.42% at pH 4.75, a 55.5 mg/l copper concentration and 33.91 min of contact time. R2 value 0.9992 demonstrated that the deviation between the CCD model and the experimental data was very small. CCD experimental data was used to obtain the optimal architecture of ANN. The correlation coefficient of the trained neutral network (0.967) signified the compatibility of the neural model with the experimental data. The RMSE (root mean squared error) was used to compare the efficiency of the RSM model and the constructed ANN. The RMSE results showed a clear advantage of RSM over ANN for both data fitting and the possibility of estimation capabilities. This may be due to the fact that ANN required more experiments to build an effective model. Many articles focus attention on batch studies but among them the papers related to RSM studies in dynamic systems are also interesting. Hasan et al. (2010) evaluated free and immobilized biomass of Aeromonashydrophila as a biosorbent for the removal of Pb(II) from aqueous solution in the batch and column mode, using the 22 full factorial CCD model. The effect of two independent variables, bed height and flow rate, proved to be highly significant (p = 0.000) on lead ions removal. The main effect plots revealed the positive effect of bed height and the negative effect of flow rate on the removal of Pb(II). The optimum predicted response of 88.23% was found for the lowest flow rate and the highest bed height. The optimization of Se(IV) and Se(VI) biosorption using CCD was developed by Ranjan and Hasan (2010) in the batch and continuous mode. For the batch mode 24 full factorial CCD was applied with four variables such as initial pH and solute concentration, adsorbent dose and temperature. For the column mode 23 full factorial CCD with variables as bed height, flow rate and initial selenium concentration was chosen. In both systems selenium uptake (lg/g) was taken as response. Interesting dynamic studies were conducted by Dora et al. (2013). The researchers proposed a gas–liquid–solid fluidized bed reactor for removal of As(III) from water solution by biosorption onto activated carbon derived from cashew nut shale. They evaluated the effects of four parameters (gas and liquid velocity, initial static bed height, average particle size) on As(III) adsorption efficiency (%) using the 24 full factorial CCD. The predictable responses were compared with the experimental data, R2 was found to be 0.961.

Apart from widely used methods such as BB and CCD, Taguchi approach may also be employed as an effective tool in designating of experiments. The L9 Taguchi orthogonal array was applied for the optimization of commercial carbon modification for heavy metal biosorption (Guijarro-Aldaco et al., 2011). Carbon type (coconut shell, bituminous, lignite), acid treatment (none, HCl, H3PO4), concentration of impregnating calcium solution and temperature of thermal treatment served as modelling and optimization factors, while Zn(II) uptake was used as a response. The conducted experiments indicated that the optimal activated carbon for Zn(II) biosorption is coconut shell carbon treated with H3PO4, impregnated with 100% vol calcium solution and heated at 400 °C. This combination provides a Zn(II) uptake of 10.70 mg/g. Among the many applications of RSM in biosorption, the approach proposed by Alslaibi et al. (2013) seems to be very interesting. They applied the central composite design (CCD) method to optimize the preparation of olive stone activated carbon (OSAC) using a microwave technique followed by biosorption of heavy metals (Cu2+, Cd2+, Ni2+, Pb2+, Fe2+, and Zn2+) from solution. Three parameters such as radiation power (W), radiation time (min) and the chemical impregnation ratio were chosen for optimization, while the responses were removal efficiencies and the OSAC yield. The impregnation ratio and radiation time had a greater significant effect on heavy metal removal than radiation power, however the radiation power had the highest significant effect on OSAC yield. 2.2. Dyes and organics Apart from removal of heavy metals, dyes and organics biosorption may be also optimized by RSM. Mao et al. (2011) and Hameed et al. (2009) for instance optimized the preparation of biosorbents for removal of dye (Reactive Red 4) and 2,4,6-trichlorophenol. The former prepared polyethyleneimine (PEI)-coated bacterial biosorbent using fermentation waste biomass of Corynebacteriumglutamicum. They employed RSM with the 22 full factorial central composite design with a maximum uptake of RR4 as a response. The amounts of polymer (PEI) and cross-linker glutaraldehyde (GA) were chosen as independent factors. The latter used RSM with the CCD to optimize the production of activated carbon from KOH impregnated oil palm entry fruit bunch (EFB). In this case, the researchers investigated the effect of three variables: impregnation rate IR, CO2 activation time, and CO2 activation temperature on activated carbon yield and 2,4,6-trichlorophenol adsorption capacity for manufactured biosorbent. The main aim of Singh’s et al. (2012) investigation was to evaluate the influence of the same as in research mentioned above, operating parameters as the most significant factors on malachite green uptake. In this study the Box–Behnken design with a second order polynominal was used to find the optimal conditions in which the maximum decolorization was equal to 96.25%. In the first part of this study they applied Plackett–Burman design to determine which of the six variables (pH, contact time, sorbent dosage, initial dye concentration, agitation rate and temperature) has a significant effect on the decolorization process. It was found that the contributions of the agitation rate and temperature were below 95%, therefore those factors were excluded from the second phase of studies. Similarly to studies referred above, Plackett–Burman design was successfully applied in proposed by Du et al. (2010) and Ong and Seou (2013), a two-step RSM procedure to identify significant factors for optimization of the biosorption of Acid Black 172 on Pseudomonas sp. DY1 and Reactive Black 5 (RB5) on chitosan beads. Du et al. (2010), using PBD, have selected the most significant factors: pH, concentrations of NaH2PO4 and FeCl3. These three main factors were used for modelling and optimization by the CCD and a second order polynomial. In the latter investigation, contact time,

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

9

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

initial dye concentration, pH and agitation rate were employed as operating factors. The analysis of both models (PB and CCD) showed that contact time and pH were the most significant factors. Additionally, decolorization studies were supplemented by investigation on sorption kinetics and dynamics.

3. Optimization of biosorption process using artificial neural networks (ANN) Artificial neural networks (ANN) allow for predicting the output on the basis of the input data without the need to explicitly define the relationship between them, which is especially important in the case of complex issues such as biosorption. Therefore, this approach has been widely used by many researchers for describing systems containing a lot of different sorbents and sorbates. Table 2 presents a scientific literature review of the application of ANN based techniques for modelling of biosorption. Bingöl et al. (2012) compared RSM to ANN in the modelling of lead biosorption, using black cumin. A feed-forward MLP ANN model was employed as a network to predict lead(II) uptake from aqueous solution. The best-fitting algorithm was reported to be the Marquardt–Levenberg with the minimum mean squared error (MSE). The results of the investigation showed that the ANN approach is more appropriate for the modelling of sorption by black cumin than RSM based on the CCD. Modelling of the removal of Pb(II) was also investigated by Kardam et al. (2012) in biosorption on nanocellulose fibers derived from rice straw. The ANN technique based on the single-layer recurrent backpropagation algorithm (Fig. 3) was successfully applied for predicting the sorption efficiency in this case. The researchers also conducted the sensitivity analysis in order to determine the degree of the effectiveness of variables. They observed that the most valid parameter is pH, while contact time produces low sensitivity and could be removed from the network. Modelling with ANN can be used for predicting biosorption not only of metal ions but also of dyes and organic compounds. The

performances in competitive adsorption of phenol and resorcinol from aqueous solution by activated carbon, wood charcoal and rice husk ash was modelled by Aghav et al. (2011). In their study the percentage removal of the organics was evaluated using the three feed-forward networks with a backpropagation learning algorithm with the Levenberg–Marquardt (LM) training, with the parameters such as contact time, adsorbent dose, pH and initial sorbates concentration being taking into account. It was reported that errors of the investigated ANN model output are around 10%, while the correlation coefficient was in all cases higher than 0.95, which indicates that the ANN approach can be successfully and effectively used for prediction organic compounds uptake in competitive adsorption processes. Apart from batch biosorption processes, the ANN approach was used in modelling of column issues such as fixed-bed and packedbed adsorption systems. Saha et al. (2010) modelled the breakthrough curves in the fixed-bed biosorption of cadmium, lanthanum and acid green-3 (by Ascophyllum nodosum, Pseudomonas aeruginosa, Azolla filiculoides), using the radial basis function network. Feed velocity, bed height and feed concentration were employed as input variables in a network based on the orthogonal least square (OLS) learning algorithm. The values of errors for all biosorption systems are lower than 6%, which indicates that the results of ANN predictions show a low discrepancy between experimental and calculated data. A conclusion similar to the mentioned above was put forward by Oguz and Ersoy (2010). The scientists developed an ANN model for removal of copper(II) ions using shells of Sunflower in a fixed-bed adsorption column. The performance factor, which was a breakthrough characteristic, was predicted with reference to the inlet Cu2+ concentration, feed flow rate, bed height, initial solution pH and particle size. A one-layered back propagation neural network was used for modelling the uptake of Cu2+ ions from aqueous solutions. The value of R2 (0.986) and RMSE (0.018) confirmed that the predicted ANN model can be satisfactorily used in this case. Texier et al. (2002) conducted a fixed-bed study using cells of Pseudomonas aeruginosa immobilized in polyacrylamide gel for

Table 2 Optimization of biosorption using ANN. Sorbent

Sorbate

Algorithm

Input variables

Output variables

Software

References

AC, wood charcoal, rice husk ash

Phenol, resorcinol

C0, t, pH

%R

MATLAB 7.0

Black cumin

Pb(II)

Lenvenberg–Marquardt BP Lenvenberg–Marquardt BP

pH, X, T

q

Shells of Sunflower

Cu(II)

Lenvenberg–Marquardt BP First order gradient descent BP

C0, Q, H, pH, d C0, t, Q

C(t)

Meadn d MLP mass, temperature timetrations of phenol (mg/l) Minitab 16 Statistica

Aghav et al. (2011) Bingöl et al. (2012)

C/C0

Fortran

Lenvenberg–Marquardt BP Three-layered feed forward back propagation Lenvenberg–Marquardt BP Orthogonal least square (OLS) learning algorithm

d, X, pH, C0, t C0, T, pH, t

q

SigmaPlot version 11

qmax

MATLAB

pH, C0, T, X, h, Q Q, h, C0

q

MATLAB 7.0

C/C0

MATLAB

Lenvenberg–Marquardt BP Feed forward backpropagation Levenberg–Marquardt BP The Quasi-Newton BP

X, T, C0, pH pH, t, C0

%S

Neural Network Toolbox Neuro Solution 6.0 MATLAB 7.1

pH, C0, X, t, V pH, X

%S

Bone char

Fluoride

Walnut husk

Lanaset Red G

Penicillium YW 01

Acid Black 172, Congo Red

Rice polish

As(III), As(V)

Ascophyllum nodosum, Pseudomonas aeruginosa, Azolla filiculoides Zea mays Cob powder

Cadmium, lanthanum, acid green-3 (AG3, a dye) As(III), As(V)

Bacillus sp.

Total chromium

Nanocellulose fibers

Pb(II)

Rice husks

Safranin

qe

%R

Ò

Neural Network Toolbox Neuro Solution 6.0 MATLAB

Oguz and Ersoy (2010) TovarGómez et al. (2013) Çelekli et al. (2012) Yang et al. (2011) Ranjan et al. (2011) Saha et al. (2010) Raj et al. (2013) Masood et al. (2012) Kardam et al. (2012) Saha (2013)

Please cite this article in press as: Witek-Krowiak, A., et al. Application of response surface methodology and artificial neural network methods in modelling and optimization of biosorption process. Bioresour. Technol. (2014), http://dx.doi.org/10.1016/j.biortech.2014.01.021

10

A. Witek-Krowiak et al. / Bioresource Technology xxx (2014) xxx–xxx

biosorption of lanthanide (La, Eu, Yb) ions from aqueous solutions. They used a back propagation ANN to predict breakthrough curves by the three neural networks input neurons consisting of initial metal ions concentration, bed depth, Reynolds number and time, maximum adsorption capacity and adsorption rate constant. It was found that the incorporation of the maximum sorption capacity as an input caused a decrease in the neural network prediction capacity, while the adsorption rate constant had a rather insignificant effect on the column kinetic modelling. The results show the root mean square error of the ANN model of 0.1697, which indicates a rather low divergence between predicted and experimental data. The application of an artificial neural network (for prediction of biosorption in fixed-bed column was also developed by Cavas et al. (2011) as well as Tovar-Gómez et al. (2013). Both studies were related to modelling breakthrough curves based on the Thomas equation in dynamic processes. Tovar-Gómez and his coworkers assessed the performance of the hybrid ANNs-Thomas model in biosorption of fluoride on bone char, while the latter compared these two models with each other for the fixed-bed adsorption of methylene blue by beach waste Posidonia oceanic (L.) dead leaves. As for the hybrid feed-forward ANNs-Thomas model with backpropagation algorithm, the input data included the feed fluoride concentration, the operation time of packed bed column and the feed flow obtained from fluoride breakthrough curves arranged in order of higher sensitivity. The relatively high correlation coefficient (>0.9) and low mean square error (