Toward Improved Hydrologic Prediction with Reduced Uncertainty Using Sequential Multi-Model Combination

The contemporary usage of hydrologic models has been to rely on a single model to perform the simulation and predictions. Despite the tremendous progress, efforts and investment put into developing more hydrologic models, there is no convincing claim that any particular model in existence is superior to other models for various applications and under all circumstances. This results to reducing the size of the plausible model space and often leads to predictions that may well-represent some phenomena or events at the expenses of others. Assessment of predictive uncertainty based on a single model is subject to statistical bias and most likely underestimation of uncertainty. This endorses the implementation of multi-model methods for more accurate estimation of uncertainty in hydrologic prediction. In this study, we present two methods for the combination of multiple model predictors using Bayesian Model Averaging (BMA) and Sequential Bayesian Model Combination (SBMC). Both methods are statistical schemes to infer a combined probabilistic prediction that possess more reliability and skill than the original model members produced by several competing models. This paper discusses the features of both methods and explains how the limitation of BMA can be overcome by SBMC. Three hydrologic models are considered and it is shown that multi-model combination can result in higher prediction accuracy than individual models.


Introduction
Over the past two decades significant effort has gone into development of watershed models by hydrologists.These models have been used to simulate and predict the behavior of the underlying physical processes in the natural system.The reliability on these models to accurately and precisely predict the nonlinear and complex behavior of the hydrologic system is dependent on the perception of the modeler from the governing processes in the system, followed by model 2 conceptualization, mathematical and computer modeling.Hydrologic and water resources engineers often use these models for calculating the watershed runoff, water resources planning and management including reservoir operation, storm water management, water distribution system and groundwater protection.Prediction with these models are often deterministic, relying on the most probable forecast without explicitly accounting for the uncertainty in the incomplete system representation (model structural uncertainty) or the uncertainty in the system initial condition, in the observation of system input and output and also in the parameters that identify the system.It is obvious that quantifying these uncertainties is necessary to evaluate model quality and predictive competence.
Owing to availability of massive data from various sources including ground-based and remotely-sensed observations, hydrologists have developed various types of models including data-driven or black box models, conceptual models and also physically-based models for both lump and distributed representation of a hydrologic system.The general practice by hydrologist and water resources engineers is to use a single model by complete reliance that the model can perform the simulation to their best advantage ignoring the fact that there is no such a perfect model in existence that fully represent the processes in all conditions (Beven 2006;Smith et al., 2004).Most of the predictive uncertainty analysis techniques developed by far are implemented on single models which are believed to result in underestimation of uncertainty and overconfidence in the model predictive capability.Multi-model combination methods have recently been advocated to benefit from the strength of various models in predicting the hydrologic variable of interest (Neuman, 2003;Duan et al., 2007;Ajami et al., 2007;Vrugt and Robinson, 2007).These studies have been motivated by the Bayesian Model Averaging (BMA) development by Raftery et al., (1993Raftery et al., ( , 2003Raftery et al., ( , 2005) ) and Hoeting et al., (1999).BMA prediction is essentially the weighted average of the individual model predictions which has gained popularity in recent years.The BMA aims at providing the unconditional mean and variance of the predictant on the basis of several model forecasts.The main characteristic of the BMA is to rely on a set of time-invariant weighting parameters that are assumed to be normally distributed.
Experience with modeling and simulation have shown that various models may perform differently at different periods (e.g.wet season vs. dry season or dry soil vs. wet soil initial condition) and reliance on just one model for simulating the processes in all conditions would be overconfidence on the model.Assuming the fixed parameter weight, as part of BMA technique, does not provide the flexibility to models participate dynamically according to the conditions.Therefore, in this study we present a sequential Bayesian model combination method to overcome the fixed parameter assumption in the BMA method and the results are interpreted.

Bayesian Model Averaging (BMA)
Bayesian Model Averaging (BMA) is a statistical method which was originally developed to combine inferences and prediction from several statistical models (Leamer 1978;Kass and Raftery 1995).In other word, BMA was designed to postprocess the forecast model ensembles to deduce a predictive probability density function (PDF) of combined prediction that is more skillful and reliable than that of the original model members (model ensemble).Raftery et al. 2005) extended the BMA application to the ensemble of dynamic models (mainly weather forecasting models).The BMA predictive PDF of a quantity of interest is a weighted average of PDFs providing that the individual forecasts are unbiased or bias-corrected. Where is the posterior probability of prediction from model M k given in below: enabling the procedure be applied in real time forecasting.

Sequential Bayesian Model Combination (SBMC)
The posterior distribution presented in eq. ( 2) as the original form of Bayes law is in the batch form where the available historical data is taken for the uncertainty estimation through that conditional probability.However, this form makes no attempt to include information from new observations when becoming available.The flexibility required to use the new information is provided by a sequential Bayesian scheme.Moradkhani et al. (2005a &b) showed that the methods based on sequential Bayesian estimation seem better able to benefit from the temporal organization and structure of information achieving better conformity of the model output with observations.In the multi-model selection process, it is intuitive that if certain models give better predictions than others for a specific portion of the process (time period), those models should be given higher level of participation in predicting the quantity of interest while still not ignoring the level (probability) of success of other models in prediction, i.e., using them may generate a better solution than using a fixed weighting factor as is done in BMA method.
Therefore, implementing the model recursively provides a flexible framework to update models' posterior probability using newly-available observations (Hsu et al., 2008).
Let y t denote the observation of predictive variable at time t and ) | ( be the model prior distribution of the j th model at time t, then the model posterior distribution in sequential form is written as: (3)

Hydr
where a i and b j are parameters, and y(t) and r(t) are the observed streamflow and rainfall sequences, respectively.The time unit t is one day, and ε t+1 is the error of streamflow estimation.
The case study uses three previous time intervals of rainfall and streamflow observations as the inputs to the model (i.e., n 1 = n 2 = 2).
ANN models are black box or data-driven models that have been found suitable in many hydrology studies including streamflow prediction and precipitation estimation (Hsu et al., 1995;Moradkhani et al., 2004;Hong et al., 2005).In this study we used the Self Organizing Radial Basis (SORB) model developed by Moradkhani et al. (2004) (Figure 3).

Study Test Basin
The Leaf River Basin with 1949 km 2 area is located north of Collins, Mississippi and was chosen as the test basin in this study.We used 36 years (1953~1988) of daily rainfall and streamflow data for this basin.For the models to be used in one-time-increment forward forecasting, their parameters must be calibrated from a set of historical data.For the two conceptual models, SAC-SMA and HyMOD, we used 11 years of data just for the calibration period as suggested by Yapo et al. (1996) to obtain the stable and reliable parameters.We used the same number of years for the ARX and ANN models in order for their parameters to be calibrated.

Analysis and Discussion
As mentioned in previous section, all models needed to be calibrated.The calibration of SAC-SMA and HyMOD were done using the Shuffled Complex Evolution-UA (SCE-UA) developed by Duan et al. (1992).The SCE-UA is known as an effective and efficient global optimization technique which has been tested and used in many other disciplines.The calibration of ARX model and SORB-ANN were done by means of simple least square (SLS) method.The common purpose in hydrological prediction is to maximize the predictive accuracy, precision We used these measures as different objective functions to calibrate each model.Therefore, by having four models and four objective functions, we create a sixteen-member ensemble of individual model predictions for the Leaf River basin.As a result, the model combination using BMA yields sixteen weights which are fixed for the whole period of simulation (calibration or evaluation).For the SBMC method, at each time of simulation, there exists sixteen-member ensemble which are changing throughout the simulation as explained in section 2.2.The verification statistics are seen for the individual models and also the model combination schemes in Figure 4. We also extended our verification to probabilistic form.Probabilistic verification methods have been used extensively in the evaluation of meteorological and climate forecasts.One of the measures that is useful in our verification scheme is Ranked Probability Score (RPS) which is essentially the mean-squared error of the probability forecasts averaged over multiple events.In other words, RPS is the mean square error of probabilistic multi-category forecasts where observations are 1 (occurrence) for the observed category and 0 for all other categories.To generate forecast probability from the ensemble of models, the cumulative distribution function of all available historical observations is used to determine threshold streamflow values for nonexceedence probability categories.The pre-specified categories we used in our study are 10%, 35%, 70%, 90% and 100% nonexceedence.For example, the observed value for a given forecast category takes on the value of 1 if the observed flow value is less than the threshold for that category, otherwise the value is 0. The mathematical expression of RPS is given by: ∑ where i t F is the forecast probability and i t O is the observed value at each threshold category, i=1,…, J.The streamflow value for each day is treated as an event.The average RPS for a group of n evaluation period is given by: When RPS is viewed in absolute value, it may not be that meaningful, therefore, we use the Rank Probability Skill Score (RPSS) which is a skill score based on RPS values given as: RPS ref is the reference RPS value which here we computed from the original model ensemble.
For the present study we found the two values of RPSS for each of BMA and SBC methods.we get 43.8% better prediction than that of the original ensemble predictions, and for Sequential Bayesian Combination about 51.2% better than the original ensemble predictions is resulted

Summary and Conclusion
The motivation for this study stems from the fact that there is no such a perfect hydrologic model available to date that performs best in all conditions.Therefore, we conducted an experiment to compare a well-known technique for multi-model combination, Bayesian Model Averaging (BMA), with Sequential Bayesian Model Combination (SBMC) for the one day ahead streamflow forecasting.Four hydrologic models were employed with different natures from times series type to black box and finally to conceptual models to provide deterministic forecast for the Leaf River Basin.Both deterministic and probabilistic verifications were made.The results show that in its current implementation, SBMC achieves a slightly better performance in terms of forecast accuracy, precision and skill score.It appears that sequential estimation in general can take advantage of structural organization of information content in the data to find the flexible weights that can adapt to hydrometeorological condition better than when the whole process is treated in batch and the weighting parameters are estimated once and for all.
In the present study we used four hydrologic models for one basin; however, more models with different development philosophy can be employed where the procedure is applied over more basins with various climate conditions.
If y is the variable of interest to be forecasted (predictant), D = [d 1 , d 2 , …, d n ] is the vector of observations (calibration data) and M = [M 1 ,M 2 , …, M k ] denotes the ensemble of individual model predictions, then the posterior distribution of y is represented as follows.
absence of prior knowledge in selecting the models at the beginning of prediction, all models are treated equally, i.e., World Environmental and Water Resources Congress 2008 Ahupua'a World Environmental and Water Resources Congress 2008: Ahupua'a World Environmental and Water Resources Congress 2008 posterior of model M k can be regarded as likelihood of calibration model M k in predicting y.The likelihood of model M k , in fact, quantifies the probability of success of model M k to closely fit the observation D.One of the challenges in BMA application lies in estimating the parameters, weights w k and variance.One method proposed byRaftery et al. (2005) is to employ the Expectation-Maximization (EM) algorithm as an iterative procedure.BMA is essentially a post-processing of existing retrospective model simulations or hindcastings and the with time meaning that once they are obtained, they remain fixed and used for for rest of the prediction.In the next section, we explain a procedure on sequentially estimating the ) Figure 1

Figure 4 .
Figure 4. Verification statistics for both Calibration and evaluation periods As seen in figure 4, the multi-model combination prediction resulted to better performance measures no matter what objective function used for the deterministic predictions resulted from the individual models.In other words, It is also seen that SBC outperforms BMA which is World Environmental and Water Resources Congress 2008 Ahupua'a World Environmental and Water Resources Congress 2008: Ahupua'a World Environmental and Water Resources Congress 2008 World Environmental and Water Resources Congress 2008 Ahupua'a World Environmental and Water Resources Congress 2008 Ahupua'a World Environmental and Water Resources Congress 2008: Ahupua'a World Environmental and Water Resources Congress 2008 BMA =43.8% and RPSS SBC = 51.2%suggesting that in case of Bayesian Model Averaging, World Environmental and Water Resources Congress 2008: Ahupua'a World Environmental and Water Resources Congress 2008