Evaluating several satellite precipitation estimates and global ground-based dataset on Sicily (Italy)

The developing of satellite-based precipitation retrieval systems, presents great potentialities for several ap plications ranging from weather and meteorological applications to hydrological modelling. Evaluating perfor mances for these estimates is essential in order to understand their real capabilities and suitability related to each application. In this study an evaluation analysis of satellite precipitation retrieval systems has been carried out for the area of Sicily (Italy). Sicily is an island in the Mediterranean sea with a particular climatology and morphology, which is considered as an interesting test site for satellite precipitation products on the European mid-latitude area. A high density rain-gauges network has been used to evaluate selected satellite precipitation products. Sicily has an area of 26,000 km2 and the gauge density of the network considered in this study is about 250 km2 /gauge. Four satellite products (CMORPH, PERSIANN, TMPA-RT, PERSIANN-CCS) along with two adjusted products (TMPA and PERSIANN Adjusted) have been selected for the evaluation. Evalua tion and comparisons among selected products is performed with reference to the data provided by the gauge network of Sicily and using statistical and visualization tools. Results show that bias is relevant for all satellite products and climatic considerations are reported to address this issue. Moreover bias errors are observed for the adjusted products even though they are reduced respect to only-satellite products. In order to analyze this result, the ground-based precipitation dataset used by adjusted products (GPCC dataset), has been examined and weaknesses arising from spatial sampling of precipitation process have been identified for the study area. Therefore possible issues deriving from using global ground-based datasets for local scales are pointed out from this application.


INTRODUCTION
Potentialities of remote sensing in hydrology are related to the possibility of retrieving information about several variables related to the hydrological cycle.While with regard to some hydrological variables, such as evapotranspiration, the usage of remote sensing data has been widely recognized, other variables still are not considered as information that can confidently being estimated by means of remote sensing information.This is the case of precipitation, that is still strongly related to ground measurement by means of rain-gauge networks.
Nevertheless, in the last years precipitation estimation by satellite data has experienced a considerable development mainly due the to key role of such estimates for climatological purposes.Indeed, satellite information is the only available source that can be used to achieve estimates on a global and continuous scale.One of the most relevant weakness for using precipitation estimates from satellite for hydrological applications is related to the spatial and temporal resolutions of climatological products that usually is not suitable for such cases.Actually, estimates with more detailed resolutions can be derived at the price of increasing the errors and the uncertainty in the estimates.This kind of estimates are nowadays performed within operative frameworks operated by national agencies and university centers that produce global precipitation maps in quasi-real time with observations from satellites, with typical resolutions of 0.25 o lat-lon and 3 hours.These features make satellite precipitation very attractive for hydrological purposes, but an effort to characterize the reliability and the errors that affect these data has to be performed.It has been widely shown by a large evaluation activity that performances and features of precipitation estimates from satellites show geographic heterogeneity and relationships with morphological and physical features such as elevation, coastlines presence etc.These local characteristics affecting the performances of satellite precipitation estimates usually are not accounted by operative products, that routinely retrieve satellite data and produce precipitation estimates.
The objective of this study is to analyze the capabilities of satellite estimates obtained by operative estimation algorithms, namely characterizing their performances, with reference to a given area that is the Sicily Island (Italy) placed in the Mediterranean Sea.
Since satellite precipitation operative algorithms share a number of features as well as data sources, namely sensors considered to retrieve precipitation information, it does not result being useful to consider a complete list of satellite precipitation products available from literature, and a selection of the more important can be considered representative.Therefore a set of products that represent the most important approaches for getting precipitation estimates have been selected.These products are evaluated and compared against the rain-gauges data from a local network.Particular issues like the homogeneity of gauges in the study area are not specifically addressed because main objectives of this study are related to potentialities and weaknesses of such products in the area, not considering spatial dynamics.However the good network density allows for assuming these issues not particularly affecting the results.
In the first section datasets and procedures are presented, then the results of analyses are reported considering a first analysis of evaluation among selected products against ground network and further analyses to delve deeper into some aspects.Finally discussion of results are illustrated along with considerations about possible future improvements of this study.

Satellite datasets
Derivation of precipitation estimates from satellite is fundamentally based on two kind of data: passive microwave (PMW) data retrieved from Low Earth Orbiting (LEO) satellites, and infrared (IR) data from Geostationary Earth Orbiting (GEO) satellites.These two sources of information are characterized by opposite features as the LEO-PMW data are physically related to precipitation processes and characterized by low spatio-temporal resolutions, whereas the GEO-IR data can be used just to infer precipitation from cloud top temperature and are available with high resolutions.The most consolidated estimate methods are based on blending these two sources following different approaches.Therefore different products, based on such approaches, have been selected for this study.The CPC (NOAA Climate Prediction Center) Morphing method (CMORPH), 1 uses IR information only to infer hydrometeor movements allowing for translating in time PMW precipitation estimates; the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) method 2, 3  uses PMW precipitation estimates to calibrate IR-Rainfall relationships by means of an artificial neural network; the TRMM Multisatellite Precipitation Analysis, Real Time (TMPA-RT) method 4 uses calibrated IR precipitation estimates to infill gaps on precipitation estimates provided by PMW data.Based on the same PERSIANN algorithm structure, the PERSIANN Cloud-patch Classification System (PERSIANN-CCS 5 ) introduces a classification scheme of hydrometeors based on IR images.Since it differs from other products because of a more relevant relationship with IR data, it has been considered in this analysis.Finally other two adjusted products have been considered in this study, the PERSIANN Adjusted and the TMPA Research Version, that make use of rain-gauges information, by means of the Global Precipitation Climatology Project (GPCP) dataset. 6, 7  Some useful technical and operative characteristics are reported on table 1

Reference precipitation dataset
Rain-gauge dataset used in the evaluation analysis is provided by SIAS (Servizio Informativo Agrometeorologico Siciliano) i.e. the agro-meteorological informative system of Sicilia that collects information and makes available a quality-controlled meteoclimatic dataset.It is constituted by 104 tipping bucket rain-gauges and, as shown in figure 1, spatial distribution is rather homogeneous in the territory with an average density equal to about 250 Km 2 /gauge.Data are retrieved with high temporal resolution (10 minutes) allowing time aggregation as necessary.In order to obtain information about general performances of selected satellite products, a long reference period has been chosen than allowing to obtain information about both average features presented by satellite products and those related to seasonality and eventually on particular events.Therefore the 2007-2008 period has been selected.Still such a two years period could not give a full depiction of performances behavior of satellite products, since some trend variability, perhaps related to climate changes, may not being captured, but this is out the scope of this study and the representativeness of the selected study period is assumed.
A gridded reference data from SIAS rain-gauge network, for this selected time period, has been prepared using an interpolation procedure based on the Natural Neighbor 8 interpolation algorithm.Beyond the spatial framework of the analysis, in order to be able to compare estimates from different sources, it is necessary considering the same temporal framework.The reference temporal resolution considered is 3 hours that can be considered the typical resolution of most operational satellite products that are nowadays available.Therefore the SIAS gridded data has been opportunely aggregated.

Evaluation indexes
In order to describe different aspects of satellite precipitation performances related to their analysis respect to the reference rain-gauges network datasets, the following set of indexes has been chosen.They have been classified as continuous and categorical indexes considering respectively those related to precipitation values, and those related to precipitation occurrences.

Continuous evaluation indexes
obs and P (i) est are respectively the precipitation value provided by gauge data and the precipitation estimation provided by a satellite product for a single position/pixel, at the i -th time step.
σ(P est ) • σ(P obs ) where P est and P obs are respectively the gauge and satellite time series data for a single position/pixel.

• Taylor diagram
Taylor diagram 9 is based on the geometrical relationship between correlation coefficient, series standard deviation and centered mean square error.It is useful to summerize error statistical performances and it can be used to illustrate satellite precipitation products relative performances (e.g.see Ref. 10).

Categorical indexes
• Probability of detection obs > t where t is a threshold value and I is an indicator function indicating the number of occurrences respecting the given condition.Threshold value for categorical indexes t is fixed equal to 0.125 mm/3hr according to Ref. 10. POD indicates the rainfall occurrences correctly detected by estimation product.It is given by the ratio between the number of occurrences registered by both reference and test dataset, and occurrence registered only by reference dataset.It is equal to 1 if the analyzed dataset is able to represent every occurrences and 0 if no occurrences is detected.
• False alarm ratio est > t FAR indicates the amount of rainfall occurrences detected by considered satellite product when reference dataset is not indicating rainfall.It is equal to 0 if it does not reproduce any false occurrence and 1 if all its registered occurrences do not correspond to the observed data.

Spatial analysis
The availability of a long observation period, allows to make some general considerations about satellite precipitation products performances.In order to retrieve and display information about evaluation indexes spatial distribution, temporal series analysis has been performed for each grid cell within the study area.
Temporal mean and standard deviation maps, obtained considering temporal series for each grid, are shown in figure 2 and summary mean statistics, corresponding to spatial averaged values, are reported in table 2. These results show great differences between magnitude of precipitation estimated by satellite products and reference data, resulting in a strong underestimation by satellite products.In particular only-satellite PMW mainly based products, (CMORPH, PERSIANN and TMPA-RT) underestimate more than 50% of gauge mean value whereas PERSIANN-CCS does not seem reproducing the same behavior reporting only a 20% underestimation probably because of its more IR-based derivation.The mean gauge precipitation map on figure 2 appears related to the morphology of the area, with higher mean precipitation values in the high elevation areas (where even snow precipitation occurs), as it is observable comparing mean maps with elevation pattern (see figure 1).Underestimation is reduced for adjusted products (PERSIANN Adj. and TMPA) but it remains still relevant in spite of the GPCP correction.To address this latter issue, further analysis are needed particularly on the suitability of GPCP dataset for precipitation depiction at local scale.
Coefficient of variation (CV) values from the temporal mean maps (table 2) give a measure of spatial variability of the average precipitation that is still underestimated by all satellite products, particularly by PERSIANN (with CV=0.167 against 0.288 from gauge) and TMPA (CV=0.135) in spite of its bias adjustment that probably leads to a flattening of spatial distribution in the study area.
Temporal variability, observed by means of standard deviation (STD) maps (figure 2) and related mean values (table 2), is even underestimated by CMORPH, PERSIANN and TMPA-RT (underestimation between 51%-28%), while PERSIANN-CCS shows a closer level to gauge reference data, but this result is related to the less biased distribution that is detected on mean values.Coherently with this observation, adjusted products report STD values closer to reference dataset level with TMPA even greater than it.
In order to have a quantitative and comparative estimation of satellite-product performances, indexes spatial distributions have been computed (figure 3).Threshold value adopted for categorical indexes is fixed equal to 0.125 mm/3hr according to Ref. 10. MBE maps confirm that higher bias occur on more elevated areas, where mean rainfall magnitude is greater, and highlight the underestimation reduction by adjusted products.PERSIANN-CCS, even if not adjusted, displays low bias levels probably because of its estimation structure based on a stronger IR relationship.RMSE maps display the elevation pattern already observed in the mean maps and do not show large differences among different satellite-products.The greater values on the east side could be due to both the high-elevation area with related greater precipitation magnitude and based on different mechanism of precipitation (i.e.orographic rather than cyclonic).
Correlation coefficient (CC) maps report a slightly better performance of CMORPH compared to others products.One can observe that these maps indicate the best performing area in the center of the island.This could be due to a problem arising from coastal treatment since PMW retrieval algorithms suffer for some weaknesses due to different radiative properties of hydrometeors respectively over the land and over the ocean. 11  POD and FAR mean levels are related to the time scale considered (3 hours) where it is challenging, for satellite estimates, achieving good results.For this time scale low values are expected, however plots allow to compare products performance levels.In particular PERSIANN-CCS and CMORPH report relatively good results.In the TMPA-RT and TMPA POD maps, central area displays better results than coastal pixels confirming the existence of problems for coastal areas.TMPA-RT and TMPA turn out being the best FAR performing.They show even some issues with pixels on the eastern high-area.From performance maps on figure 3    Taylor diagram realized considering mean correlation coefficient and standard deviation values from map distributions is reported in figure 4.
Taylor diagram summarizes relationship between testing and reference series standard deviations, correlation coefficient and RMSD (root mean square difference) between series centered pattern by means of a trigonometric similitude.Taylor diagram explains how error performance, measured by means of RMSD centered pattern, is given by a combination of correlation coefficient and standard deviations.Since reducing underestimation bias, overall precipitation variance increases along with its magnitude, adjusted products (PERSIANN-Adj.and TMPA) result performing worse, in terms of RMSD centered pattern, than most of not adjusted (CMORPH, PERSIANN and TMPA-RT).This effect affects the mean RMSE values (table 3) as well.
Such a result highlights that for this area, and generally where an underestimation bias is observed, an adjustment procedure that reduces underestimation not producing a significant increase of correlation features, may lead to worse performances in terms of root mean square error.

Large scale considerations
The issue of the relevant bias in all the satellite products, needs to be addressed trying to understand the nature of this inconsistence.As a first step it has been investigated whether it is a problem for the particular study area or it involves a wider area.
In order to address this question the accumulated monthly rainfall global data from GPCP version 2.1 with spatial resolution equal to 2.5 o has been retrieved and compared to similar maps obtained from CMORPH, PERSIANN, TMPA-RT, PERSIANN-Adjusted and TMPA datasets (see figure 5) referring to an extension ranging from northern Africa coast to mid-Europe (30 o -50 o latitude).
From a simple observation of these maps it seems that the passage from north Africa climatic regime to the continental European, characterized by a greater amount of annual rain, is to not well captured by satellite products.
Such a result is congruent with Ref. 13 findings that, in a study on the uncertainties of satellite precipitation, observed that they are more reliable over tropical oceans and flat surfaces while complex terrains, coastlines and water bodies, high latitudes and light precipitation show larger measurement uncertainties.In their analysis Europe and Mediterranean area result being characterized by high uncertainty especially during winter.Issues on Europe area have been recently addressed as well by Ref. 14 that reported the overall underestimation by satellite products and addressed some difficulties arising on mid and high latitude such as those related to low intensities, frozen precipitation occurrences and issues with the surface backgrounds.Therefore, weaknesses on the precipitation retrieval process and related improvements, are to be pursued reviewing retrieval algorithms structure and implementation, that is one of the most addressed open issue about satellite precipitation.As described by the developers of the GPROF algorithm (Goddard Profiling algorithm), 11 retrieval inconsistencies could be due to the PMW algorithm as the meteorological model simulations, currently used in the database feeding the algorithm, are tropical in nature and probably it results in a poor representation of extratropical zones.Ref. 15   of accuracy of the microphysical details provided by the Cloud Resolving Model (CRM) in the a priori database, the completeness of the CRM database and its suitability to represents differences in climate regimes.Even Ref. 17 pointed out how effective upwelling PMW brightness temperatures and associated radiances profile from CRMs may differ because of uncertainty in microphysical parameterizations.Ref. 18 observed some differences among PMW radiances captured by the TRMM Microwave Imager (TMI) and those obtained from GPROF for the characteristics of rain systems over the Korean Peninsula.The authors introduced some customization on the CRM simulations that lead to quality results improvement, therefore demonstrating weaknesses of the general algorithm at the local scale.Another case of considerable satellite products biased estimates is reported by Ref. 12 for the Korean Peninsula.Authors highlight that a general underestimation pattern is described by several products due to shared PMW precipitation algorithms and related weaknesses.Moreover they show that even though the gauge-adjusted TMPA seems to have less bias and shows a similar pattern to climatology, it reports increased RMSE values.Therefore authors suggest that TMPA works best when correlation between preadjusted values and gauge measurements is high because adjustments can be made homogeneously throghout the rainfall range.

GPCC suitability analysis
Another issue highlighted by the evaluation analysis regards the bias reported by adjusted products that are computed incorporating ground-based information by means of GPCP data.Indeed, adjusted products, even though able to reduce the underestimation bias displayed by corresponding only-satellite product, still show a considerable gap referred to the reference rain-gauge data used in the analysis.This discrepancy could be attributed to different performances between SIAS and the GPCC ground data used by GPCP.Since introducing adjustment procedures is considered as the main direction on reaching reliable estimates, understanding this discrepancy is a key activity on characterizing the potentiality on using GPCP data as reference ground data.In particular the illustrated case study, points out potential weaknesses related to local scales of observation.Here a direct comparison between SIAS dataset used in the evaluation, and the GPCC dataset providing the rain-gauge information to GPCP (then in turn to the adjusted satellite precipitation products) is performed.
The GPCC Full Data Reanalysis monthly dataset with spatial resolution equal to 0.5 o has been retrieved from the web-based delivering service made available by the DWD German meteorological service for the period 2003-2009.These data are analyzed in comparison with the SIAS data for the same period, interpolated at the same spatial and time resolution according with the Natural Neighbor method as previously described.Figure 6 shows the monthly spatial averaged precipitation from both datasets.The two series generally show a good agreement.About 80% of occurrences differ less than 20 mm/month and particular strong differences can be observed for specific months.Both underestimation and overestimation by GPCC with respect to SIAS are observed with a prevalence of underestimation occurrences (about 70%).
The MBE, calculated as difference between SIAS and GPCC data, is equal to 6.25 mm/month.Referring to the same time period of the evaluation analysis, that is the 2007-2008 period, the mean bias between GPCC and SIAS is 9.31 mm/month.This value can be compared with the corresponding values reported by PERSIANN Adjusted and TMPA, respectively equal to 21.36 and 12.24 mm/month.This difference is to be attributed to the use of the GPCC monitoring product by GPCP whereas here the Full data reanalysis has been used, and on the adjusting algorithms used by precipitation products.Spatial distribution maps of CC, MBE and RMSE, reported on figures 8, indicate that the high elevation area on the eastern, shows low performances for all the indexes.Even for some pixels on the western side lower values than those on central area are observed.However CC map displays values always greater than 0.6.MBE map, reports values greater than 40 mm/month for a couple of pixel on the eastern area, where evidently particular issues due to poor sampling on high elevation area are observed.Other underestimation occurrences of GPCC respect to SIAS are observed on the eastern side while some overestimation occurrences are observed, up to about 10 mm/month, in the central area.On the RMSE map, particular high values are observed on the same western pixels where high MBE was detected, while a few poor performing values can be identified on the eastern side and the best performing pixels can be localized on the central area.Given these considerations, it can be supposed that an imperfect depiction of precipitation spatial dynamics, may origin from a poor sampling of average precipitation amounts within each of these large pixels.Both SIAS and GPCC gridded estimates used so far, origin from rain-gauges placed on different positions around the study area.Obviously spatial sampling can affect the statistics of precipitation derived from the same events.Then location of stations used in the GPCC analysis have been requested to the DWD office that made these data available.Figure 9   Some GPCC stations are labeled as WMO stations, so it has been supposed that these are characterized by different managing procedures and that the different number of station available for different periods may in part be attributed to the different network sources.One can observe that location of GPCC stations miss a large high precipitation area around Etna volcano and Peloritan mountains range.On figure 8 (a) the correlation coefficient map obtained from the temporal series for each pixel, reveals that the area around Etna volcano is characterized by a very low level of agreement between SIAS and GPCC.In order to investigate the dependence of network sampling on long term statistics, the mean precipitation map from Ref. 19 here considered as the true precipitation distribution on Sicilia, has been sampled using three different network position schemes: the SIAS network, the GPCC stations network and the only WMO GPCC stations.Samples than have been spatially interpolated at the same resolution of GPCC using Natural Neighbor method as described for the previous analysis.It does not correspond to the interpolation method used by GPCC, but here the objective is not to reconstruct the exact GPCC estimate, but to obtain and compare spatial estimates from different sampling schemes.Mean annual precipitation maps, corresponding to each scheme, are reported in figure 10.
Both GPCC and GPCC-WMO schemes report average mean value lower then SIAS that in turn is lower than that provided by the reference map equal to 680 mm/year.Underestimations can be attributed to the sampling gap on the high-precipitation rate area on the Etna volcano and Peloritan mountain range at high elevations.Mean and standard deviation of maps values reported on table 4 show that the missed sampling on areas with high mean precipitation leads to the underestimation of both spatial averaged mean and spatial variability of precipitation in the area.

SUMMARY AND CONCLUSION
The evaluation analysis presented in this study has been carried out in order to derive some insights about performances of satellite precipitation estimates for the area of Sicily (Italy).This area presents several morphological and geographical features that make it an interesting test site for such precipitation products.Indeed, many evaluation studies available in literature, highlight the link between the satellite estimates features and the geographical area.
A large bias level has been observed for all the satellite products, that has been verified being present even on a wider spatial scale.In particular it regards the entire Mediterranean area above the Africa coastlines.
The bias level shown by adjusted products (PERSIANN adjusted and TMPA) has been pointed out examining GPCC ground-based dataset that is introduced by means of GPCP within these products.Datasets resulted being in general good agreement, with GPCC showing mean values generally lower than SIAS, then confirming the bias level observed on the previous evaluation analysis.Elaborations shown that particular lower performances resulted localized on high elevation areas.Therefore it has been formulated the hypothesis of an inadequate spatial sampling of precipitation with missing measurements on such areas and producing the observed bias level.Further analyses confirmed the missed sampling by GPCC dataset around high elevation areas therefore related to high precipitation rates, resulting in the underestimation level observed on previous analyses.
Weaknesses displayed by satellite precipitation products and highlighted by this study, are able to inform about potentialities and suitability of such estimates, and give some useful insights for their application and further developments.

Figure 1 :
Figure 1: Study area location with digital elevation model and rain-gauges distribution

Figure 2 :
Figure 2: Temporal mean and standard deviation (STD) maps of precipitation obtained from rain-gauges data and satellite products (mm/3hr).

6 MBE
one can conclude Proc. of SPIE Vol.8531 853106-procedures, although allow for the bias reduction, do not achieve relavant improvements on other skills represented by RMSE, CC and categorical indexes.

Figure 4 :
Figure 4: Taylor diagram from averaged values.Points are represented by means of polar coordinates with standard deviation as radius and cos −1 (Corr) as angle.Distances from Gauge point give the centered pattern Root Mean Square Error (indicated as RMSD) assumed as performance index.

Figure 7
Figure 7 (a) displays the averaged annual values from spatial averaged time series again.It is confirmed the general underestimation presented by GPCC, with 2006 being the only year showing GPCC values greater than SIAS. Figure 7 (b), with mean monthly values, shows that the GPCC underestimation is distributed along all the year with the exception of July and October.

Figure 7 :
Figure 7: Total annual precipitation (a) and mean monthly contribution (b).
displays positions of stations over Sicilia used in GPCC procedures, SIAS network and a mean annual precipitation map elaborated according to Ref.19.
Other stations used in GPCC

Table 2 :
Mean value and coefficient of variation (CV) from mean maps and mean values from standard deviation (STD) maps.Values in brackets represent the variation, as percentage ratio, of satellite product performances with respect to gauge performance.

Table 3 :
Mean and standard deviation (STD) of the evaluation indexes maps.
and 16 showed that bayesian PMW retrieval algorithm approaches are characterized by errors due to lack Proc. of SPIE Vol.8531 853106-8