Abstract
Use of the satellite and reanalysis precipitation products, as supplementary data sources, are steadily rising for hydrometeorological applications, especially in data-sparse areas. However, the accuracy of these data sets is often lacking, especially in Turkey. This study evaluates the accuracy of satellite precipitation product (TRMM 3B42V7) and reanalysis precipitation product (NCEP-CFSR) against rain gauge observations for the 1998–2010 periods. Average annual precipitation for the 25 basins in Turkey was calculated using rain gauge precipitation data from 225 stations. The inverse distance weighting (IDW) method was used to calculate areal precipitation for each basin using GIS. According to the results of statistical analysis, the coefficient of determination for the TRMM product gave satisfactory results (R2 > 0.88). However, R2 for the CFSR data set ranges from 0.35 for the Eastern Black Sea basin to 0.93 for the West Mediterranean basin. RMSE was calculated to be 95.679 mm and 128.097 mm for the TRMM and CFSR data, respectively. The NSE results of TRMM data showed very good performance for 6 basins, while the PBias value showed very good performance for 7 basins. The NSE results of CFSR data showed very good performance for 3 basins, while the PBias value showed very good performance for 6 basins.
1. Introduction
The precipitation, which is one of the most important components of the hydrologic cycle, provides the basic input data for hydrology, climatology, ecology, and agricultural models [1]. The reliability of these models depends on the accuracy and continuity of the precipitation data in spatial and temporal resolutions [2] or even in area with complex orography [3].
The precipitation data were generally obtained from the sparse and discrete observation stations. In most areas, the desired data cannot be easily accessed in regions where there is no or insufficient reliable precipitation data [4, 5]. Also, the precipitation data obtained from ground-based stations contain spatial representation errors [6, 7]. In spite of their importance, insufficient current meteorological stations that are sparsely and unevenly distributed especially over the mountainous region make difficult to obtain accurate precipitation data in Turkey. Besides, most of the stations are located in specific places (i.e., cities and airports); therefore, these stations may not precisely represent the areas situated among them. However, satellite-based precipitation products provide more accurate and continuous observation data [8] to determine the areal precipitation distribution used for the whole world.
Some of the satellite-based precipitation products used today are as follows: tropical rainfall measuring mission (TRMM) [8], stereoscopic imagery based on Meteosat satellites precipitation products (H-SAF of EUMETSAT) [9], Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) (Hsu et al., 1997), Integrated Multi-Satellite Retrievals for Global Precipitation Measurement (GPM-IMERG) [10], Climate Forecast System Reanalysis (CFSR) [11], and Asian Precipitation-Highly Resolved Observational Data Integration Towards Evaluation of Water Resources (APHRODITE) [12].
TRMM, with the cooperation of NASA (National Aeronautics and Space Administration) and JAXA (The Japan Aerospace Exploration Agency), was launched to a height of approximately 350 km in 1997 in order to determine the spatial and temporal distributions of precipitation in tropical regions and to provide a better understanding of climate changes [13].
In the study conducted for the hydrological modelling of the Zambezi Basin within the scope of the African Dam Project, since there are not enough observation stations in the area, precipitation data were analyzed by using satellite-based precipitation products, which are the TRMM 3B42, FEWS, RFE 2.0, NOAA/CPC, and CMORPH, and by comparing their accuracy. As a result, although it was determined that CMORPH predicts rainfall with more than 50% accuracy, TRMM 3B42 product, which is close to it but contains longer data records, was preferred for modelling [14].
The study in China also compared PERSIANN-CDR, TRMM, and CFSR, which are commonly used high-resolution global precipitation products, with observed precipitation. As a result of the analyses carried out, it was determined that the precipitation data obtained from TRMM gave closer results to the observed data than others on a monthly basis. While it was declared that PERSIANN-CDR and TRMM products might be reliable incorrectly predicting the runoff in the basin, CFSR was found to differ from basin to basin [2].
In a study conducted in Ethiopia, researchers tested three satellite-based precipitation products to improve the spatial forecasting of precipitation. The test showed that MPEG, the product of EUMETSAT, called Multisensor Precipitation Estimate, and CFSR provide the most accurate precipitation estimates. The MPEG and CFSR satellites accounted for the explanation of approximately 78–86% of rainfall variation for 38 stations, while the TRMM explained only 17% of the variation [15].
In a study which was carried out in The Mekong River Basin, China, they compared satellite-based data and reanalysis products (TRMM, PERSIANN-CDR, APHRODITE, MERRA2, CFSR, and ERA) against ground-based precipitation data. The APHRODITE was chosen as the reference for the comparison. Generally, TRMM and PERSIANN-CDR satellite data show higher reliability than reanalysis products at both spatial and temporal scales. MERRA2 reanalysis product is more reliable in terms of temporal variability but with some underestimation of precipitation. The other two reanalysis products CFSR and ERA-Interim are relatively unreliable due to large overestimations [16].
The accuracy of the CFSR satellite-based solar radiation data was examined for Hatay Province in Turkey. Statistical results showed that monthly basis data were weakly correlated (R2 = 0.02–0.73). According to these results, it is not advisable to use the CFSR data set in the absence of observed solar radiation data [17].
The reliable use of satellite-based precipitation products, which are considered as precipitation data in many hydrological, climatological, and agricultural models, needs to be verified according to different regions and climatic zones. Therefore, data from precipitation observation stations are used to validate these data and determine their errors [18].
In this study, the accuracy of TRMM satellite-based and CFSR reanalysis precipitation data covering the years 1998–2010 for Turkey’s all basins was evaluated. Annual averaged areal precipitation data sets were compared with observed data using four statistical analyses. This is the first study of comparison of areal precipitation data with TRMM and CFSR product in Turkey.
2. Materials and Methods
2.1. Study Area
Turkey is located between 36° and 42° North latitudes and 26°–45° East longitudes. Its total rainfall area is 779 452 km2. Turkey has 25 large-scale hydrological basins. The average annual total runoff of the basins is 186 billion m3 [19]. Locations of the basins are given in Figure 1.
2.2. Observed Precipitation Data
Observations have been done by the General Directorate of Meteorology (DMI) in Turkey using rain gauges. Rain gauges are bucket type and are of 0.2 mm accuracy. The ground base data from the observation stations belonging to DMI within the basins were obtained from the previous study done by Kayhan and Alan [19]. They calculated annual average precipitation for all basins using the IDW method. These data were used to evaluate the accuracy of satellite-based and reanalysis products by comparing them with ground-based data. The numbers of stations, size of basins, and observed averaged areal precipitation are given in Table 1.
Location and elevation of meteorological stations for the study area are shown in Figure 2.
2.3. TRMM 3B42V7 Precipitation Data
TRMM 3B42V7 monthly precipitation data covering the years 1998–2018 for Turkey have been downloaded from https://mirador.gsfc.nasa.gov/ in NetCDF format. The spatial resolution of the TRMM satellite is 0.25° (approx. 27.8 km). The downloaded data were transferred to ArcGIS software.
2.4. CFSR Precipitation Data
The CFSR (Climate Forecast System Reanalysis), the coupled atmosphere-ocean-land system, is developed at NOAA-NCEP and provides meteorological parameters including precipitation, temperature, wind speed, relative humidity, and radiation, available from 1979 to present [21–23]. The CFSR data set consists of 6 hours of weather forecast data generated by the US National Weather Service. CFSR provides the maximum and minimum temperatures (°C), precipitation (mm), wind velocity (m/s), humidity (%), and solar radiation (MJ/m2) values of any point in the world for the years 1979–2010. The spatial and temporal resolution of the CFSR satellite is 0.35° (nearly 38 km) and 6 hours, respectively. CFSR precipitation data for Turkey (1979–2010) were obtained from https://rda.ucar.edu/ as a file with an extended csv. format.
2.5. Inverse Distance Weighting (IDW)
Inverse distance weighting (IDW) is the most widely used nongeostatistical interpolation method, requiring very few parameters from the operators. It can particularly be used where the data set is lacking and other techniques are affected by errors. The IDW method is a local intermediate value estimation method because it generates and estimates from neighboring points. The weight of a data point is inversely proportional to the square of its distance from the grid cell. The IDW method performs the estimation of unknown points by using the distances of points from each other in the weight calculation. The calculation formula of IDW is given in the following equation [24]:where Zj is the unsampled location value, Zi is the known cell’s value, β is the weight, and δ is the smoothing parameter. The separation distance hijk is measured by a three-dimensional Euclidian distance. hijk is calculated by the following equation [24]:where Δx and Δy are the distances between the unknown and known points according to the reference axes, respectively, and Δz refers to the height as the third point of measure.
2.6. Statistical Methods
The four statistical methods coefficient of determination (R2), root-mean-square error (RMSE), PBias (percent bias), and the Nash–Sutcliffe model efficiency coefficient (NSE) were used for evaluation of precipitation products against the observation station data.
The coefficient of determination of the degree of linear relationship between the data of the satellite-based precipitation and the observation station is calculated by the following equation:
The root-mean-square error (RMSE) represents the mean standard deviation of prediction with respect to the observation [25–28]. The RMSE value between the satellite-based precipitation data and the observation station data is calculated by the following equation:
Percent bias (PBias) measures the average tendency of the simulated data to be larger or smaller than their observed counterparts [29]. The PBias value is calculated using the following equation:
The Nash–Sutcliffe model efficiency coefficient (NSE) is a normalized statistic that determines the relative magnitude of the residual variance compared to the measured data variance [30]. A positive value indicates that the estimate is good, while a negative value indicates that the estimation ability is poor [31]. NSE is computed as shown in the following equation:where Gi is the observation station measurement, is the average of the observation station measurements, Si is the satellite-based estimation, and n is the number of data pairs.
The NSE can take values of between −1 and 1. The performance ratings proposed by Moriasi et al. [31] were applied in this study. The result of a simulation is unsatisfactory if the NSE is lower than 0.50, satisfactory if between 0.50 and 0.65, good if between 0.65 and 0.75, and very good if between 0.75 and 1. NSE and PBias evaluation classes are given in Table 2.
3. Results and Discussion
3.1. Observed Annual Average Precipitation
The ground-based precipitation data from 225 observation stations belong to the General Directorate of Meteorology, Turkey, were used to calculate annual average precipitation for all basins using the IDW method from the previous study done by Kayhan and Alan [19]. Results of the total annual average precipitation are given in Table 3.
3.2. Spatial Distribution of TRMM Satellite-Based Precipitation Data
Data downloaded in NetCDF format have been transferred to ArcGIS software. Areal precipitation maps were created for all basins by using the IDW interpolation method, and the areal distribution of precipitation map for December 2005 is given in Figure 3. Total annual TRMM precipitation obtained from areal distribution maps with helping GIS histogram calculations is given in Table 4.
3.3. Spatial Distribution of CFSR Precipitation Data
The daily CFSR precipitation data obtained from 1161 stations for Turkey from 1979 to 2010 were transferred into MS Excel Program. Averaged monthly and annual precipitations were calculated for the basins. Areal precipitation maps were created for all basins by using the IDW method. Areal distributions of annual precipitation maps for all basins of Turkey were obtained, and one of them for 2005 is given in Figure 4. Average annual CFSR precipitation obtained from areal distribution maps with helping GIS histogram calculations is given in Table 5.
Comparison of observed, CFSR, and TRMM average precipitation data is shown in Figure 5. Average precipitations are higher than observed precipitations for all basins. Also, CFSR data have a higher amount of precipitation than TRMM data except the Western Black Sea basin where is mostly mountainous area.
3.4. Results of R2 and RMSE Analysis
The results of statistical analyses of TRMM precipitation data are shown in Table 6. The coefficient of determination (R2) was observed to be generally above 0.90. This shows that TRMM data are in a high linear relationship with the data measured from the precipitation observation station. The average RMSE value of the basins for the total annual precipitation amount was calculated as 95.679 mm. The highest RMSE value was calculated for the Coruh Basin (215.11 mm), and the lowest RMSE value was determined for the Van Lake Basin (32.85 mm).
The results of statistical analyses of CFSR precipitation data are given in Table 7. The coefficient of determination (R2) was observed to be generally above 0.60. The Eastern Black Sea Basin (R2 = 0.35), the Coruh Basin (R2 = 0.51), and the Aras Basin (R2 = 0.58) remained under this value so that the accuracy of CFSR precipitation data was not acceptable for these basins. It is observed that CFSR data in other basins are in a good linear relationship with the data measured from the precipitation observation station.
The average RMSE value of the basins for the total annual precipitation amount was calculated as 128.10 mm. The highest RMSE value was calculated for the Coruh Basin (328.47 mm), and the lowest RMSE value was determined for the Ceyhan Basin (34.40 mm).
3.5. Results of PBias Evaluation
The optimal value of PBias is 0.0, with low-magnitude values indicating accurate model simulation. Positive values indicate model underestimation bias, and negative values indicate model overestimation bias [29]. In this study, PBias values were calculated between −28.79% and 2.73% in basins as given in Figure 6. PBias values were found positive 4.59% and 2.73% for the Asi and Eastern Black Sea basins, respectively. These positive values are indicating that the areal precipitation for these basins was underestimated by TRMM. Negative PBias values which mean overestimated precipitation were found for the 23 basins.
PBias values were calculated between −43.63% and 0.47% in basins as shown in Figure 7. According to PBias values, estimation abilities were found very good for Susurluk, Antalya, Burdur Lakes, Akarcay Ceyhan, and Lake Van basins. These low-magnitude values are indicating that CFSR data are more accurate for these basins. Negative PBias values which mean overestimated precipitation were found for the 22 basins.
3.6. Results of NSE Evaluation
Results of NSE statistical analysis are given in Figure 8. As shown in Figure 8, a positive value indicates that the estimate is good, while a negative value indicates that the estimation ability is poor. According to results, estimation abilities were found good for the 10 basins. These basins have uniform rain gauge distributions and relatively high precipitation. In contrast, estimation abilities were determined unsatisfactory for 15 basins. These basins have relatively low precipitation and located in mountainous areas.
Results of NSE statistical analysis for CFSR data are shown in Figure 9. For the 9 basins, positives values indicate that the estimate is good, while 17 basins having negative values indicate that the estimation ability is poor. It was seen that the Antalya, Burdur Lakes, and Ceyhan basins showed the best estimation performance.
4. Conclusions
In this study, the accuracy of TRMM satellite-based and CFSR reanalysis precipitation products was evaluated statistically by comparing the observed areal average annual precipitation data for all basins in Turkey.
According to the correlation of determination results, the consistency of TRMM precipitation data was better than the consistency of CFSR precipitation data with the data of observation stations. RMSE values for TRMM data was found lower than CFSR data. The highest RMSE value was calculated for the Coruh Basin (215.115 mm), and the lowest RMSE value was determined for the Van Lake Basin (32.848 mm).
Results of PBias evaluation of TRMM data showed that estimation abilities for at least 23 basins were found satisfactory and only two basins have unsatisfactory estimations. According to PBias evaluation of CFSR data, estimation abilities were found to be very good for Susurluk, Antalya, Burdur Lakes, Akarcay Ceyhan, and Lake Van basins. However, seven basins were found to have unsatisfactory estimations.
Estimation abilities of TRMM data in terms of NSE were found to be good and satisfactory for the 10 basins and poor for 15 basins. It was seen that the Susurluk, North Aegean, Asi, Lake Van, West Mediterranean, and Antalya basins showed the best estimation ability. As a result of NSE statistical analysis for CFSR data, estimation abilities for 8 basins are good and satisfactory, while 17 basins are poor. It was seen that the Antalya, Burdur lakes, and Ceyhan basins showed the best estimation ability.
Barbosa et al. [32] explained that the TRMM precipitation data gave very different results from the precipitation data of ground observation station and did not recommend using it on an annual scale. However, in our study, it has been determined that TRMM data can be used as an areal precipitation input in 10 basins in Turkey. These basins have uniform rain gauge distributions and relatively high precipitation. In contrast, estimation abilities were determined unsatisfactory for 15 basins that have relatively low precipitation and are located in mountainous areas. Both TRMM and CFSR products have satisfactory prediction ability in the four basins. The basins are Antalya, Lake Van, Susurluk, and Gediz.
Wang et al. [33] explained that hydrological modelling of the Mekong River in Southeast Asia was performed by using observation station measurement data and TRMM data as the precipitation input. They concluded that the usage of TRMM precipitation data is more suitable for hydrological modelling especially in basins with low quality and inadequate data. Worqlul et al. [15] compared CFSR and TRMM data and found that CFSR yielded better results. However, in this study, it was observed that TRMM precipitation data make better predictions than CFSR. Similar results were found by Chen et al. [16].
Both NSE and PBias evaluation results indicated that TRMM precipitation data can be used in 10 basins in Turkey as an areal precipitation input in the hydrological, climatological, and agricultural studies created on the basin basis in Turkey. It was suggested that different precipitation prediction products have to be analyzed in terms of accuracy for future studies in Turkey.
Data Availability
Tropical Rainfall Measuring Mission (TRMM) 3B42V7 monthly precipitation data covering the years 1998–2018 for Turkey were taken from https://mirador.gsfc.nasa.gov/ in NetCDF format. Climate Forecast System Reanalysis (CFSR) precipitation data for Turkey (1979–2010) were taken from https://rda.ucar.edu/datasets/ds093.2. in extended csv. format. The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors gratefully acknowledge the NASA Goddard Space Flight Center for the provision of TRMM 3B42V7 data and developers of the NCEP-CFSR precipitation data set for making the climatic data open-access.