ABSTRACT
Geospatial three-dimensional (3D) raster data have been widely used for simple representations and analysis, such as geological models, spatio-temporal satellite data, hyperspectral images, and climate data. With the increasing requirements of resolution and accuracy, the amount of geospatial 3D raster data has grown exponentially. In recent years, the processing of large raster data using Hadoop has gained popularity. However, data uploaded to Hadoop are randomly distributed onto datanodes without consideration of the spatial characteristics. As a result, the direct processing of geospatial 3D raster data produces a massive network data exchange among the datanodes and degrades the performance of the cluster. To address this problem, we propose an efficient group-based replica placement policy for large-scale geospatial 3D raster data, aiming to optimize the locations of the replicas in the cluster to reduce the network overhead. An overlapped group scheme was designed for three replicas of each file. The data in each group were placed in the same datanode, and different colocation patterns for three replicas were implemented to further reduce the communication between groups. The experimental results show that our approach significantly reduces the network overhead during data acquisition for 3D raster data in the Hadoop cluster, and maintains the Hadoop replica placement requirements.
ABSTRACT
As the largest hydroelectric project worldwide, previous studies indicate that the Three Gorges Dam (TGD) affects the local climate because of the changes of hydrological cycle caused by the impounding and draining of the TGD. However, previous studies do not analyze the long-term precipitation changes before and after the impoundment, and the variation characteristics of local precipitation remain elusive. In this study, we use precipitation anomaly data derived from the CN05.1 precipitation dataset between 1988 and 2017 to trace the changes of precipitation before and after the construction of the TGD (i.e., 1988-2002 and 2003-2017), in the Three Gorges Reservoir Area (TGRA). Results showed that the annual and dry season precipitation anomaly in the TGRA presented an increasing trend, and the precipitation anomaly showed a slight decrease during the flood season. After the impoundment of TGD, the precipitation concentration degree in the TGRA decreased, indicating that the precipitation became increasingly uniform, and the precipitation concentration period insignificantly increased. A resonance phenomenon between the monthly average water level and precipitation anomaly occurred in the TGRA after 2011 and showed a positive correlation. Our findings revealed the change of local precipitation characteristics before and after the impoundment of TGD and showed strong evidence that this change had a close relationship with the water level.
