Design and implementation of big data ad-hoc queries and statistics subsystem for miliary electronic health records / 军事医学

ABSTRACT

Objective To improve the analysis efficiency and interactive experience of the Military Electronic Health Records System(MEHRS)and to realize quick response of ad-hoc queries and statistics in the MEHRS with big data columnar storage and processing technologies.Methods We carried out requirement analysis and functional design of the ad-hoc queries and statistics subsystem of the MEHRS,proposed a three-tier architecture which included the archive storage layer,statistical pretreatment layer and statistical application layer.After the selection and evaluation of big data processing technologies,CarbonData columnar storage was used to store preprocessed data and executed statistics with Spark SQL on the basis of medical business data modeling and preprocessing.Results Five testing tasks were executed on two million archives in the following two subsystemsone with modeless and non-preprocessed MongoDB storage,the other with modeled and preprocessed CarbonData storage.The latter could finish these tasks within seconds and was dozens of times more efficient than the former statistically.Conclusion This study designs and implements a big data technology proposal that satisfies the quick response of ad-hoc queries and statistics in the MEHRS, providing powerful and flexible technical support for big data statistical analysis.