Investigation of severe weather event using big data and deep Learning

Abstract

Weather forecasting is vital in numerous domains including energy, newlineindustry, agriculture, stock market and many others. Weather is considered as newlinea complex phenomenon because of the dynamic interaction of several forces. newlineAccurate weather forecasting is crucial to avoid the loss of human life and newlineproperty destruction. For accurate weather event prediction, it is necessary to newlinehandle the vast amount of text and image datasets. Severe weather event newlineprediction would assist the government to take precautionary measures to newlineavoid such significant losses and to save human lives. The distributed data newlineprocessing frameworks like Hadoop and Spark are used for parallelizing newlinemultiple tasks simultaneously and handling the data effectively. Two newlineparadigms of weather prediction are theory-driven and data-driven newlineapproaches. Numerical Weather Prediction (NWP) models are theory-driven newlinewhich are based on the atmospheric model equations. Some of the NWP newlinebased models are Lagrangian forecast models, Rapid Fresh, Hi-resolution newlineRapid Fresh models and the like Hadoop is only used for batch processing newlinewhereas Spark is utilized for the real-time or near-real-time processing of newlinedata. Spark has an in-memory computing framework that is employed to newlineameliorate the computational speed. During recent years, the generation and newlineconsumption of data take place using big data technology frameworks. newlineMachine learning and deep learning models have applied in all the newlinespecialization currently for achieving accurate prediction. Deep learning newlinemodels provide exceptional accuracy for the large volume of datasets. newlineFurther, the feature space can be increased by using automatic feature newlineextraction and feature selection process in the deep neural networks. newlineAcceptable prediction accuracy can be obtained using deep learning models newlinewhich are capable of handling higher-level feature representation. newline