Handling multilevel elasticity for distributed stream processing in cloud environment

Abstract

newlineAn increasing number of individuals and organizations are taking advantage of newlineservices available over the Internet due to its ease of access and constant availability. newlineCloud computing is a paradigm for delivering computing resources over the Internet newlinein a highly scalable and on-demand manner. Cloud computing offers multifarious newlineessential services to its users, ranging from infrastructure and system development newlineenvironments to software as a service over the Internet. Various users consuming the newlinecloud services to deploy different applications have their service requirements defined newlinein a Service Level Agreement (SLA). Such applications can be real-time services, i.e., newlinesatellite data processing, banking transactions, healthcare applications, social media, newlineetc. A cloud service provider (CSP) should deliver all its services swiftly to these newlineapplications, which demand fluctuating computational processing, on time. Real-time newlinestream computations are perennial, receiving processing requests unpredictably and newlinerequiring a fair amount of resources for their processing in a constrained timeframe. newlineSuch a dynamic nature of applications leads to resource elasticity at runtime. In newlinea cloud resource hierarchy, multiple resources with different processing capabilities newlineand costs exist. In order to optimally utilize the cloud resources and ensure their newlineuninterrupted availability for real-time processing requirements, it is required to scale newlinethe resources at each processing level efficiently. This work proposes MeghMesa, newlinethe multilevel elasticity framework in a cloud environment for processing real-time newlinestreaming applications and collectively optimizing the elasticity concern of multilevel newlineresources while attaining SLAs and quality of service (QoS) parameters. newlineThe MeghMesa framework consists of a multilevel, multivariable-multistep (MLMVMS) newlineresource forecasting and scaling module as primary functional modules. The newlineML-MVMS model plays a significant role in accurately identifying resources required newlineat multiple processing le