MF-Storm: a maximum flow-based job scheduler for stream processing engines on computational clusters to increase throughput

ABSTRACT:

Background

A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource

SUBMITTER:  

PROVIDER: S-EPMC9575846 | biostudies-literature | 2022 Jan

REPOSITORIES: biostudies-literature