A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints - Publication - Bridge of Knowledge

Search

A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints

Abstract

The paper proposes a model, design, and implementation of an efficient multithreaded engine for execution of distributed service-based workflows with data streaming defined on a per task basis. The implementation takes into account capacity constraints of the servers on which services are installed and the workflow data footprint if needed. Furthermore, it also considers storage space of the workflow execution engine and its cost. Caching service output data is implemented to speed up the execution of the workflow. Input data is partitioned into data packets, which are passed and processed by services previously selected for workflow tasks so that the aforementioned constraints are met. Performance impact of the proposed mechanisms is investigated for workflow structures common in acyclic directed graph workflow applications. It is shown for a real workflow with distributed processing of digital media content that the initial budget needs to be properly distributed between both the cost of services, but also the cost of intermediate storage to obtain good workflow execution times.

Citations

  • 1 0

    CrossRef

  • 0

    Web of Science

  • 1 2

    Scopus

Cite as

Full text

full text is not available in portal

Keywords

Details

Category:
Articles
Type:
artykuł w czasopiśmie wyróżnionym w JCR
Published in:
JOURNAL OF SUPERCOMPUTING no. 63, edition 3, pages 919 - 945,
ISSN: 0920-8542
Language:
English
Publication year:
2013
Bibliographic description:
Czarnul P.: A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints// JOURNAL OF SUPERCOMPUTING. -Vol. 63, iss. 3 (2013), s.919-945
DOI:
Digital Object Identifier (open in new tab) 10.1007/s11227-012-0837-z
Verified by:
Gdańsk University of Technology

seen 71 times

Recommended for you

Meta Tags