Search
A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints
Abstract
The paper proposes a model, design, and implementation of an efficient multithreaded engine for execution of distributed service-based workflows with data streaming defined on a per task basis. The implementation takes into account capacity constraints of the servers on which services are installed and the workflow data footprint if needed. Furthermore, it also considers storage space of the workflow execution engine and its cost. Caching service output data is implemented to speed up the execution of the workflow. Input data is partitioned into data packets, which are passed and processed by services previously selected for workflow tasks so that the aforementioned constraints are met. Performance impact of the proposed mechanisms is investigated for workflow structures common in acyclic directed graph workflow applications. It is shown for a real workflow with distributed processing of digital media content that the initial budget needs to be properly distributed between both the cost of services, but also the cost of intermediate storage to obtain good workflow execution times.
Citations
-
1 0
CrossRef
-
0
Web of Science
-
1 2
Scopus
Cite as
Full text
- Publication version
- Accepted or Published Version
- DOI:
- Digital Object Identifier (open in new tab) 10.1007/s11227-012-0837-z
- License
-
open in new tab
Keywords
Details
- Category:
- Articles
- Type:
- artykuł w czasopiśmie wyróżnionym w JCR
- Published in:
-
JOURNAL OF SUPERCOMPUTING
no. 63,
edition 3,
pages 919 - 945,
ISSN: 0920-8542 - Language:
- English
- Publication year:
- 2013
- Bibliographic description:
- Czarnul P.: A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints// JOURNAL OF SUPERCOMPUTING. -Vol. 63, iss. 3 (2013), s.919-945
- DOI:
- Digital Object Identifier (open in new tab) 10.1007/s11227-012-0837-z
- Verified by:
- Gdańsk University of Technology
seen 115 times