Search
KernelHive: a new workflow-based framework for multilevel high performance computing using clusters and workstations with CPUs and GPUs
Abstract
The paper presents a new open-source framework called KernelHive for multilevel parallelization of computations among various clusters, cluster nodes, and finally, among both CPUs and GPUs for a particular application. An application is modeled as an acyclic directed graph with a possibility to run nodes in parallel and automatic expansion of nodes (called node unrolling) depending on the number of computation units available. A methodology is proposed for parallelization and mapping of an application to the environment that includes selection of devices using a chosen optimizer, selection of best grid configurations for compute devices, optimization of data partitioning and the execution. One of possibly many scheduling algorithms can be selected considering execution time, power consumption, and so on. An easy-to-use GUI is provided for modeling and monitoring with a repository of ready-to-use constructs and computational kernels. The methodology, execution times, and scalability have been demonstrated for a distributed and parallel password-breaking example run in a heterogeneous environment with a cluster and servers with different numbers of nodes and both CPUs and GPUs. Additionally, performance of the framework has been compared with an MPI + OpenCL implementation using a parallel geospatial interpolation application employing up to 40 cluster nodes and 320 cores.
Citations
-
1 1
CrossRef
-
0
Web of Science
-
1 4
Scopus
Authors (4)
Cite as
Full text
full text is not available in portal
Keywords
Details
- Category:
- Articles
- Type:
- artykuł w czasopiśmie wyróżnionym w JCR
- Published in:
-
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
no. 28,
edition 9,
pages 2586 - 2607,
ISSN: 1532-0626 - Language:
- English
- Publication year:
- 2016
- Bibliographic description:
- Rościszewski P., Czarnul P., Lewandowski R., Schally - Kacprzak M.: KernelHive: a new workflow-based framework for multilevel high performance computing using clusters and workstations with CPUs and GPUs// CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE. -Vol. 28, iss. 9 (2016), s.2586-2607
- DOI:
- Digital Object Identifier (open in new tab) 10.1002/cpe.3719
- Verified by:
- Gdańsk University of Technology
seen 158 times