Search
Search results for: cluster computing
-
Considerations of Computational Efficiency in Volunteer and Cluster Computing
PublicationIn the paper we focus on analysis of performance and power consumption statistics for two modern environments used for computing – volunteer and cluster based systems. The former integrate computational power donated by volunteers from their own locations, often towards social oriented or targeted initiatives, be it of medical, mathematical or space nature. The latter is meant for high performance computing and is typically installed...
-
Performance and Power-Aware Modeling of MPI Applications for Cluster Computing
PublicationThe paper presents modeling of performance and power consumption when running parallel applications on modern cluster-based systems. The model includes basic so-called blocks representing either computations or communication. The latter includes both point-to-point and collective communication. Real measurements were performed using MPI applications and routines run on three different clusters with both Infiniband and Gigabit Ethernet...
-
Cluster Computing-The Journal of Networks Software Tools and Applications
Journals -
IEEE International Conference on Cluster Computing
Conferences -
IEEE International Workshop on Cluster Computing and the Grid
Conferences -
IEEE International Symposium on Cluster, Cloud and Grid Computing
Conferences -
Network-assisted processing of advanced IoT applications: challenges and proof-of-concept application
PublicationRecent advances in the area of the Internet of Things shows that devices are usually resource-constrained. To enable advanced applications on these devices, it is necessary to enhance their performance by leveraging external computing resources available in the network. This work presents a study of computational platforms to increase the performance of these devices based on the Mobile Cloud Computing (MCC) paradigm. The main...
-
Process arrival pattern aware algorithms for acceleration of scatter and gather operations
PublicationImbalanced process arrival patterns (PAPs) are ubiquitous in many parallel and distributed systems, especially in HPC ones. The collective operations, e.g. in MPI, are designed for equal process arrival times (PATs), and are not optimized for deviations in their appearance. We propose eight new PAP-aware algorithms for the scatter and gather operations. They are binomial or linear tree adaptations introducing additional process...
-
Paweł Czarnul dr hab. inż.
PeoplePaweł Czarnul obtained a D.Sc. degree in computer science in 2015, a Ph.D. in computer science granted by a council at the Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology in 2003. His research interests include:parallel and distributed processing including clusters, accelerators, coprocessors; distributed information systems; architectures of distributed systems; programming mobile devices....
-
Long Distance Geographically Distributed InfiniBand Based Computing
PublicationCollaboration between multiple computing centres, referred as federated computing is becom- ing important pillar of High Performance Computing (HPC) and will be one of its key components in the future. To test technical possibilities of future collaboration using 100 Gb optic fiber link (Connection was 900 km in length with 9 ms RTT time) we prepared two scenarios of operation. In the first one, Interdisciplinary Centre for Mathematical...
-
Data Partitioning and Task Management in the Clustered Server Layer of the Volunteer-based Computation System
PublicationWhile the typical volunteer-based distributed computing system focus on the computing performance, the Comcute system was designed especially to keep alive in the emergency situations. This means that designers had to take into account not only performance, but the safety of calculations as well. Quadruple-layered architecture was proposed to separate the untrusted components from the core of the system. The main layer (W) consists...
-
Recognition of hazardous acoustic events employing parallel processing on a supercomputing cluster . Rozpoznawanie niebezpiecznych zdarzeń dźwiękowych z wykorzystaniem równoległego przetwarzania na klastrze superkomputerowym
PublicationA method for automatic recognition of hazardous acoustic events operating on a super computing cluster is introduced. The methods employed for detecting and classifying the acoustic events are outlined. The evaluation of the recognition engine is provided: both on the training set and using real-life signals. The algorithms yield sufficient performance in practical conditions to be employed in security surveillance systems. The...
-
Prediction of Processor Utilization for Real-Time Multimedia Stream Processing Tasks
PublicationUtilization of MPUs in a computing cluster node for multimedia stream processing is considered. Non-linear increase of processor utilization is described and a related class of algorithms for multimedia real-time processing tasks is defined. For such conditions, experiments measuring the processor utilization and output data loss were proposed and their results presented. A new formula for prediction of utilization was proposed...
-
Electronically Excited States in Solution via a Smooth Dielectric Model Combined with Equation-of-Motion Coupled Cluster Theory
PublicationWe present a method for computing excitation energies for molecules in solvent, based on the combination of a minimal parameter implicit solvent model and the equation-of-motion coupled-cluster singles and doubles method (EOM-CCSD). In this method, the solvent medium is represented by a smoothly varying dielectric function, constructed directly from the quantum mechanical electronic density using only two tunable parameters. The...
-
Towards Scalable Simulation of Federated Learning
PublicationFederated learning (FL) allows to train models on decentralized data while maintaining data privacy, which unlocks the availability of large and diverse datasets for many practical applications. The ongoing development of aggregation algorithms, distribution architectures and software implementations aims for enabling federated setups employing thousands of distributed devices, selected from millions. Since the availability of...
-
Distributed NVRAM Cache – Optimization and Evaluation with Power of Adjacency Matrix
PublicationIn this paper we build on our previously proposed MPI I/O NVRAM distributed cache for high performance computing. In each cluster node it incorporates NVRAMs which are used as an intermediate cache layer between an application and a file for fast read/write operations supported through wrappers of MPI I/O functions. In this paper we propose optimizations of the solution including handling of write requests with a synchronous mode,...
-
Use of ICT infrastructure for teaching HPC
PublicationIn this paper we look at modern ICT infrastructure as well as curriculum used for conducting a contemporary course on high performance computing taught over several years at the Faculty of Electronics Telecommunications and Informatics, Gdansk University of Technology, Poland. We describe the infrastructure in the context of teaching parallel programming at the cluster level using MPI, node level using OpenMP and CUDA. We present...
-
Project-Based Collaborative Research and Training Roadmap for Manufacturing Based on Industry 4.0
PublicationThe importance of the economy being up to date with the latest developments, such as Industry 4.0, is more evident than ever before. Successful implementation of Industry 4.0 principles requires close cooperation of industry and state authorities with universities. A paradigm of such cooperation is described in this paper stemming from university partners with partly overlapping and partly complementary areas of expertise in manufacturing....
-
Three levels of fail-safe mode in MPI I/O NVRAM distributed cache
PublicationThe paper presents architecture and design of three versions for fail-safe data storage in a distributed cache using NVRAM in cluster nodes. In the first one, cache consistency is assured through additional buffering write requests. The second one is based on additional write log managers running on different nodes. The third one benefits from synchronization with a Parallel File System (PFS) for saving data into a new file which...
-
A Parallel MPI I/O Solution Supported by Byte-addressable Non-volatile RAM Distributed Cache
PublicationWhile many scientific, large-scale applications are data-intensive, fast and efficient I/O operations have become of key importance for HPC environments. We propose an MPI I/O extension based on in-system distributed cache with data located in Non-volatile Random Access Memory (NVRAM) available in each cluster node. The presented architecture makes effective use of NVRAM properties such as persistence and byte-level access behind...
-
DATABASE AND BIGDATA PROCESSING SYSTEM FOR ANALYSIS OF AIS MESSAGES IN THE NETBALTIC RESEARCH PROJECT
PublicationA specialized database and a software tool for graphical and numerical presentation of maritime measurement results has been designed and implemented as part of the research conducted under the netBaltic project (Internet over the Baltic Sea – the implementation of a multi-system, self-organizing broadband communications network over the sea for enhancing navigation safety through the development of e-navigation services.) The...
-
Processing of Satellite Data in the Cloud
PublicationThe dynamic development of digital technologies, especially those dedicated to devices generating large data streams, such as all kinds of measurement equipment (temperature and humidity sensors, cameras, radio-telescopes and satellites – Internet of Things) enables more in-depth analysis of the surrounding reality, including better understanding of various natural phenomenon, starting from atomic level reactions, through macroscopic...
-
MERPSYS: An environment for simulation of parallel application execution on large scale HPC systems
PublicationIn this paper we present a new environment called MERPSYS that allows simulation of parallel application execution time on cluster-based systems. The environment offers a modeling application using the Java language extended with methods representing message passing type communication routines. It also offers a graphical interface for building a system model that incorporates various hardware components such as CPUs, GPUs, interconnects...
-
KernelHive: a new workflow-based framework for multilevel high performance computing using clusters and workstations with CPUs and GPUs
PublicationThe paper presents a new open-source framework called KernelHive for multilevel parallelization of computations among various clusters, cluster nodes, and finally, among both CPUs and GPUs for a particular application. An application is modeled as an acyclic directed graph with a possibility to run nodes in parallel and automatic expansion of nodes (called node unrolling) depending on the number of computation units available....
-
Parallelization of Selected Algorithms on Multi-core CPUs, a Cluster and in a Hybrid CPU+Xeon Phi Environment
PublicationIn the paper we present parallel implementations as well as execution times and speed-ups of three different algorithms run in various environments such as on a workstation with multi-core CPUs and a cluster. The parallel codes, implementing the master-slave model in C+MPI, differ in computation to communication ratios. The considered problems include: a genetic algorithm with various ratios of master processing time to communication...
-
Investigation into MPI All-Reduce Performance in a Distributed Cluster with Consideration of Imbalanced Process Arrival Patterns
PublicationThe paper presents an evaluation of all-reduce collective MPI algorithms for an environment based on a geographically-distributed compute cluster. The testbed was split into two sites: CI TASK in Gdansk University of Technology and ICM in University of Warsaw, located about 300 km from each other, both connected by a fast optical fiber Ethernet-based 100 Gbps network (900 km part of the PIONIER backbone). Each site hosted a set...
-
A Solution to Image Processing with Parallel MPI I/O and Distributed NVRAM Cache
PublicationThe paper presents a new approach to parallel image processing using byte addressable, non-volatile memory (NVRAM). We show that our custom built MPI I/O implementation of selected functions that use a distributed cache that incorporates NVRAMs located in cluster nodes can be used for efficient processing of large images. We demonstrate performance benefits of such a solution compared to a traditional implementation without NVRAM...
-
Real-Time connection Between Immerse 3D Vizualization Laboratory and Kaskada Platform
PublicationMultimedia stream processing into two cooperative different systems (cluster platform and virtual lab) is considered. The considered selected information about the systems is presented and the idea of its communication when executing the distributed application is proposed. A general schema of the communication architecture is given. Tests of data transmission quality are considered and their results are presented.