Search
Filters
total: 22
Search results for: distributed cluster
-
Investigation into MPI All-Reduce Performance in a Distributed Cluster with Consideration of Imbalanced Process Arrival Patterns
PublicationThe paper presents an evaluation of all-reduce collective MPI algorithms for an environment based on a geographically-distributed compute cluster. The testbed was split into two sites: CI TASK in Gdansk University of Technology and ICM in University of Warsaw, located about 300 km from each other, both connected by a fast optical fiber Ethernet-based 100 Gbps network (900 km part of the PIONIER backbone). Each site hosted a set...
-
Agent-Based Non-distributed and Distributed Clustering
Publication -
Real terrain visualisation with a distributed PC-cluster
PublicationJednym z zadań aplikacyjnych ogólnopolskiego projektu CLUSTERIX było opracowanie i wdrożenie metody interaktywnej wizualizacji terenu rzeczywistego z danych przestrzennych systemów GIS. Przedstawione wyniki osiągnięte w tym zadaniu wskazują na duży potencjał wykorzystania opracowanego modelu wizualizacji 3D dla potrzeb systemów zarządzania kryzysowego.
-
Paweł Czarnul dr hab. inż.
PeoplePaweł Czarnul obtained a D.Sc. degree in computer science in 2015, a Ph.D. in computer science granted by a council at the Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology in 2003. His research interests include:parallel and distributed processing including clusters, accelerators, coprocessors; distributed information systems; architectures of distributed systems; programming mobile devices....
-
Distributed NVRAM Cache – Optimization and Evaluation with Power of Adjacency Matrix
PublicationIn this paper we build on our previously proposed MPI I/O NVRAM distributed cache for high performance computing. In each cluster node it incorporates NVRAMs which are used as an intermediate cache layer between an application and a file for fast read/write operations supported through wrappers of MPI I/O functions. In this paper we propose optimizations of the solution including handling of write requests with a synchronous mode,...
-
A Solution to Image Processing with Parallel MPI I/O and Distributed NVRAM Cache
PublicationThe paper presents a new approach to parallel image processing using byte addressable, non-volatile memory (NVRAM). We show that our custom built MPI I/O implementation of selected functions that use a distributed cache that incorporates NVRAMs located in cluster nodes can be used for efficient processing of large images. We demonstrate performance benefits of such a solution compared to a traditional implementation without NVRAM...
-
Long Distance Geographically Distributed InfiniBand Based Computing
PublicationCollaboration between multiple computing centres, referred as federated computing is becom- ing important pillar of High Performance Computing (HPC) and will be one of its key components in the future. To test technical possibilities of future collaboration using 100 Gb optic fiber link (Connection was 900 km in length with 9 ms RTT time) we prepared two scenarios of operation. In the first one, Interdisciplinary Centre for Mathematical...
-
Evaluation of multimedia applications in a cluster oriented environment
PublicationIn the age of Information and Communication Technology (ICT), Web and the Internet have changed significantly the way applications are developed, deployed and used. One of recent trends is modern design of web-applications based on SOA. This process is based on the composition of existing web services into a single scenario from the point of view of a particular user or client. This allows IT companies to shorten product time-to-market....
-
The role of clusters as collective actors in the energy transformation: the case of Mazovia Cluster ICT
PublicationPurpose – In our paper we aim to show the possibilities of implementing the idea of sustainability in the context of energy transformation using the concept of an industrial cluster. The implementation of the idea of sustainability is of particular importance from the perspective of the functioning of the meta-organisations involved in the implementation and promotion of decarbonisation processes. Industrial clusters, as collective...
-
Real-Time connection Between Immerse 3D Vizualization Laboratory and Kaskada Platform
PublicationMultimedia stream processing into two cooperative different systems (cluster platform and virtual lab) is considered. The considered selected information about the systems is presented and the idea of its communication when executing the distributed application is proposed. A general schema of the communication architecture is given. Tests of data transmission quality are considered and their results are presented.
-
Data Partitioning and Task Management in the Clustered Server Layer of the Volunteer-based Computation System
PublicationWhile the typical volunteer-based distributed computing system focus on the computing performance, the Comcute system was designed especially to keep alive in the emergency situations. This means that designers had to take into account not only performance, but the safety of calculations as well. Quadruple-layered architecture was proposed to separate the untrusted components from the core of the system. The main layer (W) consists...
-
Three levels of fail-safe mode in MPI I/O NVRAM distributed cache
PublicationThe paper presents architecture and design of three versions for fail-safe data storage in a distributed cache using NVRAM in cluster nodes. In the first one, cache consistency is assured through additional buffering write requests. The second one is based on additional write log managers running on different nodes. The third one benefits from synchronization with a Parallel File System (PFS) for saving data into a new file which...
-
A Parallel MPI I/O Solution Supported by Byte-addressable Non-volatile RAM Distributed Cache
PublicationWhile many scientific, large-scale applications are data-intensive, fast and efficient I/O operations have become of key importance for HPC environments. We propose an MPI I/O extension based on in-system distributed cache with data located in Non-volatile Random Access Memory (NVRAM) available in each cluster node. The presented architecture makes effective use of NVRAM properties such as persistence and byte-level access behind...
-
Towards Scalable Simulation of Federated Learning
PublicationFederated learning (FL) allows to train models on decentralized data while maintaining data privacy, which unlocks the availability of large and diverse datasets for many practical applications. The ongoing development of aggregation algorithms, distribution architectures and software implementations aims for enabling federated setups employing thousands of distributed devices, selected from millions. Since the availability of...
-
KernelHive: a new workflow-based framework for multilevel high performance computing using clusters and workstations with CPUs and GPUs
PublicationThe paper presents a new open-source framework called KernelHive for multilevel parallelization of computations among various clusters, cluster nodes, and finally, among both CPUs and GPUs for a particular application. An application is modeled as an acyclic directed graph with a possibility to run nodes in parallel and automatic expansion of nodes (called node unrolling) depending on the number of computation units available....
-
IEEE International Conference on Cluster Computing
Conferences -
IEEE International Workshop on Cluster Computing and the Grid
Conferences -
Collision‐Aware Routing Using Multi‐Objective Seagull Optimization Algorithm for WSN‐Based IoT
PublicationIn recent trends, wireless sensor networks (WSNs) have become popular because of their cost, simple structure, reliability, and developments in the communication field. The Internet of Things (IoT) refers to the interconnection of everyday objects and sharing of information through the Internet. Congestion in networks leads to transmission delays and packet loss and causes wastage of time and energy on recovery. The routing protocols...
-
Distributed System For Noise Threat Evaluation Based On Psychoacoustic Measurements
PublicationAn innovative system designed for the continuous monitoring of acoustic climate of urban areas was presentedin the paper. The assessment of environmental threats is performed using online data, acquired through a grid ofengineered monitoring stations collecting comprehensive information about the acoustic climate of urban areas.The grid of proposed devices provides valuable data for the purpose of long and short time acoustic climateanalysis....
-
IEEE International Symposium on Cluster, Cloud and Grid Computing
Conferences -
Architecture and implementation of distributed data storage using Web Services, CORBA i PVM. W: Proceedings. PPAM 2003. Parallel Processing and Applied Mathematics. Fifth International Conference. Częstochowa, 7-10 September 2003. Architektura i implementacja rozproszonego zarządzania danymi używając systemów Web Services, CORBA i PVN.
PublicationProponujemy architekturę i jej implementację PVMWeb Cluster I/O przeznaczoną do rozproszonego zarządzania danymi. Dane zapisywane są w systemie Web Services z geograficznie odległych klientów lub przez wywołania CORBA z wewnątrz danego klastra co oferuje lepsze osiągi.
-
Process arrival pattern aware algorithms for acceleration of scatter and gather operations
PublicationImbalanced process arrival patterns (PAPs) are ubiquitous in many parallel and distributed systems, especially in HPC ones. The collective operations, e.g. in MPI, are designed for equal process arrival times (PATs), and are not optimized for deviations in their appearance. We propose eight new PAP-aware algorithms for the scatter and gather operations. They are binomial or linear tree adaptations introducing additional process...