Search
Search results for: nvram distributed cache
-
Three levels of fail-safe mode in MPI I/O NVRAM distributed cache
PublicationThe paper presents architecture and design of three versions for fail-safe data storage in a distributed cache using NVRAM in cluster nodes. In the first one, cache consistency is assured through additional buffering write requests. The second one is based on additional write log managers running on different nodes. The third one benefits from synchronization with a Parallel File System (PFS) for saving data into a new file which...
-
Distributed NVRAM Cache – Optimization and Evaluation with Power of Adjacency Matrix
PublicationIn this paper we build on our previously proposed MPI I/O NVRAM distributed cache for high performance computing. In each cluster node it incorporates NVRAMs which are used as an intermediate cache layer between an application and a file for fast read/write operations supported through wrappers of MPI I/O functions. In this paper we propose optimizations of the solution including handling of write requests with a synchronous mode,...
-
A Solution to Image Processing with Parallel MPI I/O and Distributed NVRAM Cache
PublicationThe paper presents a new approach to parallel image processing using byte addressable, non-volatile memory (NVRAM). We show that our custom built MPI I/O implementation of selected functions that use a distributed cache that incorporates NVRAMs located in cluster nodes can be used for efficient processing of large images. We demonstrate performance benefits of such a solution compared to a traditional implementation without NVRAM...
-
Multi-agent large-scale parallel crowd simulation with NVRAM-based distributed cache
PublicationThis paper presents the architecture, main components and performance results for a parallel and modu-lar agent-based environment aimed at crowd simulation. The environment allows to simulate thousandsor more agents on maps of square kilometers or more, features a modular design and incorporates non-volatile RAM (NVRAM) with a fail-safe mode that can be activated to allow to continue computationsfrom a recently analyzed state in...
-
Cache service for maps presentation in distributed information data exchange system
PublicationThe paper presents the proposition of caches implementation for map presentation in distributed information data exchange system. The concept of cache service is described in the context of distributed information data exchange system elements which control and present on maps positions and other identification data of vessels and other suspicious objects on the territorial sea, sea-coast and the internal sea-waters. The proposed...
-
A Parallel MPI I/O Solution Supported by Byte-addressable Non-volatile RAM Distributed Cache
PublicationWhile many scientific, large-scale applications are data-intensive, fast and efficient I/O operations have become of key importance for HPC environments. We propose an MPI I/O extension based on in-system distributed cache with data located in Non-volatile Random Access Memory (NVRAM) available in each cluster node. The presented architecture makes effective use of NVRAM properties such as persistence and byte-level access behind...
-
A Fail-Safe NVRAM Based Mechanism for Efficient Creation and Recovery of Data Copies in Parallel MPI Applications
PublicationThe paper presents a fail-safe NVRAM based mechanism for creation and recovery of data copies during parallel MPI application runtime. Specifically, we target a cluster environment in which each node has an NVRAM installed in it. Our previously developed extension to the MPI I/O API can take advantage of NVRAM regions in order to provide an NVRAM based cache like mechanism to significantly speed up I/O operations and allow to preload...
-
Multi-agent large-scale parallel crowd simulation
PublicationThis paper presents design, implementation and performance results of a new modular, parallel, agent-based and large scale crowd simulation environment. A parallel application, implemented with C and MPI, was implemented and run in this parallel environment for simulation and visualization of an evacuation scenario at Gdansk University of Technology, Poland and further in the area of districts of Gdansk. The application uses a...
-
Taking advantage of the shared explicit cache system based critical sections in the shared memory parallel architectures
PublicationArtykuł prezentuje nową metodę implementacji sekcji krytycznych w równoległych architekturach z pamięcią współdzieloną, takich jak systemy zintegrowane wielowątkowe wieloprocesorowe. Metoda stanowi modyfikację i rozbudowanie metody zwanej Folding, dostępnej w procesorach sieciowych oraz jest w założeniach podobna do techniki zwanej cache-based locking. W porównaniu do dostępnych metod, nowa metoda usuwa problemy skalowalności i...
-
Using Redis supported by NVRAM in HPC applications
PublicationNowadays, the efficiency of storage systems is a bottleneck in many modern HPC clusters. High performance in the traditional approach – processing using files – is often difficult to obtain because of a model’s complexity and its read/write patterns. An alternative approach is to apply a key-value database, which usually has low latency and scales well. On the other hand, many key-value stores suffer from a limitation of memory...
-
Distributed graph searching with a sense of direction
PublicationIn this work we consider the edge searching problem for vertex-weighted graphs with arbitrarily fast and invisible fugitive. The weight function w provides for each vertex v the minimum number of searchers required to guard v, i.e., the fugitive may not pass through v without being detected only if at least w(v) searchers are present at v. This problem is a generalization of the classical edge searching problem, in which one has...
-
Distributed Databases
e-Learning Courses -
NVRAM as Main Storage of Parallel File System
PublicationModern cluster environments' main trouble used to be lack of computational power provided by CPUs and GPUs, but recently they suffer more and more from insufficient performance of input and output operations. Apart from better network infrastructure and more sophisticated processing algorithms, a lot of solutions base on emerging memory technologies. This paper presents evaluation of using non-volatile random-access memory as a...
-
DISTRIBUTED COMPUTING
Journals -
Distributed VoIP telecommunication system
PublicationIn the paper a distributed VoIP telecommunication system architecture with load balancing is described. Important features of this architecture are the high level of system reliability and the possibility of using lowcost hardware solutions.
-
Telemetric Distributed Systems - 2022
e-Learning Courses -
Telemetric Distributed Systems - 2023
e-Learning Courses -
Distributed representation of information on cyclic events
PublicationA representation of information on cyclic events has been proposed which is advantageous for computing environments where a distributed set of Receivers reacts to cyclic events generated by distributed sources. In such scenario no immanent central information repository exist on event timing or volume. Receivers are able to learn the event cycles without communicating with each other, merely on the basis of the fact that an event...
-
Modelling of Mechatronic Systems with Distributed Parameter Components
PublicationThe paper presents an uniform, port-based approach to modelling of both lumped and distributed parameter systems. Port-based model of distributed system has been defined by application of the bond graph methodology and the distributed transfer function method (DTFM). The proposed method of modelling enables to formulate input data for computer analysis by application of the DTFM. The computational package for the analysis of complex...
-
Port-based approach to distributed transfer function method
PublicationIn the paper there is presented an uniform, port-based approach to modeling of both lumped and distributed parameter systems. Port-based model of distributed system has been defined by application of distributed transfer function method (DTFM). The approach proposed combines versatility of port - based modeling and accuracy of distributed transfer function method. It enables to formulate appropriate input data for computer analysis...
-
Port-Based Modelling of Distributed-Lumped Parameter Systems
PublicationThere in the paper is presented an uniform, port-based approach to modelling of both lumped and distributed parameter systems. Port-based model of the distributed system has been defined by application of bond graph methodology and distributed transfer function method (DTFM). The approach proposed combines versatility of port-based modelling and accuracy of distributed transfer function method. The concise representation of lumped-distributed...
-
Paweł Czarnul dr hab. inż.
PeoplePaweł Czarnul obtained a D.Sc. degree in computer science in 2015, a Ph.D. in computer science granted by a council at the Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology in 2003. His research interests include:parallel and distributed processing including clusters, accelerators, coprocessors; distributed information systems; architectures of distributed systems; programming mobile devices....
-
Depth Images Filtering In Distributed Streaming
PublicationIn this paper, we propose a distributed system for point cloud processing and transferring them via computer network regarding to effectiveness-related requirements. We discuss the comparison of point cloud filters focusing on their usage for streaming optimization. For the filtering step of the stream pipeline processing we evaluate four filters: Voxel Grid, Radial Outliner Remover, Statistical Outlier Removal and Pass Through....
-
DEPTH IMAGES FILTERING IN DISTRIBUTED STREAMING
PublicationIn this paper we discuss the comparison of point cloud filters focusing on their applicability for streaming optimization. For the filtering stage within a stream pipeline processing we evaluate three filters: Voxel Grid, Pass Through and Statistical Outlier Removal. For the filters we perform series of the tests aiming at evaluation of changes of point cloud size and transmitting frequency (various fps ratio). We propose a distributed...
-
Optimization issues in distributed computing systems design
PublicationIn recent years, we observe a growing interest focused on distributed computing systems. Both industry and academia require increasing computational power to process and analyze large amount of data, including significant areas like analysis of medical data, earthquake, or weather forecast. Since distributed computing systems – similar to computer networks – are vulnerable to failures, survivability mechanisms are indispensable...
-
Image Processing Techniques for Distributed Grid Applications
PublicationParallel approaches to 2D and 3D convolution processing of series of images have been presented. A distributed, practically oriented, 2D spatial convolution scheme has been elaborated and extended into the temporal domain. Complexity of the scheme has been determined and analysed with respect to coefficients in convolution kernels. Possibilities of parallelisation of the convolution operations have been analysed and the results...
-
Geometric Algebra Model of Distributed Representations
PublicationFormalism based on GA is an alternative to distributed representation models developed so far-Smolensky's tensor product, Holographic Reduced Representations (HRR) and Binary Spatter Code (BSC). Convolutions are replaced by geometric products, interpretable in terms of geometry which seems to be the most natural language for visualization of higher concepts. This paper recalls the main ideas behind the GA model and investigates...
-
Agent-Based Non-distributed and Distributed Clustering
Publication -
Long Distance Geographically Distributed InfiniBand Based Computing
PublicationCollaboration between multiple computing centres, referred as federated computing is becom- ing important pillar of High Performance Computing (HPC) and will be one of its key components in the future. To test technical possibilities of future collaboration using 100 Gb optic fiber link (Connection was 900 km in length with 9 ms RTT time) we prepared two scenarios of operation. In the first one, Interdisciplinary Centre for Mathematical...
-
Distributed infrastructure of self-organizing service servers
PublicationThis paper presents an idea of creating distributed system consisting of autonomous self-organizing service providers. It shows a real implemented system allowing for dynamic service search without interaction with client based on the inter-servers communication. Moreover presented system can be easily enriched with new elements without the need for restarting already existing servers.
-
Improving flexibility and performance of PVM applications by distributed partial evaluation
PublicationA new framework for developing both flexible and efficient PVM applications is described. We propose Architecture Templates Interface (ATI) that allows to control application granularity and parallelism. To ensure high application efficiency we extend partial evaluation strategy into domain of distributed applications obtaining Distributed Partial Evaluation (DPE). Both ATI and DPE were implemented using a new distributed programming...
-
Integration of cloud-based services into distributed workflow systems: challenges and solutions
PublicationThe paper introduces the challenges in modern workflow management in distributed environments spanning multiplecluster, grid and cloud systems. Recent developments in cloud computing infrastructures are presented and are referring howclouds can be incorporated into distributed workflow management, aside from local and grid systems considered so far. Severalchallenges concerning workflow definition, optimisation and execution are...
-
Tracing of dynamic objects in distributed interactive simulation systems
PublicationDistributed interactive simulation systems require integration of several areas of computer science and applied mathematics to enable each individual simulation object to visualize effectively dynamic states of other objects. Objects are unpredictable,i.e., controlled by their local operators, and are remote, i.e., must rely on some transmission media to visualize dynamic scene from their local perspectives. The paper...
-
Application capabilities of the maximum distributed generation estimate methodology
PublicationThe paper presents application capabilities of the maximum distributed generation estimate methodology. This subject is an example of solutions to the problem that today face the transmission system operator and distribution system operators, which is related to the high saturation with wind power generation predicted for the near future.
-
Hybrid, Approximate Models of Distributed-Parameter Systems
PublicationThe paper introduces the method of distributed-parameter systems modelling. It enables to obtain low order modal model of the system that experiences Coriolis acceleration component and gyroscopic effect. In such cases, corresponding system equations are non-self-adjoined. To solve this problem modal reduced model is built up for the system without Coriolis acceleration or gyroscopic effect terms. These phenomena are next included...
-
A Distributed Description of Facts in a Distributed Semantic Web Ontology
PublicationInicjatywa Sieci Semantycznej (ang. Semantic Web) zakłada tworzenie w Internecie ontologii zawierających semantyczny opis dużych dziedzin wiedzy. Okazało się jednak, że przekroczenie pewnej wielkości ontologii powoduje szereg negatywnych zjawisk. Aby tym zjawiskom zapobiec, wiele prac koncentruje się na dzieleniu baz wiedzy. Celem tego opracowania jest zaprezentowanie problemów związanych z modularyzacją oraz przedstawienie kilku...
-
Issues of Estimating the Maximum Distributed Generation at High Wind Power Participation
PublicationThis paper presents the methods of estimating the maximum power that can be connected to the power system in distributed generation sources. Wind turbine generator systems (WTGS) were selected as the subject for analysis. Nonetheless, the considerations presented in this paper are only general and also apply to other types of power sources, including the sources that are not considered part of distributed generation.
-
Distributed wireless sensor for measuring humidity and temperature
PublicationHumidity and temperature monitoring is often required in industrial halls and laboratories and can be crucial, because many properties of electrical devices deteriorates under influence of these parameters. There are several useful commercial methods for measuring humidity and temperature, such as commercial hygrometers and humidity recorders. However, they do not allow data acquisition simultaneously from multiple sensors. This...
-
Distributed state estimation using a network of asynchronous processing nodes
PublicationWe consider the problem of distributed state estimation of continuous-time stochastic processes using a~network of processing nodes. Each node performs measurement and estimation using the Kalman filtering technique, communicates its results to other nodes in the network, and utilizes similar results from the other nodes in its own computations. We assume that the connection graph of the network is not complete, i.e. not all nodes...
-
Distributed state estimation using a network of asynchronous processing nodes
PublicationWe consider the problem of distributed state estimation of continuous-time stochastic processes using a~network of processing nodes. Each node performs measurement and estimation using the Kalman filtering technique, communicates its results to other nodes in the network, and utilizes similar results from the other nodes in its own computations. We assume that the connection graph of the network is not complete, i.e. not all nodes...
-
Distributed Evacuation in Graphs with Multiple Exits
PublicationWe consider the problem of efficient evacuation using multiple exits. We formulate this problem as a discrete problem on graphs where mobile agents located in distinct nodes of a given graph must quickly reach one of multiple possible exit nodes, while avoiding congestion and bottlenecks. Each node of the graph has the capacity of holding at most one agent at each time step. Thus, the agents must choose their movements strategy...
-
CMMI - an Assurance of Transferability of Knowledge andProjects in a Distributed Enterprise
PublicationCMMI (Capability Maturity Model Integrated)is a reference model fordefining and performing processes in software and systems engineeringorganizations.An organization implementing CMMI,among other things,assures thatacit knowledge is properly externalized into formal knowledge.This externalization gives an organization opportunity to assure that knowledge and development projects are easily transferable across adistributed enterprise...
-
Crowdsourcing and Volunteer Computing as Distributed Approach for Problem Solving
PublicationIn this paper, a combination between volunteer computing and crowdsourcing is presented. Two paradigms of the web computing are described, analyzed and compared in detail: grid computing and volunteer computing. Characteristics of BOINC and its contribution to global Internet processing are shown with the stress put onto applications the system can facilitate and problems it can solve. An alternative instance of a grid computing...
-
Distributed Evolutionary Algorithm for Path Planning in Navigation Situation
PublicationThis article presents the use of a multi-population distributed evolutionary algorithm for path planning in navigation situation. The algorithm used is with partially exchanged population and migration between independently evolving populations. In this paper a comparison between a multi-population and a classic single-population algorithm takes place. The impact on the ultimate solution has been researched. It was shown that using...
-
Multi-criterion decision making in distributed systems by quantum evolutionary algorithms
PublicationDecision making by the AQMEA (Adaptive Quantum-based Multi-criterion Evolutionary Algorithm) has been considered for distributed computer systems. AQMEA has been extended by a chromosome representation with the registry of the smallest units of quantum information. Evolutionary computing with Q-bit chromosomes has been proofed to characterize by the enhanced population diversity than other representations, since individuals represent...
-
On Configurability of Distributed Volunteer-Based Computing in the Comcute System
PublicationThe chapter proposes additional solutions that can be implemented within the Comcute system to increase its configurability. This refers to configuration of the reliability level in the W and S server layers, static or on-the-fly data partitioning and integration, configuration of the system for processing in the data streaming fashion, extending the system for selection of a project that the client wants to contribute to, ease...
-
Simulation of Parallel Applications on Large-scale Distributed Systems
PublicationThis chapter has a form of a review article in the field of simulating High-Performance Computing systems. We justify the need for a new versatile simulator considering heterogeneity, energy efficiency and reliability of HPC systems. We sketch the problems that need to be solved by such simulator and rationalize using discrete-event simulation for this purpose. Based on a review of existing discrete-event HPC simulation solutions...
-
Distributed Detection of Selected Features in Data Streams Using Grid-class Systems
PublicationThis chapter describes basic methodology of distributed digital signal processing. A choice of distributed methods of detection of selected features in data streams using grid-class systems is discussed. Problems related to distribution of data for processing are addressed. A mitigating method for data distribution and result merging is described.
-
Computational algorithm for the analysis of mechatronic systems with distributed parameter elements
PublicationThe paper presents a systematic computational package for analysis of complex systems composed of multiple lumped and distributed parameter subsystems. The algorithm is based on the transfer function method (DTFM). With this algorithm, a bond graph technique for the modelling is developed to simplify computations. Analysis of different systems requires only changing the inputs data in the form of the bond graph diagram
-
EVENTS VISUALIZATION POST IN A DISTRIBUTED TELEINFORMATION SYSTEM FOR THE BORDER GUARD
PublicationEvents Visualization Post is a part of the STRADAR project, which is dedicated to streaming real-time data in distributed dispatcher and teleinformation systems of the Border Guard. Events Visualization Post is a software designed for simultaneous visualization of data of different types. In the paper, the structure of the software is presented, the process of generation of tasks is described, and the visualization of audio, files,...