Search
dr hab. inż. Paweł Czarnul
Employment
- Chief Specialist at Dział Usług Chmurowych
- Vice-Dean for Cooperation and Development at Faculty of Electronics, Telecommunications and Informatics
- Head of Department at Department of Computer Architecture
Publications
Filters
total: 120
Catalog Publications
Year 2017
-
Parallelization of Selected Algorithms on Multi-core CPUs, a Cluster and in a Hybrid CPU+Xeon Phi Environment
PublicationIn the paper we present parallel implementations as well as execution times and speed-ups of three different algorithms run in various environments such as on a workstation with multi-core CPUs and a cluster. The parallel codes, implementing the master-slave model in C+MPI, differ in computation to communication ratios. The considered problems include: a genetic algorithm with various ratios of master processing time to communication...
-
Performance evaluation of unified memory and dynamic parallelism for selected parallel CUDA applications
PublicationThe aim of this paper is to evaluate performance of new CUDA mechanisms—unified memory and dynamic parallelism for real parallel applications compared to standard CUDA API versions. In order to gain insight into performance of these mechanisms, we decided to implement three applications with control and data flow typical of SPMD, geometric SPMD and divide-and-conquer schemes, which were then used for tests and experiments. Specifically,...
Year 2016
-
A Fail-Safe NVRAM Based Mechanism for Efficient Creation and Recovery of Data Copies in Parallel MPI Applications
PublicationThe paper presents a fail-safe NVRAM based mechanism for creation and recovery of data copies during parallel MPI application runtime. Specifically, we target a cluster environment in which each node has an NVRAM installed in it. Our previously developed extension to the MPI I/O API can take advantage of NVRAM regions in order to provide an NVRAM based cache like mechanism to significantly speed up I/O operations and allow to preload...
-
A Parallel MPI I/O Solution Supported by Byte-addressable Non-volatile RAM Distributed Cache
PublicationWhile many scientific, large-scale applications are data-intensive, fast and efficient I/O operations have become of key importance for HPC environments. We propose an MPI I/O extension based on in-system distributed cache with data located in Non-volatile Random Access Memory (NVRAM) available in each cluster node. The presented architecture makes effective use of NVRAM properties such as persistence and byte-level access behind...
-
Application of Web-GIS and Cloud Computing to Automatic Satellite Image Correction
PublicationRadiometric calibration of satellite imagery requires coupling of atmospheric and topographic parameters, which constitutes serious computational problems in particular in complex geographical terrain. Successful application of topographic normalization algorithms for calibration purposes requires integration of several types of high-resolution geographic datasets and their processing in a common context. This paper presents the...
-
Benchmarking Performance of a Hybrid Intel Xeon/Xeon Phi System for Parallel Computation of Similarity Measures Between Large Vectors
PublicationThe paper deals with parallelization of computing similarity measures between large vectors. Such computations are important components within many applications and consequently are of high importance. Rather than focusing on optimization of the algorithm itself, assuming specific measures, the paper assumes a general scheme for finding similarity measures for all pairs of vectors and investigates optimizations for scalability...
-
Checkpointing of Parallel MPI Applications using MPI One-sided API with Support for Byte-addressable Non-volatile RAM
PublicationThe increasing size of computational clusters results in an increasing probability of failures, which in turn requires application checkpointing in order to survive those failures. Traditional checkpointing requires data to be copied from application memory into persistent storage medium, which increases application execution time as it is usually done in a separate step. In this paper we propose to use emerging byte-addressable...
-
Considerations of Computational Efficiency in Volunteer and Cluster Computing
PublicationIn the paper we focus on analysis of performance and power consumption statistics for two modern environments used for computing – volunteer and cluster based systems. The former integrate computational power donated by volunteers from their own locations, often towards social oriented or targeted initiatives, be it of medical, mathematical or space nature. The latter is meant for high performance computing and is typically installed...
-
KernelHive: a new workflow-based framework for multilevel high performance computing using clusters and workstations with CPUs and GPUs
PublicationThe paper presents a new open-source framework called KernelHive for multilevel parallelization of computations among various clusters, cluster nodes, and finally, among both CPUs and GPUs for a particular application. An application is modeled as an acyclic directed graph with a possibility to run nodes in parallel and automatic expansion of nodes (called node unrolling) depending on the number of computation units available....
-
Modeling energy consumption of parallel applications
PublicationThe paper presents modeling and simulation of energy consumption of two types of parallel applications: geometric Single Program Multiple Data (SPMD) and divide-and-conquer (DAC). Simulation is performed in a new MERPSYS environment. Model of an application uses the Java language with extension representing message exchange between processes working in parallel. Simulation is performed by running threads representing distinct process...
-
Modeling Parallel Applications in the MERPSYS Environment
PublicationThe chapter presents how to model parallel computational applications for which simulation of execution in a large-scale parallel or distributed environment is performed within the MERPSYS environment. Specifically, it is shown what approaches can be adopted to model key paradigms often used for parallel applications: master-slave, geometric parallelism (single program multiple data), pipelined and divide-and-conquer applications....
-
Performance and Power-Aware Modeling of MPI Applications for Cluster Computing
PublicationThe paper presents modeling of performance and power consumption when running parallel applications on modern cluster-based systems. The model includes basic so-called blocks representing either computations or communication. The latter includes both point-to-point and collective communication. Real measurements were performed using MPI applications and routines run on three different clusters with both Infiniband and Gigabit Ethernet...
Year 2015
-
BeesyCluster as Front-End for High Performance Computing Services
PublicationThe paper presents the BeesyCluster system as a middleware allowing invocation of services on high performance computing resources within the NIWA Centre of Competence project. Access is possible through both WWW and SOAP Web Service interfaces. The former allows non-experienced users to invoke both simple and complex services exposed through easyto-use servlets. The latter is meant for integration of external applications with...
-
Integration of Services into Workflow Applications
PublicationDescribing state-of-the-art solutions in distributed system architectures, Integration of Services into Workflow Applications presents a concise approach to the integration of loosely coupled services into workflow applications. It discusses key challenges related to the integration of distributed systems and proposes solutions, both in terms of theoretical aspects such as models and workflow scheduling algorithms, and technical...
-
Simulation of parallel similarity measure computations for large data sets
PublicationThe paper presents our approach to implementation of similarity measure for big data analysis in a parallel environment. We describe the algorithm for parallelisation of the computations. We provide results from a real MPI application for computations of similarity measures as well as results achieved with our simulation software. The simulation environment allows us to model parallel systems of various sizes with various components...
Year 2014
-
A Workflow Application for Parallel Processing of Big Data from an Internet Portal
PublicationThe paper presents a workflow application for efficient parallel processing of data downloaded from an Internet portal. The workflow partitions input files into subdirectories which are further split for parallel processing by services installed on distinct computer nodes. This way, analysis of the first ready subdirectories can start fast and is handled by services implemented as parallel multithreaded applications using multiple...
-
Comparison of selected algorithms for scheduling workflow applications with dynamically changing service availability
PublicationThis paper compares the quality and execution times of several algorithms for scheduling service based workflow applications with changeable service availability and parameters. A workflow is defined as an acyclic directed graph with nodes corresponding to tasks and edges to dependencies between tasks. For each task, one out of several available services needs to be chosen and scheduled to minimize the workflow execution time and...
-
Optimization of Execution Time under Power Consumption Constraints in a Heterogeneous Parallel System with GPUs and CPUs
PublicationThe paper proposes an approach for parallelization of computations across a collection of clusters with heterogeneous nodes with both GPUs and CPUs. The proposed system partitions input data into chunks and assigns to par- ticular devices for processing using OpenCL kernels defined by the user. The sys- tem is able to minimize the execution time of the application while maintaining the power consumption of the utilized GPUs and...
-
Parallel computations in the volunteer based Comcute system
PublicationThe paper presents Comcute which is a novel multi-level implemen- tation of the volunteer based computing paradigm. Comcute was designed to let users donate the computing power of their PCs in a simplified manner, requiring only pointing their web browser at a specific web address and clicking a mouse. The server side appoints several servers to be in charge of execution of particular tasks. Thanks to that the system can survive...
-
Performance Modeling and Prediction of Real Application Workload in a Volunteer-based System
PublicationThe goal of this paper is to present a model that predicts the real workload placed on a volunteer based system by an application, with incorporation of not only performance but also availability of volunteers. The application consists of multiple data packets that need to be processed. Knowing the computational workload demand of a single data packet we show how to estimate the application workload in a volunteer based system. Furthermore,...
-
Runtime Visualization of Application Progress and Monitoring of a GPU-enabled Parallel Environment
PublicationThe paper presents design, implementation and real life uses of a visualization subsystem for a distributed framework for parallelization of workflow-based computations among clusters with nodes that feature both CPUs and GPUs. Firstly, the proposed system presents a graphical view of the infrastructure with clusters, nodes and compute devices along with parameters and runtime graphs of load, memory available, fan speeds etc. Secondly,...
-
Teaching High Performance Computing Using BeesyCluster and Relevant Usage Statistics
PublicationThe paper presents motivations and experiences from using the BeesyCluster middleware for teaching high performance computing at the Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology. Features of BeesyCluster well suited for conducting courses are discussed including: easy-to-use WWW interface for application development and running hiding queuing systems, publishing applications as services...
Year 2013
-
A Concept of Modeling and Optimization of Applications in Large Scale Systems
PublicationThe chapter presents the idea that includes modeling and subsequent optimization of application execution on large scale parallel and distributed systems. The model considers performance, reliability and power consumption. It should allow easy modeling of various classes of applications while reflecting key parameters of both the applications and two classes of target systems: clusters and volunteer based systems. The chapter presents...
-
A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints
PublicationThe paper proposes a model, design, and implementation of an efficient multithreaded engine for execution of distributed service-based workflows with data streaming defined on a per task basis. The implementation takes into account capacity constraints of the servers on which services are installed and the workflow data footprint if needed. Furthermore, it also considers storage space of the workflow execution engine and its cost....
-
An Evaluation Engine for Dynamic Ranking of Cloud Providers
PublicationThe paper focuses on creation of an effective dynamic ranking service for IaaS, PaaS and SaaS cloud providers. It considers building a quality model for this purpose along with definition of quality measure- ment procedures. The paper discusses several techniques known from already existing price comparison engines that could be modified and adopted for comparison of cloud providers. A technique for filtering measured data is proposed,...
-
Block-based Representation of Application Execution on Modern Parallel Systems
PublicationThe chapter presents how to model execution of a parallel computational application that is to be executed in a large-scale parallel or distributed environment with potentially thousands to millions of execution units. The representation uses pre- viously attributes and factors representative of modern high performance systems including multicore CPUs, GPUs, dedicated accelerators such as Intel Phi.
-
Design of a Distributed System using Mobile Devices and Workflow Management for Measurement and Control of a Smart Home and Health
PublicationThe paper presents design of a distributed system for measurements and control of a smart home including temper- atures, light, fire danger, health problems of inhabitants such as increased body temperature, a person falling etc. This is done by integration of mobile devices and standards, distributed service based middleware BeesyCluster and a workflow management system. Mobile devices are used to measure the parameters and are...
-
Funkcjonalność platformy KASKADA
PublicationPrzedstawiono opis funkcjonalności platformy KASKADA, a w szczególności jej architektury oprogramowania, mechanizmów zarządzania oraz biblioteki usług aplikacyjnych. Zaprezentowano poszczególne komponenty platformy, opisano ich budowę i implementację podstawowych funkcji, jak również zasygnalizowano problemy jakie wystąpiły podczas jej implementacji.
Year 2012
-
Analiza istniejących obliczeń uruchamianych w systemach rozproszonych
PublicationZaprezentowano analizę algorytmów równoległych tradycyjnie uruchamianych w systemach klastrowych wysokiej wydajności, a następnie pokazano charakterystykę algorytmów ze względu na parametry istotne przy implementacji ich rozwiązań w rozproszonym środowisku Comcute. Następnie przedstawiono ocenę możliwości ich przeniesienia do tego środowiska.
-
Automatyczna adaptacja usług sieciowych do usług systemu Be-esyCluster na potrzeby realizacji rozproszonych scenariuszy
PublicationRozdział przedstawia adaptację tradycyjnych usług sieciowych (ang. Web Services) do usług platformy BeesyCluster. Pozwalato na ich wykorzystanie w module zarządzania rozproszonymi scenariuszami usług, w tym: modelowaniu scenariuszy, automatycznym doborzeusług oraz wykonaniu. Przedstawiono także rozszerzenia opisu usług związane ze strumieniowym przetwarzaniem danych, ograniczeniami na dane, wyszukiwaniem zasobów do wykonania usług...
-
Dynamic ranking of cloud providers
PublicationThe paper focuses on how to create an effective dynamic ranking service for IaaS,PaaS and SaaS cloud providers. It considers building of a quality model for thispurpose along with definition of quality measurement procedures. The paper discussesseveral techniques known from already existing price comparison engines that couldbe modified and adopted for comparison of cloud providers. Furthermore, a tech-nique for filtering measured...
-
Evaluation of multimedia applications in a cluster oriented environment
PublicationIn the age of Information and Communication Technology (ICT), Web and the Internet have changed significantly the way applications are developed, deployed and used. One of recent trends is modern design of web-applications based on SOA. This process is based on the composition of existing web services into a single scenario from the point of view of a particular user or client. This allows IT companies to shorten product time-to-market....
-
Integration of cloud-based services into distributed workflow systems: challenges and solutions
PublicationThe paper introduces the challenges in modern workflow management in distributed environments spanning multiplecluster, grid and cloud systems. Recent developments in cloud computing infrastructures are presented and are referring howclouds can be incorporated into distributed workflow management, aside from local and grid systems considered so far. Severalchallenges concerning workflow definition, optimisation and execution are...
-
On Configurability of Distributed Volunteer-Based Computing in the Comcute System
PublicationThe chapter proposes additional solutions that can be implemented within the Comcute system to increase its configurability. This refers to configuration of the reliability level in the W and S server layers, static or on-the-fly data partitioning and integration, configuration of the system for processing in the data streaming fashion, extending the system for selection of a project that the client wants to contribute to, ease...
-
Rozproszona aplikacja do śledzenia komputerów w systemie po ich przemieszczeniu oraz do monitorowania wykorzystania komputerów
PublicationZaprezentowano propozycję aplikacji internetowej pozwalającej na zdalne śledzenie lokalizacji jak również wykorzystania zasobów komputerów takich jak: zużycie procesora czy pamięci dyskowej. Przedstawiono dwa warianty aplikacji: uruchamianej jako niezależna aplikacja, bądź jako aplikacja działająca w przeglądarce internetowej. Proponowane rozwiązanie zwiększy możliwość odszukania komputerów po kradzieży jak również pozwoli na sporządzenie...
-
Sposoby definiowana zadań i zlecania wykonywania obliczeń w systemie Comcute
PublicationZaprezentowano specyfikację elementów definiowanych przez zleceniodawcę systemu Comcute przy uruchamianiu nowych zadań obliczeniowych w systemie, w tym parametrów niezawodnościowo-wydajnościowych jak również kodów obliczeniowych, partycjonowania i scalania. Przedstawiono także wymagania związane z konfigurowalnością systemu i jego bezpieczeństwem a także wskazówki technologiczne.
Year 2011
-
BeesyBees: A mobile agent-based middleware for a reliable and secure execution of service-based workflow applications in BeesyCluster
PublicationIntegrating distributed services into workflows comes with its own set of challenges, including security, coordination, fault tolerance and optimisation of execution time. This paper presents an architecture and implementation - nicknamed BeesyBees - that allows distributed execution of workflow applications in BeesyCluster using agents. BeesyCluster is a middleware that allows users to access distributed resources as well as publish...
-
Dynamic Compatibility Matching of Services for Distributed Workflow Execution
PublicationThe paper presents a concept and an implementation of dynamic learn-ing of compatibilities of services used in a workflow application. While services may have the same functionality, they may accept input and produce output in different formats. The proposed solution learns matching of outputs and inputs at runtime and uses this knowledge in subsequent runs of workflow applications. The presented solution was implemented in an...
-
Parallelization of Compute Intensive Applications into Workflows based on Services in BeesyCluster
PublicationThe paper presents an approach for modeling, optimization and execution of workflow applications based on services that incorporates both service selection and partitioning of input data for parallel processing by parallel workflow paths. A compute-intensive workflow application for parallel integration is presented. An impact of the input data partitioning on the scalability is presented. The paper shows a comparison of the theoretical...
Year 2010
-
Automatic conversion of legacy applications into services in beesycluster
PublicationPrzedstawiono rozwiązanie, dzieki któremu uzytkownikw prosty sposób uzyskuje mozliwość automatycznej konwersji aplikacji dostępnych w systemach Unix do usług w systemie BeesyCluster. system BeesyCluster stanowi warstę pośredniczącą w dostepie do sieci klasrów poprzez WWW. Aby zapewnić szeroki zakres dostępnych usług mozliwa jest konwersja wielu pakietów linuksowych jednoczesnie. narzędzie umozliwia na podstawie wydobytych informacji...
-
Beesybees-Agent-Based, Adaptive & Learning Workflow Execution Module for BeesyCluster
PublicationPrezentujemy projekt oraz implementację adaptacyjnego i uczącego się modułu przeznaczonego dowykonywania scenariuszy w środowisku BeesyCluster. BeesyCluster pozwala na modelowaniescenariuszy w formie acyklicznego grafu skierowanego, w którym wierzchołki oznaczają zadania,a krawędzie określają zależności między nimi. Przedstawiamy także kooperatywne wykonaniescenariusza przez grupę agentów zdolnych do zbierania, składowania i korzystania...
-
Modeling, run-time optimization and execution of distributed workflow applications in the JEE-based BeesyCluster environment
PublicationArtykuł prezentuje kompletne rozwiązanie do modelowania naukowych i biznesowych scenariuszy. statycznego i dynamicznego wyboru usług z uwzględnieniem parametrów jakościowych oraz wykonanie scenariuszy w rzeczywistym środowisku. Scenariusz jest modelowany jako acykliczny graf skierowany, w którym węzły reprezentują zadania zaś krawędzie zależności pomiędzy zadaniami. Warstwa pośrednia BeesyCluster jest wykorzystana do umożliwienia...
-
Nowoczesne koncepcje integracji usług w systemie BeesyCluster
PublicationOpisano funkcje aktualnej wersji systemu BeesyCluster jakowarstwy pośredniej w dostępie do rozproszonych zasobów wraz podsystemami integracji usług, wyboru usług oraz ich wykonania. Zaprezentowano rozszerzenia podsystemu integracji usług zorientowane na green computing. Omówiono problemy inteligentnego wyszukiwania usług, wykorzystanie GPU, współpracę z urządzeniami mobilnymi oraz przetwarzanie w przestrzeniach inteligentnych.Dodatkowo...
-
Workflow application for detection of unwanted events
PublicationZaprezentowano rozproszoną aplikację do wykrywania potencjalnie niebezpiecznych zdarzeń z wejściowych strumieni wideo. Rozpoznanie niepożądanych zdarzeń wywołuje alarmy i wysyła powiadomienia do odpowiednich służb, jak również powoduje zarejestrowanie filmu. Model aplikacji składa się z węzłów z kamerami, pobierajacych strumienie danych, przetwarzajacych dane, wysyłajacych powiadomienia i zapisujacych dane. Zaimplementowana aplikacja...
Year 2009
-
A JEE-based modelling and execution environment for workflow applications with just-in-time service selection
PublicationArtykuł prezentuje model scenariusza z dynamicznym wyborem usług do wykonania zadań scenariusza. Adekwatność rozwiązania została zaprezentowana dla zastosowań naukowych i biznesowych w warunkach kiedy dostępność usług się zmienia i usługi powinny być wybierane w trakcie działania scenariusza a nie przed jego rozpoczęciem. Pokazano, że dla scenariusza naukowego z powtarzalnymi symulacjami, algorytm dobiera usługi, aby zminimalizować...
-
Metodologie doboru usług sieciowych na potrzeby automatycznego wykonywania scenariuszy
PublicationPrzedstawiono metodologie tworzenia scenariuszy jak równieżdoboru usług do optymalnego ich wykonania. Złożone scenariusze pojawiają się w różnych kontekstach: zastosowań biznesowych, aplikacji naukowych jak również w przetwarzaniu wszechobecnym i mobilnym. Wymienione są problemy i stosowane rozwiązania począwszy od publikacji informacji o usłudze, przez optymalizację doboru usług do wykorzystywanych protokołów i standardów.
-
Wykorzystanie ontologii do wyszukiwania usług w systemie BeesyCluster
PublicationNiniejszy rozdział przedstawia koncepcję wykorzystania ontologiido wyszukiwania usług na platformie BeesyCluster, która udostępniausługi różnych dostawców i pozwala na ich wywołanie. Ontologia dziedziny wykorzystana jest zarówno przez dostawców usług w procesie publikacji informacji o usłudze jak również przez inteligetny algorytm wyszukiwania i dopasowania usług do wymagań klienta systemu BeesyCluster. Przedstawiono architekturę...
Year 2007
-
BC-MPI: running an mpi application on multiple clusters with beesycluster connectivity
PublicationW artykule zaproponowano nowy pakiet BC-MPI, który umożliwiauruchomienie aplikacji MPI na wielu klastrach z różnymi implementacjami MPI. Wykorzystuje dedykowane implementacje MPIdo komunikacji wewnątrz klastrów oraz tryb MPI THREAD MULTIPLE dokomunikacji pomiędzy klastrami w dodatkowych wątkach aplikacji MPI. Ponadto, aplikacja BC-MPI może być automatycznie skompilowanai uruchomiona przez warstwę pośrednią BeesyCluster. BeesyClusterumożliwia...
Year 2006
-
Integration of compute-intensive tasks into scientific workflows in BeesyCluster
PublicationArtykuł prezentuje szczegóły projektowe i implementacyjne jak również symulacje naukowych scenariuszy z wymagającymi zadaniami na klastrach i komputerach PC. Autor wdrożył wsparcie dla scenariuszy naukowych do wcześniej opracowanego systemu BeesyCluster wdrożonego w Akademickim Centrum Komputerowym sieci TASK w Gdańsku na klastrze holk z 288 procesorami Itanium2. BeesyCluster pozwala użytkownikom na zarządzanie różnymi kontami...
-
Internetowy portal wspomagający symulacje i analizę sytuacji krytycznych w regionie oraz zespołowe podejmowanie decyzji
PublicationOmówiono internetowy portal dostepowy zintegrowanego systemu służącego do analizy sytuacji wyjatkowych, wizualizacji przestrzennej kluczowych obiektów infrastruktury wraz z numerycznymi ocenami różnego typu zagrożeń terrorystycznych lub katastrof naturalnych. Portal zapewnia interakcję pomiędzy rozproszonymi grupami terenowymi oraz centrami zarządzania sytuacjami kryzysowymi, również decydentami, współdzieloną wizualizację wyników...
seen 5140 times