Search
Search results for: deep neural network training benchmarking parallel computations caffe mkl
-
Benchmarking Deep Neural Network Training Using Multi- and Many-Core Processors
PublicationIn the paper we provide thorough benchmarking of deep neural network (DNN) training on modern multi- and many-core Intel processors in order to assess performance differences for various deep learning as well as parallel computing parameters. We present performance of DNN training for Alexnet, Googlenet, Googlenet_v2 as well as Resnet_50 for various engines used by the deep learning framework, for various batch sizes. Furthermore,...
-
Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training
PublicationIn the paper we tackle bi-objective execution time and power consumption optimization problem concerning execution of parallel applications. We propose using a discrete-event simulation environment for exploring this power/time trade-off in the form of a Pareto front. The solution is verified by a case study based on a real deep neural network training application for automatic speech recognition. A simulation lasting over 2 hours...
-
The impact of the AC922 Architecture on Performance of Deep Neural Network Training
PublicationPractical deep learning applications require more and more computing power. New computing architectures emerge, specifically designed for the artificial intelligence applications, including the IBM Power System AC922. In this paper we confront an AC922 (8335-GTG) server equipped with 4 NVIDIA Volta V100 GPUs with selected deep neural network training applications, including four convolutional and one recurrent model. We report...
-
Performance and Energy Aware Training of a Deep Neural Network in a Multi-GPU Environment with Power Capping
PublicationIn this paper we demonstrate that it is possible to obtain considerable improvement of performance and energy aware metrics for training of deep neural networks using a modern parallel multi-GPU system, by enforcing selected, non-default power caps on the GPUs. We measure the power and energy consumption of the whole node using a professional, certified hardware power meter. For a high performance workstation with 8 GPUs, we were...
-
Paweł Rościszewski dr inż.
PeoplePaweł Rościszewski received his PhD in Computer Science at Gdańsk University of Technology in 2018 based on PhD thesis entitled: "Optimization of hybrid parallel application execution in heterogeneous high performance computing systems considering execution time and power consumption". Currently, he is an Assistant Professor at the Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, Poland....
-
Poprawa jakości klasyfikacji głębokich sieci neuronowych poprzez optymalizację ich struktury i dwuetapowy proces uczenia
PublicationW pracy doktorskiej podjęto problem realizacji algorytmów głębokiego uczenia w warunkach deficytu danych uczących. Głównym celem było opracowanie podejścia optymalizującego strukturę sieci neuronowej oraz zastosowanie uczeniu dwuetapowym, w celu uzyskania mniejszych struktur, zachowując przy tym dokładności. Proponowane rozwiązania poddano testom na zadaniu klasyfikacji znamion skórnych na znamiona złośliwe i łagodne. W pierwszym...
-
Resource constrained neural network training
PublicationModern applications of neural-network-based AI solutions tend to move from datacenter backends to low-power edge devices. Environmental, computational, and power constraints are inevitable consequences of such a shift. Limiting the bit count of neural network parameters proved to be a valid technique for speeding up and increasing efficiency of the inference process. Hence, it is understandable that a similar approach is gaining...
-
Neural network training with limited precision and asymmetric exponent
PublicationAlong with an extremely increasing number of mobile devices, sensors and other smart utilities, an unprecedented growth of data can be observed in today’s world. In order to address multiple challenges facing the big data domain, machine learning techniques are often leveraged for data analysis, filtering and classification. Wide usage of artificial intelligence with large amounts of data creates growing demand not only for storage...
-
A Bayesian regularization-backpropagation neural network model for peeling computations
PublicationA Bayesian regularization-backpropagation neural network (BRBPNN) model is employed to predict some aspects of the gecko spatula peeling, viz. the variation of the maximum normal and tangential pull-off forces and the resultant force angle at detachment with the peeling angle. K-fold cross validation is used to improve the effectiveness of the model. The input data is taken from finite element (FE) peeling results. The neural network...
-
Deep neural network architecture search using network morphism
PublicationThe paper presents the results of the research on neural architecture search (NAS) algorithm. We utilized the hill climbing algorithm to search for well-performing structures of deep convolutional neural network. Moreover, we used the function preserving transformations which enabled the effective operation of the algorithm in a short period of time. The network obtained with the advantage of NAS was validated on skin lesion classification...
-
Olgun Aydin dr
PeopleOlgun Aydin finished his PhD by publishing a thesis about Deep Neural Networks. He works as a Principal Machine Learning Engineer in Nike, and works as Assistant Professor in Gdansk University of Technology in Poland. Dr. Aydin is part of editorial board of "Journal of Artificial Intelligence and Data Science" Dr. Aydin served as Vice-Chairman of Why R? Foundation and is member of Polish Artificial Intelligence Society. Olgun is...
-
Benchmarking overlapping communication and computations with multiple streams for modern GPUs
PublicationThe paper presents benchmarking a multi-stream application processing a set of input data arrays. Tests have been performed and execution times measured for various numbers of streams and various compute intensities measured as the ratio of kernel compute time and data transfer time. As such, the application and benchmarking is representative of frequently used operations such as vector weighted sum, matrix multiplication etc....
-
Deep neural networks for data analysis 24/25
e-Learning CoursesThis course covers introduction to supervised machine learning, construction of basic artificial deep neural networks (DNNs) and basic training algorithms, as well as the overview of popular DNNs architectures (convolutional networks, recurrent networks, transformers). The course introduces students to popular regularization techniques for deep models. Besides theory, large part of the course is the project in which students apply...
-
Neural networks and deep learning
PublicationIn this chapter we will provide the general and fundamental background related to Neural Networks and Deep Learning techniques. Specifically, we divide the fundamentals of deep learning in three parts, the first one introduces Deep Feed Forward Networks and the main training algorithms in the context of optimization. The second part covers Convolutional Neural Networks (CNN) and discusses their main advantages and shortcomings...
-
Simulation of parallel similarity measure computations for large data sets
PublicationThe paper presents our approach to implementation of similarity measure for big data analysis in a parallel environment. We describe the algorithm for parallelisation of the computations. We provide results from a real MPI application for computations of similarity measures as well as results achieved with our simulation software. The simulation environment allows us to model parallel systems of various sizes with various components...
-
Speaker Recognition Using Convolutional Neural Network with Minimal Training Data for Smart Home Solutions
PublicationWith the technology advancements in smart home sector, voice control and automation are key components that can make a real difference in people's lives. The voice recognition technology market continues to involve rapidly as almost all smart home devices are providing speaker recognition capability today. However, most of them provide cloud-based solutions or use very deep Neural Networks for speaker recognition task, which are...
-
Categorization of emotions in dog behavior based on the deep neural network
PublicationThe aim of this article is to present a neural system based on stock architecture for recognizing emotional behavior in dogs. Our considerations are inspired by the original work of Franzoni et al. on recognizing dog emotions. An appropriate set of photographic data has been compiled taking into account five classes of emotional behavior in dogs of one breed, including joy, anger, licking, yawning, and sleeping. Focusing on a particular...
-
Minimizing Distribution and Data Loading Overheads in Parallel Training of DNN Acoustic Models with Frequent Parameter Averaging
PublicationIn the paper we investigate the performance of parallel deep neural network training with parameter averaging for acoustic modeling in Kaldi, a popular automatic speech recognition toolkit. We describe experiments based on training a recurrent neural network with 4 layers of 800 LSTM hidden states on a 100-hour corpora of annotated Polish speech data. We propose a MPI-based modification of the training program which minimizes the...
-
Deep convolutional neural network for predicting kidney tumour malignancy
PublicationPurpose: According to the statistics, up to 15-20% of removed solid kidney tumors turn out to be benign in postoperative histopathological examination, despite having been identified as malignant by a radiologist. The aim of the research was to limit the number of unnecessary nephrectomies of benign tumors. Methods or Background: We propose a machine-aided diagnostic system for kidney...
-
Deep Learning Optimization for Edge Devices: Analysis of Training Quantization Parameters
PublicationThis paper focuses on convolution neural network quantization problem. The quantization has a distinct stage of data conversion from floating-point into integer-point numbers. In general, the process of quantization is associated with the reduction of the matrix dimension via limited precision of the numbers. However, the training and inference stages of deep learning neural network are limited by the space of the memory and a...
-
Training of Deep Learning Models Using Synthetic Datasets
PublicationIn order to solve increasingly complex problems, the complexity of Deep Neural Networks also needs to be constantly increased, and therefore training such networks requires more and more data. Unfortunately, obtaining such massive real world training data to optimize neural networks parameters is a challenging and time-consuming task. To solve this problem, we propose an easy-touse and general approach to training deep learning...
-
GPU Power Capping for Energy-Performance Trade-Offs in Training of Deep Convolutional Neural Networks for Image Recognition
PublicationIn the paper we present performance-energy trade-off investigation of training Deep Convolutional Neural Networks for image recognition. Several representative and widely adopted network models, such as Alexnet, VGG-19, Inception V3, Inception V4, Resnet50 and Resnet152 were tested using systems with Nvidia Quadro RTX 6000 as well as Nvidia V100 GPUs. Using GPU power capping we found other than default configurations minimizing...
-
Comparative study of methods for artificial neural network training.
PublicationPrzedstawiono wyniki badań porównawczych następujących metod uczenia sieci neuronowych: propagacji wstecznej błędów, rekursywnej metody najmniejszych kwadratów, metody Zangwill'a i algorytmów ewolucyjnych. Badania dotyczyły projektowania adaptacyjnego regulatora neuronowego napięcia generatora synchronicznego.
-
Benchmarking Parallel Chess Search in Stockfish on Intel Xeon and Intel Xeon Phi Processors
PublicationThe paper presents results from benchmarking the parallel multithreaded Stockfish chess engine on selected multi- and many-core processors. It is shown how the strength of play for an n-thread version compares to 1-thread version on both Intel Xeon and latest Intel Xeon Phi x200 processors. Results such as the number of wins, losses and draws are presented and how these change for growing numbers of threads. Impact of using particular...
-
Selected Technical Issues of Deep Neural Networks for Image Classification Purposes
PublicationIn recent years, deep learning and especially Deep Neural Networks (DNN) have obtained amazing performance on a variety of problems, in particular in classification or pattern recognition. Among many kinds of DNNs, the Convolutional Neural Networks (CNN) are most commonly used. However, due to their complexity, there are many problems related but not limited to optimizing network parameters, avoiding overfitting and ensuring good...
-
From Linear Classifier to Convolutional Neural Network for Hand Pose Recognition
PublicationRecently gathered image datasets and the new capabilities of high-performance computing systems have allowed developing new artificial neural network models and training algorithms. Using the new machine learning models, computer vision tasks can be accomplished based on the raw values of image pixels instead of specific features. The principle of operation of deep neural networks resembles more and more what we believe to be happening...
-
Dataset Related Experimental Investigation of Chess Position Evaluation Using a Deep Neural Network
PublicationThe idea of training Articial Neural Networks to evaluate chess positions has been widely explored in the last ten years. In this paper we investigated dataset impact on chess position evaluation. We created two datasets with over 1.6 million unique chess positions each. In one of those we also included randomly generated positions resulting from consideration of potentially unpredictable chess moves. Each position was evaluated...
-
Neural network model of ship magnetic signature for different measurement depths
PublicationThis paper presents the development of a model of a corvette-type ship’s magnetic signature using an artificial neural network (ANN). The capabilities of ANNs to learn complex relationships between the vessel’s characteristics and the magnetic field at different depths are proposed as an alternative to a multi-dipole model. A training dataset, consisting of signatures prepared in finite element method (FEM) environment Simulia...
-
Deep neural networks for data analysis
e-Learning CoursesThe aim of the course is to familiarize students with the methods of deep learning for advanced data analysis. Typical areas of application of these types of methods include: image classification, speech recognition and natural language understanding. Celem przedmiotu jest zapoznanie studentów z metodami głębokiego uczenia maszynowego na potrzeby zaawansowanej analizy danych. Do typowych obszarów zastosowań tego typu metod należą:...
-
Creating neural models using an adaptive algorithm for optimal size of neural network and training set.
PublicationZaprezentowano adaptacyjny algorytm generujący modele neuronowe liniowych układów mikrofalowych, zdolny do oszacowania optymalnego rozmiaru zbiory uczącego i sieci neuronowej. Stworzono kilka modeli nieciągłości falowodowych i mokropaskowych, a następnie zweryfikowano ich poprawność porównując wyniki analiz metodą dopasowania rodzajów i metodą momentów filtrów pasmowo-przepustowych.
-
An Improved Convolutional Neural Network for Steganalysis in the Scenario of Reuse of the Stego-Key
PublicationThe topic of this paper is the use of deep learning techniques, more specifically convolutional neural networks, for steganalysis of digital images. The steganalysis scenario of the repeated use of the stego-key is considered. Firstly, a study of the influence of the depth and width of the convolution layers on the effectiveness of classification was conducted. Next, a study on the influence of depth and width of fully connected...
-
LEGO bricks for training classification network
Open Research DataThe data set contains images of 447 different classes of LEGO bricks used for training LEGO bricks classification network. The dataset contains two types of images: photos (10%) and renders (90%) aggregated into respective directories. Each directory (photos and renders) contains 447 directories labeled as the official brick type number. The images...
-
Neural network agents trained by declarative programming tutors
PublicationThis paper presents an experimental study on the development of a neural network-based agent, trained using data generated using declarative programming. The focus of the study is the application of various agents to solve the classic logic task – The Wumpus World. The paper evaluates the effectiveness of neural-based agents across different map configurations, offering a comparative analysis to underline the strengths and limitations...
-
Deep neural networks for human pose estimation from a very low resolution depth image
PublicationThe work presented in the paper is dedicated to determining and evaluating the most efficient neural network architecture applied as a multiple regression network localizing human body joints in 3D space based on a single low resolution depth image. The main challenge was to deal with a noisy and coarse representation of the human body, as observed by a depth sensor from a large distance, and to achieve high localization precision....
-
Leveraging Training Strategies of Artificial Neural Network for Classification of Multiday Electromyography Signals
Publication -
Parallel Computations of Text Similarities for Categorization Task
PublicationIn this chapter we describe the approach to parallel implementation of similarities in high dimensional spaces. The similarities computation have been used for textual data categorization. A test datasets we create from Wikipedia articles that with their hyper references formed a graph used in our experiments. The similarities based on Euclidean distance and Cosine measure have been used to process the data using k-means algorithm....
-
Parallel computations in the volunteer based Comcute system
PublicationThe paper presents Comcute which is a novel multi-level implemen- tation of the volunteer based computing paradigm. Comcute was designed to let users donate the computing power of their PCs in a simplified manner, requiring only pointing their web browser at a specific web address and clicking a mouse. The server side appoints several servers to be in charge of execution of particular tasks. Thanks to that the system can survive...
-
Network-aware Data Prefetching Optimization of Computations in a Heterogeneous HPC Framework
PublicationRapid development of diverse computer architectures and hardware accelerators caused that designing parallel systems faces new problems resulting from their heterogeneity. Our implementation of a parallel system called KernelHive allows to efficiently run applications in a heterogeneous environment consisting of multiple collections of nodes with different types of computing devices. The execution engine of the system is open for...
-
Neural Network Subgraphs Correlation with Trained Model Accuracy
PublicationNeural Architecture Search (NAS) is a computationally demanding process of finding optimal neural network architecture for a given task. Conceptually, NAS comprises applying a search strategy on a predefined search space accompanied by a performance evaluation method. The design of search space alone is expected to substantially impact NAS efficiency. We consider neural networks as graphs and find a correlation between the presence...
-
Global Surrogate Modeling by Neural Network-Based Model Uncertainty
PublicationThis work proposes a novel adaptive global surrogate modeling algorithm which uses two neural networks, one for prediction and the other for the model uncertainty. Specifically, the algorithm proceeds in cycles and adaptively enhances the neural network-based surrogate model by selecting the next sampling points guided by an auxiliary neural network approximation of the spatial error. The proposed algorithm is tested numerically...
-
Explainable AI for Inspecting Adversarial Attacks on Deep Neural Networks
PublicationDeep Neural Networks (DNN) are state of the art algorithms for image classification. Although significant achievements and perspectives, deep neural networks and accompanying learning algorithms have some important challenges to tackle. However, it appears that it is relatively easy to attack and fool with well-designed input samples called adversarial examples. Adversarial perturba-tions are unnoticeable for humans. Such attacks...
-
Selection of an artificial pre-training neural network for the classification of inland vessels based on their images
PublicationArtificial neural networks (ANN) are the most commonly used algorithms for image classification problems. An image classifier takes an image or video as input and classifies it into one of the possible categories that it was trained to identify. They are applied in various areas such as security, defense, healthcare, biology, forensics, communication, etc. There is no need to create one’s own ANN because there are several pre-trained...
-
Diagnosing wind turbine condition employing a neural network to the analysis of vibroacoustic signals
PublicationIt is important from the economic point of view to detect damage early in the wind turbines before failures occur. For this purpose, a monitoring device was built that analyzes both acoustic signals acquired from the built-in non-contact acoustic intensity probe, as well as from the accelerometers, mounted on the internal devices in the nacelle. The signals collected in this way are used for long-term training of the autoencoder...
-
Comparison of Deep Neural Network Learning Algorithms for Mars Terrain Image Segmentation
PublicationThis paper is dedicated to the topic of terrain recognition on Mars using advanced techniques based on the convolutional neural networks (CNN). The work on the project was conducted based on the set of 18K images collected by the Curiosity, Opportunity and Spirit rovers. The data were later processed by the model operating in a Python environment, utilizing Keras and Tensorflow repositories. The model benefits from the pretrained...
-
Deep neural networks for data analysis 27/28
e-Learning Courses -
Deep neural networks for data analysis 25/26
e-Learning Courses -
Deep neural networks for data analysis 26/27
e-Learning Courses -
Neural Network World
Journals -
Controlling computer by lip gestures employing neural network
PublicationResults of experiments regarding lip gesture recognition with an artificial neural network are discussed. The neural network module forms the core element of a multimodal human-computer interface called LipMouse. This solution allows a user to work on a computer using lip movements and gestures. A user face is detected in a video stream from a standard web camera using a cascade of boosted classifiers working with Haar-like features....
-
A Simple Neural Network for Collision Detection of Collaborative Robots
PublicationDue to the epidemic threat, more and more companies decide to automate their production lines. Given the lack of adequate security or space, in most cases, such companies cannot use classic production robots. The solution to this problem is the use of collaborative robots (cobots). However, the required equipment (force sensors) or alternative methods of detecting a threat to humans are usually quite expensive. The article presents...