Wyszukiwarka
Filtry
wszystkich: 620
-
Katalog
Wyniki wyszukiwania dla: DEEP NEURAL NETWORK TRAINING BENCHMARKING PARALLEL COMPUTATIONS CAFFE MKL
-
Benchmarking Deep Neural Network Training Using Multi- and Many-Core Processors
PublikacjaIn the paper we provide thorough benchmarking of deep neural network (DNN) training on modern multi- and many-core Intel processors in order to assess performance differences for various deep learning as well as parallel computing parameters. We present performance of DNN training for Alexnet, Googlenet, Googlenet_v2 as well as Resnet_50 for various engines used by the deep learning framework, for various batch sizes. Furthermore,...
-
Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training
PublikacjaIn the paper we tackle bi-objective execution time and power consumption optimization problem concerning execution of parallel applications. We propose using a discrete-event simulation environment for exploring this power/time trade-off in the form of a Pareto front. The solution is verified by a case study based on a real deep neural network training application for automatic speech recognition. A simulation lasting over 2 hours...
-
The impact of the AC922 Architecture on Performance of Deep Neural Network Training
PublikacjaPractical deep learning applications require more and more computing power. New computing architectures emerge, specifically designed for the artificial intelligence applications, including the IBM Power System AC922. In this paper we confront an AC922 (8335-GTG) server equipped with 4 NVIDIA Volta V100 GPUs with selected deep neural network training applications, including four convolutional and one recurrent model. We report...
-
Performance and Energy Aware Training of a Deep Neural Network in a Multi-GPU Environment with Power Capping
PublikacjaIn this paper we demonstrate that it is possible to obtain considerable improvement of performance and energy aware metrics for training of deep neural networks using a modern parallel multi-GPU system, by enforcing selected, non-default power caps on the GPUs. We measure the power and energy consumption of the whole node using a professional, certified hardware power meter. For a high performance workstation with 8 GPUs, we were...
-
Paweł Rościszewski dr inż.
OsobyPaweł Rościszewski received his PhD in Computer Science at Gdańsk University of Technology in 2018 based on PhD thesis entitled: "Optimization of hybrid parallel application execution in heterogeneous high performance computing systems considering execution time and power consumption". Currently, he is an Assistant Professor at the Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, Poland....
-
Resource constrained neural network training
PublikacjaModern applications of neural-network-based AI solutions tend to move from datacenter backends to low-power edge devices. Environmental, computational, and power constraints are inevitable consequences of such a shift. Limiting the bit count of neural network parameters proved to be a valid technique for speeding up and increasing efficiency of the inference process. Hence, it is understandable that a similar approach is gaining...
-
Neural network training with limited precision and asymmetric exponent
PublikacjaAlong with an extremely increasing number of mobile devices, sensors and other smart utilities, an unprecedented growth of data can be observed in today’s world. In order to address multiple challenges facing the big data domain, machine learning techniques are often leveraged for data analysis, filtering and classification. Wide usage of artificial intelligence with large amounts of data creates growing demand not only for storage...
-
A Bayesian regularization-backpropagation neural network model for peeling computations
PublikacjaA Bayesian regularization-backpropagation neural network (BRBPNN) model is employed to predict some aspects of the gecko spatula peeling, viz. the variation of the maximum normal and tangential pull-off forces and the resultant force angle at detachment with the peeling angle. K-fold cross validation is used to improve the effectiveness of the model. The input data is taken from finite element (FE) peeling results. The neural network...
-
Deep neural network architecture search using network morphism
PublikacjaThe paper presents the results of the research on neural architecture search (NAS) algorithm. We utilized the hill climbing algorithm to search for well-performing structures of deep convolutional neural network. Moreover, we used the function preserving transformations which enabled the effective operation of the algorithm in a short period of time. The network obtained with the advantage of NAS was validated on skin lesion classification...
-
Olgun Aydin dr
OsobyOlgun Aydin finished his PhD by publishing a thesis about Deep Neural Networks. He works as a Principal Machine Learning Engineer in Nike, and works as Assistant Professor in Gdansk University of Technology in Poland. Dr. Aydin is part of editorial board of "Journal of Artificial Intelligence and Data Science" Dr. Aydin served as Vice-Chairman of Why R? Foundation and is member of Polish Artificial Intelligence Society. Olgun is...
-
Benchmarking overlapping communication and computations with multiple streams for modern GPUs
PublikacjaThe paper presents benchmarking a multi-stream application processing a set of input data arrays. Tests have been performed and execution times measured for various numbers of streams and various compute intensities measured as the ratio of kernel compute time and data transfer time. As such, the application and benchmarking is representative of frequently used operations such as vector weighted sum, matrix multiplication etc....
-
Neural networks and deep learning
PublikacjaIn this chapter we will provide the general and fundamental background related to Neural Networks and Deep Learning techniques. Specifically, we divide the fundamentals of deep learning in three parts, the first one introduces Deep Feed Forward Networks and the main training algorithms in the context of optimization. The second part covers Convolutional Neural Networks (CNN) and discusses their main advantages and shortcomings...
-
Simulation of parallel similarity measure computations for large data sets
PublikacjaThe paper presents our approach to implementation of similarity measure for big data analysis in a parallel environment. We describe the algorithm for parallelisation of the computations. We provide results from a real MPI application for computations of similarity measures as well as results achieved with our simulation software. The simulation environment allows us to model parallel systems of various sizes with various components...
-
Categorization of emotions in dog behavior based on the deep neural network
PublikacjaThe aim of this article is to present a neural system based on stock architecture for recognizing emotional behavior in dogs. Our considerations are inspired by the original work of Franzoni et al. on recognizing dog emotions. An appropriate set of photographic data has been compiled taking into account five classes of emotional behavior in dogs of one breed, including joy, anger, licking, yawning, and sleeping. Focusing on a particular...
-
Speaker Recognition Using Convolutional Neural Network with Minimal Training Data for Smart Home Solutions
PublikacjaWith the technology advancements in smart home sector, voice control and automation are key components that can make a real difference in people's lives. The voice recognition technology market continues to involve rapidly as almost all smart home devices are providing speaker recognition capability today. However, most of them provide cloud-based solutions or use very deep Neural Networks for speaker recognition task, which are...
-
Minimizing Distribution and Data Loading Overheads in Parallel Training of DNN Acoustic Models with Frequent Parameter Averaging
PublikacjaIn the paper we investigate the performance of parallel deep neural network training with parameter averaging for acoustic modeling in Kaldi, a popular automatic speech recognition toolkit. We describe experiments based on training a recurrent neural network with 4 layers of 800 LSTM hidden states on a 100-hour corpora of annotated Polish speech data. We propose a MPI-based modification of the training program which minimizes the...
-
Deep convolutional neural network for predicting kidney tumour malignancy
PublikacjaPurpose: According to the statistics, up to 15-20% of removed solid kidney tumors turn out to be benign in postoperative histopathological examination, despite having been identified as malignant by a radiologist. The aim of the research was to limit the number of unnecessary nephrectomies of benign tumors. Methods or Background: We propose a machine-aided diagnostic system for kidney...
-
Deep Learning Optimization for Edge Devices: Analysis of Training Quantization Parameters
PublikacjaThis paper focuses on convolution neural network quantization problem. The quantization has a distinct stage of data conversion from floating-point into integer-point numbers. In general, the process of quantization is associated with the reduction of the matrix dimension via limited precision of the numbers. However, the training and inference stages of deep learning neural network are limited by the space of the memory and a...
-
Training of Deep Learning Models Using Synthetic Datasets
PublikacjaIn order to solve increasingly complex problems, the complexity of Deep Neural Networks also needs to be constantly increased, and therefore training such networks requires more and more data. Unfortunately, obtaining such massive real world training data to optimize neural networks parameters is a challenging and time-consuming task. To solve this problem, we propose an easy-touse and general approach to training deep learning...
-
Deep neural networks for data analysis
Kursy OnlineThe aim of the course is to familiarize students with the methods of deep learning for advanced data analysis. Typical areas of application of these types of methods include: image classification, speech recognition and natural language understanding. Celem przedmiotu jest zapoznanie studentów z metodami głębokiego uczenia maszynowego na potrzeby zaawansowanej analizy danych. Do typowych obszarów zastosowań tego typu metod należą:...
-
GPU Power Capping for Energy-Performance Trade-Offs in Training of Deep Convolutional Neural Networks for Image Recognition
PublikacjaIn the paper we present performance-energy trade-off investigation of training Deep Convolutional Neural Networks for image recognition. Several representative and widely adopted network models, such as Alexnet, VGG-19, Inception V3, Inception V4, Resnet50 and Resnet152 were tested using systems with Nvidia Quadro RTX 6000 as well as Nvidia V100 GPUs. Using GPU power capping we found other than default configurations minimizing...
-
Comparative study of methods for artificial neural network training.
PublikacjaPrzedstawiono wyniki badań porównawczych następujących metod uczenia sieci neuronowych: propagacji wstecznej błędów, rekursywnej metody najmniejszych kwadratów, metody Zangwill'a i algorytmów ewolucyjnych. Badania dotyczyły projektowania adaptacyjnego regulatora neuronowego napięcia generatora synchronicznego.
-
Selected Technical Issues of Deep Neural Networks for Image Classification Purposes
PublikacjaIn recent years, deep learning and especially Deep Neural Networks (DNN) have obtained amazing performance on a variety of problems, in particular in classification or pattern recognition. Among many kinds of DNNs, the Convolutional Neural Networks (CNN) are most commonly used. However, due to their complexity, there are many problems related but not limited to optimizing network parameters, avoiding overfitting and ensuring good...
-
Benchmarking Parallel Chess Search in Stockfish on Intel Xeon and Intel Xeon Phi Processors
PublikacjaThe paper presents results from benchmarking the parallel multithreaded Stockfish chess engine on selected multi- and many-core processors. It is shown how the strength of play for an n-thread version compares to 1-thread version on both Intel Xeon and latest Intel Xeon Phi x200 processors. Results such as the number of wins, losses and draws are presented and how these change for growing numbers of threads. Impact of using particular...
-
From Linear Classifier to Convolutional Neural Network for Hand Pose Recognition
PublikacjaRecently gathered image datasets and the new capabilities of high-performance computing systems have allowed developing new artificial neural network models and training algorithms. Using the new machine learning models, computer vision tasks can be accomplished based on the raw values of image pixels instead of specific features. The principle of operation of deep neural networks resembles more and more what we believe to be happening...
-
Dataset Related Experimental Investigation of Chess Position Evaluation Using a Deep Neural Network
PublikacjaThe idea of training Articial Neural Networks to evaluate chess positions has been widely explored in the last ten years. In this paper we investigated dataset impact on chess position evaluation. We created two datasets with over 1.6 million unique chess positions each. In one of those we also included randomly generated positions resulting from consideration of potentially unpredictable chess moves. Each position was evaluated...
-
An Improved Convolutional Neural Network for Steganalysis in the Scenario of Reuse of the Stego-Key
PublikacjaThe topic of this paper is the use of deep learning techniques, more specifically convolutional neural networks, for steganalysis of digital images. The steganalysis scenario of the repeated use of the stego-key is considered. Firstly, a study of the influence of the depth and width of the convolution layers on the effectiveness of classification was conducted. Next, a study on the influence of depth and width of fully connected...
-
LEGO bricks for training classification network
Dane BadawczeThe data set contains images of 447 different classes of LEGO bricks used for training LEGO bricks classification network. The dataset contains two types of images: photos (10%) and renders (90%) aggregated into respective directories. Each directory (photos and renders) contains 447 directories labeled as the official brick type number. The images...
-
Creating neural models using an adaptive algorithm for optimal size of neural network and training set.
PublikacjaZaprezentowano adaptacyjny algorytm generujący modele neuronowe liniowych układów mikrofalowych, zdolny do oszacowania optymalnego rozmiaru zbiory uczącego i sieci neuronowej. Stworzono kilka modeli nieciągłości falowodowych i mokropaskowych, a następnie zweryfikowano ich poprawność porównując wyniki analiz metodą dopasowania rodzajów i metodą momentów filtrów pasmowo-przepustowych.
-
Deep neural networks for human pose estimation from a very low resolution depth image
PublikacjaThe work presented in the paper is dedicated to determining and evaluating the most efficient neural network architecture applied as a multiple regression network localizing human body joints in 3D space based on a single low resolution depth image. The main challenge was to deal with a noisy and coarse representation of the human body, as observed by a depth sensor from a large distance, and to achieve high localization precision....
-
Leveraging Training Strategies of Artificial Neural Network for Classification of Multiday Electromyography Signals
Publikacja -
Parallel Computations of Text Similarities for Categorization Task
PublikacjaIn this chapter we describe the approach to parallel implementation of similarities in high dimensional spaces. The similarities computation have been used for textual data categorization. A test datasets we create from Wikipedia articles that with their hyper references formed a graph used in our experiments. The similarities based on Euclidean distance and Cosine measure have been used to process the data using k-means algorithm....
-
Parallel computations in the volunteer based Comcute system
PublikacjaThe paper presents Comcute which is a novel multi-level implemen- tation of the volunteer based computing paradigm. Comcute was designed to let users donate the computing power of their PCs in a simplified manner, requiring only pointing their web browser at a specific web address and clicking a mouse. The server side appoints several servers to be in charge of execution of particular tasks. Thanks to that the system can survive...
-
Explainable AI for Inspecting Adversarial Attacks on Deep Neural Networks
PublikacjaDeep Neural Networks (DNN) are state of the art algorithms for image classification. Although significant achievements and perspectives, deep neural networks and accompanying learning algorithms have some important challenges to tackle. However, it appears that it is relatively easy to attack and fool with well-designed input samples called adversarial examples. Adversarial perturba-tions are unnoticeable for humans. Such attacks...
-
Global Surrogate Modeling by Neural Network-Based Model Uncertainty
PublikacjaThis work proposes a novel adaptive global surrogate modeling algorithm which uses two neural networks, one for prediction and the other for the model uncertainty. Specifically, the algorithm proceeds in cycles and adaptively enhances the neural network-based surrogate model by selecting the next sampling points guided by an auxiliary neural network approximation of the spatial error. The proposed algorithm is tested numerically...
-
Neural Network Subgraphs Correlation with Trained Model Accuracy
PublikacjaNeural Architecture Search (NAS) is a computationally demanding process of finding optimal neural network architecture for a given task. Conceptually, NAS comprises applying a search strategy on a predefined search space accompanied by a performance evaluation method. The design of search space alone is expected to substantially impact NAS efficiency. We consider neural networks as graphs and find a correlation between the presence...
-
Network-aware Data Prefetching Optimization of Computations in a Heterogeneous HPC Framework
PublikacjaRapid development of diverse computer architectures and hardware accelerators caused that designing parallel systems faces new problems resulting from their heterogeneity. Our implementation of a parallel system called KernelHive allows to efficiently run applications in a heterogeneous environment consisting of multiple collections of nodes with different types of computing devices. The execution engine of the system is open for...
-
Selection of an artificial pre-training neural network for the classification of inland vessels based on their images
PublikacjaArtificial neural networks (ANN) are the most commonly used algorithms for image classification problems. An image classifier takes an image or video as input and classifies it into one of the possible categories that it was trained to identify. They are applied in various areas such as security, defense, healthcare, biology, forensics, communication, etc. There is no need to create one’s own ANN because there are several pre-trained...
-
Diagnosing wind turbine condition employing a neural network to the analysis of vibroacoustic signals
PublikacjaIt is important from the economic point of view to detect damage early in the wind turbines before failures occur. For this purpose, a monitoring device was built that analyzes both acoustic signals acquired from the built-in non-contact acoustic intensity probe, as well as from the accelerometers, mounted on the internal devices in the nacelle. The signals collected in this way are used for long-term training of the autoencoder...
-
An Intelligent Approach to Short-Term Wind Power Prediction Using Deep Neural Networks
PublikacjaIn this paper, an intelligent approach to the Short-Term Wind Power Prediction (STWPP) problem is considered, with the use of various types of Deep Neural Networks (DNNs). The impact of the prediction time horizon length on accuracy, and the influence of temperature on prediction effectiveness have been analyzed. Three types of DNNs have been implemented and tested, including: CNN (Convolutional Neural Networks), GRU (Gated Recurrent...
-
Outlier detection method by using deep neural networks
PublikacjaDetecting outliers in the data set is quite important for building effective predictive models. Consistent prediction can not be made through models created with data sets containing outliers, or robust models can not be created. In such cases, it may be possible to exclude observations that are determined to be outlier from the data set, or to assign less weight to these points of observation than to other points of observation....
-
Neural Network World
Czasopisma -
Controlling computer by lip gestures employing neural network
PublikacjaResults of experiments regarding lip gesture recognition with an artificial neural network are discussed. The neural network module forms the core element of a multimodal human-computer interface called LipMouse. This solution allows a user to work on a computer using lip movements and gestures. A user face is detected in a video stream from a standard web camera using a cascade of boosted classifiers working with Haar-like features....
-
A Simple Neural Network for Collision Detection of Collaborative Robots
PublikacjaDue to the epidemic threat, more and more companies decide to automate their production lines. Given the lack of adequate security or space, in most cases, such companies cannot use classic production robots. The solution to this problem is the use of collaborative robots (cobots). However, the required equipment (force sensors) or alternative methods of detecting a threat to humans are usually quite expensive. The article presents...
-
A Comprehensive Analysis of Deep Neural-Based Cerebral Microbleeds Detection System
PublikacjaMachine learning-based systems are gaining interest in the field of medicine, mostly in medical imaging and diagnosis. In this paper, we address the problem of automatic cerebral microbleeds (CMB) detection in magnetic resonance images. It is challenging due to difficulty in distinguishing a true CMB from its mimics, however, if successfully solved it would streamline the radiologists work. To deal with this complex three-dimensional...
-
Deep neural networks approach to skin lesions classification — A comparative analysis
PublikacjaThe paper presents the results of research on the use of Deep Neural Networks (DNN) for automatic classification of the skin lesions. The authors have focused on the most effective kind of DNNs for image processing, namely Convolutional Neural Networks (CNN). In particular, three kinds of CNN were analyzed: VGG19, Residual Networks (ResNet) and the hybrid of VGG19 CNN with the Support Vector Machine (SVM). The research was carried...
-
Intelligent turbogenerator controller based on artifical neural network
PublikacjaThe paper presents a desing of an intelligent controller based on neural network (ICNN). The ICNN ensures at the same time two fundamental functions : the maintaining of generator voltage at the desired value and the damping of the electromechanical oscillations. Its performance is evaluted on a single machine infinite bus power system through computer simulations. The dynamic and transient operation of the proposed controller...
-
Electromagnetic Modeling of Microstrip Elements Aided with Artificial Neural Network
PublikacjaThe electromagnetic modeling principle aided withartificial neural network to designing the microwave widebandelements/networks prepared in microstrip technology is proposedin the paper. It is assumed that the complete information is knownfor the prototype design which is prepared on certain substratewith certain thickness and electric permittivity. The longitudinaland transversal dimensions of new design...
-
Using GPUs for Parallel Stencil Computations in Relativistic Hydrodynamic Simulation
PublikacjaThis paper explores the possibilities of using a GPU for complex 3D finite difference computation. We propose a new approach to this topic using surface memory and compare it with 3D stencil computations carried out via shared memory, which is currently considered to be the best approach. The case study was performed for the extensive computation of collisions between heavy nuclei in terms of relativistic hydrodynamics.
-
Evaluation of Facial Pulse Signals Using Deep Neural Net Models
PublikacjaThe reliable measurement of the pulse rate using remote photoplethysmography (PPG) is very important for many medical applications. In this paper we present how deep neural networks (DNNs) models can be used in the problem of PPG signal classification and pulse rate estimation. In particular, we show that the DNN-based classification results correspond to parameters describing the PPG signals (e.g. peak energy in the frequency...