Search
Search results for: deep neural network training
-
The impact of the AC922 Architecture on Performance of Deep Neural Network Training
PublicationPractical deep learning applications require more and more computing power. New computing architectures emerge, specifically designed for the artificial intelligence applications, including the IBM Power System AC922. In this paper we confront an AC922 (8335-GTG) server equipped with 4 NVIDIA Volta V100 GPUs with selected deep neural network training applications, including four convolutional and one recurrent model. We report...
-
Benchmarking Deep Neural Network Training Using Multi- and Many-Core Processors
PublicationIn the paper we provide thorough benchmarking of deep neural network (DNN) training on modern multi- and many-core Intel processors in order to assess performance differences for various deep learning as well as parallel computing parameters. We present performance of DNN training for Alexnet, Googlenet, Googlenet_v2 as well as Resnet_50 for various engines used by the deep learning framework, for various batch sizes. Furthermore,...
-
Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training
PublicationIn the paper we tackle bi-objective execution time and power consumption optimization problem concerning execution of parallel applications. We propose using a discrete-event simulation environment for exploring this power/time trade-off in the form of a Pareto front. The solution is verified by a case study based on a real deep neural network training application for automatic speech recognition. A simulation lasting over 2 hours...
-
Resource constrained neural network training
PublicationModern applications of neural-network-based AI solutions tend to move from datacenter backends to low-power edge devices. Environmental, computational, and power constraints are inevitable consequences of such a shift. Limiting the bit count of neural network parameters proved to be a valid technique for speeding up and increasing efficiency of the inference process. Hence, it is understandable that a similar approach is gaining...
-
Performance and Energy Aware Training of a Deep Neural Network in a Multi-GPU Environment with Power Capping
PublicationIn this paper we demonstrate that it is possible to obtain considerable improvement of performance and energy aware metrics for training of deep neural networks using a modern parallel multi-GPU system, by enforcing selected, non-default power caps on the GPUs. We measure the power and energy consumption of the whole node using a professional, certified hardware power meter. For a high performance workstation with 8 GPUs, we were...
-
Neural network training with limited precision and asymmetric exponent
PublicationAlong with an extremely increasing number of mobile devices, sensors and other smart utilities, an unprecedented growth of data can be observed in today’s world. In order to address multiple challenges facing the big data domain, machine learning techniques are often leveraged for data analysis, filtering and classification. Wide usage of artificial intelligence with large amounts of data creates growing demand not only for storage...
-
Comparative study of methods for artificial neural network training.
PublicationPrzedstawiono wyniki badań porównawczych następujących metod uczenia sieci neuronowych: propagacji wstecznej błędów, rekursywnej metody najmniejszych kwadratów, metody Zangwill'a i algorytmów ewolucyjnych. Badania dotyczyły projektowania adaptacyjnego regulatora neuronowego napięcia generatora synchronicznego.
-
Deep neural network architecture search using network morphism
PublicationThe paper presents the results of the research on neural architecture search (NAS) algorithm. We utilized the hill climbing algorithm to search for well-performing structures of deep convolutional neural network. Moreover, we used the function preserving transformations which enabled the effective operation of the algorithm in a short period of time. The network obtained with the advantage of NAS was validated on skin lesion classification...
-
Creating neural models using an adaptive algorithm for optimal size of neural network and training set.
PublicationZaprezentowano adaptacyjny algorytm generujący modele neuronowe liniowych układów mikrofalowych, zdolny do oszacowania optymalnego rozmiaru zbiory uczącego i sieci neuronowej. Stworzono kilka modeli nieciągłości falowodowych i mokropaskowych, a następnie zweryfikowano ich poprawność porównując wyniki analiz metodą dopasowania rodzajów i metodą momentów filtrów pasmowo-przepustowych.
-
Categorization of emotions in dog behavior based on the deep neural network
PublicationThe aim of this article is to present a neural system based on stock architecture for recognizing emotional behavior in dogs. Our considerations are inspired by the original work of Franzoni et al. on recognizing dog emotions. An appropriate set of photographic data has been compiled taking into account five classes of emotional behavior in dogs of one breed, including joy, anger, licking, yawning, and sleeping. Focusing on a particular...
-
Deep convolutional neural network for predicting kidney tumour malignancy
PublicationPurpose: According to the statistics, up to 15-20% of removed solid kidney tumors turn out to be benign in postoperative histopathological examination, despite having been identified as malignant by a radiologist. The aim of the research was to limit the number of unnecessary nephrectomies of benign tumors. Methods or Background: We propose a machine-aided diagnostic system for kidney...
-
Leveraging Training Strategies of Artificial Neural Network for Classification of Multiday Electromyography Signals
Publication -
Selection of an artificial pre-training neural network for the classification of inland vessels based on their images
PublicationArtificial neural networks (ANN) are the most commonly used algorithms for image classification problems. An image classifier takes an image or video as input and classifies it into one of the possible categories that it was trained to identify. They are applied in various areas such as security, defense, healthcare, biology, forensics, communication, etc. There is no need to create one’s own ANN because there are several pre-trained...
-
Speaker Recognition Using Convolutional Neural Network with Minimal Training Data for Smart Home Solutions
PublicationWith the technology advancements in smart home sector, voice control and automation are key components that can make a real difference in people's lives. The voice recognition technology market continues to involve rapidly as almost all smart home devices are providing speaker recognition capability today. However, most of them provide cloud-based solutions or use very deep Neural Networks for speaker recognition task, which are...
-
Dataset Related Experimental Investigation of Chess Position Evaluation Using a Deep Neural Network
PublicationThe idea of training Articial Neural Networks to evaluate chess positions has been widely explored in the last ten years. In this paper we investigated dataset impact on chess position evaluation. We created two datasets with over 1.6 million unique chess positions each. In one of those we also included randomly generated positions resulting from consideration of potentially unpredictable chess moves. Each position was evaluated...
-
GPU Power Capping for Energy-Performance Trade-Offs in Training of Deep Convolutional Neural Networks for Image Recognition
PublicationIn the paper we present performance-energy trade-off investigation of training Deep Convolutional Neural Networks for image recognition. Several representative and widely adopted network models, such as Alexnet, VGG-19, Inception V3, Inception V4, Resnet50 and Resnet152 were tested using systems with Nvidia Quadro RTX 6000 as well as Nvidia V100 GPUs. Using GPU power capping we found other than default configurations minimizing...
-
Using Deep Neural Network Methods for Forecasting Energy Productivity Based on Comparison of Simulation and DNN Results for Central Poland—Swietokrzyskie Voivodeship
Publication -
Using Deep Neural Network Methods for Forecasting Energy Productivity Based on Comparison of Simulation and DNN Results for Central Poland – Swietokrzyskie Voivodeship
Publication -
Zastosowanie algorytmu ewolucyjnego do uczenia neuronowego regulatora napięcia generatora synchronicznego. Evolutionary algorithm for training a neural network of synchronous generator voltage controller
PublicationNajpopularniejsza metoda uczenia wielowarstwowych sieci neuronowych -metoda wstecznej propagacji błędu - charakteryzuje się słabą efektywnością. Z tego względu podejmowane są próby stosowania innych metod do uczenia sieci. W pracy przedstawiono wyniki uczenia sieci realizującej regulator neuronowy, za pomocą algorytmu ewolucyjnego. Obliczenia symulacyjne potwierdziły dobrą zbieżność algorytmu ewolucyjnego w tym zastosowaniu.
-
Neural networks and deep learning
PublicationIn this chapter we will provide the general and fundamental background related to Neural Networks and Deep Learning techniques. Specifically, we divide the fundamentals of deep learning in three parts, the first one introduces Deep Feed Forward Networks and the main training algorithms in the context of optimization. The second part covers Convolutional Neural Networks (CNN) and discusses their main advantages and shortcomings...
-
Deep Learning Optimization for Edge Devices: Analysis of Training Quantization Parameters
PublicationThis paper focuses on convolution neural network quantization problem. The quantization has a distinct stage of data conversion from floating-point into integer-point numbers. In general, the process of quantization is associated with the reduction of the matrix dimension via limited precision of the numbers. However, the training and inference stages of deep learning neural network are limited by the space of the memory and a...
-
Training of Deep Learning Models Using Synthetic Datasets
PublicationIn order to solve increasingly complex problems, the complexity of Deep Neural Networks also needs to be constantly increased, and therefore training such networks requires more and more data. Unfortunately, obtaining such massive real world training data to optimize neural networks parameters is a challenging and time-consuming task. To solve this problem, we propose an easy-touse and general approach to training deep learning...
-
From Linear Classifier to Convolutional Neural Network for Hand Pose Recognition
PublicationRecently gathered image datasets and the new capabilities of high-performance computing systems have allowed developing new artificial neural network models and training algorithms. Using the new machine learning models, computer vision tasks can be accomplished based on the raw values of image pixels instead of specific features. The principle of operation of deep neural networks resembles more and more what we believe to be happening...
-
Selected Technical Issues of Deep Neural Networks for Image Classification Purposes
PublicationIn recent years, deep learning and especially Deep Neural Networks (DNN) have obtained amazing performance on a variety of problems, in particular in classification or pattern recognition. Among many kinds of DNNs, the Convolutional Neural Networks (CNN) are most commonly used. However, due to their complexity, there are many problems related but not limited to optimizing network parameters, avoiding overfitting and ensuring good...
-
Deep neural networks for human pose estimation from a very low resolution depth image
PublicationThe work presented in the paper is dedicated to determining and evaluating the most efficient neural network architecture applied as a multiple regression network localizing human body joints in 3D space based on a single low resolution depth image. The main challenge was to deal with a noisy and coarse representation of the human body, as observed by a depth sensor from a large distance, and to achieve high localization precision....
-
An Improved Convolutional Neural Network for Steganalysis in the Scenario of Reuse of the Stego-Key
PublicationThe topic of this paper is the use of deep learning techniques, more specifically convolutional neural networks, for steganalysis of digital images. The steganalysis scenario of the repeated use of the stego-key is considered. Firstly, a study of the influence of the depth and width of the convolution layers on the effectiveness of classification was conducted. Next, a study on the influence of depth and width of fully connected...
-
Minimizing Distribution and Data Loading Overheads in Parallel Training of DNN Acoustic Models with Frequent Parameter Averaging
PublicationIn the paper we investigate the performance of parallel deep neural network training with parameter averaging for acoustic modeling in Kaldi, a popular automatic speech recognition toolkit. We describe experiments based on training a recurrent neural network with 4 layers of 800 LSTM hidden states on a 100-hour corpora of annotated Polish speech data. We propose a MPI-based modification of the training program which minimizes the...
-
A Comprehensive Analysis of Deep Neural-Based Cerebral Microbleeds Detection System
PublicationMachine learning-based systems are gaining interest in the field of medicine, mostly in medical imaging and diagnosis. In this paper, we address the problem of automatic cerebral microbleeds (CMB) detection in magnetic resonance images. It is challenging due to difficulty in distinguishing a true CMB from its mimics, however, if successfully solved it would streamline the radiologists work. To deal with this complex three-dimensional...
-
Paweł Rościszewski dr inż.
PeoplePaweł Rościszewski received his PhD in Computer Science at Gdańsk University of Technology in 2018 based on PhD thesis entitled: "Optimization of hybrid parallel application execution in heterogeneous high performance computing systems considering execution time and power consumption". Currently, he is an Assistant Professor at the Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, Poland....
-
Data augmentation for improving deep learning in image classification problem
PublicationThese days deep learning is the fastest-growing field in the field of Machine Learning (ML) and Deep Neural Networks (DNN). Among many of DNN structures, the Convolutional Neural Networks (CNN) are currently the main tool used for the image analysis and classification purposes. Although great achievements and perspectives, deep neural networks and accompanying learning algorithms have some relevant challenges to tackle. In this...
-
Musical Instrument Identification Using Deep Learning Approach
PublicationThe work aims to propose a novel approach for automatically identifying all instruments present in an audio excerpt using sets of individual convolutional neural networks (CNNs) per tested instrument. The paper starts with a review of tasks related to musical instrument identification. It focuses on tasks performed, input type, algorithms employed, and metrics used. The paper starts with the background presentation, i.e., metadata...
-
Self-Supervised Learning to Increase the Performance of Skin Lesion Classification
PublicationTo successfully train a deep neural network, a large amount of human-labeled data is required. Unfortunately, in many areas, collecting and labeling data is a difficult and tedious task. Several ways have been developed to mitigate the problem associated with the shortage of data, the most common of which is transfer learning. However, in many cases, the use of transfer learning as the only remedy is insufficient. In this study,...
-
Accurate Modeling of Antenna Structures by Means of Domain Confinement and Pyramidal Deep Neural Networks
PublicationThe importance of surrogate modeling techniques has been gradually increasing in the design of antenna structures over the recent years. Perhaps the most important reason is a high cost of full-wave electromagnetic (EM) analysis of antenna systems. Although imperative in ensuring evaluation reliability, it entails considerable computational expenses. These are especially pronounced when carrying out EM-driven design tasks such...
-
Optimized Deep Learning Model for Flood Detection Using Satellite Images
PublicationThe increasing amount of rain produces a number of issues in Kerala, particularly in urban regions where the drainage system is frequently unable to handle a significant amount of water in such a short duration. Meanwhile, standard flood detection results are inaccurate for complex phenomena and cannot handle enormous quantities of data. In order to overcome those drawbacks and enhance the outcomes of conventional flood detection...
-
Tagged images with bees
Open Research DataImages taken from bee hive with tagged bees. The images are prepared for training yolo5 deep neural network (supplied with the data).
-
How to Sort Them? A Network for LEGO Bricks Classification
PublicationLEGO bricks are highly popular due to the ability to build almost any type of creation. This is possible thanks to availability of multiple shapes and colors of the bricks. For the smooth build process the bricks need to properly sorted and arranged. In our work we aim at creating an automated LEGO bricks sorter. With over 3700 different LEGO parts bricks classification has to be done with deep neural networks. The question arises...
-
MP3vec: A Reusable Machine-Constructed Feature Representation for Protein Sequences
Publication—Machine Learning (ML) methods have been used with varying degrees of success on protein prediction tasks, with two inherent limitations. First, prediction performance often depends upon the features extracted from the proteins. Second, experimental data may be insufficient to construct reliable ML models. Here we introduce MP3vec, a transferable representation for protein sequences that is designed to be used specifically for sequence-to-sequence...
-
Poprawa jakości klasyfikacji głębokich sieci neuronowych poprzez optymalizację ich struktury i dwuetapowy proces uczenia
PublicationW pracy doktorskiej podjęto problem realizacji algorytmów głębokiego uczenia w warunkach deficytu danych uczących. Głównym celem było opracowanie podejścia optymalizującego strukturę sieci neuronowej oraz zastosowanie uczeniu dwuetapowym, w celu uzyskania mniejszych struktur, zachowując przy tym dokładności. Proponowane rozwiązania poddano testom na zadaniu klasyfikacji znamion skórnych na znamiona złośliwe i łagodne. W pierwszym...
-
Method for Clustering of Brain Activity Data Derived from EEG Signals
PublicationA method for assessing separability of EEG signals associated with three classes of brain activity is proposed. The EEG signals are acquired from 23 subjects, gathered from a headset consisting of 14 electrodes. Data are processed by applying Discrete Wavelet Transform (DWT) for the signal analysis and an autoencoder neural network for the brain activity separation. Processing involves 74 wavelets from 3 DWT families: Coiflets,...
-
Assessing the attractiveness of human face based on machine learning
PublicationThe attractiveness of the face plays an important role in everyday life, especially in the modern world where social media and the Internet surround us. In this study, an attempt to assess the attractiveness of a face by machine learning is shown. Attractiveness is determined by three deep models whose sum of predictions is the final score. Two annotated datasets available in the literature are employed for training and testing...
-
Buried Object Characterization Using Ground Penetrating Radar Assisted by Data-Driven Surrogate-Models
PublicationThis work addresses artificial-intelligence-based buried object characterization using 3-D full-wave electromagnetic simulations of a ground penetrating radar (GPR). The task is to characterize cylindrical shape, perfectly electric conductor (PEC) object buried in various dispersive soil media, and in different positions. The main contributions of this work are (i) development of a fast and accurate data driven surrogate modeling...
-
BP-EVD: Forward Block-Output Propagation for Efficient Video Denoising
PublicationDenoising videos in real-time is critical in many applications, including robotics and medicine, where varying light conditions, miniaturized sensors, and optics can substantially compromise image quality. This work proposes the first video denoising method based on a deep neural network that achieves state-of-the-art performance on dynamic scenes while running in real-time on VGA video resolution with no frame latency. The backbone...
-
A Bayesian regularization-backpropagation neural network model for peeling computations
PublicationA Bayesian regularization-backpropagation neural network (BRBPNN) model is employed to predict some aspects of the gecko spatula peeling, viz. the variation of the maximum normal and tangential pull-off forces and the resultant force angle at detachment with the peeling angle. K-fold cross validation is used to improve the effectiveness of the model. The input data is taken from finite element (FE) peeling results. The neural network...
-
Musical Instrument Tagging Using Data Augmentation and Effective Noisy Data Processing
PublicationDeveloping signal processing methods to extract information automatically has potential in several applications, for example searching for multimedia based on its audio content, making context-aware mobile applications (e.g., tuning apps), or pre-processing for an automatic mixing system. However, the last-mentioned application needs a significant amount of research to reliably recognize real musical instruments in recordings....
-
Diagnosing wind turbine condition employing a neural network to the analysis of vibroacoustic signals
PublicationIt is important from the economic point of view to detect damage early in the wind turbines before failures occur. For this purpose, a monitoring device was built that analyzes both acoustic signals acquired from the built-in non-contact acoustic intensity probe, as well as from the accelerometers, mounted on the internal devices in the nacelle. The signals collected in this way are used for long-term training of the autoencoder...
-
Performance improvement of NN based RTLS by customization of NN structure - heuristic approach
PublicationThe purpose of this research is to improve performance of the Hybrid Scene Analysis – Neural Network indoor localization algorithm applied in Real-time Locating System, RTLS. A properly customized structure of Neural Network and training algorithms for specific operating environment will enhance the system’s performance in terms of localization accuracy and precision. Due to nonlinearity and model complexity, a heuristic analysis...
-
An Intelligent Approach to Short-Term Wind Power Prediction Using Deep Neural Networks
PublicationIn this paper, an intelligent approach to the Short-Term Wind Power Prediction (STWPP) problem is considered, with the use of various types of Deep Neural Networks (DNNs). The impact of the prediction time horizon length on accuracy, and the influence of temperature on prediction effectiveness have been analyzed. Three types of DNNs have been implemented and tested, including: CNN (Convolutional Neural Networks), GRU (Gated Recurrent...
-
Deep Learning Basics 2023/24
e-Learning CoursesA course about the basics of deep learning intended for students of Computer Science. It includes an introduction to supervised machine learning, the architecture of basic artificial neural networks and their training algorithms, as well as more advanced architectures (convolutional networks, recurrent networks, transformers) and regularization and optimization techniques.
-
Employing Subjective Tests and Deep Learning for Discovering the Relationship between Personality Types and Preferred Music Genres
PublicationThe purpose of this research is two-fold: (a) to explore the relationship between the listeners’ personality trait, i.e., extraverts and introverts and their preferred music genres, and (b) to predict the personality trait of potential listeners on the basis of a musical excerpt by employing several classification algorithms. We assume that this may help match songs according to the listener’s personality in social music networks....
-
Feature Weighted Attention-Bidirectional Long Short Term Memory Model for Change Detection in Remote Sensing Images
PublicationIn remote sensing images, change detection (CD) is required in many applications, such as: resource management, urban expansion research, land management, and disaster assessment. Various deep learning-based methods were applied to satellite image analysis for change detection, yet many of them have limitations, including the overfitting problem. This research proposes the Feature Weighted Attention (FWA) in Bidirectional Long...