Search
Department of Multimedia Systems
Publications
Filters
total: 908
Catalog Publications
Year 2019
-
Automatic labeling of traffic sound recordings using autoencoder-derived features
PublicationAn approach to detection of events occurring in road traffic using autoencoders is presented. Extensions of existing algorithms of acoustic road events detection employing Mel Frequency Cepstral Coefficients combined with classifiers based on k nearest neighbors, Support Vector Machines, and random forests are used. In our research, the acoustic signal gathered from the microphone placed near the road is split into frames and converted...
-
Combining Road Network Data from OpenStreetMap with an Authoritative Database
PublicationComputer modeling of road networks requires detailed and up-to-date dataset. This paper proposes a method of combining authoritative databases with OpenStreetMap (OSM) system. The complete route is established by finding paths in the graph constructed from partial data obtained from OSM. In order to correlate data from both sources, a method of coordinate conversion is proposed. The algorithm queries road data from OSM and provides...
-
Comparative study on the effectiveness of various types of road traffic intensity detectors
PublicationVehicle detection and speed measurements are crucial tasks in traffic monitoring systems. In this work, we focus on several types of electronic sensors, operating on different physical principles in order to compare their effectiveness in real traffic conditions. Commercial solutions are based on road tubes, microwave sensors, LiDARs, and video cameras. Distributed traffic monitoring systems require a high number of monitoring...
-
Comparison of Lithuanian and Polish Consonant Phonemes Based on Acoustic Analysis – Preliminary Results
PublicationThe goal of this research is to find a set of acoustic parameters that are related to differences between Polish and Lithuanian language consonants. In order to identify these differences, an acoustic analysis is performed, and the phoneme sounds are described as the vectors of acoustic parameters. Parameters known from the speech domain as well as those from the music information retrieval area are employed. These parameters are...
-
Comparison of the effectiveness of automatic EEG signal class separation algorithms
PublicationIn this paper, an algorithm for automatic brain activity class identification of EEG (electroencephalographic) signals is presented. EEG signals are gathered from seventeen subjects performing one of the three tasks: resting, watching a music video and playing a simple logic game. The methodology applied consists of several steps, namely: signal acquisition, signal processing utilizing z-score normalization, parametrization and...
-
Database of speech and facial expressions recorded with optimized face motion capture settings
PublicationThe broad objective of the present research is the analysis of spoken English employing a multiplicity of modalities. An important stage of this process, discussed in the paper, is creating a database of speech accompanied with facial expressions. Recordings of speakers were made using an advanced system for capturing facial muscle motion. A brief historical outline, current applications, limitations and the ways of capturing face...
-
Deep neural networks for human pose estimation from a very low resolution depth image
PublicationThe work presented in the paper is dedicated to determining and evaluating the most efficient neural network architecture applied as a multiple regression network localizing human body joints in 3D space based on a single low resolution depth image. The main challenge was to deal with a noisy and coarse representation of the human body, as observed by a depth sensor from a large distance, and to achieve high localization precision....
-
Development of Intelligent Road Signs with V2X Interface for Adaptive Traffic Controlling
PublicationThe objective of this paper is to present a practical project of intelligent road signs, under which a series of new products for the regulation of traffic is being created. The engineering part of the project, described in this paper, was preceded by a series of experimental studies, the results of which were described in another paper accepted for publication at the MTS-ITS conference 2019, entitled "Comparative study on the effectiveness...
-
Diagnosing wind turbine condition employing a neural network to the analysis of vibroacoustic signals
PublicationIt is important from the economic point of view to detect damage early in the wind turbines before failures occur. For this purpose, a monitoring device was built that analyzes both acoustic signals acquired from the built-in non-contact acoustic intensity probe, as well as from the accelerometers, mounted on the internal devices in the nacelle. The signals collected in this way are used for long-term training of the autoencoder...
-
Discovering Rule-Based Learning Systems for the Purpose of Music Analysis
PublicationMusic analysis and processing aims at understanding information retrieved from music (Music Information Retrieval). For the purpose of music data mining, machine learning (ML) methods or statistical approach are employed. Their primary task is recognition of musical instrument sounds, music genre or emotion contained in music, identification of audio, assessment of audio content, etc. In terms of computational approach, music databases...
-
Estimating Traffic Intensity Employing Passive Acoustic Radar and Enhanced Microwave Doppler Radar Sensor
PublicationInnovative road signs that can autonomously display the speed limit in cases where the trac situation requires it are under development. The autonomous road sign contains many types of sensors, of which the subject of interest in this article is the Doppler sensor that we have improved and the constructed and calibrated acoustic probe. An algorithm for performing vehicle detection and tracking, as well as vehicle speed measurement,...
-
Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries
PublicationOne of the first clinical signs differentiating the minimally conscious state from the vegetative state is the presence of smooth pursuit eye movements occurring in direct response to moving salient stimuli. Glasgow Coma Scale (GCS) is one of the most commonly used diagnostic tools for acute phase assessment of the level of consciousness, together with a neurological examination. These classic measures are limited to qualitative...
-
Human verbal memory encoding is hierarchically distributed in a continuous processing stream
PublicationProcessing of memory is supported by coordinated activity in a network of sensory, association, and motor brain regions. It remains a major challenge to determine where memory is encoded for later retrieval. Here we used direct intracranial brain recordings from epilepsy patients performing free recall tasks to determine the temporal pattern and anatomical distribution of verbal memory encoding across the entire human cortex. High...
-
Influence of the Delay in Monitor System on the Motor Coordination of Musicians while Performing
PublicationThis paper provides a description and results of measurements of the maximum acceptable value of delay tolerated by a musician, while playing an instrument, that does not cause de-synchronization and discomfort. First, methodology of measurements comprising audio recording and a fast camera is described. Then, themeasurement procedure for acquiring the maximum value of delay conditioning...
-
Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech
PublicationWe present a novel deep learning model for the detection and reconstruction of dysarthric speech. We train the model with a multi-task learning technique to jointly solve dysarthria detection and speech reconstruction tasks. The model key feature is a low-dimensional latent space that is meant to encode the properties of dysarthric speech. It is commonly believed that neural networks are black boxes that solve problems but do not...
-
Labeler-hot Detection of EEG Epileptic Transients
PublicationPreventing early progression of epilepsy and sothe severity of seizures requires effective diagnosis. Epileptictransients indicate the ability to develop seizures but humansoverlook such brief events in an electroencephalogram (EEG)what compromises patient treatment. Traditionally, trainingof the EEG event detection algorithms has relied on groundtruth labels, obtained from the consensus...
-
Localization of sound sources with dual acoustic vector sensor
PublicationThe aim of the work is to estimate the position of sound sources. The proposed method uses a setup of two acoustic vector sensors (AVS). The intersection of azimuth rays from each AVS should indicate the position of a source. In practice, the result of position estimation using this method is an area rather than a point. This is a result of inaccuracy of the individual sensors, but more importantly, of the influence of a source...
-
MACHINE LEARNING–BASED ANALYSIS OF ENGLISH LATERAL ALLOPHONES
PublicationAutomatic classification methods, such as artificial neural networks (ANNs), the k-nearest neighbor (kNN) and selforganizing maps (SOMs), are applied to allophone analysis based on recorded speech. A list of 650 words was created for that purpose, containing positionally and/or contextually conditioned allophones. For each word, a group of 16 native and non-native speakers were audio-video recorded, from which seven native speakers’...
-
Method for Clustering of Brain Activity Data Derived from EEG Signals
PublicationA method for assessing separability of EEG signals associated with three classes of brain activity is proposed. The EEG signals are acquired from 23 subjects, gathered from a headset consisting of 14 electrodes. Data are processed by applying Discrete Wavelet Transform (DWT) for the signal analysis and an autoencoder neural network for the brain activity separation. Processing involves 74 wavelets from 3 DWT families: Coiflets,...
-
Metoda i system adaptacyjnego sterowania parametrami algorytmu syntezy niskich częstotliwości dźwięków muzycznych
PublicationW ostatnich latach można zaobserwować bardzo wyraźny i systematyczny wzrost wykorzystywania urządzeń mobilnych jako środka do odtwarzania muzyki, czy odtwarzania filmów w dowolnych warunkach akustycznych. Ich użytkownicy oczekują przy tym jak najlepszych walorów brzmieniowych dźwięku. W niniejszej rozprawie zostały zaproponowane metody, mające na celu poprawę brzmienia urządzeń mobilnych w zakresie niskich częstotliwości i korekcji...
-
MULTIMODALNE POMIARY DRGAŃ STRUNY
PublicationW artykule zostały przedstawione badania drgań struny zrealizowane przy użyciu szybkich kamer wizyjnych, mikrofonu oraz akcelerometru. Obiektem badań były instrumenty muzyczne. Opisano zjawiska zachodzące w instrumencie podczas tworzenia się i wydobywania z niego dźwięku. Celem pracy było zbadanie różnic w wynikach otrzymanych poprzez pomiary wykonane z użyciem zróżnicowanych reprezentacji obrazowych i sygnałowych. Zaproponowano...
-
Music information retrieval—The impact of technology, crowdsourcing, big data, and the cloud in art.
PublicationThe exponential growth of computer processing power, cloud data storage, and crowdsourcing model of gathering data bring new possibilities to music information retrieval (mir) field. Mir is no longer music content retrieval only; the area also comprises the discovery of expressing feelings and emotions contained in music, incorporating other than hearing modalities for helping this issue, users’ profiling, merging music with social...
-
Music signal equalization in a changing environment
PublicationThe paper presents the concept of an automatic system for music signal correction, considering room frequency response and music genre being played. The proposed algorithm, based on the room frequency response, compensates acoustic conditions surrounding the sound source. Additionally, the compensation process considers the signal content by recognizing music genre. As part of the described research, a series of subjective tests...
-
New applications of sound and vision engineering
PublicationMultimedia, Sound & Vision Engineering are relatively new fields within the area of science and technology, but teaching and research in this area has been carried out at Gdansk University of Technology (Gdansk, Poland) for nearly 5 decades. Current project carried-out in the Multimedia Systems Department are in the scope of the paper.
-
Porównanie klasycznych i ewolucyjnych metod projektowanie dyfuzorów akustycznych w warunkach odsłuchowych symulowanych metodą FDTD
PublicationW niniejszym rozdzaiel przedstawione zostanie porównanie dwóch podejść do projektowania dyfuzorów akustycznych. Pierwsze z nich bazuje na klasycznych założeniach dotyczących wykorzystywania sekwewncji pseudolosowych. Drugie z nich wykorzystuje automatyczne podejście bazujące na wykorzystaniu algorytmów genetycznych. Porównanie takie pozwala na określenie zalet i wad podejśc klasycznych oraz podejśc bazujących na zastosowaniu algorytmu...
-
Post-comatose patients with minimal consciousness tend to preserve reading comprehension skills but neglect syntax and spelling
PublicationModern eye tracking technology provides a means for communication with patients suffering from disorders of consciousness (DoC) or remaining in locked-in-state. However, being able to use an eye tracker for controlling text-based contents by such patients requires preserved reading ability in the first place. To our knowledge, this aspect, although of great social importance, so far has seemed to be neglected. In the paper, we...
-
Projektowanie oraz implementacja cyfrowego multiefektu gitarowego z wykorzystaniem procesora sygnałowego
PublicationW artykule został przedstawiony proces projektowania i realizacji cyfrowego multiefektu gitarowego z wykorzystaniem procesora sygnałowegoTMS320C5535 firmy Texas Instruments, dla którego oprogramowanie napisano w języku C. Omówiono zasady działania oraz algorytmy wybranych efektów dźwiękowych, które zostały zaimplementowane w procesorze sygnałowym. Zaprojektowano również uniwersalny moduł wejściowy zawierający wzmacniacz z regulowanym...
-
Real and Virtual Instruments in Machine Learning – Training and Comparison of Classification Results
PublicationThe continuous growth of the computing power of processors, as well as the fact that computational clusters can be created from combined machines, allows for increasing the complexity of algorithms that can be trained. The process, however, requires expanding the basis of the training sets. One of the main obstacles in music classification is the lack of high-quality, real-life recording database for every instrument with a variety...
-
Recovering Sound Produced by Wind Turbine Structures Employing Video Motion Magnification
PublicationThe recordings were made with a fast video camera and with a microphone. Using fast cameras allowed for observation of the micro vibrations of the object structure. Motion-magnified video recordings of wind turbines on a wind farm were made for the purpose of building a damage prediction system. An idea was to use video to recover sound & vibrations in order to obtain a contactless diagnostic method for wind turbines. The recovered signals...
-
Relationship between album cover design and music genres.
PublicationThe aim of the study is to find out whether there exists a relationship between typographic, compositional and coloristic elements of the music album cover design and music contained in the album. The research study involves basic statistical analysis of the manually extracted data coming from the worldwide album covers. The samples represent 34 different music genres, coming from nine countries from around the world. There are...
-
Sound engineering as our commitment to its creators in Poland
PublicationSound engineering is an interdisciplinary and rapidly expanding domain. It covers many aspects, such as sound perception, studio and sound mastering technology, music information retrieval including content-based search systems and automatic music transcription frameworks, sound synthesis, sound restoration, electroacoustics, and other ones constituting multimedia technology. Moreover, machine learning methods applied to the topics...
-
Special techniques and future perspectives: Simultaneous macro- and micro-electrode recordings
PublicationThere are many approaches to studying the inner workings of the brain and its highly interconnected circuits. One can look at the global activity in different brain structures using non-invasive technologies like positron emission tomography (PET) or functional magnetic resonance imaging (fMRI), which measure physiological changes, e.g. in the glucose uptake or blood flow. These can be very effectively used to localize active patches...
-
Speech Analytics Based on Machine Learning
PublicationIn this chapter, the process of speech data preparation for machine learning is discussed in detail. Examples of speech analytics methods applied to phonemes and allophones are shown. Further, an approach to automatic phoneme recognition involving optimized parametrization and a classifier belonging to machine learning algorithms is discussed. Feature vectors are built on the basis of descriptors coming from the music information...
-
Style Transfer for Detecting Vehicles with Thermal Camera
PublicationIn this work we focus on nighttime vehicle detection for intelligent traffic monitoring from the thermal camera. To train a Convolutional Neural Network (CNN) detector we create a stylized version of COCO (Common Objects in Context) dataset using Style Transfer technique that imitates images obtained from thermal cameras. This new dataset is further used for fine-tuning of the model and as a result detection accuracy on images...
-
Subjective tests for gathering knowledge for applying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot, and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with or...
-
Subjective tests for gathering konwledge for applaying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot,and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with...
-
Unsupervised machine-learning classification of electrophysiologically active electrodes during human cognitive task performance
PublicationIdentification of active electrodes that record task-relevant neurophysiological activity is needed for clinical and industrial applications as well as for investigating brain functions. We developed an unsupervised, fully automated approach to classify active electrodes showing event-related intracranial EEG (iEEG) responses from 115 patients performing a free recall verbal memory task. Our approach employed new interpretable...
-
Validating data acquired with experimental multimodal biometric system installed in bank branches
PublicationAn experimental system was engineered and implemented in 100 copies inside a real banking environment comprising: dynamic handwritten signature verification, face recognition, bank client voice recognition and hand vein distribution verification. The main purpose of the presented research was to analyze questionnaire responses reflecting user opinions on: comfort, ergonomics, intuitiveness and other aspects of the biometric enrollment...
-
Variable length sliding models for banking clients face biometry
PublicationAn experiment was organized in 100 bank branches to acquire biometric samples from nearly 5000 clients including face images. A procedure for creating face verification models based on continuously expanding database of biometric samples is proposed, implemented, and tested. The presented model applies to circumstances where it is possible to collect and to take into account new biometric samples after each positive verification...
-
Vehicle detector training with minimal supervision
PublicationRecently many efficient object detectors based on convolutional neural networks (CNN) have been developed and they achieved impressive performance on many computer vision tasks. However, in order to achieve practical results, CNNs require really large annotated datasets for training. While many such databases are available, many of them can only be used for research purposes. Also some problems exist where such datasets are not...
-
Weryfikacja autentyczności kolorów na zdjęciach wykonanych w technice analogowej
PublicationW artykule opisano zagadnienie odróżniania historycznych fotografii pomiędzy oryginalnie kolorowe a koloryzowane. Rozważono problem doboru zdjęć pod względem technologii, w jakiej zostały wykonane. Następnie wykorzystując sieci neuronowe już w części wyuczone na innych zbiorach danych, sprawdzono ich efektywność w rozwiązywaniu badanego problemu. Rozważono wpływ rozmiaru obrazu podanego na wejściu, architektury zastosowanej sieci,...
-
Wind Turbines Modeling as the Tool for Developing Algorithms of Processing their Video Recordings
PublicationIn the real world, many factors exist disturbing observation of the examined phenomena and causing various noises and distortions in recorded signals. It very often makes it difficult or even impossible to optimize various signal processing algorithms, through finding appropriate parameters. In this paper, we show an application, that retrieves wind turbine rotor speed from recorded video. Next, we describe the process of reduction...
-
Wind Turbines Modeling as the Tool for Developing Algorithms of Processing their Video Recordings
PublicationIn the real world, many factors exist disturbing observation of the examined phenomena and causing various noises and distortions in recorded signals. It very often makes it difficult or even impossible to optimize various signal processing algorithms, through finding appropriate parameters. In this paper, we show an application, that retrieves wind turbine rotor speed from recorded video. Next, we describe the process of reduction...
-
Wpływ kolorystyki ujęć oraz ścieżki dźwiękowej na emocje widza - wstępne eksperymenty
PublicationBrak
-
Wykorzystanie sieci neuronowych do syntezy mowy wyrażającej emocje
PublicationW niniejszym artykule przedstawiono analizę rozwiązań do rozpoznawania emocji opratych na mowie i możliwości ich wykprzystania w syntezie mowy z emocjami stosując do tego celu sieci neuronowe. Wskazano również przydatnośc parametrów typowo stosowanych do rozpoznawania mowy w detekcji emocji w śpiewie i rozróżnianiu tych emocji w obu przypadkach. Przedstawiono aktualne rozwiązania dotyczące rozpoznawania emocji w mowie i metod syntezy...
Year 2018
-
A comparative study of English viseme recognition methods and algorithm
PublicationAn elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector...
-
A comparative study of English viseme recognition methods and algorithms
PublicationAn elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector construction...
-
A Comparison of STI Measured by Direct and Indirect Methods for Interiors Coupled with Sound Reinforcement Systems
PublicationThis paper presents a comparison of STI (Speech Transmission Index) coefficient measurement results carried out by direct and indirect methods. First, acoustic parameters important in the context of public address and sound reinforcement systems are recalled. A measurement methodology is presented that employs various test signals to determine impulse responses. The process of evaluating sound system performance, signals enabling...
-
A Device for Measuring Auditory Brainstem Responses to Audio
PublicationStandard ABR devices use clicks and tone bursts to assess subjects’ hearing in an objective way. A new device was developed that extends the functionality of a standard ABR audiometer by collecting and analyzing auditory brainstem responses (ABR). The developed accessory allows for the use of complex sounds (e.g., speech or music excerpts) as stimuli. Therefore, it is possible to find out how efficiently different types of sounds...
-
A fast time-frequency multi-window analysis using a tuning directional kernel
PublicationIn this paper, a novel approach for time-frequency analysis and detection, based on the chirplet transform and dedicated to non-stationary as well as multi-component signals, is presented. Its main purpose is the estimation of spectral energy, instantaneous frequency (IF), spectral delay (SD), and chirp rate (CR) with a high time-frequency resolution (separation ability) achieved by adaptive fitting of the transform kernel. We...