Search
Department of Multimedia Systems
Publications
Filters
total: 908
Catalog Publications
Year 2021
-
Direct electrical stimulation of the human brain has inverse effects on the theta and gamma neural activities
PublicationObjective: Our goal was to analyze the electrophysiological response to direct electrical stimulation (DES) systematically applied at a wide range of parameters and anatomical sites, with particular focus on neural activities associated with memory and cognition. Methods: We used a large set of intracranial EEG (iEEG) recordings with DES from 45 subjects with electrodes...
-
Estimation of Average Speed of Road Vehicles by Sound Intensity Analysis
PublicationConstant monitoring of road traffic is important part of modern smart city systems. The proposed method estimates average speed of road vehicles in the observation period, using a passive acoustic vector sensor. Speed estimation based on sound intensity analysis is a novel approach to the described problem. Sound intensity in two orthogonal axes is measured with a sensor placed alongside the road. Position of the apparent sound...
-
Evaluation of aspiration problems in L2 English pronunciation employing machine learning
PublicationThe approach proposed in this study includes methods specifically dedicated to the detection of allophonic variation in English. This study aims to find an efficient method for automatic evaluation of aspiration in the case of Polish second-language (L2) English speakers’ pronunciation when whole words are analyzed instead of particular allophones extracted from words. Sample words including aspirated and unaspirated allophones...
-
Highlighting interlanguage phoneme differences based on similarity matrices and convolutional neural network
PublicationThe goal of this research is to find a way of highlighting the acoustic differences between consonant phonemes of the Polish and Lithuanian languages. For this purpose, similarity matrices are employed based on speech acoustic parameters combined with a convolutional neural network (CNN). In the first experiment, we compare the effectiveness of the similarity matrices applied to discerning acoustic differences between consonant...
-
Independent dynamics of low, intermediate, and high frequency spectral intracranial EEG activities during human memory formation
PublicationA wide spectrum of brain rhythms are engaged throughout the human cortex in cognitive functions. How the rhythms of various frequency ranges are coordinated across the space of the human cortex and time of memory processing is inconclusive. They can either be coordinated together across the frequency spectrum at the same cortical site and time or induced independently in particular bands. We used a large dataset of human intracranial...
-
Independent dynamics of slow, intermediate, and fast intracranial EEG spectral activities during human memory formation
PublicationA wide spectrum of brain rhythms are engaged throughout the human cortex in cognitive functions. How the rhythms of various low and high frequencies are spatiotemporally coordinated across the human brain during memory processing is inconclusive. They can either be coordinated together across a wide range of the frequency spectrum or induced in specific bands. We used a large dataset of human intracranial electroencephalography...
-
Leveraging spatio-temporal features for joint deblurring and segmentation of instruments in dental video microscopy
PublicationIn dentistry, microscopes have become indispensable optical devices for high-quality treatment and micro-invasive surgery, especially in the field of endodontics. Recent machine vision advances enable more advanced, real-time applications including but not limited to dental video deblurring and workflow analysis through relevant metadata obtained by instrument motion trajectories. To this end, the proposed work addresses dental...
-
Mispronunciation Detection in Non-Native (L2) English with Uncertainty Modeling
PublicationA common approach to the automatic detection of mispronunciation in language learning is to recognize the phonemes produced by a student and compare it to the expected pronunciation of a native speaker. This approach makes two simplifying assumptions: a) phonemes can be recognized from speech with high accuracy, b) there is a single correct way for a sentence to be pronounced. These assumptions do not always hold, which can result...
-
Production of six-degrees-of-freedom (6DoF) navigable audio using 30 Ambisonic microphones
PublicationThis paper describes a method for planning, recording, and post-production of six-degrees-of-freedom audio recorded with multiple 3rd order Ambisonic microphone arrays. The description is based on the example of recordings conducted in August 2020 with the Poznan Philharmonic Orchestra using 30 units of Zylia ZM-1S. A convenient way to prepare and organize such a big project is proposed – this involves details of stage planning,...
-
Robustness in Compressed Neural Networks for Object Detection
PublicationModel compression techniques allow to significantly reduce the computational cost associated with data processing by deep neural networks with only a minor decrease in average accuracy. Simultaneously, reducing the model size may have a large effect on noisy cases or objects belonging to less frequent classes. It is a crucial problem from the perspective of the models' safety, especially for object detection in the autonomous driving...
-
Selective monitoring of noise emitted by vehicles involved in road traffic
PublicationAn acoustic intensity probe was developed measures the sound intensity in three orthogonal directions, making possible to calculate the azimuth and elevation angles, describing the sound source position. The acoustic sensor is made in the form of a cube with a side of 10 mm, on the inner surfaces of which the digital MEMS microphones are mounted. The algorithm works in two stages. The first stage is based on the analysis of sound...
-
Skuteczność klasyfikacji gatunków muzycznych za pomocą sieci neuronowej w zależności od typu danych wejściowych
PublicationRozpoznawanie gatunku muzycznego jest jednym z podstawowych elementów inteligentnych systemów tworzenia automatycznych list muzyki. Platformy strumieniowe oferujące taką usługę wymagają rozwiązań, które umożliwią jak najdokładniej określić przynależność utworu do gatunku muzycznego. Zgodnie z aktualnym stanem wiedzy – najskuteczniejszym klasyfikatorem są sztuczne sieci neuronowe (w tym w wersji uczenia głębokiego), dla których...
-
Techniki wielokanałowe wykorzystywane w koncertach i nagraniach muzycznych na odległość
PublicationW czasie pandemii koronawirusa COVID-19 nowego znaczenia nabrały możliwości transmisji dźwięku z obrazem – zwłaszcza do pracy zdalnej, która w przypadku muzyków jest szczególnym wyzwaniem zarówno w kontekście wspólnych ćwiczeń i prób, jak i koncertów. Wynikła konieczność wieloźródłowego połączenia ujawniła potrzebę uprzestrzennienia dźwięku w celu łatwiejszej lokalizacji źródeł dźwięku. Tworzenie zdalnych nagrań muzycznych stało...
-
Towards Cancer Patients Classification Using Liquid Biopsy
PublicationLiquid biopsy is a useful, minimally invasive diagnostic and monitoring tool for cancer disease. Yet, developing accurate methods, given the potentially large number of input features, and usually small datasets size remains very challenging. Recently, a novel feature parameterization based on the RNA-sequenced platelet data which uses the biological knowledge from the Kyoto Encyclopedia of Genes and Genomes, combined with a classifier...
Year 2020
-
1D convolutional context-aware architectures for acoustic sensing and recognition of passing vehicle type
PublicationA network architecture that may be employed to sensing and recognition of a type of vehicle on the basis of audio recordings made in the proximity of a road is proposed in the paper. The analyzed road traffic consists of both passenger cars and heavier vehicles. Excerpts from recordings that do not contain vehicles passing sounds are also taken into account and marked as ones containing silence....
-
A Study of Cross-Linguistic Speech Emotion Recognition Based on 2D Feature Spaces
PublicationIn this research, a study of cross-linguistic speech emotion recognition is performed. For this purpose, emotional data of different languages (English, Lithuanian, German, Spanish, Serbian, and Polish) are collected, resulting in a cross-linguistic speech emotion dataset with the size of more than 10.000 emotional utterances. Despite the bi-modal character of the databases gathered, our focus is on the acoustic representation...
-
Adaptive traffic optimization using Variable Speed Limits; Adaptacyjna optymalizacja ruchu drogowego przy pomocy zmiennych ograniczeń prędkości
PublicationVariable speed limits (VSL) is an intelligent transportation system (ITS) solution for traffic management. The speed limits can be changed dynamically in order to adapt to traffic, weather, or road surface conditions. This paper presents an approach for such an adaptive traffic control where the primary goal is to ensure traffic safety and efficiency of the traffic control system (fast response to dynamically changing traffic,...
-
Ambisoniczna mapa wybranych miejsc w Trójmieście
PublicationProjekt miał na celu stworzenie ambisonicznej mapy Trójmiasta w formie aplikacji internetowej. Materiały wideo w technologii 360 z dźwiękiem w postaci sygnału ambisonicznego zostały zarejestrowane w lokalizacjach Trójmiasta, które uznano za charakterystyczne dla tej aglomeracji. Celem badawczym projektu było porównanie dostępnych algorytmów miksowania sygnałów ambisonicznych poprzez przeprowadzenie testów odsłuchowych. Przeprowadzono...
-
Analiza ruchu drogowego z wykorzystaniem analizy akustycznej
PublicationTematyka pracy porusza zagadnienia dotyczące pozyskiwania informacji o ruchu drogowym z wykorzystaniem monitoringu akustycznego. Przybliżono podstawowe techniki nadzoru nad ruchem drogowym. Przedstawiono założenia akustycznego detektora ruchu i zbadano jego skuteczność na trzech płaszczyznach działania – zliczania pojazdów, klasyfikacji rodzajowej i klasyfikacji warunków pogodowych panujących na nawierzchni
-
Analyzing the Effectiveness of the Brain–Computer Interface for Task Discerning Based on Machine Learning
PublicationThe aim of the study is to compare electroencephalographic (EEG) signal feature extraction methods in the context of the effectiveness of the classification of brain activities. For classification, electroencephalographic signals were obtained using an EEG device from 17 subjects in three mental states (relaxation, excitation, and solving logical task). Blind source separation employing independent component analysis (ICA) was...
-
Audio Feature Analysis for Precise Vocalic Segments Classification in English
PublicationAn approach to identifying the most meaningful Mel-Frequency Cepstral Coefficients representing selected allophones and vocalic segments for their classification is presented in the paper. For this purpose, experiments were carried out using algorithms such as Principal Component Analysis, Feature Importance, and Recursive Parameter Elimination. The data used were recordings made within the ALOFON corpus containing audio signal...
-
Automatic Marking of Allophone Boundaries in Isolated English spoken Words
PublicationThe work presents a method that allows delimiting the borders of allophones in isolated English words. The described method is based on the DTW algorithm combining two signals, a reference signal and an analyzed one. As the reference signal, recordings from the MODALITY database were used, from which the words were extracted. This database was also used for tests, which were described. Test results show that the automatic determination...
-
Chór wirtualny
PublicationWiosna roku 2020 została zapisana emocjami, które należy zaliczać do tych niepożądanych. Praca on-line stała się jedyną możliwą formą pracy z zespołem. Prekursorem pomysłu wirtualnego chóru był amerykański kompozytor i dyrygent Eric Whitacre. Eric wybrał do wykonania przez chór wirtualny utwory posiadające wspólne cechy. Kolejnym poruszanym zagadnieniem jest stworzenie przestrzennego dźwięku. Technologia na której opiera się dźwięk...
-
Comparing traffic intensity estimates employing passive acoustic radar and microwave Doppler radar sensor
PublicationThe purpose of our applied research project is to develop an autonomous road sign with built-in radar devices of our design. In this paper, we show that it is possible to calibrate the acoustic vector sensor so that it can be used to measure traffic volume and count the vehicles involved in the traffic through the analysis of the noise emitted by them. Signals obtained from a Doppler radar are used as a reference source. Although...
-
Comparison of sound of organ pipes in contemporary and historical instruments
PublicationThe aim of this research is to examine the differences in the timbre of organ pipes’ sound between a historical and a contemporary organ instrument. The historical instrument is the Oliwa organ from Gdansk, Poland, and the contemporary one is from Kartuzy, Poland. Recordings are made of single notes played by an open labial pipe that belongs to the Principal rank. The analyses and comparison of several sound features compatible...
-
Comparison of two methods of sound extraction from guitar string video recordings
PublicationA comparison of two sound extraction methods from guitar string video recordings is presented in the paper. A brief overview of highframe rate camera technology and possible applications are included. The method using the image analysis from two such cameras is presented. The cameras are placed at the angle of 90 degrees for recording the image in three planes. The results achieved...
-
Constructing a Dataset of Speech Recordingswith Lombard Effect
PublicationThepurpose of therecordings was to create a speech corpus based on the ISLEdataset, extended with video and Lombard speech. Selected from a set of 165sentences, 10, evaluatedas having thehighest possibility to occur in the context ofthe Lombard effect,were repeated in the presence of the so-called babble speech to obtain Lombard speech features. Altogether,15speakers were recorded, and speech parameterswere...
-
Employing Subjective Tests and Deep Learning for Discovering the Relationship between Personality Types and Preferred Music Genres
PublicationThe purpose of this research is two-fold: (a) to explore the relationship between the listeners’ personality trait, i.e., extraverts and introverts and their preferred music genres, and (b) to predict the personality trait of potential listeners on the basis of a musical excerpt by employing several classification algorithms. We assume that this may help match songs according to the listener’s personality in social music networks....
-
Evaluating calibration and robustness of pedestrian detectors
PublicationIn this work robustness and calibration of modern pedestrian detectors are evaluated. Pedestrian detection is a crucial perception com- ponent in autonomous driving and here we study its performance under different image corruptions. Furthermore, we provide analysis of classifi- cation calibration of pedestrian detectors and we show a positive effect of using style-transfer augmentation technique. Our analysis is aimed as a step...
-
Evaluation of Lombard Speech Models in the Context of Speech in Noise Enhancement
PublicationThe Lombard effect is one of the most well-known effects of noise on speech production. Speech with the Lombard effect is more easily recognizable in noisy environments than normal natural speech. Our previous investigations showed that speech synthesis models might retain Lombard-effect characteristics. In this study, we investigate several speech models, such as harmonic, source-filter, and sinusoidal, applied to Lombard speech...
-
Improving Objective Speech Quality Indicators in Noise Conditions
PublicationThis work aims at modifying speech signal samples and test them with objective speech quality indicators after mixing the original signals with noise or with an interfering signal. Modifications that are applied to the signal are related to the Lombard speech characteristics, i.e., pitch shifting, utterance duration changes, vocal tract scaling, manipulation of formants. A set of words and sentences in Polish, recorded in silence,...
-
Investigating Feature Spaces for Isolated Word Recognition
PublicationThe study addresses the issues related to the appropriateness of a two-dimensional representation of speech signal for speech recognition tasks based on deep learning techniques. The approach combines Convolutional Neural Networks (CNNs) and time-frequency signal representation converted to the investigated feature spaces. In particular, waveforms and fractal dimension features of the signal were chosen for the time domain, and...
-
Microscopic traffic simulation models for connected and automated vehicles (CAVs) – state-of-the-art
PublicationResearch on connected and automated vehicles (CAVs) has been gaining substantial momentum in recent years. However, thevast amount of literature sources results in a wide range of applied tools and datasets, assumed methodology to investigate thepotential impacts of future CAVs traffic, and, consequently, differences in the obtained findings. This limits the scope of theircomparability and applicability and calls for a proper standardization...
-
Multifactor consciousness level assessment of participants with acquired brain injuries employing human–computer interfaces
PublicationBackground A lack of communication with people suffering from acquired brain injuries may lead to drawing erroneous conclusions regarding the diagnosis or therapy of patients. Information technology and neuroscience make it possible to enhance the diagnostic and rehabilitation process of patients with traumatic brain injury or post-hypoxia. In this paper, we present a new method for evaluation possibility of communication and the...
-
Multimedia Communications, Services and Security MCSS. 10th International Conference, MCSS 2020, Preface
PublicationMultimedia surrounds us everywhere. It is estimated that only a part of the recorded resources are processed and analyzed. These resources offer enormous opportunities to improve the quality of life of citizens. As a result, of the introduction of a new type of algorithms to improve security by maintaining a high level of privacy protection. Among the many articles, there are examples of solutions for improving the operation of...
-
Musical Instrument Tagging Using Data Augmentation and Effective Noisy Data Processing
PublicationDeveloping signal processing methods to extract information automatically has potential in several applications, for example searching for multimedia based on its audio content, making context-aware mobile applications (e.g., tuning apps), or pre-processing for an automatic mixing system. However, the last-mentioned application needs a significant amount of research to reliably recognize real musical instruments in recordings....
-
O nadjeżdżającej rewolucji w transporcie
Publication1,3 miliona – tyle osób rocznie na świecie ginie w wypadkach drogowych. Ponad 20 milionów zostaje rannych! 4 miliardy złotych – prawie tyle rocznie tracą kierowcy w 7 największych miastach w Polsce z powodu korków (a są to jedynie szacowane koszty straconego czasu i paliwa, bez uwzględnienia np. negatywnego wpływu na środowisko). Czy możemy coś z tym zrobić?
-
Projekt INZNAK - aktywne znaki drogowe
PublicationW Politechnice Gdańskiej na Wydziale Elektroniki, Telekomunikacji i Informatyki we współpracy z Akademią Górniczo-Hutniczą w Krakowie i dwiema firmami z województwa pomorskiego (Siled Sp. z o.o. i Microsystems Sp. z o.o.) od 2017 r. realizowany jest projekt badawczy pt. „INZNAK – inteligentne znaki drogowe do adaptacyjnego sterowania ruchem pojazdów, komunikujące się w technologii V2X”. Projekt jest dofinansowywany przez NCBR w...
-
Ranking Speech Features for Their Usage in Singing Emotion Classification
PublicationThis paper aims to retrieve speech descriptors that may be useful for the classification of emotions in singing. For this purpose, Mel Frequency Cepstral Coefficients (MFCC) and selected Low-Level MPEG 7 descriptors were calculated based on the RAVDESS dataset. The database contains recordings of emotional speech and singing of professional actors presenting six different emotions. Employing the algorithm of Feature Selection based...
-
System for monitoring road slippery based on CCTV cameras and convolutional neural networks
PublicationThe slipperiness of the surface is essential for road safety. The growing number of CCTV cameras opens the possibility of using them to automatically detect the slippery surface and inform road users about it. This paper presents a system of developed intelligent road signs, including a detector based on convolutional neural networks (CNNs) and the transferlearning method employed to the processing of images acquired with video...
-
Toward Robust Pedestrian Detection With Data Augmentation
PublicationIn this article, the problem of creating a safe pedestrian detection model that can operate in the real world is tackled. While recent advances have led to significantly improved detection accuracy on various benchmarks, existing deep learning models are vulnerable to invisible to the human eye changes in the input image which raises concerns about its safety. A popular and simple technique for improving robustness is using data...
-
Vehicle Detection with Self-Training for Adaptative Video Processing Embedded Platform
PublicationTraffic monitoring from closed-circuit television (CCTV) cameras on embedded systems is the subject of the performed experiments. Solving this problem encounters difficulties related to the hardware limitations, and possible camera placement in various positions which affects the system performance. To satisfy the hardware requirements, vehicle detection is performed using a lightweight Convolutional Neural Network (CNN), named...
Year 2019
-
A Computationally Efficient Model for Predicting Successful Memory Encoding Using Machine-Learning-based EEG Channel Selection
PublicationComputational cost is an important consideration for memory encoding prediction models that use data from dozens of implanted electrodes. We propose a method to reduce computational expense by selecting a subset of all the electrodes to build the prediction model. The electrodes were selected based on their likelihood of measuring brain activity useful for predicting memory encoding better than chance (in terms of AUC). A logistic...
-
A Concept of Automatic Film Color Grading Based on Music Recognition and Evoked Emotions
PublicationThe article presents the aspects of the final selection of the color of shots in film production based on the psychology of color. First of all, the elements of color processing, contrast, saturation or white balance in the film shots were presented and the definition of color grading was given. In the second part of the article the analysis of film music was conducted in the context of stimulating appropriate emotions while watching...
-
Akustyczna analiza natężenia ruchu drogowego dla systemów zarządzania ruchem
PublicationW pracy przybliżono wybrane zagadnienia z dziedziny zarządzania transportem drogowym w Polsce i na świecie. W tym kontekście pzredstawiono potrzeby rynkowe, wymagania jak i możliwości w zakresie pozyskiwania informacji o aktualnym stanie sieci drogowych. Zaproponowano akustyczną metodę nadzorowania ruchu drogowego i jej możliwości w kontekście systemów zarządzania ruchem. Przedstawiono schemat akwizycji sygnału wraz z danymi odniesienia....
-
An Attempt to Create Speech Synthesis Model That Retains Lombard Effect Characteristics
PublicationThe speech with the Lombard effect has been extensively studied in the context of speech recognition or speech enhancement. However, few studies have investigated the Lombard effect in the context of speech synthesis. The aim of this paper is to create a mathematical model that allows for retaining the Lombard effect. These models could be used as a basis of a formant speech synthesizer. The proposed models are based on dividing...
-
ANALIZA KOLORÓW SCEN FILMOWYCH W KONTEKŚCIE COLOR GRADINGU
PublicationW artykule przedstawiono zagadnienia związane z kolorowaniem sceny filmowej. W pracy przedyskutowano główne aspekty obróbki koloru obrazu filmowego oraz omówiono definicje pojęć związanych z kolorowaniem sceny, tj.: color correction oraz color gradingu. Opisano teorie psychologii koloru oraz ich praktyczne wykorzystanie w filmie i odniesiono je do podstawowych gatunków filmowych i modeli emocji. Następnie przedyskutowano założenia...
-
ANALIZA PARAMETRÓW SYGNAŁU MOWY W KONTEKŚCIE ICH PRZYDATNOŚCI W AUTOMATYCZNEJ OCENIE JAKOŚCI EKSPRESJI ŚPIEWU
PublicationPraca dotyczy podejścia do parametryzacji w przypadku klasyfikacji emocji w śpiewie oraz porównania z klasyfikacją emocji w mowie. Do tego celu wykorzystano bazę mowy i śpiewu nacechowanego emocjonalnie RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song), zawierającą nagrania profesjonalnych aktorów prezentujących sześć różnych emocji. Następnie obliczono współczynniki mel-cepstralne (MFCC) oraz wybrane deskryptory...
-
Application of autoencoder to traffic noise analysis
PublicationThe aim of an autoencoder neural network is to transform the input data into a lower-dimensional code and then to reconstruct the output from this code representation. Applications of autoencoders to classifying sound events in the road traffic have not been found in the literature. The presented research aims to determine whether such an unsupervised learning method may be used for deploying classification algorithms applied to...
-
Assessment of the Effectiveness of a Short-term Hearing Aid Use in Patients with Different Degrees of Hearing Loss
PublicationThe study presents evaluating the effectiveness of the hearing aid fitting process in the short-term use (7 days). The evaluation method consists of a survey based on the APHAB (Abbreviated Profile of Hearing Aid Benefit) questionnaire. Additional criteria such as a degree of hearing loss, number of hours and days of hearing aid use as well as the user’s experience were also taken into consideration. The outcomes of the benefit...