Search
Department of Multimedia Systems
Publications
Filters
total: 924
Catalog Publications
Year 2020
-
Multimedia Communications, Services and Security MCSS. 10th International Conference, MCSS 2020, Preface
PublicationMultimedia surrounds us everywhere. It is estimated that only a part of the recorded resources are processed and analyzed. These resources offer enormous opportunities to improve the quality of life of citizens. As a result, of the introduction of a new type of algorithms to improve security by maintaining a high level of privacy protection. Among the many articles, there are examples of solutions for improving the operation of...
-
Musical Instrument Tagging Using Data Augmentation and Effective Noisy Data Processing
PublicationDeveloping signal processing methods to extract information automatically has potential in several applications, for example searching for multimedia based on its audio content, making context-aware mobile applications (e.g., tuning apps), or pre-processing for an automatic mixing system. However, the last-mentioned application needs a significant amount of research to reliably recognize real musical instruments in recordings....
-
O nadjeżdżającej rewolucji w transporcie
Publication1,3 miliona – tyle osób rocznie na świecie ginie w wypadkach drogowych. Ponad 20 milionów zostaje rannych! 4 miliardy złotych – prawie tyle rocznie tracą kierowcy w 7 największych miastach w Polsce z powodu korków (a są to jedynie szacowane koszty straconego czasu i paliwa, bez uwzględnienia np. negatywnego wpływu na środowisko). Czy możemy coś z tym zrobić?
-
Projekt INZNAK - aktywne znaki drogowe
PublicationW Politechnice Gdańskiej na Wydziale Elektroniki, Telekomunikacji i Informatyki we współpracy z Akademią Górniczo-Hutniczą w Krakowie i dwiema firmami z województwa pomorskiego (Siled Sp. z o.o. i Microsystems Sp. z o.o.) od 2017 r. realizowany jest projekt badawczy pt. „INZNAK – inteligentne znaki drogowe do adaptacyjnego sterowania ruchem pojazdów, komunikujące się w technologii V2X”. Projekt jest dofinansowywany przez NCBR w...
-
Ranking Speech Features for Their Usage in Singing Emotion Classification
PublicationThis paper aims to retrieve speech descriptors that may be useful for the classification of emotions in singing. For this purpose, Mel Frequency Cepstral Coefficients (MFCC) and selected Low-Level MPEG 7 descriptors were calculated based on the RAVDESS dataset. The database contains recordings of emotional speech and singing of professional actors presenting six different emotions. Employing the algorithm of Feature Selection based...
-
System for monitoring road slippery based on CCTV cameras and convolutional neural networks
PublicationThe slipperiness of the surface is essential for road safety. The growing number of CCTV cameras opens the possibility of using them to automatically detect the slippery surface and inform road users about it. This paper presents a system of developed intelligent road signs, including a detector based on convolutional neural networks (CNNs) and the transferlearning method employed to the processing of images acquired with video...
-
Toward Robust Pedestrian Detection With Data Augmentation
PublicationIn this article, the problem of creating a safe pedestrian detection model that can operate in the real world is tackled. While recent advances have led to significantly improved detection accuracy on various benchmarks, existing deep learning models are vulnerable to invisible to the human eye changes in the input image which raises concerns about its safety. A popular and simple technique for improving robustness is using data...
-
Vehicle Detection with Self-Training for Adaptative Video Processing Embedded Platform
PublicationTraffic monitoring from closed-circuit television (CCTV) cameras on embedded systems is the subject of the performed experiments. Solving this problem encounters difficulties related to the hardware limitations, and possible camera placement in various positions which affects the system performance. To satisfy the hardware requirements, vehicle detection is performed using a lightweight Convolutional Neural Network (CNN), named...
Year 2019
-
A Computationally Efficient Model for Predicting Successful Memory Encoding Using Machine-Learning-based EEG Channel Selection
PublicationComputational cost is an important consideration for memory encoding prediction models that use data from dozens of implanted electrodes. We propose a method to reduce computational expense by selecting a subset of all the electrodes to build the prediction model. The electrodes were selected based on their likelihood of measuring brain activity useful for predicting memory encoding better than chance (in terms of AUC). A logistic...
-
A Concept of Automatic Film Color Grading Based on Music Recognition and Evoked Emotions
PublicationThe article presents the aspects of the final selection of the color of shots in film production based on the psychology of color. First of all, the elements of color processing, contrast, saturation or white balance in the film shots were presented and the definition of color grading was given. In the second part of the article the analysis of film music was conducted in the context of stimulating appropriate emotions while watching...
-
Akustyczna analiza natężenia ruchu drogowego dla systemów zarządzania ruchem
PublicationW pracy przybliżono wybrane zagadnienia z dziedziny zarządzania transportem drogowym w Polsce i na świecie. W tym kontekście pzredstawiono potrzeby rynkowe, wymagania jak i możliwości w zakresie pozyskiwania informacji o aktualnym stanie sieci drogowych. Zaproponowano akustyczną metodę nadzorowania ruchu drogowego i jej możliwości w kontekście systemów zarządzania ruchem. Przedstawiono schemat akwizycji sygnału wraz z danymi odniesienia....
-
An Attempt to Create Speech Synthesis Model That Retains Lombard Effect Characteristics
PublicationThe speech with the Lombard effect has been extensively studied in the context of speech recognition or speech enhancement. However, few studies have investigated the Lombard effect in the context of speech synthesis. The aim of this paper is to create a mathematical model that allows for retaining the Lombard effect. These models could be used as a basis of a formant speech synthesizer. The proposed models are based on dividing...
-
ANALIZA KOLORÓW SCEN FILMOWYCH W KONTEKŚCIE COLOR GRADINGU
PublicationW artykule przedstawiono zagadnienia związane z kolorowaniem sceny filmowej. W pracy przedyskutowano główne aspekty obróbki koloru obrazu filmowego oraz omówiono definicje pojęć związanych z kolorowaniem sceny, tj.: color correction oraz color gradingu. Opisano teorie psychologii koloru oraz ich praktyczne wykorzystanie w filmie i odniesiono je do podstawowych gatunków filmowych i modeli emocji. Następnie przedyskutowano założenia...
-
ANALIZA PARAMETRÓW SYGNAŁU MOWY W KONTEKŚCIE ICH PRZYDATNOŚCI W AUTOMATYCZNEJ OCENIE JAKOŚCI EKSPRESJI ŚPIEWU
PublicationPraca dotyczy podejścia do parametryzacji w przypadku klasyfikacji emocji w śpiewie oraz porównania z klasyfikacją emocji w mowie. Do tego celu wykorzystano bazę mowy i śpiewu nacechowanego emocjonalnie RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song), zawierającą nagrania profesjonalnych aktorów prezentujących sześć różnych emocji. Następnie obliczono współczynniki mel-cepstralne (MFCC) oraz wybrane deskryptory...
-
Application of autoencoder to traffic noise analysis
PublicationThe aim of an autoencoder neural network is to transform the input data into a lower-dimensional code and then to reconstruct the output from this code representation. Applications of autoencoders to classifying sound events in the road traffic have not been found in the literature. The presented research aims to determine whether such an unsupervised learning method may be used for deploying classification algorithms applied to...
-
Assessment of the Effectiveness of a Short-term Hearing Aid Use in Patients with Different Degrees of Hearing Loss
PublicationThe study presents evaluating the effectiveness of the hearing aid fitting process in the short-term use (7 days). The evaluation method consists of a survey based on the APHAB (Abbreviated Profile of Hearing Aid Benefit) questionnaire. Additional criteria such as a degree of hearing loss, number of hours and days of hearing aid use as well as the user’s experience were also taken into consideration. The outcomes of the benefit...
-
Automatic labeling of traffic sound recordings using autoencoder-derived features
PublicationAn approach to detection of events occurring in road traffic using autoencoders is presented. Extensions of existing algorithms of acoustic road events detection employing Mel Frequency Cepstral Coefficients combined with classifiers based on k nearest neighbors, Support Vector Machines, and random forests are used. In our research, the acoustic signal gathered from the microphone placed near the road is split into frames and converted...
-
Combining Road Network Data from OpenStreetMap with an Authoritative Database
PublicationComputer modeling of road networks requires detailed and up-to-date dataset. This paper proposes a method of combining authoritative databases with OpenStreetMap (OSM) system. The complete route is established by finding paths in the graph constructed from partial data obtained from OSM. In order to correlate data from both sources, a method of coordinate conversion is proposed. The algorithm queries road data from OSM and provides...
-
Comparative study on the effectiveness of various types of road traffic intensity detectors
PublicationVehicle detection and speed measurements are crucial tasks in traffic monitoring systems. In this work, we focus on several types of electronic sensors, operating on different physical principles in order to compare their effectiveness in real traffic conditions. Commercial solutions are based on road tubes, microwave sensors, LiDARs, and video cameras. Distributed traffic monitoring systems require a high number of monitoring...
-
Comparison of Lithuanian and Polish Consonant Phonemes Based on Acoustic Analysis – Preliminary Results
PublicationThe goal of this research is to find a set of acoustic parameters that are related to differences between Polish and Lithuanian language consonants. In order to identify these differences, an acoustic analysis is performed, and the phoneme sounds are described as the vectors of acoustic parameters. Parameters known from the speech domain as well as those from the music information retrieval area are employed. These parameters are...
-
Comparison of the effectiveness of automatic EEG signal class separation algorithms
PublicationIn this paper, an algorithm for automatic brain activity class identification of EEG (electroencephalographic) signals is presented. EEG signals are gathered from seventeen subjects performing one of the three tasks: resting, watching a music video and playing a simple logic game. The methodology applied consists of several steps, namely: signal acquisition, signal processing utilizing z-score normalization, parametrization and...
-
Database of speech and facial expressions recorded with optimized face motion capture settings
PublicationThe broad objective of the present research is the analysis of spoken English employing a multiplicity of modalities. An important stage of this process, discussed in the paper, is creating a database of speech accompanied with facial expressions. Recordings of speakers were made using an advanced system for capturing facial muscle motion. A brief historical outline, current applications, limitations and the ways of capturing face...
-
Deep neural networks for human pose estimation from a very low resolution depth image
PublicationThe work presented in the paper is dedicated to determining and evaluating the most efficient neural network architecture applied as a multiple regression network localizing human body joints in 3D space based on a single low resolution depth image. The main challenge was to deal with a noisy and coarse representation of the human body, as observed by a depth sensor from a large distance, and to achieve high localization precision....
-
Development of Intelligent Road Signs with V2X Interface for Adaptive Traffic Controlling
PublicationThe objective of this paper is to present a practical project of intelligent road signs, under which a series of new products for the regulation of traffic is being created. The engineering part of the project, described in this paper, was preceded by a series of experimental studies, the results of which were described in another paper accepted for publication at the MTS-ITS conference 2019, entitled "Comparative study on the effectiveness...
-
Diagnosing wind turbine condition employing a neural network to the analysis of vibroacoustic signals
PublicationIt is important from the economic point of view to detect damage early in the wind turbines before failures occur. For this purpose, a monitoring device was built that analyzes both acoustic signals acquired from the built-in non-contact acoustic intensity probe, as well as from the accelerometers, mounted on the internal devices in the nacelle. The signals collected in this way are used for long-term training of the autoencoder...
-
Discovering Rule-Based Learning Systems for the Purpose of Music Analysis
PublicationMusic analysis and processing aims at understanding information retrieved from music (Music Information Retrieval). For the purpose of music data mining, machine learning (ML) methods or statistical approach are employed. Their primary task is recognition of musical instrument sounds, music genre or emotion contained in music, identification of audio, assessment of audio content, etc. In terms of computational approach, music databases...
-
Estimating Traffic Intensity Employing Passive Acoustic Radar and Enhanced Microwave Doppler Radar Sensor
PublicationInnovative road signs that can autonomously display the speed limit in cases where the trac situation requires it are under development. The autonomous road sign contains many types of sensors, of which the subject of interest in this article is the Doppler sensor that we have improved and the constructed and calibrated acoustic probe. An algorithm for performing vehicle detection and tracking, as well as vehicle speed measurement,...
-
Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries
PublicationOne of the first clinical signs differentiating the minimally conscious state from the vegetative state is the presence of smooth pursuit eye movements occurring in direct response to moving salient stimuli. Glasgow Coma Scale (GCS) is one of the most commonly used diagnostic tools for acute phase assessment of the level of consciousness, together with a neurological examination. These classic measures are limited to qualitative...
-
Human verbal memory encoding is hierarchically distributed in a continuous processing stream
PublicationProcessing of memory is supported by coordinated activity in a network of sensory, association, and motor brain regions. It remains a major challenge to determine where memory is encoded for later retrieval. Here we used direct intracranial brain recordings from epilepsy patients performing free recall tasks to determine the temporal pattern and anatomical distribution of verbal memory encoding across the entire human cortex. High...
-
Influence of the Delay in Monitor System on the Motor Coordination of Musicians while Performing
PublicationThis paper provides a description and results of measurements of the maximum acceptable value of delay tolerated by a musician, while playing an instrument, that does not cause de-synchronization and discomfort. First, methodology of measurements comprising audio recording and a fast camera is described. Then, themeasurement procedure for acquiring the maximum value of delay conditioning...
-
Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech
PublicationWe present a novel deep learning model for the detection and reconstruction of dysarthric speech. We train the model with a multi-task learning technique to jointly solve dysarthria detection and speech reconstruction tasks. The model key feature is a low-dimensional latent space that is meant to encode the properties of dysarthric speech. It is commonly believed that neural networks are black boxes that solve problems but do not...
-
Labeler-hot Detection of EEG Epileptic Transients
PublicationPreventing early progression of epilepsy and sothe severity of seizures requires effective diagnosis. Epileptictransients indicate the ability to develop seizures but humansoverlook such brief events in an electroencephalogram (EEG)what compromises patient treatment. Traditionally, trainingof the EEG event detection algorithms has relied on groundtruth labels, obtained from the consensus...
-
Localization of sound sources with dual acoustic vector sensor
PublicationThe aim of the work is to estimate the position of sound sources. The proposed method uses a setup of two acoustic vector sensors (AVS). The intersection of azimuth rays from each AVS should indicate the position of a source. In practice, the result of position estimation using this method is an area rather than a point. This is a result of inaccuracy of the individual sensors, but more importantly, of the influence of a source...
-
MACHINE LEARNING–BASED ANALYSIS OF ENGLISH LATERAL ALLOPHONES
PublicationAutomatic classification methods, such as artificial neural networks (ANNs), the k-nearest neighbor (kNN) and selforganizing maps (SOMs), are applied to allophone analysis based on recorded speech. A list of 650 words was created for that purpose, containing positionally and/or contextually conditioned allophones. For each word, a group of 16 native and non-native speakers were audio-video recorded, from which seven native speakers’...
-
Method for Clustering of Brain Activity Data Derived from EEG Signals
PublicationA method for assessing separability of EEG signals associated with three classes of brain activity is proposed. The EEG signals are acquired from 23 subjects, gathered from a headset consisting of 14 electrodes. Data are processed by applying Discrete Wavelet Transform (DWT) for the signal analysis and an autoencoder neural network for the brain activity separation. Processing involves 74 wavelets from 3 DWT families: Coiflets,...
-
Metoda i system adaptacyjnego sterowania parametrami algorytmu syntezy niskich częstotliwości dźwięków muzycznych
PublicationW ostatnich latach można zaobserwować bardzo wyraźny i systematyczny wzrost wykorzystywania urządzeń mobilnych jako środka do odtwarzania muzyki, czy odtwarzania filmów w dowolnych warunkach akustycznych. Ich użytkownicy oczekują przy tym jak najlepszych walorów brzmieniowych dźwięku. W niniejszej rozprawie zostały zaproponowane metody, mające na celu poprawę brzmienia urządzeń mobilnych w zakresie niskich częstotliwości i korekcji...
-
MULTIMODALNE POMIARY DRGAŃ STRUNY
PublicationW artykule zostały przedstawione badania drgań struny zrealizowane przy użyciu szybkich kamer wizyjnych, mikrofonu oraz akcelerometru. Obiektem badań były instrumenty muzyczne. Opisano zjawiska zachodzące w instrumencie podczas tworzenia się i wydobywania z niego dźwięku. Celem pracy było zbadanie różnic w wynikach otrzymanych poprzez pomiary wykonane z użyciem zróżnicowanych reprezentacji obrazowych i sygnałowych. Zaproponowano...
-
Music information retrieval—The impact of technology, crowdsourcing, big data, and the cloud in art.
PublicationThe exponential growth of computer processing power, cloud data storage, and crowdsourcing model of gathering data bring new possibilities to music information retrieval (mir) field. Mir is no longer music content retrieval only; the area also comprises the discovery of expressing feelings and emotions contained in music, incorporating other than hearing modalities for helping this issue, users’ profiling, merging music with social...
-
Music signal equalization in a changing environment
PublicationThe paper presents the concept of an automatic system for music signal correction, considering room frequency response and music genre being played. The proposed algorithm, based on the room frequency response, compensates acoustic conditions surrounding the sound source. Additionally, the compensation process considers the signal content by recognizing music genre. As part of the described research, a series of subjective tests...
-
New applications of sound and vision engineering
PublicationMultimedia, Sound & Vision Engineering are relatively new fields within the area of science and technology, but teaching and research in this area has been carried out at Gdansk University of Technology (Gdansk, Poland) for nearly 5 decades. Current project carried-out in the Multimedia Systems Department are in the scope of the paper.
-
Porównanie klasycznych i ewolucyjnych metod projektowanie dyfuzorów akustycznych w warunkach odsłuchowych symulowanych metodą FDTD
PublicationW niniejszym rozdzaiel przedstawione zostanie porównanie dwóch podejść do projektowania dyfuzorów akustycznych. Pierwsze z nich bazuje na klasycznych założeniach dotyczących wykorzystywania sekwewncji pseudolosowych. Drugie z nich wykorzystuje automatyczne podejście bazujące na wykorzystaniu algorytmów genetycznych. Porównanie takie pozwala na określenie zalet i wad podejśc klasycznych oraz podejśc bazujących na zastosowaniu algorytmu...
-
Post-comatose patients with minimal consciousness tend to preserve reading comprehension skills but neglect syntax and spelling
PublicationModern eye tracking technology provides a means for communication with patients suffering from disorders of consciousness (DoC) or remaining in locked-in-state. However, being able to use an eye tracker for controlling text-based contents by such patients requires preserved reading ability in the first place. To our knowledge, this aspect, although of great social importance, so far has seemed to be neglected. In the paper, we...
-
Projektowanie oraz implementacja cyfrowego multiefektu gitarowego z wykorzystaniem procesora sygnałowego
PublicationW artykule został przedstawiony proces projektowania i realizacji cyfrowego multiefektu gitarowego z wykorzystaniem procesora sygnałowegoTMS320C5535 firmy Texas Instruments, dla którego oprogramowanie napisano w języku C. Omówiono zasady działania oraz algorytmy wybranych efektów dźwiękowych, które zostały zaimplementowane w procesorze sygnałowym. Zaprojektowano również uniwersalny moduł wejściowy zawierający wzmacniacz z regulowanym...
-
Real and Virtual Instruments in Machine Learning – Training and Comparison of Classification Results
PublicationThe continuous growth of the computing power of processors, as well as the fact that computational clusters can be created from combined machines, allows for increasing the complexity of algorithms that can be trained. The process, however, requires expanding the basis of the training sets. One of the main obstacles in music classification is the lack of high-quality, real-life recording database for every instrument with a variety...
-
Recovering Sound Produced by Wind Turbine Structures Employing Video Motion Magnification
PublicationThe recordings were made with a fast video camera and with a microphone. Using fast cameras allowed for observation of the micro vibrations of the object structure. Motion-magnified video recordings of wind turbines on a wind farm were made for the purpose of building a damage prediction system. An idea was to use video to recover sound & vibrations in order to obtain a contactless diagnostic method for wind turbines. The recovered signals...
-
Relationship between album cover design and music genres.
PublicationThe aim of the study is to find out whether there exists a relationship between typographic, compositional and coloristic elements of the music album cover design and music contained in the album. The research study involves basic statistical analysis of the manually extracted data coming from the worldwide album covers. The samples represent 34 different music genres, coming from nine countries from around the world. There are...
-
Sound engineering as our commitment to its creators in Poland
PublicationSound engineering is an interdisciplinary and rapidly expanding domain. It covers many aspects, such as sound perception, studio and sound mastering technology, music information retrieval including content-based search systems and automatic music transcription frameworks, sound synthesis, sound restoration, electroacoustics, and other ones constituting multimedia technology. Moreover, machine learning methods applied to the topics...
-
Special techniques and future perspectives: Simultaneous macro- and micro-electrode recordings
PublicationThere are many approaches to studying the inner workings of the brain and its highly interconnected circuits. One can look at the global activity in different brain structures using non-invasive technologies like positron emission tomography (PET) or functional magnetic resonance imaging (fMRI), which measure physiological changes, e.g. in the glucose uptake or blood flow. These can be very effectively used to localize active patches...
-
Speech Analytics Based on Machine Learning
PublicationIn this chapter, the process of speech data preparation for machine learning is discussed in detail. Examples of speech analytics methods applied to phonemes and allophones are shown. Further, an approach to automatic phoneme recognition involving optimized parametrization and a classifier belonging to machine learning algorithms is discussed. Feature vectors are built on the basis of descriptors coming from the music information...
-
Style Transfer for Detecting Vehicles with Thermal Camera
PublicationIn this work we focus on nighttime vehicle detection for intelligent traffic monitoring from the thermal camera. To train a Convolutional Neural Network (CNN) detector we create a stylized version of COCO (Common Objects in Context) dataset using Style Transfer technique that imitates images obtained from thermal cameras. This new dataset is further used for fine-tuning of the model and as a result detection accuracy on images...