Wyszukiwarka
Katedra Systemów Multimedialnych
Publikacje
Filtry
wszystkich: 902
Katalog Publikacji
Rok 2018
-
Selection of Features for Multimodal Vocalic Segments Classification
PublikacjaEnglish speech recognition experiments are presented employing both: audio signal and Facial Motion Capture (FMC) recordings. The principal aim of the study was to evaluate the influence of feature vector dimension reduction for the accuracy of vocalic segments classification employing neural networks. Several parameter reduction strategies were adopted, namely: Extremely Randomized Trees, Principal Component Analysis and Recursive...
-
Sound quality metrics applied to road noise evaluation
PublikacjaRoad noise monitoring systems typically measure sound levels in specific time periods. The more insightful approach suggests to measure also the nature of noise. Sound quality of sounds such as car noise can be objectively evaluated by several parameters. One of them is psychoacoustic annoyance, described by loudness, tone color, and the temporal structure of sound. In this paper the assessment of several sound quality parameters, such...
-
Support Vector Machine Applied to Road Traffic Event Classification
PublikacjaThe aim of this paper is to present results of road traffic event signal recognition. First, several types of systems for road traffic monitoring, including Intelligent Transport System (ITS) are shortly described. Then, assumptions of creating a database of vehicle signals recorded in different weather and road conditions are outlined. Registered signals were edited as single vehicle pass by. Using the Matlab-based application...
-
Suppression of distortions in signals received from Doppler sensor for vehicle speed measurement
PublikacjaDoppler sensors are commonly used for movement detection and speed measurement. However, electromagnetic interference and imperfections in sensor construction result in degradation of the signal to noise ratio. As a result, detection of signals reflected from moving objects becomes problematic. The paper proposes an algorithm for reduction of distortions and noise in the signal received from a simple, dual-channel type of a Doppler...
-
The influence of sound track on the viewer’s emotions and correction of the color in the film
PublikacjaThe article presents the aspects of the final selection of colors in film production based on the emotions caused by the soundtrack of the film. First, the processing of colors, contrast, saturation and white balance of shots in the film was presented. The definition of color grading is also described, i.e. the color changes in the film's views. In the second part of the article, the soundtracks of the film were analyzed, in particular...
-
The influence of time of hearing aid use on auditory perception in various acoustic situations
PublikacjaThe assessment of sound perception in hearing aids, especially in the context of benefits that a prosthesis can bring, is a complex issue. The objective parameters of the hearing aids can easily be determined. These parameters, however, do not always have a direct and decisive influence on the subjective assessment of quality of the patient’s hearing while using a hearing aid. The paper presents the development of a method for...
-
Towards Audio Signal Equalization Based on Spectral Characteristics of a Listening Room and Music Content Reproduced
PublikacjaThis study presents investigations of the influence of the room acoustics on the frequency characteristic of the audio signal playback. First, the concept of a novel spectral equalization method of the room acoustic conditions is introduced. On the basis of the room spectral response, a system for room acoustics compensation based on an equalizer designed is proposed. The system settings depend on music genre recognized automatically....
-
Vehicle detector training with labels derived from background subtraction algorithms in video surveillance
PublikacjaVehicle detection in video from a miniature station- ary closed-circuit television (CCTV) camera is discussed in the paper. The camera provides one of components of the intelligent road sign developed in the project concerning the traffic control with the use of autonomous devices being developed. Modern Convolutional Neural Network (CNN) based detectors need big data input, usually demanding their manual labeling. In the presented...
-
Visual and Auditory Attention Stimulator for Assisting Pedagogical Therapy
PublikacjaVisual and auditory attention stimulator provides a system developed in order to improve reading skills using simultaneous presentation of text in its visual form and in transformed auditory form accompanied by related movie material. The described research employed 40 children at the age of 8 13 years having difficulties in learning of reading, who were diagnosed as having developmental dyslexia. It was shown that application...
-
Visual perception of vowels from static and dynamic cues
PublikacjaThe purpose of the study was to analyse human identification of Polish vowels from static and dynamic durationally slowed visual cues. A total of 152 participants identified 6 Polish vowels produced by 4 speakers from static (still images) and dynamic (videos) cues. The results show that 59% of static vowels and 63% of dynamic vowels were successfully identified. There was a strong confusion between vowels within front, central,...
-
Vocalic Segments Classification Assisted by Mouth Motion Capture
PublikacjaVisual features convey important information for automatic speech recognition (ASR), especially in noisy environment. The purpose of this study is to evaluate to what extent visual data (i.e. lip reading) can enhance recognition accuracy in the multi-modal approach. For that purpose motion capture markers were placed on speakers' faces to obtain lips tracking data during speaking. Different parameterizations strategies were tested...
-
WYKORZYSTANIE SIECI NEURONOWYCH DO SYNTEZY MOWY WYRAŻAJĄCEJ EMOCJE
PublikacjaW niniejszym artykule przedstawiono analizę rozwiązań do rozpoznawania emocji opartych na mowie i możliwości ich wykorzystania w syntezie mowy z emocjami, wykorzystując do tego celu sieci neuronowe. Przedstawiono aktualne rozwiązania dotyczące rozpoznawania emocji w mowie i metod syntezy mowy za pomocą sieci neuronowych. Obecnie obserwuje się znaczny wzrost zainteresowania i wykorzystania uczenia głębokiego w aplikacjach związanych...
-
ZASTOSOWANIE APLIKACJI INTERNETOWEJ W OCENIE JAKOŚCI DOPASOWANIA APARATÓW SŁUCHOWYCH
PublikacjaW pracy opisano zastosowanie aplikacji internetowej do oceny jakości dopasowania aparatów słuchowych. Metoda oceny polega na badaniu ankietowym, uzupełnionym testem rozumienia słów jednosylabowych w polu swobodnym. Opisywana aplikacja internetowa pozwala na przeprowadzenie badania z dowolnego komputera z dostępem do sieci. Dzięki implementacji metody w postaci aplikacji internetowej, można w systematyczny i uporządkowany sposób...
-
Zastosowanie sieci neuronowych w cyfrowej syntezie dźwięku
PublikacjaRozwój technik związanych z uczeniem maszynowym umożliwia nowe podejście i nowe definiowanie wielu dotychczasowych problemów. Heurystyczne algorytmy stosowane do problemów takich jak klasyfikacja danych w postaci wektorów cech, czy wyróżnianie grup obiektów o podobnych własnościach mogą znaleźć także zastosowanie w takich dziedzinach jak analiza i synteza dźwięków muzycznych. W referacie przybliżone zostały podstawowe zasady projektowania...
Rok 2017
-
A Method of Object Re-identiciation Applicable to Multicamera Surveillance Systems
PublikacjaThe paper addresses some challenges pertaining to the methods for tracking of objects in multi-camera systems. The tracking methods related to a single Field of Vision (FOV) are quite different from inter-camera tracking, especially in case of non-overlapping FOVs. In this case, the processing is directed to determine the probability of a particular object’s identity seen in a pair of cameras in the presence of places non-observed...
-
An audio-visual corpus for multimodal automatic speech recognition
Publikacjareview of available audio-visual speech corpora and a description of a new multimodal corpus of English speech recordings is provided. The new corpus containing 31 hours of recordings was created specifically to assist audio-visual speech recognition systems (AVSR) development. The database related to the corpus includes high-resolution, high-framerate stereoscopic video streams from RGB cameras, depth imaging stream utilizing Time-of-Flight...
-
Analysis of allophones based on audio signal recordings and parameterization
PublikacjaThe aim of this study is to develop an allophonic description of English plosive consonants based on recordings of 600 specially selected words. Allophonic variations addressed in the study may have two sources: positional and contextual. The former one depends on the syllabic or prosodic position in which a particular phoneme occurs. Contextual allophony is conditioned by the local phonetic environment. Co-articulation overlapping...
-
ANN for human pose estimation in low resolution depth images
PublikacjaThe paper presents an approach to localize human body joints in 3D coordinates based on a single low resolution depth image. First a framework to generate a database of 80k realistic depth images from a 3D body model is described. Then data preprocessing and normalization procedure, and DNN and MLP artificial neural networks architectures and training are presented. The robustness against camera distance and image noise is analysed....
-
Assessment of hearing in coma patients employing auditory brainstem response, electroencephalography, and eye-gaze-tracking
PublikacjaThe results of the study conducted by Tagliaferri et al. in 12 European countries indicate that the ratio of registered brain injury cases in Europe amounts to 150-300 per 100 000 people, with the European mean value of 235 cases per 100 000 people. The project presented in the paper assumes development of a combined metric of patients’ state remaining in coma by intelligent fusion of GCS (subjective Glasgow Coma Scale or its derivatives)...
-
Automatic music set organizatio based on mood of music / Automatyczna organizacja bazy muzycznej na podstawie nastroju muzyki
PublikacjaThis work is focused on an approach based on the emotional content of music and its automatic recognition. A vector of features describing emotional content of music was proposed. Additionally, a graphical model dedicated to the subjective evaluation of mood of music was created. A series of listening tests was carried out, and results were compared with automatic mood recognition employing SOM (Self Organizing Maps) and ANN (Artificial...
-
Badanie wierności brzmienia dźwięku instrumentów wirtualnych VST/TRTAS
PublikacjaTematem referatu jest subiektywne badanie wierności brzmienia instrumentów wirtualnych (VST/TRTAS) wykorzystujących próbkowanie dźwięków rzeczywistych instrumentów muzycznych. Na potrzeby przedstawionej pracy wybrano kilka utworów muzyki orkiestrowej z epoki romantyzmu i klasycyzmu, nagranych przy użyciu instrumentów akustycznych. Następnie zaaranżowano fragmenty tych utworów, wykorzystując do tego instrumenty wirtualne i efekty...
-
Building Knowledge for the Purpose of Lip Speech Identification
PublikacjaConsecutive stages of building knowledge for automatic lip speech identification are shown in this study. The main objective is to prepare audio-visual material for phonetic analysis and transcription. First, approximately 260 sentences of natural English were prepared taking into account the frequencies of occurrence of all English phonemes. Five native speakers from different countries read the selected sentences in front of...
-
Classifying type of vehicles on the basis of data extracted from audio signal characteristics
PublikacjaThe aim of this study is to find and optimize a feature vector for an automatic recognition of the type of vehicles, extracted form an audio signal. First, the influence of weather-based conditions of road surface on spectral characteristic of the audio signal recorded from a passing vehicle in close proximity to the road is discussed. Next, parameterization of the recorded audio signal is performed. For that purpose, the MIRtoolbox,...
-
Comparative Study of Self-Organizing Maps vs. Subjective Evaluation of Quality of Allophone Pronunciation for Nonnative English Speakers
PublikacjaThe purpose of this study was to apply Self-Organizing Maps to differentiate between the correct and the incorrect allophone pronunciations and to compare the results with subjective evaluation. Recordings of a list of target words, containing selected allophones of English plosive consonants, the velar nasal and the lateral consonant, were made twice. First, the target words were read from the list by 9 non-native speakers and...
-
Comparison of selected electroencephalographic signal classification methods
PublikacjaA variety of methods exists for electroencephalographic (EEG) signals classification. In this paper, we briefly review selected methods developed for such a purpose. First, a short description of the EEG signal characteristics is shown. Then, a comparison between the selected EEG signal classification methods, based on the overview of research studies on this topic, is presented. Examples of methods included in the study are: Artificial...
-
Detection of the Incoming Sound Direction Employing MEMS Microphones and the DSP
PublikacjaA 3D acoustic vector sensor based on MEMS microphones and its application to road traffic monitoring is presented in the paper. The sensor is constructed from three pairs of digital MEMS microphones, mounted on the orthogonal axes. Signals obtained from the microphones are used to compute sound intensity vectors in each direction. With this data, it is possible to compute the horizontal and vertical angle of an incoming sound....
-
Determining Pronunciation Differences in English Allophones Utilizing Audio Signal Parameterization
PublikacjaAn allophonic description of English plosive consonants, based on audio-visual recordings of 600 specially selected words, was developed. First, several speakers were recorded while reading words from a teleprompter. Then, every word was played back from the previously recorded sample read by a phonology expert and each examined speaker repeated a particular word trying to imitate correct pronunciation. The next step consisted...
-
Evaluation of Face Detection Algorithms for the Bank Client Identity Verification
PublikacjaResults of investigation of face detection algorithms efficiency in the banking client visual verification system are presented. The video recordings were made in real conditions met in three bank operating outlets employing a miniature industrial USB camera. The aim of the experiments was to check the practical usability of the face detection method in the biometric bank client verification system. The main assumption was to provide...
-
Examining Feature Vector for Phoneme Recognition / Analiza parametrów w kontekście automatycznej klasyfikacji fonemów
PublikacjaThe aim of this paper is to analyze usability of descriptors coming from music information retrieval to the phoneme analysis. The case study presented consists in several steps. First, a short overview of parameters utilized in speech analysis is given. Then, a set of time and frequency domain-based parameters is selected and discussed in the context of stop consonant acoustical characteristics. A toolbox created for this purpose...
-
Handwritten signature verification system employing wireless biometric pen
PublikacjaThe handwritten signature verification system being a part of the developed multimodal biometric banking stand is presented. The hardware component of the solution is described with a focus on the signature acquisition and on verification procedures. The signature is acquired employing an accelerometer and a gyroscope built-in the biometric pen plus pressure sensors for the assessment of the proper pen grip and then the signature...
-
Intelligent equalizer solution employing music genre and the room characteristics analysis
PublikacjaThe paper presents an intelligent equalizer solution based on room acoustic conditions and music genre analysis. A series of acoustic characteristic measurements are performed for checking the concept proposed. White noise (reference signal) and audio excerpts belonging to six music genres are utilized as excitation signals in measurements. This results in registration of frequency responses of rooms and reverberation times. Signals...
-
Komputerowe oko świadomości
PublikacjaZnane metody badania osób w śpiączce nie dają odpowiedzi na pytanie jak funkcjonuje poznawczo osoba wybudzona ze śpiączki z obniżoną świadomością. Książka podsumowuje wyniki badań przybliżających odpowiedź na powyższe pytanie. Część prezentowanych badań była prowadzona przez autorów i współpracowników już wcześniej, z wykorzystaniem skonstruowanego urządzenia do śledzenia wzroku, zaś nowsze prezentowane badania dostarczyły wyników...
-
Measurement and visualization of sound intensity vector distribution in proximity of acoustic diffusers
PublikacjaIn this work, we would like to present analyses and visualizations of sound intensity distribution measured in proximity of an acoustic diffuser. Such distribution may be used for estimation of basic acoustic parameters of a diffuser. Measurement is performed with the use of a logarithmic sine sweep which allows for the analysis of waves scattered by the diffuser and rejecting the direct sound signal component. Pressure and sound...
-
METODA OCENY EFEKTYWNOŚCI KRÓTKOTERMINOWEGO STOSOWANIA APARATÓW SŁUCHOWYCH Z WYKORZYSTANIEM APLIKACJI INTERNETOWEJ
PublikacjaW pracy przedstawiono opracowanie metody oceny efektywności protezowania osób niedosłyszących aparatami słuchowymi. Metoda polega na badaniu ankietowym opartym na kwestionariuszu oceny APHAB uzupełnionym testem rozumienia słów jednosylabowych w polu swobodnym. Uwzględniono dodatkowe kryteria, takie jak: stopień ubytku słuchu, pomiar liczby dni i godzin korzystania z aparatów słuchowych oraz doświadczenia pacjenta. Metoda została...
-
Modelowanie procesów słuchowych w celu oceny ryzyka uszkodzeń słuchu
PublikacjaW rozdziale przedstawiono sposób modelowania fizjologicznych procesów zachodzących w systemie słuchowym. Opisano szczegółowo autorską metodę oceny szkodliwego oddziaływania hałasu na słuch. W rozdziale opisano również kocepcję psychoakustycznej dozymetrii hałasowej. Bazuje ona na wyznaczaniu wartości czasowego przesunięcia progu słyszenia w pasmach krytycznych słuchu. Uwzględnia również efekty wywołane przez refleks akustyczny....
-
Modified dynamic time warping method applied to handwritten signature authenticity verification
PublikacjaA signature verification system based on static features and time-domain functions of signals obtained using a tablet has been presented in the paper. The signature verification method, based mainly on dynamic time warping coupled with some signature image features, has been described. The FRR measures reflecting the method’s efficiency have been evaluated for verification attempts performed directly after obtaining model signatures...
-
Multimodal Approach For Polysensory Stimulation And Diagnosis Of Subjects With Severe Communication Disorders
Publikacjais evaluated on 9 patients, data analysis methods are described, and experiments of correlating Glasgow Coma Scale with extracted features describing subjects performance in therapeutic exercises exploiting EEG and eyetracker are presented. Performance metrics are proposed, and k-means clusters used to define concepts for mental states related to EEG and eyetracking activity. Finally, it is shown that the strongest correlations...
-
Multimodal system for diagnosis and polysensory stimulation of subjects with communication disorders
PublikacjaAn experimental multimodal system, designed for polysensory diagnosis and stimulation of persons with impaired communication skills or even non-communicative subjects is presented. The user interface includes an eye tracking device and the EEG monitoring of the subject. Furthermore, the system consists of a device for objective hearing testing and an autostereoscopic projection system designed to stimulate subjects through their...
-
OCENA EFEKTYWNOŚCI KRÓTKOTERMINOWEGO ZASTOSOWANIA APARATÓW SŁUCHOWYCH Z WYKORZYSTANIEM APLIKACJI INTERNETOWEJ
PublikacjaW pracy przedstawiono opracowanie metody oceny efektywności protezowania osób niedosłyszących aparatami słuchowymi. Metoda polega na badaniu ankietowym opartym na kwestionariuszu oceny APHAB uzupełnionym testem rozumienia słów jednosylabowych w polu swobodnym. Uwzględniono dodatkowe kryteria, takie jak: stopień ubytku słuchu, po-miar liczby godzin i dni korzystania z aparatów słuchowych oraz doświadczenia pacjenta. Metoda została...
-
Performance Evaluation of Selected Parallel Object Detection and Tracking Algorithms on an Embedded GPU Platform
PublikacjaPerformance evaluation of selected complex video processing algorithms, implemented on a parallel, embedded GPU platform Tegra X1, is presented. Three algorithms were chosen for evaluation: a GMM-based object detection algorithm, a particle filter tracking algorithm and an optical flow based algorithm devoted to people counting in a crowd flow. The choice of these algorithms was based on their computational complexity and parallel...
-
Pilot Testing of Developed Multimodal Biometric Identity Verification System
PublikacjaThe bank client identity verification system developed in the course of the IDENT project is presented. The total number of five biometric modalities including: dynamic signature proofing, voice recognition, face image verification, face contour extraction and hand blood vessels distribution comparison have been developed and studied. The experimental data were acquired employing multiple biometric sensors installed at engineered...
-
Real and imaginary motion classification based on rough set analysis of EEG signals for multimedia applications
PublikacjaRough set-based approach to the classification of EEG signals of real and imaginary motion is presented. The pre-processing and signal parametrization procedures are described, the rough set theory is briefly introduced, and several classification scenarios and parameters selection methods are proposed. Classification results are provided and discussed with their potential utilization for multimedia applications controlled by the...
-
Road surface roughness estimation employing integrated position and acceleration sensor
PublikacjaAssessment of a surface quality being an essential task for the authorities supervising the roads provides the subject of the paper. Information about riding quality of a pavement, important for drivers, both in terms of their comfort and safety is collected during experiments employing mobile sensors. The paper describes the use of a miniature position and acceleration sensor for evaluation of the roughness of the road surface....
-
Shape-Based Pose Estimation of Robotic Surgical Instruments
PublikacjaWe describe a detector of robotic instrument parts in image-guided surgery. The detector consists of a huge ensemble of scale-variant and pose-dedicated, rigid appearance templates. The templates, which are equipped with pose-related keypoints and segmentation masks, allow for explicit pose estimation and segmentation of multiple end-effectors as well as fine-grained non-maximum suppression. We train the templates by grouping examples...
-
Sound intensity distribution around organ pipe
PublikacjaThe aim of the paper was to compare acoustic field around the open and stopped organ pipes. The wooden organ pipe was located in the anechoic chamber and activated with a constant air flow, produced by an external air-compressor. Thus, long-term steady state response was possible to obtain. Multichannel acoustic vector sensor was used to measure the sound intensity distribution of radiated acoustic energy. Measurements have been...
-
Stworzenie stereofonicznej ścieżki dźwiękowej do filmu symulującej dźwięk wielokanałowy
PublikacjaCelem referatu pracy jest przedstawienie procesu tworzenia stereofonicznej ścieżki dźwiękowej do filmu, symulującej dźwięk wielokanałowy w odsłuchu słuchawkowym. Opracowana symulacja dźwięku wielokanałowego wykorzystuje filtrację HRTF (ang. Head-Related-Transfer-Function). W celu umożliwienia jednoczesnego odsłuchu kilku partii instrumentalnych składających się na ścieżkę dźwiękową stworzona została aplikacja wraz z graficznym...
-
Surgical tool tracking by on-line selection of structural correlation filters
PublikacjaIn visual tracking of surgical instruments, correlation filtering finds the best candidate with maximal correlation peak. However, most trackers only consider capturing target appearance but not target structure. In this paper we propose surgical instrument tracking approach that integrates prior knowledge related to rotation of both shaft and tool tips. To this end, we employ rigid parts mixtures model of an instrument. The rigidly...
-
SYMULACJA DŹWIĘKU PRZESTRZENNEGO W ŚCIEŻCE DŹWIĘKOWEJ W ODSŁUCHU BINAURALNYM
PublikacjaCelem pracy jest przedstawienie aplikacji umożliwiającej tworzenie stereofonicznej ścieżki dźwiękowej do filmu, symulującej dźwięk przestrzenny w odsłuchu słuchawkowym. Interfejs przygotowanej aplikacji pozwala użytkownikowi na wybór rozmieszczenia konkretnych partii instrumentalnych w odpowiednich miejscach w przestrzeni dźwiękowej oraz jednoczesny odsłuch wszystkich ścieżek wraz z przygotowanym materiałem filmowym. Symulacja...
-
The project IDENT: Multimodal biometric system for bank client identity verification
PublikacjaBiometric identity verification methods are implemented inside a real banking environment comprising: dynamic handwritten signature verification, face recognition, bank cli-ent voice recognition and hand vein distribution verification. A secure communication system based on an intra-bank client-server architecture was designed for this purpose. Hitherto achieved progress within the project is reported in this paper with a focus...
-
Tracking Moving Objects in Video Surveillance Systems with Kalman and Particle Filters – A Practical Approach
PublikacjaThis Chapter focuses on the first type of object tracking algorithms, namely on Kalman and particle filters. A theory of these algorithms may be found in many publications, there are also reports on implementation of these approaches to object tracking in video. However, developers of VCA systems still face two important problems. The first one is related to obtaining accurate measurements of positions and sizes of the tracked...