Wyszukiwarka
Wyniki wyszukiwania dla: AUDIO ANALYSIS
-
Multimodal human-computer interfaces based on advanced video and audio analysis
PublikacjaMultimodal interfaces development history is reviewed briefly in the introduction. Examples of applications of multimodal interfaces to education software and for the disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and the audio interface for speech stretching for hearing impaired and stuttering people. The Smart...
-
Multimodal human-computer interfaces based on advanced video and audio analysis
PublikacjaMultimodal interfaces development history is reviewed briefly in the introduction. Some applications of multimodal interfaces to education software for disabled people are presented. One of them, the LipMouse is a novel, vision-based human-computer interface that tracks user’s lip movements and detect lips gestures. A new approach to diagnosing Parkinson’s disease is also shown. The progression of the disease can be measured employing...
-
Objectivization of Audio-Visual Correlation analysis
PublikacjaSimultaneous perception of audio and visual stimuli often causes the concealment or misrepresentation of information actually contained in these stimuli. Such effects are called the ''image proximity effect'' or the ''ventriloquism effect'' in literature. Until recently, most research carried out to understand their nature was based on subjective assessments. The Authors of this paper propose a methodology based on both subjective...
-
An new method of audio-visual correlation analysis
PublikacjaThis paper presents a new methodology of conducting the audio-visual correlation analysis employing the gaze tracking system. Interaction between two perceptual modalities, seeing and hearing, their interaction and mutual reinforcement in a complex relationship was a subject of many research studies. Earlier stage of the carried out experiments at the Multimedia Systems Department (MSD) showed that there exists a relationship between...
-
Audio content analysis in the urban area telemonitoring system
PublikacjaArtykuł przedstawia możliwości rozwinięcie monitoringu miejskiego o automatyczną analizę dźwięku. Przedstawiono metody parametryzacji dźwięku, które możliwe są do zastosowania w takim systemie oraz omówiono aspekty techniczne implementacji. W kolejnej części przedstawiono system decyzyjny oparty na drzewach zastosowany w systemie. System ten rozpoznaje dźwięki niebezpieczne (strzał, rozbita szyba, krzyk) wśród dźwięków zarejestrowanych...
-
Using concentrated spectrogram for analysis of audio acoustic signals
PublikacjaThe paper presents results of time-frequency analysis of audio acoustic signals using the method of Concentrated Spectrograph also known as ''Cross-spectral method'' or ''Reassignment method''. Presented algorithm involves signal's local group delay and channelized instantaneous frequency to relevantly redistribute all Short-time Fourier transform lines in time-frequency plain. The main intention of the paper is to compare various...
-
Analysis of allophones based on audio signal recordings and parameterization
PublikacjaThe aim of this study is to develop an allophonic description of English plosive consonants based on recordings of 600 specially selected words. Allophonic variations addressed in the study may have two sources: positional and contextual. The former one depends on the syllabic or prosodic position in which a particular phoneme occurs. Contextual allophony is conditioned by the local phonetic environment. Co-articulation overlapping...
-
Audio Feature Analysis for Precise Vocalic Segments Classification in English
PublikacjaAn approach to identifying the most meaningful Mel-Frequency Cepstral Coefficients representing selected allophones and vocalic segments for their classification is presented in the paper. For this purpose, experiments were carried out using algorithms such as Principal Component Analysis, Feature Importance, and Recursive Parameter Elimination. The data used were recordings made within the ALOFON corpus containing audio signal...
-
Analysis of impact of audio modifications on the robustness of watermark for non-blind architecture
PublikacjaThe aim of this paper is to assess the robustness of the non-blind audio content watermarking scheme proposed by the authors. The authors present the architecture of the designed system along with the employed workflows for embedding and extracting the watermark followed by the implementation phase description and the analysis of the experimental results. Some possible attack simulations on the embedded watermarks are reviewed,...
-
Analysis of the Usefulness of Cheap Audio Recorders for Spectral Measurement of Environmental Noise
PublikacjaEnvironmental noise pollution is nowadays one of the most serious health threats. The impact of noise on the human body depends not only on the sound level but also on its spectral distribution. Reliable measurements of the environmental noise spectrum are often hampered by the very high price of top quality measuring devices. This paper explores the possibility of using much cheaper audio recorders for the frequency analysis....
-
Quality Analysis of Audio-Video Transmission in an OFDM-Based Communication System
PublikacjaApplication of a reliable audio-video communication system, brings many advantages. With the spoken word we can exchange ideas, provide descriptive information, as well as aid to another person. With the availability of visual information one can monitor the surrounding, working environment, etc. As the amount of available bandwidth continues to shrink, researchers focus on novel types of transmission. Currently, orthogonal frequency...
-
Further developments of parameterization methods of audio stream analysis for secuirty purposes
PublikacjaThe paper presents an automatic sound recognition algorithm intended for application in an audiovisual security monitoring system. A distributed character of security systems does not allow for simultaneous observation of multiple multimedia streams, thus an automatic recognition algorithm must be introduced. In the paper, a module for the parameterization and automatic detection of audio events is described. The spectral analyses...
-
A study on of music features derived from audio recordings examples – a quantitative analysis
PublikacjaThe paper presents a comparative study of music features derived from audio recordings, i.e. the same music pieces but representing different music genres, excerpts performed by different musicians, and songs performed by a musician, whose style evolved over time. Firstly, the origin and the background of the division of music genres were shortly presented. Then, several objective parameters of an audio signal were recalled that...
-
Gaze-tracking based audio-visual correlation analysis employing quality of experience methodology
PublikacjaThis paper investigates a new approach to audio-visual correlation assessment based on the gaze-tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). The gaze-tracking methodology, having roots in Human-Computer Interaction borrows the relevance feedback through gaze-tracking and applies it to the new area of interests, which is Quality of Experience. Results of subjective...
-
Analysis of impact of lossy audio compression on the robustness of watermark embedded in the DWT domain for non-blind copyright protection
PublikacjaA methodology of non-blind watermarking of the audio content is proposed. The outline of audio copyright problem and motivation for practical applications are discussed. The algorithmic theory pertaining watermarking techniques is briefly introduced. The system architecture together with employed workflows for embedding and extracting the watermarks are described. The implemented approach is described and obtained results are reported....
-
Automatic audio-visual threat detection
PublikacjaThe concept, practical realization and application of a system for detection and classification of hazardous situations based on multimodal sound and vision analysis are presented. The device consists of new kind multichannel miniature sound intensity sensors, digital Pan Tilt Zoom and fixed cameras and a bundle of signal processing algorithms. The simultaneous analysis of multimodal signals can significantly improve the accuracy...
-
Intelligent video and audio applications for learning enhancement
PublikacjaThe role of computers in school education is briefly discussed. Multimodal interfaces development history is shortly reviewed. Examples of applications of multimodal interfaces for learners with special educational needs are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with facial expression and speech stretching audio interface representing audio modality....
-
Objectivization of audio-video correlation assessment experiments
PublikacjaThe purpose of this paper is to present a new method of conducting an audio-visual correlation analysis employing a head-motion-free gaze tracking system. First, a review of related works in the domain of sound and vision correlation is presented. Then assumptions concerning audio-visual scene creation are shortly described. The objectivization process of carrying out correlation tests employing gaze-tracking system is outlined....
-
Exploiting audio-visual correlation by means of gaze tracking
PublikacjaThis paper presents a novel means for increasing audio-visual correlation analysis reliability. This is done based on gaze tracking technology engineered at the Multimedia Systems Department of the Gdansk University of Technology, Poland. In the paper, the past history and current research in the area of audio-visual perception analysis are shortly reviewed. Then the methodology employing gaze tracking is presented along with the...
-
Elimination of Impulsive Disturbances From Archive Audio Signals Using Bidirectional Processing
PublikacjaIn this application-oriented paper we consider the problem of elimination of impulsive disturbances, such as clicks, pops and record scratches, from archive audio recordings. The proposed approach is based on bidirectional processing—noise pulses are localized by combining the results of forward-time and backward-time signal analysis. Based on the results of specially designed empirical tests (rather than on the results of theoretical analysis),...
-
Audio-visual aspect of the Lombard effect and comparison with recordings depicting emotional states.
PublikacjaIn this paper an analysis of audio-visual recordings of the Lombard effect is shown. First, audio signal is analyzed indicating the presence of this phenomenon in the recorded sessions. The principal aim, however, was to discuss problems related to extracting differences caused by the Lombard effect, present in the video , i.e. visible as tension and work of facial muscles aligned to an increase in the intensity of the articulated...
-
Automatic audio signal mixing system based on one-dimensional Wave-U-Net autoencoders
PublikacjaThe purpose of this dissertation is to develop an automatic song mixing system that is capable of automatically mixing a song with good quality in any music genre. This work recalls first the audio signal processing methods used in audio mixing, and it describes selected methods for automatic audio mixing. Then, a novel architecture built based on one-dimensional Wave-U-Net autoencoders is proposed for automatic music mixing. Models...
-
AUDIO SIGNAL EQUALIZATION BASED ON IMPULSE RESPONSE OF A LISTENING ROOM AND MUSIC CONTENT REPRODUCED
PublikacjaA research study presents investigations of the influence of the room acoustics on the frequency characteristic of the audio signal playback. First, a concept of a novel spectral equalization method of the room acoustic conditions is introduced. On the basis of the room spectral response, a system for room acoustics compensation based on an equalizer designed is proposed. The system settings depend on music genre recognized automatically....
-
New semi-causal and noncausal techniques for detection of impulsive disturbances in multivariate signals with audio applications
PublikacjaThis paper deals with the problem of localization of impulsive disturbances in nonstationary multivariate signals. Both unidirectional and bidirectional (noncausal) detection schemes are proposed. It is shown that the strengthened pulse detection rule, which combines analysis of one-step-ahead signal prediction errors with critical evaluation of leave-one-out signal interpolation errors, allows one to noticeably improve detection results...
-
Elimination of impulsive disturbances from archive audio files – comparison of three noise pulse detection schemes
PublikacjaThe problem of elimination of impulsive disturbances (such as clicks, pops, ticks, crackles, and record scratches) from archive audio recordings is considered and solved using autoregressive modeling. Three classical noise pulse detection schemes are examined and compared: the approach based on open-loop multi-step-ahead signal prediction, the approach based on decision-feedback signal prediction, and the double threshold approach,...
-
Piotr Szczuko dr hab. inż.
OsobyDr hab. inż. Piotr Szczuko w 2002 roku ukończył studia na Wydziale Elektroniki, Telekomunikacji i Informatyki Politechniki Gdańskiej zdobywając tytuł magistra inżyniera. Tematem pracy dyplomowej było badanie zjawisk jednoczesnej percepcji obrazu cyfrowego i dźwięku dookólnego. W roku 2008 obronił rozprawę doktorską zatytułowaną "Zastosowanie reguł rozmytych w komputerowej animacji postaci", za którą otrzymał nagrodę Prezesa Rady...
-
EXAMINING INFLUENCE OF VIDEO FRAMERATE AND AUDIO/VIDEO SYNCHRONIZATION ON AUDIO-VISUAL SPEECH RECOGNITION ACCURACY
PublikacjaThe problem of video framerate and audio/video synchronization in audio-visual speech recogni-tion is considered. The visual features are added to the acoustic parameters in order to improve the accuracy of speech recognition in noisy conditions. The Mel-Frequency Cepstral Coefficients are used on the acoustic side whereas Active Appearance Model features are extracted from the image. The feature fusion approach is employed. The...
-
EXAMINING INFLUENCE OF VIDEO FRAMERATE AND AUDIO/VIDEO SYNCHRONIZATION ON AUDIO-VISUAL SPEECH RECOGNITION ACCURACY
PublikacjaThe problem of video framerate and audio/video synchronization in audio-visual speech recognition is considered. The visual features are added to the acoustic parameters in order to improve the accuracy of speech recognition in noisy conditions. The Mel-Frequency Cepstral Coefficients are used on the acoustic side whereas Active Appearance Model features are extracted from the image. The feature fusion approach is employed. The...
-
Retrospecting Polish Audio Engineering Society Membership on 20th Anniversary of the Polish Section of the Audio Engineering Society
PublikacjaIn this article some key events concerning founding Polish Section of the Audio Engineering Society were presented. In addition, the history covering International Symposia on Sound Engineering and Mastering was outlined. Also, papers contained in this issue were shortly reviewed.
-
Measurement of Latency in the Android Audio Path
PublikacjaThis paper provides a description of experimental investigations concerning comparison between the audio path characteristics of various Android versions. First, information about the changes in each system version in the context of latency caused by them is presented. Then, a measurement procedure employing available applications to measure latency is described comparing to results contained in the Internet. Finally, a comparison...
-
Data Analysis in Bridge of Data
PublikacjaThe chapter presents the data analysis aspects of the Bridge of Data project. The software framework used, Jupyter, and its configuration are presented. The solution’s architecture, including the TRYTON supercomputer as the underlying infrastructure, is described. The use case templates provided by the Stat-reducer application are presented, including data analysis related to spatial points’ cloud-, audio- and wind-related research.
-
Detection of impulsive disturbances in archive audio signals
PublikacjaIn this paper the problem of detection of impulsive disturbances in archive audio signals is considered. It is shown that semi-causal/noncausal solutions based on joint evaluation of signal prediction errors and leave-one-out signal interpolation errors, allow one to noticeably improve detection results compared to the prediction-only based solutions. The proposed approaches are evaluated on a set of clean audio signals contaminated...
-
Personal adaptive tuning of mobile computer audio
PublikacjaAn integrated methodology for enhancing audio quality in mobile computers is presented. The key features are adaptation of the characteristics of the acoustic track to the changing conditions and to the user's individual preferences. Original signal processing algorithms are introduced, which concern: linearization of frequency response, dialogue intelligibility enhancement and dynamics processing tuned up to the user's preferences....
-
A Study on Audio Signal Processed by "Instant Mastering"
PublikacjaAn increasing amount of music produced in home- and project-studios results in development and growth of "automatic mastering services". The presented investigation explores changes introduced to audio signal by various online mastering platforms. A music set consisting of 10 songs produced in small facilities was processed by eight on-line automatic mastering services. Additionally, some laboratory-constructed signals were tested....
-
Elimination of impulsive disturbances from stereo audio recordings
PublikacjaThis paper presents a new approach to elimination of impulsive disturbances from stereo audio recordings. The proposed solution is based on vector autoregressive modeling of audio signals. On-line tracking of signal model parameters is performed using the stability-preserving Whittle-Wiggins-Robinson algorithm with exponential data weighting. Detection of noise pulses and model-based interpolation of the irrevocably distorted samples...
-
Digital Audio Broadcasting or Webcasting: A Network Quality Perspective
PublikacjaIn recent years, many alternative technologies of delivering audio content have emerged, with different advantages and disadvantages. In this paper pros and cons of digital audio broadcasting and webcasting transmission techniques in a network quality perspective are described. A case study of user expectations with respect to currently available services is analyzed, and the perceived quality of real digital broadcasted and webcasted...
-
Automatic system for audio-video material reconstruction and archiving
PublikacjaReferat przedstawia propozycję modelu systemu automatycznej archiwizacji i rekonstrukcji nagrań audio-wideo. Założeniem tego rozwiązania jest uczynienie procesu rekonstrukcji nagrań bardziej niezależnym od człowieka. Ma to na celu redukcję kosztów rekonstrukcji przetwarzanych nagrań. Z powodu dużej liczby archiwalnych nagrań audio-wideo istnieje potrzeba stworzenia systemu który umożliwi automatyczną indeksację ich treści. Pomoże...
-
An audio-visual corpus for multimodal automatic speech recognition
Publikacjareview of available audio-visual speech corpora and a description of a new multimodal corpus of English speech recordings is provided. The new corpus containing 31 hours of recordings was created specifically to assist audio-visual speech recognition systems (AVSR) development. The database related to the corpus includes high-resolution, high-framerate stereoscopic video streams from RGB cameras, depth imaging stream utilizing Time-of-Flight...
-
Multimodal Audio-Visual Recognition of Traffic Events
PublikacjaPrzedstawiono demonstrator systemu wykrywania niebezpiecznych zdarzeń w ruchu drogowym oparty na jednoczesnej analizie danych wizyjnych i akustycznych. System jest częścią systemu automatycznego nadzoru bezpieczeństwa. Wykorzystuje on kamery i mikrofony jako źródła danych. Przedstawiono wykorzystane algorytmy - algorytmy rozpoznawania zdarzeń dźwiękowych oraz analizy obrazu. Zaprezentowano wyniki działania algorytmów na przykładzie...
-
A double-talk detector using audio watermarking
Publikacjaa novel approach to double-talk detection in the acoustic echo canceler is proposed. a hidden signature is embedded into the arriving signal, using the echo-hiding method. next detection of the presence of this signature in the microphone signal is performed. the results of the signature detection may be used by the acoustic echo canceler to stop or restart the adaptation process.
-
Adaptive filter for reconstruction of stereo audio signals.
PublikacjaArtykuł poświęcony jest omówieniu metody rekonstrukcji zakłóconych impulsowo sygnałów stereofonicznych. W pracy zdefiniowano model sygnału stereofonicznego i przedstawiono zaprojektowany dla tego modelu filtr Kalmana. Przedstawiono modyfikacje filtru, w wyniku których algorytm dokonuje rekonstrukcji zakłóconego impulsowo sygnału w jednym kanale z wykorzystaniem dodatkowej informacji zawartej w niezakłóconych próbkach sygnału pochodzącego...
-
Intelligent algorithms for optical track audio restoration
PublikacjaW referacie przedstawiono dwa algorytmy dedykowane redukcji pasożytniczych zniekształceń dźwięku spotykanych w optycznych ścieżkach dźwiękowych. Pierwszy algorytm umożliwia redukcję szerokopasmowego szumu w nagraniach fonicznych. Wykorzystano w nim psycho-akustyczny model słuchu oparty o miarę nieprzewidywalność sygnału (ang. Unpredictability Measure). Ocena jakości redukcji szumu została wykonana z wykorzystaniem metod inteligentnych....
-
A Device for Measuring Auditory Brainstem Responses to Audio
PublikacjaStandard ABR devices use clicks and tone bursts to assess subjects’ hearing in an objective way. A new device was developed that extends the functionality of a standard ABR audiometer by collecting and analyzing auditory brainstem responses (ABR). The developed accessory allows for the use of complex sounds (e.g., speech or music excerpts) as stimuli. Therefore, it is possible to find out how efficiently different types of sounds...
-
Testing Watermark Robustness against Application of Audio Restoration Algorithms
PublikacjaThe purpose of this study was to test to what extent watermarks embedded in distorted audio signals are immune to audio restoration algorithm performing. Several restoration routines such as noise reduction, spectrum expansion, clipping or clicks reduction were applied in the online website system. The online service was extended with some copyright protection mechanisms proposed by the authors. They contain low-level music features...
-
Localization of impulsive disturbances in audio signals using template matching
PublikacjaIn this paper, a new solution to the problem of elimination of impulsive disturbances from audio signals, based on the matched filtering technique, is proposed. The new approach stems from the observation that a large proportion of noise pulses corrupting audio recordings have highly repetitive shapes that match several typical “patterns”. In many cases a representative set of exemplary pulse waveforms can be extracted from the...
-
Objectivization of phonological evaluation of speech elements by means of audio parametrization
PublikacjaThis study addresses two issues related to both machine- and subjective-based speech evaluation by investigating five phonological phenomena related to allophone production. Its aim is to use objective parametrization and phonological classification of the recorded allophones. These allophones were selected as specifically difficult for Polish speakers of English: aspiration, final obstruent devoicing, dark lateral /l/, velar nasal...
-
Audio-visual surveillance system for application in bank operating room
PublikacjaAn audio-visual surveillance system able to detect, classify and to localize acoustic events in a bank operating room is presented. Algorithms for detection and classification of abnormal acoustic events, such as screams or gunshots are introduced. Two types of detectors are employed to detect impulsive sounds and vocal activity. A Support Vector Machine (SVM) classifier is used to discern between the different classes of acoustic...
-
Quality Evaluation of Novel DTD Algorithm Based on Audio Watermarking
PublikacjaEcho cancellers typically employ a doubletalk detection (DTD) algorithm in order to keep the adaptive filter from diverging in the presence of near-end speech signal or other disruptive sounds in the microphone signal. A novel doubletalk detection algorithm based on techniques similar to those used for audio signal watermarking was introduced by the authors. The application of the described DTD algorithm within acoustic echo cancellation...
-
Wireless intelligent audio-video surveillance prototyping system
PublikacjaThe presented system is based on the Virtex6 FPGA and several supporting devices like a fast DDR3 memory, small HD camera, microphone with A/D converter, WiFi radio communication module, etc. The system is controlled by the Linux operating system. The Linux drivers for devices implemented in the system have been prepared. The system has been successfully verified in a H.264 compression accelerator prototype in which the most demanding...
-
Audio codec employing frequency-derived tonality measure
PublikacjaA transform codec employing efficient algorithm for detection of spectral tonal components is presented. The tonality measure used in MPEG psychoacoustic model is replaced with the method providing adequate tonality estimates even if the tonal components are deeply frequency modulated. The reliability of hearing threshold estimated using psychoacoustic model with standardized tonality measure and the proposed one is investigated...