Search
Filters
total: 1385
filtered: 899
Search results for: sound extracted from video
-
Comparison of two methods of sound extraction from guitar string video recordings
PublicationA comparison of two sound extraction methods from guitar string video recordings is presented in the paper. A brief overview of highframe rate camera technology and possible applications are included. The method using the image analysis from two such cameras is presented. The cameras are placed at the angle of 90 degrees for recording the image in three planes. The results achieved...
-
New Aspects of Virtual Sound Source Localization Research—Impact of Visual Angle and 3-D Video Content on Sound Perception
PublicationThe influence of image on virtual sound source localization, called the “image proximity effect” or the “ventriloquism effect”, is a well known phenomenon. This paper focuses on other aspects related to this effect, namely the impact of the visual angle of the presented object and 3D video content on sound perception. The research conducted confirmed that the visual angle of the presented object determines the image proximity effect...
-
Recovering Sound Produced by Wind Turbine Structures Employing Video Motion Magnification
PublicationThe recordings were made with a fast video camera and with a microphone. Using fast cameras allowed for observation of the micro vibrations of the object structure. Motion-magnified video recordings of wind turbines on a wind farm were made for the purpose of building a damage prediction system. An idea was to use video to recover sound & vibrations in order to obtain a contactless diagnostic method for wind turbines. The recovered signals...
-
Automatic sound source localization in disturbing conditions using acoustic vector sensors
PublicationA concept, practical realization and applications of a passive acoustic radar to automatic localization and tracking of sound sources in disturbing conditions were presented in the paper. The device consists of the new kind of multichannel miniature sound intensity sensors and a group of digital signal processing algorithms. The sensitivity of the realized acoustic radar was examined in free sound field. Several kinds of sound...
-
Detection of moving objects in images combined from video and thermal cameras
PublicationAn algorithm for detection of moving objects in video streams from the monitoring cameras is presented. A system composed of a standard video camera and a thermal camera, mounted in close proximity to each other, is used for object detection. First, a background subtraction is performed in both video streams separately, using the popular Gaussian Mixture Models method. For the next processing stage, the authors propose an algorithm...
-
EXAMINING INFLUENCE OF VIDEO FRAMERATE AND AUDIO/VIDEO SYNCHRONIZATION ON AUDIO-VISUAL SPEECH RECOGNITION ACCURACY
PublicationThe problem of video framerate and audio/video synchronization in audio-visual speech recogni-tion is considered. The visual features are added to the acoustic parameters in order to improve the accuracy of speech recognition in noisy conditions. The Mel-Frequency Cepstral Coefficients are used on the acoustic side whereas Active Appearance Model features are extracted from the image. The feature fusion approach is employed. The...
-
EXAMINING INFLUENCE OF VIDEO FRAMERATE AND AUDIO/VIDEO SYNCHRONIZATION ON AUDIO-VISUAL SPEECH RECOGNITION ACCURACY
PublicationThe problem of video framerate and audio/video synchronization in audio-visual speech recognition is considered. The visual features are added to the acoustic parameters in order to improve the accuracy of speech recognition in noisy conditions. The Mel-Frequency Cepstral Coefficients are used on the acoustic side whereas Active Appearance Model features are extracted from the image. The feature fusion approach is employed. The...
-
Guitar String Sound Retrieved from Moving Pixels
PublicationThe aim of this study was to develop a method of visual recording and analyzing the vibrations of guitar strings using high-speed cameras and dedicated video processing algorithms. The recording of a plucked string reveals the way in which the deformations propagate, composing the standing and travelling wave. The paper compares the results for a few selected models of classical and acoustic guitars, and it involves processing...
-
Gesture-controlled Sound Mixing System With a Sonified Interface
PublicationIn this paper the Authors present a novel approach to sound mixing. It is materialized in a system that enables to mix sound with hand gestures recognized in a video stream. The system has been developed in such a way that mixing operations can be performed both with or without visual support. To check the hypothesis that the mixing process needs only an auditory display, the influence of audio information visualization on sound...
-
Improving automatic surveillance by sound analysis
PublicationAn automatic surveillance system, based on event detection in the video image can be improved by implementing algorithms for audio analysis. Dangerous or illegal actions are often connected with distinctive sound events like screams or sudden bursts of energy. A method for detection and classification of alarming sound events is presented. Detection is based on the observation of sudden changes in sound level in distinctive sub-bands...
-
Subjective tests for gathering konwledge for applaying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot,and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with...
-
Subjective tests for gathering knowledge for applying color grading to video clips automatically
PublicationThe analysis of film music concerning caused emotions may allow for a more accurate adaptation of the color of the film in the context of color grading. Therefore, this paper aims to gather knowledge on the correlation between the applied color palette to a video clip, music associated with a particular shot, and emotions evoked. For that purpose, subjective tests are prepared in which several video clips are presented with or...
-
Objectivization of audio-video correlation assessment experiments
PublicationThe purpose of this paper is to present a new method of conducting an audio-visual correlation analysis employing a head-motion-free gaze tracking system. First, a review of related works in the domain of sound and vision correlation is presented. Then assumptions concerning audio-visual scene creation are shortly described. The objectivization process of carrying out correlation tests employing gaze-tracking system is outlined....
-
Postprodukcja nagrania wideo z dzwiekiem dookolnym
PublicationOne of the aims of this paper is to present issues related to audio-video correlation. This is presented on the basis of a short film realization employing surround microphone techniques. First, some related works in the domain of sound and vision correlation are presented. Then assumptions concerning scene creation related to both audio and video are shortly described. Another objective is to discuss results of subjective tests...
-
Detection of vehicles stopping in restricted zones in video from surveillance cameras
PublicationAn algorithm for detection of vehicles that stop in restricted areas, e.g. excluded by traffic rules, is proposed. Classic approaches based on object tracking are inefficient in high traffic scenes because of tracking errors caused by frequent object merging and splitting. The proposed algorithm uses the background subtraction results for detection of moving objects, then pixels belonging to moving objects are tested for stability....
-
Acoustic radar employing particle velocity sensors
PublicationA concept, practical realization and applications of a passive acoustic radar to automatic localization, tracking of sound sources were presented in the paper. The device consist of the new kind of multichannel miniature sound intensity sensors and a group of digital signal processing algorithms. Contrary to active radars, it does not emit the scanning beam but after receiving surroundings sounds it provide information about the...
-
Eulerian motion magnification applied to structural health monitoring of wind turbines
PublicationSeveral types of defects may occur in wind turbines, as physical damage of blades or gearbox malfunction. A wind farm monitoring and damage prediction system is built to observe abnormal vibrations of elements of wind turbine: blades, nacelle, and tower. Contactless methods are developed which do not require turbine stopping. In this work, structural health monitoring of a wind turbine is evaluated using a conversion from the captured...
-
Standards on Cyber Security Assessment of Smart Grid
PublicationSecurity evaluation of communication systems in smart grid poses a great challenge to the developers and operators. In recent years many new smart grid standards were proposed, which paradoxically results in the difficulty in finding a relevant publication in this plethora of literature. This paper presents the results of a systematic analysis which aimed at addressing this issue by identifying standards that present sound security...
-
Visual Traffic Noise Monitoring in Urban Areas
PublicationThe paper presents an advanced system for railway and road traffic noise monitoring in metropolitan areas. This system is a functional part of a more complex solution designed for environmental monitoring in cities utilizing analyses of sound, vision and air pollution, based on a ubiquitous computing approach. The system consists of many autonomous, universal measuring units and a multimedia server, which gathers, processes and...
-
Methodology and technology for the polymodal allophonic speech transcription
PublicationA method for automatic audiovisual transcription of speech employing: acoustic and visual speech representations is developed. It adopts a combining of audio and visual modalities, which provide a synergy effect in terms of speech recognition accuracy. To establish a robust solution, basic research concerning the relation between the allophonic variation of speech, i.e. the changes in the articulatory setting of speech organs for...
-
Methodology and technology for the polymodal allophonic speech transcription
PublicationA method for automatic audiovisual transcription of speech employing: acoustic, electromagnetical articulography and visual speech representations is developed. It adopts a combining of audio and visual modalities, which provide a synergy effect in terms of speech recognition accuracy. To establish a robust solution, basic research concerning the relation between the allophonic variation of speech, i.e., the changes in the articulatory...
-
Visual Lip Contour Detection for the Purpose of Speech Recognition
PublicationA method for visual detection of lip contours in frontal recordings of speakers is described and evaluated. The purpose of the method is to facilitate speech recognition with visual features extracted from a mouth region. Different Active Appearance Models are employed for finding lips in video frames and for lip shape and texture statistical description. Search initialization procedure is proposed and error measure values are...
-
Traffic Noise Analysis Applied to Automatic Vehicle Counting and Classification
PublicationProblems related to determining traffic noise characteristics are discussed in the context of automatic dynamic noise analysis based on noise level measurements and traffic prediction models. The obtained analytical results provide the second goal of the study, namely automatic vehicle counting and classification. Several traffic prediction models are presented and compared to the results of in-situ noise level measurements. Synchronized...
-
The Innovative Faculty for Innovative Technologies
PublicationA leaflet describing Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology. Multimedia Systems Department described laboratories and prototypes of: Auditory-visual attention stimulator, Automatic video event detection, Object re-identification application for multi-camera surveillance systems, Object Tracking and Automatic Master-Slave PTZ Camera Positioning System, Passive Acoustic Radar,...
-
Extraction of stable foreground image regions for unattended luggage detection
PublicationA novel approach to detection of stationary objects in the video stream is presented. Stationary objects are these separated from the static background, but remaining motionless for a prolonged time. Extraction of stationary objects from images is useful in automatic detection of unattended luggage. The proposed algorithm is based on detection of image regions containing foreground image pixels having stable values in time and...
-
Layered background modeling for automatic detection of unattended objects in camera images
PublicationAn algorithm for automatic detection of unattended objects in video camera images is presented. First, background subtraction is performed, using an approach based on the codebook method. Results of the detection are then processed by assigning the background pixels to time slots, based on the codeword age. Using this data, moving objects detected during a chosen period may be extracted from the background model. The proposed approach...
-
Counting and tracking vehicles using acoustic vector sensors
PublicationA method is presented for counting vehicles and for determining their movement direction by means of acoustic vector sensor application. The assumptions of the method employing spatial distribution of sound intensity determined with the help of an integrated 3D intensity probe are discussed. The intensity probe developed by the authors was used for the experiments. The mode of operation of the algorithm is presented in conjunction...
-
Application of autoencoder to traffic noise analysis
PublicationThe aim of an autoencoder neural network is to transform the input data into a lower-dimensional code and then to reconstruct the output from this code representation. Applications of autoencoders to classifying sound events in the road traffic have not been found in the literature. The presented research aims to determine whether such an unsupervised learning method may be used for deploying classification algorithms applied to...
-
Multimodal English corpus for automatic speech recognition
PublicationA multimodal corpus developed for research of speech recognition based on audio-visual data is presented. Besides usual video and sound excerpts, the prepared database contains also thermovision images and depth maps. All streams were recorded simultaneously, therefore the corpus enables to examine the importance of the information provided by different modalities. Based on the recordings, it is also possible to develop a speech...
-
Broadening the scope of measurement and analysis of vibrations of an organ pipe employing intensity probe, simulations, and highspeed camera
PublicationThis paper shows an integrated approach to measure, analyze, and model phenomena occurring in an organ pipe driven by pressurized air. The aim of this paper is two-fold, i.e., to measure the pressure signal and the intensity field around the mouth by means of an intensity probe and to visualize and observe the motion of the air jet, which represents the excitation mechanism of the system. This is realized through two techniques,...
-
On the Consumption of Multimedia Content Using Mobile Devices: a Year to Year User Case Study
PublicationIn the early days, consumption of multimedia content related with audio signals was only possible in a stationary manner. The music player was located at home, with a necessary physical drive. An alternative way for an individual was to attend a live performance at a concert hall or host a private concert at home. To sum up, audio-visual effects were only reserved for a narrow group of recipients. Today, thanks to portable players,...
-
A comparative study of English viseme recognition methods and algorithm
PublicationAn elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector...
-
A comparative study of English viseme recognition methods and algorithms
PublicationAn elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector construction...
-
Monitoring of Caged Bluefin Tuna Reactions to Ship and Offshore Wind Farm Operational Noises
PublicationUnderwater noise has been identified as a relevant pollution affecting marine ecosystems in different ways. Despite the numerous studies performed over the last few decades regarding the adverse effect of underwater noise on marine life, a lack of knowledge and methodological procedures still exists, and results are often tentative or qualitative. A monitoring methodology for the behavioral response of bluefin tuna (Thunnus thynnus)...
-
Verification of the Parameterization Methods in the Context of Automatic Recognition of Sounds Related to Danger
PublicationW artykule opisano aplikację, która automatycznie wykrywa zdarzenia dźwiękowe takie jak: rozbita szyba, wystrzał, wybuch i krzyk. Opisany system składa się z bloku parametryzacji i klasyfikatora. W artykule dokonano porównania parametrów dedykowanych dla tego zastosowania oraz standardowych deskryptorów MPEG-7. Porównano też dwa klasyfikatory: Jeden oparty o Percetron (sieci neuronowe) i drugi oparty o Maszynę wektorów wspierających....
-
An Empirical Study on the Impact of Gender on Mobile Applications Usability
PublicationIn the area of broadband wireless Internet, mobile applications have already replaced their desktop equivalents and are recognized as valuable tools for any size of businesses and for private use. With the emergence of millions of apps, the quality of their interaction with the user remains an open question for software vendors. While female and male requirements and preferences are not always similar, to the best of our knowledge,...
-
Audio content analysis in the urban area telemonitoring system
PublicationArtykuł przedstawia możliwości rozwinięcie monitoringu miejskiego o automatyczną analizę dźwięku. Przedstawiono metody parametryzacji dźwięku, które możliwe są do zastosowania w takim systemie oraz omówiono aspekty techniczne implementacji. W kolejnej części przedstawiono system decyzyjny oparty na drzewach zastosowany w systemie. System ten rozpoznaje dźwięki niebezpieczne (strzał, rozbita szyba, krzyk) wśród dźwięków zarejestrowanych...
-
Affect aware video games
PublicationIn this chapter a problem of affect aware video games is described, including such issue as: emotional model of the player, design, development and UX testing of affect-aware video games, multimodal emotion recognition and a featured review of affect-aware video games.
-
Reversible Video Stream Anonymization for Video Surveillance Systems Based on Pixels Relocation and Watermarking
PublicationA method of reversible video image regions of interest anonymization for applications in video surveillance systems is described. A short introduction to theanonymization procedures is presented together with the explanation of its relation to visual surveillance. A short review of state of the art of sensitive data protection in media is included. An approach to reversible Region of Interest (ROI) hiding in video is presented,...
-
Superresolution algorithm to video surveillance system
PublicationAn application of a multiframe SR (superresolution) algorithm applied to video monitoring is described. The video signal generated by various types of video cameras with different parameters and signal distortions which may be very problematic for superresolution algorithms. The paper focuses on disadvantages in video signal which occur in video surveillance systems. Especially motion estimation and its influence on superresolution...
-
The nonlinear effects of sound in a liquid with relaxation losses
PublicationThe nonlinear effects of sound in electrolyte with a chemical reaction are examined. The dynamic equations that govern non-wave modes in the field of intense sound are derived, and acoustic forces of vortex, entropy, and relaxation modes are determined in the cases of low-frequency sound and high-frequency sound. The difference in the nonlinear effects of sound in electrolyte and in a gas with excited vibrational degrees of molecules,...
-
generation of the vorticity mode by sound in a bingham plastic
PublicationThis study investigates interaction between acoustic and non-acoustic modes, such as vorticity mode,in some class of a non-newtonian fluid called Bingham plastic. The instantaneous equations describinginteraction between different modes are derived. The attention is paid to the nonlinear effects in the fieldof intense sound. The resulting equations which describe dynamics of both sound and the vorticity modeapply to both periodic...
-
Multi-task Video Enhancement for Dental Interventions
PublicationA microcamera firmly attached to a dental handpiece allows dentists to continuously monitor the progress of conservative dental procedures. Video enhancement in video-assisted dental interventions alleviates low-light, noise, blur, and camera handshakes that collectively degrade visual comfort. To this end, we introduce a novel deep network for multi-task video enhancement that enables macro-visualization of dental scenes. In particular,...
-
Adaptive Personal Tuning of Sound in Mobile Computers
PublicationAn integrated methodology for enhancing audio quality in mobile computers is presented. The key features are adaptation of the characteristics of their acoustic track to changing acoustic conditions of the environment and to users’ individual preferences. Signal processing algorithms are introduced that concern: linearization of frequency response, dialogue intelligibility enhancement, and dynamics processing tuned up to the users’...
-
Analysis of the parameters of respiration patterns extracted from thermal image sequences
PublicationRemote estimation of vital signs is an important and active area of research. The goal of this work was to analyze the feasibility of estimating respiration parameters from video sequences of faces recorded using a mobile thermal camera. Different estimators were analyzed and experimentally verified. It was demonstrated that the respiration rate, periodicity of respiration, and presence and length of apnea periods could be reliably...
-
Systematic approach to binary classification of images in video streams using shifting time windows
Publicationin the paper, after pointing out of realistic recordings and classifications of their frames, we propose a new shifting time window approach for improving binary classifications. We consider image classification in tewo steps. in the first one the well known binary classification algorithms are used for each image separately. In the second step the results of the previous step mare analysed in relatively short sequences of consecutive...
-
Classifying type of vehicles on the basis of data extracted from audio signal characteristics
PublicationThe aim of this study is to find and optimize a feature vector for an automatic recognition of the type of vehicles, extracted form an audio signal. First, the influence of weather-based conditions of road surface on spectral characteristic of the audio signal recorded from a passing vehicle in close proximity to the road is discussed. Next, parameterization of the recorded audio signal is performed. For that purpose, the MIRtoolbox,...
-
Sound intensity distribution around organ pipe
PublicationThe aim of the paper was to compare acoustic field around the open and stopped organ pipes. The wooden organ pipe was located in the anechoic chamber and activated with a constant air flow, produced by an external air-compressor. Thus, long-term steady state response was possible to obtain. Multichannel acoustic vector sensor was used to measure the sound intensity distribution of radiated acoustic energy. Measurements have been...
-
Sound engineering as our commitment to its creators in Poland
PublicationSound engineering is an interdisciplinary and rapidly expanding domain. It covers many aspects, such as sound perception, studio and sound mastering technology, music information retrieval including content-based search systems and automatic music transcription frameworks, sound synthesis, sound restoration, electroacoustics, and other ones constituting multimedia technology. Moreover, machine learning methods applied to the topics...
-
Image Classification Based on Video Segments
PublicationIn the dissertation a new method for improving the quality of classifications of images in video streams has been proposed and analyzed. In multiple fields concerning such a classification, the proposed algorithms focus on the analysis of single frames. This class of algorithms has been named OFA (One Frame Analyzed).In the dissertation, small segments of the video are considered and each image is analyzed in the context of its...