Search
Filters
total: 561
filtered: 358
Search results for: audio-visual speech recognition system
-
Gesture-based computer control system
PublicationIn the paper a system for controlling computer applications by hand gestures is presented. First, selected methods used for gesture recognition are described. The system hardware and a way of controlling a computer by gestures are described. The architecture of the software along with hand gesture recognition methods and algorithms used are presented. Examples of basic and complex gestures recognized by the system are given.
-
Video Semantic Analysis Framework based on Run-time Production Rules - Towards Cognitive Vision
PublicationThis paper proposes a service-oriented architecture for video analysis which separates object detection from event recognition. Our aim is to introduce new tools to be considered in the pathway towards Cognitive Vision as a support for classical Computer Vision techniques that have been broadly used by the scientific community. In the article, we particularly focus in solving some of the reported scalability issues found in current...
-
A concept of Signal Equalization Method Based on Music Genre and the Listener's Room Characteristics
PublicationA research study that investigates the influence of the room acoustics environment on the frequency characteristic of the audio signal playback is presented. First, a novel spectral equalization method of the room acoustic conditions is introduced. On the basis of the frequency response of the room, a system for room acoustics compensation based on eight-band equalizer is proposed. The system settings depend on music genre. In...
-
Speech synthesis controlled by eye gazing
PublicationA method of communication based on eye gaze controlling is presented. Investigations of using gaze tracking have been carried out in various context applications. The solution proposed in the paper could be referred to as ''talking by eyes'' providing an innovative approach in the domain of speech synthesis. The application proposed is dedicated to disabled people, especially to persons in a so-called locked-in syndrome who cannot...
-
Innovative method of localization airplanes in VCS (VCS-MLAT) distributed system
PublicationThe article presents the concept and the structure of the localization module. The prototype module is the part of the VCS (VCS-MLAT) localization distributed system. The device receives the audio signal transmitted in airplanes band (118 MHz – 136 MHz). Received data with the timestamps are send to the main server. The data from multiple devices estimates the localization of the airplane. The main aim of the project is the analysis...
-
Study Analysis of Transmission Efficiency in DAB+ Broadcasting System
PublicationDAB+ is a very innovative and universal multimedia broadcasting system. Thanks to its updated multimedia technologies and metadata options, digital radio keeps pace with changing consumer expectations and the impact of media convergence. Broadcasting analog and digital radio services does vary, concerning devices on both transmitting and receiving side, as well as content processing mechanisms. However, the biggest difference is...
-
On the Consumption of Multimedia Content Using Mobile Devices: a Year to Year User Case Study
PublicationIn the early days, consumption of multimedia content related with audio signals was only possible in a stationary manner. The music player was located at home, with a necessary physical drive. An alternative way for an individual was to attend a live performance at a concert hall or host a private concert at home. To sum up, audio-visual effects were only reserved for a narrow group of recipients. Today, thanks to portable players,...
-
Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training
PublicationIn the paper we tackle bi-objective execution time and power consumption optimization problem concerning execution of parallel applications. We propose using a discrete-event simulation environment for exploring this power/time trade-off in the form of a Pareto front. The solution is verified by a case study based on a real deep neural network training application for automatic speech recognition. A simulation lasting over 2 hours...
-
Examining Feature Vector for Phoneme Recognition / Analiza parametrów w kontekście automatycznej klasyfikacji fonemów
PublicationThe aim of this paper is to analyze usability of descriptors coming from music information retrieval to the phoneme analysis. The case study presented consists in several steps. First, a short overview of parameters utilized in speech analysis is given. Then, a set of time and frequency domain-based parameters is selected and discussed in the context of stop consonant acoustical characteristics. A toolbox created for this purpose...
-
Hand gesture recognition supported by fuzzy rules and Kalman filters
PublicationThe paper presents a system based on camera and multimediaprojector enabling a user to control computer applications by dynamic hand gestures. Gesture recognition methodology based on representing hand movement trajectory by motion vectors analysed using fuzzy rule-based inference is first given. For effective hand position tracking Kalman filters are employed. The system engineered is developed using J2SE and C++/OpenCV technology....
-
Personal adaptive tuning of mobile computer audio
PublicationAn integrated methodology for enhancing audio quality in mobile computers is presented. The key features are adaptation of the characteristics of the acoustic track to the changing conditions and to the user's individual preferences. Original signal processing algorithms are introduced, which concern: linearization of frequency response, dialogue intelligibility enhancement and dynamics processing tuned up to the user's preferences....
-
Measurement of Latency in the Android Audio Path
PublicationThis paper provides a description of experimental investigations concerning comparison between the audio path characteristics of various Android versions. First, information about the changes in each system version in the context of latency caused by them is presented. Then, a measurement procedure employing available applications to measure latency is described comparing to results contained in the Internet. Finally, a comparison...
-
Transmitting Alarm Information in DAB+ Broadcasting System
PublicationThe main goal of digital broadcasting is to deliver high-quality content with the lowest possible bitrate. This paper is focused on transmitting alarm information, such as emergency warning and alerting, in the DAB+ (Digital Audio Broadcasting plus) broadcasting system. These additional services should be available at the lowest possible bitrate, in order to provide a clear and understandable voice message to people. Furthermore, additional...
-
Modeling and Designing Acoustical Conditions of the Interior – Case Study
PublicationThe primary aim of this research study was to model acoustic conditions of the Courtyard of the Gdańsk University of Technology Main Building, and then to design a sound reinforcement system for this interior. First, results of measurements of the parameters of the acoustic field are presented. Then, the comparison between measured and predicted values using the ODEON program is shown. Collected data indicate a long reverberation...
-
Adaptive Personal Tuning of Sound in Mobile Computers
PublicationAn integrated methodology for enhancing audio quality in mobile computers is presented. The key features are adaptation of the characteristics of their acoustic track to changing acoustic conditions of the environment and to users’ individual preferences. Signal processing algorithms are introduced that concern: linearization of frequency response, dialogue intelligibility enhancement, and dynamics processing tuned up to the users’...
-
Interactions with recognized patients using smart glasses
PublicationRecently, different smart glasses solutions have been proposed on the market. The rapid development of this wearable technology has led to several research projects related to applications of smart glasses in healthcare. In this paper we propose a general architecture of the system enabling data integration for the recognized person. In the proposed system smart glasses integrates data obtained for the recognized patient from health...
-
Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review
PublicationThe automatic emotion recognition domain brings new methods and technologies that might be used to enhance therapy of children with autism. The paper aims at the exploration of methods and tools used to recognize emotions in children. It presents a literature review study that was performed using a systematic approach and PRISMA methodology for reporting quantitative and qualitative results. Diverse observation channels and modalities...
-
The data exchange between smart glasses and healthcare information systems using the HL7 FHIR standard
PublicationIn this study we evaluated system architecture for the use of smart glasses as a viewer of information, as a source of medical data (vital sign measurements: temperature, pulse rate, and respiration rate), and as a filter of healthcare information. All activities were based on patient/device identification procedures using graphical markers or features based on visual appearance. The architecture and particular use cases were implemented...
-
Smart Virtual Bass Synthesis Algorithm Based on Music Genre Classification
PublicationThe aim of this paper is to present a novel approach to the Virtual Bass Synthesis (VBS) algorithms applied to portable computers. The proposed algorithm employed automatic music genre recognition to determine the optimum parameters for the synthesis of additional frequencies. The synthesis was carried out using the non-linear device (NLD) and phase vocoder (PV) methods depending on the music excerpt genre. Classification of musical...
-
MACHINE LEARNING–BASED ANALYSIS OF ENGLISH LATERAL ALLOPHONES
PublicationAutomatic classification methods, such as artificial neural networks (ANNs), the k-nearest neighbor (kNN) and selforganizing maps (SOMs), are applied to allophone analysis based on recorded speech. A list of 650 words was created for that purpose, containing positionally and/or contextually conditioned allophones. For each word, a group of 16 native and non-native speakers were audio-video recorded, from which seven native speakers’...
-
Pracujący w czasie rzeczywistym system detekcji gazów wykorzystujący przenośny komputer Raspberry PI oraz matrycę półprzewodnikowych czujników gazu
PublicationThe gas-analyzing systems based on the array of partially selective gas sensors and pattern-recognition techniques are potentially fast and lowcost alternative for other devices, like gas‑analysers. They give the possibility of recognition the type and the concentration of measured volatile compounds in their working environment. In this work we present the implementation of gas recognition system, in which the signals from an...
-
A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors
PublicationIn recent years, emotion recognition algorithms have achieved high efficiency, allowing the development of various affective and affect-aware applications. This advancement has taken place mainly in the environment of personal computers offering the appropriate hardware and sufficient power to process complex data from video, audio, and other channels. However, the increase in computing and communication capabilities of smartphones,...
-
Minimizing Distribution and Data Loading Overheads in Parallel Training of DNN Acoustic Models with Frequent Parameter Averaging
PublicationIn the paper we investigate the performance of parallel deep neural network training with parameter averaging for acoustic modeling in Kaldi, a popular automatic speech recognition toolkit. We describe experiments based on training a recurrent neural network with 4 layers of 800 LSTM hidden states on a 100-hour corpora of annotated Polish speech data. We propose a MPI-based modification of the training program which minimizes the...
-
Visual Features for Endoscopic Bleeding Detection
PublicationAims: To define a set of high-level visual features of endoscopic bleeding and evaluate their capabilities for potential use in automatic bleeding detection. Study Design: Experimental study. Place and Duration of Study: Department of Computer Architecture, Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, between March 2014 and May 2014. Methodology: The features have...
-
Noise profiling for speech enhancement employing machine learning models
PublicationThis paper aims to propose a noise profiling method that can be performed in near real-time based on machine learning (ML). To address challenges related to noise profiling effectively, we start with a critical review of the literature background. Then, we outline the experiment performed consisting of two parts. The first part concerns the noise recognition model built upon several baseline classifiers and noise signal features...
-
Automatic Classification of Polish Sign Language Words
PublicationIn the article we present the approach to automatic recognition of hand gestures using eGlove device. We present the research results of the system for detection and classification of static and dynamic words of Polish language. The results indicate the usage of eGlove allows to gain good recognition quality that additionally can be improved using additional data sources such as RGB cameras.
-
Sensors integration in the smart home environment - a proposal to solve the problem with user identification
PublicationIn this preliminary study we, investigate the possibility of user recognition techniques suitable on smart home devices like chairs, beds, aiming for low–power, high accuracy and quick response time. We propose the two well know technique: voice speaker recognition and accelerometer signal from device mounted on the chair, and the third one optical system basing on IR LED transmitter/receiver circuit. The preliminary results proved...
-
Employing a biofeedback method based on hemispheric synchronization in effective learning
PublicationIn this paper an approach to build a brain computer-based hemispheric synchronization system is presented. The concept utilizes the wireless EEG signal registration and acquisition as well as advanced pre-processing methods. The influence of various filtration techniques of EOG artifacts on brain state recognition is examined. The emphasis is put on brain state recognition using band pass filtration for separation of individual...
-
Online sound restoration system for digital library applications
PublicationAudio signal processing algorithms were introduced to the new online non-commercial service for audio restoration intended to enhance the content of digitized audio repositories. Missing or distorted audio samples are predicted using neural networks and a specific implementation of the Jannsen interpolation method based on the autoregressive model (AR) combined with the iterative restoring of missing signal samples. Since the distortion...
-
Gesture-based computer control system applied to the interactive whiteboard
PublicationIn the paper the gesture-based computer control system coupled with the dedicated touchless interactive whiteboard is presented. The system engineered enables a user to control any top-most computer application by using one or both hands gestures. First, a review of gesture recognition applications with a focus on methods and algorithms applied is given. Hardware and software solution of the system consisting of a PC, camera, multimedia...
-
Gesture-based computer control system applied to the interactive whiteboard
PublicationIn the paper the gesture-based computer control system coupled with the dedicated touchless interactive whiteboard is presented. The system engineered enables a user to control any top-most computer application by using one or both hands gestures. First, a review of gesture recognition applications with a focus on methods and algorithms applied is given. Hardware and software solution of the system consisting of a PC, camera, multimedia...
-
Computer-Supported Polysensory Integration Technology for Educationally Handicapped Pupils
PublicationIn this paper, a multimedia system providing technology for hearing and visual attention stimulation is shortly presented. The system aims to support the development of educationally handicapped pupils. The system has been presented in the context of its configuration, architecture, and therapeutic exercise implementation issues. Results of pupils’ improvements after 8 weeks of training with the system are also provided. Training...
-
Fully Automated AI-powered Contactless Cough Detection based on Pixel Value Dynamics Occurring within Facial Regions
PublicationIncreased interest in non-contact evaluation of the health state has led to higher expectations for delivering automated and reliable solutions that can be conveniently used during daily activities. Although some solutions for cough detection exist, they suffer from a series of limitations. Some of them rely on gesture or body pose recognition, which might not be possible in cases of occlusions, closer camera distances or impediments...
-
Video content analysis in the urban area telemonitoring system
PublicationThe task of constant monitoring of video streams from a large number of cameras and reviewing the recordings in order to find a specified event requires a considerable amount of time and effort from the system operators and it is prone to errors. A solution to this problem is an automatic system for constant analysis of camera images being able to raise an alarm if a predefined event is detected. The chapter presents various aspects...
-
Analysis of impact of audio modifications on the robustness of watermark for non-blind architecture
PublicationThe aim of this paper is to assess the robustness of the non-blind audio content watermarking scheme proposed by the authors. The authors present the architecture of the designed system along with the employed workflows for embedding and extracting the watermark followed by the implementation phase description and the analysis of the experimental results. Some possible attack simulations on the embedded watermarks are reviewed,...
-
Online Sound Restoration for Digital Library Applications
PublicationA system for sound restoration was conceived and engineered having the following features: no special sound restoration software is needed to perform audio restoration by the user, the process of restoration employs automatic reduction of noise, wow and impulse distortions performed in the online mode, no skills in digital signal processing from the user are needed. The principles of the created system and its features as well...
-
Fuzzy rule-based dynamic gesture recognition employing camera & multimedia projector
PublicationIn the paper the system based on camera and multimedia projector enabling a user to control computer applications by dynamic hand gestures is presented. The main objective is to present the gesture recognition methodology which bases on representing hand movement trajectory by motion vectors analyzed using fuzzy rule-based inference. The approach was engineered in the system developed with J2SE and C++ / OpenCV technology. OpenCV...
-
The project IDENT: Multimodal biometric system for bank client identity verification
PublicationBiometric identity verification methods are implemented inside a real banking environment comprising: dynamic handwritten signature verification, face recognition, bank cli-ent voice recognition and hand vein distribution verification. A secure communication system based on an intra-bank client-server architecture was designed for this purpose. Hitherto achieved progress within the project is reported in this paper with a focus...
-
Gesture recognition framework for multimedia content viewer controlling
PublicationIn the paper a system for controlling a multimedia content viewer by hand gestures is presented. First, selected methods used for gesture recognition are described. Two different application cases of the system, i.e. for multimedia presentation purposes and for multimedia content viewing are outlined. Moreover, a proposal of improvement of the system combining these approaches is also given. The system work cycle is reviewed. The...
-
Analysis of impact of lossy audio compression on the robustness of watermark embedded in the DWT domain for non-blind copyright protection
PublicationA methodology of non-blind watermarking of the audio content is proposed. The outline of audio copyright problem and motivation for practical applications are discussed. The algorithmic theory pertaining watermarking techniques is briefly introduced. The system architecture together with employed workflows for embedding and extracting the watermarks are described. The implemented approach is described and obtained results are reported....
-
Database of speech and facial expressions recorded with optimized face motion capture settings
PublicationThe broad objective of the present research is the analysis of spoken English employing a multiplicity of modalities. An important stage of this process, discussed in the paper, is creating a database of speech accompanied with facial expressions. Recordings of speakers were made using an advanced system for capturing facial muscle motion. A brief historical outline, current applications, limitations and the ways of capturing face...
-
Online sound restoration system for digital library applications.
PublicationAudio signal processing algorithms were introduced to the new online non-commercial service for audio restoration intended to enhance the content of digitized audio repositories. Missing or distorted audio samples are predicted using neural networks and a specific implementation of the Jannsen interpolation method based on the autoregressive model (AR) combined with the iterative restoring of missing signal samples. Since the distortion...
-
A Comparison of STI Measured by Direct and Indirect Methods for Interiors Coupled with Sound Reinforcement Systems
PublicationThis paper presents a comparison of STI (Speech Transmission Index) coefficient measurement results carried out by direct and indirect methods. First, acoustic parameters important in the context of public address and sound reinforcement systems are recalled. A measurement methodology is presented that employs various test signals to determine impulse responses. The process of evaluating sound system performance, signals enabling...
-
Towards Precise Visual Navigation and Direct Georeferencing for MAV Using ORB-SLAM2
PublicationA low accuracy of positioning using Global Navigation Satellite System (GNSS) are not meet geodetic requirements for direct images georeferencing for Unmanned Aerial Vehicle (UAV) photogrammetry. A majority of UAVs are equipped with a monocular or stereo non-metric cameras for either visual data gathering or live video feed for operator. A cheap positioning techniques used on board commercial UAVs are not that precise as geodetic...
-
A video monitoring system using ontology-driven identification of threats
PublicationIn this paper, we present a video monitoring systemthat leverages image recognition and ontological reasoningabout threats. In the solution, an image processing subsystemuses video recording of a monitored area and recognizesknown concepts in scenes. Then, a reasoning subsystem uses anontological description of security conditions and informationfrom image recognition to check if a violation of a conditionhas occurred. If a threat...
-
Implementation Of The Innovative Radiolocalization System VCS-MLAT (Voice Communication System Multilateration)
PublicationIn the article the concept of the radiolocalization subsystem of the VHF communication for aviation VCS-MLAT (Voice Communication System – Multilateration) is presented. The distributed localization system can estimate the position of the aircraft using the audio signals from aircraft transmitters in the VHF band (118-136 MHz). This paper shows initial verification of the possibility to use voice airband communication to estimate...
-
Evaluation of a Novel Approach to Virtual Bass Synthesis Strategy
PublicationThe aim of this paper is to present a novel approach to the Virtual Bass Synthesis (VBS) strategy applied to portable computers. The developed algorithms involve intelligent, rule-based settings of bass synthesis parameters with regard to music genre of an audio excerpt and the type of a portable device in use. The Smart VBS algorithm performs the synthesis based on a nonlinear device (NLD) with artificial controlling synthesis...
-
Metoda i algorytmy sterowania procesami miksowania dźwięku za pomocą gestów w oparciu o analizę obrazu wizyjnego
PublicationGłównym celem rozprawy było opracowanie systemu miksowania dźwięku za pomocą gestów rąk wykonywanych w powietrzu oraz zbadanie możliwości oferowanych przez takie rozwiązanie w porównaniu ze współczesną metodą miksowania sygnałów fonicznych, wykorzystującą środowisko komputera. Opracowany system rozpoznaje zarówno dynamiczne jak i statyczne gesty rąk. Rozpoznawanie gestów dynamicznych zrealizowano w oparciu o metody logiki rozmytej...
-
Recognition of Hand Drawn Flowcharts
PublicationIn this paper the problem of hand drawn flowcharts recognition is presented. There are described two attitudes to this problem: on-line and off-line. A concept of FCE, a system for recognizing and understanding of freehand drawn on-line flow charts on desktop computer and mobile devices is presented. The first experiments with the FCE system and the planes for future are also described.
-
Impact of Visual Image Quality on Lymphocyte Detection Using YOLOv5 and RetinaNet Algorithms
PublicationLymphocytes, a type of leukocytes, play a vital role in the immune system. The precise quantification, spatial arrangement and phenotypic characterization of lymphocytes within haematological or histopathological images can serve as a diagnostic indicator of a particular lesion. Artificial neural networks, employed for the detection of lymphocytes, not only can provide support to the work of histopathologists but also enable better...