Search results for: multimodal
-
Joint workshop on Multimodal Interaction and Related Machine Learning Algorithms (now ICMI-MLMI)
Conferences -
MULTIMODALNE POMIARY DRGAŃ STRUNY
PublicationW artykule zostały przedstawione badania drgań struny zrealizowane przy użyciu szybkich kamer wizyjnych, mikrofonu oraz akcelerometru. Obiektem badań były instrumenty muzyczne. Opisano zjawiska zachodzące w instrumencie podczas tworzenia się i wydobywania z niego dźwięku. Celem pracy było zbadanie różnic w wynikach otrzymanych poprzez pomiary wykonane z użyciem zróżnicowanych reprezentacji obrazowych i sygnałowych. Zaproponowano...
-
Typoszereg komputerowych interfejsów multimodalnych
PublicationW referacie opisano opracowywane w ramach realizowanego projektu, multimodalne interfejsymultimodalne, ułatwiające użytkowanie urządzeń komputerowych, w tym również terminali mobilnych.Przedstawiono zasady działania poszczególnych interfejsów oraz dotychczasowo uzyskane rezultaty.Wyniki uzyskane zostały drogą prób i eksperymentów z udziałem grup użytkowników docelowych,obejmujących zarówno użytkowników standardowych, jak również...
-
KORPUS MOWY ANGIELSKIEJ DO CELÓW MULTIMODALNEGO AUTOMATYCZNEGO ROZPOZNAWANIA MOWY
PublicationW referacie zaprezentowano audiowizualny korpus mowy zawierający 31 godzin nagrań mowy w języku angielskim. Korpus dedykowany jest do celów automatycznego audiowizualnego rozpoznawania mowy. Korpus zawiera nagrania wideo pochodzące z szybkoklatkowej kamery stereowizyjnej oraz dźwięk zarejestrowany przez matrycę mikrofonową i mikrofon komputera przenośnego. Dzięki uwzględnieniu nagrań zarejestrowanych w warunkach szumowych korpus...
-
Automatyczna weryfikacja klienta bankowego w oparciu o multimodalne technologie biometryczne
PublicationW referacie przedstawiono przegląd rozwiązań wykorzystywanych w bankach do weryfikacji tożsamości klientów. Ponadto zawarto opis metod biometrycznych aktualnie wykorzystywanych w placówkach bankowych wraz z odniesieniem do skuteczności i wygody korzystania z dostępnych rozwiązań. Zaproponowano rozszerzenie zakresu wykorzystania technologii biometrycznych, wskazując kierunek rozwoju systemów bezpieczeństwa dla poprawy dostępu do...
-
Automatyczne określanie stopnia oparzenia. Klasyfikacja na podstawie multimodalnych badań termicznych
Publication -
Multimodalne stanowisko do polisensorycznej diagnozy i stymulacji osób z zaburzeniami komunikacji
PublicationCelem komunikatu plakatowego jest prezentacja eksperymentalnego zintegrowanego systemu multimodalnego, przeznaczonego do wykorzystania w diagnozowaniu i stymulacji polisensorycznej osób niekomunikujących się, w szczególności osób z ciężkimi urazami mózgu. Interfejs użytkownika wykorzystuje śledzenie wzroku i monitorowanie elektroencefalograficzne. Ponadto elementami tego stanowiska są: emiter bodźców zapachowych oraz urządzenie...
-
Specyfikacja niebezpiecznych i podejrzanych zdarzeń w strumieniach wizyjnych, fonicznych i multimodalnych
PublicationWspółczesne systemy monitoringu wizyjnego są złożone z wielu kamer pokrywających rozległe obszary i liczne pomieszczenia. Zakres zdarzeń zachodzących w tych kamerach, mogących stanowić poważne zagrożenia bezpieczeństwa, jest bardzo szeroki \cite{rau}. Operatorowi złożonego systemu monitoringu trudno jest zaobserwować na ekranach monitorów każde zachodzące zdarzenie, wiele praktycznie działających systemów monitoringu wizyjnego...
-
Koncepcja urbanistyczno-architektoniczna zagospodarowania rejonu dworca kolejowego w Białymstoku w ramach programu funkcjonalno-przestrzennego węzła multimodalnego w Białymstoku
Publicationekspertyza projektowa dotyczyła opracowania wariantowej, urbanistycznej koncepcji reorganizacji przestrzeni miejskiej w rejonie dworca PKP/PKS w kontekście poprawy integracji środków transportu
-
IDENT nd.
ProjectsProject realized in Department of Multimedia Systems according to PBS3/B3/26/2015 agreement from 2015-04-20
-
Typoszeregi Opracowanie typoszeregu komputerowych interfejsów multimodalnych oraz ich wdrożenie w zastosowaniach edukacyjnych, medycznych, w obronności i w przemyśle
ProjectsProject realized in Faculty of Electronics, Telecommunications and Informatics according to POIG-01.03.01-22-017/08-00 agreement from 2008-12-01
-
Zespół Systemów Multimedialnych
Research Teams* technologie archiwizacji, rekonstrukcji i dostępu do nagrań archiwalnych * technologie inteligentnego monitoringu wizyjnego i akustycznego * multimedialne technologie telemedyczne * multimodalne interfejsy komputerowe
-
Zespół Systemów Multimedialnych
Research Teams* technologie archiwizacji, rekonstrukcji i dostępu do nagrań archiwalnych * technologie inteligentnego monitoringu wizyjnego i akustycznego * multimedialne technologie telemedyczne * multimodalne interfejsy komputerowe
-
Piotr Odya dr inż.
PeoplePiotr Odya was born in Gdansk in 1974. He received his M.Sc. in 1999 from the Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, Poland. His thesis was related to the problem of sound quality improvement in the contemporary broadcasting studio. He is interested in video editing and multichannel sound systems. The goal of Mr. Odya Ph.D. thesis concerned methods and algorithms for correcting...
-
Michał Lech dr inż.
PeopleMichał Lech was born in Gdynia in 1983. In 2007 he graduated from the faculty of Electronics, Telecommunications and Informatics of Gdansk University of Technology. In June 2013, he received his Ph.D. degree. The subject of the dissertation was: “A Method and Algorithms for Controlling the Sound Mixing Processes by Hand Gestures Recognized Using Computer Vision”. The main focus of the thesis was the bias of audio perception caused...
-
Bożena Kostek prof. dr hab. inż.
People -
Intelligent multimedia solutions supporting special education needs.
PublicationThe role of computers in school education is briefly discussed. Multimodal interfaces development history is shortly reviewed. Examples of applications of multimodal interfaces for learners with special educational needs are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with facial expression and speech stretching audio interface representing audio modality....
-
Intelligent video and audio applications for learning enhancement
PublicationThe role of computers in school education is briefly discussed. Multimodal interfaces development history is shortly reviewed. Examples of applications of multimodal interfaces for learners with special educational needs are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with facial expression and speech stretching audio interface representing audio modality....
-
Krzysztof Kutt dr inż.
PeopleComputer scientist and psychologist trying to combine expertise from both disciplines into something cool. My research activity focuses on the development of affective HCI/BCI interfaces (based on multimodal fusion of signals and contextual data), methods for processing sensory data (including semantization of such data) and the development of knowledge-based systems (in particular knowledge graphs and semantic web systems).
-
Automatic audio-visual threat detection
PublicationThe concept, practical realization and application of a system for detection and classification of hazardous situations based on multimodal sound and vision analysis are presented. The device consists of new kind multichannel miniature sound intensity sensors, digital Pan Tilt Zoom and fixed cameras and a bundle of signal processing algorithms. The simultaneous analysis of multimodal signals can significantly improve the accuracy...
-
Emotion recognition and its application in software engineering
PublicationIn this paper a novel application of multimodal emotion recognition algorithms in software engineering is described. Several application scenarios are proposed concerning program usability testing and software process improvement. Also a set of emotional states relevant in that application area is identified. The multimodal emotion recognition method that integrates video and depth channels, physiological signals and input devices...
-
A Study in Experimental Methods of Human-Computer Communication for Patients After Severe Brain Injuries
PublicationExperimental research in the domain of multimedia technology applied to medical practice is discussed, employing a prototype of integrated multimodal system to assist diagnosis and polysensory stimulation of patients after severe brain injury. The system being developed includes among others: eye gaze tracker, and EEG monitoring of non-communicating patients after severe brain injuries. The proposed solutions are used for collecting...
-
Affect aware video games
PublicationIn this chapter a problem of affect aware video games is described, including such issue as: emotional model of the player, design, development and UX testing of affect-aware video games, multimodal emotion recognition and a featured review of affect-aware video games.
-
Emotion Recognition for Affect Aware Video Games
PublicationIn this paper the idea of affect aware video games is presented. A brief review of automatic multimodal affect recognition of facial expressions and emotions is given. The first result of emotions recognition using depth data as well as prototype affect aware video game are presented
-
Smart Approach for Glioma Segmentation in Magnetic Resonance Imaging using Modified Convolutional Network Architecture (U-NET)
PublicationSegmentation of a brain tumor from magnetic resonance multimodal images is a challenging task in the field of medical imaging. The vast diversity in potential target regions, appearance and multifarious intensity threshold levels of various tumor types are few of the major factors that affect segmentation results. An accurate diagnosis and its treatment demand strict delineation of the tumor affected tissues. Herein, we focus on...
-
Time-domain prosodic modifications for text-to-speech synthesizer
PublicationAn application of prosodic speech processing algorithms to Text-To-Speech synthesis is presented. Prosodic modifications that improve the naturalness of the synthesized signal are discussed. The applied method is based on the TD-PSOLA algorithm. The developed Text-To-Speech Synthesizer is used in applications employing multimodal computer interfaces.
-
Aktywny system RFID do lokalizacji i identyfikacji obiektów w wielomodalnej infrastrukturze bezpieczeństwa
PublicationPrzedstawiono prace koncepcyjne, badawcze oraz implementacyjne skoncentrowane na praktycznej realizacji systemu detekcji obiektów z wykorzystaniem kamer wizyjnych i identyfikacji radiowej. Zaproponowano rozbudowę wielomodalnego teleinformatycznego systemu bezpieczeństwa o warstwę identyfikacji radiowej obiektów. Omówiono założenia zaprojektowanego systemu oraz opracowaną warstwę sprzętową. Zaproponowano i przedyskutowano praktyczne...
-
System Weryfikacji Autentyczności Podpisu Odręcznego
PublicationW referacie przedstawiono system statycznej i dynamicznej weryfikacji autentyczności podpisu odręcznego, składanego piórem biometrycznym, wyposażonym w 2 akcelerometry, 2 żyroskopy i 3 czujniki ścisku, na rezystancyjnej powierzchni dotykowej, łączącym się bezprzewodowo z urządzeniami komputerowymi. We wstępie przedstawiono architekturę sieciową wielomodalnego systemu biometrii. Przedstawiono warstwę sprzętową systemu weryfikacji...
-
Comparative study on the effectiveness of various types of road traffic intensity detectors
PublicationVehicle detection and speed measurements are crucial tasks in traffic monitoring systems. In this work, we focus on several types of electronic sensors, operating on different physical principles in order to compare their effectiveness in real traffic conditions. Commercial solutions are based on road tubes, microwave sensors, LiDARs, and video cameras. Distributed traffic monitoring systems require a high number of monitoring...
-
Handwritten signature verification system employing wireless biometric pen
PublicationThe handwritten signature verification system being a part of the developed multimodal biometric banking stand is presented. The hardware component of the solution is described with a focus on the signature acquisition and on verification procedures. The signature is acquired employing an accelerometer and a gyroscope built-in the biometric pen plus pressure sensors for the assessment of the proper pen grip and then the signature...
-
MODALITY corpus - SPEAKER 38 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 29 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 37 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 28 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 28 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 38 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 27 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 36 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 36 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 27 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 37 - COMMANDS C1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
MODALITY corpus - SPEAKER 29 - SEQUENCE S1
Open Research DataThe MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system,...
-
Multimedia industrial and medical applications supported by machine learning
PublicationThis article outlines a keynote paper presented at the Intelligent DecisionTechnologies conference providing a part of the KES Multi-theme Conference “Smart Digital Futures” organized in Rome on June 14–16, 2023. It briefly discusses projects related to traffic control using developed intelligent traffic signs and diagnosing the health of wind turbine mechanisms and multimodal biometric authentication for banking branches to provide...
-
Testing Stability of Digital Filters Using Optimization Methods with Phase Analysis
PublicationIn this paper, novel methods for the evaluation of digital-filter stability are investigated. The methods are based on phase analysis of a complex function in the characteristic equation of a digital filter. It allows for evaluating stability when a characteristic equation is not based on a polynomial. The operation of these methods relies on sampling the unit circle on the complex plane and extracting the phase quadrant of a function...
-
Once in a season – the pragmatic function of fuck in “BoJack Horseman” TV Show
PublicationThis article investigates the use and pragmatic functions of the swear word fuck in the “BoJack Horseman” produced by Netflix and bridges the gap in the linguistic research on this particular TVshow. Incorporating corpus linguistics tools, the BoJack Horseman Corpus was compiled and thelemma fuck has been investigated and analysed from the multimodal perspective....
-
Effect of some organic solvent - water mixtures composion on precipitated calcium carbonate in carbonation process
PublicationPrecipitated calcium carbonate particles were obtained during carbonation of calcium hydroxide slurry with carbon dioxide. Aqueous solutions of isopropyl alcohol, n-butanol and glycerol were used as solvents. Concentration of organic additives in the reactive mixture was from 0 to 20 % (vol.). Precipitation process were performed in a stirred tank reactor equipped with gas distributor. Multimodal courses of particles size distribution...
-
Virtual immersive environments
PublicationYet a higher level of active systems may be achieved when users are fully immersed in an interface which is a 3D computer generated virtual world and can interact with surrounding objects of that world as they were in a real one. This is the issue covered by Chapter 7. Interaction in such a world is both multidimensional and multimodal, with the possibility of free movement of the user in any direction and the simultaneous stimulation...
-
Controlling computer by lip gestures employing neural network
PublicationResults of experiments regarding lip gesture recognition with an artificial neural network are discussed. The neural network module forms the core element of a multimodal human-computer interface called LipMouse. This solution allows a user to work on a computer using lip movements and gestures. A user face is detected in a video stream from a standard web camera using a cascade of boosted classifiers working with Haar-like features....
-
Human-Computer Interface Based on Visual Lip Movement and Gesture Recognition
PublicationThe multimodal human-computer interface (HCI) called LipMouse is presented, allowing a user to work on a computer using movements and gestures made with his/her mouth only. Algorithms for lip movement tracking and lip gesture recognition are presented in details. User face images are captured with a standard webcam. Face detection is based on a cascade of boosted classifiers using Haar-like features. A mouth region is located in...
-
Molecular Imaging and Nanotechnology—Emerging Tools in Diagnostics and Therapy
PublicationPersonalized medicine is emerging as a new goal in the diagnosis and treatment of diseases. This approach aims to establish differences between patients suffering from the same disease, which allows to choose the most effective treatment. Molecular imaging (MI) enables advanced insight into molecule interactions and disease pathology, improving the process of diagnosis and therapy and, for that reason, plays a crucial role in personalized...