Search
Search results for: gaze interaction
-
EMG and gaze based interaction with graphic interface of smart glasses application
PublicationIn this paper we investigate the effectiveness of the interaction using eye tracking and electromyography. Smart glasses requires reliable interfaces for controlling the graphic content displayed directly in front of the user's eye. Presented research is related with the eGlasses project, which is focused on the development of an open platform in the form of multisensory electronic glasses and related interaction methods. One of...
-
Brain-computer interaction based on EEG signal and gaze-tracking information = Analiza interackji mózg-komputer wykorzystująca sygnał EEg i informacje z systemu śledzenia punktu fiksacji wzroku
PublicationThe article presents an attempt to integrate EEG signal analysis with information about human visual activities, i.e. gaze fixation point. The results from gaze-tracking-based measurement were combined with the standard EEG analysis. A search for correlation between the brain activity and the region of the screen observed by the user was performed. The preliminary stage of the study consists in electrooculography (EOG) signal processing....
-
Gaze tracking in multi-display environment
PublicationThis paper presents the basic ideas of eye and gaze tracking in multiple-display environment. The algorithm for display detection and identification is described as well as the rules for gaze interaction in multi display environment. The core of the method is to use special LED markers and eye and scene tracking glasses. Scene tracking camera registers markers position which is then represented as a cloud of points. Analyzing the...
-
Eye-tracking everywhere - software supporting disabled people in interaction with computers
PublicationIn this paper we present comprehensive system for communication with computer by gaze. One of the main assumptions behind this work was to provide solution that can be used with standard RGB webcam. The proposed comprehensive system included the eye tracking module and user interface for convenient gaze interaction with computer. As a result a fully functional application was developed. The average accuracy of the eye tracking...
-
Application of gaze tracking technology to quality of experience domain
PublicationA new methodological approach to study subjective assessment results employing gaze tracking technology is shown. Notions of Human-Computer Interaction (HCI) and Quality of Experience (QoE) are shortly introduced in the context of their common application. Then, the gaze tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT) is presented. A series of audio-visual subjective...
-
Gaze-tracking based audio-visual correlation analysis employing quality of experience methodology
PublicationThis paper investigates a new approach to audio-visual correlation assessment based on the gaze-tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). The gaze-tracking methodology, having roots in Human-Computer Interaction borrows the relevance feedback through gaze-tracking and applies it to the new area of interests, which is Quality of Experience. Results of subjective...
-
Using deep learning to increase accuracy of gaze controlled prosthetic arm
PublicationThis paper presents how neural networks can be utilized to improve the accuracy of reach and grab functionality of hybrid prosthetic arm with eye tracing interface. The LSTM based Autoencoder was introduced to overcome the problem of lack of accuracy of the gaze tracking modality in this hybrid interface. The gaze based interaction strongly depends on the eye tracking hardware. In this paper it was presented how the overall the...
-
The evaluation of eGlasses eye tracking module as an extension for Scratch
PublicationIn this paper we present the possibility of using eGlasses eye tracking module as an extension for Scratch programming tool which is a visual programming language supporting computer skills learning. The main concept behind this project is to setup the interface for rapid interaction design. Eye tracking is a powerful tool for hands free communication but for that requires a dedicated software. This software is rarely tailored...
-
An new method of audio-visual correlation analysis
PublicationThis paper presents a new methodology of conducting the audio-visual correlation analysis employing the gaze tracking system. Interaction between two perceptual modalities, seeing and hearing, their interaction and mutual reinforcement in a complex relationship was a subject of many research studies. Earlier stage of the carried out experiments at the Multimedia Systems Department (MSD) showed that there exists a relationship between...
-
Eye tracking within near-to-eye display
PublicationIn this paper we investigate the effectiveness of the gaze interaction within near-to-eye display. The practical aspect of the paper is about combining an eye tracker with smart glasses. Presented research is related with the eGlasses project, which is focused on the development of an open platform in the form of multisensory electronic glasses and related interaction methods. One of the implemented interaction methods is the one...
-
Facial feature extraction from a monochromatic picture.
PublicationFace pose determination represents an important area of research in Human Machine Interaction. In this paper, I describe a new method of extracting facial feature locations from a single monochromatic monocular camera for the purpose of estimating and tracking the three dimensional pose of human face and eye-gaze direction.
-
Facial Feature extraction from a monochromatic picture
PublicationFace pose determination represents an important area of research in Human Machine Interaction. In this paper, I describe a new method of extracting facial feature locations from a single monochromatic monocular camera for the purpose of estimating and tracking the three dimensional pose of human face and eye-gaze direction.
-
CyberEye: New Eye-Tracking Interfaces for Assessment and Modulation of Cognitive Functions beyond the Brain
PublicationThe emergence of innovative neurotechnologies in global brain projects has accelerated research and clinical applications of BCIs beyond sensory and motor functions. Both invasive and noninvasive sensors are developed to interface with cognitive functions engaged in thinking, communication, or remembering. The detection of eye movements by a camera offers a particularly attractive external sensor for computer interfaces to monitor,...
-
Objectivization of Audio-Visual Correlation analysis
PublicationSimultaneous perception of audio and visual stimuli often causes the concealment or misrepresentation of information actually contained in these stimuli. Such effects are called the ''image proximity effect'' or the ''ventriloquism effect'' in literature. Until recently, most research carried out to understand their nature was based on subjective assessments. The Authors of this paper propose a methodology based on both subjective...
-
Objectivization of audio-video correlation assessment experiments
PublicationThe purpose of this paper is to present a new method of conducting an audio-visual correlation analysis employing a head-motion-free gaze tracking system. First, a review of related works in the domain of sound and vision correlation is presented. Then assumptions concerning audio-visual scene creation are shortly described. The objectivization process of carrying out correlation tests employing gaze-tracking system is outlined....
-
Head movement compensation algorithm in multi-display communication by gaze
PublicationAn influence of head movements on the gaze estimation accuracy when using a head mounted eye tracking system is discussed in the paper. This issue has been examined for a multi-display environment. It was found that head movement (rotation) to some extent does not influence on the gaze estimation accuracy seriously. Acceptable results were obtained when using eye-tracker to communicate with a computer via in two displays simultaneously.
-
Awareness evaluation of patients in vegetative state employing eye-gaze tracking system
PublicationApplication of eye-gaze tracking system to awareness evaluation is demonstrated. Hitherto awareness evaluation methods are presented. The assumptions of proposed method based on analysis of visual activity of patients in vegetative state are demonstrated. The eye-gaze tracking system ''Cyber-Eye'' developed at the Multimedia Systems Department employed to conducted experiments is presented. Research described in the paper indicates...
-
Speech synthesis controlled by eye gazing
PublicationA method of communication based on eye gaze controlling is presented. Investigations of using gaze tracking have been carried out in various context applications. The solution proposed in the paper could be referred to as ''talking by eyes'' providing an innovative approach in the domain of speech synthesis. The application proposed is dedicated to disabled people, especially to persons in a so-called locked-in syndrome who cannot...
-
Observing uncertainty in music tagging by automatic gaze tracking
PublicationIn this paper, a new approach to observe music file tagging process by employing a gaze tracking system is proposed. The study was conducted with the participation of twenty subjects having different musical experience. For the purpose of the experiments a website survey based on a musical database was prepared. It allowed to gather information about music experience of subjects along with music characteristics such as genre, tempo,...
-
Exploiting audio-visual correlation by means of gaze tracking
PublicationThis paper presents a novel means for increasing audio-visual correlation analysis reliability. This is done based on gaze tracking technology engineered at the Multimedia Systems Department of the Gdansk University of Technology, Poland. In the paper, the past history and current research in the area of audio-visual perception analysis are shortly reviewed. Then the methodology employing gaze tracking is presented along with the...
-
Eye-Gaze Tracking-Based Telepresence System for Videoconferencing
PublicationAn approach to the teleimmersive videoconferencing system enhanced by the pan-tilt-zoom (PTZ) camera, controlled by the eye-gaze tracking system, is presented in this paper. An overview of the existing telepresence systems, especially dedicated to videoconferencing is included. The presented approach is based on the CyberEye eye-gaze tracking system engineered at the Multimedia Systems Department (MSD) of Gdańsk University of Technology...
-
Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries
PublicationOne of the first clinical signs differentiating the minimally conscious state from the vegetative state is the presence of smooth pursuit eye movements occurring in direct response to moving salient stimuli. Glasgow Coma Scale (GCS) is one of the most commonly used diagnostic tools for acute phase assessment of the level of consciousness, together with a neurological examination. These classic measures are limited to qualitative...
-
Consciousness Study of Subjects with Unresponsive Wakefulness Syndrome Employing Multimodal Interfaces
PublicationThe paper presents a novel multimodal-based methodology for consciousness study of individuals with unresponsive wakefulness syndrome. Two interfaces were employed in the experiments: eye gaze tracking system – CyberEye developed at the Multimedia Systems Department, and EEG device with electrode placement in the international 10-20 standard. It was a pilot study for checking if it is possible to determine objective methods based...
-
Comparison of developed gaze point estimation methods
PublicationThis paper presents the software part of an inexpensive hands-free eye tracking system. The system works using infrared illumination like most of the available eye trackers. Two methods allowing estimation the gaze point on computer screen are compared. Research on effectiveness of these methods is discussed and the better one is indicated.
-
Music Information Retrieval in Music Repositories
PublicationThis chapter reviews the key concepts associated with automated Music Information Retrieval (MIR). First, current research trends and system solutions in terms of music retrieval and music recommendation are discussed. Next, experiments performed on a constructed music database are presented. A proposal for music retrieval and annotation aided by gaze tracking is also discussed.
-
Binocular Vision Impairments Therapy Supported By Contactless Eye-gaze Tracking System
PublicationBinocular vision impairments often result in partial or total loss of stereoscopic vision. The lack of binocular vision is a serious vision impairment that deserves more attention. Very important result of the binocular vision impairments is a binocular depth perception. This paper describes also a concept of a measurement and therapy system for the binocular vision impairments by using eye-gaze tracking system.
-
AUDITORY DISPLAY FROM THE MUSIC TECHNOLOGY PERSPECTIVE . Obecność wirtualnego środowiska dźwiękowego w technologiach muzycznych
PublicationThis paper presents some applications of Auditory Displays (AD) in the domain of music technology. First, the scope of music technology and auditory display areas are shortly outlined. Then, the research trends and system solutions within the fields of music technology, music information retrieval and music recommendation are discussed. Finally, an example of an auditory display that facilities music annotation process based on...
-
Post-comatose patients with minimal consciousness tend to preserve reading comprehension skills but neglect syntax and spelling
PublicationModern eye tracking technology provides a means for communication with patients suffering from disorders of consciousness (DoC) or remaining in locked-in-state. However, being able to use an eye tracker for controlling text-based contents by such patients requires preserved reading ability in the first place. To our knowledge, this aspect, although of great social importance, so far has seemed to be neglected. In the paper, we...
-
Auditory Display Applied to Research in Music and Acoustics . Obrazowanie dźwiękowe w muzyce i akustyce.
PublicationThis paper presents a relationship between Auditory Display (AD) and the domains of music and acoustics. First, some basic notions of the Auditory Display area are shortly outlined. Then, the research trends and system solutions within the fields of music technology, music information retrieval and music recommendation and acoustics that are within the scope of AD are discussed. Finally, an example of AD solution based on gaze...
-
Zachowania starszych klientów na rynku usług on-line
PublicationOsoby starsze stanowić będą w przyszłości stale zwiększający się rynek konsumencki. Jednocześnie stale rośnie rynek usług online. Dlatego problem korzystania osób starszych z usług on-line nabiera szczególnego znaczenia. W artykule przedstawione zostały wyniki badania sposobu korzystania z usługi pośrednictwa pracy w serwisach internetowych, uzyskane z wykorzystaniem badań okulograficznych - nagrań Gaze Plot oraz wywiadów ThinkAloud....
-
A Study in Experimental Methods of Human-Computer Communication for Patients After Severe Brain Injuries
PublicationExperimental research in the domain of multimedia technology applied to medical practice is discussed, employing a prototype of integrated multimodal system to assist diagnosis and polysensory stimulation of patients after severe brain injury. The system being developed includes among others: eye gaze tracker, and EEG monitoring of non-communicating patients after severe brain injuries. The proposed solutions are used for collecting...
-
Virtual keyboard controlled by eye gaze employing speech synthesis
PublicationThe article presents the speech synthesis integrated into the eye gaze tracking system. This approach can significantly improve the quality of life of physically disabled people who are unable to communicate. The virtual keyboard (QWERTY) is an interface which allows for entering the text for the speech synthesizer. First, this article describes a methodology of determining the fixation point on a computer screen. Then it presents...
-
Virtual Keyboard controlled by eye gaze employing speech synthesis
PublicationThe article presents the speech synthesis integrated into the eye gaze tracking system. This approach can significantly improve the quality of life of physically disabled people who are unable to communicate. The virtual keyboard (QWERTY) is an interface which allows for entering the text for the speech synthesizer. First, this article describes a methodology of determining the fixation point on a computer screen. Then it presents...
-
New Aspects of Virtual Sound Source Localization Research—Impact of Visual Angle and 3-D Video Content on Sound Perception
PublicationThe influence of image on virtual sound source localization, called the “image proximity effect” or the “ventriloquism effect”, is a well known phenomenon. This paper focuses on other aspects related to this effect, namely the impact of the visual angle of the presented object and 3D video content on sound perception. The research conducted confirmed that the visual angle of the presented object determines the image proximity effect...
-
Cross-domain applications of multimodal human-computer interfaces
PublicationDeveloped multimodal interfaces for education applications and for disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and audio interface for speech stretching for hearing impaired and stuttering people and intelligent pen allowing for diagnosing and ameliorating developmental dyslexia. The eye-gaze tracking system named...
-
Intracranial electrophysiological recordings from the human brain during memory tasks with pupillometry
PublicationData comprise intracranial EEG (iEEG) brain activity represented by stereo EEG (sEEG) signals, recorded from over 100 electrode channels implanted in any one patient across various brain regions. The iEEG signals were recorded in epilepsy patients (N=10) undergoing invasive monitoring and localization of seizures when they were performing a battery of four memory tasks lasting approx. 1 hour in total. Gaze tracking on the task...
-
New methods for assessment and stimulation of non-communicative patients employing advanced multimodal HCI . Nowe metody oceny i stymulacji pacjentów niekomunikatywnych z wykorzystaniem zaawansowanych interfejsów multimodalnych człowiek-komputer
PublicationIn most cases of patients with locomotor system damage it is possible to find a solution to the medical problems originating from the injury. However, it is much more difficult to prevent cognitive and emotional impairments. Therefore, we believe that the technological support of therapists working with such patients on an everyday basis may be essential. We have acquired experience in designing and providing diagnostic and therapeutic...
-
Assessment of hearing in coma patients employing auditory brainstem response, electroencephalography, and eye-gaze-tracking
PublicationThe results of the study conducted by Tagliaferri et al. in 12 European countries indicate that the ratio of registered brain injury cases in Europe amounts to 150-300 per 100 000 people, with the European mean value of 235 cases per 100 000 people. The project presented in the paper assumes development of a combined metric of patients’ state remaining in coma by intelligent fusion of GCS (subjective Glasgow Coma Scale or its derivatives)...
-
Analysis of the relationship between the length of eye fixation and the parameters of advertisements visible from the road
PublicationIn Poland, many advertising signs addressed to motorists are located along the roadside. Advertisements do not serve any traffic related purpose and can distract drivers making them less reliable and as a consequence, reduce road safety. The existing regulations only apply to the carriageway. The research is conducted as part of the RID project. The objective of the RID project is to create a "Manual for positioning roadside advertising...
-
Multimodal human-computer interfaces based on advanced video and audio analysis
PublicationMultimodal interfaces development history is reviewed briefly in the introduction. Some applications of multimodal interfaces to education software for disabled people are presented. One of them, the LipMouse is a novel, vision-based human-computer interface that tracks user’s lip movements and detect lips gestures. A new approach to diagnosing Parkinson’s disease is also shown. The progression of the disease can be measured employing...
-
Multimodal human-computer interfaces based on advanced video and audio analysis
PublicationMultimodal interfaces development history is reviewed briefly in the introduction. Examples of applications of multimodal interfaces to education software and for the disabled people are presented, including interactive electronic whiteboard based on video image analysis, application for controlling computers with mouth gestures and the audio interface for speech stretching for hearing impaired and stuttering people. The Smart...
-
User Authentication by Eye Movement Features Employing SVM and XGBoost Classifiers
PublicationDevices capable of tracking the user’s gaze have become significantly more affordable over the past few years, thus broadening their application, including in-home and office computers and various customer service equipment. Although such devices have comparatively low operating frequencies and limited resolution, they are sufficient to supplement or replace classic input interfaces, such as the keyboard and mouse. The biometric...
-
Multifactor consciousness level assessment of participants with acquired brain injuries employing human–computer interfaces
PublicationBackground A lack of communication with people suffering from acquired brain injuries may lead to drawing erroneous conclusions regarding the diagnosis or therapy of patients. Information technology and neuroscience make it possible to enhance the diagnostic and rehabilitation process of patients with traumatic brain injury or post-hypoxia. In this paper, we present a new method for evaluation possibility of communication and the...
-
New Applications of Multimodal Human-Computer Interfaces
PublicationMultimodal computer interfaces and examples of their applications to education software and for the disabled people are presented. The proposed interfaces include the interactive electronic whiteboard based on video image analysis, application for controlling computers with gestures and the audio interface for speech stretching for hearing impaired and stuttering people. Application of the eye-gaze tracking system to awareness...