Challenges in Observing the Emotions of Children with Autism Interacting with a Social Robot

Duygun Erol Barkana; Katrin D. Bartl-Pokorny; Hatice Kose; Agnieszka Landowska; Manuel Milling; Ben Robins; Bjorn Schuller; Pinar Uluer; Michał Wróbel; Tatjana Zorcec

doi:10.1007/s12369-024-01185-3

Challenges in Observing the Emotions of Children with Autism Interacting with a Social Robot

Abstrakt

This paper concerns the methodology of multi-modal data acquisition in observing emotions experienced by children with autism while they interact with a social robot. As robot-enhanced therapy gains more and more attention and proved to be effective in autism, such observations might influence the future development and use of such technologies. The paper is based on an observational study of child-robot interaction, during which multiple modalities were captured and then analyzed to retrieve information on a child’s emotional state. Over 30 children on the autism spectrum from Macedonia, Turkey, Poland, and the United Kingdom took part in our study and interacted with the social robot Kaspar. We captured facial expressions/body posture, voice/vocalizations, physiological signals, and eyegaze-related data. The main contribution of the paper is reporting challenges and lessons learned with regard to interaction, its environment, and observation channels typically used for emotion estimation. The main challenge is the limited availability of channels, especially eyegaze-related (29%) and voice-related (6%) data are not available throughout the entire session. The challenges are of a diverse nature—we distinguished task-based, child-based, and environment-based ones. Choosing the tasks (scenario) and adapting environment, such as room, equipment, accompanying person, is crucial but even with those works done, the child-related challenge is the most important one. Therapists have pointed out to a good potential of those technologies, however, the main challenge to keep a child engaged and focused, remains. The technology must follow a child’s interest, movement, and mood. The main observations are the necessity to train personalized models of emotions as children with autism differ in level of skills and expressions, and emotion recognition technology adaptation in real time (e. g., switching modalities) to capture variability in emotional outcomes.

Cytowania

0

CrossRef
0

Web of Science
0

Scopus

Autorzy (10)

Duygun Erol Barkana
Katrin D. Bartl-Pokorny
Hatice Kose
Agnieszka Landowska dr hab. inż.
Manuel Milling
Ben Robins
Bjorn Schuller
Pinar Uluer
Michał Wróbel dr inż.
Tatjana Zorcec

Cytuj jako

Pełna treść

pełna treść publikacji nie jest dostępna w portalu

pełna treść artykułu zobacz w serwisie zewnętrznym otwiera się w nowej karcie

Słowa kluczowe

Informacje szczegółowe

Kategoria:

Publikacja w czasopiśmie

Typ:

artykuły w czasopismach

Opublikowano w:

International Journal of Social Robotics nr 16, strony 2261 - 2276,
ISSN: 1875-4791

Język:

angielski

Rok wydania:

2024

Opis bibliograficzny:

Erol Barkana D., Bartl-Pokorny K. D., Kose H., Landowska A., Milling M., Robins B., Schuller B., Uluer P., Wróbel M., Zorcec T.: Challenges in Observing the Emotions of Children with Autism Interacting with a Social Robot// International Journal of Social Robotics -Vol. 16,iss. 11-12 (2024), s.2261-2276

DOI: