Automatic recognition of therapy progress among children with autism - Publication - Bridge of Knowledge


Automatic recognition of therapy progress among children with autism


The article presents a research study on recognizing therapy progress among children with autism spectrum disorder. The progress is recognized on the basis of behavioural data gathered via five specially designed tablet games. Over 180 distinct parameters are calculated on the basis of raw data delivered via the game flow and tablet sensors - i.e. touch screen, accelerometer and gyroscope. The results obtained confirm the possibility of recognizing progress in particular areas of development. The recognition accuracy exceeds 80%. Moreover, the study identifies a subset of parameters which appear to be better predictors of therapy progress than others. The proposed method - consisting of data recording, parameter calculation formulas and prediction models - might be implemented in a tool to support both therapists and parents of autistic children. Such a tool might be used to monitor the course of the therapy, modify it and report its results.


  • 1 2


  • 1 0

    Web of Science

  • 1 1


Cite as

Full text

download paper
downloaded 29 times
Publication version
Accepted or Published Version
Creative Commons: CC-BY open in new tab



artykuł w czasopiśmie wyróżnionym w JCR
Published in:
Scientific Reports no. 7, pages 1 - 14,
ISSN: 2045-2322
Publication year:
Bibliographic description:
Kołakowska A., Landowska A., Anzulewicz A., Sobota K.: Automatic recognition of therapy progress among children with autism// Scientific Reports. -Vol. 7, (2017), s.1-14
Digital Object Identifier (open in new tab) 10.1038/s41598-017-14209-y
Bibliography: test
  1. Jedrzejewska-Szczerska, M., Karpienko, K. & Landowska, A. System supporting behavioral therapy for children with autism. J. Innov. Opt. Heal. Sci 8 (2015). open in new tab
  2. Landowska, A. & Smiatacz, M. Mobile activity plan applications for behavioral therapy of autistic children. In Machine Interactions 4, Advances in Intelligent Systems and Computing, 15-125, (Springer International Publishing, 2015). open in new tab
  3. Cheng, L., Kimberly, G. & Orlich, F. Kidtalk: online therapy for aspergers syndrome. Technical Report MSR-TR-2002-08, Microsoft Research, Redmont, WA (2002). open in new tab
  4. Scientific REPORTS | 7: 13863 | DOI:10.1038/s41598-017-14209-y open in new tab
  5. Tanaka, J. W. et al. Using computerized games to teach face recognition skills to children with autism spectrum disorder: The lets face it! Program. J. Child Psychol. Psychiatry 51, 944-952, (2010). open in new tab
  6. Deriso, D., Susskind, J., Krieger, L. & Bartlett, M. Emotion mirror: a novel intervention for autism based on real-time expression recognition. In Computer Vision-ECCV2012. Workshops and Demonstration, 671-674 (Springer International Publishing, Florence, 2012). open in new tab
  7. Kaliouby, R. & Robinson, P. The emotional hearing aid: An assistive tool for children with Asperger syndrome. Univers. Access Inf. Soc. 4, 121-134, (2005). open in new tab
  8. Robins, D., Fein, D., Barton, M. & Green, J. The modified-checklist for autism in toddlers (m-chat): An initial investigation in the early detection of autism and pervasive developmental disorders. J. Autism. Dev. Disord. 31, 131-144 (2001). open in new tab
  9. Saitovitch, A. et al. Studying gaze abnormalities in autism: Which type of stimulus to use? Open J. Psychiatry 3, 32-38, https://doi. org/10.4236/ojpsych.2013.32A006 (2013). open in new tab
  10. Jones, W. & Klin, A. Attention to eyes is present but in decline in 2-6-month-old infants later diagnosed with autism. Nat. 504, 427-431, (2013). open in new tab
  11. Hashemi, J. et al. Computer vision tools for the noninvasive assessment of autism-related behavioral markers. In IEEE International Conference on Development and Learning and Epigenetic Robotics, (San Diego, 2012). open in new tab
  12. Torres, E. et al. Autism: the micromovement perspective. Front. Integr. Neurosci. 7, (2013). open in new tab
  13. Wall, D. P., Dally, R., Luyster, R., Jung, J.-Y. & DeLuca, T. F. Use of artificial intelligence to shorten the behavioral diagnosis of autism. PloS one 7, (2012). open in new tab
  14. Wall, D., Kosmicki, J., Deluca, T., Harstad, E. & Fusaro, V. Use of machine learning to shorten observation-based screening and diagnosis of autism. Transl. psychiatry, (2012). open in new tab
  15. Yampolskiy, R. & Govindaraju, V. Behavioural biometrics: a survey and classification. Int. J. Biom. 1, 81-113, IJBM.2008.018665 (2008). open in new tab
  16. Kolakowska, A. User authentication based on keystroke dynamics analysis. In Springer-Verlag (ed.) Computer Recognition Systems 4, vol. 95 of Advances in Intelligent and Soft Computing, 667-675, (2011). open in new tab
  17. Hansen, J. H. L. & Hasan, T. Speaker recognition by machines and humans: A tutorial review. IEEE Signal Process. Mag. 32, 74-99, (2015). open in new tab
  18. Nickel, C., Wirtl, T. & Busch, C. Authentication of smartphone users based on the way they walk using k-nn algorithm. In Proceedings of the 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP '12, 16-20, (IEEE Computer Society, Washington, DC, USA, 2012). open in new tab
  19. Kolakowska, A. Recognizing emotions on the basis of keystroke dynamics. In Proc. of the 8th International Conference on Human System Interaction, 667-675, (Warsaw, 2015). open in new tab
  20. Kolakowska, A., Landowska, A., Jarmolkowicz, P., Jarmolkowicz, M. & Sobota, K. Automatic recognition of males and females among web browser users based on behavioural patterns of peripherals usage. Internet Res. 26, 1093-1111, intr-04-2015-0100 (2016). open in new tab
  21. Luyster, R. et al. The autism diagnostic observation schedule-toddler module: a new module of a standardized diagnostic measure for autism spectrum disorders. J. Autism. Dev. Disord. 39, 1305-1320, (2009). open in new tab
  22. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (Fifth ed.) (2013). open in new tab
  23. Taffoni, F. et al. Sensor-based technology in the study of motor skills in infants at risk for ASD. In IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 1879-1883, (2012). open in new tab
  24. Trevarthen, C. & Delafield-Butt, J. Autism as a developmental disorder in intentional movement and affective engagement. Front. Integr. Neurosci. 7, (2013). open in new tab
  25. Cook, J., Blakemore, S. & Press, C. Atypical basic movement kinematics in autism spectrum conditions. Brain 136, 2816-2824, (2013). open in new tab
  26. David, F., Baranek, G., Wiesen, C., Miao, A. F. & Thorpe, D. Coordination of precision grip in 2-6 years-old children with autism spectrum disorders compared to children developing typically and children with developmental disabilities. Front. Integr. Neurosci. 6, (2012). open in new tab
  27. Anzulewicz, A., Sobota, K. & Delafield-Butt, J. Toward the autism motor signature: Gesture patterns during smart tablet gameplay identify children with autism. Sci. Reports 6, (2016). open in new tab
  28. Dowd, A., McGinley, J., Taffe, J. & Rinehart, N. Do planning and visual integration difficulties underpin motor dysfunction in autism? A kinematic study of young children with autism. J. Autism. Dev. Disord. 42, 1539-1548, 011-1385-8 (2012). open in new tab
  29. von Hofsten, C. Action in development. Dev. Sci. 10, 54-60, (2007). open in new tab
  30. MacDonald, M., Lord, C. & Ulrich, D. The relationship of motor skills and adaptive behavior skills in young children with autism spectrum disorders. Res. Autism. Spectr. Discord. 7, 1383-1390 (2013). open in new tab
  31. Wilkins, J. The relationship between social skills and challenging behaviors in children with autism spectrum disorders. Ph.D. thesis (2010).
  32. Adolf, K., Tamis-Lemonda, C. & Karasik, L. Cinderella indeed -a commentary on Iverson's 'Developing language in a developing body: the relationship between motor development and language development' . J. Child. Lang. 37, 269-273, S030500090999047X (2010). open in new tab
  33. Iverson, J. Developing language in a developing body: the relationship between motor development and language development. J. Child. Lang. 37, 229-261, (2010). open in new tab
  34. Lord, C. et al. Austism diagnostic observation schedule: A standardized observation of communicative and social behavior. J. Autism. Dev. Disord. 19, 185-212, (1989). open in new tab
  35. AUTMON. Automated therapy monitoring for children with developmental disorders of autism spectrum. http://autmon.eti.
  36. Kolakowska, A. et al. Applications for investigating therapy progress of autistic children. In Federated Conference on Computer Science and Information Systems, 1693-1697, (Gdansk, 2016). open in new tab
  37. Kolakowska, A., Landowska, A. & Karpienko, K. Gyroscope-based game revealing progress of children with autism. In Int. Conf. Machine Learning and Soft Computing (Ho Chi Minh, 2017). open in new tab
  38. Wolpert, D. H. The lack of a priori distinctions between learning algorithms. Neural. Comput. 8, 1341-1390, neco.1996.8.7.1341 (1996). open in new tab
  39. Breiman, L. Bagging predictors. Mach. Learn. 24, 123-140, (1996). open in new tab
  40. Breiman, L. Random forests. Mach. Learn. 45, 5-32, (2001). open in new tab
  41. Rodriguez, J. J., Kuncheva, L. I. & Alonso, C. J. Rotation forest: A new classifier ensemble method. IEEE. Trans. Pattern. Anal. Mach. Intell. 28, 1619-1630, (2006). open in new tab
  42. Freund, Y. & Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119-139, (1997). open in new tab
  43. Kohavi, R. & Quinlan, R. Decision tree discovery. In Handbook of data mining and knowledge discovery, chap. 16, 267-276 (Oxford University Press, 2002).
  44. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning, chap. 6 (MIT Press, 2016).
  45. Scientific REPORTS | 7: 13863 | DOI:10.1038/s41598-017-14209-y 44. Friedman, N., Geiger, D. & Goldszmidt, M. Bayesian network classifiers. Mach. Learn. 29, 131-163, 1007465528199 (1997). open in new tab
  46. Hall, M. et al. The WEKA data mining software: an update. SIGKDD Explorations 11, 10-18 (2009). open in new tab
  47. Witten, I., Frank, E. & M.A., H. Data Mining: Practical Machine Learning Tools and Techniques, chap. 5 (Morgan Kaufmann Publishers Inc., 2011). open in new tab
  48. Nadeau, C. & Bengio, Y. Inference for the generalization error. Mach. Learn. 52, 239-281, (2003). open in new tab
  49. Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861-874, (2006). open in new tab
  50. Cook, J. From movement kinematics to social cognition: the case of autism. Philos. Transactions of the Royal Soc. Lond. B: Biol. Sci. 371, (2016). open in new tab
  51. Travers, B. et al. Brainstem white matter predicts individual differences in manual motor difficulties and symptom severity in autism. J. Autism. Dev. Disord. 45, 3030-3040, (2015). open in new tab
  52. Gernsbacher, M., Sauer, E., Geye, H., Schweigert, E. & Hill Goldsmith, H. Infant and toddler oral-and manual-motor skills predict later speech fluency in autism. J. Child. Psychol. Psychiatry. 49, 43-50, (2008). open in new tab
Verified by:
Gdańsk University of Technology

seen 133 times

Recommended for you

Meta Tags