Gesture Recognition With the Linear Optical Sensor and Recurrent Neural Networks - Publikacja - MOST Wiedzy

Twoje konto

zaloguj

Wyszukiwarka

Gesture Recognition With the Linear Optical Sensor and Recurrent Neural Networks

Abstrakt

In this paper, the optical linear sensor, a representative of low-resolution sensors, was investigated in the multiclass recognition of near-field hand gestures. The recurrent neural network (RNN) with a gated recurrent unit (GRU) memory cell was utilized as a gestures classifier. A set of 27 gestures was collected from a group of volunteers. The 27 000 sequences obtained were divided into training, validation, and test subsets. The primary research goal was to define the most appropriate model architecture in terms of the accurate recognition of each gesture. An additional aim of the research was to investigate the kind of input data, i.e., raw data or preprocessed (feature) data, which generally produces better results. Therefore, three datasets were generated: raw data, simple features data, and high-level features data. (This includes information about hand poses which are already recognized.) The random search method was applied to achieve hyperparameter optimization to find the best possible topology for the neural network. The analysis performed shows that selected models were characterized by a test score at a level of 96.89% for the raw data, 95.75% for simple features, and 93.38% for high-level features. Results indicate that the direct use of raw data obtained from the optical linear sensor evaluated on the RNN with GRU memory cells allows for the reliable recognition of even complex gestures. Therefore, such solutions may have the potential to serve as a support to or as an alternative to video-based sensors especially for mobile devices.

Cytowania

  • 2 8

    CrossRef

  • 0

    Web of Science

  • 3 2

    Scopus

Autorzy (3)

Cytuj jako

Pełna treść

pobierz publikację
pobrano 151 razy
Wersja publikacji
Accepted albo Published Version
Licencja
Creative Commons: CC-BY-NC-SA otwiera się w nowej karcie

Słowa kluczowe

Informacje szczegółowe

Kategoria:
Publikacja w czasopiśmie
Typ:
artykuł w czasopiśmie wyróżnionym w JCR
Opublikowano w:
IEEE SENSORS JOURNAL nr 18, wydanie 13, strony 5429 - 5438,
ISSN: 1530-437X
Język:
angielski
Rok wydania:
2018
Opis bibliograficzny:
Czuszyński K., Rumiński J., Kwaśniewska A.: Gesture Recognition With the Linear Optical Sensor and Recurrent Neural Networks// IEEE SENSORS JOURNAL. -Vol. 18, iss. 13 (2018), s.5429-5438
DOI:
Cyfrowy identyfikator dokumentu elektronicznego (otwiera się w nowej karcie) 10.1109/jsen.2018.2834968
Bibliografia: test
  1. J. S. Kim, S. J. Yun, and Y. S. Kim, "Low-power motion gesture sensor with a partially open cavity package," Opt. Express, vol. 24, no. 10, pp. 10537-10546, 2016. otwiera się w nowej karcie
  2. S. K. Tang, W. C. Tseng, W. W. Luo, K. C. Chiu, S. T. Lin, and Y. P. Liu, "Virtual Mouse: A Low Cost Proximity-Based Gestural Pointing Device," in Human-Computer Interaction. Interaction Techniques and Environments: 14th Int. Conf., HCI International 2011, Orlando, USA, Proc., Part II, 2011, pp. 491-499. otwiera się w nowej karcie
  3. A. Withana, R. Peiris, N. Samarasekara, and S. Nanayakkara, "zSense : Enabling Shallow Depth Gesture Recognition for Greater Input Expressivity on Smart Wearables," in CHI '15 Proc. of the 33rd Annual ACM Conf. on Human Factors in Computing Systems, Seoul, Korea, 2015, pp. 3661-3670. otwiera się w nowej karcie
  4. K. Murakami and H. Taguchi, "Gesture Recognition using Recurrent Neural Networks," in Proc. of the SIGCHI conf. on Human factors in computing systems, New Orleans, USA, 1991, pp. 237-242. otwiera się w nowej karcie
  5. P. Vamplew and A. Adams, "Recognition and anticipation of hand motions using a recurrent neural network," in IEEE International Conf. on Neural Networks, Perth, Australia, 1995, pp. 3-6. otwiera się w nowej karcie
  6. C. W. Ng and S. Ranganath, "Gesture recognition via pose classification," in Proc. -Int. Conf. on Pattern Recognition, Barcelona, Spain, 2000, vol. 15, no. 3, pp. 699-704.
  7. E. Tsironi, P. Barros, and S. Wermter, "Gesture Recognition with a Convolutional Long Short-Term Memory Recurrent Neural Network," in Eur. Symp. on Artificial Neural Networks, Bruges, Belgium, 2016, pp. 213-218. otwiera się w nowej karcie
  8. M. Maraqa and R. Abu-Zaiter, "Recognition of Arabic Sign Language (ArSL) using recurrent neural networks," in 1st Int. Conf. on the Applications of Digital Information and Web Technologies, ICADIWT 2008, Ostrava, Czech Republic, 2008, pp. 478-481. otwiera się w nowej karcie
  9. K. Stefanov and J. Beskow, "Gesture Recognition System for Isolated Sign Language Signs," in The 4th European and 7th Nordic Symposium on Multimodal Communication, 29-30 September 2016, University of Copenhagen, Denmark, 2016, pp. 57-59.
  10. H. Stern, K. Smilansky, and S. Berman, "Depth Based Dual Component Dynamic Gesture Recognition," in IPCV'13 -The 2013 Int. Conf. on Image Processing and Computer Vision, Las Vegas, USA, 2013.
  11. T. Liu, W. Zhou, and H. Li, "Sign language recognition with long short- term memory," in 2016 IEEE Int. Conf. on Image Processing (ICIP), Phoenix, USA, 2016, pp. 2871-2875. otwiera się w nowej karcie
  12. C.-H. Chuan, E. Regina, and C. Guardino, "American Sign Language Recognition Usin Leap Motion Sensor," in 2014 13th Int. Conf. on Machine Learning and Applications, Detroit, USA, 2014, pp. 541-544. otwiera się w nowej karcie
  13. Y. Chen, Z. Ding, Y. L. Chen, and X. Wu, "Rapid recognition of dynamic hand gestures using leap motion," in 2015 IEEE Int. Conf. on Information and Automation, ICIA 2015 -In conjunction with 2015 IEEE Int. Conf. on Automation and Logistics, Lijiang, China, 2015, pp. 1419-1424. otwiera się w nowej karcie
  14. P. Hong, M. Turk, and T. S. Huang, "Constructing Finite State Machines for Fast Gesture Recognition," in Pattern Recognition, 2000. Proc. 15th Int. Conf. on, Barcelona, Spain, 2000, pp. 691-694.
  15. T. E. Starner and A. Pentland, "Visual Recognition of American Sign Language Using Hidden Markov Models," MIT Cambridge Dept Of Brain And Cognitive Sciences, pp. 189-194, 1995. otwiera się w nowej karcie
  16. S. Marcel, O. Bernier, J. E. Viallet, and D. Collobert, "Hand gesture recognition using input-output hidden Markov models," in Automatic Face and Gesture Recognition, 2000. Proc. Fourth IEEE International Conference on, Grenoble, France, 2000, pp. 456-461. otwiera się w nowej karcie
  17. P. Premaratne, S. Yang, P. Vial, and Z. Ifthikar, "Centroid tracking based dynamic hand gesture recognition using discrete Hidden Markov Models," Neurocomputing, vol. 228, no. October 2016, pp. 79-83, 2017. otwiera się w nowej karcie
  18. J. Galka, M. Masior, M. Zaborski, and K. Barczewska, "Inertial Motion Sensing Glove for Sign Language Gesture Acquisition and Recognition," IEEE Sens. J., vol. 16, no. 16, pp. 6310-6316, 2016. otwiera się w nowej karcie
  19. S. Y. Kim, H. G. Han, J. W. Kim, S. Lee, and T. W. Kim, "A Hand Gesture Recognition Sensor Using Reflected Impulses," IEEE Sens. J., vol. 17, no. 10, pp. 2975 -2976, 2017. otwiera się w nowej karcie
  20. P. Wang, W. Li, C. Li, and Y. Hou, "Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networks," https://arxiv.org/abs/1612.09401, 2016. otwiera się w nowej karcie
  21. S. Y. Kim, H. G. Han, J. W. Kim, S. Lee, and T. W. Kim, "A Hand Gesture Recognition Sensor Using Reflected Impulses," IEEE Sens. J., vol. 17, no. 10, pp. 2975 -2976, 2017. otwiera się w nowej karcie
  22. J. Elman, "Finding structure in time* 1," Cogn. Sci., vol. 14, no. 1 990, pp. 179-211, 1990. otwiera się w nowej karcie
  23. D. Bukhari, Y. Wang, and H. Wang, "Multilingual Convolutional, Long Short-Term Memory, Deep Neural Networks for Low Resource Speech Recognition," Procedia Comput. Sci., vol. 107, pp. 842-847, 2017. otwiera się w nowej karcie
  24. G. Bailador, D. Roggen, G. Tröster, and G. Trivino, "Real time gesture recognition using continuous time recurrent neural networks," in Proc. of the ICST 2nd Int Conf. on Body area networks, Florence, Italy, 2007. otwiera się w nowej karcie
  25. Y. Araga, M. Shirabayashi, K. Kaida, and H. Hikawa, "Real time gesture recognition system using posture classifier and Jordan recurrent neural network," in Proc. of the International Joint Conference on Neural Networks, Brisbane, Australia, 2012, pp. 10-15. otwiera się w nowej karcie
  26. N. Neverova, C. Wolf, G. Paci, G. Sommavilla, G. W. Taylor, and F. Nebout, "A multi-scale approach to gesture detection and recognition," in Proc. of the IEEE International Conference on Computer Vision, Sydney, Australia, 2013, pp. 484-491. otwiera się w nowej karcie
  27. R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training Recurrent Neural Networks," in ICML'13 Proc. of the 30th Int. Conf. on Machine Learning, Atlanta, USA, 2013, pp. 1310-1318.
  28. S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997. otwiera się w nowej karcie
  29. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling," https://arxiv.org/abs/1412.3555, pp. 1-9, 2014.
  30. S. Oprea, A. Garcia-Garcia, J. Garcia-Rodriguez, S. Orts-Escolano, and M. Cazorla, "A recurrent neural network based Schaeffer gesture recognition system," in 2017 Int. Joint Conf. on Neural Networks (IJCNN), Anchorage, USa, 2017, pp. 425-431. otwiera się w nowej karcie
  31. A. Géron, Hands-On Machine Learning with Scikit-Learn & TensorFlow. O'Reilly Media Inc., 2017.
  32. S. Shin and W. Sung, "Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks," in IEEE Int. Symp. Circ. and Sys. (ISCAS), Montreal, Canada, 2016, pp. 2274-2277. otwiera się w nowej karcie
  33. S. Otte, D. Krechel, M. Liwicki, and A. Dengel, "Local Feature Based Online Mode Detection with Recurrent Neural Networks," in 2012 Int. Conf. on Frontiers in Hand. Rec., Barti, Italy, 2012, pp. 533-537. otwiera się w nowej karcie
  34. E. Tsironi, P. Barros, C. Weber, and S. Wermter, "An analysis of Convolutional Long Short-Term Memory Recurrent Neural Networks for gesture recognition," Neurocomputing, vol. 268, pp. 76-86, 2017. otwiera się w nowej karcie
  35. X. Chai, Z. Liu, F. Yin, Z. Liu, and X. Chen, "Two Streams Recurrent Neural Networks for Large-Scale Continuous Gesture Recognition," in 2016 23rd Int. Conf. on Pat. Rec. (ICPR), Cancun, Mexico, 2016. otwiera się w nowej karcie
  36. K. Czuszynski, J. Ruminski, and J. Wtorek, "Analysis of the properties of the active linear gesture sensor," Metrol. Meas. Syst., vol. 24, no. 4, pp. 617-630, 2017. otwiera się w nowej karcie
  37. K. Czuszynski, J. Ruminski, and J. Wtorek, "The passive operating mode of the linear optical gesture sensor," Adv. Electr. Comput. Eng., vol. 18, no. 1, pp. 145-156, 2018. otwiera się w nowej karcie
  38. K. Czuszynski, J. Ruminski, and J. Wtorek, "Pose classification in the gesture recognition using the linear optical sensor," in Human System Interactions (HSI), 2017 10th Int. Conf. on, Ulsan, Korea, 2017, pp. 18- 24. otwiera się w nowej karcie
  39. J. O. Wobbrock, M. R. Morris, and A. D. Wilson, "User-defined gestures for surface computing," in Proc. of the 27th Int. Conf. on Human factors in comp. sys. -CHI 09, Boston, USA, 2009, pp. 1083- 1092. otwiera się w nowej karcie
  40. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay, "Scikit-learn: Machine Learning in Python," J. Mach. Learn. Res., vol. 12, pp. 2825-2830, 2012.
  41. J. Bergstra and Y. Bengio, "Random Search for Hyper-Parameter Optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, 2012.
  42. G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter, "Self- Normalizing Neural Networks," https://arxiv.org/abs/1706.02515, 2017. otwiera się w nowej karcie
  43. T. Shen, T. Zhou, G. Long, J. Jiang, S. Pan, and C. Zhang, "DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding," https://arxiv.org/abs/1709.04696, 2017. otwiera się w nowej karcie
  44. N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima," https://arxiv.org/abs/1609.04836, pp. 1-16, 2016.
  45. J. Bergstra, B. Komer, C. Eliasmith, D. Yamins, and D. D. Cox, "Hyperopt: A Python library for model selection and hyperparameter optimization," Comput. Sci. Discov., vol. 8, 2015 otwiera się w nowej karcie
  46. K. Czuszynski, J. Ruminski, A. Bujnowski, and J. Wtorek, "Semi complex navigation with an active optical gesture sensor," in UbiComp '16 Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany, 2016, pp. 269-272. otwiera się w nowej karcie
Źródła finansowania:
  • This work was supported by NCBiR, FWF, SNSF, ANR, and FNR in the framework of ERA-NET CHIST-ERA II through Project eGLASSES—The interactive eyeglasses for mobile, perceptual computing, and in part by Statutory Funds of the Electronics, Telecommunications and Informatics Faculty, Gdansk University of Technology.
Weryfikacja:
Politechnika Gdańska

wyświetlono 171 razy

Publikacje, które mogą cię zainteresować

Optical Sensor Based Gestures Inference Using Recurrent Neural Network in Mobile Conditions

2018

Pose classification in the gesture recognition using the linear optical sensor

2017

From Linear Classifier to Convolutional Neural Network for Hand Pose Recognition

2017

Analysis of Properties of an Active Linear Gesture Sensor

2017
Meta Tagi