Super-resolved Thermal Imagery for High-accuracy Facial Areas Detection and Analysis - Publikacja - MOST Wiedzy

Twoje konto

zaloguj

Wyszukiwarka

Super-resolved Thermal Imagery for High-accuracy Facial Areas Detection and Analysis

Abstrakt

In this study, we evaluate various Convolutional Neural Networks based Super-Resolution (SR) models to improve facial areas detection in thermal images. In particular, we analyze the influence of selected spatiotemporal properties of thermal image sequences on detection accuracy. For this purpose, a thermal face database was acquired for 40 volunteers. Contrary to most of existing thermal databases of faces, we publish our dataset in a raw, original format (14-bit depth) to preserve all important details. In our experiments, we utilize two metrics usually used for image enhancement evaluation: Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM). In addition, we present how to design a SR network with a widened receptive field to mitigate the problem of contextual information being spread over larger image regions due to the heat flow in thermal images. Finally, we determine whether there is a relation between achieved PSNR and accuracy of facial areas detection that can be analyzed for vital signs extraction (e.g. nostril region). The performed evaluation showed that PSNR can be improved even by 60\% if full bit depth resolution data is used instead of 8 bits. Also, we showed that the application of image enhancement solution is necessary for low resolution images to achieve a satisfactory accuracy of object detection.

Cytowania

  • 1 8

    CrossRef

  • 0

    Web of Science

  • 1 8

    Scopus

Autorzy (4)

Cytuj jako

Pełna treść

pobierz publikację
pobrano 50 razy
Wersja publikacji
Accepted albo Published Version
Licencja
Creative Commons: CC-BY-NC-ND otwiera się w nowej karcie

Słowa kluczowe

Informacje szczegółowe

Kategoria:
Publikacja w czasopiśmie
Typ:
artykuły w czasopismach
Opublikowano w:
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE nr 87,
ISSN: 0952-1976
Język:
angielski
Rok wydania:
2020
Opis bibliograficzny:
Kwaśniewska A., Rumiński J., Szankin M., Kaczmarek M.: Super-resolved Thermal Imagery for High-accuracy Facial Areas Detection and Analysis// ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE -Vol. 87, (2020), s.103263-
DOI:
Cyfrowy identyfikator dokumentu elektronicznego (otwiera się w nowej karcie) 10.1016/j.engappai.2019.103263
Bibliografia: test
  1. R. Keys, Cubic convolution interpolation for digital image processing, IEEE transactions on acoustics, speech, and signal processing 29 (6) (1981) 1153-1160. otwiera się w nowej karcie
  2. L. Zhang, X. Wu, An edge-guided image interpolation algorithm via directional filtering and data fusion, IEEE transactions on Image Processing 15 (8) (2006) 2226-2238.
  3. Y. Romano, M. Protter, M. Elad, Single image interpolation via adaptive nonlocal sparsity-based modeling, IEEE Transactions on Image Processing 23 (7) (2014) 3085-3098. otwiera się w nowej karcie
  4. D. Glasner, S. Bagon, M. Irani, Super-resolution from a single image, in: Com- puter Vision, 2009 IEEE 12th International Conference on, IEEE, 2009, pp. 349- 455 356. otwiera się w nowej karcie
  5. G. Freedman, R. Fattal, Image and video upscaling from local self-examples, ACM Transactions on Graphics (TOG) 30 (2) (2011) 12. otwiera się w nowej karcie
  6. Z. Cui, H. Chang, S. Shan, B. Zhong, X. Chen, Deep network cascade for image super-resolution, in: European Conference on Computer Vision, Springer, 2014, 460 pp. 49-64. otwiera się w nowej karcie
  7. H. Chang, D.-Y. Yeung, Y. Xiong, Super-resolution through neighbor embedding, in: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 1, IEEE, 2004, pp. I-I.
  8. K. I. Kim, Y. Kwon, Single-image super-resolution using sparse regression and 465 natural image prior, IEEE transactions on pattern analysis & machine intelli- gence (6) (2010) 1127-1133.
  9. M. Bevilacqua, A. Roumy, C. Guillemot, M. L. Alberi-Morel, Low-complexity single-image super-resolution based on nonnegative neighbor embedding. otwiera się w nowej karcie
  10. K. Jia, X. Wang, X. Tang, Image transformation based on learning dictionaries 470 across image spaces, IEEE transactions on pattern analysis and machine intelli- gence 35 (2) (2013) 367-380. otwiera się w nowej karcie
  11. C. Dong, C. C. Loy, K. He, X. Tang, Image super-resolution using deep convolu- tional networks, IEEE transactions on pattern analysis and machine intelligence 38 (2) (2016) 295-307. otwiera się w nowej karcie
  12. J. Kim, J. Kwon Lee, K. Mu Lee, Accurate image super-resolution using very deep convolutional networks, in: Proceedings of the IEEE conference on com- puter vision and pattern recognition, 2016, pp. 1646-1654. otwiera się w nowej karcie
  13. J. Kim, J. Kwon Lee, K. Mu Lee, Deeply-recursive convolutional network for image super-resolution, in: Proceedings of the IEEE conference on computer 480 vision and pattern recognition, 2016, pp. 1637-1645. otwiera się w nowej karcie
  14. Y. Tai, J. Yang, X. Liu, Image super-resolution via deep recursive residual net- work, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, 2017, p. 5. otwiera się w nowej karcie
  15. J. Liu, W. Yang, X. Zhang, Z. Guo, Retrieval compensated group structured spar- 485 sity for image super-resolution, IEEE Transactions on Multimedia 19 (2) (2017) 302-316. otwiera się w nowej karcie
  16. X. Li, M. T. Orchard, New edge-directed interpolation, IEEE transactions on im- age processing 10 (10) (2001) 1521-1527.
  17. H. Chen, X. He, L. Qing, Q. Teng, Single image super-resolution via adaptive 490 transform-based nonlocal self-similarity modeling and learning-based gradient regularization, IEEE Transactions on Multimedia 19 (8) (2017) 1702-1717. otwiera się w nowej karcie
  18. V. Jain, S. Seung, Natural image denoising with convolutional networks, in: Ad- vances in Neural Information Processing Systems, 2009, pp. 769-776.
  19. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, 495 otwiera się w nowej karcie
  20. A. Tejani, J. Totz, Z. Wang, et al., Photo-realistic single image super-resolution using a generative adversarial network, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 105-114.
  21. M. S. Sajjadi, B. Schölkopf, M. Hirsch, Enhancenet: Single image super- resolution through automated texture synthesis, in: Computer Vision (ICCV), 500 2017 IEEE International Conference on, IEEE, 2017, pp. 4501-4510. otwiera się w nowej karcie
  22. S. C. Park, M. K. Park, M. G. Kang, Super-resolution image reconstruction: a technical overview, IEEE signal processing magazine 20 (3) (2003) 21-36.
  23. L. G. Villanueva, G. M. Callicó, F. Tobajas, S. López, V. De Armas, J. F. López, R. Sarmiento, Medical diagnosis improvement through image quality enhance- 505 ment based on super-resolution, in: Digital System Design: Architectures, Meth- ods and Tools (DSD), 2010 13th Euromicro Conference on, IEEE, 2010, pp. 259- 262. otwiera się w nowej karcie
  24. M. Abdel-Nasser, J. Melendez, A. Moreno, O. A. Omer, D. Puig, Breast tumor classification in ultrasound images using texture analysis and super-resolution 510 methods, Engineering Applications of Artificial Intelligence 59 (2017) 84-92. otwiera się w nowej karcie
  25. Y. Gao, H. Li, J. Dong, G. Feng, A deep convolutional network for medical image super-resolution, in: Chinese Automation Congress (CAC), 2017, IEEE, 2017, pp. 5310-5315. otwiera się w nowej karcie
  26. A. Kwaśniewska, A. Giczewska, J. Rumiński, Big data significance in remote 515 medical diagnostics based on deep learning techniques, Task Quarterly 21 (2017) 309-319.
  27. M. Lewandowska, J. Rumiński, T. Kocejko, J. Nowak, Measuring pulse rate with a webcama non-contact method for evaluating cardiac activity, in: Computer Sci- ence and Information Systems (FedCSIS), 2011 Federated Conference on, IEEE, 520 2011, pp. 405-410.
  28. J. Ruminski, A. Kwasniewska, Evaluation of respiration rate using thermal imag- ing in mobile conditions, in: Application of Infrared to Biomedical Sciences, Springer, 2017, pp. 311-346. otwiera się w nowej karcie
  29. M. Hanmandlu, et al., A new entropy function and a classifier for thermal face 525 recognition, Engineering Applications of Artificial Intelligence 36 (2014) 269- 286.
  30. A. Kwaśniewska, J. Rumiński, Face detection in image sequences using a portable thermal camera, in: Proceedings of the 13th Quantitative Infrared Ther- mography Conference, 2016. otwiera się w nowej karcie
  31. A. Kwaśniewska, J. Rumiński, K. Czuszyński, M. Szankin, Real-time facial fea- tures detection from low resolution thermal images with deep classification mod- els, Journal of Medical Imaging and Health Informatics 8 (5) (2018) 979-987. otwiera się w nowej karcie
  32. M. Szankin, A. Kwasniewska, T. Sirlapu, M. Wang, J. Ruminski, R. Nicolas, M. Bartscherer, Long distance vital signs monitoring with person identification 535 for smart home solutions, in: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2018, pp. 1558-1561. otwiera się w nowej karcie
  33. I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, Deep learning, Vol. 1, MIT press Cambridge, 2016.
  34. R. Gade, T. B. Moeslund, Thermal cameras and applications: a survey, Machine vision and applications 25 (1) (2014) 245-262. otwiera się w nowej karcie
  35. Flir lepton camera modules, https://www.flir.com/products/ lepton/, accessed: 2018-11-10. otwiera się w nowej karcie
  36. J. Rumiński, Analysis of the parameters of respiration patterns extracted from 545 thermal image sequences, in: Biocybernetics and Biomedical Engineering, Vol. 36, 2016, pp. 732-741. otwiera się w nowej karcie
  37. B. Qi, V. John, Z. Liu, S. Mita, Pedestrian detection from thermal images: A sparse representation based approach, Infrared Physics & Technology 76 (2016) otwiera się w nowej karcie
  38. K. A. R. J. N. R. Szankin, Maciej, Road condition evaluation using fusion of multiple deep models on always-on vision processor, in: Proc. Of the 44th Annual Conference of the IEEE Industrial Electronics Society, in print, 2018. otwiera się w nowej karcie
  39. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, A. C. Berg, Ssd: Single shot multibox detector, in: European conference on computer vision, 555 otwiera się w nowej karcie
  40. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. otwiera się w nowej karcie
  41. V. Nair, G. E. Hinton, Rectified linear units improve restricted boltzmann ma- 560 chines, in: Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807-814.
  42. K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human- level performance on imagenet classification, in: Proceedings of the IEEE inter- national conference on computer vision, 2015, pp. 1026-1034. otwiera się w nowej karcie
  43. D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980. otwiera się w nowej karcie
  44. Tensorflow implementation of sr models, https://github.com/ LoSealL/VideoSuperResolution, accessed: 2018-10-10. otwiera się w nowej karcie
  45. Drrn repository, https://github.com/tyshiwo/DRRN_CVPR17, ac- 570 cessed: 2018-10-10. otwiera się w nowej karcie
  46. L. Torrey, J. Shavlik, Transfer learning, in: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, IGI Global, 2010, pp. 242-264. otwiera się w nowej karcie
  47. Tensorflow detection model zoo, https://github.com/tensorflow/ 575 models/blob/master/research/object_detection/g3doc/ detection_model_zoo.md, accessed: 2018-11-10. otwiera się w nowej karcie
  48. J. Bergstra, Y. Bengio, Random search for hyper-parameter optimization, Journal of Machine Learning Research 13 (Feb) (2012) 281-305.
  49. A. Hore, D. Ziou, Image quality metrics: Psnr vs. ssim, in: 2010 20th Interna- 580 tional Conference on Pattern Recognition, IEEE, 2010, pp. 2366-2369. otwiera się w nowej karcie
Weryfikacja:
Politechnika Gdańska

wyświetlono 123 razy

Publikacje, które mogą cię zainteresować

Improving Accuracy of Contactless Respiratory Rate Estimation by Enhancing Thermal Sequences with Deep Neural Networks

2019

Evaluating Accuracy of Respiratory Rate Estimation from Super Resolved Thermal Imagery

2019

Thermal Images Analysis Methods using Deep Learning Techniques for the Needs of Remote Medical Diagnostics

2020

Influence of Thermal Imagery Resolution on Accuracy of Deep Learning based Face Recognition

2019
Meta Tagi