Abstrakt
In this study, we evaluate various Convolutional Neural Networks based Super-Resolution (SR) models to improve facial areas detection in thermal images. In particular, we analyze the influence of selected spatiotemporal properties of thermal image sequences on detection accuracy. For this purpose, a thermal face database was acquired for 40 volunteers. Contrary to most of existing thermal databases of faces, we publish our dataset in a raw, original format (14-bit depth) to preserve all important details. In our experiments, we utilize two metrics usually used for image enhancement evaluation: Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM). In addition, we present how to design a SR network with a widened receptive field to mitigate the problem of contextual information being spread over larger image regions due to the heat flow in thermal images. Finally, we determine whether there is a relation between achieved PSNR and accuracy of facial areas detection that can be analyzed for vital signs extraction (e.g. nostril region). The performed evaluation showed that PSNR can be improved even by 60\% if full bit depth resolution data is used instead of 8 bits. Also, we showed that the application of image enhancement solution is necessary for low resolution images to achieve a satisfactory accuracy of object detection.
Cytowania
-
1 8
CrossRef
-
0
Web of Science
-
1 9
Scopus
Autorzy (4)
Cytuj jako
Pełna treść
- Wersja publikacji
- Accepted albo Published Version
- Licencja
- otwiera się w nowej karcie
Słowa kluczowe
Informacje szczegółowe
- Kategoria:
- Publikacja w czasopiśmie
- Typ:
- artykuły w czasopismach
- Opublikowano w:
-
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
nr 87,
ISSN: 0952-1976 - Język:
- angielski
- Rok wydania:
- 2020
- Opis bibliograficzny:
- Kwaśniewska A., Rumiński J., Szankin M., Kaczmarek M.: Super-resolved Thermal Imagery for High-accuracy Facial Areas Detection and Analysis// ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE -Vol. 87, (2020), s.103263-
- DOI:
- Cyfrowy identyfikator dokumentu elektronicznego (otwiera się w nowej karcie) 10.1016/j.engappai.2019.103263
- Bibliografia: test
-
- R. Keys, Cubic convolution interpolation for digital image processing, IEEE transactions on acoustics, speech, and signal processing 29 (6) (1981) 1153-1160. otwiera się w nowej karcie
- L. Zhang, X. Wu, An edge-guided image interpolation algorithm via directional filtering and data fusion, IEEE transactions on Image Processing 15 (8) (2006) 2226-2238.
- Y. Romano, M. Protter, M. Elad, Single image interpolation via adaptive nonlocal sparsity-based modeling, IEEE Transactions on Image Processing 23 (7) (2014) 3085-3098. otwiera się w nowej karcie
- D. Glasner, S. Bagon, M. Irani, Super-resolution from a single image, in: Com- puter Vision, 2009 IEEE 12th International Conference on, IEEE, 2009, pp. 349- 455 356. otwiera się w nowej karcie
- G. Freedman, R. Fattal, Image and video upscaling from local self-examples, ACM Transactions on Graphics (TOG) 30 (2) (2011) 12. otwiera się w nowej karcie
- Z. Cui, H. Chang, S. Shan, B. Zhong, X. Chen, Deep network cascade for image super-resolution, in: European Conference on Computer Vision, Springer, 2014, 460 pp. 49-64. otwiera się w nowej karcie
- H. Chang, D.-Y. Yeung, Y. Xiong, Super-resolution through neighbor embedding, in: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 1, IEEE, 2004, pp. I-I.
- K. I. Kim, Y. Kwon, Single-image super-resolution using sparse regression and 465 natural image prior, IEEE transactions on pattern analysis & machine intelli- gence (6) (2010) 1127-1133.
- M. Bevilacqua, A. Roumy, C. Guillemot, M. L. Alberi-Morel, Low-complexity single-image super-resolution based on nonnegative neighbor embedding. otwiera się w nowej karcie
- K. Jia, X. Wang, X. Tang, Image transformation based on learning dictionaries 470 across image spaces, IEEE transactions on pattern analysis and machine intelli- gence 35 (2) (2013) 367-380. otwiera się w nowej karcie
- C. Dong, C. C. Loy, K. He, X. Tang, Image super-resolution using deep convolu- tional networks, IEEE transactions on pattern analysis and machine intelligence 38 (2) (2016) 295-307. otwiera się w nowej karcie
- J. Kim, J. Kwon Lee, K. Mu Lee, Accurate image super-resolution using very deep convolutional networks, in: Proceedings of the IEEE conference on com- puter vision and pattern recognition, 2016, pp. 1646-1654. otwiera się w nowej karcie
- J. Kim, J. Kwon Lee, K. Mu Lee, Deeply-recursive convolutional network for image super-resolution, in: Proceedings of the IEEE conference on computer 480 vision and pattern recognition, 2016, pp. 1637-1645. otwiera się w nowej karcie
- Y. Tai, J. Yang, X. Liu, Image super-resolution via deep recursive residual net- work, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, 2017, p. 5. otwiera się w nowej karcie
- J. Liu, W. Yang, X. Zhang, Z. Guo, Retrieval compensated group structured spar- 485 sity for image super-resolution, IEEE Transactions on Multimedia 19 (2) (2017) 302-316. otwiera się w nowej karcie
- X. Li, M. T. Orchard, New edge-directed interpolation, IEEE transactions on im- age processing 10 (10) (2001) 1521-1527.
- H. Chen, X. He, L. Qing, Q. Teng, Single image super-resolution via adaptive 490 transform-based nonlocal self-similarity modeling and learning-based gradient regularization, IEEE Transactions on Multimedia 19 (8) (2017) 1702-1717. otwiera się w nowej karcie
- V. Jain, S. Seung, Natural image denoising with convolutional networks, in: Ad- vances in Neural Information Processing Systems, 2009, pp. 769-776.
- C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, 495 otwiera się w nowej karcie
- A. Tejani, J. Totz, Z. Wang, et al., Photo-realistic single image super-resolution using a generative adversarial network, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 105-114.
- M. S. Sajjadi, B. Schölkopf, M. Hirsch, Enhancenet: Single image super- resolution through automated texture synthesis, in: Computer Vision (ICCV), 500 2017 IEEE International Conference on, IEEE, 2017, pp. 4501-4510. otwiera się w nowej karcie
- S. C. Park, M. K. Park, M. G. Kang, Super-resolution image reconstruction: a technical overview, IEEE signal processing magazine 20 (3) (2003) 21-36.
- L. G. Villanueva, G. M. Callicó, F. Tobajas, S. López, V. De Armas, J. F. López, R. Sarmiento, Medical diagnosis improvement through image quality enhance- 505 ment based on super-resolution, in: Digital System Design: Architectures, Meth- ods and Tools (DSD), 2010 13th Euromicro Conference on, IEEE, 2010, pp. 259- 262. otwiera się w nowej karcie
- M. Abdel-Nasser, J. Melendez, A. Moreno, O. A. Omer, D. Puig, Breast tumor classification in ultrasound images using texture analysis and super-resolution 510 methods, Engineering Applications of Artificial Intelligence 59 (2017) 84-92. otwiera się w nowej karcie
- Y. Gao, H. Li, J. Dong, G. Feng, A deep convolutional network for medical image super-resolution, in: Chinese Automation Congress (CAC), 2017, IEEE, 2017, pp. 5310-5315. otwiera się w nowej karcie
- A. Kwaśniewska, A. Giczewska, J. Rumiński, Big data significance in remote 515 medical diagnostics based on deep learning techniques, Task Quarterly 21 (2017) 309-319.
- M. Lewandowska, J. Rumiński, T. Kocejko, J. Nowak, Measuring pulse rate with a webcama non-contact method for evaluating cardiac activity, in: Computer Sci- ence and Information Systems (FedCSIS), 2011 Federated Conference on, IEEE, 520 2011, pp. 405-410.
- J. Ruminski, A. Kwasniewska, Evaluation of respiration rate using thermal imag- ing in mobile conditions, in: Application of Infrared to Biomedical Sciences, Springer, 2017, pp. 311-346. otwiera się w nowej karcie
- M. Hanmandlu, et al., A new entropy function and a classifier for thermal face 525 recognition, Engineering Applications of Artificial Intelligence 36 (2014) 269- 286.
- A. Kwaśniewska, J. Rumiński, Face detection in image sequences using a portable thermal camera, in: Proceedings of the 13th Quantitative Infrared Ther- mography Conference, 2016. otwiera się w nowej karcie
- A. Kwaśniewska, J. Rumiński, K. Czuszyński, M. Szankin, Real-time facial fea- tures detection from low resolution thermal images with deep classification mod- els, Journal of Medical Imaging and Health Informatics 8 (5) (2018) 979-987. otwiera się w nowej karcie
- M. Szankin, A. Kwasniewska, T. Sirlapu, M. Wang, J. Ruminski, R. Nicolas, M. Bartscherer, Long distance vital signs monitoring with person identification 535 for smart home solutions, in: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2018, pp. 1558-1561. otwiera się w nowej karcie
- I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, Deep learning, Vol. 1, MIT press Cambridge, 2016.
- R. Gade, T. B. Moeslund, Thermal cameras and applications: a survey, Machine vision and applications 25 (1) (2014) 245-262. otwiera się w nowej karcie
- Flir lepton camera modules, https://www.flir.com/products/ lepton/, accessed: 2018-11-10. otwiera się w nowej karcie
- J. Rumiński, Analysis of the parameters of respiration patterns extracted from 545 thermal image sequences, in: Biocybernetics and Biomedical Engineering, Vol. 36, 2016, pp. 732-741. otwiera się w nowej karcie
- B. Qi, V. John, Z. Liu, S. Mita, Pedestrian detection from thermal images: A sparse representation based approach, Infrared Physics & Technology 76 (2016) otwiera się w nowej karcie
- K. A. R. J. N. R. Szankin, Maciej, Road condition evaluation using fusion of multiple deep models on always-on vision processor, in: Proc. Of the 44th Annual Conference of the IEEE Industrial Electronics Society, in print, 2018. otwiera się w nowej karcie
- W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, A. C. Berg, Ssd: Single shot multibox detector, in: European conference on computer vision, 555 otwiera się w nowej karcie
- K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. otwiera się w nowej karcie
- V. Nair, G. E. Hinton, Rectified linear units improve restricted boltzmann ma- 560 chines, in: Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807-814.
- K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human- level performance on imagenet classification, in: Proceedings of the IEEE inter- national conference on computer vision, 2015, pp. 1026-1034. otwiera się w nowej karcie
- D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980. otwiera się w nowej karcie
- Tensorflow implementation of sr models, https://github.com/ LoSealL/VideoSuperResolution, accessed: 2018-10-10. otwiera się w nowej karcie
- Drrn repository, https://github.com/tyshiwo/DRRN_CVPR17, ac- 570 cessed: 2018-10-10. otwiera się w nowej karcie
- L. Torrey, J. Shavlik, Transfer learning, in: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, IGI Global, 2010, pp. 242-264. otwiera się w nowej karcie
- Tensorflow detection model zoo, https://github.com/tensorflow/ 575 models/blob/master/research/object_detection/g3doc/ detection_model_zoo.md, accessed: 2018-11-10. otwiera się w nowej karcie
- J. Bergstra, Y. Bengio, Random search for hyper-parameter optimization, Journal of Machine Learning Research 13 (Feb) (2012) 281-305.
- A. Hore, D. Ziou, Image quality metrics: Psnr vs. ssim, in: 2010 20th Interna- 580 tional Conference on Pattern Recognition, IEEE, 2010, pp. 2366-2369. otwiera się w nowej karcie
- Weryfikacja:
- Politechnika Gdańska
wyświetlono 123 razy