Neural Architecture Search for Skin Lesion Classification - Publikacja - MOST Wiedzy

Wyszukiwarka

Neural Architecture Search for Skin Lesion Classification

Abstrakt

Deep neural networks have achieved great success in many domains. However, successful deployment of such systems is determined by proper manual selection of the neural architecture. This is a tedious and time-consuming process that requires expert knowledge. Different tasks need very different architectures to obtain satisfactory results. The group of methods called the neural architecture search (NAS) helps to find effective architecture in an automated manner. In this paper, we present the use of an architecture search framework to solve the medical task of malignant melanoma detection. Unlike many other methods tested on benchmark datasets, we tested it on practical problem, which differs greatly in terms of difficulty in distinguishing between classes, resolution of images, data balance within the classes, and the number of data available. In order to find a suitable network structure, the hill-climbing search strategy was employed along with network morphism operations to explore the search space. The network morphism operations allow for incremental increases in the network size with the use of the previously trained network. This kind of knowledge reusing allows significantly reducing the computational cost. The proposed approach produces structures that achieve similar results to those provided by manually designed structures, at the same time making use of almost 20 times fewer parameters. What is more, the search process lasts on average only 18h on single GPU.

Cytowania

  • 4 4

    CrossRef

  • 0

    Web of Science

  • 4 5

    Scopus

Cytuj jako

Pełna treść

pobierz publikację
pobrano 98 razy
Wersja publikacji
Accepted albo Published Version
Licencja
Creative Commons: CC-BY otwiera się w nowej karcie

Słowa kluczowe

Informacje szczegółowe

Kategoria:
Publikacja w czasopiśmie
Typ:
artykuły w czasopismach
Opublikowano w:
IEEE Access nr 8, strony 9061 - 9071,
ISSN: 2169-3536
Język:
angielski
Rok wydania:
2020
Opis bibliograficzny:
Kwasigroch A., Grochowski M., Mikołajczyk A.: Neural Architecture Search for Skin Lesion Classification// IEEE Access -Vol. 8, (2020), s.9061-9071
DOI:
Cyfrowy identyfikator dokumentu elektronicznego (otwiera się w nowej karcie) 10.1109/access.2020.2964424
Bibliografia: test
  1. Y. LeCun, Y. Bengio, and G. Hinton, ''Deep learning,'' Nature, vol. 521, no. 7553, pp. 436-444, May 2015. otwiera się w nowej karcie
  2. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ''ImageNet classification with deep convolutional neural networks,'' in Proc. Adv. Neural Inf. Pro- cess. Syst., Lake Tahoe, NV, USA, Dec. 2012, pp. 1106-1114. otwiera się w nowej karcie
  3. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, ''ImageNet: A large-scale hierarchical image database,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009. otwiera się w nowej karcie
  4. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, ''You only look once: Unified, real-time object detection,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 779-788. otwiera się w nowej karcie
  5. S. Ren, K. He, R. B. Girshick, and J. Sun, ''Faster R-CNN: Towards real- time object detection with region proposal networks,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, Jun. 2017. otwiera się w nowej karcie
  6. Y. Zhu, K. Sapra, F. A. Reda, K. J. Shih, S. Newsam, A. Tao, and B. Catanzaro, ''Improving semantic segmentation via video propagation and label relaxation,'' Dec. 2018, arXiv:1812.01593. [Online]. Available: https://arxiv.org/abs/1812.01593 otwiera się w nowej karcie
  7. T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, ''Unsupervised learning of depth and ego-motion from video,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6612-6619. otwiera się w nowej karcie
  8. Z. Chen and X. Huang, ''End-to-end learning for lane keeping of self- driving cars,'' in Proc. IEEE Intell. Vehicles Symp. (IV), Jun. 2017, pp. 1856-1860. otwiera się w nowej karcie
  9. C. Barata, M. E. Celebi, and J. S. Marques, ''A survey of feature extraction in dermoscopy image analysis of skin cancer,'' IEEE J. Biomed. Health Inform., vol. 23, no. 3, pp. 1096-1109, May 2019. otwiera się w nowej karcie
  10. X. Liu, W. Liu, T. Mei, and H. Ma, ''A deep learning-based approach to progressive vehicle re-identification for urban surveillance,'' in Proc. ECCV, 2016, pp. 869-884. otwiera się w nowej karcie
  11. A. Mikolajczyk and M. Grochowski, ''Data augmentation for improving deep learning in image classification problem,'' in Proc. Int. Interdiscipl. PhD Workshop (IIPhDW), May 2018, pp. 117-122. otwiera się w nowej karcie
  12. L. Gatys, A. Ecker, and M. Bethge, ''A neural algorithm of artistic style,'' J. Vis., vol. 16, no. 12, p. 326, Sep. 2016. otwiera się w nowej karcie
  13. X. Yi, E. Walia, and P. Babyn, ''Generative adversarial network in medical imaging: A review,'' Med. Image Anal., vol. 58, Dec. 2019, Art. no. 101552. otwiera się w nowej karcie
  14. M. Pietikäinen, A. Hadid, G. Zhao, and T. Ahonen, Computer Vision Using Local Binary Patterns. London, U.K.: Springer-Verlag, 2011. otwiera się w nowej karcie
  15. K. Simonyan and A. Zisserman, ''Very deep convolutional networks for large-scale image recognition,'' in Proc. 3rd Int. Conf. Learn. Repre- sent. (ICLR), San Diego, CA, USA, May 2015, pp. 1-10.
  16. G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, ''Densely connected convolutional networks,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2261-2269. otwiera się w nowej karcie
  17. K. He, X. Zhang, S. Ren, and J. Sun, ''Deep residual learning for image recognition,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770-778. otwiera się w nowej karcie
  18. S. Zagoruyko and N. Komodakis, ''Wide residual networks,'' in Proc. Brit. Mach. Vis. Conf. (BMVC), York, U.K., Sep. 2016, pp. 1-15. otwiera się w nowej karcie
  19. M. Tan and Q. V. Le, ''EfficientNet: Rethinking model scaling for convo- lutional neural networks,'' in Proc. 36th Int. Conf. Mach. Learn. (ICML), Long Beach, CA, USA, 2019, pp. 6105-6114.
  20. D. P. Kingma and J. Ba, ''Adam: A method for stochastic optimization,'' in Proc. 3rd Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, May 2015, pp. 1-15.
  21. I. Loshchilov and F. Hutter, ''SGDR: Stochastic gradient descent with warm restarts,'' in Proc. 5th Int. Conf. Learn. Represent. (ICLR), Toulon, France, Apr. 2017.
  22. S. Ioffe and C. Szegedy, ''Batch normalization: Accelerating deep net- work training by reducing internal covariate shift,'' in Proc. 32nd
  23. Int. Conf. Mach. Learn. (ICML), Lille, France, vol. 37, Jul. 2015, pp. 448-456.
  24. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi- nov, ''Dropout: A simple way to prevent neural networks from overfitting,'' J. Mach. Learn. Res., vol. 15, pp. 1929-1958, Jan. 2014.
  25. A. L. Maas, ''Rectifier nonlinearities improve neural network acoustic models,'' in Proc. ICML, 2013, vol. 30, no. 1, p. 3.
  26. G. Zhang and H. Li, ''Effectiveness of scaled exponentially-regularized linear units (SERLUs),'' Jul. 2018, arXiv:1807.10117. [Online]. Available: https://arxiv.org/abs/1807.10117
  27. T. DeVries and G. W. Taylor, ''Improved regularization of convolutional neural networks with cutout,'' Aug. 2017, arXiv:1708.04552. [Online].
  28. A. Mikolajczyk and M. Grochowski, ''Style transfer-based image syn- thesis as an efficient regularization technique in deep learning,'' in Proc. 24th Int. Conf. Methods Models Autom. Robot. (MMAR), Aug. 2019, pp. 42-47. otwiera się w nowej karcie
  29. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, ''A survey on deep transfer learning,'' in Proc. 27th Int. Conf. Artif. Neural Netw. (ICANN), Rhodes, Greece, vol. 11141, Oct. 2018, pp. 270-279. otwiera się w nowej karcie
  30. T. Elsken, J. H. Metzen, and F. Hutter, ''Neural architecture search: A sur- vey,'' J. Mach. Learn. Res., vol. 20, pp. 55:1-55:21, Mar. 2019. otwiera się w nowej karcie
  31. B. Zoph and Q. V. Le, ''Neural architecture search with reinforcement learning,'' in Proc. 5th Int. Conf. Learn. Represent. (ICLR), Toulon, France, Apr. 2017, pp. 1-37.
  32. E. Real, ''Large-scale evolution of image classifiers,'' in Proc. 34th Int. Conf. Mach. Learn. (ICML), Sydney, NSW, Australia, vol. 70, Aug. 2017, pp. 2902-2911.
  33. T. Chen, I. J. Goodfellow, and J. Shlens, ''Net2Net: Accelerating learning via knowledge transfer,'' in Proc. 4th Int. Conf. Learn. Represent. (ICLR), San Juan, Puerto Rico, May 2016, pp. 1-12. otwiera się w nowej karcie
  34. H. Liu, K. Simonyan, and Y. Yang, ''DARTS: Differentiable architecture search,'' in Proc. 7th Int. Conf. Learn. Represent. (ICLR), New Orleans, LA, USA, May 2019, pp. 1-13.
  35. ISIC Archive. Accessed: Sep. 24, 2019. [Online]. Available: https:// www.isic-archive.com otwiera się w nowej karcie
  36. J. Bergstra and Y. Bengio, ''Random search for hyper-parameter optimiza- tion,'' J. Mach. Learn. Res., vol. 13, pp. 281-305, Feb. 2012.
  37. K. O. Stanley and R. Miikkulainen, ''Evolving neural networks through augmenting topologies,'' Evol. Comput., vol. 10, no. 2, pp. 99-127, Jun. 2002. otwiera się w nowej karcie
  38. D. Floreano, P. Dürr, and C. Mattiussi, ''Neuroevolution: From architec- tures to learning,'' Evol. Intel., vol. 1, no. 1, pp. 47-62, Mar. 2008. otwiera się w nowej karcie
  39. B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, ''Learning transferable architectures for scalable image recognition,'' in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake City, UT, USA, Jun. 2018, pp. 8697-8710. otwiera się w nowej karcie
  40. E. Real, A. Aggarwal, Y. Huang, and Q. V. Le, ''Regularized evolu- tion for image classifier architecture search,'' in Proc. AAAI, vol. 33, pp. 4780-4789, Sep. 2019. otwiera się w nowej karcie
  41. H. Mendoza, A. Klein, M. Feurer, J. T. Springenberg, and F. Hutter, ''Towards automatically-tuned neural networks,'' in Proc. 33rd Workshop Autom. Mach. Learn. (AutoML) 2016, Int. Conf. Mach. Learn. (ICML), New York, NY, USA, Jun. 2016, vol. 64, pp. 58-65. otwiera się w nowej karcie
  42. J. Bergstra, D. Yamins, and D. D. Cox, ''Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures,'' in Proc. 30th Int. Conf. Mach. Learn. (ICML), Atlanta, GA, USA, Jun. 2013, vol. 28, pp. 115-123. otwiera się w nowej karcie
  43. H. Cai, L. Zhu, and S. Han, ''ProxylessNAS: Direct neural architecture search on target task and hardware,'' in Proc. 7th Int. Conf. Learn. Repre- sent. (ICLR), New Orleans, LA, USA, May 2019, pp. 1-13.
  44. Y. Weng, T. Zhou, L. Liu, and C. Xia, ''Automatic convolutional neural architecture search for image classification under different scenes,'' IEEE Access, vol. 7, pp. 38495-38506, 2019. otwiera się w nowej karcie
  45. Y. Weng, T. Zhou, Y. Li, and X. Qiu, ''NAS-Unet: Neural architec- ture search for medical image segmentation,'' IEEE Access, vol. 7, pp. 44247-44257, 2019. otwiera się w nowej karcie
  46. A. Zela, A. Klein, S. Falkner, and F. Hutter, ''Towards automated deep learning: Efficient joint neural architecture and hyperparame- ter search,'' Jul. 2018, arXiv:1807.06906. [Online]. Available: https:// arxiv.org/abs/1807.06906
  47. A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter, ''Fast Bayesian optimization of machine learning hyperparameters on large datasets,'' in Proc. 20th Int. Conf. Artif. Intell. Statist. (AISTATS), Fort Lauderdale, FL, USA, vol. 54, Apr. 2017, pp. 528-536. otwiera się w nowej karcie
  48. P. Chrabaszcz, I. Loshchilov, and F. Hutter, ''A downsampled vari- ant of ImageNet as an alternative to the CIFAR datasets,'' Jul. 2017, arXiv:1707.08819. [Online]. Available: https://arxiv.org/abs/1707.08819
  49. B. Baker, O. Gupta, R. Raskar, and N. Naik, ''Accelerating neural architec- ture search using performance prediction,'' in Proc. 6th Int. Conf. Learn. Represent. (ICLR), Vancouver, BC, Canada, Apr./May 2018, pp. 1-14.
  50. S. Hochreiter and J. Schmidhuber, ''Long short-term memory,'' Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997. otwiera się w nowej karcie
  51. R. J. Williams, ''Simple statistical gradient-following algorithms for connectionist reinforcement learning,'' Mach. Learn., vol. 8, nos. 3-4, pp. 229-256, May 1992. otwiera się w nowej karcie
  52. T. Wei, C. Wang, Y. Rui, and C. W. Chen, ''Network morphism,'' in Proc. 33nd Int. Conf. Mach. Learn. (ICML), New York City, NY, USA, vol. 48, Jun. 2016, pp. 564-572. otwiera się w nowej karcie
  53. H. Cai, T. Chen, W. Zhang, Y. Yu, and J. Wang, ''Efficient architec- ture search by network transformation,'' in Proc. 32nd AAAI Conf. Artif. Intell., (AAAI), 30th Innov. Appl. Artif. Intell. (IAAI), 8th AAAI Symp. Educ. Adv. Artif. Intell. (EAAI), New Orleans, LA, USA, Feb. 2018, pp. 2787-2794. otwiera się w nowej karcie
  54. H. Cai, J. Yang, W. Zhang, S. Han, and Y. Yu, ''Path-level network trans- formation for efficient architecture search,'' in Proc. 35th Int. Conf. Mach. Learn. (ICML), Stockholm, Sweden, vol. 80, Jul. 2018, pp. 677-686.
  55. T. Elsken, J. H. Metzen, and F. Hutter, ''Simple and efficient architecture search for convolutional neural networks,'' in Proc. 6th Int. Conf. Learn. Represent. (ICLR), Vancouver, BC, Canada, Apr./May 2018, pp. 1-14. otwiera się w nowej karcie
  56. F. Nachbar, W. Stolz, T. Merkle, A. B. Cognetta, T. Vogt, M. Landthaler, P. Bilek, O. Braun-Falco, and G. Plewig, ''The ABCD rule of der- matoscopy: High prospective value in the diagnosis of doubtful melanocytic skin lesions,'' J. Amer. Acad. Dermatol., vol. 30, no. 4, pp. 551-559, Apr. 1994. otwiera się w nowej karcie
  57. A. Mikołajczyk, A. Kwasigroch, and M. Grochowski, ''Intelligent system supporting diagnosis of malignant melanoma,'' in Proc. Polish Control Conf., 2017, pp. 828-837. otwiera się w nowej karcie
  58. M. Grochowski, A. Kwasigroch, and A. Mikołajczyk, ''Selected technical issues of deep neural networks for image classification purposes,'' Bull. Polish Acad. Sci., Tech. Sci., vol. 67, no. no. 2, pp. 363-376, 2019.
  59. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. VOLUME 8, 2020
Weryfikacja:
Politechnika Gdańska

wyświetlono 191 razy

Publikacje, które mogą cię zainteresować

Meta Tagi