Abstrakt
Deep neural networks have achieved great success in many domains. However, successful deployment of such systems is determined by proper manual selection of the neural architecture. This is a tedious and time-consuming process that requires expert knowledge. Different tasks need very different architectures to obtain satisfactory results. The group of methods called the neural architecture search (NAS) helps to find effective architecture in an automated manner. In this paper, we present the use of an architecture search framework to solve the medical task of malignant melanoma detection. Unlike many other methods tested on benchmark datasets, we tested it on practical problem, which differs greatly in terms of difficulty in distinguishing between classes, resolution of images, data balance within the classes, and the number of data available. In order to find a suitable network structure, the hill-climbing search strategy was employed along with network morphism operations to explore the search space. The network morphism operations allow for incremental increases in the network size with the use of the previously trained network. This kind of knowledge reusing allows significantly reducing the computational cost. The proposed approach produces structures that achieve similar results to those provided by manually designed structures, at the same time making use of almost 20 times fewer parameters. What is more, the search process lasts on average only 18h on single GPU.
Cytowania
-
3 7
CrossRef
-
2 7
Web of Science
-
3 6
Scopus
Autorzy (3)
Cytuj jako
Pełna treść
- Wersja publikacji
- Accepted albo Published Version
- Licencja
-
otwiera się w nowej karcie
Słowa kluczowe
Informacje szczegółowe
- Kategoria:
- Publikacja w czasopiśmie
- Typ:
- artykuły w czasopismach
- Opublikowano w:
-
IEEE Access
nr 8,
strony 9061 - 9071,
ISSN: 2169-3536 - Język:
- angielski
- Rok wydania:
- 2020
- Opis bibliograficzny:
- Kwasigroch A., Grochowski M., Mikołajczyk A.: Neural Architecture Search for Skin Lesion Classification// IEEE Access -Vol. 8, (2020), s.9061-9071
- DOI:
- Cyfrowy identyfikator dokumentu elektronicznego (otwiera się w nowej karcie) 10.1109/access.2020.2964424
- Bibliografia: test
-
- Y. LeCun, Y. Bengio, and G. Hinton, ''Deep learning,'' Nature, vol. 521, no. 7553, pp. 436-444, May 2015. otwiera się w nowej karcie
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, ''ImageNet classification with deep convolutional neural networks,'' in Proc. Adv. Neural Inf. Pro- cess. Syst., Lake Tahoe, NV, USA, Dec. 2012, pp. 1106-1114. otwiera się w nowej karcie
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, ''ImageNet: A large-scale hierarchical image database,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009. otwiera się w nowej karcie
- J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, ''You only look once: Unified, real-time object detection,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 779-788. otwiera się w nowej karcie
- S. Ren, K. He, R. B. Girshick, and J. Sun, ''Faster R-CNN: Towards real- time object detection with region proposal networks,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, Jun. 2017. otwiera się w nowej karcie
- Y. Zhu, K. Sapra, F. A. Reda, K. J. Shih, S. Newsam, A. Tao, and B. Catanzaro, ''Improving semantic segmentation via video propagation and label relaxation,'' Dec. 2018, arXiv:1812.01593. [Online]. Available: https://arxiv.org/abs/1812.01593 otwiera się w nowej karcie
- T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, ''Unsupervised learning of depth and ego-motion from video,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6612-6619. otwiera się w nowej karcie
- Z. Chen and X. Huang, ''End-to-end learning for lane keeping of self- driving cars,'' in Proc. IEEE Intell. Vehicles Symp. (IV), Jun. 2017, pp. 1856-1860. otwiera się w nowej karcie
- C. Barata, M. E. Celebi, and J. S. Marques, ''A survey of feature extraction in dermoscopy image analysis of skin cancer,'' IEEE J. Biomed. Health Inform., vol. 23, no. 3, pp. 1096-1109, May 2019. otwiera się w nowej karcie
- X. Liu, W. Liu, T. Mei, and H. Ma, ''A deep learning-based approach to progressive vehicle re-identification for urban surveillance,'' in Proc. ECCV, 2016, pp. 869-884. otwiera się w nowej karcie
- A. Mikolajczyk and M. Grochowski, ''Data augmentation for improving deep learning in image classification problem,'' in Proc. Int. Interdiscipl. PhD Workshop (IIPhDW), May 2018, pp. 117-122. otwiera się w nowej karcie
- L. Gatys, A. Ecker, and M. Bethge, ''A neural algorithm of artistic style,'' J. Vis., vol. 16, no. 12, p. 326, Sep. 2016. otwiera się w nowej karcie
- X. Yi, E. Walia, and P. Babyn, ''Generative adversarial network in medical imaging: A review,'' Med. Image Anal., vol. 58, Dec. 2019, Art. no. 101552. otwiera się w nowej karcie
- M. Pietikäinen, A. Hadid, G. Zhao, and T. Ahonen, Computer Vision Using Local Binary Patterns. London, U.K.: Springer-Verlag, 2011. otwiera się w nowej karcie
- K. Simonyan and A. Zisserman, ''Very deep convolutional networks for large-scale image recognition,'' in Proc. 3rd Int. Conf. Learn. Repre- sent. (ICLR), San Diego, CA, USA, May 2015, pp. 1-10.
- G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, ''Densely connected convolutional networks,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2261-2269. otwiera się w nowej karcie
- K. He, X. Zhang, S. Ren, and J. Sun, ''Deep residual learning for image recognition,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770-778. otwiera się w nowej karcie
- S. Zagoruyko and N. Komodakis, ''Wide residual networks,'' in Proc. Brit. Mach. Vis. Conf. (BMVC), York, U.K., Sep. 2016, pp. 1-15. otwiera się w nowej karcie
- M. Tan and Q. V. Le, ''EfficientNet: Rethinking model scaling for convo- lutional neural networks,'' in Proc. 36th Int. Conf. Mach. Learn. (ICML), Long Beach, CA, USA, 2019, pp. 6105-6114.
- D. P. Kingma and J. Ba, ''Adam: A method for stochastic optimization,'' in Proc. 3rd Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, May 2015, pp. 1-15.
- I. Loshchilov and F. Hutter, ''SGDR: Stochastic gradient descent with warm restarts,'' in Proc. 5th Int. Conf. Learn. Represent. (ICLR), Toulon, France, Apr. 2017.
- S. Ioffe and C. Szegedy, ''Batch normalization: Accelerating deep net- work training by reducing internal covariate shift,'' in Proc. 32nd
- Int. Conf. Mach. Learn. (ICML), Lille, France, vol. 37, Jul. 2015, pp. 448-456.
- N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi- nov, ''Dropout: A simple way to prevent neural networks from overfitting,'' J. Mach. Learn. Res., vol. 15, pp. 1929-1958, Jan. 2014.
- A. L. Maas, ''Rectifier nonlinearities improve neural network acoustic models,'' in Proc. ICML, 2013, vol. 30, no. 1, p. 3.
- G. Zhang and H. Li, ''Effectiveness of scaled exponentially-regularized linear units (SERLUs),'' Jul. 2018, arXiv:1807.10117. [Online]. Available: https://arxiv.org/abs/1807.10117
- T. DeVries and G. W. Taylor, ''Improved regularization of convolutional neural networks with cutout,'' Aug. 2017, arXiv:1708.04552. [Online].
- A. Mikolajczyk and M. Grochowski, ''Style transfer-based image syn- thesis as an efficient regularization technique in deep learning,'' in Proc. 24th Int. Conf. Methods Models Autom. Robot. (MMAR), Aug. 2019, pp. 42-47. otwiera się w nowej karcie
- C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, ''A survey on deep transfer learning,'' in Proc. 27th Int. Conf. Artif. Neural Netw. (ICANN), Rhodes, Greece, vol. 11141, Oct. 2018, pp. 270-279. otwiera się w nowej karcie
- T. Elsken, J. H. Metzen, and F. Hutter, ''Neural architecture search: A sur- vey,'' J. Mach. Learn. Res., vol. 20, pp. 55:1-55:21, Mar. 2019. otwiera się w nowej karcie
- B. Zoph and Q. V. Le, ''Neural architecture search with reinforcement learning,'' in Proc. 5th Int. Conf. Learn. Represent. (ICLR), Toulon, France, Apr. 2017, pp. 1-37.
- E. Real, ''Large-scale evolution of image classifiers,'' in Proc. 34th Int. Conf. Mach. Learn. (ICML), Sydney, NSW, Australia, vol. 70, Aug. 2017, pp. 2902-2911.
- T. Chen, I. J. Goodfellow, and J. Shlens, ''Net2Net: Accelerating learning via knowledge transfer,'' in Proc. 4th Int. Conf. Learn. Represent. (ICLR), San Juan, Puerto Rico, May 2016, pp. 1-12. otwiera się w nowej karcie
- H. Liu, K. Simonyan, and Y. Yang, ''DARTS: Differentiable architecture search,'' in Proc. 7th Int. Conf. Learn. Represent. (ICLR), New Orleans, LA, USA, May 2019, pp. 1-13.
- ISIC Archive. Accessed: Sep. 24, 2019. [Online]. Available: https:// www.isic-archive.com otwiera się w nowej karcie
- J. Bergstra and Y. Bengio, ''Random search for hyper-parameter optimiza- tion,'' J. Mach. Learn. Res., vol. 13, pp. 281-305, Feb. 2012.
- K. O. Stanley and R. Miikkulainen, ''Evolving neural networks through augmenting topologies,'' Evol. Comput., vol. 10, no. 2, pp. 99-127, Jun. 2002. otwiera się w nowej karcie
- D. Floreano, P. Dürr, and C. Mattiussi, ''Neuroevolution: From architec- tures to learning,'' Evol. Intel., vol. 1, no. 1, pp. 47-62, Mar. 2008. otwiera się w nowej karcie
- B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, ''Learning transferable architectures for scalable image recognition,'' in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake City, UT, USA, Jun. 2018, pp. 8697-8710. otwiera się w nowej karcie
- E. Real, A. Aggarwal, Y. Huang, and Q. V. Le, ''Regularized evolu- tion for image classifier architecture search,'' in Proc. AAAI, vol. 33, pp. 4780-4789, Sep. 2019. otwiera się w nowej karcie
- H. Mendoza, A. Klein, M. Feurer, J. T. Springenberg, and F. Hutter, ''Towards automatically-tuned neural networks,'' in Proc. 33rd Workshop Autom. Mach. Learn. (AutoML) 2016, Int. Conf. Mach. Learn. (ICML), New York, NY, USA, Jun. 2016, vol. 64, pp. 58-65. otwiera się w nowej karcie
- J. Bergstra, D. Yamins, and D. D. Cox, ''Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures,'' in Proc. 30th Int. Conf. Mach. Learn. (ICML), Atlanta, GA, USA, Jun. 2013, vol. 28, pp. 115-123. otwiera się w nowej karcie
- H. Cai, L. Zhu, and S. Han, ''ProxylessNAS: Direct neural architecture search on target task and hardware,'' in Proc. 7th Int. Conf. Learn. Repre- sent. (ICLR), New Orleans, LA, USA, May 2019, pp. 1-13.
- Y. Weng, T. Zhou, L. Liu, and C. Xia, ''Automatic convolutional neural architecture search for image classification under different scenes,'' IEEE Access, vol. 7, pp. 38495-38506, 2019. otwiera się w nowej karcie
- Y. Weng, T. Zhou, Y. Li, and X. Qiu, ''NAS-Unet: Neural architec- ture search for medical image segmentation,'' IEEE Access, vol. 7, pp. 44247-44257, 2019. otwiera się w nowej karcie
- A. Zela, A. Klein, S. Falkner, and F. Hutter, ''Towards automated deep learning: Efficient joint neural architecture and hyperparame- ter search,'' Jul. 2018, arXiv:1807.06906. [Online]. Available: https:// arxiv.org/abs/1807.06906
- A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter, ''Fast Bayesian optimization of machine learning hyperparameters on large datasets,'' in Proc. 20th Int. Conf. Artif. Intell. Statist. (AISTATS), Fort Lauderdale, FL, USA, vol. 54, Apr. 2017, pp. 528-536. otwiera się w nowej karcie
- P. Chrabaszcz, I. Loshchilov, and F. Hutter, ''A downsampled vari- ant of ImageNet as an alternative to the CIFAR datasets,'' Jul. 2017, arXiv:1707.08819. [Online]. Available: https://arxiv.org/abs/1707.08819
- B. Baker, O. Gupta, R. Raskar, and N. Naik, ''Accelerating neural architec- ture search using performance prediction,'' in Proc. 6th Int. Conf. Learn. Represent. (ICLR), Vancouver, BC, Canada, Apr./May 2018, pp. 1-14.
- S. Hochreiter and J. Schmidhuber, ''Long short-term memory,'' Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997. otwiera się w nowej karcie
- R. J. Williams, ''Simple statistical gradient-following algorithms for connectionist reinforcement learning,'' Mach. Learn., vol. 8, nos. 3-4, pp. 229-256, May 1992. otwiera się w nowej karcie
- T. Wei, C. Wang, Y. Rui, and C. W. Chen, ''Network morphism,'' in Proc. 33nd Int. Conf. Mach. Learn. (ICML), New York City, NY, USA, vol. 48, Jun. 2016, pp. 564-572. otwiera się w nowej karcie
- H. Cai, T. Chen, W. Zhang, Y. Yu, and J. Wang, ''Efficient architec- ture search by network transformation,'' in Proc. 32nd AAAI Conf. Artif. Intell., (AAAI), 30th Innov. Appl. Artif. Intell. (IAAI), 8th AAAI Symp. Educ. Adv. Artif. Intell. (EAAI), New Orleans, LA, USA, Feb. 2018, pp. 2787-2794. otwiera się w nowej karcie
- H. Cai, J. Yang, W. Zhang, S. Han, and Y. Yu, ''Path-level network trans- formation for efficient architecture search,'' in Proc. 35th Int. Conf. Mach. Learn. (ICML), Stockholm, Sweden, vol. 80, Jul. 2018, pp. 677-686.
- T. Elsken, J. H. Metzen, and F. Hutter, ''Simple and efficient architecture search for convolutional neural networks,'' in Proc. 6th Int. Conf. Learn. Represent. (ICLR), Vancouver, BC, Canada, Apr./May 2018, pp. 1-14. otwiera się w nowej karcie
- F. Nachbar, W. Stolz, T. Merkle, A. B. Cognetta, T. Vogt, M. Landthaler, P. Bilek, O. Braun-Falco, and G. Plewig, ''The ABCD rule of der- matoscopy: High prospective value in the diagnosis of doubtful melanocytic skin lesions,'' J. Amer. Acad. Dermatol., vol. 30, no. 4, pp. 551-559, Apr. 1994. otwiera się w nowej karcie
- A. Mikołajczyk, A. Kwasigroch, and M. Grochowski, ''Intelligent system supporting diagnosis of malignant melanoma,'' in Proc. Polish Control Conf., 2017, pp. 828-837. otwiera się w nowej karcie
- M. Grochowski, A. Kwasigroch, and A. Mikołajczyk, ''Selected technical issues of deep neural networks for image classification purposes,'' Bull. Polish Acad. Sci., Tech. Sci., vol. 67, no. no. 2, pp. 363-376, 2019.
- I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. VOLUME 8, 2020
- Weryfikacja:
- Politechnika Gdańska
wyświetlono 176 razy