Analysis of Denoising Autoencoder Properties Through Misspelling Correction Task - Publication - Bridge of Knowledge

Your account

login

Search

Analysis of Denoising Autoencoder Properties Through Misspelling Correction Task

Abstract

The paper analyzes some properties of denoising autoencoders using the problem of misspellings correction as an exemplary task. We evaluate the capacity of the network in its classical feed-forward form. We also propose a modification to the output layer of the net, which we called multi-softmax. Experiments show that the model trained with this output layer outperforms traditional network both in learning time and accuracy. We test the influence of the noise introduced to training data on the learning speed and generalization quality. The proposed approach of evaluating various properties of autoencoders using misspellings correction task serves as an open framework for further experiments, e.g. incorporating other neural network topologies into an autoencoder setting.

Citations

  • 2

    CrossRef

  • 0

    Web of Science

  • 3

    Scopus

Authors (2)

Cite as

Full text

download paper
downloaded 141 times
Publication version
Accepted or Published Version
License
Copyright (Springer International Publishing AG 2017)

Keywords

Details

Category:
Conference activity
Type:
materiały konferencyjne indeksowane w Web of Science
Title of issue:
9th International Conference on Computational Collective Intelligence (ICCCI) strony 438 - 447
Language:
English
Publication year:
2017
Bibliographic description:
Draszawka K., Szymański J..: Analysis of Denoising Autoencoder Properties Through Misspelling Correction Task, W: 9th International Conference on Computational Collective Intelligence (ICCCI), 2017, ,.
DOI:
Digital Object Identifier (open in new tab) 10.1007/978-3-319-67077-5_42
Bibliography: test
  1. Fausett, L.V.: Fundamentals of neural networks. Prentice-Hall (1994) open in new tab
  2. Kukich, K.: Techniques for automatically correcting words in text. ACM Comput- ing Surveys (CSUR) 24 (1992) 377439 open in new tab
  3. Baeza-Yates, G. Navarro, R.: Faster approximate string matching. Algorithmica 23 (1999) 127158
  4. Szymanski, J., Boinski, T.: Improvement of imperfect string matching based on asymmetric n-grams. In: Computational Collective Intelligence. Technologies and Applications -5th International Conference, ICCCI 2013, Craiova, Romania, September 11-13, 2013, Proceedings. (2013) 306315 open in new tab
  5. Astrain, J.J., Garitagoitia, J.R., Villadangos, J.E., Fariña, F., Córdoba, A., de Mendvil, J.G.: An imperfect string matching experience using deformed fuzzy automata. In: HIS. (2002) 115123 open in new tab
  6. Boguszewski, A., Szyma«ski, J., Draszawka, K.: Towards increasing f-measure of approximate string matching in o(1) complexity. In: 2016 Federated Conference on Computer Science and Information Systems (FedCSIS). (2016) 527532 open in new tab
  7. Hantler, S.L., Laker, M.M., Lenchner, J., Milch, D.: Methods and apparatus for performing spelling corrections using one or more variant hash tables (2006) US Patent App. 11/513,782. open in new tab
  8. Udupa, R., Kumar, S.: Hashing-based approaches to spelling correction of personal names. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics (2010) 12561265
  9. Rubinstein, R.Y., Kroese, D.P.: The cross-entropy method: a unied approach to combinatorial optimization, Monte-Carlo simulation and machine learning. Springer Science & Business Media (2013)
  10. Jollie, I.: Principal component analysis. Wiley Online Library (2002) open in new tab
  11. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. science 313 (2006) 504507 open in new tab
  12. Bengio, Y., et al.: Learning deep architectures for ai. Foundations and trends R in Machine Learning 2 (2009) 1127 open in new tab
  13. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denois- ing autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11 (2010) 33713408 open in new tab
  14. Lu, X., Tsao, Y., Matsuda, S., Hori, C.: Speech enhancement based on deep de- noising autoencoder. In: Interspeech. (2013) 436440
  15. Agostinelli, F., Anderson, M.R., Lee, H.: Adaptive multi-column deep neural net- works with application to robust image denoising. In: Advances in Neural Infor- mation Processing Systems. (2013) 14931501
  16. Chollet, F.: Keras. https://github.com/fchollet/keras (2016) open in new tab
  17. Theano Development Team: Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints abs/1605.02688 (2016) open in new tab
  18. Zeiler, M.D.: Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012) open in new tab
  19. Kohonen, T.: The self-organizing map. Neurocomputing 21 (1998) 16 open in new tab
  20. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014) open in new tab
  21. Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016) open in new tab
  22. Botvinick, M.M., Plaut, D.C.: Short-term memory for serial order: a recurrent neural network model. Psychological review 113 (2006) 201 open in new tab
Verified by:
Gdańsk University of Technology

seen 151 times

Recommended for you

Automatic labeling of traffic sound recordings using autoencoder-derived features

2019

Condition-Based Monitoring of DC Motors Performed with Autoencoders

2022

Application of autoencoder to traffic noise analysis

2019

Using Long-Short term Memory networks with Genetic Algorithm to predict engine condition

2022
Meta Tags