Evolutionary construction of convolutional neural networks

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rivaling hand-crafted neural networks in terms of accuracy.
A two-step approach is introduced where a convolutional autoencoder is created that efficiently compresses the input data in the first step, and a convolutional neural network is created to classify the compressed data in the second step. The creation of networks in both steps is guided by by an evolutionary process, where new networks are constantly being generated by mutating members of a collection of existing networks. Additionally, a method is introduced that considers the trade-off between compression and information loss of different convolutional autoencoders. This is used to select the optimal convolutional autoencoder from among those evolved to compress the data for the second step.
The complete framework is implemented, tested on the popular CIFAR-10 data set, and the results are discussed. Finally, a number of possible directions for future work with this particular framework in mind are considered, including opportunities to improve its efficiency and its application in particular areas.
LanguageEnglish
Title of host publicationMachine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers
EditorsGiuseppe Nicosia, Giovanni Giuffrida, Giuseppe Nicosia, Panos Pardalos, Vincenzo Sciacca, Renato Umeton
Place of PublicationCham
PublisherSpringer
Pages293-304
Number of pages12
ISBN (Electronic)978-3-030-13709-0
ISBN (Print)978-3-030-13708-3
DOIs
StatePublished - Feb 2019
Event4th International Conference on Machine Learning, Optimization, and Data Science, LOD 2018 - Volterra, Italy
Duration: 13 Sep 201816 Sep 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11331 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Conference on Machine Learning, Optimization, and Data Science, LOD 2018
CountryItaly
CityVolterra
Period13/09/1816/09/18

Fingerprint

Neural networks
Evolutionary algorithms
Deep neural networks
Deep learning

Keywords

  • Convolutional autoencoders
  • Convolutional neural networks
  • Genetic algorithms
  • Neuro-evolution

Cite this

van Knippenberg, M. S., Menkovski, V., & Consoli, S. (2019). Evolutionary construction of convolutional neural networks. In G. Nicosia, G. Giuffrida, G. Nicosia, P. Pardalos, V. Sciacca, & R. Umeton (Eds.), Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers (pp. 293-304). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11331 LNCS). Cham: Springer. DOI: 10.1007/978-3-030-13709-0_25
van Knippenberg, M.S. ; Menkovski, V. ; Consoli, Sergio. / Evolutionary construction of convolutional neural networks. Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers. editor / Giuseppe Nicosia ; Giovanni Giuffrida ; Giuseppe Nicosia ; Panos Pardalos ; Vincenzo Sciacca ; Renato Umeton. Cham : Springer, 2019. pp. 293-304 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{d38584dfd23e4f839d90481db43a95ff,
title = "Evolutionary construction of convolutional neural networks",
abstract = "Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rivaling hand-crafted neural networks in terms of accuracy.A two-step approach is introduced where a convolutional autoencoder is created that efficiently compresses the input data in the first step, and a convolutional neural network is created to classify the compressed data in the second step. The creation of networks in both steps is guided by by an evolutionary process, where new networks are constantly being generated by mutating members of a collection of existing networks. Additionally, a method is introduced that considers the trade-off between compression and information loss of different convolutional autoencoders. This is used to select the optimal convolutional autoencoder from among those evolved to compress the data for the second step.The complete framework is implemented, tested on the popular CIFAR-10 data set, and the results are discussed. Finally, a number of possible directions for future work with this particular framework in mind are considered, including opportunities to improve its efficiency and its application in particular areas.",
keywords = "Convolutional autoencoders, Convolutional neural networks, Genetic algorithms, Neuro-evolution",
author = "{van Knippenberg}, M.S. and V. Menkovski and Sergio Consoli",
year = "2019",
month = "2",
doi = "10.1007/978-3-030-13709-0_25",
language = "English",
isbn = "978-3-030-13708-3",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer",
pages = "293--304",
editor = "Giuseppe Nicosia and Giovanni Giuffrida and Giuseppe Nicosia and Panos Pardalos and Vincenzo Sciacca and Renato Umeton",
booktitle = "Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers",
address = "Germany",

}

van Knippenberg, MS, Menkovski, V & Consoli, S 2019, Evolutionary construction of convolutional neural networks. in G Nicosia, G Giuffrida, G Nicosia, P Pardalos, V Sciacca & R Umeton (eds), Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11331 LNCS, Springer, Cham, pp. 293-304, 4th International Conference on Machine Learning, Optimization, and Data Science, LOD 2018, Volterra, Italy, 13/09/18. DOI: 10.1007/978-3-030-13709-0_25

Evolutionary construction of convolutional neural networks. / van Knippenberg, M.S.; Menkovski, V.; Consoli, Sergio.

Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers. ed. / Giuseppe Nicosia; Giovanni Giuffrida; Giuseppe Nicosia; Panos Pardalos; Vincenzo Sciacca; Renato Umeton. Cham : Springer, 2019. p. 293-304 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11331 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Evolutionary construction of convolutional neural networks

AU - van Knippenberg,M.S.

AU - Menkovski,V.

AU - Consoli,Sergio

PY - 2019/2

Y1 - 2019/2

N2 - Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rivaling hand-crafted neural networks in terms of accuracy.A two-step approach is introduced where a convolutional autoencoder is created that efficiently compresses the input data in the first step, and a convolutional neural network is created to classify the compressed data in the second step. The creation of networks in both steps is guided by by an evolutionary process, where new networks are constantly being generated by mutating members of a collection of existing networks. Additionally, a method is introduced that considers the trade-off between compression and information loss of different convolutional autoencoders. This is used to select the optimal convolutional autoencoder from among those evolved to compress the data for the second step.The complete framework is implemented, tested on the popular CIFAR-10 data set, and the results are discussed. Finally, a number of possible directions for future work with this particular framework in mind are considered, including opportunities to improve its efficiency and its application in particular areas.

AB - Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rivaling hand-crafted neural networks in terms of accuracy.A two-step approach is introduced where a convolutional autoencoder is created that efficiently compresses the input data in the first step, and a convolutional neural network is created to classify the compressed data in the second step. The creation of networks in both steps is guided by by an evolutionary process, where new networks are constantly being generated by mutating members of a collection of existing networks. Additionally, a method is introduced that considers the trade-off between compression and information loss of different convolutional autoencoders. This is used to select the optimal convolutional autoencoder from among those evolved to compress the data for the second step.The complete framework is implemented, tested on the popular CIFAR-10 data set, and the results are discussed. Finally, a number of possible directions for future work with this particular framework in mind are considered, including opportunities to improve its efficiency and its application in particular areas.

KW - Convolutional autoencoders

KW - Convolutional neural networks

KW - Genetic algorithms

KW - Neuro-evolution

UR - http://www.scopus.com/inward/record.url?scp=85063596775&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-13709-0_25

DO - 10.1007/978-3-030-13709-0_25

M3 - Conference contribution

SN - 978-3-030-13708-3

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 293

EP - 304

BT - Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers

PB - Springer

CY - Cham

ER -

van Knippenberg MS, Menkovski V, Consoli S. Evolutionary construction of convolutional neural networks. In Nicosia G, Giuffrida G, Nicosia G, Pardalos P, Sciacca V, Umeton R, editors, Machine Learning, Optimization, and Data Science - 4th International Conference, LOD 2018, Revised Selected Papers. Cham: Springer. 2019. p. 293-304. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Available from, DOI: 10.1007/978-3-030-13709-0_25