On flexible neural networks: Some system-theoretic properties and a new class

Yazdan Bavafa-Toosi, Hiromitsu Ohmori

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Although flexible neural networks (FNNs) have been used more successfully than classical neural networks (CNNs), nothing is rigorously known about their properties. In fact, they are not even well known to the systems and control community. In this paper, theoretical evidence is given for their superiority over CNNs. Following an overview of flexible bipolar sigmoid functions (FBSFs), several fundamental properties of feedforward and recurrent FNNs are established. For the feedforward case, it is proven that similar to CNNs, FNNs with as few as a single hidden layer (SHL) are universal approximators. It is also proven that unlike irreducible SHL CBSNNs, irreducible SHL FBSNNs are nonuniquely determined by their input-output (I-O) maps, up to a finite group of symmetries. Then, recurrent FNNs are introduced. It is observed that they can be interpreted as a generalization of the conventional state-space framework. For the recurrent case, it is substantiated that similar to CBSNNs, FBSNNs are universal approximators. Necessary and sufficient conditions for the controllability and observability of a generic class of them are established. For a subclass of this class, it is proven that unlike CBSNNs, FBSNNs are nonuniquely determined by their I-O maps, up to a finite group of symmetries, and that every system inside this subclass is minimal. Finally, a new class of FNNs, namely, flexible bipolar radial basis neural networks (FBRBNNs) is introduced. It is proven that as in the case of classical radial basis neural networks (CRBNNs), feedforward SHL FBRBNNs are universal approximators.

Original languageEnglish
Title of host publicationProceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05
Pages2554-2561
Number of pages8
DOIs
Publication statusPublished - 2005 Dec 1
Event44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05 - Seville, Spain
Duration: 2005 Dec 122005 Dec 15

Publication series

NameProceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05
Volume2005

Other

Other44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05
CountrySpain
CitySeville
Period05/12/1205/12/15

    Fingerprint

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Bavafa-Toosi, Y., & Ohmori, H. (2005). On flexible neural networks: Some system-theoretic properties and a new class. In Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05 (pp. 2554-2561). [1582547] (Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference, CDC-ECC '05; Vol. 2005). https://doi.org/10.1109/CDC.2005.1582547