The on-line learning of Radial Basis Function neural networks (RBFNs) is analyzed. Our approach makes use of a master equation that describes the dynamics of the weight space probability density. An approximate solution of the master equation is obtained in the limit of a small learning rate. In this limit, the on line learning dynamics is analyzed and it is shown that, since fluctuations are small, dynamics can be well described in terms of evolution of the mean. This allows us to analyze the learning process of RBFNs in which the number of hidden nodes K is larger than the typically small number of input nodes N. The work represents a complementary analysis of on-line RBFNs, with respect to the previous works ( and ), in which RBFNs with N≫K have been analyzed. The generalization error equation and the equations of motion of the weights are derived for generic RBF architectures, and numerically integrated in specific cases. Analytical results are then confirmed by numerical simulations. Unlike the case of large N>K we find that the dynamics in the case N<K is not affected by the problems of symmetric phases and subsequent symmetry breaking.

On-line learning in RBF neural networks: a stochastic approach

MARINARO, Maria;SCARPETTA, Silvia
2000

Abstract

The on-line learning of Radial Basis Function neural networks (RBFNs) is analyzed. Our approach makes use of a master equation that describes the dynamics of the weight space probability density. An approximate solution of the master equation is obtained in the limit of a small learning rate. In this limit, the on line learning dynamics is analyzed and it is shown that, since fluctuations are small, dynamics can be well described in terms of evolution of the mean. This allows us to analyze the learning process of RBFNs in which the number of hidden nodes K is larger than the typically small number of input nodes N. The work represents a complementary analysis of on-line RBFNs, with respect to the previous works ( and ), in which RBFNs with N≫K have been analyzed. The generalization error equation and the equations of motion of the weights are derived for generic RBF architectures, and numerically integrated in specific cases. Analytical results are then confirmed by numerical simulations. Unlike the case of large N>K we find that the dynamics in the case N
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11386/3167277
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 24
  • ???jsp.display-item.citation.isi??? 15
social impact