Publication: Elements for a general memory structure: properties of recurrent neural networks used to form situation models
Loading...
Full text at PDC
Publication Date
2008-05
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Springer Verlag
Citations
Exportar
Abstract
We study how individual memory items are stored assuming that situations given in the environment can be represented in the form of synaptic-like couplings in recurrent neural networks. Previous numerical investigations have shown that specific architectures based on suppression or max units can successfully learn static or dynamic stimuli (situations). Here we provide a theoretical basis concerning the learning process convergence and the network response to a novel stimulus. We show that, besides learning "simple" static situations, a nD network can learn and replicate a sequence of up to n different vectors or frames. We find limits on the learning rate and show coupling matrices developing during training in different cases including expansion of the network into the case of nonlinear interunit coupling. Furthermore, we show that a specific coupling matrix provides low-pass-filter properties to the units, thus connecting networks constructed by static summation units with continuous-time networks. We also show under which conditions such networks can be used to perform arithmetic calculations by means of pattern completion.
Description
UCM subjects
Unesco subjects
Keywords
Citation
Beer RD (2006) Parameter space structure of continuous-time recurrent neural networks. Neural Comput 18: 3009–3051
Cruse H, Hübner D (2008) Selforganizing memory: active learning of landmarks used for navigation. Biol Cybern (submitted)
Cruse H, Sievers K (2008) A general network structure for learning Pavlovian paradigms (in preparation)
Elman JL (1990) Finding structure in time. Cogn Sci 14: 179–211
Feynman R (2001) In: Hawking SW (ed) The universe in a nutshell. Bantam Press, New York
Fuster JM (1995) Memory in the cerebral cortex: an empirical approach to neural networks in the human and nonhuman primate. MIT Press, Cambridge
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci 79: 2554–2558
Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two state neurons. Proc Natl Acad Sci 81: 3088–3092
Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 2:78–80
Kühn S, Beyn WJ, Cruse H (2007) Modelling memory functions with recurrent neural networks consisting of input compensation units: I. Static situations. Biol Cybern 96: 455–470
Kühn S, Cruse H (2007) Modelling memory functions with recurrent neural networks consisting of input compensation units: II. Dynamic situations. Biol Cybern 96: 471–486
Kindermann T, Cruse H (2002) MMC— a new numerical approach to the kinematics of complex manipulators. Mech Mach Theory 37: 375–94
Palm G, Sommer FT (1996) Associative data storage and retrieval in neural networks. In: Domany E, van Hemmen JL, Schulten K(eds) Models of neural networks III. Association, generalization, and representation. Springer, New York, pp 79–18
Pasemann F (2002) Complex dynamics and the structure of small neural networks. Netw: Comput Neural Syst 13: 195–16
Steinkühler U, Cruse H (1998) A holistic model for an internal representation to control the movement of a manipulator with redundant degrees of freedom. Biol Cybern 79: 457–66
Strang G (2003) Introduction to linear algebra. Wellesley - Cambridge Press, Cambridge
Tani J (2003) Learning to generate articulated behavior through the bottom-up and the top-down interaction processes. Neural Netw 16: 11–3
Wessnitzer J, Webb B (2006) Multimodal sensory integration in insects—towards insect brain control architectures. Bioinspir Biomim 1: 63–5