OFTNAI

The foundation is currently supporting a computer modelling centre within the Oxford University Department of Experimental Psychology

Members of the university research centre are using 3D virtual reality software to embed brain models within simulated virtual environments. The use of 3D virtual reality allows careful control of the positions and velocities of visual stimuli, as well as the point of view of the brain model within the environment. We have found that using this kind of realistic sensory input is critical to how, for example, models of the visual system develop their synaptic connections. We are involved in modelling various areas of brain function, including vision, spatial representation, motor behaviour and navigation. However, these systems interact and hence so should the models. The use of 3D virtual reality will allow us to explore how models of different brain areas can work together given realistic sensory input. This approach will permit the integration of various sources of experimental data into a unified theoretical framework, in which complete brain models are embedded in simulated 3D environments. We believe that this approach will eventually become invaluable for guiding further empirical research in brain science.

Case study: Learning invariant visual object recognition with Continuous Transformation Learning

Simulations using 3D virtual reality software have helped us to investigate how the continuous transformation of visual stimuli in the world may help the visual areas of the brain to learn to recognise objects and faces from different viewpoints. In simulation studies, we have exposed a hierarchical neural network model of the visual system to visual input from rotating objects generated using 3D virtual reality software. Continuous Transformation Learning utilizes the spatial continuity inherent in how objects transform in the real world, combined with associative learning of the feedforward connection weights, to enable the network to develop view invariant representations of visual objects (Stringer, S.M., Perry, G., Rolls, E.T. and Proske, J.H. (2006). Biological Cybernetics 94: 128-142).

Views of the two objects used in the simulations

Views of the two objects used in the simulations, (left) a cube, and (right) a tetrahedron. During training, the network was exposed to views of rotations around the vertical axis with variable step sizes between the views (18 degrees in this figure).

Results from a computer simulation after training.

Results from a computer simulation after training. (a) The firing rate response profile of a fourth layer output neuron to the cube and tetrahedron stimuli as they are rotated through 180 degrees.This cell has learned to respond to the cube in all viewing angles, but does not respond to the tetrahedron from any view. (b) The firing rate response profiles of two fourth layer cells that have learned to respond to the tetrahedron. These cells respond to different complementary parts of the view-space of the tetrahedron. (c) The view of the tetrahedron at 65 degrees, at which a new face comes into view triggering a switch in the firing of the two tetrahedron cells.

3D Publications Date Authors Journal Volume/Pages Download
Spatial scene representations formed by self-organizing learning in a hippocampal extension of the ventral visual system2008Rolls, E.T., Tromans, J.M. and Stringer, S.M.Neural Networks28: 2116-2127 Download
Learning transform invariant object recognition in the visual system with multiple stimuli present during training2008Stringer, S.M. and RollsNeural Networks21: 888-903 Download
Invariant object recognition with trace learning and multiple stimuli present during training2007Stringer, S.M., Rolls, E.T. and Tromans, J.M. Network: Computation in Neural Systems18:2 161-187 Download
Invariant global motion recognition in the dorsal visual system: a unifying theory2006Rolls, E.T. and Stringer,Neural Computation19: 139- 169 Download
Spatial vs temporal continuity in view invariant visual object recognition learning.2006Perry, G., Rolls, E.T. and Stringer, S.M.Vision Research46: 3994-4006 Download
Learning invariant object recognition in the visual sy stem with continuous transformations2006Stringer, S.M., Perry, G., Rolls, E.T. and Proske, J.H.Biological Cybernetics94: 128-142 Download