Vision
Computer modellers within the university research centre are exploring various aspects of visual processing in the brain, including motion detection, face recognition in natural scenes, and the recognition of objects from novel views. Over successive stages, the primate visual system develops neurons that respond with view, size and position invariance to objects or faces. Our models explain how such neurons may develop their firing properties, and hence allow the visual system to recognise objects in natural environments.
This research has direct bearing on understanding disorders of visual perception such as amblyopia, in which one eye suffers reduced vision due to interference during early visual development. Amblyopia is the leading cause of vision loss in persons under 40 years of age. Other disabilities include prosopagnosia where subjects have difficulty recognising faces, or spatial neglect where patients ignore part of their field of vision.
Recently, our computer simulations have revealed a powerful new algorithm, Continuous Transformation Learning, that may account for how the brain learns to recognise objects and faces from different viewpoints. This discovery represents a major breakthrough in understanding the operation of the visual system, and should help to guide the treatment of visual disorders arising from developmental problems.
In addition to potential medical benefits, possible engineering applications of this research range from visual control and quality inspection in manufacturing to automated CCTV monitoring.The new Continuous Transformation Learning algorithm may help robots to operate more flexibly in real-world environments by enabling them to recognise objects from different viewpoints.
Case study Recognising objects from novel views
One of the major challenges in computer vision is how to recognise objects from different viewpoints which have not been encountered during training. Our neural network model of the ventral visual system is able to accomplish this by first learning how elemental features in the environment transform across different viewpoints during early visual development (Stringer, S.M. and Rolls, E.T. (2002). Neural Computation, 14: 2585-2596).
(Left) Architecture of a 4-layer hierarchical neural network model of the ventral visual processing stream. Convergence through the network is designed to provide fourth-layer neurons with information across the entire input retina. (Right) Convergence through successive layers of the visual system.
Six visual stimuli with three surface features that occur in three relative positions. Each row shows one of the stimuli rotated through the five different rotational views, in which the stimulus is presented to the network. From left to right, the rotational views shown are -60 degrees, -30 degrees, 0 degree (central position), 30 degrees, and 60 degrees. To simulate early visual development, layers 1 and 2 are trained on pairs of surface features across all five views.Then layers 3 and 4 are trained on the complete stimuli at only four out of the five views.
Results from the neural network simulation after training. The figure shows the response profiles of a top layer neuron to the 6 stimuli across all 5 views. It can be seen that this cell has learned to respond invariantly to one of the stimuli across all views.The network has learned to discriminate between the 6 objects from all views, including the novel view not encountered during training.
Selected publications
Stringer, S. M. and Rolls, E.T. (2000). Position invariant recognition in the visual system with cluttered environments, Neural Networks, 13: 305-315
Rolls, E.T. and Stringer, S. M. (2001). Invariant object recognition in the visual system with error correction and temporal difference learning, Network: Computation in Neural Systems, 12: 111-129.
Stringer, S.M. and Rolls, E.T. (2002). Invariant object recognition in the visual system with novel views of 3D objects, Neural Computation, 14: 2585-2596.
Elliffe, M. C. M., Rolls, E.T. and Stringer, S. M. (2002). Invariant recognition of feature combinations in the visual system, Biological Cybernetics, 86: 59-71.
Trappenberg, T.P., Rolls, E.T. and Stringer, S.M. (2002). Effective size of receptive fields of inferior temporal cortex neurons in natural scenes, Advances in Neural Information Processing Systems, 14: 293-300.
Rolls, E.T. and Stringer, S. M. (2006). Invariant visual object recognition: A model, with lighting invariance, Journal of Physiology - Paris, 100: 43-62.