Brilliant cameras will soon become a reality thanks to a research project from Bristol and Manchester’s Universities. Scientists have used convolutional neural networks and developed cameras that identify what is happening without recording an image.
Nowadays, modern systems perceive and process the world around them, not well enough. They still combine sensors to record images (cameras) with computing devices (GPUs). As a result, AI systems perceive the world only after recording and transferring visual information between sensors and processors. But many of the things that cameras can see are often irrelevant to the task at hand. For example, detailing the leaves on the trees near the road along which the car with the autopilot is passing. All this unnecessary information is captured by sensors with the smallest details and clogs the system with irrelevant data, consuming energy and processing time. A different approach is needed to provide an effective vision for intelligent machines.
Two studies from Bristol and Manchester have shown how perception and learning can create new cameras for AI systems. They presented a convolutional neural network (CNN) in the SCAMP-5D computer vision system. She can classify what is happening in front of her at a speed of 8,200 frames per second.
Recall that a convolutional neural network is a kind of AI algorithm. The CNNs developed by the team can recognize images without the need to record them or send them further to processing systems.
This approach will make the systems much more efficient and safer as there is no need to record images to analyze them.
Thanks to the SCAMP architecture designed by Peter Dudek, a professor at the University of Manchester, and his team, the work was made possible. SCAMP is a camera processor chip, which scientists describe as a Pixel Processor Array (PPA). The PPA has a processor built into each pixel that can communicate to process data in parallel. This is perfect for CNN and vision algorithms.
The development of scientists is presented at the European Conference on Computer Vision (ECCV).