New AI model understands hand gestures

The new model of artificial intelligence (AI) defines gestures with an accuracy of 85%. To create it, scientists have studied how the human brain works.

Researchers from Nanyang and Sydney University of Technology have developed a machine learning system that can recognize hand gestures. To do this, she analyzes the images using stretch strain gauges. The architecture of artificial intelligence (AI) is described in the journal Nature Electronics, scientists were inspired by the device of the human brain.

They noticed that when people solve practical problems, they usually combine visual and somatosensory information obtained from the environment. These two types of information complement each other since in combination they give a better idea of ​​all the data that is needed to solve the problem.

“Our idea came about after we studied the properties of the brain to process information. It turned out that in the human brain, the activity of perception depends not only on specific sensory information but also on the integrated integration of several sensors. This inspired us to combine visual and somatosensory information (touch, temperature) to implement a system that accurately recognizes gestures”.

lead author of the study Xiaodong Chen

Developing his gesture recognition technique, Chen and his colleagues provided the opportunity to integrate various types of sensory information collected using several sensors. Supplementing them with computer vision, they created a system that can recognize gestures with an accuracy of 85%.

Its multilayer and hierarchical structure imitates the structure of the brain, but with artificial neural networks instead of the usual ones. In addition, some networks in their architecture process the same modal sensory data processed by neural networks in the brain. For example, a sectional neural network (CNN) specifically performs conversion operations, artificially reproducing the function of the local receptor field inside biological nervous systems. So it imitates the initial processing of visual information that occurs in those parts of the human brain that are responsible for vision.