Researchers at Google and Oxford University have unveiled AI that can edit the movements of individual objects. Now work with video can be reduced by several times.
The new neural network can distinguish between individuals and objects in a video, as well as speed up or slow down their movements individually while ensuring smooth interaction with each other. Now video editors can change the action of the movie using just one tool.
To do this, a team from Google and Oxford University divided each video frame into separate layers and taught AI to identify people or objects in them. This neural network focuses on things on each layer, concentrating on their movements, then it can separate each object and change their movement.
Previously, Google came up with a model that can read sign language during video calls. AI can identify “actively speaking”, but ignores the interlocutor if he just moves his hands or head. A new development by Google researchers is capable of doing this with great efficiency and low latency. While the researchers note that the detection of sign language leads to a delay or degraded video quality, this problem can be solved, and the model itself remains light and reliable.
First, the system runs the video through a model called PoseNet, which estimates the position of the body and limbs in each frame. Simplified visual information is sent to a model trained to position data from videos of people using sign language and compares the image to how people usually display certain words.