Researchers at Tokyo Institute of Technology and Carnegie Mellon University have jointly developed a new human motion capture system that consists of a single ultra-wide fisheye camera mounted on the user’s chest.
Conventional motion capture systems in dedicated studios typically rely on multiple synchronized cameras attached to ceilings and walls to capture the movements of a person wearing a protective suit equipped with multiple sensors. Such systems are often very expensive and limited in terms of the space and environment in which the owner can move.
A team of researchers led by Tokyo Tech’s Hideki Koike unveils a new motion capture system that consists of a single ultra-wide fisheye camera mounted on a user’s chest. Their design not only overcomes space constraints in existing systems but is also cost-effective.
The system, dubbed MonoEye, can capture a user’s body movements as well as their perspective or “viewport”.
Our ultra-wide fisheye lens has a 280-degree field of view and can capture the user’s limbs, face, and environment.
To provide robust multimodal motion capture, the system has been designed with three deep neural networks capable of evaluating 3D body posture, head posture, and camera posture in real-time.
Researchers have already trained these neural networks using a vast dataset of 680,000 images of people with varying body shapes, clothing, actions, backgrounds, and lighting conditions, as well as 16,000 frames of photorealistic images.
The researchers speculate that in the future, the chest camera could be turned into an everyday accessory such as a tie clip, brooch, or sports gear.