A team of researchers from the University of Maryland has demonstrated that popular household robotic vacuum cleaners can be remotely hacked and used as microphones. Scientists presented their work at the SenSys 2020 conference in Japan.
The researchers collected information from a laser navigation system in a popular vacuum robot vacuum cleaner and applied signal processing and deep learning techniques to recover speech and detect TV sounds in the same room as the device.
This work demonstrates the potential of any device using distance sensing technology (lidar) that can be used to collect sound despite the lack of a microphone.
“We welcome these devices to our homes and have no idea how they can be used,” explains Nirupam Roy, assistant professor of computer science at the University of Maryland and one of the authors of the study. “But we have shown that even though robotic vacuums do not have microphones, it is possible to repurpose the systems they use for navigation to spy on users and potentially reveal personal information.”
Lidar navigation systems in home vacuum robotic vacuums illuminate a room with a laser beam and read laser reflections from nearby objects. The robot uses reflected signals to avoid collisions when moving around the house.
Privacy experts have suggested that cards created by vacuum robots, often stored in the cloud, potentially violate users’ privacy. Advertisers can easily access information about the size of the house, which implies the level of income, and other information related to lifestyle. Roy and his team wondered if these robots ‘lidar could also pose a potential security threat as sound recording devices in users’ homes or workplaces.
Sound waves cause objects to vibrate, and these vibrations cause small changes in the light reflected from the object. Laser microphones, used in espionage since the 1940s, are capable of converting these variations back into sound waves. But laser microphones rely on a directional laser beam that bounces off very smooth surfaces such as glass windows.
On the other hand, a vacuum lidar scans the environment with a laser and captures light backscattered by objects of irregular shape and density. The scattered signal received by the robot’s sensor provides only a fraction of the information needed to reconstruct the sound waves. The researchers were not sure if the lidar system of the robot vacuum cleaner could be used to act as a microphone, and if the signal could be interpreted into meaningful audio signals in principle.
Yet the researchers hacked into the robot vacuum cleaner to show that they were controlling the position of the laser beam and sending the received data to their laptops via Wi-Fi without interfering with the device’s navigation.
They then experimented with two sound sources. One of the sources was a human voice, and the other was sound from various shows played through the TV soundbar. Roy and his colleagues then captured a laser signal picked up by the robot’s navigation system as it bounced off various objects near the sound source. Items included a trash can, a cardboard box, a food container, and a trash bag – items that can usually be found on the floor.
The researchers transmitted the received signals through deep learning algorithms. Their computer system, which they call the LidarPhone, identified speech in conversations and TV shows from minute recordings with over 90% accuracy.