This AI Can See You Through Walls

Intentions are good, but what if the device was capable of harm?

Leah Zitter
Read +
Follow Us

Researchers at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT) created an artificial intelligence software called RF-Pose that can see you through walls. Other systems using wireless technology also detect human movement through walls, but their images are grainy.

They either identify only one limb at a time, or give us a static and fuzzy image of the body. According to the experiment of Zao and team, any one of us can walk around a room and an AI camera located in another room will “peek” through the dividing wall and photograph our precise movements in 2D representation. Even if I turn off the lights and draw down the blinds, the AI device still captures my movements as though it were day.

The Tech Behind RF-Pose

The system uses radio waves which bounce off human bodies and pass through walls. The image appears as a fluorescent skeletal figure or as a stick figure made off fuzzy dotted blue and yellow specks. It’s possible I can hide my actual identity, but there’s no hiding my movement. Even through a 40mm thick wall (which is quite thick), even if I ducked into a cupboard or hid behind a chair, this AI’s sharp eyes can pierce through that dividing wall and see me as though the wall were glass.

“So we had to do this indirect training of the network,” Professor Dina Katabi of MIT CSAIL told BBC. "So, we used a camera and we took images of people in the scene as we ran the radio signals there. Ad we give those examples of the images from the machine to train it.”

This process is called a deep learning neural network, where the researchers fed the computer with thousands of radio wave profiles and images corresponding to hundreds and thousands of postures like sitting, walking, or opening a door. In time, the AI learned to associate any particular radio signal with a certain movement. Once trained, the AI algorithm worked only from radio waves, just as surveillance machines do when photographing people walking through security barriers. 

Describing the Study

The team of seven researchers, Zao, Li, Alsheikh, Tian, Zhao , Torralba, and Katabi, described their pilot tests in an open-access paper.

"We conducted person identification experiment with 100 people in two settings: visible environment, where the subject and RF device are in the same room, and through-wall environment, where the RF device captured the person’s reflections through a wall. In each setting, every person walked naturally and randomly inside the area covered by our RF device, and we collected eight and two minutes data separately for training and testing."

For each setting, the team trained a special computer trained to identify images and fed it 50 consecutive frames of skeleton heatmaps (each of these frames had numerous photographs of different postures). At the end, the researchers found that the AI accurately identified movement 83.4% of the time in visible scenes, where light was on. This occurs “even when a wall blocked the device and our pose extractor never saw these people and such environments during training.”

"We achieved more than 96% in both settings [both light and dark], demonstrating that the extracted skeleton can preserve most of the discriminative information for identification even though the pose extractor is never trained or fine-tuned on the identification task. Right now, the computer only captures rough movement and only in 2D."

The researchers say they're planning to create 3D images to tone in on finer moments like squatting, sitting, crawling, and even tremors in an elderly person.

The Louis Story

This reminds me of the story captured by a painting done by Napoleon's second wife, Marie Louise, that I saw just this week in the Virginia Museum of Fine Arts. Louis XIV was extremely sharp-sighted. While visiting one of his mistresses, he lunched by her: candied fruit, wine, some oysters, a rice salad.

Suddenly, he took up a hard-boiled egg, bent down and proffered it to a man hidden by screens under his mistress’ bed. “That person,” he said, “is probably hungry, too.” Those AI are programmed to see people hiding under even more opaque constructions such as in cupboards, under floorboards, and so forth. There’s little way to hide from the RF-Pose AI.

What’s the Purpose?

The researchers plan to use the device for altruistic motives, like helping rescuers find victims under rubble after an earthquake or helping caregivers “peek” in on people in advanced dementia or other states of serious illnesses (like Parkinson’s disease, multiple sclerosis or muscular dystrophy).

Katabi and her team say they want to help elderly people live independently by preventing falls. “A key advantage of our approach,” said Katabi, “is that patients do not have to wear sensors or remember to charge their devices.”In advanced stages such patients are apt to forget. The camera captures all.” Sounds good.

What about if this gets out of control, however, and we use it for spying on individuals, in any and all situations, without them knowing? Wouldn't an unscrupulous government just love such a device? And how can we prevent such a government or others from spying on us? So how do we put controls in place?