Sep 9 2015
In today's smart home, technologies can track how much energy a particular appliance like a refrigerator or television or hair dryer is gobbling up. What they don't typically show is which person in the house actually flicked the switch.
A new wearable technology developed at the University of Washington called MagnifiSense can sense what devices and vehicles the user interacts with throughout the day, which can help track that individual's carbon footprint, enable smart home applications or even assist with elder care.
In a study to be presented this week at the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, MagnifiSense correctly classified 94 percent of users' interactions with 12 common devices after a quick one-time calibration, including microwaves, blenders, remote controls, electric toothbrushes, laptops, light dimmers, and even cars and buses. Even without the calibration, MagnifiSense was still correct 83 percent of the time.
The sensor worn on the wrist uses unique electromagnetic radiation signatures generated by electrical components or motors in those devices to pinpoint when its wearer flicks a light switch, turns on a stove or even boards a train.
"It's another way to log what you're interacting with so at the end of the day or month you can see how much energy you used," said Shwetak Patel, Washington Research Foundation Endowed Professor of Computer Science & Engineering and Electrical Engineering, who directs the UW Ubicomp Lab.
"Right now, we can know that lights are 20 percent of your energy use. With this, we divvy it up and say who consumed that energy," said Patel.
In a 24-hour test in which a single user did everything from read on a laptop to cook dinner and take a bus ride, the system correctly identified 25 out of 29 interactions with various devices and vehicles.
MagnifiSense also has potential for other smart home applications, such as recognizing a user's preference for interacting with an appliance or device. By sensing whether an adult or child is turning on a television or tablet, for instance, a system could automatically display their favorite programs or tailor the device with appropriate selections.
In assisted living settings or nursing homes, the wearable sensor could help keep track of how efficiently elderly people are going about everyday tasks such as cooking or grooming. It could also detect when a stove has been left on for a long period of time and help alert someone to that danger.
"The nice thing with MagnifiSense is that you don't have to instrument every single appliance in your house, which gets expensive and cumbersome," said lead author Edward Wang, a UW electrical engineering doctoral student. "It can also sense some of the blank spots that other technologies can't, like battery-powered devices."
The team combined three simple, off-the-shelf sensors that use inductors, or coils of wire wound around magnets. Those proved to be the most accurate without being so power-hungry that wearing them would be impractical.
The sensors also capture a broad frequency range that allows the system to differentiate between electromagnetic radiation emanating from the unique combinations of electronic components such as motors, rectifiers and modulators embedded in everyday devices.
"When a blender turns on, for instance, modulators change the current profile of the device and create something similar to a vocal cord pattern," Wang said. "A blender 'sings' quite differently than a hair dryer even though to our ears they sound similar."
The team also developed innovative signal processing and machine learning algorithms to help the system correctly match those patterns with a particular type of device.
One advantage to a wearable option is that anyone concerned about privacy issues can control when they use it, researchers said, or simply take it off.
Next steps include testing MagnifiSense on a wider variety of devices and distinguishing between multiple devices operating in close proximity. In preliminary tests, for instance, MagnifiSense had the most trouble correctly classifying a handful of particular toothbrushes, shavers and cars.
The researchers also plan to work on miniaturizing their proof-of-concept device into something that could be embedded into a watch or band. Based on its investigation, the team believes that with slight improvement to the update rate of magnetic sensors in current smartphones and smartwatches, MagnifiSense could soon be enabled on new devices with a simple software upgrade.
"We think it could be integrated into any wrist-sized product," said Patel. "The next steps are really to look at what other devices we can detect and work on a prototype that's wearable."
Co-authors include UW electrical engineering doctoral student Tien-Jui Lee, UW computer science and engineering doctoral students Alex Mariakakis and Mayank Goel, and Sidhant Gupta.