Find out how a new programming abstraction for sensors known as synthetic sensors could simplify IoT app development, especially for industrial IoT situations.
A new programming abstraction called synthetic sensors promises to dramatically reduce the complexity of IoT app development. The concept creates a better balance between the raw data gathered by sensors and the kinds of real-world events that developers and users care about. While this kind of approach can be useful with a single sensor, its power and utility grow when it reflects data fused from multiple sensor types.
Researchers at Carnegie Mellon University have been pushing the limits of this research with the development of a set of hardware and software tools for synthetic sensors. This platform translates data from a single low-cost IoT device into dozens of different events, like faucet running, appliance usage and paper towel usage.
Higher-order synthetic sensors can be programmed on top of these virtual sensors for monitoring more sophisticated events. For example, one could create a synthetic sensor to measure how many paper towels have been dispensed based on the sound of the dispenser, even when there is no physical sensor on the paper towel dispenser. Programmers could then use this synthetic sensor to drive an app for ordering more paper towels or to alert the janitor.
In the short run, there is probably more opportunity in using synthetic sensor principles for industrial IoT app development.
This represents a departure from single-use sensors that measure things like occupancy or whether a door is open. This first implementation of this project used sensors for measuring vibration, audio, ambient temperature, noncontact temperature, humidity, air pressure, illumination, color, motion, magnetic and electromagnetic interferences, and received radio signal strength. In many cases, a single primary sensor could provide enough data for measuring an event, but other sensors could help resolve cases when there was signal noise.
It certainly is not the first effort to drive IoT applications from multiple IoT devices. Platforms such as IFTTT already allow developers and consumers to weave together their own IoT measurement and activation flows. But in the long run, synthetic sensors could provide a way for developers to think about creating apps using object-oriented, event-driven programming models rather than declarative models driven by sensor thresholds.
Carnegie Mellon University researchers' synthetic sensor.
Practical constraints
The Carnegie Mellon University researchers created many constraints they believe make this more practical for consumer use cases. First, the platform had to be able to do its readings while plugged into a wall. This eliminated the need to change batteries, which could be an issue if hundreds were deployed in each room in a large facility.
Second, they wanted to be able to eliminate the need for cameras, which introduce privacy and security concerns. The platform also needed to be able to correlate events at the sensor platform level to reduce bandwidth requirements and to reduce privacy concerns. They found that it was possible to accurately correlate about 85 events in the real world with surprising accuracy, such as oven usage, paper towel usage, water consumption and the frequency of fridge use, just from a single sensor platform in the room.
A key element of this was the ability to measure multiple signals to help distinguish between different kinds of events. For example, if background music is playing, it may otherwise overwhelm the ability to discern the faucet running. However, by measuring the vibration of the countertop and listening for the sounds of running water, it can more accurately determine if a faucet is indeed running.
Likewise, the sound of a blender and a mixer might be similar, but these tend to generate different electromagnetic fields, which make it easier to distinguish between these and other appliances. Overall, the team created 38 synthetic sensors that achieved 98% accuracy. Many of these achieved nearly 100% accuracy, with three sensors performing in the 60% range.
How synthetic sensors work
A synthetic sensor platform might detect that a specific sound, vibration and movement pattern is associated with someone getting a paper towel. The combination of these signals is correlated at the sensor platform level into a synthetic sensor for detecting paper towel usage. This allows developers to create applications that use these synthetic sensors to drive other application behavior.
A synthetic sensor reports that a paper towel has been dispensed.
The simplest synthetic sensor only correlates raw physical data on the sensor and is called a first-order synthetic sensor. These can be woven together into higher-level abstraction for more sophisticated second-order, third-order and so on synthetic sensors. For example, a series of events associated with cooking a particular meal might correlate with a pasta dinner compared with a stir fry. These, in turn, could be correlated with a higher-order abstraction like total kitchen usage.
Using a synthetic sensor board, a single device can monitor a room full of devices and events.
If privacy concerns could be addressed, it is conceivable that this data could be correlated with other information about the house, such as food purchase history, to improve the correlation of cooking patterns with different kinds of meals.
Many vendors are starting to commercialize consumer sensor platforms with less sophistication. For example, wireless home monitoring company Notion has developed a small system with sensors for light, motion, sound, angular rate, water, light, natural frequency, acceleration and temperature. These battery-operated devices can be affixed to doors, walls or windows to detect events like a smoke alarm going off, a door opening or water leaking.
Building a better industrial internet
The types of consumer applications demonstrated by the Carnegie Mellon University researchers look like fun. But it is not clear how much the mass market of consumers is willing to pay to track events in their home. In the long run, this kind of technology is likely to get subsumed into better iterations of platforms like Amazon Echo or Google Home. An app store for synthetic sensors would make it easier for consumers to customize different kinds of experiences, the discovery of insight or automate shopping for more paper towels.
In the short run, there is probably more opportunity in using synthetic sensor principles for industrial IoT app development, such as those for the $80 billion heating, ventilation and air conditioning (HVAC) industry. For example, Augury Inc. has developed a specialized sensor platform that can be placed on expensive pumps, compressors, chiller, motors and HVAC equipment, called Halo. The current pod uses sensors for vibration, ultrasonic sounds, magnetic fields and temperature. It takes incremental readings every 15 minutes, and a separate low-power vibration sensor can wake up the pod when an anomaly is detected.
Halo devices allow technicians and facilities managers to ascertain the status of expensive HVAC machinery to take care of repairs and replacements before they become more costly. The devices use machine learning to correlate various sounds and vibrations with the health of machines. Going forward, Augury plans to develop a low-cost device that could be built into washing machines, refrigerators and other household appliances.
Saar Yoskovitz, CEO at Augury, said the three main values for this platform are reduced repair costs, reduced operational costs and increased uptime. Repair technicians can replace parts when required rather than on a fixed schedule, which can reduce maintenance costs by 30%. They can also optimize the settings of this equipment which can increase efficiency by 20%. These savings are significant because each machine can cost $15,000 to $150,000.
IoT developers can simplify the use of their applications by translating raw data into the kinds of events that users care about and can track in their monitoring tool of choice. Yoskovitz explained, "One of the biggest pitfalls of industrial IoT is that vendors connect everything to the cloud and to one proprietary dashboard. Enterprise users are overwhelmed by the amount of data coming to their screens and don't know what to do about it. A better approach is to focus on the algorithms for converting raw data into insight."
