3 machine learning best practices to use in IoT projects
IoT machine learning takes a whole team of experts that can approach the project with the right mindset, effectively communicate, and facilitate user feedback and testing.
In an ideal world, organizations could easily plan machine learning for IoT with the right technical experts, and components would perfectly integrate with each other. Unfortunately, no matter how well a team researches IoT machine learning before it starts development, some of their assumptions will be proven wrong and they will face unexpected challenges.
During her IoT Tech Expo 2020 presentation, "Building Machine Learning Products -- a Best Practice Approach," Jenn Gamble, data science practice lead at Very, identified the required skills to implement machine learning with IoT and how teams can adopt best practices, approach software development and handle unexpected difficulties.
"A lot of the data science development lifecycle is actually very different than what the Agile software development lifecycle is. A lot of those hard-won best practices from software engineering, the data science community was not always aware of, or, at least, not fully embracing or benefiting from," Gamble said.
The differences between tech stacks and experts who must work together to create machine learning for IoT adds complexity. A strong understanding of the product, end users and goals will carry the team throughout development to make informed decisions about what to build and when.
Development teams must follow these three machine learning best practices and additional IoT-specific considerations.
1. Design the machine learning system to solve a problem
Three levels make up machine learning: product management, technical architecture and implementation detail.
At the product management level, the development team must decide which problem they must solve and which IoT system will solve the issue. This foundational level requires an extensive understanding of what is technically possible and meets the requirements and constraints, such as time, money, experts and skill sets.
The technical architecture is made up of different components and interface specifications between the components that must meet the requirements within constraints.
Implementation detail involves the development team understanding what each component in the machine learning system should do to make design decisions.
Machine learning projects share common components, such as firmware development, hardware engineering and data pipelines. A major component of machine learning is data flow from external sources. Developers must design an interface and determine when the back end passes the data to the machine learning pipeline. The pipeline can also write the data back into the front access location.
The best way to end up actually building the right thing in the end is, as early as possible, getting something in the hands of users. Then that feedback can be used to inform ongoing development.
Jenn GambleData science practice lead, Very
Alternatively, data can reside in the back end and only processed data goes into the machine learning pipeline. Developers must have a strong understanding of who uses the data and what decisions or actions are made from the processed data.
The challenges of meeting each level are the motivating impetus for many Agile software development best practices that can now serve machine learning development.
"Anyone who has ever built a new product can tell you that it just doesn't work [the way you initially assume it will] at all. You don't have all of the information that you need upfront," Gamble said.
Technical challenges will emerge that the development team couldn't predict during the initial plans and designs. Development teams must seek out information, such as how users actually use the product and data, through feedback mechanisms to improve their product throughout development.
3. Create user feedback and testing mechanisms
The only way to find out if a product fulfills its purpose is to use it. Teams can think of each component as its own product that provides something to the system, and then determine if the component works through full product assembly. Instead of developing each part in isolation, teams implement user feedback mechanisms to conclude that their product or component does the right thing.
"The best way to end up actually building the right thing in the end is, as early as possible, getting something in the hands of users. Then that feedback can be used to inform ongoing development," Gamble said.
Development teams must answer two questions throughout the development process with every component and level: "Am I doing the right thing?" and "Is it working properly?" These questions ensure that the product ultimately achieves its goals.
Regular testing also ensures a product's viability. Connecting components to test features for functionality means teams must make very simple choices for each component early on. Testing might include implementing a component that isn't finalized, but doing so still provides value and informs the team if the interfaces don't work.
3. Build a team of experts that can effectively work together
IoT machine learning requires a team with many different skills, including software developers, user experience experts, mobile app developers, product managers, IoT experts and a tech lead.
With IoT-based machine learning, teams need IoT experts in firmware and hardware. People with different development styles and backgrounds, often with different knowledge bases, must come together to build a machine learning product.
The right tech lead, sometimes called the AI product manager, can be one of the most important aspects of a successful project because that person must understand and be able to translate the requirements of each developer. This manager must get the team to effectively work together and regularly communicate any technical tradeoffs and requirements.
Every single person involved in a machine learning project should understand -- at a high level -- the product, the end users, the intended workflows the product enables and the technical architecture. It's not enough for experts to just understand their individual component and how it interfaces with other project components. They need an overall understanding of the project to make the right technical choices and feedback, Gamble said.
Regular discussions of technical design and product requirements can inform each other and lead to reduced costs, improved efficiency and better technical setup. Peer programming, a common practice in software engineering, can also increase the team's understanding of each component and the whole system.
IoT-based machine learning calls for additional considerations
With just machine learning, teams might not know if their system works or meets a certain level of accuracy until they use real data. With IoT, engineers can't change IoT devices once they are produced, unlike the way that they can edit software. Development teams must be confident that the IoT components of the system will match their requirements.
IoT machine learning requires developers to write code for multiple hardware targets. Developers should complete local digital testing at the component level for hardware, even if they know how to assemble the components, Gamble said.
End-of-the-line testing capabilities during the first production runs ensures a product's success. The developers and hardware engineers must understand the data flow to and from IoT devices, what information the devices gather, and the devices' interactions.
Developers often must simulate data to prototype their data pipeline and test IoT devices and machine learning components. To get the most accurate results, data simulations should take place as close to the data source as possible.
If the developers simulate the firmware, they can send data to the cloud and store it in the final product's storage hardware. This process provides a more accurate picture of the whole machine learning system before end-user data becomes available. A simulation that is as close to the final product as possible confirms the system will work as intended once it is in the field -- and in end users' hands.
