How AI and machine learning enable testing to keep up with IoT

The internet of things has penetrated every aspect of our lives, and it’s transforming how we test and develop software. With the IoT market projected to be worth $1,490.31 billion by 2024, this explosive growth is set to continue. However, testing procedures and processes are not ready to test IoT and, if no change occurs, it’s likely to result in delays in deployment, updates and user acceptance.

So what does this frenzied growth mean for testing?

As IoT continues to snowball, it presents testing with a myriad of challenges thanks to the vast number of systems and variations involved. IoT applications make end-to-end software testing even more complicated, and many organizations have underestimated the issues with test strategies. Within IoT, companies need to understand all the elements of an application, how they are working together and how people are using them, from the vast array of mobile devices and complex back ends to microservices managing a wide range of sensors and devices at the edge, as well as data collection, analytics and decisions in the cloud. Suffice to say that this is the final nail in the coffin for manual testing; however, test automation also requires an overhaul to try and keep up.

As so much of an IoT application is composed of services outside the application’s control, the only way that test automation strategies can keep up is to embrace AI and machine learning. Automation of test execution is not enough as the entire testing process from creation through to analysis needs to be automated. To do this requires intelligent models to auto-generate tests, with AI and machine learning allowing teams to analyze data from testing and to identify the patterns with bugs.

AI-driven intelligent testing can build models by itself to continually test and monitor a system. Different AI approaches can look at the user interfaces, network traffic, system resources and so forth to build complete verification models. Expanding test automation beyond simple test execution and automating the creation of tests is the only way to achieve high levels of test coverage and confidence in complex IoT systems.

Once tests can be auto-generated, machine learning is then required to focus testing on the areas that really matter, “learning” the areas of an IoT system that are the riskiest and the areas that matter the most to users and then focusing testing in that area. Today, most teams only have a few hundred test scripts, so it’s easy to simply run all tests; but once we can auto-generate billions of tests, we need machine learning to help ensure we are executing the right ones to deliver a robust product.

Machine learning is also increasingly being used to understand which behaviors are bugs and which are features. Currently, development teams manually define what correct behavior is for each test case. However, once we have billions of test cases, highly complex systems, IoT systems interacting with each other and emergent behavior, it’s clearly not feasible for a human to define what correct is in each case. Especially since correct will be a range of values, not just one, and may in fact differ depending on the user and the context.

So, AI and machine learning are at the core of the future of IoT testing as auto-generating tests, deciding which tests to run and deciding which tests have failed.

If you think about different examples of IoT systems — from wearables to smart fridges to Amazon Go — they are complex applications that include a vast array of interconnected systems often built on entirely different technology stacks, so intelligent testing becomes essential. Intelligent testing, using AI and machine learning, is about demanding more actionable intelligence from testing: when automation, cognitive systems and advanced analytics become integral parts of a testing ecosystem, we should have high expectations, especially when proactively dealing with issues surrounding user interfaces and user experience.

For IoT to continue on its path of rapid growth, intelligent testing is no longer an aspiration, but an essential. By incorporating AI and machine learning, testing is not only more intelligent, but it will become a critical business accelerator for those IoT businesses that embrace it.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.