Diffblue Cover is gaining traction amidst a trend toward AI-augmented testing, say industry analysts. Cover can now run a suite of unit tests without developer intervention.
Diffblue Cover, an AI-powered unit test writing platform, gained additional capabilities this week that give developers a hands-off option to test Java code.
Cover, which launched in 2020, writes and optimizes unit tests in Java and Spring codebases, which are used for writing enterprise Java applications. The new capabilities allow developers to autonomously write application test suites, analyzing a codebase and creating tests without developer intervention. In addition, Cover can identify problem areas such as untestable code and automatically refactor Java code to improve testability.
AI-augmented testing is the next step in the evolution of software testing and will become the most impactful driver of change in the testing market, said Joachim Herschmann, senior director and analyst at Gartner.
"When compared to traditional software testing tools, AI-augmented software testing tools provide software engineering teams with a clear advantage in accelerating testing cycle times and improving software quality," he said. "It is also a crucial element for a strategy to reduce significant business continuity risks when critical applications and services are severely compromised or stop working altogether."
Diffblue steps into market gap
Up to this point, enterprise developers have often relied on time-consuming manual testing methods or older testing products that are no longer state of the art, Herschmann said.
"Many organizations are still using testing solutions that are more than 10 years old, since they haven't updated in a long time," he said. "Or [they] are using tools that came of age in the days of waterfall and are not suitable for the modern world of Agile and DevOps."
Automated unit testing tools for Java such as JUnit and TestNG can speed up unit testing with better efficiency and better test coverage. AI-augmented testing goes even further and performs faster, more autonomous work with benefits such as lower costs and improved quality, according to Garner research. But despite the abundance of automated unit testing tools, competitors in the AI-augmented testing space are thin on the ground. In addition to Diffblue Cover, tools include open source EvoSuite, which is under development on GitHub, and Ponicode, which was acquired by CircleCI in March and performs unit tests in JavaScript and Python but not Java.
Lack of test coverage is a real problem in application modernization, so it's a bit of a sweet spot.
James GovernorCo-founder and analyst, RedMonk
"Lack of test coverage is a real problem in application modernization, so it's a bit of a sweet spot," said James Governor, co-founder and analyst at RedMonk.
Application modernization includes tasks such as rewriting or repackaging legacy software to run on modern platforms. Experts have warned that the growing number of data breaches and the developer shortage in recent years have added urgency to modernization efforts.
As such, Diffblue Cover is gaining traction -- especially in financial services, Governor said. Diffblue's customer base reflects this traction; although the platform has fewer than a dozen paying customers, it includes enterprises such as AWS, Citi, Goldman Sachs and JPMorgan.
Diffblue Cover upgrades to autonomous testing
While Diffblue Cover originally helped developers run unit tests, the platform's upgrade means that developers can now run their unit tests autonomously, said Mathew Lodge, CEO at Diffblue. Cover analyzes code in the background and can write a complete set of unit tests overnight without developer intervention, he said. For example, at Goldman Sachs, Cover wrote 3,000 tests in about eight hours, a task the financial services firm estimated would take its developers 268 workdays, according to one case study.
Diffblue Cover's new features include Reports for insights into test coverage, and Refactor, which suggests and applies code changes. A third feature, Replay, can create new unit tests by using existing tests and live production behavior, which is useful for dynamic applications that need testing at runtime, Lodge said. The final feature in the pack, Optimize, cuts the amount of time it takes to run unit tests in half compared with the previous edition of Cover, he said.
Cover's AI uses a probabilistic search to determine the best tests to run out of a potential infinite number of possibilities.
This means spending less time running less-than-optimal tests, RedMonk's Governor said.
"One nice touch with the platform enhancements is identifying which tests you don't need to run," he said. "As such, it can be used to cut your continuous integration overheads and costs. In this economy, anything that improves test coverage -- improving code quality -- while also cutting your CI bill, is probably going to be seen as something worth trying."
The new features are available for only Diffblue Enterprise users; prices start at $2,100 per developer, per year. The Community Edition, a stripped-down free version for individual developers, does not include the new capabilities.
To date, Diffblue has raised more than $40 million in venture funding including an $8 million round led by AlbionVC, announced this week. Diffblue Cover runs locally on Linux, Windows and MacOS environments and is available as a command-line interface tool and as an IntelliJ IDEA plugin.
