Xandr's bumpy ride to AI software test automation success

Low-code Mabl a software test automation standout

Although Xandr presents one UI to clients, a dozen UI microservices owned by different teams process millions of requests per day behind the scenes. Silverstein's initial attempt at UI testing used an in-house testing framework built with WebDriver-based Node.js Selenium tools.

You can add assertions every step along the way, and it makes writing the test significantly easier than any other product we explored. It's certainly easier than the things that I had built.

"It was sufficient for highly motivated engineers, but the barrier to entry was too high," he said. "You needed to learn too much about the testing framework, the way to use the framework, and the capabilities and shortcomings of the testing framework in order to get started writing tests."

Rather than build another testing framework from scratch, Silverstein began to look for a software test automation tool that might fit the bill. His team considered AI-assisted tools Functionize and Appvance, but decided on Mabl because of its low-code approach.

"You pull up your product, click 'record' in the trainer and then just use the product, and it records everything that you did," he said. "You can add assertions every step along the way, and it makes writing the test significantly easier than any other product we explored. It's certainly easier than the things that I had built."

Early struggles with Mabl

Initially, Mabl's secure network tunneling mechanism was incompatible with the naming conventions one Xandr team used for Kubernetes deployments. When Mabl triggered the performance testing apparatus for continuous integration, the identifier it provided would correspond to the most recently created ephemeral Kubernetes resource, which would override the intended environment target.

Jesse Silverstein

After adding a test to the plan, the team sometimes had trouble getting the trainer to target a different ephemeral environment as well, Silverstein said.

Mabl's engineering team addressed these problems in less than a week, according to Silverstein. "They built a whole new interface for how you launch the trainer," he said. "It's now working perfectly."

Xandr's team had been launching Mabl environments into Azure Kubernetes, creating a separate Kubernetes cluster for each of those instances. Mabl provided a Docker container image that opened a secure tunnel from wherever the container was deployed to Mabl's cloud for testing, but that connection was unreliable.

"If we tried to [give it] the same name as an ephemeral environment, it wouldn't connect," Silverstein said.

While earlier versions of the software test automation tool did not support the reuse of environment names for Mabl links, that limitation has since been removed and all customers can now reuse environment names, a Mabl spokesperson said.

Mabl's quick response to Xandr's implementation issues impressed Silverstein.

"They partnered with us closely and figured out all the things that were hard to do -- and in some cases, impossible -- and immediately got to work," he said.