SmartBear LoadNinja API automates performance testing for CI/CD
A REST API for SmartBear LoadNinja enables development teams to add load testing directly into their CI/CD workflows to automate performance testing for developers.
Software tools provider SmartBear has added a REST API to its LoadNinja load testing platform that enables developers and testers to integrate performance testing directly into their continuous development and deployment workflows.
While performance testing examines speed, responsiveness and resource usage, it does not look for bugs and anomalies that crop up in a system pressed to its limit. That's where load testing comes in -- to test a system under a heavy load of concurrent users to see how well it holds up. Performance testing is a superset of load testing, so the terms are often used interchangeably.
Traditional load testing requires developers to manually build load testing scripts that track the behavior of simulated clients, which can be tedious. The process is complicated, because most performance testing tools record not only transactions under the protocol layer, but also all the traffic between the server and the client, said Keshav Vasudevan, product manager for SmartBear's LoadNinja.
LoadNinja is a cloud-based platform to load-test web applications in browsers and gauge a system's performance under typical, real-world load conditions. An InstaPlay recording feature lets developers play back recorded scripts immediately, which can reduce development teams' load testing efforts by 50% to 70%, Vasudevan said. A reporting feature enables developers to get a baseline on performance and correct any performance anomalies before an application goes into production.
Now, with a SmartBear LoadNinja REST API, developers can access, manage and interact with LoadNinja through simple HTTP calls from any third-party software or component in the pipeline, such as Jenkins or other CI engines. Development teams can build and run load-test scenarios in their CI/CD workflows and analyze performance data to find and debug issues in real time.
SmartBear also has a Jenkins plug-in for LoadNinja, so developers can incorporate load testing using browsers at scale into their Jenkins pipeline.
SmartBear's LoadNinja Pro supports up to 250 virtual users and costs $2,399 a year, while LoadNinja Premium supports up to 2,500 virtual users and costs $7,399 a year. The enterprise version of the product, which supports more than 2,500 users, requires custom pricing.
Screenshot of SmartBear LoadNinja InstaPlay Recorder
Performance testing in CI/CD improves quality, reduces risks
Performance testing is one of the most overlooked nonfunctional requirements of software engineering, along with security, said Theresa Lanowitz, an analyst at Voke Inc. in Minden, Nev.
Waiting until the end to conduct performance testing can be disastrous, because the architectural flaws uncovered during the performance test are already part of the application.
Theresa LanowitzAnalyst, Voke
"By making performance testing easy and integrated to the workflow via an API, performance is relegated to a top-tier priority and cannot be overlooked," she said.
Developers frequently conduct performance testing at the end of the development cycle, whether it's a two-week sprint or a monthslong project. Incorporating performance testing in CI/CD as part of that workload decreases risk and increases quality.
"Waiting until the end to conduct performance testing can be disastrous, because the architectural flaws uncovered during the performance test are already part of the application," Lanowitz said.
Moreover, saving performance testing until the end is antithetical to the DevOps mantra to always keep software in a shippable state, which means you don't defer testing. Most DevOps organizations perform some type of performance testing with every build.
"That can be as basic as using timing functions in Selenium or using JMeter or other performance tools," said Thomas Murphy, an analyst at Gartner. "You are failing the build if performance degrades from the prior 'working build.'"
In addition to Selenium and JMeter, other tools in this category include LoadRunner, BlazeMeter and IBM Rational Performance Tester.
