The secret to DevOps test automation? Pick your battles
The pain of software test automation is real, particularly in a DevOps environment. But what if it was possible to approach it all differently? App maker Airtasker has a plan.
Want to make DevOps test automation easier? Let go of the idea that software testing can be fully automated.
That somewhat revolutionary thought comes from Kanak Kalburgi, senior automation engineer at Sydney-based Airtasker Pty Ltd., an app that connects temp workers with employers who need a job done. Airtasker, which recently expanded service to London, has 2 million users on its platform currently.
The real-time nature of the app meant the design team had an enormous manual workload when it came to testing, Kalburgi explained. To address this, Airtasker turned to automated testing framework BrowserStack. But, while very helpful, automation didn't mean the hard work was done, Kalburgi said. In fact, it required top-down changes through the entire company to make DevOps test automation and continuous integration and delivery really work.
DevOps culture means ownership
The company started with a lot of the engineering team focused on DevOps, but it's taken time for the culture to take hold. Airtasker is in the midst of a significant transition now, Kalburgi said, with a goal that everyone in the company will take ownership of quality and metrics.
"I truly believe that a [DevOps] leadership philosophy has to be provided, and the culture has to be forced," he said. "We are really starting to see the changes in our company now."
It's fundamentally a fallacy that all test can be automated.
Kanak Kalburgisenior automation engineer, Airtasker
One of the most important steps toward DevOps test automation was to ensure communication among the groups was clear and frequent. "Developers and our business side come together so we can formulate expected user behavior and make amendments and changes to our core scenarios as we go," Kalburgi said. "That's one of the main tenants in our practice. And we have a set of routines that are maintainable."
DevOps testing takes selective automation
With the communication in place, the company could focus on DevOps test automation by asking a simple question: What's the difference between a test and an automated check?
"With a large niche app such as ours, there has got to be a lot of areas where real tests need to happen," Kalburgi said. "Every time we talk about a typical testing strategy, you are always going to have things that can be automated and things that are not optimal for automation at all. You need to realize that and carve out those areas that are simpler to have critical checks for automation so you are not spending too much time in doing the wrong things."
Narrow your testing strategy
Airtasker makes this work by treating each phase of the application as a state machine.
"This means we verify whether or not a single feature is working well for us," he said. "We do not really have to fundamentally test all of the preceding parts of the app. If you treat it as a state machine with very fine transitive features you can pin down and narrow down your testing strategy."
The bottom line for Kalburgi is that until true, intelligent AI-powered testing exists DevOps test automation will remain a struggle for companies unless the focus and attitudes change.
"It's fundamentally a fallacy that all test can be automated," he added. "But you do need to make sure you're doing the right thing at the right time."
Dig Deeper on Agile, DevOps and software development methodologies
