Streamlining user acceptance testing (UAT) with Agile
Organizations that implement successful user acceptance testing can greatly reduce errors and business risks.
Pairing user acceptance testing (UAT) with process-streamlining Agile development practices seems like a no-brainer. UAT helps ensure that applications will be used effectively once deployed, and Agile promises to shorten the UAT process. However, moving UAT to Agile requires due diligence. Using Agile practices requires organizations to re-evaluate how and when UAT is executed. Building a UAT Center of Excellence (CoE) framework will help project, test and development managers improve UAT test coverage and efficiency with less Agile adoption pain.
What is user acceptance testing?
When business teams define their requirements, it is often assumed that developers and QA testers will develop a system that precisely meets all defined business requirements. However, real-time complexities and vast data combinations, limit the spelling out of all business requirements explicitly at such an early stage of a project life cycle. This limitation drives the need for an additional layer of testing, after the unit and system integration testing phase and before the movement of the application into the development phase. This is known as user acceptance testing (UAT), which is ideally done by business users -- end users who would actually use the systems. They check the functionality for delivery as per the defined business requirements.
The objective of user acceptance testing is to:
- Ensure compliance of the application with business requirements
- Check the business functionalities or logic from a risk perspective and
- Certify that the system meets all business requirements
UAT includes testing requirements for functional, operational, performance and interface areas. The fact that UAT takes place after the development of an application in the project lifecycle makes it even more critical, as it’s the last chance to screen and solve issues which were not detected earlier in the development lifecycle. UAT is about identifying the gaps between how the completed application works and how it’s expected to perform in a business environment.
Challenges in UAT
UAT’s primary objective is to eliminate the risks of production failures and ensure that the end application or system meets all desired business requirements. However, an application being developed in an Agile mode leads to reduced and frequent cycles of testing, which in turn mandates UAT testers to develop skills of optimization testing techniques, automation and work in cohesion with the development and QA teams.
The challenges for UAT in an Agile development model can be broadly classified into four categories which include business challenges, people & process, governance and tools & automation as listed in the figure 1 below:
Figure 1: UAT Challenges in an Agile mode of development
Overcoming challenges with a UAT Center of Excellence
The recommended framework
The UAT Center of Excellence (CoE) addresses all challenges of UAT in an Agile development model, where the testing processes, domain and technical skills are developed and enhanced through:
- Sharing best practices
- Reusing key assets like people, processes and tools
- Establishing a governance model
- Optimizing testing through tools & accelerators and
- Establishing a knowledge management framework
The UAT CoE would result in efficiencies with standardized processes, a metrics-driven governance model, reusable automation scripts and test cases that can be used across various sprints & artifacts. Figure 2 is a snapshot of the proposed UAT Center of Excellence.
Figure 2: The UAT Centre of Excellence
The UAT approach
We recommend a unique UAT approach that addresses all its challenges in an Agile development model, where the UAT team would work in tandem with the development and QA teams using CoE best practices to enhance test coverage and efficiency.
The UAT team gets involved early in the project life cycle and is engaged in the entire iterative process of the recommended approach as seen in figure 3. The UAT team works with a story card acceptance criteria for each iteration. This helps eliminate many potential defects with early business validations and improves efficiencies through optimum automation of regression test beds.
Figure 3 defines the UAT team’s responsibility at each stage of the iterative process.
Figure 3: The Recommended UAT Approach within the UAT COE, part of the iterative process
Proposed methodologies in the recommended UAT approach
The following innovative methodologies are recommended in the proposed UAT approach:
1. Infosys business driven test ideas technique (iBDTI). The iBDTI strategy is built on an Agile test design approach. The strategy includes building test scenarios using one-liner test ideas that help capture test requirements and improve testing coverage and effectiveness.
2. Multi-dimensional requirements traceability matrix. The UAT team would leverage the multi-dimensional requirements traceability matrix which has five layers. The UAT scenarios generated from user stories and business requirements are a core part of this process.
3. Test data management. To ensure comprehensive test coverage, the UAT team is involved earlier in the life cycle to identify test data, resulting in better control and reduced dependency on the development and QA teams for the same.
4. Risk-based testing (RBT). RBT helps minimize failure and optimize testing.
5. Metrics-driven governance model. A UAT level metrics dashboard helps assess and govern the health of the UAT by monitoring the project quality information, status and productivity details via the metrics-driven governance model.
Benefits of the proposed UAT Approach
The key benefits of this approach are:
1. Reduced testing costs. Early involvement of the UAT team in the life cycle leads to the early identification of defects and reduces costs associated with rework significantly.
2. Reduced UAT cycle. Optimized test coverage and accelerated test execution lead to a shorter UAT cycle as per Agile requirements.
3. Business value. Deployment of robust processes, tools, accelerators, optimized testing techniques and automation lead to applications or systems that meet all business requirements, timelines and are devoid of production failures. This helps create business value for organizations.
4. Stakeholder involvement. The metrics driven governance model ensures that all stakeholders get the needed information on UAT and its progress.
5. Reduced test design effort and increased test coverage. While the test design effort reduces, the testing coverage increases with methodologies like iBDTI, multi-dimensional requirements traceability metrics, risk-based testing and a metrics driven governance model.
Conclusion
With most projects moving to an Agile mode of testing, the software development lifecycle has become shorter, which makes it mandatory for UAT to use efficient testing processes, automation and metrics in order to deliver high quality end systems or applications that are low in cost and require minimal effort. UAT in a CoE framework can help organizations achieve these benefits for an Agile development program.
A successful UAT is crucial for deploying systems devoid of errors and business risks. This in turn helps an organization achieve its intended business goals earlier.
How does your organization execute UAT? If operating in an Agile environment, have you made changes to your UAT processes to meet the challenges of changing requirements? Let SSQ know by sending an email to [email protected].
About the authors:
Ravindra Kambhampati ([email protected]) is a Senior Project Manager with Independent Validation Solutions Unit. His core competency is in designing and executing projects across the SDLC spectrum (ADM and Testing). He has been involved in various proposals in the Insurance and Healthcare domain. His areas of specialization are mainframes application testing and development, presales activities, design and implementation of TCoE. He holds a Master’s Degree in Industrial Engineering.
Srinivas Yeluripaty ([email protected]) is a Group Project Manager working with Infosys FSI Testing Practice. He has over 13 years of experience in verification & validation services for large US corporations in Capital Markets, Insurance and Healthcare payer domains. His interests and experience is involved in Test Center of Excellence strategy development and implementation, consultation in setting up independent and centralized quality control functions. He is PMP certified and has completed two six sigma implementation (Infosys BrITe framework – Gold Certification) for testing productivity and business results improvement.
