Business Acceptance Testing – Is Your Product User Ready?
– By Ashwin Kumar Chandrashekaraiah
Enterprise
QE
Introduction
Over the years, it has been observed that a new product or service that meets all its stated technical
requirements might still fail to achieve the actual purpose for which it was conceptualized. This is because,
the stakeholders grossly ignored Business Acceptance Testing (BAT) which can have a huge impact on the
business of the customer. Perry and Rice, in their book ‘Surviving the Top Ten Challenge of Software
Testing’, say that “Users should be most concerned with validating that the system will support the needs
of the organization. The question to be answered by Business Acceptance Testing is ‘will the system meet
the business or operational needs in the real world’?’
Perry and Rice add that Business Acceptance Testing (BAT) includes “Identifying all the business processes
to be tested; decomposing these processes to the lowest level of complexity, and testing real-life test cases
through those processes.”
Business Acceptance Testing (BAT) can include functional, system and regression testing. However, its
overall purpose is to validate the software against business needs. BAT is conducted to verify whether the
system can support day-to-day business and user scenarios to validate rules, various workflows, data
correctness, and overall fit for use and ensure whether the system fits business requirements.
BAT is a relatively new phenomenon and is similar in many ways to User Acceptance Testing (UAT) and User
Testing (UT). However, before embarking on explaining BAT in detail, it is important to understand
the difference between BAT, UAT, and UT.
User Acceptance Testing (UAT) and User Testing (UT) are the testing processes where the software is tested
for ‘real world’ application. UAT ensures that the software meets business usage requirement and its
behavior is consistent with the requirements. The customer specifies scenarios to test when a user story is
written and each scenario can have multiple acceptance tests. Customers are responsible for verifying the
correctness of the test. In this scenario, it is not expected for the developer or the testing engineer to have
actual domain knowledge. He or she goes by the user stories and the test conditions written by the
customer. Without domain knowledge, these stories are implemented but not necessarily understood.
BAT is a combination of UAT, and UT and a little more than that. It is not a mere contractual ritual, where the
product passes a user acceptance test as a condition of sale. In the case of UAT, for example, on actual
usage, customers find bugs which were not visible earlier and the project suddenly slows down, costs go
up and timeframes affected for the end customer
BAT is testing based not only on user stories but an understanding of end user behavior. This calls for
actual domain knowledge, especially for the testing team. If possible, every individual working on a project,should have a clear understanding of the domain and the customer’s business. If there is a knowledge gap,
it needs to be addressed by including domain and business specific training programs.
Trigent Software’s customer, a market leader in GPS satellite tracking and advanced solutions for
optimized management of distributed assets was looking to consolidate verification and validation
activities to reduce dependency, create a flexible and scalable model in line with growing needs,
standardize tools and follow a stringent testing methodology. They hoped that this would help to reduce
time-to-market and cost while maintaining quality and performance.
Trigent developed an automated functional/ regression testing of the Web application, automated Web
Services in multiple environments, achieved better test coverage and developed framework to support
Automation Execution.
As a result of the attention to business acceptance testing, the Web Service tests had 90 percent coverage
and the customer experienced improved time-to-market with minimal schedule variance. The highlight of
the project was – reduced defect leakage with zero critical post-release bugs.
| DIFFERENCE BETWEEN UT, UAT, AND BAT | |||
|---|---|---|---|
| Testing | User Testing(UT) | User Acceptance Testing(UAT) | Business Acceptance Testing (BAT) |
| Related to | Product Engineering | Product Engineering | Product Engineering and Product Marketing |
| Scope | User experience, Performance | Functionality | Concept, idea or users’ preferences and ease of use |
| Focues is on | Scenarios, tasks and user goals | Functionality | Concept, idea or prototype |
| Measures | Performance with users | Functionality | End-customers’ preferences |
| Mode | Individual | Black-box testing | Group |
| Team participants | User community | User community & QA team | Independent testing team |
| Conducted by | Test moderator | Developer organization | Developers, marketers, and test moderators |
| Tone of testing | Evaluation and Validating | Quality control | Stage-gate check and go or no-go |
BAT's Differentiators
Developers, as well as functional testers, are technical people and they test the product against functional
specifications. The bugs and issues which are an outcome of functional testing can be fixed.
BAT plays an important role in validating that all the business requirements are fulfilled before the release
of the software in the market. Use of live data and real use cases differentiate BAT from other forms of
testing.
BAT tests are created to verify that the systems functionality is consistent to requirements and it is a sure
way to reduce or eliminate change requests and reduce project costs.
The following points further clarify what BAT is really meant to do:
- It is the last stage of testing before shipping
- It tests to a standard of compliance with requirements, based on specific examples
- It is a set of tests that are run, for a customer, to demonstrate functionality
- It is a set of prescribed tests that absolutely must pass before the user will take the product
- It is a set of prescribed tests that absolutely must pass as stipulated in contracts
- It is any testing that is not done by a developer
- It is tests that are done by real users
- It is tests that are done by stand-ins or surrogates for real users
Some of the critical reasons for effective BAT by the customer are:
- Ensure that the product delivered meets business criteria.
- Ensure that the product supports the dependent business.
- Reduces or negates cost for fixing issues after launch.
- Reduce errors arising due to factors such as:
- Poorly defined requirements
- Misinterpretation of business requirements
- Incorrect translation of business requirements into functional flows
BAT Process
The BAT team forms an integral part of the product development process and has various important tasks
to be performed such as:
- BAT Requirement Review to ensure requirements are tied to business needs
- Test Planning for BAT
- Business Test Case Design
- Business Test Execution
The work associated with BAT begins while requirements are written and continues through the final stage
of testing. It is important to understand business requirements to prepare tests and test data which is
realistic. It also helps to define the context in which the system will be used and assess its fit as described
in the diagram below:
The BAT team uses several templates/documents for effective testing of the product. All such documents
will become part of the deliverables to the customer for making informative decisions on the quality of the
product.
| Used by the BAT Tester | Created by BAT Tester |
|---|---|
| Business Process Models | Test Plan |
| System Requirements Specification | Test Cases |
| Business Requirements | Defect Log |
| Use Cases | |
| Testing Guidelines |
While it makes sense to have the testing team work with the customer, the time needed to train the testing
team on the business side of the project can be long and expensive. On the other hand, working with a
software services provider who has domain expertise will already have the expertise, experience, and
domain knowledge to make the process of Business Acceptance Testing simple and straightforward.
Trigent’s customer, a Fortune 500 leading truck maker in the US, manufactures and sells commercial and
military trucks, buses, diesel engines, recreational vehicles, as well as provides service parts for trucks and
trailers across the US through a network of 100s of dealers. The company facilitated ‘made-to-order’
custom truck configuration based on customer’s requirements
Processing a customer’s request required interaction between two applications. One of the applications
maintained product’s configuration information while the other maintained engineering rules and
international standards. The system was meant to enable customers to choose parts configurations and
check whether they adhered to engineering rules and international standards. These data-driven
applications were integrated with a number of other functional applications. Ensuring data accuracy and
data flow consistency across these applications was of supreme concern as it had a direct impact on the
client’s business
Each product had several rules, and a minor defect would affect downstream applications, thus affecting
the entire business.
Trigent team of experts’ used test case modularity framework and prepared independent test scripts at a
modular level to test the components. This modularity of test case scenarios accommodated faster
changes in business functionality, reduced complexity and time by optimizing the test case scenarios. A set of automated test suites was created to keep the quality at an optimum level and ensure quality code for
each major module release. Trigent’s solution also enabled the applications to increase its user load
capacity. The team performed load testing to assess loads at different load scenarios
Benefits included:
- Modularizing the testing framework helped improved component performance
- Automated performance test suite helped save time and reduced time to market
- Faster performance on more than 200 users load
- Improved data Flow and accuracy across the applications
BAT should be planned with clear acceptance test plan during requirement analysis and in the design
phase. In strategic planning, a set of real life scenarios should be identified for execution. If objectives for
BAT are not clearly defined at the beginning, then implementing BAT at the end will not be successful.
Summary
Delivering effective software that meets business needs is always a challenge, even more challenging
when delivered in a globally distributed model, outsourced to a partner and with ever-present challenges
of scope, timelines, costs and effort. The test team needs all the help it can get to effectively test the
software and make sure that it meets quality expectations and does so without causing time and budget
overruns.
A test manager’s job does not end with the successful completion of testing. The BAT team needs to be
ramped up on any relevant aspect (usage, features, CRs, etc.) as may be required. They need to be
informed of the correct and current state of the application, including the known/open bugs. The BAT and
test managers need to be on the same page with respect to deployment schedules, defect lifecycles, defect
reporting mechanisms and triaging, test environment issues, patches to be applied, exit criteria/quality
thresholds and the like.
While the BAT team is very much a part of the client’s business, and is a completely distinct entity, it
nevertheless can often add a wealth of information and provide valuable inputs, not to mention be a part
of the all-important relationship-building effort.
Trigent Testing Centre of Excellence draws upon 15+ years of testing experience with an accumulated
knowledge base, proven track record, and deep domain expertise. We have dedicated resources certified
to International standards such as ISTQB, CSTE, CSQA, and Equivalent, with expert onshore/onsite and
offshore capability. With scalable resource and delivery flexibility – there is always high skilled capacity to
accommodate increased demand.
References:
- Overview of User Acceptance Testing (UAT) for Business Analysts (BAs) by Vince Bordo (DevelopMentor)
- User Acceptance testing a user guide- Brian Hambling, Pauline van Goethem
- en.wikipedia.org