Business Acceptance Testing – Is Your Product User Ready?

Over the years, it has been observed that a new product or service that meets all its stated technical requirements might still fail to achieve the actual purpose for which it was conceptualized. This is because, the stakeholders grossly ignored Business Acceptance Testing (BAT) which can have a huge impact on the business of the customer. Perry and Rice, in their book ‘Surviving the Top Ten Challenge of Software Testing’, say that “Users should be most concerned with validating that the system will support the needs of the organization. The question to be answered by Business Acceptance Testing is ‘will the system meet the business or operational needs in the real world’?”

Perry and Rice add that Business Acceptance Testing (BAT) includes “Identifying all the business processes to be tested; decomposing these processes to the lowest level of complexity, and testing real-life test cases through those processes.”

Business Acceptance Testing (BAT) can include functional, system and regression testing. However, its overall purpose is to validate the software against business needs. BAT is conducted to verify whether the system can support day-to-day business and user scenarios to validate rules, various workflows, data correctness, and overall fit for use and ensure whether the system fits business requirements.

BAT is a relatively new phenomenon and is similar in many ways to User Acceptance Testing (UAT) and User Testing (UT). However, before embarking on explaining BAT in detail, it is important to understand the difference between BAT, UAT, and UT.

User Acceptance Testing (UAT) and User Testing (UT) are the testing processes where the software is tested for ‘real world’ application. UAT ensures that the software meets business usage requirement and its behavior is consistent with the requirements. The customer specifies scenarios to test when a user story is written and each scenario can have multiple acceptance tests. Customers are responsible for verifying the correctness of the test. In this scenario, it is not expected for the developer or the testing engineer to have actual domain knowledge. He or she goes by the user stories and the test conditions written by the customer. Without domain knowledge, these stories are implemented but not necessarily understood.

BAT is a combination of UAT and UT and a little more than that. It is not a mere contractual ritual, where the product passes a user acceptance test as a condition of sale. In the case of UAT, for example, on actual usage, customers find bugs which were not visible earlier and the project suddenly slows down, costs go up and timeframes affected for the end customer.

BAT is testing based not only on user stories but an understanding of end user behavior. This calls for actual domain knowledge, especially for the testing team. If possible, every individual working on a project should have a clear understanding of the domain and the customer’s business. If there is a knowledge gap, it needs to be addressed by including domain and business specific training programs.

Trigent Software’s customer, a market leader in GPS satellite tracking and advanced solutions for optimized management of distributed assets, was looking to consolidate verification and validation activities to reduce dependency, create a flexible and scalable model in line with growing needs, standardize tools and follow a stringent testing methodology. They hoped that this would help to reduce time-to-market and cost while maintaining quality and performance.

Trigent developed an automated functional/regression testing of the Web application, automated Web Services in multiple environments, achieved better test coverage and developed framework to support Automation Execution.

As a result of the attention to business acceptance testing, the Web Service tests had 90 percent coverage and the customer experienced improved time-to-market with minimal schedule variance. The highlight of the project was – reduced defect leakage with zero critical post-release bugs.

The BAT team forms an integral part of the product development process and has various important tasks to be performed such as:

• BAT Requirement Review to ensure requirements are tied to business needs.
• Test Planning for BAT.
• Business Test Case Design.
• Business Test Execution.

The work associated with BAT begins while requirements are written and continues through the final stage of testing. It is important to understand business requirements to prepare tests and test data that are realistic. It also helps to define the context in which the system will be used and assess its fit, as described in the diagram below:

The BAT team uses several templates/documents for effective testing of the product. All such documents will become part of the deliverables to the customer for making informative decisions on the quality of the product.

While it makes sense to have the testing team work with the customer, the time needed to train the testing team on the business side of the project can be long and expensive. On the other hand, working with a software services provider who has domain expertise will already have the expertise, experience, and domain knowledge to make the process of Business Acceptance Testing simple and straightforward.

Trigent’s customer, a Fortune 500 leading truck maker in the US, manufactures and sells commercial and military trucks, buses, diesel engines, recreational vehicles, as well as provides service parts for trucks and trailers across the US through a network of hundreds of dealers. The company facilitated ‘made-to-order’ custom truck configuration based on customer requirements.

Processing a customer’s request required interaction between two applications. One of the applications maintained product configuration information, while the other maintained engineering rules and international standards. The system enabled customers to choose parts configurations and verify whether they adhered to engineering rules and international standards. These data-driven applications were integrated with several other functional applications. Ensuring data accuracy and data flow consistency across these applications was of paramount importance, as it had a direct impact on the client’s business.

Each product had several rules, and even a minor defect could affect downstream applications, thereby impacting the entire business.

Trigent’s team of experts used a test case modularity framework and prepared independent test scripts at a modular level to test the components. This modularity of test case scenarios accommodated faster changes in business functionality, reduced complexity, and saved time by optimizing the test case scenarios. A set of automated test suites was created to maintain optimum quality and ensure high-quality code for each major module release. Trigent’s solution also enabled the applications to increase their user load capacity. The team performed load testing to assess performance under different load scenarios.

Benefits included:

• Modularizing the testing framework helped improve component performance.
• The automated performance test suite helped save time and reduce time to market.
• Faster performance with a load of more than 200 users.
• Improved data flow and accuracy across the applications.

BAT should be planned with a clear acceptance test plan during the requirement analysis and design phases. During strategic planning, a set of real-life scenarios should be identified for execution. If the objectives for BAT are not clearly defined at the beginning, implementing BAT at the end of the project will not be successful.