Navigating the future of testing

How new-age tech will impact QA

– By Diwakar Menon, Jagadish Anandhan & Raghukiran HB

Generic

Executive summary

Software testing is integral to software development. Software development companies look to integrate testing from the early stages of development to mitigate challenges that may arise using new-technologies and developing complex products. Integrating testing from the early stages helps offset high costs that may occur if the product or software shows errors in later development or after release stages. Software testing has become more exhaustive as testers have to contend with testing for futuristic scenarios, and new-age devices encompassing many internet-enabled products such as wearables or bots.

The need and demand for digital products buoyed by the digital transformation pace and the evolution of new-age technologies will require testing organizations to adopt new tools, processes, and upgrade skills. Testing teams will need to be aligned with the various stakeholders to envisage requirements and enable the delivery of quality products at scale and speed.

This whitepaper outlines the current digital transformation stage and its influence as a driving force for growth and enablement. It captures the disruption the ongoing pandemic has had on digital transformation, diving into this disruption drivers.

It touches on the imminent challenges testers face in this evolving technology landscape. It espouses the philosophy of responsible testing and articulates its eight essentials attributes that pave way for frictionless customer experience.

Digital transformation and the impact of COVID19

The evolution of technology is the harbinger of digital transformation. As technology evolves, creating new possibilities to enhance processes and experiences, the demand and need for digital transformation accelerate. Digital transformation is an enabler that helps organizations create new business models, elevate workforce productivity, and advance marketplace positioning.

The digital transformation market valued at $342 billion in 2019 is expected to shoot up to $923 billion by 2025, clearly emphasizing that businesses and organizations are gearing up for a digitally-driven future. Companies that embrace modern technology to drive value will be able to leverage the multifold benefits of digital transformation and stay ahead of the curve.

Many digital-first businesses are already reaping the benefits of being the frontrunners in the digital transformation realm. The Standard & Poor’s (S&P) 500 is continuously changing, a reflection on the pace of disruption in the market. Young companies like Google, Amazon, Netflix, and Uber have fast evolved into business unicorns and made it to the top of S&P lists.

The future of an organization will be driven by its agility to respond to changing market conditions, evolving regulatory norms, and innovation in technologies. To derive business value, organizations must enhance their processes, competencies, and business efficiencies by embracing digital transformation. Further, product features and competencies must be primed to maximize digital value. Quality must remain at the center of digital transformation efforts.

Tech companies are fast replacing the traditional leaders

Covid-19 has either shuttered unprepared businesses or hurtled businesses into the future by one giant leap. Overnight, companies had to rethink their strategy on serving customers without exposing their employees to threat. As new economics, lifestyles, and ways of working become more evident, businesses now need to adopt digital technologies to overcome the crisis rapidly. The change is particularly perceptible in sectors like retail, education, and healthcare. The thrust on contactless/touchless technology has increased manifold, and the innovation is focussed on seamlessly delivering these experiences.

Organizations are building a future-ready enterprise today to support an environment of dynamic and constant change, both in business needs and user expectations.

They are embracing a commitment to the transformation that keeps them relevant, agile, and competitive

The demand for contactless and virtual consulting amid the pandemic has given rise to telemedicine and test packs. A similar impact can be seen across industries, as illustrated in the image.

Drivers of disruption

The thrust on contactless experiences, while preserving the human warmth of relationships, will be the defining theme of the next decade. Contactless/touchless experience can be pushed by digital technology like Augmented Reality (AR), Virtual Reality (VR), Internet of Things (IoT), Blockchain, Robotic Process Automation (RPA), Over-the-top media service (OTT), and Voice Assistance (UI).

The signs were everywhere, but COVID19 catalyzed and crystallized the change.

IoT-enabled wearables were picking up pace, but the prospect of using health data from Fitbit devices to customize health insurance plans is no longer a distant idea. Blockchain, already finding applications beyond its cryptocurrency origins, can be the engine of trust in many use cases. Instances centering on profile validation, as in insurance and HR processes, come to fore instantly. Chatbots and voice-enabled UIs are expected to make a quick transition from margins to the mainstream. Reports suggest that (RPA) automation initiatives have received a big push in the BPO industry and are poised to gain further prominence in the manufacturing sector.

But the elephant in the room is the significant shift in the way we work: working remotely. Many businesses were already open to offer this flexibility, but it was more of an exception than the rule. The forced adoption of remote working, spurred by the pandemic, has settled doubts about large-scale remote workforce feasibility. Much to their surprise, organizations found that they could continue business as usual at a better rate than previously imagined. But working remotely exposes the enterprise applications to a whole new level of complexities and security issues. The need to scale the infrastructure necessary to support remote working will fuel process acceleration and automation initiatives.

Organizations are increasingly adopting newer processes to collaborate effectively, improve speed-to-market, and increase product release velocity. Thus we are witnessing a rise of agile scrum practices, DevOps, DevSecOps, AIOps, ITSecOps, Continuous Integration and Continuous Deployment (CI/CD), and Continuous Testing. To maximize the benefits accrued from these approaches, it’s incumbent on organizations to use automation.

Witnessing a rise in adoption of…

The principles underlying the new-age architectures like microservices and containerization allow for deployment at scale. And these changes are finding their way into the digital transformation equation.

Imminent challenges testers face in the new age

Digital transformation is accelerating the pace of change in every industry. Processes are evolving faster, a wider variety of tools are available, and new architectures are being explored. All these developments pose new challenges to testers

Customer experience & touchpoint validation

Customer experience or CX is a significant differentiator and a giant propeller for new-age tech. Testers have their task cut out, trying to test software powered across multiple devices and platforms for a frictionless omnichannel experience. Ensure that all touchpoints are in sync, irrespective of where the user starts his journey and the pattern he follows. The application or software should handle vast volumes of data or a sudden spike in the users. All this to enable a frictionless customer experience. Ensuring a frictionless experience for customers across devices and platforms can be arduous for testers.

Regulatory knowledge in their industry

Modern-day software development requires testing teams to be aware and vigilant of change in regulations to stay compliant. Data protection concerns and laws such as the European GDPR have made security a vital element for the smooth launch and use of the software. Such regulations require that security testing be included in the software development lifecycle (SDLC) from the early stages of development. It is also vital that testers know the industry-specific regulations that can become a bottleneck if not planned early in software development. For example, the Health Insurance Portability and Accountability Act or HIPAA is a regulation created to safeguard private information about patients. It is a must-have regulatory standard for any business looking to deal with patient or healthcare information.

Similarly, different regulations emphasize accessibility to ensure the differently-abled can access and utilize services and offerings. Extensive testing must be carried out to ensure the software meets the accessibility requirements such as the Web Content Accessibility Guidelines and others. Staying well-versed with changing regulations must be a mandate for testers.

Artificial Intelligence / Machine Learning

Many organizations are increasing the uptake of artificial intelligence (AI) and machine learning (ML) in their products to enhance, enrich and empower their products. This enables them to sharpen their competitive edge in the marketplace. Therefore, the testing tools and frameworks must necessarily harness the power of AI ML too. Therefore, new-age testers must keep apace with these technologies as well and skill themselves accordingly.

Effective test data management

Data is an immensely valued resource. Effective management of test data is essential to understand and interpret the function, response, performance of particular software. Effective test data management is more critical in data-exhaustive domains such as insurance, fintech, healthcare, etc. Managing complex and sensitive data can turn out to be a challenge.

Choosing the right tools and frameworks

A successful software product requires many elements to work in cohesion. A couple of essential features that enable the delivery of quality software are frameworks and tools. The abundance of tools and frameworks makes it challenging to choose the right mix for a given project.

Responsible testing is the key

Many consider ‘responsible’ testing a tautology, given that every testing effort is intended to be ‘responsible.’ But the approach has a proven record of ensuring customer delight, making it a vital element for testers. From our recent survey where we asked participants what ‘responsible’ testing is, we received the following result.

A testing unit defines the role and responsibility of its people, which is then rigorously followed. A responsible approach goes beyond the call of pre-defined duties and facilitates end-to-end stakeholder assurance and business value creation. It focuses on balancing value with risk and business outcomes while putting customer experience at all efforts’ heart.

Designing test cases and strategies from the customer perspective can ensure apt code logic and better customer experience. A responsible testing culture ensures that the team adopts the collaborative test design approach, shift left, continuous integration, and continuous testing along with judicious and early adoption of automation testing. As responsible testers, the team must do all the necessary due diligence to assess operational readiness and risk coverage.

Responsible testing approach and philosophy: 8 essential attributes

While technology is an enabler, the decisions still rest with humans. A culture of responsibility harnesses the man-machine symbiosis to full potential. Here are the eight essential attributes of responsible testing that can help you maximize business value.

Risk coverage

This approach considers a risk-based strategy versus the more common coverage-based strategy. A traditional coverage-based strategy would try to address questions like How many tests did you design? What was the test coverage that you achieved? In a risk-based strategy, one attempts to optimize (minimize) the number of tests to cover most of the risks. Thus, we can have 10% of tests cover the maximum number of risks or 80% of tests still not covering all risks.

A risk-based strategy or approach is different in terms of how we want to evaluate the product's performance or what it might mean once it is released. This strategy ensures the product promises to customers is upheld. It needs to minimize the possible fallout of risk (for example, of revenue loss) due to poor customer experience because of a feature or functionality failure.

To mitigate the negative impact on business due to a product release, one needs to identify all the risks involved by collaborating and discussing with all the key stakeholders, including product owners and domain experts. Identify all the business-critical functions and features that will be in high demand or usage. Understand the high-risk areas and where the application can fail. Even old testing philosophy says that 80% of bugs will be found in 20% of the application, therefore identifying the high priority application. This way, a 20% coverage will ensure 80% risk coverage. One critical parameter is a good understanding and analysis of past failure data. Understanding failure patterns and identifying the root cause may not be a functional problem but a data migration issue.

Collaborative test design

Over the years, it has been observed that test design is performed by one person, working in isolation gathering inputs from documents or other items. A test design prepared and managed in isolation can miss critical perspectives and conditions that should be tested.

Our responsible testing approach recommends a collaborative test design approach as opposed to an individual performing it alone. We all very well know that collaboration is key to successful testing in an Agile and DevOps approach. Also, while testing complex applications, a tester needs to be creative and cover functionality, and go beyond what may be described in a requirements document, use case, or user story.

We recommend a collaboration across business analysts, product owners, developers, and quality assurance teams to get a holistic and better understanding of the process, external interfaces, usage, and dependencies concerning the software to be tested. One also needs to understand how support teams or client services will be involved to address software failures. Another critical consideration for a responsible tester is to ensure that all the stakeholders in this collaborative process can understand the test design and test cases. For example, one can use a Gherkin lang

Benefits of a collaborative test design approach are:

Shared problem-solving ability
Practical perspective in prioritizing tests
Good knowledge of the application
Faster and effective communication about the project, the tests, and the problems to be solved
Increases stakeholder ownership in the test design activity
Greater creativity for user stories or to build test cases

Customer experience focus

In the end, it all boils down to the quality of customer experience as the ‘Customer is the King’. A responsible tester gets into the shoes of the customer and designs the tests accordingly.

One needs to understand the business flows, how the customer or the user interacts with the system. Different types of users will also interact with the application (end-user, administrator, back-office, customer support, etc.). Therefore there is a need to understand how they go about interacting? What are the touchpoints that each user has? What promises have been made to different users at these touchpoints? It's essential to get real user data and try to use that in a way that can allow us to simulate those customer touchpoints much before we go live. Advanced analytics tools will help you find the behavior patterns of the customer.

Different types of users will also interact with the application (end-user, administrator, back-office, customer support, etc.). Therefore there is a need to understand how they go about interacting? What are the touchpoints that each user has? What promises have been made to different users at these touchpoints? It's essential to get real user data and try to use that in a way that can allow us to simulate those customer touchpoints much before we go live. Advanced analytics tools will help you find the behavior patterns of the customer

While conducting the test based on the roles, it is vital to consider the usage permissions provided for that specific role and the user's production or actual environment. Sometimes, testing in a controlled or lab environment is not reflective of the real end-user experience based on role, privileges, and environment.

Test early, test often

Everyone talks about testing shifting left early into the development phase. What does that mean?

Many projects widely use agile scrum practices and DevOps methodologies that witness a high frequency of build, implementation, releases, and feedback. Therefore, a responsible testing approach proposes to test early and often test, i.e., a tester joins the SDLC process right at the start. Thus, a tester can ask questions about the customer journey, which can lead to changes. Working with back-end developers and ideating with them helps create test ideas and 'what if' type scenarios. Engagement with the API developers results in writing out tests for new services while they are being developed. And sometimes, pairing up with the UI and API developers can lead to testing something new before it hit a build. CI/CD pipeline must be implemented to get early feedback on the builds and run frequent automation regressions.

Testing may still be necessary before the final release. But it’s faster because many problems would already have been identified and resolved earlier on

Early focus on non-functional

It is good to plan for the non-functional testing needs at the beginning of the application life cycle. Many tools allow you to do a quick assessment, if not complete assessments, of non-functional testing needs. There are automated security testing tools that give you a sense of vulnerabilities. There are code review tools that provide you with leakage issues. Now, these are not a substitute for running useful security tests or code security analysis of your product, but these are good early indicators

Organizations must be concerned if testers focus on functional requirements and stories during the initial sprints, and only then look back at non-functional testing. The application that may be compatible with one browser, can crash on another.

Some pointer for successful non-functional testing:

Test the application for performance continuously for the defined concurrent user load sprint on a sprint basis access is consistent and supports the localization needs
Ensure the application is compliant with the accessibility guidelines
Ensure that the user response times are in the acceptable range
Continually assess for security violations
Ensure that the application's geographical

Leverage automation

There is much hype around test automation. Testers measure automation by the amount of coverage that is achieved. Is it really the silver bullet that testers think it is? Probably not, but it’s still a vital parameter to make testing more efficient

The agile approach requires early and continuous testing. Not just the newly-appended code, but the code from previous iterations, too, must be a part of test coverage. While this ensures unbroken functionality, it further burdens the testers that can lead to quality issues. Automation frees the tester from repetitive tasks, enabling them to focus on exploratory testing.

The below-mentioned elements are apt candidates for automation:

Release readiness

The focus of the product release should mainly be on operational readiness. Ensure the application is stable enough and ready for the release on various parameters like capability, usability, compatibility, reliability, security, scalability, performance, and installability. The product shouldn't behave differently in varied scenarios.

As a part of a go-live, it’s vital to create a mechanism to capture and relay customer feedback to product owners promptly. Consequently, this enables testers to respond to customer issues as and when they arise quickly.

Continuous feedback

Lastly, but surely not the least important, is continuous feedback. This is one of the critical things that we have included as a part of our responsible testing philosophy. It is about understanding what happens when a release goes live

A responsible tester is glued to how the application is performing. What are the kinds of issues that customers are facing? They walk the extra mile of not just understanding the customer issues but provide a fix for the issue as soon as it is reported.

One could look at continuous feedback as a shift right mechanism wherein we are entering the production environment arena. Analyzing the end-user logs from production gives insights into real user experience, identify usage patterns, peak load time period, and failure points.

Conclusion

The wave of digital transformation is growing in enormity. The speed of digital transformation for long has been in the state of acceleration buoyed by technology as a propeller. The pandemic has further boosted the transformation.

Testers have to be integral to this transformation. As reports indicate, testers must support a growing market slated to increase to $70 billion by 2024 from the current $40 billion value. Change is imperative that testing teams have to acknowledge and adopt. Awareness about the critical trends mentioned below can help testing organizations embrace change with ease.

Automation

The adoption and dependence on automation are poised for healthy growth. Codeless and self-healing test automation will make it simpler. A combination of AI and RPA for test automation will enable organizations to automate reports and analysis.

Tools strategy

QA tool strategy among organizations will not be limited to just commercial-off-the-shelf tools (COTs) but will consider open source tools. The future QA strategy in terms of tools will look at a combination of open source tools and COTS as no one tool will address all the needs.

Upskill capabilities

The future driven by AI applications will not just be dependent on Software Development Engineers in Test (SDET). It will require complementary skill sets in data science and statistical modeling, giving rise to requirements for software development artificial intelligence engineers in test (SDAIET). This means that QA experts will need knowledge not just about development but also have experience in data science and statistical modeling

Redefined KPIs

Testing teams will have the same KPIs as other members in the software development process, evaluated against the customer experience across the end-to-end business transactions. Integrated KPIs will be the way forward for organizations to encourage innovation and collaboration.