Evolving towards 5th generation test automation frameworks
– By Jagadish Anandan & Madhav Kanchiraju
Generic
QE
There are many testing tools in the market today that claim to be easy to use and maintain. However, the focus is still on harnessing scripting to push for better automation coverage.
As the lines of code grow longer, so does the complexity, resulting in higher maintenance costs for scripts and frameworks. There is a growing need to minimize the use of test scripts while improving maintainability and quality. This idea forms the foundation of Scriptless Test Automation.
This paper traces the evolution of test automation and discusses why enterprises must move towards the 5th generation test automation frameworks.
Evolution of test automation frameworks
1st gen: record & playback
A browser extension or a plug-in records a tester's interaction with an application. Thereafter, the test-scripts are auto-generated and executed for testing. Often used when the same tests must be repeated many times over.
2nd gen: use & re-use
The test cases are divided into building blocks (i.e., modular approach) to ensure proper coverage. By breaking them into reusable pieces and placing them in a library, these units become easily referenceable. Further, these units can be combined to build larger end-to-end testing scenarios.
3rd gen: data-driven scripts
Application-specific scripts are coded with the provision of accommodating variable datasets. The data is stored in a database table or spreadsheet. Thus, a single test script is capable of executing tests for all the test data. Input values are extracted from the data files and stored as a variable in test scripts. It enables exercising both positive and negative test cases through a single test script.
4th gen: action keyword scripts
Here, a keyword is mapped to an individual testing action. For example, a mouse click, keystrokes, selecting a menu item, etc. A sequence of operations is executed when a keyword is invoked. By dragging and dropping the keyword, the tester can simulate the user actions that correspond with the keyword. Keyword-driven scripts combined with data-driven philosophers formed the backbone of this generation. Keywords can also represent a series of actions that undertake a standard business flow.
Shortcomings of past generations of test automation
- Traditional test automation tools are complex, time-consuming, and expensive to maintain. A steep learning curve is required to master the use of these tools.
- Recorders have limited capabilities and can break easily. A small change in the presentation layer code is capable of breaking the recording, requiring the testers to start over again.
- Most tools require strong coding skills as any changes in software must be addressed in the testing code. The scripts are written in a tool-specific language, which disempowers other business stakeholders and consequently increases workload.
- Varying tester competencies and individual capability result in inconsistent scripting standards. This adds to the project overhead and derails the quest for achieving operational efficiencies.
- UI-based test scripts are disadvantaged right from the start because their execution is reliant on a graphical user interface (GUI) that does not change.
What is the 5th generation test automation?
Scriptless testing, the 5th generation testing, comprises simplified methodology embodied in objects, descriptors, and keywords. It relies on fueling the execution engine to initiate actions. In a significant departure from the fourth-generation testing which involves scripts, keywords, graphs, etc., the 5th generation testing is based on recorded test sequence. Some of the of 5th generation test automation tools include Mabl, Ranorex, Worksoft, TOSCA, etc.
Let us briefly explore how the 5th generation test automation tools function.
When the application is first set up, the tool executes the default tests that are a prerequisite for any application. To finish these tests, the tool only requires access credentials to log into the application. The default tests, over time, grow smart enough to understand how to login and navigate through the entire application.
The tool accesses the application through the browser sessions running in containers or virtual engines that reside in the cloud. It runs the tests to finish certain operations, records the output, processes the results, and supplies the insights back to the user.
The 5th generation testing lends itself for easy participation and review from business stakeholders of all hues, including development teams, testing teams, business analysts, domain experts, etc. If introduced early in the software development lifecycle, it improves the ease of script design and results in higher return-on-investment. Therefore, a good agile practice can be leveraged to implement scriptless automation.
Advantages of 5th generation testing frameworks
- Speed: Faster time-to-market and enables agile testing.
- Maintainability: Less maintenance overhead cost
- Configurability: Make testing assets easy to conform to different requirements with minimal changes.
- Robustness: Enables easy plug and play anywhere
- Operational efficiency: Scriptless test automation can reduce up to 80% of the test automation scripting efforts, helping testers focus on core value areas instead of repetitive and peripheral works.
Top myths to avoid
There are many myths surrounding the scriptless test automation. Below, we address the top four:
Scriptless means "no-code"
Code does exist and needs to be adapted.
It translates to "no effort"
No. Factually, it is a record and playback with the ability to configure quickly, thus making it faster (but not without requiring effort).
It functions automatically
Automation checks must be well designed to accurately locate the chinks in the armor.
It requires zero programming skills
Basic programming skills to understand objects and interaction are still required.
Conclusion
Scriptless test automation is a disruptive testing approach that fundamentally changes the way testing is done. Advantages such as superior quality and cost-containment are manifest from the beginning of the development phase. It can also be seen as an enabler to early automation in an agile team.
Scriptless test automation is a disruptive testing approach that fundamentally changes the way testing is done. Advantages such as superior quality and cost-containment are manifest from the beginning of the development phase. It can also be seen as an enabler to early automation in an agile team.
It frees the testers’ minds from non-value-added activities and helps them focus on their core goals. Scriptless test automation empowers testers to shift-left and serves the requirements of Agile testing.