Case Study

AI Oncology Company Achieves 95% Reduction in Testing Cycle Time with Test Case Automation

Trigent's automation solution enhances test coverage and accelerates development for an AI-driven precision oncology company

Healthcare
AI Testing QE Test Automation

About the Client​

The client is a leading AI-driven precision oncology company based in Chicago. It leverages AI tech to predict the effectiveness of treatments, accelerate drug development, and diagnose multiple diseases earlier. With a commitment to excellence, it continues pushing boundaries, transforming disease understanding, diagnosis, and treatment worldwide.

Business Challenge

The client offers a suite of healthcare solutions based on AI analysis of genomic data. These applications require accurate DNA sequencing, specialized expertise to handle complex business rules, strict adherence to HIPAA and HL7 standards for patient privacy, and specialized testing for custom APIs.

The complexity of the applications and the multi-dimensional attributes of genomic data pose unique challenges during quality testing. While extensive manual testing is initially required to validate new features, automating the test cases requires specialized domain knowledge and expertise in testing AI models while complying with patient privacy requirements. Given the paucity of genomic domain expertise and AI application testing skills at the client’s end resulted in:

  • A significant number of test cases are being executed manually, impacting QA timelines.
  • Increased backlog of test cases that could be automated pending resource availability.
  • Test automation framework unable to support the specialized requirements of genomic testing.

Trigent’s prior experience in quality assurance of AI applications for genomic solutions and its established HealthTech domain credentials addressed the client’s partner criteria.

Trigent Solution

Trigent assembled a dedicated team of automation specialists, including a Senior Test Automation Architect and five Automation Engineers with deep expertise in Playwright and related technologies. This team tackled the client’s existing manual testing processes head-on.

Solution Scope and Project Plan

Following a detailed discovery effort, the Trigent team, in consensus with the client, identified the following activities and milestones for the project:

  • Automate Test Creation: Leverage tools like Playwright, Concourse, and internal solutions to automate testing of 900+ manual test cases across multiple applications.
  • Enhance Automation Coverage: Address the gaps in the test automation framework for critical applications like Customer Systems, Laboratory Information Management Systems (LIMS), and the Reporting Pipeline.
  • CI/CD Integration: Develop custom Playwright APIs to integrate with the client CI/CI system to streamline testing and deployment.
  • Set up Comprehensive Testing Services: In addition to the test automation scope, Trigent proposed implementing smoke tests, comprehensive end-to-end test authoring, and detailed documentation for developer tools encompassing CLI, cloud services, and web interfaces.

This is where Trigent’s expertise in automation came into play. It offered the necessary technological solutions and resources to overcome these obstacles.

Automating AI Application Test Cases

To succinctly address gaps in test automation coverage, Trigent developed detailed test automation scripts for crucial applications such as Reports and Laboratory Information Management System (LIMS), which manage, track, and streamline various laboratory processes. Designing test cases for AI algorithms requires specialized expertise, curated test data sets, and an appropriate combination of cloud and on-prem infrastructure capable of handling the volume of genomic data. Trigent collaborated with onsite automation test leads to ensure smooth implementation.

Customization of the Test Automation Framework

To bolster the existing Playwright and TypeScript frameworks, Trigent focused on:

  • Enhancements to address specific testing requirements and improve overall functionality.
  • Strengthening the automation framework to scale and handle the demands of a growing test suite.
The approach involved developing a robust API testing framework using Playwright and Python. Unlike generic testing frameworks, these APIs were specially designed to meet the client’s genetic API testing needs, ensuring precise functionality. Trigent enhanced the existing Playwright and TypeScript frameworks by adding critical utilities such as:
  • Logging and reporting mechanisms for clear insights and traceability.
  • Configuration loaders for smooth and flexible test execution.
  • Data parsers for robust and adaptable tests.
  • Parameter management for flexible and reusable test scripts.
  • API clients for interacting with common APIs.
  • Credential management for secure storage and retrieval of tokens.
  • Authentication utilities are used to handle mechanisms like OAuth and JWT.
  • Storage state restoration for login efficiency.
  • Functions to remove low-quality reads or trim adapter sequences.
  • Utilities to validate genetic data integrity and quality.
  • Tools to track and manage biological samples accurately.
This helped significantly enhance test automation coverage and reduce test execution time, contributing to faster, more reliable, and efficient quality assurance.

Data-Driven Testing for Efficiency

Trigent implemented data-driven testing to maximize coverage and efficiency. Manual test flows were converted into data-driven combinational tests. This approach utilizes a single specification file to execute numerous tests with diverse data inputs, eliminating the need for individual tests for every scenario. This ensured thorough testing of various situations, encompassing user roles, insurance options, and payment methods.
For instance, the client’s login functionality was tested with different user roles. The framework executed a global setup for login across applications, storing credentials in JSON format and using the “StorageState” feature to manage context. These credentials were injected into the browser to facilitate the login process, ensuring efficient and comprehensive testing.

CI/CD Integration and Extended Testing Services

The Enhanced Test Automation framework was seamlessly integrated with the client’s CI/CD pipelines to streamline testing and deployment processes, ensuring continuous integration and delivery. Beyond core test case automation, Trigent provided additional testing services, including:
  • Smoke Testing: Implementing smoke tests ensured basic functionality before further investment in detailed testing.
  • End-to-end Testing: Comprehensive end-to-end tests validated the entire user journey.
  • Developer Tools Documentation: Detailed documentation for Developer Tools (CLI, cloud services, and web-based tools) enhanced usability and understanding.

Client Benefits

The automation solution offered the following benefits

Reduction in Manual Effort
  • 95% decrease in testing cycle time by automating regression tests and critical functionalities.
  • 90 hours of manual regression execution cycle reduced to 4 hours.
Enhanced Test Automation Coverage
  • 80% of the testing is automated with test case automation across Customer Systems, LIMS, and Reporting Pipeline applications.
  • Data-driven testing allowed the client to cover diverse scenarios with different data sets, ensuring robust testing.
Accelerated Development Cycle
  • Automated initial login and user role tests in one week, significantly faster than the estimated four weeks.
  • Completed converting all tests to data-driven tests in just 3 days, significantly exceeding expectations of a 2-week timeframe.
Improved Reliability and Quality
  • Enhancements to the Playwright and TypeScript frameworks improved logging, reporting, and data management, resulting in more reliable tests.
  • Automated data validation and cleanup ensured genetic data quality.
  • End-to-end testing services thoroughly validated all application aspects, reducing production issues.
Scalability and Flexibility
  • Common functions, storage state restoration, and parameter management enhanced the framework’s scalability and adaptability.
  • Improved configuration loaders and data parsers facilitated flexible and smooth test execution.

Trigent’s tailored test automation solution for AI-powered genomic applications enabled the client to significantly reduce manual efforts, enhance test automation coverage, accelerate development cycles, and improve its AI-driven healthcare applications’ overall reliability and quality. This allowed the client to concentrate on its core mission of advancing precision medicine.

Technology Stack

Concourse_logo
playwright
slack
unittest
xray