Case Study

Modernizing Workforce Intelligence: How Trigent Migrated a Legacy HR Platform to Databricks on GCP

Information Technology (IT)
Cloud Services

About the Client​

The client is a prominent provider of cloud-based HR and workforce management solutions, trusted by organizations of all sizes to optimize the employee lifecycle. Their platform spans recruitment, onboarding, performance management, payroll, and learning, empowering companies to enhance employee engagement and drive business success.

Business Challenge​

As the client scaled, their legacy infrastructure based on Cloudera struggled to meet growing performance and scalability demands. Key challenges included:

  • Limited scalability and sluggish performance during complex data transformations and streaming workloads
  • Inflexible ETL workflows that made real-time reporting and analytics difficult to manage
  • High operational overhead associated with maintaining a Cloudera-based big data ecosystem
  • Complex MapReduce jobs that were increasingly outdated and difficult to maintain

To continue supporting dynamic HR needs and rich analytics, the client needed a modern data platform with cloud-native flexibility and streaming-first architecture.

Trigent Solution

Trigent partnered with the client to migrate their data infrastructure from Cloudera to Databricks on Google Cloud Platform (GCP), enabling real-time performance and streamlined data operations. Key elements of the solution included:

  • Full migration of the Cloudera platform to Databricks on GCP, ensuring future scalability and lower total cost of ownership
  • Deployment of Kafka, Debezium, and Kafka Connect to enable robust real-time data streaming from source systems
  • Implementation of Databricks Autoloader with Spark Streaming Jobs, designed to handle incremental data merges across 800 tables into the Silver Layer
  • Migration of legacy ETL workflows, creating 120 Gold tables from Silver, and generating 200 reporting tables through 50 new transformation jobs
  • Conversion of MapReduce jobs to Spark for an improved Attendance domain, boosting maintainability and performance
  • Enablement of incremental refresh of Kylin Cube, preserving advanced analytical capabilities while modernizing the underlying pipeline

Client Benefits

Trigent’s migration to Databricks on GCP delivered a high-performance, future-ready data platform for the client.

  • Up to 40% reduction in data transformation time, improving SLA compliance for HR reporting
  • Real-time workforce insights, which made possible by a streaming-first architecture and automated ETL pipelines
  • Lower maintenance effort and faster time-to-deploy with modern Spark-based workflows
  • Scalable infrastructure that supports growing data volumes and evolving analytics needs
  • Improved data governance and traceability, aligning with compliance requirements in HR and payroll domains