Lead Data Engineer

Lead Data Engineer
Chennai
9+Years

Role Summary

Technical leader responsible for the modernization and future-state architecture of the Sales Validation platform.

Responsibilities

Lead Databricks architecture and delivery

Drive Synapse-to-Databricks migration

Establish engineering standards and reusable frameworks

Mentor engineers and conduct design reviews

Own technical delivery across major workstreams

Drive performance optimization and scalability

Required Skills

Databricks (Highest Priority)

Databricks

PySpark

Spark SQL

Delta Lake

Unity Catalog

Databricks Workflows

Databricks Asset Bundles

Lakehouse Architecture

Azure Platform

Azure Data Factory (ADF)

ADLS Gen2

Azure SQL

Azure Security Patterns

Migration Experience

Synapse Pipelines

Synapse Spark

Synapse SQL Pools

Synapse Notebooks

Synapse Data Flows

Modernization of legacy ETL workloads

Engineering

Advanced SQL

Data Modeling

GitLab CI/CD

Performance Tuning

Code Reviews

Preferred

Apache Airflow

Snowflake

Data Governance

Enterprise-scale Azure platform implementation

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.