Data Engineering Lead – Databricks & Cloud Data Migration

Experience: 6 to 10 years

Joining Timeline: Immediate to a maximum of 15 days

Job Overview

We are looking for an experienced Data Engineer/Architect to lead cloud data platform modernization and migration initiatives, with strong expertise in Databricks (Unity Catalog, Delta Lake, Delta Sharing, Lakebase), enterprise ETL/ELT tools, and data governance frameworks. The candidate will drive Oracle-to-modern-platform migrations and support AI/ML-ready data product development.

Key Responsibilities

• Design and implement metadata governance, data cataloging, and secure data-sharing solutions using Databricks Unity Catalog and Delta Sharing.
• Build and manage cloud-native transactional/operational data capabilities leveraging Databricks Lakebase.
• Architect and execute data migration strategies from Oracle to modern platforms (Delta Lake, Snowflake, Azure SQL Database, Azure Synapse, PostgreSQL, MongoDB, etc.).
• Develop and orchestrate ETL/ELT pipelines using Azure Data Factory, dbt, Informatica, Fivetran, or equivalent tools.
• Implement enterprise data governance using Purview, Collibra, Alation, or Informatica EDC — covering lineage, data quality, and master data management (MDM).
• Support building AI/ML-ready data products, including feature engineering pipelines for downstream analytics/ML use cases.
• Collaborate with cross-functional teams to ensure compliance and governance standards, especially in regulated industries (healthcare, financial services, insurance).

Required Skills

• Hands-on experience with Databricks Unity Catalog, Delta Sharing, and metadata governance.
• Experience with Databricks Lakebase or similar cloud-native transactional/operational database capabilities.
• Strong cloud platform experience (Azure, AWS, or GCP).
• Proficiency with ETL/ELT tools — Azure Data Factory, dbt, Informatica, Fivetran, etc.
• Experience with data governance tools — Microsoft Purview, Collibra, Alation, or Informatica EDC.
• Proven experience migrating workloads from Oracle to Databricks Delta Lake, Snowflake, Azure SQL Database, Azure Synapse, PostgreSQL, MongoDB, or similar.
• Knowledge of data quality, lineage, MDM, and metadata management practices.
• Experience supporting AI/ML-ready data pipelines and feature engineering.
• Domain knowledge in healthcare, financial services, insurance, or other regulated industries is a plus.