Azure Data Engineer

We are looking for an experienced Azure Data Engineer to design, develop, and optimize scalable data platforms and pipelines on Azure. The ideal candidate should have strong expertise in Azure Databricks, PySpark, Azure Data Factory, ADLS Gen2, and Delta Lake, with experience building modern Lakehouse architectures.

Key Responsibilities

• Develop and maintain ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize Lakehouse solutions using Delta Lake and Medallion Architecture (Bronze, Silver, Gold).
• Design scalable data ingestion, transformation, and data quality frameworks.
• Work with structured and semi-structured data formats such as Parquet, Avro, JSON, and ORC.
• Implement data governance, lineage, and access controls using Unity Catalog and/or Microsoft Purview.
• Support AI-ready data products, RAG pipelines, and vector search use cases.
• Monitor, troubleshoot, and optimize pipeline performance and cloud costs.
• Collaborate with Data Scientists, AI Engineers, Architects, and business stakeholders.

Required Skills

• 5+ years of Data Engineering experience.
• Strong hands-on experience with Azure Databricks, PySpark, Python, and SQL.
• Experience with Azure Data Factory (ADF), ADLS Gen2, and Azure Synapse.
• Expertise in Delta Lake, Lakehouse, and Medallion Architecture.
• Good understanding of ETL/ELT, Data Warehousing, and Data Modeling.
• Experience with data governance tools like Unity Catalog or Microsoft Purview.

Preferred Skills

• Azure AI Search, RAG, Vector Search, Embeddings.
• Streaming technologies such as Kafka, Event Hubs, or Structured Streaming.
• dbt, Great Expectations, Delta Live Tables.
• Terraform, Bicep, Azure DevOps, GitHub Actions.