Lead Data Engineer

Role Overview

We are looking for a Lead Data Engineer with strong experience in Azure Data Engineering, Python/PySpark, SQL, and Data Warehousing. The role involves designing scalable data pipelines, leading Azure ETL solutions, and providing technical guidance to the team.

Key Responsibilities

• Design and develop scalable data pipelines using Python/PySpark.
• Architect and implement Azure ETL/ELT solutions using Databricks, Data Factory, Blob Storage, Synapse, Azure SQL, and Lakebase.
• Work with cross-functional teams to translate business requirements into technical solutions.
• Lead code reviews, ensure engineering best practices, and maintain technical documentation.
• Mentor junior engineers and provide technical leadership.
• Troubleshoot data, pipeline, and performance issues.
• Use Git, Azure DevOps, and Jira for source control and project delivery.

Required Skills

• 8+ years of experience in Data Engineering/Data Warehousing.
• 5+ years of experience with Python/PySpark.
• 8+ years of experience with SQL, including complex queries, stored procedures, and functions.
• Strong experience with Azure Databricks, Data Factory, Blob Storage, Synapse, Azure SQL, Lakebase, and Unity Catalog.
• Knowledge of Azure Functions, Logic Apps, Azure VMs, Git, and Azure DevOps.
• Strong problem-solving, communication, and leadership skills.

Preferred

• Azure Data Engineer certification is a plus.
• Experience with Lakehouse architecture and data governance.

Role Details

Role: Lead Data Engineer

Employment: Full-Time, Permanent

Experience: 8+ Years

Education: B.Tech/B.E., BCA, B.Sc., or any relevant postgraduate qualification