Senior Azure Data Engineer

Role & Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Data Factory, Databricks, PySpark, and SQL.
• Build and optimize ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop data solutions using Azure Databricks, Delta Lake, and Azure Synapse Analytics.
• Implement data transformation and processing using PySpark and Python.
• Work with Medallion Architecture (Bronze, Silver, Gold) to build efficient and maintainable data platforms.
• Develop and maintain data models to support reporting, analytics, and downstream business requirements.
• Ensure data quality, accuracy, reliability, and performance across data pipelines.
• Troubleshoot and optimize existing pipelines and data processing jobs.
• Collaborate with data engineers, architects, analysts, and business stakeholders to understand requirements and deliver effective data solutions.
• Follow best practices for security, governance, version control, and CI/CD in data engineering projects.
• Mentor junior data engineers and contribute to technical discussions and solution design.

Preferred Candidate Profile

• 5+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.
• Strong hands-on experience with Azure Data Factory, Azure Databricks, PySpark, Python, and SQL.
• Good experience in Delta Lake, Azure Synapse Analytics, and data warehousing concepts.
• Strong understanding of ETL/ELT, data modeling, data pipelines, and data architecture.
• Experience working with Bronze, Silver, and Gold data layers / Medallion Architecture.
• Good understanding of performance tuning and optimization of Spark and SQL workloads.
• Experience with Microsoft Fabric and Unity Catalog would be an added advantage.
• Strong problem-solving and analytical skills.
• Good communication skills with the ability to work effectively with cross-functional teams.
• Prior experience mentoring team members or taking ownership of data engineering initiatives is preferred.