Azure Data Engineer

Job Title: Azure Data Engineer 8+ Years

Experience: 8+ Years
Location: Pan India
Notice Period: Immediate to 30 Days only

Job Summary

We are looking for an experienced Azure Data Engineer with 8+ years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure Data Factory, Azure Databricks, PySpark, SQL, Azure Data Lake Storage, and data warehousing.

The candidate will be responsible for building robust data pipelines, implementing ETL/ELT processes, optimizing data platforms, and working closely with data architects, analysts, and business stakeholders.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines on Microsoft Azure.
• Develop and optimize ETL/ELT workflows using Azure Data Factory (ADF).
• Build data processing solutions using Azure Databricks and PySpark.
• Develop complex SQL queries, stored procedures, views, and performance-optimized data solutions.
• Work with Azure Data Lake Storage Gen2 (ADLS) for large-scale data ingestion and processing.
• Implement data integration from various sources including databases, APIs, files, and cloud applications.
• Design and implement data warehouse and data lake solutions.
• Perform data transformation, cleansing, validation, and quality checks.
• Implement incremental data loads, CDC, partitioning, and performance optimization.
• Monitor, troubleshoot, and optimize data pipelines and production workflows.
• Implement appropriate security, access control, and data governance practices.
• Collaborate with Data Architects, Data Scientists, BI Developers, and business teams.
• Participate in code reviews, technical design discussions, and production support.
• Follow best practices for CI/CD, version control, testing, and deployment of data engineering solutions.

Required Technical Skills

• 8+ years of experience in Data Engineering / ETL development

• Strong hands-on experience with Microsoft Azure

• Strong expertise in:

• Azure Data Factory (ADF)

• Azure Databricks

• PySpark

• Python

• SQL

• Azure Data Lake Storage Gen2 (ADLS Gen2)

• Good understanding of Azure Synapse Analytics / Azure SQL

• Strong knowledge of ETL/ELT concepts and data warehousing

• Experience with data modeling, dimensional modeling, and large-scale data processing

• Experience working with relational and non-relational data sources

• Good understanding of Delta Lake / Lakehouse architecture

• Experience with Git and CI/CD pipelines

• Strong understanding of performance tuning and troubleshooting

Good to Have

• Experience with Azure Synapse Analytics
• Knowledge of Microsoft Fabric
• Experience with Kafka / Event Hubs
• Knowledge of Power BI and BI data platforms
• Experience implementing CI/CD using Azure DevOps
• Understanding of Data Governance, Data Quality, and Security
• Experience with Agile/Scrum methodologies
• Azure certifications such as Azure Data Engineer Associate (DP-203)