Azure Data Engineer

Role & responsibilities

Key Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines using Azure services.
• Build and manage data pipelines using Azure Data Factory (ADF).
• Develop scalable data solutions using Azure Data Lake Storage (ADLS Gen2).
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Use Python/PySpark for data processing and transformation.
• Work with Azure Databricks for large-scale data processing and analytics.
• Perform data extraction, transformation, cleansing, and integration from multiple sources.
• Implement data quality, validation, monitoring, and error-handling processes.
• Optimize data pipelines and queries for performance and cost efficiency.
• Work with Azure Synapse Analytics / Microsoft Fabric where required.
• Implement CI/CD practices using Azure DevOps.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and other engineering teams.
• Ensure data security, governance, reliability, and scalability of data solutions.

Mandatory Technical Skills

• Microsoft Azure
• Azure Data Factory (ADF)
• Azure Data Lake Storage Gen2 (ADLS)
• SQL
• Python / PySpark
• Azure Databricks
• ETL / ELT
• Data Warehousing
• Azure Synapse Analytics
• Azure DevOps / CI-CD

Good to Have

• Microsoft Fabric
• Delta Lake
• Apache Spark
• Power BI
• Azure Functions
• Event Hub
• Azure Key Vault
• Unity Catalog
• Data Governance
• Infrastructure as Code / Terraform

Candidate Profile

• 39 years of experience in Data Engineering / Cloud Data Engineering.
• Strong hands-on experience with Microsoft Azure data services.
• Good understanding of data warehousing concepts, dimensional modeling, and ETL/ELT processes.
• Strong problem-solving and analytical skills.
• Experience working in Agile development environments.
• Good communication and stakeholder-management skills.

Education

Bachelor’s/Master’s degree in Computer Science, Information Technology, Engineering, or a related field

Preferred candidate profile