Azure Data Engineer

Job Summary

We are looking for an experienced Azure Data Engineer with strong hands-on expertise in Azure Databricks, PySpark, Python, and SQL. The candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data processing solutions on the Azure cloud platform.

Mandatory Skills

• Azure Databricks
• PySpark
• Python
• SQL
• Strong experience in Data Engineering / ETL / ELT
• Experience in developing and optimizing data pipelines
• Strong understanding of data transformation, data cleansing, and data integration
• Good knowledge of Azure cloud data services
• Experience working with large-volume datasets and distributed processing

Key Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Write efficient Python and SQL code for data extraction, transformation, and loading.
• Perform data transformation and processing using PySpark DataFrames and Spark SQL.
• Design and implement ETL/ELT workflows for structured and semi-structured data.
• Optimize Spark jobs, SQL queries, and Databricks workloads for performance and cost.
• Work with Azure data services to build cloud-based data solutions.
• Perform data validation, cleansing, and quality checks.
• Troubleshoot data pipeline failures and resolve performance issues.
• Collaborate with Data Architects, Analysts, Developers, and business teams.
• Follow coding standards, CI/CD practices, version control, and Agile methodologies.
• Participate in design discussions, code reviews, testing, and production deployments.