Senior Databricks / Azure Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure Databricks.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Azure DevOps environment.
• Ensure high availability of the system by implementing monitoring tools such as Prometheus/Grafana.

Job Requirements :

• 5-8 years of experience in Data Engineering with expertise in ADF, Azure Databricks, PySpark.
• Strong understanding of Git version control system for managing code repositories.
• Experience working on big data technologies like Hadoop/Hive/Pig/Sqoop etc. .
• Proficiency in writing complex SQL queries for querying relational databases.