Azure Data Engineer

Roles and Responsibilities

• Utilize PySpark/DataBricks for big data processing tasks such as ETL processes.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Snowflake databases.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
• Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Snowflake.

Desired Candidate Profile

• 6-11 years of experience in designing and developing large-scale data pipelines using ADF or similar tools like Azure Databricks.
• Strong understanding of SQL programming language with expertise in writing efficient queries for query optimization on Snowflake DBs.
• Proficiency in working with PySpark/DataBricks for big data processing tasks such as ETL processes.