Azure Data Engineer

Required Skills:

• Strong experience in Data Engineering and ETL/ELT processes
• Proficiency in Python
• Hands-on experience with Apache Spark (PySpark and Spark SQL)
• Experience with Azure Databricks (Unity Catalog), Snowflake, Azure Functions, and Service Bus
• Strong SQL development and optimization skills
• Experience with Infrastructure as Code using Terraform
• Hands-on experience with CI/CD tools such as GitHub Actions, Git, Artifactory, and SonarQube
• Experience with AI/ML concepts and Large Language Models (LLMs)

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions
• Build and optimize data processing workflows using Python and Spark
• Develop cloud-based data solutions using Azure Databricks and Snowflake
• Implement and manage infrastructure using Terraform
• Develop and maintain CI/CD pipelines for data engineering projects
• Collaborate with cross-functional teams to deliver high-quality data solutions
• Support AI/ML and LLM-based initiatives as required