Azure Data Engineer

JD:

The Azure Databricks Engineer will be responsible for designing, developing, and implementing large-scale data processing solutions on the Azure Databricks platform. The role involves collaborating with data scientists, analysts, and business stakeholders to deliver robust, scalable, and high-performance analytics solutions that support business objectives.

Key Responsibilities

Develop and optimise data pipelines using Azure Databricks, Spark, and related technologies.

Design and implement ETL (Extract, Transform, Load) workflows to ingest data from diverse sources into Azure Data Lake and other storage solutions.

Collaborate with cross-functional teams to understand data requirements and deliver end-to-end solutions.

Monitor, troubleshoot, and enhance Databricks jobs for performance, reliability, and scalability.

Ensure data quality, security, and compliance with organisational and regulatory standards.

Document processes, technical designs, and best practices for ongoing maintenance and knowledge sharing.