Azure Databricks Engineer

Azure Databricks Engineer

Location: Remote

Contract: Freelance / Contract

The Opportunity

We are looking for an experienced Azure Databricks Engineer to join a technology team working on data engineering and cloud data platform initiatives.

This is an excellent opportunity for a hands-on engineer with strong Azure and Databricks experience to build, optimise and maintain scalable data solutions, working with modern cloud technologies and large-scale data processing.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build and optimise ETL/ELT processes to ingest, transform and process large volumes of data.
• Develop and maintain data lakehouse solutions using Delta Lake.
• Work with Azure Data Factory to orchestrate and manage data workflows.
• Implement data quality, monitoring, performance optimisation and error-handling processes.
• Collaborate with data engineers, architects and wider technical teams to deliver reliable data solutions.
• Troubleshoot complex data engineering issues and improve pipeline performance.
• Follow best practices around security, governance, documentation and deployment.

Essential Skills & Experience

• Strong commercial experience with Azure Databricks .
• Excellent Python and PySpark skills.
• Experience building production-grade data pipelines and ETL/ELT processes.
• Strong knowledge of SQL and data modelling.
• Experience working with Delta Lake and lakehouse architectures.
• Hands-on experience with Azure data services, ideally Azure Data Factory and ADLS Gen2.
• Understanding of data quality, performance tuning and pipeline optimisation.
• Comfortable working independently in a remote environment.

Desirable Experience

• Experience with Azure DevOps, Git and CI/CD pipelines.
• Knowledge of Databricks Unity Catalog and data governance.
• Experience with Terraform or infrastructure as code.
• Familiarity with streaming technologies and real-time data processing.
• Experience with medallion architectures and Bronze, Silver and Gold data layers.

Apply Now

If you are an experienced Azure Databricks Engineer looking for your next remote contract opportunity, we would love to hear from you.

Please submit your CV to discuss the role in more detail.