Databricks Engineer

Job Title: Databricks Engineer

Rate: £400 per day (OUTSIDE IR35)

Contract Length: 6 months (extendable)

Location: Remote with occasional site visits in either London or Birmingham

Databricks engineer - Role & Responsibilities

• Build, maintain and optimise Databricks data pipelines using PySpark, SQL and Delta Lake to transform raw data into curated, business-ready datasets.
• Manage production deployments through Azure DevOps (ADO), including code reviews, testing, release management and work tracking.
• Monitor and troubleshoot data pipelines, resolving issues and optimising performance, reliability and cost.
• Implement data quality checks and governance controls using Unity Catalog to ensure data integrity, security and compliance.
• Collaborate with business stakeholders and the Platform team to translate requirements into technical solutions, maintaining clear documentation and support runbooks.

Databricks engineer - Required Experience

• Proven commercial experience as a Databricks Engineer, Data Engineer or Data Platform Engineer, with hands-on experience supporting production environments.
• Strong technical expertise in Databricks, PySpark, SQL and Delta Lake, including data transformation and pipeline optimisation.
• Experience with Azure DevOps (ADO), including CI/CD pipelines, code reviews, automated testing and production releases.
• Knowledge of Unity Catalog, data governance, data quality validation and production monitoring.
• Strong problem-solving and communication skills, with the ability to work collaboratively with business and technical teams

Interviews: October 2026

Start Date: November 2026

How to Apply:

Apply to Calice Mills at X4 Technology to learn more about the role or connect with me on LinkedIn to hear about regular job updates available with X4 Technology