Databricks SME

The Role

We are looking for a Databricks Platform SME to join Version 1s Data & AI Practice, with strong Databricks expertise and the ability to take primary ownership of our Databricks platform. You will be responsible for Databricks administration, governance, entitlements, performance optimisation, and overall operational management, while contributing to scalable, cloud-native data solutions that enable analytics and AI at scale.

This is a delivery-focused role for an experienced engineer who enjoys solving complex data challenges, takes ownership of platform outcomes, and applies strong engineering discipline in real-world environments.

What You’ll Be Doing

• Partnering with business and technology stakeholders to build and enhance reliable, scalable data platforms
• Taking primary ownership of the Databricks platform, including administration, governance, entitlements, optimisation, and operational management
• Managing Databricks workspaces, users, groups, access controls, platform configuration, performance, reliability, and cost efficiency
• Building and maintaining cloudnative data solutions on AWS
• Supporting production systems by troubleshooting issues and implementing continuous improvements
• Ensuring solutions are productionready: secure, tested, observable, and costefficient
• Optimising data performance, reliability, and data quality in live environments
• Designing, building, and operating batch and streaming data pipelines
• Developing highthroughput, lowlatency data processing systems
• Orchestrating data workflows using Airflow or similar orchestration tools
• Writing clean, efficient, and welltested code following software engineering best practices
• Collaborating closely with product, analytics, and platform teams

Qualifications

What You’ll Bring

• 8+ years’ experience in handson data platform engineering roles
• Strong Databricks expertise, including platform administration, entitlements/access management, governance, performance optimisation, and operational management
• Hands-on experience managing Databricks workspaces, clusters, jobs, notebooks, and production Spark workloads
• Demonstrated ability to manage and optimize Databricks platform operations at scale
• At least 2 Databricks platform implementations from design to production in the last 4 years.
• Strong handson coding skills in Python, Java, or similar languages, with strong SQL capability
• Proven experience building and operating batch and streaming pipelines in production
• Handson experience using dbt for transformations and Airflow (or equivalent) for orchestration
• Solid understanding of data modelling, data quality, and production data systems
• Experience working with relational databases such as Postgres or SQL Server
• Familiarity with CI/CD pipelines, testing frameworks, and monitoring / observability tooling

Nice to Have

• Databricks Professional Certification
• Experience enabling ML, AI, or GenAI workloads through strong data foundations
• Exposure to modern BI and analytics tooling
• Experience working in regulated or securityconscious environments
• Consulting or clientfacing delivery experience