Databricks Lead Engineer

Who you are

• Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms
• Strong consulting values with ability to collaborate effectively in client-facing environments
• Hands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumption
• Strong problem-solving, analytical, and communication skills
• Experience leading or mentoring teams of engineers to deliver high-quality scalable data solutions
• Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow)
• Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent
• Strong SQL and Python (or equivalent language) skills for data manipulation and automation
• Hands-on experience with cloud platforms (AWS, Azure, GCP)
• Familiarity with Databricks Workflows and other orchestration tools
• Knowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon)
• Familiarity with medallion architectures, data lakehouse principles and distributed data processing
• Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelines
• Understanding of data governance, security, and compliance frameworks
• Exposure to AI/ML workloads desirable
• Experience: Minimum 5–8 years in data engineering, data warehousing, or data architecture roles, with at least 3+ years working with Databricks
• Education: University degree required
• Preferred: BSc/MSc in Computer Science, Data Engineering, or related field
• Databricks certifications (Data Engineer Professional) highly desirable

What the job involves

• Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive
• Client Engagement & Delivery
• Data Pipeline Development (Batch and Streaming)
• Databricks & Lakehouse Architectures
• Data Modelling & Optimisation (Delta Lake, Medallion architecture)
• Collaboration & Best Practices
• Quality, Governance & Security
• Business Relationships:
• Solution Architects
• Data Engineers, Developers, ML Engineers, and Analysts
• Client stakeholders up to Head of Data Engineering, Chief Data Architect, and Analytics leadership
• Measures of Success:
• Delivery of high-performing, scalable, and secure data pipelines aligned to client requirements
• High client satisfaction and successful adoption of Databricks-based solutions
• Demonstrated ability to innovate and improve data engineering practices
• Contribution to the growth of the practice through reusable assets, accelerators, and technical leadership