Techical Lead Databricks

About the role

Our client is seeking an experienced Technical Lead – Databricks to support the detailed design, technical delivery and implementation of data platform components required for the Generative AI and MCU Phase 2 initiatives. The Technical Lead will be responsible for developing detailed and low-level technical designs across the Databricks platform, providing technical guidance and support to delivery teams, and contributing hands-on to the development of data platform components. This is a hands-on technical leadership role suited to someone with strong Databricks, Apache Spark and Python experience who can translate high-level requirements into robust, scalable technical designs and work effectively with both technical and business stakeholders.

Key responsibilities

• Provide technical leadership across the Databricks components supporting the Generative AI and MCU Phase 2 initiatives.
• Develop detailed and low-level technical designs for Databricks-based data platform solutions.
• Translate business and technical requirements into scalable, secure and maintainable technical solutions.
• Provide technical guidance to developers and delivery teams throughout the implementation lifecycle.
• Review technical designs and ensure solutions align with enterprise architecture and engineering standards.
• Design and develop data platform components using Databricks, Apache Spark and Python.
• Support the implementation and optimisation of Databricks workloads across the data platform.
• Contribute hands-on to coding, development and technical problem-solving where required.
• Ensure solutions are designed for scalability, performance, reliability and maintainability.
• Work closely with architecture, data engineering and project teams to ensure Databricks solutions meet program requirements.

About you

• Proven experience working as a Technical Lead, Lead Data Engineer or Senior Data Engineer within a complex enterprise environment.
• Strong hands-on experience with the Databricks platform.
• Strong coding experience with Apache Spark and Python.
• Demonstrated experience creating detailed and low-level technical designs.
• Strong understanding of data engineering and modern data platform architectures.
• Ability to translate requirements into practical, scalable technical solutions.
• Strong problem-solving and analytical skills.
• Excellent communication and stakeholder engagement skills.
• Ability to provide technical leadership while remaining hands-on with development and implementation.