Data Engineering Lead

Job Summary

This role builds and owns the data infrastructure across digital systems, market intelligence, and product operations. The data engineering lead is also the programmes data governance officer: the person who ensures that when anyone asks where does that number come from, the answer is a documented, auditable data lineage.

Responsibilities

• Build scalable ETL/ELT pipelines to handle high-velocity streaming data and batch processing.
• Implement data integrity checks, metadata management, and security protocols.
• Provide the data backbone for the MDO, Commercialisation, and engineering teams to ensure seamless data flow.
• Maintain the highest standards for code quality, including conducting rigorous code reviews, developing comprehensive unit tests, and producing clear, thorough documentation.
• Design and implement the data warehouse: raw data lake + structured analytical warehouse with clear separation of raw, cleaned, and modelled data layers.
• Implement data lineage tracking: every number that enters a model output must be traceable back through transformations to its original source record.
• Design the data access layer for the appropriate teams and create experimental, isolated, and research sandboxes.

Requirements

• 8+ years in Data Engineering with experience leading engineering teams.
• Expertise in Apache Spark/Kafka, Airflow, and cloud infrastructure (AWS/Azure/GCP).
• Advanced knowledge of NoSQL, Relational, and Vector databases.
• Proven understanding of fundamental data structures and the ability to apply them to solve complex problems.
• Cloud data warehouse experience: Schema design, query optimisation, cost management
• Comfort with large, messy, multi-source datasets
• Experience with data quality frameworks
• Experience with Containerization (Docker/Kubernetes)

Preferred Qualifications

• Experience at a data-intensive platform company where data pipelines directly powered business-critical decisions
• Exposure to time-series data pipelines: has built systems that ingest, store, and serve temporal data cleanly (traffic growth time-series, economic indicators)
• Understanding of mathematics, particularly linear algebra and probability theory

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.