Senior Data Engineer (Data Platform DevOps & Architecture)

• We’re seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform with DevOps at its core
• This permanent position focuses on building an automated, self-service architecture designed for continuous evolution
• You’ll architect solutions using modern patterns, establish automated data pipelines with CI/CD practices, implement infrastructure-as-code, and mentor our team while building systems that adapt to emerging technologies
• We’re committed to Azure but flexible on specific products – we value pragmatic solutions based on actual needs
• Reports to: Product organisation – you’ll work closely with product teams to align data capabilities with business objectives
• Location: London office 3 days per week, flexible remote work the rest of the time
• Team: You’ll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholders
• Timeline: Initial focus on migration (6 months), then platform evolution, DevOps maturity, and capability expansion
• Architect a modern data platform with DevOps principles for long-term business evolution
• Establish automated engineering best practices for data, governance, and operations
• Build robust batch and real-time processing pipelines (integrating web application data with analytics sources and external APIs)
• Create comprehensive documentation and training materials
• Translate business requirements into maintainable technical solutions
• Mentor team members and drive continuous improvement
• You’ll establish the foundation for our data platform’s next decade. This platform will power our next generation of internal and customer products while empowering our business with real-time insights for faster, smarter decisions

Benefits

• Hybrid working environment: Split your time between our Moorgate office and remote working from home; the best of both worlds.
• Continuous development: We actively encourage and enable your continued growth and development.
• Bring your dog to work: …or if you don’t have a dog, enjoy hanging out with these cuties!
• A strong sense of fun: From light-hearted chat on Slack, to quarterly company activities - the work is serious, but we find the fun.
• Dress for your day: You won’t find us suited and booted for no reason.- You’ll bring 5+ years of data engineering experience with proven expertise in implementing enterprise-scale data platforms and DevOps methodologies
• We need someone who prioritises maintainable, scalable, and adaptable systems
• Dimensional and relational data modeling expertise
• Medallion architecture implementation (bronze→silver→gold)
• Experience with PostgreSQL and SQL Server, including data replication/integration patterns for migration
• Microsoft Azure data stack expertise: ADF, Delta Lake, ADLS Gen2, Synapse, Databricks (with flexibility to integrate non-Microsoft solutions)
• Proficiency in Python and SQL for ELT/ETL pipelines
• CDC patterns and incremental load strategies at scale
• Batch, streaming, and micro-batching architectures
• Experience with Delta Change Data Feed or equivalent real-time sync mechanisms
• Experience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) into a unified lakehouse
• Performance tuning and cost optimisation
• Event-driven architectures and API integration patterns
• Azure networking and security best practices
• CI/CD with Azure DevOps or GitHub Actions
• Git version control proficiency
• Infrastructure as Code (Terraform preferred)
• Monitoring and alerting with Azure Monitor and Log Analytics
• Test automation for data pipelines (unit, integration, regression)
• Container orchestration (Kubernetes/AKS) experience beneficial
• GDPR/data protection implementation, including the right-to-be-forgotten
• Strong understanding of information security requirements
• Data lineage and quality enforcement
• RBAC, encryption strategies, and audit logging
• Experience with Microsoft Purview or similar governance frameworks
• Experience creating architecture diagrams, data flows, and runbooks
• Strong technical writing – please provide examples
• Clear presentation skills for technical and non-technical audiences
• Comfort working with product managers and translating between technical and business contexts
• Track record of training and upskilling teams
• Experience with structured project planning: from high-level initiatives through quarterly planning to executable tasks- Technical documentation samples
• A description of a challenging data platform project you’ve delivered
• Your approach to implementing GDPR compliance in a lakehouse environment