Senior Data Engineer | GCP | Kafka | BigQuery | Python

A leading FTSE 100 fintech is looking for a Senior Data Engineer to join its global Data Engineering function and contribute to a major transformation of its data platform.

The team is consolidating a complex data estate onto GCP , creating a scalable and trusted platform that supports analytics, compliance, Data Science and increasingly sophisticated AI/ML use cases .

This is a hands-on role suited to an experienced engineer who enjoys solving complex data problems and wants to have a meaningful influence on engineering standards and platform design.

The role

Working within a Data Engineering squad, you will design, build and operate production-grade data pipelines across both batch and real-time environments .

You will work across:

- GCP – BigQuery, Cloud Composer/Airflow, GCS and Cloud Run

- Apache Kafka – event-driven ingestion and real-time streaming

- Python and SQL – building reliable, production-grade data solutions

- dbt – transformation and data modelling

- Medallion architecture – Bronze, Silver and Gold layers

- Data quality and observability – contracts, lineage, monitoring and SLOs

Role responsibilities

• Designing and building scalable batch and real-time data pipelines

• Developing Kafka-based event streams and Airflow-orchestrated workflows

• Building transformations and data models in BigQuery

• Developing and maintaining ingestion integrations and connectors

• Implementing data quality controls, observability and pipeline monitoring

• Contributing to engineering standards, code reviews and CI/CD practices

• Supporting Data Science and AI-ready data initiatives

• Mentoring junior engineers and contributing to the development of the wider engineering team

Role Requirements

• Strong commercial experience building and operating production data pipelines on GCP

• Strong SQL and Python skills

• Solid experience with Apache Kafka or comparable streaming technologies

• Experience with BigQuery, Airflow/Cloud Composer and GCS

• Experience with dbt and modern data transformation approaches

• Understanding of Medallion architecture and data platform best practices

• Experience with data contracts, lineage, observability and pipeline SLOs

• Experience working within a structured or regulated environment

This is an opportunity to work on large-scale data engineering challenges within a global fintech , with the technical scope to make a real impact on the platform and how data is used across the organisation.