Python API Developer

Python Developer (API Data Ingestion)

Role purpose: Build and run reliable Python-based data ingestion pipelines that pull data from internal/external APIs, validate and transform it, and land it into GCP with strong controls, monitoring, and documentation.

Key responsibilities: Design, develop, and maintain Python services/jobs to ingest data from REST/GraphQL APIs (and where relevant SOAP). Implement secure authentication patterns (OAuth2, JWT, API keys, mTLS) and manage secrets via approved vault tooling.

Build robust ingestion patterns: Pagination, rate limiting, retries with backoff, idempotency, and checkpointing Incremental loads (watermarks), CDC-like approaches where supported Validate and reconcile data (completeness, duplicates, referential checks) and produce quality metrics. Transform and standardise data (JSON/XML to tabular formats), including PII handling and masking where required. Orchestrate pipelines using approved schedulers (e.g., Airflow or equivalent) and implement CI/CD for deployments. Implement logging, monitoring, alerting, and runbooks to support production operations. Collaborate with API providers/consumers, platform teams, and stakeholders to agree SLAs, data contracts, and support models. Produce clear technical documentation (designs, data mappings, operational procedures).

Required skills & experience: Strong Python development experience (3+ years)

Solid understanding of API integration concepts: HTTP semantics, status codes, headers, timeouts Authentication/authorisation (OAuth2 flows, token refresh)

Experience with data engineering fundamentals: ETL/ELT patterns, incremental ingestion, partitioning Data formats (JSON, CSV, Parquet, Avro), compression SQL proficiency and experience landing data into relational or analytical stores.

Production mindset:

observability, error handling, performance tuning, and operational support.

Preferred / nice-to-have Cloud experience (GCP) and managed data services. Orchestration: Airflow or similar. Streaming/event ingestion (Kafka) where applicable. Containerisation (Docker) and Kubernetes basics.