Data Engineer

Key responsibilities

• Build and maintain batch and/or streaming data pipelines end-to-end (ingest, transform, validate, publish)
• Develop reusable data transformation patterns and curated datasets for analytics and operational use cases
• Work from requirements through to production delivery: design, build, test, deploy, and support
• Implement data quality checks, reconciliations, and monitoring/alerting to keep data trustworthy
• Optimize performance and cost (query tuning, partitioning, efficient compute usage)
• Maintain clear documentation, data definitions, and runbooks for supported pipelines
• Troubleshoot production issues, perform root cause analysis, and implement permanent fixes
• Collaborate with upstream/downstream teams to resolve data issues and improve interfaces/contracts
• Follow security and governance expectations for sensitive data (access controls, auditability)

Required skills and experience

• Minimum 5 years of strong hands-on experience delivering data engineering solutions
• Strong programming skills in Python and/or Java experience, with advanced SQL and solid data modelling skills
• Experience with distributed processing (e.g. Spark) and orchestration (e.g. Airflow or equivalent)
• Experience with streaming/event platforms (e.g. Kafka/PubSub) is beneficial; ability to learn quickly if not
• Strong engineering discipline: Git, code reviews, automated testing, CI/CD, and observability
• Experience working with cloud data platforms (Azure/AWS/GCP) and lake/lakehouse/warehouse patterns
• Proven ability to manage multiple data requests, priorities effectively, and deliver at pace