Data Engineer

Role Overview

This role designs, builds and deploys a scalable, reliable data lakehouse platform on AWS. The Data Engineer will architect end-to-end data pipelines, establish data quality frameworks, and deliver production-ready solutions that form the foundation of the organisation's data infrastructure.

Key Responsibilities

Data Pipeline & Architecture Design

• Design and implement scalable ETL/ELT pipelines using AWS services including Glue, Step Functions, Lambda and S3.
• Architect and build data lakehouse solutions leveraging Apache Iceberg or S3 tables — incorporating schema evolution, partition evolution and ACID transactions.
• Optimise pipelines for performance, cost and reliability at scale.

Data Quality & Governance

• Define, implement and maintain automated data quality validation frameworks, with metrics and monitoring to ensure data accuracy, completeness and consistency.
• Enforce data governance standards and ensure compliance across the data platform.

Application Development & Deployment

• Write production-quality code and deploy solutions on AWS cloud infrastructure.
• Apply DevOps practices — CI/CD pipelines and infrastructure-as-code tools such as Terraform or CloudFormation — for repeatable, auditable deployments.

Collaboration & Handover

• Work closely with cross-functional teams and communicate technical designs clearly to non-technical stakeholders.
• Produce thorough technical documentation to support the handover of solutions to the Day 2 operations team.

Qualifications & Experience

• Degree in Computer Science, Data Engineering, Information Systems or a related field.
• 3–5 years of experience in data engineering, ETL/ELT development or data platform roles.
• Strong hands-on experience with AWS services, and proficiency in Apache Iceberg or similar open table formats.
• Experience in a Government Commercial Cloud (GCC) environment is strongly preferred.
• AWS certifications such as AWS Certified Data Analytics or AWS Certified Solutions Architect are an advantage.

Skills & Competencies

• Data Engineering & Architecture — design and build scalable, reliable data pipelines using modern data lakehouse architectures including Apache Iceberg and S3 tables.
• Data Quality & Governance — define, measure and monitor data quality metrics, implement automated validation frameworks, and ensure compliance with governance standards.
• Application Development — write production-quality code, optimise performance, and deploy solutions on cloud platforms.
• Technical Depth — deep understanding of Apache Iceberg features and S3 table optimisation techniques.
• Collaboration & Communication — work effectively in cross-functional teams and communicate technical concepts clearly to non-technical stakeholders.

Pay: $7,000.00 - $9,000.00 per month

Work Location: In person