Data Engineer – Event-Driven & Analytics Platform

Things to know before applying:

• You must have your own work equipment
• You are able to work remotely
• You can work full-time for this role under independent contractor arrangement
• Experience working with clients/companies outside PH is an advantage
• Only candidates who meet the requirements will be contacted
• You can start ASAP

Job Description:

We are seeking an experienced Data Engineer to join our Data team, with a specialized focus on building robust, scalable, event-driven data pipelines and analytics-ready data models.

In this role, you will design and maintain end-to-end ingestion frameworks that capture real-time and near-real-time business events across our global payment platforms. You will partner with upstream engineering teams to standardize event contracts, leverage dbt and Google BigQuery to transform raw event streams into governed dimensional models, and empower downstream analytics, Looker BI dashboards, and regulatory reporting.

Key Responsibilities:

Event Streaming & Ingestion Pipelines

• Design, build, and operate resilient real-time and micro-batch ingestion pipelines from event brokers, message queues, and Change Data Capture (CDC) streams (e.g., Kafka, AWS SNS/SQS, DynamoDB, Cloud Storage) into Google Cloud Platform (GCP).
• Collaborate with software engineering teams to define and govern event schemas, contracts, and evolution strategies (JSON Schema / Avro / Protobuf).
• Build robust error-handling, dead-letter queuing (DLQ), idempotency, and deduplication mechanisms for streaming datasets.

Data Modelling & Transformation (dbt)

• Develop, test, and maintain modular transformation layers using dbt and SQL, converting raw/bronze events into clean, dimensional, and domain-driven models (Silver/Gold layers).
• Implement automated data quality checks, schema validation, and unit testing within the CI/CD pipeline using dbt tests and data observability tools.
• Document data models, line-of-business definitions, and lineage in dbt and our enterprise data catalog (Alation).

BigQuery Optimization & Cloud Architecture

• Manage table partitioning, clustering, materialization strategies, and caching in BigQuery to maximize query performance while controlling compute costs.
• Orchestrate multi-stage data pipelines using workflow tools (e.g., Cloud Composer/Airflow, Cloud Functions, Cloud Run).
• Implement and uphold infrastructure-as-code (Terraform) and CI/CD deployment standards (GitHub Actions).

Data Governance, Security & Quality

• Implement data governance best practices, including role-based access controls (RBAC), column-level security/masking for PII, and regulatory compliance (e.g., audit readiness, retention policies).
• Partner with Data Architects, Product Analytics, and Finance/Compliance stakeholders to ensure reconciled, accurate metrics (e.g., transactional data tapes, active user metrics).

Qualifications:

Event-Driven & Ingestion Systems

• Proven experience designing streaming or near-real-time ingestion pipelines (e.g., Kafka, AWS Kinesis/SNS/SQS, Pub/Sub, or event-driven cloud functions).
• Strong understanding of event architectures, schema management, CDC patterns, and JSON/semi-structured data parsing.

Data Transformation & Modelling

• Advanced SQL skills and extensive hands-on experience using dbt in production environments.
• Strong grounding in dimensional data modelling concepts (Kimball, Medallion architecture, star schemas).

Cloud Data Warehouse (GCP / BigQuery)

• Deep expertise in Google BigQuery (partitioning, clustering, nested/repeated fields, query execution plans, slot and cost optimization).
• Hands-on experience across the broader GCP data ecosystem (GCS, Cloud Functions, Cloud Composer/Airflow, IAM).

Software Engineering & DevOps

• Proficiency in Python or SQL for data processing and pipeline development.
• Experience with version control (Git/GitHub), automated testing, CI/CD pipelines, and Infrastructure as Code (Terraform).

Nice to Have

• Experience with multi-cloud data flows (AWS S3/DynamoDB to GCP BigQuery).
• Exposure to modern BI tools (Looker / LookML) and Data Catalogs (Alation).
• Domain experience in FinTech, payments, foreign exchange, or financial services.

Tech Stack requirements:

• Event Streaming & Messaging: Kafka, AWS Kinesis/SNS/SQS, Pub/Sub, Change Data Capture (CDC)
• Event Schema Formats: JSON Schema, Avro, Protobuf
• Cloud Data Warehouse: Google BigQuery
• Google Cloud Platform (GCP): Cloud Storage (GCS), Cloud Functions, Cloud Run, Cloud Composer/Airflow, IAM
• AWS: S3, DynamoDB
• Data Transformation & Modelling: dbt, SQL
• Programming Languages: Python, SQL
• DevOps & Infrastructure: Git/GitHub, GitHub Actions, Terraform
• BI & Data Catalog: Looker / LookML, Alation

Hiring Process:

• Phone screening
• Technical Test
• Behavioral Interview
• Client interview

About Adaca:

Adaca was founded in 2015 to assist Australian companies in their software development initiatives. Over the past 10 years, we have successfully built and scaled a distributed team across Australia and the Philippines to over 100 staff. Now, we are entering a whole new era of artificial intelligence. Our authentic, values-led approach has allowed us to attract and retain magnificent people. Adaca operates a high-performance core team with a network of customer-facing people who have catalysed our growth. This emphasises the value that each individual holds in influencing the growth of the business.