Gcp Data Engineer (Lead)

Role & responsibilities

• Implement data quality checks, schema validation, reconciliation logic, anomaly detection, lineage hooks, metadata capture, and operational monitoring.
• Partner with platform architects to optimize BigQuery workloads, partitioning, clustering, materialized views, storage patterns, and query performance.
• Support migration engineering, historical loads, incremental cutover, parallel run validation, defect remediation, and production support transition.
• Lead code reviews, engineering standards, development accelerators, reusable templates, CI/CD integration, and team mentoring.
• Knowledge of CDC and replication tools such as Datastream, Database Migration Service, Striim, Qlik Replicate, Debezium, Informatica, Talend, or equivalent tools.
• Experience with data quality, reconciliation, observability, and testing tools such as Great Expectations, Soda, dbt tests, Deequ, Monte Carlo, Datafold, or custom SQL/Python frameworks.
• Understanding of CI/CD, Git, Terraform, Cloud Build, Jenkins/GitHub Actions, release management, and environment promotion practices.
• Awareness of AI-assisted engineering patterns using Vertex AI, GenAI coding assistants, metadata generation, test generation, anomaly detection, and documentation acceleration under human review.