GCP Data Engineer

We are looking for a Senior Data Engineer who can independently design, build, and operate production-grade ETL/ELT pipelines across a hybrid landscape of on-premises databases and cloud. Seeking a senior resource to own the technical delivery of Data platform which will be used to for Business KPI reporting .The resource will work closely with data architects, DataOps engineers, application teams, and business stakeholders to build scalable, secure, and high-performing data pipelines and analytical data models. This role requires someone proficient in Data Modelling/Profiling/Cleansing/de-duplication/ and Data Reconciliation as per the business rules.

Key Responsibilities

Lead full AI-assisted profiling of the data to precisely size duplicates, placeholder entries, and stale/conflicting records ahead of cleansing.

• Design and execute **probabilistic de-duplication** and name/address standardization across legacy and data.

• Implement unique-identifier anchoring across every record so all programs that operate from one trusted registry.

• Stand up the **ongoing data governance framework** that protects the cleansed registry going forward: stewardship roles, validation-at-entry rules, controlled reference data, change control, and data-quality KPIs.

• Define and instrument data quality KPIs/SLAs (completeness, duplication rate, freshness, identifier-match confidence) with dashboards visible to program governance and leadership.

• Ensure the API and underlying data model support conflict-safe synchronization for the offline-first intake application (store-and-forward sync from low-connectivity regions).

• Set engineering standards, review technical designs, and lead a small team of data engineers/analysts through an aggressive, AI-accelerated timeline.

• Act as the primary technical point of contact for the Deep-Dive Discovery: validating current state, running a data-profiling preview to size the cleansing effort, and locking down dependencies.

• Escalate and manage risk around known dependencies — schedule slippage, SME availability so proactive tracking and communication is essential.

• Support UAT, training, and hyper care for data-quality-related workflows; ensure data stewards are equipped to sustain governance rules post-go-live.

• Mentor Junior and mid-level data engineers

Mandatory Skills

• 4-6 years in data engineering working on large-scale Data cleansing/migration,Data reconciliation and Data quality projects.

• Hands-on data engineering experience building production ETL/ELT pipelines.

• Proficient in Python,Pyspark,PostgreSQL,SQL

• Experience with GCP Cloud(BigQuery and related tech like Cloud Storage, Cloudrun, Dataflow and dataform)

• Datawarehousing,Data Modelling,Data Normalization.

• Demonstrated, hands-on experience with large-scale data de-duplication and entity resolution (probabilistic/fuzzy matching, standardization of names/addresses, unique-identifier anchoring)

• Orchestration tools like Airflow/Dataflow

Nice to Have Skills

• Data governance and Data Lineage

• Good Understanding of API's design(RESTful/secure API layers over a system of record, versioning, documentation) suitable for multiple consumer channels (web portal, offline/mobile, future integrations).

• Understanding of PII/data protection, data masking

• Excellent stakeholder communication skills — ability to present progress, risk, and data quality metrics to leadership and program governance.

• Experience with other clouds like AWS,Azure and modern data warehouses like Databricks/Snowflake.

• CI/CD, github

• Big Data technologies Hadoop, HDFS, Hive, Apache Spark