Manager - Data Engineering (TPA)

Role Purpose:

To ensure the organization has reliable, well-governed, and analytics-ready data by building and maintaining robust data pipelines and models. The role exists to make trusted data available securely and cost-effectively to support enterprise reporting, BI, and data-driven decision-making.

Key Responsibilities:

1- Enterprise Data Ingestion and Data Engineering;

• Build reusable, parameterized mappings and task flows in Informatica IDMC to standardize data ingestion.
• Implement change data capture (CDC), idempotent loads, schema evolution, and data-quality gates.
• Optimize Big Query loads using partitioning, clustering, and the appropriate load-versus-stream approach.
• Set up version control and CI/CD pipelines for data engineering assets.

2- Data Mapping & Transformation Design;

• Profile source systems and define field-level mappings and transformation rules.
• Specify business logic, including joins, lookups, and derivations.
• Define data-quality rules, exception handling, and reject criteria.

3- Orchestration, Automation & Reliability Engineering;

• Parameterize task flows and configure schedules and dependencies.
• Implement retries, backoff, and checkpointing to ensure reliable processing.
• Integrate monitoring and alerting through the Ops console and ChatOps.

4- Data Roles and Privacy management;

• Define least-privilege IAM roles and service accounts for data access.
• Apply dataset, table, row, and column-level security and data masking.
• Enable audit logging and retention policies, and classify PHI/PII data.

5- Modern Data Architecture & Data Modelling;

• Implement Medallion Architecture across the Bronze, Silver, and Gold layers.
• Develop scalable, curated data models for analytical and reporting needs.
• Define data quality, lineage, and governance standards for curated data.
• Collaborate with business and analytics teams to create trusted datasets.

6- Workload Monitoring, Performance & Cost Optimisation;

• Track data freshness, job health, volumes, and anomalies.
• Monitor SLAs across job duration, errors, cost per TB, and slot usage.
• Tune BigQuery performance through query optimisation and resource management.
• Optimize cost through storage lifecycle management, query tuning, and caching.

Skills

• Data Engineering Tools: Informatica IDMC, Airflow, Apache Spark.
• Programming Languages: SQL, Python.
• Cloud Technologies: Google Cloud Platform (BigQuery, IAM), Azure/AWS (optional).
• Visualization & BI Platforms: Power BI, Looker, Looker Studio, Tableau.
• Strong in BigQuery (SQL, modelling, tuning).
• Proficient with Informatica IDMC/IICS ETL/ELT.
• Experience with end-to-end enterprise data pipelines using GCP and Informatica IDMC.
• Knowledge of Medallion Architecture.
• Experience building reporting-ready data marts and BI datasets.
• Experience supporting enterprise BI dashboards and self-service analytics is preferred.
• Knowledge of healthcare data (claims, providers, members).
• Understanding of cloud security (IAM, service accounts, private endpoints).

Education

Bachelor’s degree in Computer Science, Information Technology, Information Systems, or a related