Data Engineering Consultant

Key Responsibilities

Pipeline Engineering: Design and deploy reliable batch, event-streaming, and CDC ingestion pipelines with automated patterns for schema evolution, replay-safety, quarantine, and reprocessing.

Streaming & Data Modeling: Build stateful/stateless transformations handling late events, watermarks, reattributions, contract amendments, and claim adjustments. Implement dimensional, Data Vault, lakehouse, or effective-dated/bitemporal models.

Data Quality & Controls: Implement automated control totals, reconciliation, and audit controls to guarantee completeness, validity, and line-of-business integrity across GL balances and data products.

Semantic & Platform Enablement: Deliver curated datasets optimized for Cube.dev, Power BI, and AI tools. Standardize business definitions (PMPM, IBNR, MCR, capitation). Enforce RBAC, encryption, masking, and PHI compliance.

Mandatory Qualifications

6+ years of data engineering experience with advanced SQL and Python, Scala, or Java.

Hands-on experience with Kafka and distributed processing engines (Spark, Flink, Beam).

Experience building batch/streaming pipelines on cloud platforms, lakehouses, or data warehouses.

Proficiency in dimensional modeling, data partitioning, incremental processing, and performance tuning.

Experience with orchestration, CI/CD, data-quality testing, production monitoring, and access controls.

Preferred Qualifications

Domain knowledge in healthcare claims, eligibility, capitation, sub-capitation, attribution, or value-based care.

Experience with financial accounting datasets (journals, subledgers, GL balances, reconciliations).

Familiarity with healthcare standards (X12 837/835, HL7, FHIR), dbt, Cube.dev, Power BI, and knowledge graphs/ontologies.