Senior Data Engineer

Who you are

• Bachelor's degree at a minimum
• 5+ years building production data systems
• Strong SQL and data modeling fundamentals, including grain, keys, history, dimensional modeling, and the tradeoffs between different representations of the same data
• Experience designing and operating batch and asynchronous data pipelines, including the failure modes that appear in distributed systems
• Experience with a cloud data platform and an understanding of storage, query performance, and cost
• Strong instincts around data correctness, including validation, idempotency, late and duplicated records, schema changes, backfills, and reprocessing
• Working proficiency in Python for pipelines, services, tooling, and tests
• The ability to change production systems in ordered, reversible steps without breaking their consumers
• Clear written communication, including documenting technical decisions and explaining changes to people outside of engineering
• You do not need previous healthcare experience or experience with every technology we use

What the job involves

• Vida is looking for a Senior Data Engineer to help shape the data platform that supports much of Vida's business
• Our team builds and operates the systems that ingest, validate, transform, and serve data across Vida
• We work with data from health plans, employers, clinical programs, and internal systems across member identity, eligibility, claims, billing, reporting, and analytics
• You will own data systems and domains end to end, from ingestion and validation through storage, transformation, and delivery
• You will make architecture decisions, change production systems safely, and help other engineers make good decisions about data infrastructure
• Own production data systems and domains from ingestion through delivery
• Design ingestion patterns that handle incomplete, duplicated, delayed, malformed, and changing partner data
• Define the contracts between source systems, ingestion, transformation, and downstream consumers
• Design data models with clear grain, keys, history, and definitions so the same question doesn't get two different answers
• Evolve schemas, pipelines, and datasets safely, with retries, backfills, and replay that don't corrupt or duplicate data
• Improve pipeline performance, observability, and failure handling so you can tell when something is wrong and why
• Work with Engineering, Product, Operations, and Analytics to figure out what the data needs to represent when requirements are incomplete or messy
• Additional responsibilities as needed