E-IT

Senior Data Platform Engineer — Nexus Data Platform
About the team
What you'll do
<ul>
<li>Own pipeline slices end-to-end: data ingest into a distributed streaming platform, stream-processing transform jobs, and projection of clean, contract-compliant records into downstream stores.</li>
<li>Onboard new data domains and production entities — legacy schema mapping, streaming topic contracts, and stream-processing job development.</li>
<li>Build and run stream-processing jobs in the cloud — transforms along the change-data-capture → streaming → processing path and batch ingestion via managed big-data services — including checkpointing, failure recovery, and job-level runbooks.</li>
<li>Deploy and troubleshoot streaming and processing workloads on a container orchestration platform — job lifecycle and day-to-day application-level troubleshooting, in partnership with SRE on the underlying cluster.</li>
<li>Partner with downstream teams on data contracts: schema ownership, where consumers read from, and clear boundaries between the ETL pipeline and the teams that serve on top of it.</li>
</ul>
What we’re looking for
Required
<ul>
<li>5+ years experience building production grade
streaming pipeline
systems.</li>
<li>Hands-on experience with a distributed streaming/messaging platform (Kafka + Flink/Spark or similar): topic and partition design, consumer group semantics, compaction, and schema-aware serialization.</li>
<li>Production experience with a distributed stream-processing engine: SQL or DataStream-style jobs, state management, checkpointing, and operational debugging of long-running jobs.</li>
<li>CDC ingestion familiarity — WAL or log-based change capture into a streaming platform.</li>
<li>Demonstrated experience with SQL and schema design across relational sources.</li>
<li>Demonstrated experience deploying and troubleshooting streaming workloads on a container orchestration platform — basic troubleshooting of deployments and running jobs.</li>
<li>Track record shipping observable, testable pipeline components — health checks, smoke gates, benchmark-driven validation.</li>
<li>Ability to work independently on assigned roadmap items while coordinating with cross-functional partners on contract boundaries.</li>
</ul>
Preferred
<ul>
<li>Direct, hands-on experience with Apache Kafka and Apache Flink</li>
<li>Familiarity with Debezium CDC capture.</li>
<li>Experience projecting streaming data into an OLAP or analytical serving engine (StarRocks, ClickHouse, Druid, Pinot, or similar).</li>
<li>Awareness of Gov Cloud deployment constraints and data residency considerations.</li>
<li>Experience with multi-tenant SaaS data workloads (tenant partitioning, hot-partition skew).</li>
</ul>
How we work
<ul>
<li>AI-assisted development —
we use coding agents and automation to move fast; you should be comfortable reviewing and directing AI-generated changes, not writing every line by hand.</li>
<li>Local-first iteration —
every engineer has a local stack to develop and test pipeline changes before promoting to shared environments; schema and job work should be fast to try without waiting on central infrastructure.</li>
<li>Prove it before you ship —
automated smoke tests and performance benchmarks are part of how we validate changes, not a handoff to someone else at the end.</li>
<li>Cross-team by design —
data contracts between the pipeline and downstream teams are explicit; we coordinate early with external teams so downstream consumers stay aligned without slowing delivery.</li>
</ul>
What success looks like in the first 6 months
<ul>
<li>At least one production data domain onboarded through Kafka and Flink end-to-end, with automated validation passing in local and shared environments.</li>
<li>A partner team consuming pipeline output under a documented contract — clear access patterns and agreed SLA baselines.</li>
<li>Deploy and integration workflow dependable enough for routine use, not a manual heroics exercise each time.</li>
<li>Contributed runbooks, automation, or reusable patterns that make onboarding the next domain meaningfully faster.</li>
</ul>

Data Engineer with Apache Flink

Data Engineer with Apache Flink

This job is no longer available

Talents

Recruiters

About