Lead Data Engineer

Who you are

• You've owned large-scale, interdependent data systems in production, with deep Spark expertise: idiomatic, testable transformations tuned for cost and performance, built on Python, Airflow and Iceberg
• You've led engineers, setting direction, reviewing work, developing people, while staying hands-on
• You're hands-on with infrastructure: comfortable in AWS and Kubernetes, able to dig into logs and metrics to work out why a workload is failing or running slowly, and glad to pass that on
• You engage analysts and business stakeholders directly, on metric definitions, not just requirements, and turn ambiguity into a sequenced roadmap, managing dependencies across teams and saying what isn't getting done
• You're fluent with AI tooling in your own workflow, and you reason about what makes a codebase and its data legible to it
• Exchange, SSP or DSP experience: log-level auction and bidstream data at scale
• Measurement and attribution in AdTech: pixels, trackers, 3P measurement partners
• Migrating data systems: SQL to Spark, batch to streaming with Kafka or Redpanda
• Snowflake, where reporting aggregation still lives, plus Clickhouse or similar
• CI/CD with GitHub Actions/ArgoCD; monitoring with VictoriaMetrics/Prometheus/Grafana
• Building internal tooling or libraries that other engineering teams adopted
• Consolidating or migrating data systems following an acquisition

What the job involves

• Kargo is building toward a unified platform where advertisers run campaigns across CTV, web, mobile apps, and social entirely self-serve – the first time that capability goes directly into clients' hands
• It raises the bar on the pipelines behind it, which already process billions of events per hour and now need stability, speed, a consistent vocabulary across surfaces, and observability that catches problems before clients do (for engineers and AI agents alike)
• In this role you'll own that work across supply and demand, measurement, reporting, and the log processing pipelines underneath. You'll also set the technical bar for the domain's engineers
• Own business-metric reporting for data engineering : publisher performance, campaign delivery, revenue and spend, and attribution. Shape the definitions that make a cross-surface campaign read the same everywhere
• Own the domain's data sharing : the reports publishers and advertisers receive, log-level data for partners and clients, and our data sharing API
• Level up the event log processing pipelines at the heart of the domain : idiomatic Spark, refactored for testability, and a path to streaming as demand for faster insights grows
• Own and raise the bar on the domain's observability and alert response . Inventory today's signals, monitors and alerts, centralize them, and bring each to standard: a freshness and quality commitment, context for AI-assisted triage, and a runbook
• Lead and grow the domain's data engineers . Define the standards for testability, cost efficiency and the patterns worth repeating, then raise the team to them through your own code, reviews, and knowledge-sharing, recording decisions in ADRs