Staff Software Engineer

Who you are

• Proven track record leading platform migration initiatives (v1→v2 architecture transitions) while maintaining backward compatibility and system availability
• Strong command of streaming and queuing infrastructure (Kafka or equivalent), including partitioning strategy, consumer group semantics, redelivery, and idempotency
• Deep industry experience in distributed systems and data pipeline engineering, with demonstrated experience designing and operating systems at enterprise scale (terabyte-scale ingestion, high-throughput event processing)
• Fluency in both relational and document data modeling at scale (Postgres and MongoDB), including query planning, connection pooling, and online schema change on large, continuously written tables
• Experience collaborating with engineering leadership to define multi-quarter technical roadmaps and drive them through to delivery
• Solid AWS fundamentals and comfort operating in TypeScript and Node
• Able to articulate complex systems tradeoffs to both technical and non-technical audiences and bring cross-functional partners along on long-horizon technical decisions
• Open to using AI to amplify your skills and strengthen your work - demonstrating curiosity, a willingness to learn, and sound judgment in applying AI responsibly to improve efficiency and impact

What the job involves

• As a Staff Software Engineer on the Automation - Foundations team, you will lead the re-platforming of the data layer underneath Vanta’s entire compliance product
• This is a migration that spans multiple teams, has to preserve every public API contract along the way, and cannot lose a single customer’s evidence while it happens
• We are in the middle of moving our platform from a Mongo-centric architecture to a schema-aware, Postgres-backed architecture, on a Kafka and S3 pipeline that decouples data fetching from processing
• Both pipelines run in parallel today, the hard problems here are correctness under migration, eventual consistency, and multi-tenancy, in a domain where “mostly right” is not an acceptable failure mode
• Lead the migration of Vanta’s resource data model from a Mongo-centric solution to a schema-aware Postgres-backed solution and running both generations in parallel without breaking a customer integration
• Drive solutions across teams that you do not own but are dependent on the platform built by your team
• Design for correctness under eventual consistency with idempotent session handling, conditional writes that survive out-of-order delivery, reconciliation rather than strict cross-service validation, and explicitly backpressure instead of implicit database limits
• Design and evolve Vanta's data ingestion and pipeline architecture, ensuring reliable, high-throughput processing of terabyte-scale data streams across distributed systems
• Take our Query API from internal use to a production platform with schema versioning, joins and exports, per-tenant isolation, and predictable latency under load
• Build data infrastructure where correctness is externally auditable as the evidence Vanta produces has to hold up in front of an auditor
• Diagnose the failures that only appear at production data shapes and can include hot partitions, unbounded fan-out, and online rewrites of continuously-written tables
• Set architectural direction for streaming infrastructure (Kafka and event queuing), caching layers (Redis), and persistent storage (Postgres, MongoDB) across the Foundations stack
• Mentor senior engineers across the team, raising the technical bar through design reviews, architectural guidance, and hands-on contributions to the most complex systems challenges
• Champion engineering excellence in reliability, observability, and operational hygiene across the data infrastructure that powers Vanta's core product
• Leverage AI tools and systems to accelerate your own work and explore how they can improve the systems you build

Benefits

• Health & Wellness: Vanta values the health and well-being of our teammates and offers comprehensive medical, dental, and vision coverage for teammates and dependents. Vanta is excited to provide an additional range of benefits that holistically support your physical, emotional, and financial wellbeing.
• Work Your Way: Enjoy a remote-first setup if you're in the US, or go hybrid at our international offices
• Paid Parental Leave: Industry competitive paid parental leave for all parents
• Time Away: Industry competitive and generous PTO policy
• 401K Plan: 401k matching contribution plan