Staff Software Engineer (Inference / Compute Infrastructure Engineering)

• We’re looking for a Software Engineer to build the systems that treat infrastructure as software
• This role owns the software state machines that provision hardware, bring it into service, and manage its full lifecycle — turning racks of GPUs into running inference clusters without a human touching a runbook
• The Research and Inference team is your customer: today they file tickets and wait; the target state is that they issue a single API call to stand up, scale, or tear down a cluster, and the system takes care of the rest
• The platform is manifest-driven such that teams declare the desired state of a cluster or host — shape, topology, software stack — and the system is responsible for reconciling reality to that manifest, continuously, through every stage of its lifecycle
• You will design the engines that manifest the schema, the engines that execute against it, and the workflows that carry a piece of hardware or a cluster from one state to the next—taking it from bare metal to a fully functioning AI cluster for training or inference
• You’ll write production code which is typed, tested, versioned, and deployed through CI/CD that models infrastructure state and reconciles it, the same way a Kubernetes controller reconciles a cluster’s desired state
• Success looks like eliminating manual provisioning work, not documenting it better
• A product mindset - you’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship
• You build it, you own it
• You are not only responsible for delivering the software but also for operating and supporting it in production
• Build the provisioning state machine: design and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA — as explicit, versioned states and transitions
• Build the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call — no ticket, no human in the loop
• Automate self-healing: detect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically
• Own reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service
• Partner with the inference/ML platform team: understand the cluster shapes they need — topology, interconnect, scheduling constraints — and encode them as first-class abstractions in the platform
• Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code — this is a product, not a collection of Ansible playbooks

Benefits

• Competitive health insurance plans
• Mental health support and services
• Dental and vision insurance
• Income protection & retirement
• Pre-tax flexible spending accounts
• Life insurance
• AD&D insurance
• STD & LTD insurance
• 401(k) plan
• Flexible time off policy
• Monthly commuting stipend + pre-tax bene
• Monthly team lunches
• Team-driven celebrations and events- Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts
• A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship
• Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines)
• Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution
• Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure
• Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE)
• Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization
• Systems programming in Rust or Go