Senior Software Engineer (Managed Kubernetes)

• This position requires presence in our San Francisco, San Jose, or Bellevue office location 4 days per week; Lambda’s designated work from home day is currently Tuesday
• We are seeking a Senior Software Engineer to join our Managed Kubernetes (Mk8s) team
• You will play a crucial role in shaping the architecture, reliability, and automation of our Kubernetes-based infrastructure, which powers mission-critical workloads across our global platform
• Lambda is building the AI Cloud of the future
• We are seeking a Senior Software Engineer to help our development of our Managed Kubernetes platform
• Think GKE, but purpose-built for AI workloads and running on bare metal
• In this role, you will help build the infrastructure that powers the next generation of AI training and inference at scale
• As a Senior Engineer on our Orchestration team, you will contribute to Lambda’s managed orchestration services, including Managed Kubernetes, Managed Slurm on Kubernetes, and higher-level platform services for inference and AIOps
• You’ll work at the intersection of distributed systems, GPU-accelerated computing, and Cloud Native infrastructure to build systems that are reliable, performant, and elegantly simple for our customers
• This is not a role for someone who just operates Kubernetes; it’s a role for an engineer who understands how compute, network, storage, and security interact, and can build solutions that account for that context — even while focused primarily on the orchestration layer
• You’ll be working closely with NVIDIA’s open-source ecosystem, and partnering with internal teams across the stack to deliver a world-class managed platform
• Design, build, and maintain scalable control plane services, operators, and custom Kubernetes controllers; develop automation in Go/Python for end-to-end cluster lifecycle management — provisioning, upgrades, patching, and deletion
• Build GPU-aware orchestration systems, working within the platform architecture to support GPU scheduling and resource allocation
• Partner with the Network team on networking solutions for AI workloads: CNI integration (Cilium, Multus), high-performance fabrics (InfiniBand, RoCE), RDMA, and GPUDirect
• Write resilient systems that handle failure gracefully — timeouts, retries, backoff, and degraded-mode operation — across large-scale distributed environments
• Develop platform services for inference: model serving infrastructure, autoscaling based on inference load, and multi-model deployment patterns
• Build internal tools and CLIs that let ML/AI teams deploy and monitor their own inference services
• Support and debug production issues through on-call rotation

Benefits

• Health, dental vision
• 401k match
• 5 sick days and 12 paid holidays
• Flexible PTO
• Paid parental, medical, and caregiver leave- Experience with observability at scale: Prometheus, Grafana, distributed tracing, and building actionable alerting systems
• Solid grasp of distributed systems fundamentals — fault tolerance, graceful degradation, and failure handling in large-scale environments
• Strong programming skills in Go and Python; ability to collaborate effectively on shared codebases
• Experience operating the control plane and low-level pieces of large-scale Kubernetes clusters
• Take pride in owning and delivering core components of products and platforms
• Solid knowledge of Linux systems, networking, containers, and cloud infrastructure
• Deep understanding of Kubernetes internals: controllers, schedulers, operators, CRDs, CSI, CNI, and the extension patterns that make Kubernetes powerful
• If you don’t meet all of these requirements but believe you may be a good fit, please still apply and provide a cover letter that helps us understand your experience and readiness for this role
• Have 6+ years of experience in software engineering, with a track record of owning significant technical scope within a team (e.g., driving a project from design through production, or acting as a de facto tech lead on a workstream)
• Experience building and operating managed Kubernetes services (GKE, EKS, AKS, or similar) or working on Kubernetes control plane components
• Past contributions to CNCF projects or Kubernetes SIGs a plus
• Exposure to storage architecture for AI/ML workloads
• Familiarity with GPU, InfiniBand, RDMA, or high-performance computing on Kubernetes
• Familiarity with HPC and traditional job schedulers (Slurm) and Kubernetes-native batch scheduling (KAI, Volcano, Kueue)
• Hands-on experience with NVIDIA’s GPU/networking ecosystem: GPU Operator, device plugins, DCGM, MIG, Network Operator, NCCL tuning, or similar