Inference Performance Engineer

About the Company

We are an early-stage, still-in-stealth team building an AI platform that gives organizations control of their AI by delivering special models that are faster, more cost-efficient, and more secure than one-size-fits-all alternatives, and that customers can own and run on their own terms.

If you are passionate about high-scale infrastructure, hardware-software co-optimization, and shaping the foundation of the modern AI stack, we would love to talk.

Role Overview

As an Inference Performance Engineer , you will own and evolve our core execution platform and LLM serving infrastructure. You will design, deploy, and optimize highly available, resilient multi-cloud GPU and Kubernetes systems. Working closely with systems engineers and research teams, you will tackle bare-metal, network, and framework bottlenecks to keep models serving at maximum throughput and ultra-low latency.

Core Responsibilities

• Inference & Performance Optimization: Profile, benchmark, and optimize high-throughput, latency-sensitive LLM workloads like vLLM, smart routing layers, and KV-cache orchestration.
• Systems Architecture: Design, build, and operate resilient, k8s-native platforms and custom automation across multi-cloud environments.
• Low-Level & Hardware Efficiency: Locate and eliminate system-level bottlenecks to squeeze maximum performance out of compute and GPU resources.
• Developer Platform & Tooling: Build robust, self-service platform tools to enable rapid experimentation, deployment, and scalability.

Requirements

• 5+ years of software engineering, systems backend, or performance engineering experience.
• Deep expertise in Kubernetes managing production workloads including CRDs, Helm, Karpenter, affinity/anti-affinity, and custom scheduling.
• Strong background in Performance Engineering with proven experience profiling and squeezing efficiency out of high-throughput systems handling thousands of requests per second.
• Proficiency in Go or Python .
• Hands-on experience with major cloud platforms like AWS or GCP and production observability stacks like Prometheus and Grafana.

Preferred Qualifications

• Experience with GPU orchestration on Kubernetes using NVIDIA device plugins, DCGM, or GPU scheduling.
• Deep familiarity with LLM inference mechanics and procedures such as prefill/decode phases, TTFT, ITL, and custom engine tweaks.
• Experience with multi-cloud or hybrid-cloud architectures.