Infrastructure System Engineer

Berkley Hunt has partnered with a well-funded, cutting-edge startup building next-gen infrastructure to power high-performance, low-latency LLM inference at scale. This team is rethinking how to optimize global server utilization and request routing for AI workloads, enabling real-time execution across edge locations and novel hardware.

This is a founding-level opportunity for an infrastructure engineer who thrives in low-level systems, distributed environments, and mission-critical performance challenges.

Infrastructure System Engineer

Role Responsibilities

• Architect and implement global routing engines that intelligently distribute LLM inference requests with microsecond decision times.
• Build and manage high-performance backend infrastructure from scratch, focusing on concurrency, throughput, and resilience.
• Own the control plane, monitoring systems, and production documentation for all deployments.
• Lead efforts in internal security and access management systems.
• Design and optimize infrastructure for edge-based inference with maximum hardware utilization and minimal latency.
• Collaborate cross-functionally to drive infrastructure decisions from prototype to production.

Requirements

• Extensive experience building distributed, concurrent systems optimized for speed and reliability.
• Deep expertise in systems programming (Rust, C++, or Zig preferred).
• Strong Python skills for automation and control plane development.
• Proven experience designing infrastructure for LLM inference or high-scale CDN environments.
• Proficiency with AWS, Postgres, Redis, and event streaming tools like Kafka.
• Comfort with Linux internals, performance tuning, and kernel-level debugging.
• Familiarity with observability tools such as Zipkin or Jaeger.
• Clear, concise communicator with a strong documentation and operational mindset.