Senior Software Engineer (Inference)

• Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide
• We bring Claude to life by serving our models via the industry’s largest compute-agnostic inference deployments
• We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators
• The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models
• We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms
• Designing intelligent routing algorithms that optimize request distribution across thousands of accelerators
• Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads
• Building production-grade deployment pipelines for releasing new models to millions of users
• Integrating new AI accelerator platforms to maintain our hardware-agnostic competitive advantage
• Contributing to new inference features (e.g., structured sampling, prompt caching)
• Supporting inference for new model architectures
• Analyzing observability data to tune performance based on real-world production workloads
• Managing multi-region deployments and geographic routing for global customers

Benefits

• Comprehensive health, dental, and vision insurance for you and your dependents
• Inclusive fertility benefits via Carrot Fertility
• 22 weeks of paid parental leave
• Flexible paid time off and absence policies
• Mental health support for you and your dependents
• Competitive salary and equity packages
• Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant
• Retirement plans with competitive matching
• Life and income protection plans
• $500/month flexible wellness and time saver stipend
• Commuter benefits
• Annual education stipend
• Home office stipends
• Relocation support for those moving for Anthropic
• Daily meals and snacks in the office- Kubernetes and cloud infrastructure (AWS, GCP)
• Load balancing, request routing, or traffic management systems
• Implementing and deploying machine learning systems at scale
• LLM inference optimization, batching, and caching strategies
• Python or Rust
• High-performance, large-scale distributed systems
• Are results-oriented, with a bias towards flexibility and impact
• Have significant software engineering experience, particularly with distributed systems
• Pick up slack, even if it goes outside your job description
• Want to learn more about machine learning systems and infrastructure
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs
• Care about the societal impacts of your work
• Education requirements: We require at least a Bachelor’s degree in a related field or equivalent experience- Deadline to apply: None. Applications will be reviewed on a rolling basis