Senior Software Engineer (Together Cloud Infrastructure)

• As a Senior Software Engineer in Together Cloud Infrastructure, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware (GB200s/GB300s, BlueField DPUs)
• You’ll enable state-of-the-art ML practitioners with self-serve AI cloud
  services, such as on-demand + managed Kubernetes and Slurm clusters, for both our internal SaaS products (inference, fine-tuning, RL) and our external cloud customers, spanning dozens of data centers across the world.
• Design, build, and maintain performant, secure, and highly-available backend services/operators that run in our data centers and automate hardware management, such as Infiniband partitioning, in. DC parallel storage provisioning, and VM provisioning
• Design and build out the IaaS software layer for a new GB200 data center with thousands of GPUs
• Design and build distributed GPU scheduling and the global management plane that power on-demand and managed clusters across dozens of data centers
• Develop infrastructure that powers our internal inference, RL, and fine-tuning products in addition to external cloud customers
• Design and build systems that scale per-cluster capacity limits and automate the onboarding of new capacity
• Work on a global multi-exabyte high-performance object store, serving massive datasets for pretraining and model weights for large-scale inference
• Build advanced observability stacks for our customers with automated node lifecycle management for fault-tolerant distributed pretraining and large-scale inference
• Perform architecture and research work for decentralized AI workloads
• Work on the core, open-source Together AI platform
• Create services, tools, and developer documentation
• Create testing frameworks for robustness and fault-tolerance
• We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama

Benefits

• Competitive health insurance plans
• Income protection & retirement
• Mental health support and services
• Dental and vision insurance
• Pre-tax flexible spending accounts
• STD & LTD insurance
• 401(k) plan
• AD&D insurance
• Life insurance
• Team-driven celebrations and events
• Flexible time off policy
• Monthly commuting stipend + pre-tax bene
• Monthly team lunches- To be successful, you’ll need to be deeply technical and possess excellent communication, collaboration, and diplomacy skills
• You have strong fundamental software development skills. In addition, you have strong systems knowledge and troubleshooting abilities
• Strong systems knowledge across compute, networking, and storage, including concurrency, memory management, performant I/O, and scale
• 5+ years experience writing high-performance, well-tested, production quality code, and demonstrated ownership of large-scale projects driven end-to-end to completion
• Excellent communication skills – able to write clear design docs and work effectively with both technical and non-technical team members
• 5+ years of professional software development experience and proficiency in at least one backend programming language (Golang desired)
• Experience building and operating reliable, customer-facing production systems at scale with infrastructure automation tools (Terraform, Ansible), monitoring/observability stacks (Prometheus, Grafana), and CI/CD pipelines (GitHub Actions, ArgoC
• Demonstrated experience with building and operating high-performance and/or globally distributed micro-service architectures across one or more cloud providers (AWS, Azure, GCP)
• Deep experience with Kubernetes internals, such as implementing non-trivial Kubernetes operators, device/storage/network plugins, custom schedulers, or patches to Kubernetes itself
• Deep experience with VMs/hypervisors, such as QEMU/KVM, cloud-hypervisor, VFIO, virtio, PCIE passthrough, Kubevirt, SR-IOV
• Deep experience with DC networking tech + solutions, such as VLAN, VXLAN, VPN, VPC, OVS/OVN
• Experience with Cluster API or similar
• Experience working on high-performance compute, networking, and/or storage
• Experience virtualizing GPUs and/or Infiniband
• Experience building IaaS or PaaS systems at scale
• GPU programming, NCCL, CUDA knowledge
• Experience with DPUs/SmartNICs