Research Intern (Frontier Speculative Decoding, Summer 2027)

• The Speculative Decoding team at Together AI builds the draft models and algorithms that let large language models generate faster without changing their outputs. Our speculators run in production on frontier open models such as DeepSeek, Kimi and GLM. Our work sits at the intersection of model training, inference algorithms and serving systems, and we co-design all three to lower the cost and latency of generation
• As a research intern, you will dive into open questions in speculative decoding: new drafter architectures and verification schemes, knowledge distillation and training recipes that align draft and target models, and drafters that adapt as traffic shifts. You will train and fine-tune draft models at scale on dedicated GPU clusters, and test your ideas against realistic serving workloads, including long-context and agentic traffic
• Projects aim to answer important research questions in speculative decoding, enabling faster generation on frontier models, drafters that keep pace with changing workloads, and robust, reproducible evaluation under real traffic
• Research and prototype new efficient speculative decoding algorithms and draft model architectures
• Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses
• Implement optimized version for serving engines to show efficacy in real world workloads
• Communicate the plans, progress, and results of projects to the broader team
• Share findings through technical write-ups and blog posts, and help bring successful ideas into production
• Our internship program runs 12 to 14 weeks, giving you the opportunity to work alongside industry-leading engineers and researchers across multiple teams. This cohort’s internship dates span May 17th to August 6th or June 14th to September 3rd

Benefits

• Competitive health insurance plans
• Pre-tax flexible spending accounts
• Mental health support and services
• Dental and vision insurance
• Income protection & retirement
• 401(k) plan
• AD&D insurance
• STD & LTD insurance
• Life insurance
• Flexible time off policy
• Monthly team lunches
• Team-driven celebrations and events
• Monthly commuting stipend + pre-tax bene- Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
• Strong knowledge of Machine Learning and Deep Learning fundamentals
• Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
• Currently pursuing a Master’s or Ph.D. degree in Computer Science, Electrical Engineering, or a related field
• Prior research experience in speculative decoding, efficient inference, or efficient machine learning
• Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)
• Experience with knowledge distillation, post-training, or reinforcement learning
• Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)
• Contributions to open-source machine learning projects