Software Engineer

Who you are

• Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
• 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
• Strong programming skills in Python, Go, or a similar language
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)
• Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)
• 5+ years of experience in software engineering, with a focus on infrastructure or machine learning systems
• Experience with open source inference engine like vLLM, Sglang, or TRT-LLM
• Contributions to open-source infrastructure or ML projects
• Experience in building large scale ML/MLOps infrastructure

What the job involves

• As a Software Engineer on our AI Infrastructure team, you will help design the core systems that power Fireworks AI’s generative AI platform
• You will help build infrastructure and tools that ensure the reliability, performance, quality, and availability of our AI system
• Our mission is to make Fireworks AI the most reliable and user friendly generative AI platform in the world
• You will partner closely with our cloud infrastructure team, product team, and performance team to deliver infrastructure that bridges the gap between our customer and the ultra-performant proprietary Fireworks inference engine
• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes
• Solve Hard Problems: Tackle challenges at the forefront of AI infrastructure, from low-latency inference to scalable model serving
• Build What’s Next: Work with bleeding-edge technology that impacts how businesses and developers harness AI globally
• Ownership & Impact: Join a fast-growing, passionate team where your work directly shapes the future of AI—no bureaucracy, just results
• Learn from the Best: Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation