Senior AI Runtime Engineer

• ML developers today face significant friction when deploying trained models
• They work in a fragmented space with incomplete, patchwork solutions that require extensive performance tuning and model-specific optimizations
• At Modular, we are building the next-generation AI platform that will radically improve how developers build and deploy AI models
• A core part of this offering is a platform that enables customers to achieve state-of-the-art performance across model families and frameworks
• As an AI Runtime Engineer, you will own a runtime that operates on various CPU and GPU hardware platforms, optimizing performance for diverse customer AI models
• Design and develop runtime and cross-stack optimizations to improve CPU and GPU efficiency, addressing issues such as CPU overhead, caching, and data locality across multiple devices
• Port the Modular runtime stack to new hardware platforms and develop an API to streamline this process
• Collaborate with the compiler, kernels, serving, and models teams to design core technologies that achieve state-of-the-art end-to-end performance on various CPU and GPU hardware
• Collaborate with the customer success team and engage with customers to understand their performance requirements and use cases
• Collaborate with tooling and infrastructure teams to design systems for automated performance analysis and benchmarking

Benefits

• A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
• A 401k plan with up to 5% match
• Free tax advice on Carta
• Generous work-from-home stipend of $1500 to help you improve your home office
• Unlimited paid time off and flexible work hours- Experience in C++ programming and complex software systems
• 5+ years of experience working on high-performance computing systems
• Proficiency with one or more profiling tools (CPU or GPU)
• Experience with CPU or GPU runtime optimizations and performance analysis on CPUs, GPUs, or AI accelerators
• Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
• For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply as we may have upcoming openings that are lower/higher level than the ones advertised
• Advanced degree in Computer Science or a related area is a plus
• Exposure to MLIR, LLVM, and/or the Mojo programming language
• Experience with ML graph optimizations, parallel / distributed programming, heterogeneous ML computation, and/or code generation