Staff Machine Learning Engineer (VLM/LLM Evaluation)

• The mission of the Waymo AI Foundations team is to develop machine learning solutions addressing open problems in autonomous driving, towards the goal of safely operating Waymo vehicles in dozens of cities and under all driving conditions
• As part of our work, we also initiate and foster collaborations with other research teams in Alphabet. AI Foundations areas that we are currently focusing on include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation
• This role follows a hybrid work schedule and you will report to a Senior Staff Software Engineer
• Work with a creative team of people who help to build the state-of-the-art Foundation Models that are used throughout Waymo’s systems, both onboard autonomous vehicles and offboard in simulation
• Lead the development of end-to-end evaluation systems and benchmarks for Waymo Foundation models, encompassing the entire lifecycle from pretraining and supervised fine-tuning (SFT) to reinforcement learning (RL), for evaluating the quality, safety, and realism of embodied AI agents
• Partner within and across organizations to land disruptive and innovative tech in production
• Implement and extend large large scale data and evaluation pipelines

Benefits

• Medical, dental, and vision insurance for employees and dependents
• Employee assistance programs focused on mental health
• Personalized workplace adjustments for diverse needs and abilities, including physical, mental, and neurodivergent considerations
• Access to mental health apps
• Onsite wellness centers
• Support programs including menopause benefit
• Second medical opinion for you and your loved ones
• Counseling services
• Medical advocacy program for transgender employees
• Competitive compensation
• Regular bonus and equity performance grant opportunities
• Generous 401(k) and regional retirement plans
• 1-on-1 financial coaching
• Annual cross-company compensation review and pay equity analysis
• Fertility and growing family assistance
• Parental leave and baby bonding leave
• Backup childcare
• Elder care and support
• Survivor income benefit
• Caregiver leave
• Paid time off, including vacation, bereavement, sick leave, parental leave, disability, and holidays
• Jury duty leave
• Military leave
• Hybrid work model with remote work opportunities also available
• Educational reimbursement
• Peer learning and coaching platform
• Donation matching programs
• Volunteer hours
• Employee resource groups
• Internal community groups and local culture clubs
• Inspiring spaces to work, recharge, and collaborate with fellow Waymonauts
• On-site meals and snacks
• Fitness centers, massage programs, and ergonomic support
• On-demand fitness, wellbeing, and cooking classes
• Commuter benefits- Master’s degree or PhD degree in Computer Science, similar technical field of study, or equivalent practical experience
• Strong analytical and debugging skills
• 5+ years of experience in ML engineering and applied Deep Learning, with a strong portfolio of shipped products or publication record
• Proficient programming skills (eg: Python, C/C++)
• Experience with large scale distributed system
• ML infra experience: training, evaluating and deploying ML models at scale
• Deep learning experience, especially with generative models, e.g., LLMs/VLMs, and/or reinforcement learning
• Proficiency and in-depth knowledge of the inner workings of an ML framework (e.g. Pytorch, JAX, Tensorflow)