Senior Development Engineer in Test

Who you are

• The ideal candidate has a strong engineering foundation, a passion for AI/ML-powered products, and the drive to be a multiplier — helping the team accelerate velocity and accuracy through modern AI tooling and practices. Prior experience on a Data Science, ML Science, or ML Engineering team is a plus
• Strong Java Skills: Strong technical fundamentals of Java, particularly in data structures and algorithms
• AI/LLM Validation Mindset: Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks is a big plus
• Agile Delivery at Pace: Proven track record of delivering automation solutions in a fast-paced CI/CD environment, supporting rapid deployment schedules without compromising quality
• Collaboration & Communication: Excellent communication skills, with the ability to work cross-functionally with product managers, engineers, AI/ML engineers, and data scientists
• Problem-Solving & Analytical Skills: Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement
• A software engineer with a strong coding background and understanding of delivering software at pace in Agentic PDLC
• Strong experience with white-box testing and fixing code is a huge plus
• Experience working with public cloud infrastructure (AWS/Azure/GCP) and/or foundational knowledge of the Salesforce platform (Salesforce Certifications are a plus)
• Prior experience testing autonomous agents, conversational UIs, or generative AI workflows

What the job involves

• We are looking for an enthusiastic Senior Engineer who takes an AI-first approach to help build and evolve our Sales Agent and Forecasting capabilities within the Revenue Action Orchestration (RAO) organization
• This individual will work on cutting-edge AI-powered products that tackle one of the most critical problems for enterprise sales teams — accurate revenue forecasting and intelligent sales automation. You will play a key role in designing, implementing, and maintaining robust, scalable systems that bring AI-driven insights directly into the hands of sales teams worldwide
• As a member of the RAO team, you will collaborate with cross-functional partners across product development, data science, and AI teams to deliver Salesforce's AI-driven Sales Agent and Forecasting solutions to the highest quality standards for enterprise applications
• Drive Innovation & Adoption: Design, build, and maintain scalable, AI-first testing frameworks and automation suites to validate complex RAO workflows and agentic capabilities
• Execute Quality Strategies: Translate high-level quality strategies into actionable automation plans, focusing heavily on validating LLM integrations, prompt accuracy, and deterministic system behaviors
• Hands-on Code Quality & White-Boxing: Dig deep into the application code to understand system architecture, identify quality gaps, perform white-box testing, and actively contribute to fixing code and test flakiness
• Human-in-the-Loop & Model Evaluation: Implement and support evaluation processes for subjective quality dimensions of our AI features, including response relevance, user safety, and contextual accuracy
• Cross-Functional Collaboration: Partner closely with developers, product managers, and engineers within the RAO ecosystem to ensure quality is integrated early and continuously throughout the agile delivery lifecycle

Benefits

• Medical Care
• Life Insurance
• Retirement Savings
• Employee Assistance Programs
• With 9 standard holidays and four floating holidays, you get a total 13 paid days off each year