Senior Software Development Engineer in Test

Who you are

• Strong Java Skills: Strong technical fundamentals of Java, particularly in data structures and algorithms
• AI/LLM Validation Mindset: Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks is a big plus
• Agile Delivery at Pace: Proven track record of delivering automation solutions in a fast-paced CI/CD environment, supporting rapid deployment schedules without compromising quality
• Collaboration & Communication: Excellent communication skills, with the ability to work cross-functionally with product managers, engineers, AI/ML engineers, and data scientists
• Problem-Solving & Analytical Skills: Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement
• A software engineer with a strong coding background and understanding of delivering software at pace in Agentic PDLC
• Strong experience with white-box testing and fixing code is a huge plus
• Experience working with public cloud infrastructure (AWS/Azure/GCP) and/or foundational knowledge of the Salesforce platform (Salesforce Certifications are a plus)
• Prior experience testing autonomous agents, conversational UIs, or generative AI workflows

What the job involves

• We are infusing AI into every aspect of our software engineering. While AI is writing the code, validation is what needs to be done to meet enterprise quality
• We are looking for an enthusiastic Senior engineer who follows the AI-first approach to build and maintain validation of the code and ensure it is production-ready for our customers
• This engineer should be able to help the team transition from traditional testing to using AI in driving acceleration and accuracy in testing and be a multiplier using AI tools. Prior experience working with agentic products is a plus
• As a quality SMTS, you will play a key role in defining, implementing, and maintaining the quality strategy of the product
• You will collaborate with cross-functional teams, including product development and AI teams, to ensure that Salesforce’s AI-driven solutions meet the highest quality standards for enterprise applications
• Drive Innovation & Adoption: Design, build, and maintain scalable, AI-first testing frameworks and automation suites to validate complex workflows and agentic capabilities
• Execute Quality Strategies: Translate high-level quality strategies into actionable automation plans, focusing heavily on validating LLM integrations, prompt accuracy, and deterministic system behaviors
• Hands-on Code Quality & White-Boxing: Dig deep into the application code to understand system architecture, identify quality gaps, perform white-box testing, and actively contribute to fixing code and test flakiness
• Human-in-the-Loop & Model Evaluation: Implement and support evaluation processes for subjective quality dimensions of our AI features, including response relevance, user safety, and contextual accuracy
• Cross-Functional Collaboration: Partner closely with developers, product managers, and engineers to ensure quality is integrated early and continuously throughout the agile delivery lifecycle

Benefits

• Medical Care
• Life Insurance
• Retirement Savings
• Employee Assistance Programs
• With 9 standard holidays and four floating holidays, you get a total 13 paid days off each year