Software Quality Engineer

Who you are

• Are you passionate about ensuring the highest quality for cutting-edge generative AI applications?
• 5+ years of hands-on experience in software quality assurance or engineering, with a strong focus on testing complex distributed systems or AI/ML applications
• 5+ years of experience working with GitHub actions
• Proficiency in programming languages like Python or Typescript for test automation, and experience with modern testing frameworks such as Playwright
• Solid understanding of AI/ML concepts, including model evaluation metrics, data pipelines, and the unique challenges of testing generative AI outputs
• Experience with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) in a CI/CD environment is a big plus
• Exceptional analytical skills to dissect complex problems and a keen eye for detail, ensuring no bug goes unnoticed
• A collaborative spirit that helps you Connect cross-functional teams to common quality goals, Challenge existing assumptions to build better systems, and Own the end-to-end quality of our products
• Demonstrated ability to drive initiatives independently and thrive in a fast-paced, evolving startup environment

What the job involves

• As a software quality engineer at WRITER, you'll play a critical role in shaping the reliability, performance, and trustworthiness of our AI-powered work orchestration platform
• You’ll be at the forefront of defining and implementing rigorous quality strategies for our enterprise-grade LLMs and AI agents, directly impacting how hundreds of global companies unlock transformational value through AI
• This is a unique chance to dive deep into the unique challenges of AI quality assurance and make a tangible difference in a rapidly evolving field
• You will report directly to the director of engineering
• Define and implement comprehensive quality assurance strategies and test plans for our AI agents and LLM-powered applications, ensuring exceptional product reliability and performance
• Designing and developing automation frameworks: creating robust, scalable, and maintainable automated test frameworks from scratch or enhancing existing ones. You’ll need proficiency in at least one language like Typsecript, Python
• Collaborate closely with product managers, machine learning engineers, and data scientists to understand complex AI features and model behaviors, translating them into effective test cases and validation criteria
• Drive the continuous improvement of our testing processes and infrastructure, integrating automated checks within our CI/CD pipelines to ensure rapid, high-quality releases
• Identify, document, and track software defects and inconsistencies, performing root cause analysis to provide actionable feedback to development teams
• Monitor production systems and AI model performance, proactively identifying potential issues and contributing to post-release quality validation
• Champion quality best practices across engineering teams, fostering a culture of ownership and continuous improvement in delivering world-class AI solutions
• Designing, managing, and maintaining test data strategies and mock services to ensure stable, isolated, and repeatable test execution
• Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol (MCP)
• Manage the full defect lifecycle by analyzing customer feedback and debugging logs to identify, prioritize, and track software bugs, collaborating closely with development teams to ensure timely resolution

Benefits

• Health: Selection of employer-covered medical plans, plus dental, vision, life insurance, and FSA.
• Family-friendly: Competitive parental leave policy, working parents can truly own their schedules.
• Hybrid culture: We’re built around a mix of in office collaboration and remote work when needed.
• Finance: Company stock options, plus a 401k plan with employer contributions.
• Growth: Learning and development stipend and generous PTO.
• SF/NY/London office space: Three beautiful sunny offices in three incredibly exciting cities.