Quality Software Engineer

Who you are

• 2+ years of experience building and maintaining backend applications, services, or data-intensive systems
• Strong software engineering fundamentals, including system design, testing, debugging, and operational reliability
• A data-oriented approach to solving problems and measuring whether a system is working
• Experience working with APIs, databases, data pipelines, or distributed systems
• The ability to navigate ambiguous problems, break them into manageable pieces, and deliver iteratively
• Strong communication and collaboration skills across technical and non-technical teams
• A quality-focused mindset and an interest in building systems that make difficult-to-measure outcomes more rigorous
• Experience with machine learning, applied AI, or AI-enabled products
• Familiarity with model evaluation, experimentation, recommendation systems, or ranking systems
• Experience with annotation, data-labeling, tasking, or workflow-orchestration platforms
• Experience building developer platforms, internal tools, or extensible APIs
• Familiarity with MCP, agentic systems, or AI-native development tools
• Experience working in a fast-moving startup or similarly ambiguous environment

What the job involves

• We’re hiring a Software Engineer to join our Quality team. The team builds the systems that define, select, and enforce quality standards across the data and evaluations we deliver to leading AI labs
• You’ll work on infrastructure that evaluates complex outputs submitted by fellows, such as coding tasks designed to challenge advanced AI models
• These quality checks run throughout the production process: providing real-time feedback to fellows, supporting internal operators, and validating outputs before they reach customers
• Build and scale backend systems that evaluate and improve the quality of data delivered to frontier AI labs
• Develop infrastructure for running quality checks in real time and before final customer delivery
• Help design recommendation systems that select the most effective quality checks for each project
• Build data pipelines and measurement systems that make complex, subjective quality signals more observable and actionable
• Develop APIs and platform capabilities that allow other teams and tools to use the Quality platform
• Help surface Quality capabilities through MCP and other AI-native interfaces used by internal operators
• Partner with engineers, researchers, product teams, and operations stakeholders to turn ambiguous quality problems into scalable technical solutions
• Improve the reliability, efficiency, and cost-effectiveness of quality workflows as the business grows

Benefits

• Health Insurance: Comprehensive medical , dental, and vision coverage
• Fertility and family forming: Through a partnership, Handshake offers comprehensive family benefits, including infertility treatments and a $15K stipend, to its employees.
• LGTBQ+ coverage: All medical plans cover transgender services.
• Mental health: Discounts on a variety of wellness apps and an Employee Assistance Program with access to five free counseling sessions.
• Tax savings: Tax-advantaged benefits to save money on healthcare, dependent care, and other expenses.
• Life & disability insurance: Essential benefits and automatic coverage with basic life insurance, plus short-term and long-term disability coverage.
• 401k match: Dollar-for-dollar match up to 1% of pay, with a max of $1,200 per year.
• Equity: Our equity program includes grants at hire, refresher grants after two years of tenure, and grants alongside promotions.
• Financial management: Handshake partners to provide free professional financial planning services.
• Learning & development: Employees are provided with an annual stipend to use towards professional development, career development workshops, manager trainings, and 1-on-1 coaching through external partnerships
• Community leader support: Employee Resource Group leaders are compensated with a quarterly bonus for the high-impact work the role requires
• Parental leave: Leaves of 16 weeks for birth-giving parents and 10 weeks for non-birth-giving parents; the policy applies to adoptions as well. Handshake also partners to provide one-on-one coaching on preparing for leave and coming back from leave
• Flexible time off: No accrued time off or waiting period to take vacation. Two paid flexible holiday days off to observe holidays that closely align with religious, cultural, and personal needs
• Sabbatical: Paid six-week sabbatical to recharge and reset beginning at six years of continuous, full-time employment, and every four years thereafter
• Volunteer time off: Two days off for social and civil activism, volunteer work, and/or any related activities, including attending and participating in protests