Staff / Software Engineer (Safeguards)

• We are looking for software engineers to help build safety and oversight mechanisms for our AI systems
• As a software engineer on the Safeguards team, you will work to monitor models, prevent misuse, and ensure user well-being
• This role will focus on building systems to detect unwanted model behaviors and prevent disallowed use of models
• You will apply your technical skills to uphold our principles of safety, transparency, and oversight while enforcing our terms of service and acceptable use policies
• Develop monitoring systems to detect unwanted behaviors from our API partners and potentially take automated enforcement actions; surface these in internal dashboards to analysts for manual review
• Build abuse detection mechanisms and infrastructure
• Surface abuse patterns to our research teams to harden models at the training stage
• Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale

Benefits

• Comprehensive health, dental, and vision insurance for you and your dependents

• Inclusive fertility benefits via Carrot Fertility

• 22 weeks of paid parental leave

• Flexible paid time off and absence policies

• Mental health support for you and your dependents

• Competitive salary and equity packages

• Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant

• Retirement plans with competitive matching

• Life and income protection plans

• $500/month flexible wellness and time saver stipend

• Commuter benefits

• Annual education stipend

• Home office stipends

• Relocation support for those moving for Anthropic

• Daily meals and snacks in the office

• Proficiency in Python and Typescript

• Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders

• Ability to work across the stack

• Bachelor’s degree in Computer Science, Software Engineering or comparable experience

• We encourage you to apply even if you do not believe you meet every single qualification

• Have experience with integrity, spam, fraud, or abuse detection and mitigation

• 8+ years of experience in a software engineering position

• Have experience building trust and safety detection mechanisms and intervention for AI/ML systems

• Have experience with prompt engineering, jailbreak attacks, and other adversarial inputs

• Have worked closely with operational teams to build custom internal tooling