Research Engineer / Scientist (Alignment)

• As a Research Engineer on Alignment Science, you’ll contribute to exploratory experimental research on AI safety, with a focus on risks from powerful future systems (like those we would designate as ASL-3 or ASL-4 under our Responsible Scaling Policy), often in collaboration with other teams including Interpretability, Fine-Tuning, and the Frontier Red Team
• Our blog provides an overview of topics that the Alignment Science team is either currently exploring or has previously explored. Our current topics of focus include..
• Scalable Oversight: Developing techniques to keep highly capable models helpful and honest, even as they surpass human-level intelligence in various domains
• AI Control: Creating methods to ensure advanced AI systems remain safe and harmless in unfamiliar or adversarial scenarios
• Alignment Stress-testing: Creating model organisms of misalignment to improve our empirical understanding of how alignment failures might arise
• Automated Alignment Research: Building and aligning a system that can speed up & improve alignment research
• Representative projects:
• Testing the robustness of our safety techniques by training language models to subvert our safety techniques, and seeing how effective they are at subverting our interventions
• Run multi-agent reinforcement learning experiments to test out techniques like AI Debate
• Build tooling to efficiently evaluate the effectiveness of novel LLM-generated jailbreaks
• Write scripts and prompts to efficiently produce evaluation questions to test models’ reasoning abilities in safety-relevant contexts
• Contribute ideas, figures, and writing to research papers, blog posts, and talks
• Run experiments that feed into key AI safety efforts at Anthropic, like the design and implementation of our Responsible Scaling Policy

Benefits

• Comprehensive health, dental, and vision insurance for you and your dependents

• Inclusive fertility benefits via Carrot Fertility

• 22 weeks of paid parental leave

• Flexible paid time off and absence policies

• Mental health support for you and your dependents

• Competitive salary and equity packages

• Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant

• Retirement plans with competitive matching

• Life and income protection plans

• $500/month flexible wellness and time saver stipend

• Commuter benefits

• Annual education stipend

• Home office stipends

• Relocation support for those moving for Anthropic

• Daily meals and snacks in the office

• You could describe yourself as both a scientist and an engineer

• You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities

• You want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems

• Pick up slack, even if it goes outside your job description

• Care about the impacts of AI

• Prefer fast-moving collaborative projects to extensive solo efforts

• Have significant software, ML, or research engineering experience

• Have some familiarity with technical AI safety research

• Have some experience contributing to empirical AI research projects

• Have experience authoring research papers in machine learning, NLP, or AI safety

• Have experience with LLMs

• Have experience with reinforcement learning

• Have experience with Kubernetes clusters and complex shared codebases

• Education requirements: We require at least a Bachelor’s degree in a related field or equivalent experience