Senior Software Engineer (Evals)

Senior Software Engineer, Evaluations

Frontier AI | London or Paris (Hybrid)

Compensation: Up to £170k + equity package

About

A model release decision is only as good as the numbers behind it. This team owns the framework that produces them: orchestration, runtimes and observability for every benchmark, across web apps, desktop applications and the command line. It runs around 50 benchmarks today and needs to handle 100-200 soon without getting slower or more expensive.

This is a well-funded frontier AI company building computer-use agents and the models behind them. Developers use them through a managed API, and a forward-deployed team takes them into enterprise workflows.

Researchers are the biggest users. They pick checkpoints and decide what ships based on your results. The rest varies week to week: cluster tuning one week, a browser extension or desktop environment the next.

What you'll do

• Own a chunk of the framework – scaling, scheduling or observability – so the cluster isn't sitting idle while jobs queue
• Set the bar for how new benchmarks get added, and build the checks that enforce it
• Make it quicker for researchers and forward-deployed engineers to plug in new benchmarks
• Make results reproducible, so a release call rests on numbers that hold up
• Talk to customers about what they want measured, then automate it into the harnesses

What you'll need

• 5+ years of backend work with production Python at the core. Backend-leaning full-stack is fine
• You've built AI-adjacent tooling, platforms or infra that other people relied on.
• Strong familiarity with Kubernetes - at least at the application layer, even better if you have run a cluster
• Shipped things end to end: APIs, integrations, databases, queues (SQS, RabbitMQ or Kafka)
• You instrument as you build: metrics, tracing, monitoring
• You use coding agents to go faster without letting quality slip

Optional Bonus

• Evals experience – a real plus, not a requirement
• Playwright, Selenium, or a browser extension you wrote
• Temporal, FastAPI, Postgres, Grafana or Datadog

Shortlisted candidates will be contacted within 48 hours.