E
Exa
Research (Evals)
Skills
RustPythonMachine Learning
2 months ago
Share this opportunity
Share this opportunity to other talents of your network:
✓ Offer them a visibility boost with clients.
✓ Help your contacts find their next job.
Important information
Contract type:
Permanent
Salary:
$150,000 – $300,000 / year
Location:
San Francisco, CA, USA
Work mode:
Onsite
Published on:
29 August 2026
What they need
- We’re looking for an ML evals engineer to design and build our eval stack at Exa
- The role involves investigating how to evaluate search engines in an LLM world and then building the most comprehensive, creative, and effective eval suite
- You will be deciding the future of search through the evals we choose to optimize for
- Write a manifesto of what perfect search means
- Design and implement evaluation frameworks that probe the limits of search
- Build scalable, reliable eval pipelines that track regressions, drift, and quality signals across billions of documents
- Create golden datasets, synthetic benchmarks, agentic tasks, and real-world test suites that reflect how developers, agents, and humans actually use Exa
- Partner closely with ML researchers, data engineers, infra engineers, and product to shape the feedback loops that improve our search models- Have strong engineering fundamentals and can build reliable systems (Python, Rust, distributed pipelines, GPU/cluster jobs, etc.)
- Enjoy diving into data via building eval sets, inspecting edge cases, designing creative measurement strategies
- Have hands-on ML experience (training, finetuning, or evaluating models (bonus if related to embeddings or LLMs)
Other offers great for you!
These companies are also looking for great profiles
Y
Yunik
MLOps Engineer – GCP / Vertex AI / Python
Freelance
Paris, France
Hybrid
Skills
PythonGCPVertex AIVertex AI PipelinesCloud RunMLOpsCI/CDFinOpsMonitoring
1 day ago
Quick apply
V
Visian
Ingénieur MLOps IA
Freelance
Paris, France
Hybrid
Skills
PyTorchMachine LearningSoftware engineeringGPU inference optimizationONNXTensorRTMLflow
1 day ago
Quick apply
C
Cherrypick
Data Analyst Confirmé - Lamballe
Freelance
22400 Lamballe-Armor, France
Onsite, Hybrid
Skills
PythonETLAIAWSDataikuData architectureData modeling
1 hour ago
Quick apply
No company information available.