AI Evaluation Engineer

Role & responsibilities

• Design and validate complex engineering simulation problems for AI agents.
• Create engineering design tasks with multiple constraints, optimization goals, and validated solutions.
• Use open-source simulation tools and Python to build and test engineering problems.
• Analyze AI agent outputs and execution logs to identify reasoning, coding, and simulation failures.
• Refine benchmark tasks based on model performance and ensure technical accuracy and physical plausibility.

Preferred candidate profile

• Masters degree or PhD in Electrical, Mechanical, or Aerospace Engineering with 10+ years of hands-on engineering design experience.
• Strong experience with engineering simulation tools and Python scripting.
• Experience with LLMs, coding agents, AI evaluation, or failure-mode analysis.
• Strong understanding of engineering principles, physical constraints, unit consistency, and simulation validation.
• Excellent analytical, problem-solving, and technical documentation skills.