Engineering Expert / AI Evaluation Engineer – Engineering Simulation & Design

Engineering Expert / AI Evaluation Engineer – Engineering Simulation & Design

Company: Vraify

About the Role

Vraify is looking for experienced engineering professionals to contribute to AI evaluation and engineering simulation projects. You will design challenging engineering problems, develop reference solutions, build and validate simulation environments, and evaluate AI-generated solutions for technical accuracy and engineering rigor.

Key Responsibilities

• Create original, self-contained engineering design and simulation problems.
• Develop accurate reference solutions and objective grading criteria.
• Build and validate engineering simulation environments.
• Evaluate AI-generated code, solutions, and execution trajectories.
• Analyze logs to identify reasoning, coding, and tool-use failures.
• Assess physical plausibility, unit consistency, boundary conditions, convergence, and overall engineering correctness.
• Refine benchmark problems and evaluation criteria based on observed model performance.
• Collaborate with AI researchers, domain experts, and technical teams.

Required Qualifications

• 10+ years of hands-on engineering design experience — mandatory. Candidates who do not meet this requirement are not eligible.
• Strong experience in one or more of: Electrical Engineering, Mechanical Engineering, Aerospace Engineering, Control Systems, Systems Engineering, or Robotics.
• Strong Python programming skills.
• Hands-on experience with engineering simulation, modeling, or computational design.
• Experience with relevant tools/frameworks such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
• Strong understanding of engineering fundamentals and technical problem-solving.
• Experience working with modern LLMs, coding agents, or AI-assisted engineering workflows.
• Ability to analyze execution/trajectory logs and diagnose reasoning or tool-use failures.
• Strong technical documentation and communication skills.

Preferred Qualifications

• Experience in AI evaluation, data annotation, content review, QA, benchmarking, or similar work.
• Experience creating technical evaluation datasets or benchmark problems.
• Experience collaborating with AI research or machine-learning teams.

Engagement Details

• Contract duration: Up to 24 weeks.
• Up to 40 hours per week.
• Fully remote.
• Minimum 4 hours of overlap with PST.
• Weekend on-call availability may be required.

Eligible Locations

Applicants must be based in Bangladesh, India, Indonesia, Egypt, Ghana, Kenya, Nigeria, Turkey, Brazil, or Colombia only.

Compensation: $500 per task.

Mandatory Experience Requirement

10+ years of hands-on engineering design experience is mandatory and will be required when screening candidates.

Pay: ₹500.00 per week

Benefits:

• Flexible schedule

Work Location: Remote