Software Engineer – AI Coding Benchmark Review | $65/hr

Software Engineers: Paid Code Review for AI Agent Evaluation

Experience: Professional Software Engineering Experience Required

Job Type: Contractor

Location: Remote

Compensation: $65/hour

Payment: Weekly

Required Profiles: LinkedIn profile required

Role Overview

We are looking for practicing software engineers to review coding tasks and evaluation harnesses used to benchmark AI agents. You’ll assess whether the tasks reflect real-world software engineering, verify test logic, and provide feedback to improve how AI agents are evaluated on complex coding objectives.

Key Responsibilities

• Review programming tasks for accuracy, realism, and difficulty.
• Verify the logic and test cases in evaluation harnesses.
• Assess whether coding environments measure software engineering skills effectively.
• Explain your analysis of complex code structures during a guided conversation and screen-sharing exercises.

Key Requirements

• Active professional experience as a software engineer.
• Hands-on experience building or maintaining test suites and evaluation harnesses.
• Comfortable reviewing code and explaining technical concepts aloud.
• Familiarity with complex, real-world software architecture.

Preferred Technical Skills

• Backend or full-stack engineering.
• Machine learning engineering.
• Software architecture.
• Test automation, evaluation harnesses, and comprehensive code review.

Selection Process:

Step 1 – Short Screener

Step 2 – AI Interview

Step 3 – Paid Work Trial

Step 4 – Project Onboarding