Software Engineer – AI Evaluation | $40/hr

Software Engineer – AI Coding Task Evaluation (Brazil & Argentina)

Experience: Professional software engineering experience required

Job Type: Contractor

Location: Remote (Brazil or Argentina)

Compensation: $40/hour

Payment: Weekly

Required Profiles: LinkedIn profile

Role Overview

We’re seeking software engineers based in Brazil or Argentina to review coding tasks and evaluation harnesses used to assess AI agents. You’ll examine task realism and test accuracy, share your screen to walk through the environments, and suggest improvements to the challenges.

Key Responsibilities

• Review programming tasks designed for AI evaluation.
• Assess the accuracy and structure of evaluation harnesses.
• Explain your approach to checking code and task validity.
• Provide actionable feedback to improve task complexity and realism.

Key Requirements

• Professional experience as a software engineer or developer.
• Based in Brazil or Argentina.
• Familiarity with building or verifying coding tasks and test environments.
• Comfortable discussing technical architecture and evaluation frameworks.

Preferred Technical Skills

• Backend or full-stack engineering.
• QA automation.
• Coding task and test environment development.
• Evaluation harness review.

Selection Process:

Step 1 – Short Screener

Step 2 – AI Interview

Step 3 – Paid Work Trial

Step 4 – Project Onboarding