AI Evaluation Rubric Auditor

Mercor

📍 Asunción, Asunción, Bolivia, Plurinational State of On-Site ⏳ 1d ago
AI assistant

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: AI Evaluation Rubric Auditor
Type: Contract
Compensation: $60–$85/hour
Location: Remote

Role Responsibilities

  • Audit complete tasks, including prompts and grading rubrics, to ensure rubric failures are fair.
  • Evaluate if information was available and if rubrics accurately grade the prompt requirements.
  • Determine if valid answers were incorrectly marked wrong and if rubric structures follow