AI Evaluation Rubric Auditor
The work We are testing AI agents on realistic shopping and delivery tasks. Your role as a Project Auditor is to audit the work done by our Project Reviewers. You will audit complete tasks (prompt and grading rubric) and determine if their rubric failures are fair: - Was the…
- Field
- Finance & accounting
- Where
- Canada, United States, Argentina, BOL, Brazil, CHL +7 more
- Commitment
- hourly
- Hours
- up to 15/week
- Openings
- 11
- Posted
- 5 days ago
Pay data for your profession: Accounting AI trainer jobs · Finance AI trainer jobs
How this pay compares
The title points to writing prompts, problems and answers: writing questions, test tasks, rubrics or model answers for a model to learn from. What each kind of AI training work pays
At $73/hr (the midpoint of $60 to $85), this role pays more than 30% of the 164 open finance & accounting roles with a listed pay range, below the median of $85.
Among platforms with 10 or more priced finance & accounting roles, OpenTrain AI has the highest median ($100/hr across 77 roles) and Alignerr the lowest ($73/hr across 35).
Mercor roles that closed since late October stayed listed a median of 48 days (181 roles). Mercor posted this one 5 days ago.
Computed from 3,087 open roles we track across Mercor, micro1, Handshake AI, Alignerr, Outlier, Turing, Appen, OpenTrain AI and OneForma, refreshed daily. Midpoints of listed ranges; “up to” ceilings are left out of medians.
What you would earn at this rate
Per week, before tax
$1,088
Per month, if the work is steady
$4,713
- Tax
- Contract pay arrives with no tax withheld. A US contractor at this rate and these hours would set aside about $1,191 a month for self-employment tax plus federal income tax at the 12% bracket (25%, before state tax). Outside the US, your own country's rules apply.
Assumes paid hours only; unpaid assessments, onboarding and gaps between projects lower the real figure. Full take-home calculator · Track what you actually earn
Open finance & accounting roles on each platform
The same kind of work, priced on every platform we track, with the best-paid open role on each.
| Platform | Open | Median | Best-paid open role |
|---|---|---|---|
| OpenTrain AI | 100 | $100/hr | Investment Banking M&A Advisory AI Training Expert$175-300/hr |
| micro1 | 21 | $90/hr | Domain Researcher, Finance (M&A)$400k-800k/yr |
| Mercorthis role | 38 | $85/hr | Pharma Commercial Forecasting Expert — Launch Curves$130-210/hr |
| Alignerr | 35 | $73/hr | Accounting Specialist — Excel Dataset Creation$40-150/hr |
| Handshake AI | 22 | up to $83/hr | Investment BankersUp to $175/hr |
“Up to” rows list only ceilings, so their median is a maximum, not comparable with the range midpoints above them.
All 237 open finance & accounting roles →The full listing
As posted on Mercor. See the original
The work
We are testing AI agents on realistic shopping and delivery tasks. Your role as a Project Auditor is to audit the work done by our Project Reviewers. You will audit complete tasks (prompt and grading rubric) and determine if their rubric failures are fair:
- Was the information available?
- Does the rubric accurately grade what the prompt asked for?
- Would a valid answer have been marked wrong?
- Is the rubric structure correct per the guidelines?
This role includes providing feedback to reviewers regarding prompt and rubric issues.
Must-haves
- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.
- Experience providing feedback to writers/reviewers
- Consistent availability
- Can read a long agent transcript and find where it went wrong.
- Precise writing, and you check the data instead of trusting the reference answer.
From applying to your first payment on Mercor
- 1
Apply to this listing
Apply on Mercor with your resume.
- 2
AI interview
Take Mercor's recorded AI interview, about 20 minutes. A completed interview carries over to other listings that use the same one.
- 3
Work
Hourly work is counted in weeks that run Saturday to Friday.
- 4
First payment
Mercor releases pay each Wednesday by end of day Pacific through Stripe (or Wise). Stripe holds a first payout for 7 days, and a standard payout takes up to 7 business days to reach your bank.
Sources: Mercor talent docs: AI interview, Mercor talent docs: payments
Full Mercor profile: watch-outs and the public record →Pay history
Mercor has listed this role at $60-85/hr since Nov 18, 2026 with no change.
Across Mercor's finance & accounting roles, the median listed pay was $85/hr on Nov 1, 2026 (24 roles) and is $85/hr as of Nov 22, 2026 (31 roles).
What workers report about Mercor
Anonymous reports from people who applied or worked there: whether they got work, what they were paid and how fast.
Other finance & accounting roles
- Domain Researcher, Finance (M&A)micro1$400k-800k/yr
- Investment Banking M&A Advisory AI Training ExpertOpenTrain$175-300/hr
- Software Engineer (Staff), Financial Platform & Inframicro1$350k-500k/yr
- Pharma Commercial Forecasting Expert — Launch CurvesMercor$130-210/hr
- Biotech Investment Analyst — Drug Asset & Company AssessmentMercor$120-200/hr
- Finance ExpertMercor$60-180/hr