AI Evaluator

The work

You will evaluate AI assistant responses to realistic English teaching and classroom coordination scenarios. These may involve communication with students and parents, scheduling, educational events, and collaboration with colleagues.

You will give assistants detailed instructions, review their responses, and record your findings in AI training tools and platforms. Your reviews will help measure whether the writing is accurate, complete, clear, grammatically sound, appropriate for its audience, and faithful to the instructions.

• Complete standardized digital tasks based on English teaching and classroom coordination.
• Rank and annotate AI assistant responses using detailed grading guidelines.
• Assess grammar, writing conventions, tone, clarity, literary appropriateness, and instruction-following.
• Submit evaluation data through AI training tools and platforms.

What it pays and takes

This is a part-time contract role for candidates in the United States. The role is listed as entry level, with subject knowledge and practical experience required for the evaluation work.

• Pay: $15 to $40 per hour.
• Location: Open to candidates in the United States.
• Language: Fluent in English.
• Experience: Teaching English or literature in K-12, college, adult education, or ESL settings.
• Education: Bachelor's degree in English, literature, creative writing, linguistics, comparative literature, education, or a related humanities field.
• Skills: Strong grammar, writing, literary analysis, rhetoric, composition, detailed instruction-following, and rubric-based evaluation.
• Tools: Daily hands-on use of Google Classroom, plus regular teaching-related use of Gmail and Google Calendar.
• Other requirements: Practical experience using AI tools and reviewing AI-generated writing; an iPhone with iMessage; and a Facebook and/or Instagram account.

Helpful background

A master's degree or advanced coursework in English, literature, or a related field can support this work. Experience with AI evaluation, data labeling, user testing, structured feedback, or rubric development is also helpful.

• Familiarity with prompt writing.
• Understanding of AI assistant behavior and limitations.
• Experience creating or applying evaluation rubrics.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work is the human side of building artificial intelligence. Contributors review and improve examples such as written responses so AI systems can produce more accurate, useful, and appropriate results.