AI Response Evaluator (Chinese)

The work

You will test a conversational AI personalization feature by creating prompts based on personal experiences and reviewing how well the system uses relevant context. The work combines creative prompt design with careful response evaluation and written feedback.

• Create and run conversational prompts that usually cover one to five turns.
• Check whether claims about a user are supported by evidence rather than flawed inferences or hallucinations.
• Compare two responses for helpfulness, ease of use, enjoyment, naturalness, and appropriate use of context.
• Write concise rationales that cite specific turns and explain positive or negative findings.
• Inspect debug information to confirm that summaries and relevant data sources were used correctly.
• Delete evaluation conversations after review so future chat history stays clean.

What it pays and takes

This is a remote contractor engagement lasting three months. The role is open worldwide and is listed as entry level, with part-time work available.

• Pay: $15 per hour.
• Time: 20+ hours per week, including at least four hours per day and up to 40 hours per week.
• Schedule: Include four hours of overlap with Pacific Time.
• Language: High proficiency reading and writing in Chinese.
• Experience: Experience creating creative multi-turn prompts from personal context.
• Knowledge: Understand personalization quality, grounding, incorrect inferences, and forced connections.
• Skills: Notice subtle differences between model responses and provide clear, detailed annotations and constructive feedback.
• Education: A bachelor's degree or equivalent experience in policy, law, ethics, linguistics, journalism, computer science, or a related analytical field is requested.
• Preferred background: Data annotation, AI quality evaluation, content moderation, or related experience.

How it works

Apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including reviewing model responses and rating their quality. People with strong language, analytical, or subject knowledge help make these systems more accurate, useful, and natural.