Psychologist AI Evaluation Expert

The work

You will use your psychological expertise to create and assess clinical and behavioral scenarios that test AI judgment and safety. The work focuses on clinical accuracy, ethical reasoning, and safe handling of risk.

• Write realistic clinical vignettes involving risk assessment and ethical dilemmas.
• Create reference responses that explain the clinical reasoning and safety approach behind each decision.
• Rate AI responses for clinical accuracy, ethics, and risk handling.
• Identify fluent, empathetic responses that still handle risk incorrectly, and explain the problem clearly.
• Apply professional judgment to evaluation data used to measure and improve model performance.

What it pays and takes

• Pay: $90 to $155 per hour, based on experience, paid weekly via Stripe.
• Hours: 20+ hours per week. The work is asynchronous, with flexibility to scale hours from week to week.
• Work arrangement: Fully remote independent contractor work with an ongoing engagement.
• Location: Open to candidates in eligible countries; this role is not worldwide.
• Language: Professional fluency in English.
• Requirements: A PhD or PsyD in psychology and an active, unrestricted US psychologist license.
• Requirements: At least two years of clinical or research practice after licensure, with depth in assessment, psychotherapy, or psychological testing.
• Requirements: Experience with risk assessment and safety planning, and the ability to explain clinical reasoning clearly to another psychologist.
• Helpful background: ABPP board certification, an academic appointment, crisis intervention or suicide prevention protocol experience, peer-reviewed publications, or clinical supervision experience.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work uses human examples and reviews to help AI systems give better responses. Experienced psychologists are needed to judge clinical reasoning, ethics, and safety in ways that general reviewers cannot.