Principal Research Scientist in AI Evaluation

The work

You will help evaluate advanced AI systems through rigorous research in computer science and computational physics. The work focuses on problems where progress can be checked through proofs, code, numerical results, or reproducible simulations.

You will judge whether model-generated research is correct, novel, and meaningful, then document what worked, what failed, and where the models have limits.

• Frame foundational open problems, theoretical bounds, and simulation challenges.
• Direct research involving proofs, algorithms, numerical analysis, computational experiments, and high-performance computing.
• Validate outputs through proof checks, computational reproduction, and formal methods.
• Assess research results against current literature and established findings.
• Write concise research memos on research attempts, partial solutions, model limitations, and recurring failure patterns.
• Recommend subfields and open problems that AI models and computational tools may be able to advance.

What it pays and takes

This is a remote contractor opportunity with flexible part-time or full-time availability. The role requires deep research judgment and the ability to examine technical work beyond your immediate area of expertise.

• Pay: $100 to $400 per hour.
• Work arrangement: Remote contractor role with part-time or full-time availability.
• Location: Open to candidates in the listed countries, including the United States, Canada, the United Kingdom, India, Japan, and many others.
• Language: Fluent in English.
• Experience: 7 to 10 or more years of active research experience in computer science or mathematical or computational physics.
• Research record: Strong scholarly contributions and recent publications in leading venues.
• Technical background: Hands-on experience with simulation, numerical analysis, algorithmic implementation, or high-performance computing.
• Preferred evidence: Resolving named open problems or introducing influential concepts adopted by peers.
• Helpful background: Major field prizes, society fellowships, plenary or invited talks, best paper awards, program committees, editorial boards, Lean, Coq, SAT or SMT solvers, or public research involving advanced AI systems.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including reviewing model answers, testing reasoning, and checking technical results. Researchers with deep subject knowledge help determine whether model outputs are accurate, reproducible, and useful.],

bullets

About AI training work

AI training is the human work behind systems that learn from examples, including reviewing model answers, testing reasoning, and checking technical results. Researchers with deep subject knowledge help determine whether model outputs are accurate, reproducible, and useful.