Code Reviewer (Python)

The work

You will evaluate and rate AI-generated Python code for quality, performance, and its ability to fix real software issues. The work uses open-source repositories and includes hands-on testing in local development environments.

You will also help identify difficult coding tasks that can show where AI models need improvement.

• Analyze and triage issues in popular open-source Python libraries.
• Set up and configure code repositories, including Docker-based environments.
• Review unit test coverage and assess its quality.
• Run and modify codebases locally to evaluate AI-generated bug fixes.
• Work with researchers to identify repositories and issues that challenge AI coding systems.

What it pays and takes

This is a remote contract assignment expected to last two to three weeks, with an expected start next week.

• $25 per approved task.
• 20 or more hours per week.
• Open to candidates in India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Türkiye, and Mexico.
• Four hours of working-time overlap with Pacific Time.
• Professional fluency in English.
• At least two years of overall work experience.
• Strong Python experience.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfort running, modifying, and testing real-world projects locally.
• Open-source contribution or evaluation experience is helpful.
• LLM research or evaluation experience is helpful.
• Experience building or testing developer tools or automation agents is helpful.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work uses human reviews of examples, code, text, images, audio, or model responses to improve artificial intelligence systems. Code reviewers are paid for applying their software knowledge to judge whether AI-generated solutions work correctly and meet project standards.