Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Evaluate and improve AI-generated code for accuracy and quality.
- Field
- Software & coding
- Where
- United States
- Commitment
- contract
- Hours
- up to 20/week
- Asks for
- Master's
- Posted
- First posted June 2026, still open (checked today)
Pay data for your profession: Software engineer AI trainer jobs
What the listing asks for
- Experience
- 3+ years“3+ years of professional software engineering experience.”
- Hours
- 20 a week“Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.”
Read from Turing's listing text; only what it states. Full listing
How this pay compares
The title points to writing prompts, problems and answers: writing questions, test tasks, rubrics or model answers for a model to learn from. What each kind of AI training work pays
Among platforms with 10 or more priced software & coding roles, Mercor has the highest median ($88/hr across 18 roles) and OpenTrain AI the lowest ($67/hr across 89).
Computed from 3,087 open roles we track across Mercor, micro1, Handshake AI, Alignerr, Outlier, Turing, Appen, OpenTrain AI and OneForma, refreshed daily. Midpoints of listed ranges; “up to” ceilings are left out of medians.
Open software & coding roles on each platform
The same kind of work, priced on every platform we track, with the best-paid open role on each.
| Platform | Open | Median | Best-paid open role |
|---|---|---|---|
| Mercor | 20 | $88/hr | Legacy Codebase Migration Expert$200/hr |
| micro1 | 21 | $75/hr | Software Engineer (Staff), Platform Engineering$350k-600k/yr |
| Alignerr | 73 | $70/hr | DevOps / IaC Engineer$90-120/hr |
| OpenTrain AI | 148 | $67/hr | Software Engineer, AI Code Evaluation$200/hr |
| Handshake AI | 13 | up to $105/hr | Senior Software Engineer - AI Data (US)Up to $150/hr |
“Up to” rows list only ceilings, so their median is a maximum, not comparable with the range midpoints above them.
All 345 open software & coding roles →The full listing
As posted on Turing. See the original
About Turing
Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps leading AI labs improve the reasoning, problem-solving, and decision-making capabilities of large language models (LLMs) through high-quality human feedback, evaluation, and training data.
Role Overview
We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks. This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems.
What Does Day-to-Day Look Like?
- Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
- Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
- Debug code, reproduce issues, and verify fixes across different programming environments.
- Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
- Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
- Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
- Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
- Collaborate with project teams to establish quality standards and evaluation methodologies.
Requirements
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
- 3+ years of professional software engineering experience.
- Strong proficiency in one or more of the following languages: Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
- Strong understanding of data structures, algorithms, software design principles, and debugging methodologies.
- Experience performing code reviews and evaluating code quality in production or large-scale codebases.
- Ability to analyze complex technical problems and assess solution correctness with minimal supervision.
- Familiarity with version control systems (e.g., Git) and modern software development workflows.
- Strong written communication skills and attention to detail.
- Experience with AI/ML data annotation, NLP, prompt engineering, model evaluation, or LLM-related projects is a plus.
- Experience evaluating AI-generated code, benchmark creation, or software quality assessment is highly preferred.
Perks of Freelancing With Turing
- Work in a fully remote environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details
- Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
- Engagement type : Contractor assignment (no medical/paid leave)
- Duration of contract : 1 month; [expected start date is next week]
- Location: US only
Evaluation Process
- Online automated coding challenge for Python and Docker test (RHLF)
From applying to your first payment on Turing
- 1
Apply to this listing
Apply on work.turing.com with Apply Now and finish any assessments the listing asks for. Turing says reviews take 48 hours to 6 weeks.
- 2
Shortlist and client decision
Shortlisted isn't an offer: the client makes the final choice, and Turing asks you to confirm your rate, availability and terms before onboarding.
- 3
Onboarding
Verify your ID, complete tax forms and add a bank account in your own name, through Turing or Deel.
- 4
First payment
Hours are paid monthly, by the 5th business day of the following month, once your manager approves them.
Sources: Turing help: Apply for jobs at Turing, Turing help: How long does the application review take?, Turing help: What happens after being shortlisted?, Turing help: Project onboarding, Turing help: Project offboarding
Full Turing profile: watch-outs and the public record →Pay history
What workers report about Turing
Anonymous reports from people who applied or worked there: whether they got work, what they were paid and how fast.
Other software & coding roles
- Software Engineer (Staff), Platform Engineeringmicro1$350k-600k/yr
- Strategic Project Lead, AI Coding & Evaluationsmicro1$300k-500k/yr
- Legacy Codebase Migration ExpertMercor$200/hr
- Open Source GitHub MaintainerOpenTrain$120-200/hr
- Software Engineer – ($200/hr)Turing$200/hr
- Software Engineer, AI Code EvaluationOpenTrain$200/hr