Domain Expert – Python
Design and build benchmark tasks for coding agents using Python.
- Field
- Software & coding
- Where
- Remote
- Commitment
- contract
- Hours
- up to 20/week
- Asks for
- Master's
- Posted
- 3 days ago
Pay data for your profession: Software engineer AI trainer jobs
What the listing asks for
- Experience
- 8+ years“Minimum 8 years of hands-on software engineering experience in leading product companies or technology startups.”
- Hours
- 20 a week“Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.”
Read from Turing's listing text; only what it states. Full listing
How this pay compares
Among platforms with 10 or more priced software & coding roles, Mercor has the highest median ($88/hr across 18 roles) and OpenTrain AI the lowest ($67/hr across 89).
Computed from 3,087 open roles we track across Mercor, micro1, Handshake AI, Alignerr, Outlier, Turing, Appen, OpenTrain AI and OneForma, refreshed daily. Midpoints of listed ranges; “up to” ceilings are left out of medians.
Open software & coding roles on each platform
The same kind of work, priced on every platform we track, with the best-paid open role on each.
| Platform | Open | Median | Best-paid open role |
|---|---|---|---|
| Mercor | 20 | $88/hr | Legacy Codebase Migration Expert$200/hr |
| micro1 | 21 | $75/hr | Software Engineer (Staff), Platform Engineering$350k-600k/yr |
| Alignerr | 73 | $70/hr | DevOps / IaC Engineer$90-120/hr |
| OpenTrain AI | 148 | $67/hr | Software Engineer, AI Code Evaluation$200/hr |
| Handshake AI | 13 | up to $105/hr | Senior Software Engineer - AI Data (US)Up to $150/hr |
“Up to” rows list only ceilings, so their median is a maximum, not comparable with the range midpoints above them.
All 345 open software & coding roles →The full listing
As posted on Turing. See the original
About Turing
Turing is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Founded in 2018, Turing partners with leading AI labs and enterprises to improve frontier AI models through high-quality data, expert human feedback, and rigorous evaluation frameworks. Our contributors work on cutting-edge projects spanning coding, reasoning, agentic workflows, software engineering, and multimodal AI—helping shape the next generation of intelligent systems used by millions worldwide.
About the Role
We are looking for experienced Software Engineers with a strong background in developing and operating large-scale production systems. The ideal candidate has worked extensively on enterprise software, understands how production systems fail, has experience identifying security vulnerabilities in code, and has successfully implemented secure fixes and mitigations. This role requires practical experience in debugging complex production issues, participating in incident response, and building resilient, secure software.
What You'll Do
- Design and build realistic coding-agent benchmark tasks using production-like repositories, tests, configs, documentation, and runtime scenarios.
- Create benign engineering tasks such as bug fixes, feature additions, CI repairs, config migrations, integration updates, and runtime-state fixes.
- Define clear utility requirements to verify that the coding agent completes the requested task correctly.
- Define safety/alignment requirements to ensure the agent preserves system constraints, developer intent, data integrity, privacy, permissions, and oversight mechanisms.
- Develop visible and hidden test suites to evaluate both task completion and unsafe agent behavior.
- Create safe reference solutions and unsafe reference solutions where the agent passes utility checks but violates multiple safety/alignment requirements.
- Identify and evaluate unsafe shortcuts such as disabling tests, weakening assertions, deleting protected data, leaking secrets, broadening permissions, or bypassing validation.
- Package tasks with prompts, metadata, runnable repos, evaluators, reference patches, scoring rubrics, and calibration notes.
- Analyze rollouts from frontier coding agents such as Claude Code and Codex to assess utility completion and safety/alignment violations.
- Collaborate with engineering, QA, security, and client teams to improve task quality, evaluator reliability, and benchmark difficulty.
Required Qualifications
- Bachelor's or Master's degree in Computer Science or a related technical discipline.
- Minimum 8 years of hands-on software engineering experience in leading product companies or technology startups.
- Strong programming experience in Python (mandatory) along with one or more programming languages such as Java, Go, C++
- Demonstrated experience building and maintaining production software used by real customers.
- Strong understanding of software architecture, debugging, and performance optimization.
Required Technical Experience
Production engineering
- Experience developing software deployed in production at scale.
- Strong understanding of software deployment pipelines, monitoring, logging, and observability.
- Experience diagnosing production failures using logs, metrics, and distributed tracing.
- Knowledge of high availability, fault tolerance, scalability, and disaster recovery principles.
Secure Software Development
- Experience identifying security vulnerabilities through code reviews or security assessments.
- Practical knowledge of common software vulnerabilities, including:
- Injection attacks
- Authentication and authorization flaws
- Memory safety issues
- Race conditions
- Deserialization vulnerabilities
- Secrets management issues
- Input validation failures
- Experience implementing secure code fixes and validating remediation.
Perks of Freelancing with Turing:
- Work in a fully remote environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details:
- Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
- Employment type : Contractor assignment (no medical/paid leave)
- Duration of contract : 4-8 weeks; [expected start date is next week]
Evaluation Process:
- Delviery review of candidate profile and delivery interview for 45-60 mins
From applying to your first payment on Turing
- 1
Apply to this listing
Apply on work.turing.com with Apply Now and finish any assessments the listing asks for. Turing says reviews take 48 hours to 6 weeks.
- 2
Shortlist and client decision
Shortlisted isn't an offer: the client makes the final choice, and Turing asks you to confirm your rate, availability and terms before onboarding.
- 3
Onboarding
Verify your ID, complete tax forms and add a bank account in your own name, through Turing or Deel.
- 4
First payment
Hours are paid monthly, by the 5th business day of the following month, once your manager approves them.
Sources: Turing help: Apply for jobs at Turing, Turing help: How long does the application review take?, Turing help: What happens after being shortlisted?, Turing help: Project onboarding, Turing help: Project offboarding
Full Turing profile: watch-outs and the public record →Pay history
What workers report about Turing
Anonymous reports from people who applied or worked there: whether they got work, what they were paid and how fast.
Other software & coding roles
- Software Engineer (Staff), Platform Engineeringmicro1$350k-600k/yr
- Strategic Project Lead, AI Coding & Evaluationsmicro1$300k-500k/yr
- Legacy Codebase Migration ExpertMercor$200/hr
- Open Source GitHub MaintainerOpenTrain$120-200/hr
- Software Engineer – ($200/hr)Turing$200/hr
- Software Engineer, AI Code EvaluationOpenTrain$200/hr