Senior Software Engineer – Go (LLM Evaluation & Repository Validation)
Analyze GitHub issues and evaluate LLM performance on real-world code.
- Field
- Machine learning
- Where
- India, Pakistan, Nigeria, Kenya, Egypt, Ghana +3 more
- Commitment
- contract
- Hours
- up to 20/week
- Posted
- First posted July 2025, still open (checked today)
Pay data for your profession: Data scientist AI trainer jobs
What the listing asks for
- Experience
- 3+ years“Minimum 3+ years of overall experience”
- Hours
- 20 a week“Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST. (We have 3 options of time commitment: 20 hrs/week, 30 hrs/week or…”
Read from Turing's listing text; only what it states. Full listing
How this pay compares
The title points to rating and reviewing ai output: judging a model's answers or code against guidelines, such as which of two is better or whether one is correct and safe. What each kind of AI training work pays
Among platforms with 10 or more priced machine learning roles, OpenTrain AI has the highest median ($109/hr across 34 roles) and Alignerr the lowest ($63/hr across 13).
Computed from 3,087 open roles we track across Mercor, micro1, Handshake AI, Alignerr, Outlier, Turing, Appen, OpenTrain AI and OneForma, refreshed daily. Midpoints of listed ranges; “up to” ceilings are left out of medians.
Open machine learning roles on each platform
The same kind of work, priced on every platform we track, with the best-paid open role on each.
| Platform | Open | Median | Best-paid open role |
|---|---|---|---|
| OpenTrain AI | 47 | $109/hr | Bioinformatics Machine Learning Expert$175/hr |
| Mercor | 10 | $108/hr | Machine Learning Engineer Talent Network$70-250/hr |
| micro1 | 15 | $90/hr | Member of Technical Staff, Cortex Lab (applied AI research)$300k-700k/yr |
| Alignerr | 13 | $63/hr | Software Engineer - Machine Learning$90-120/hr |
| Handshake AI | 3 | up to $75/hr3 priced | Data ScientistUp to $120/hr |
“Up to” rows list only ceilings, so their median is a maximum, not comparable with the range midpoints above them.
All 128 open machine learning roles →The full listing
As posted on Turing. See the original
Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation
About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc.
About the Role: We are looking for experienced software engineers (tech lead level) who are familiar with high-quality public GitHub repositories and can contribute to this project. This role involves hands-on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality
Why Join Us? Turing is one of the world’s fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems. You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
What does day-to-day look like:
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluating unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Opportunities to lead a team of junior engineers to collaborate on projects.
Required Skills:
- Minimum 3+ years of overall experience
- Strong experience with at least one of the following languages: Go
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real-world projects locally.
- Experience contributing to or evaluating open-source projects is a plus.
Nice to Have:
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.
Perks of Freelancing With Turing:
- Work in a fully remote environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details:
- Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST. (We have 3 options of time commitment: 20 hrs/week, 30 hrs/week or 40 hrs/week)
- Employment type : Contractor assignment (no medical/paid leave)
- Location : India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico
Evaluation Process (approximately 75 mins) :
- Two rounds of interviews (60 min technical + 30 min technical & cultural discussion)
From applying to your first payment on Turing
- 1
Apply to this listing
Apply on work.turing.com with Apply Now and finish any assessments the listing asks for. Turing says reviews take 48 hours to 6 weeks.
- 2
Shortlist and client decision
Shortlisted isn't an offer: the client makes the final choice, and Turing asks you to confirm your rate, availability and terms before onboarding.
- 3
Onboarding
Verify your ID, complete tax forms and add a bank account in your own name, through Turing or Deel.
- 4
First payment
Hours are paid monthly, by the 5th business day of the following month, once your manager approves them.
Sources: Turing help: Apply for jobs at Turing, Turing help: How long does the application review take?, Turing help: What happens after being shortlisted?, Turing help: Project onboarding, Turing help: Project offboarding
Full Turing profile: watch-outs and the public record →Pay history
What workers report about Turing
Anonymous reports from people who applied or worked there: whether they got work, what they were paid and how fast.
Other machine learning roles
- Member of Technical Staff, Cortex Lab (applied AI research)micro1$300k-700k/yr
- Machine Learning Engineer Talent NetworkMercor$70-250/hr
- Member of Technical Staff, Infrastructuremicro1$320k-500k/yr
- AI/ML Engineer, LLM ApplicationsOpenTrain$70-200/hr
- Bioinformatics Machine Learning ExpertOpenTrain$175/hr
- ML Research Engineer, Inference and GPU KernelsOpenTrain$100-170/hr