SWE Bench Engineer (Java)

The work

This role evaluates Java code produced or reviewed for large language models. You will work with public Java repositories, GitHub issues, and unit-test codebases, then provide clear feedback that helps improve AI systems.

The duties and qualifications below describe Java LLM code evaluation, so review them before applying.

• Evaluate model outputs using project rubrics and record evidence-based judgments.
• Review source materials and model outputs against project guidelines.
• Explain your decisions clearly so feedback can improve model behavior.
• Run, modify, and test real-world Java projects locally.
• Maintain consistent, high-quality decisions across repeated review tasks.

What it pays and takes

This is a fully remote contractor assignment. The project is open to candidates in the countries listed below and requires English fluency.

• Pay: $25 per label.
• Time: At least 20 hours per week.
• Schedule: At least four hours per day and four hours of overlap with Pacific Standard Time.
• Location: Open to candidates in Argentina, Bangladesh, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, the Dominican Republic, Ecuador, Egypt, El Salvador, Ghana, Guatemala, Guyana, Haiti, Honduras, India, Kenya, Mexico, Nicaragua, Nigeria, Pakistan, Panama, Paraguay, Peru, Puerto Rico, Suriname, Türkiye, Uruguay, and Venezuela.
• Language: English fluency.
• Experience: At least three years of overall experience; previous AI training experience is not required.
• Technical skills: Strong Java experience, Git, Docker, basic software pipeline setup, and the ability to navigate complex codebases.
• Helpful background: Open-source contribution or evaluation, LLM research or evaluation, or experience building or testing developer tools and automation agents.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site. OpenTrain is free to join and helps people find AI training and data-labeling projects in one place.

About AI training work

AI training is the human work behind many artificial intelligence systems. Contributors review code, rate model answers, label data, and explain their judgments so AI systems can learn from reliable examples and expert feedback.