Python ML Quality Assurance Lead

The Work

As a Python ML Quality Assurance Lead, you will review AI-generated Python code, machine learning workflows, model explanations, and contributor work. You will check technical accuracy, follow project rubrics, give clear written feedback, and help maintain consistent quality across Python machine learning training tasks.

You will also answer questions about Python syntax, package use, data leakage, model validation, metrics, reproducibility, notebooks, and rubric interpretation. The role includes contributor communication, documentation, onboarding support, calibration tasks, and process improvements.

• Review Python code, machine learning pipelines, preprocessing, model training workflows, evaluation logic, and technical explanations.
• Check work for data leakage, statistical validity, debugging accuracy, formatting, readability, maintainability, reproducibility, and instruction-following.
• Identify flawed methods, incorrect metrics, made-up or incorrect APIs, misleading conclusions, and incomplete explanations.
• Give precise feedback, escalate recurring or serious quality issues, and help contributors meet detailed review standards.
• Update guidance, trackers, frequently asked questions, examples, honeypots, and onboarding materials.
• Support onboarding and training calls for Python machine learning contributors.
• Help maintain scalable quality assurance processes for remote technical teams.

What It Pays and Takes

This is an intermediate-level, part-time contractor role for someone who can make careful technical judgments and explain them clearly in English.

• Pay: $120 per hour.
• Time: 20 or more hours per week.
• Location: United States only.
• Language: Strong English communication skills are required.
• Python: Strong knowledge of data structures, functions, classes, iterators, comprehensions, exception handling, virtual environments, package management, testing, and debugging.
• Machine learning: Strong knowledge of supervised and unsupervised learning, feature engineering, train/test splits, cross-validation, model selection, data leakage, regression, classification, clustering, metrics, bias and variance, regularization, and reproducibility.
• Review skills: Ability to assess machine learning content against detailed rubrics and explain technical problems clearly.
• Preferred background: Experience in AI training, data annotation, large language model evaluation, code QA, machine learning QA, or rubric-based technical review.
• Leadership experience: Experience leading or supporting remote teams of trainers, reviewers, engineers, data scientists, or machine learning researchers is preferred.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that generate code, text, and other outputs: people review examples, rate results, and explain what is correct or incorrect. OpenTrain helps people find and build careers in this field, where strong Python and machine learning judgment is valuable because it improves the quality and reliability of AI systems.