SHIP Medicare Counselor for AI Evaluation

The work

You will help improve AI systems that answer Medicare questions. Your work will focus on whether questions sound like real beneficiary concerns and whether generated answers are accurate, complete, useful, and current.

You will write rubrics, evaluate model responses, and provide clear feedback asynchronously. Accuracy comes first, while usefulness for people from diverse backgrounds also matters.

• Review sample Medicare questions and identify questions that do not reflect real beneficiary concerns.
• Create rubrics for initial enrollment, special enrollment periods, and Part B late penalties.
• Define guidance for comparing Medigap, Medicare Advantage, and Part D plans, including key differences and suitable use cases.
• Evaluate AI-generated answers and flag errors, missing information, and outdated guidance.
• Provide precise written feedback on model responses.

What it pays and takes

This is a remote, short-term contractor project for contributors in the United States. Medicare counseling expertise is the key qualification, and prior AI experience is not required.

• Pay: $30 to $60 per hour.
• Schedule: 20 to 30 hours per week.
• Location: Open to candidates based in the United States.
• Language: Professional fluency in English.
• Experience: Active or recent SHIP certification, or former SSA Medicare enrollment experience.
• Experience: At least one year advising on initial enrollment, special enrollment periods, Part B late penalties, Medigap versus Medicare Advantage, and Part D.
• Requirement: Clear, precise written communication and the ability to contribute consistently to a short-term project.
• Helpful background: Creating Medicare guides, training materials, or counseling rubrics.
• Helpful background: Counseling diverse beneficiaries, including rural and non-English-speaking clients.

How it works

Apply on OpenTrain with your resume and then complete the application on the hiring site. OpenTrain is a platform where people find AI training and data-labeling work, create a profile, and apply to projects.

About AI training work

AI training work is the human side of building artificial intelligence. People review questions and answers, label information, and give feedback so AI systems become more accurate and useful; subject-matter experts are especially valuable when the work requires careful judgment.