What are the responsibilities and job description for the Sports Subject Matter Experts - AI Model Evaluator (US-Remote) position at Braintrust?
Job Description
We are seeking multiple Sports Subject Matter Experts (SMEs) such as commentators, analysts, journalists, content creators, or ex-players to support the evaluation and development of Large Language Models (LLMs).
The selected individuals will act as domain-specific instructors to help improve model accuracy, contextual understanding, and performance by identifying gaps and providing detailed feedback.
- To Note: This is for an immediate project need. Project is approved for 3-months initially, with possibility to extend based on project/client demands.
- To Note: This is a fixed hourly rate project in USD ($57/hr)
Responsibilities:
- Evaluate LLM Responses: Formulate domain-relevant questions, assess AI-generated outputs, and identify gaps or inaccuracies.
- Provide Structured Feedback: Clearly document findings in a systematic format to support improvement efforts.
- Act as a Teacher to AI: Reinforce correct knowledge and address model misconceptions.
- Collaborate with AI Teams: Translate expert insights into actionable refinements for the AI.
- Maintain Expertise: Stay current on trends, rules, and events within your sport to ensure accurate and up-to-date evaluations.
Requirements:
- Proven domain expertise in at least one of the following sports: Soccer, Cricket, Football, Baseball, Basketball, or Formula 1.
- Strong analytical and critical thinking skills.
- Excellent written communication, particularly for feedback documentation.
- Familiarity with AI/ML is a plus, not a requirement.
- Ability to work independently and collaboratively in a dynamic environment.
Preferred Skills:
- Background in teaching, journalism, research, or professional analysis.
- Experience evaluating complex reasoning and contextual information.
- A curious, detail-oriented mindset with a drive to enhance AI systems.
Salary : $57