What are the responsibilities and job description for the Food & Travel Subject Matter Experts - AI Model Evaluator (US-Remote) position at Braintrust?
Job Description
We are seeking multiple Food & Travel Subject Matter Experts (SMEs) such as food critics, local journalists, travel agents, etc.) to support the evaluation and development of Large Language Models (LLMs).
The selected individuals will act as domain-specific instructors to help improve model accuracy, contextual understanding, and performance by identifying gaps and providing detailed feedback.
- To Note: This is for an immediate project need. Project is approved for 3-months initially, with possibility to extend based on project/client demands.
- To Note: This is a fixed hourly rate project in USD ($57/hr)
Responsibilities:
- Evaluate LLM Responses: Formulate domain-relevant questions, assess AI-generated outputs, and identify gaps or inaccuracies.
- Provide Structured Feedback: Clearly document findings in a systematic format to support improvement efforts.
- Act as a Teacher to AI: Reinforce correct knowledge and address model misconceptions.
- Collaborate with AI Teams: Translate expert insights into actionable refinements for the AI.
- Maintain Expertise: Stay current on trends, rules, and events within your sport to ensure accurate and up-to-date evaluations.
Requirements:
- Proven domain expertise in at least one of the following domains: Food Critic, Local Journalist, Travel Agent, etc.
- Strong analytical and critical thinking skills.
- Excellent written communication, particularly for feedback documentation.
- Familiarity with AI/ML is a plus, not a requirement.
- Ability to work independently and collaboratively in a dynamic environment.
Preferred Skills:
- Background in teaching, journalism, research, or professional analysis.
- Experience evaluating complex reasoning and contextual information.
- A curious, detail-oriented mindset with a drive to enhance AI systems.
Salary : $57