What are the responsibilities and job description for the Insurance SME - AI Evaluation (US-Remote) position at Braintrust?
Job Description:
We are seeking multiple Insurance Subject Matter Experts (SME) to help design, evaluate, and improve AI-driven workflows across insurance use cases.
You’ll define real-world processes, create rigorous evaluations, curate data, and partner with AI/product teams to boost LLM accuracy and relevance.
- To Note: This is for an immediate project need. Project is approved for 3-months initially, with possibility to extend based on project/client demands.
Responsibilities:
- Collaborate with AI/engineering to define common research workflows.
- Create evaluations/test cases with clear inputs, expected outputs, and success criteria.
- Design capability assessments for LLMs (data extraction, financial statements, market trends, risk).
- Guide data curation/annotation of high-quality financial content.
- Provide feedback and iterate with product to improve model performance.
Requirements:
- 5 years practical experience in Insurance related field.
- Strong analytical/critical thinking to structure complex processes.
- Excellent communication; can explain finance to non-finance audiences.
- Technical acumen: basics of AI/ML or working with technical teams a plus.
- Meticulous attention to detail for accurate, robust evaluation criteria.