What are the responsibilities and job description for the Researcher position at Theory AI?
- Fully paid medical, dental and vision
- 401(k) with employer contribution
- Unlimited paid time off with a four-week minimum
- $3,000 a year for conferences, courses and certifications
- Home office budget and a choice of hardware
- This is a Remote (work from home) position.
Client work surfaces the same unsolved problems repeatedly: how to evaluate refusal behaviour in a way that transfers across deployments, how to measure annotation quality when there is no gold standard, how to say something honest about a model whose failure modes are rare and consequential.
This role exists to work on those problems properly rather than around them each time. Part of your output is internal: methodology the delivery teams adopt. Part of it is external, because publishing is how a firm our size builds a reputation for knowing what it is talking about.
You will not be isolated from delivery. The problems come from real engagements, and the answers go back into them.
What you will do- Investigate evaluation and measurement problems arising from client engagements
- Develop methodology the delivery teams can actually adopt, with the tooling to make it usable
- Publish findings externally where the work supports it
- Review the methodological soundness of what we claim to clients
- Keep the firm current on relevant research and translate it into practice
- A research background in machine learning, statistics, or a quantitative field. A PhD is one route, not the only one
- Demonstrated original work, published or otherwise verifiable
- Strong Python and the ability to build your own experimental tooling
- Writes clearly for readers who are not specialists in your subfield
- Published on evaluation, benchmarking, or measurement validity
- Experience with human annotation as a research instrument
- Comfortable presenting to non-research audiences, including clients
Salary : $3,000