What are the responsibilities and job description for the Founding AI Engineer position at Aimhire?
Founding AI Engineer
đź“Ť San Francisco, CA | đź•’ Full-Time, On-Site |
About the Company
Our client is a Y Combinator–backed startup transforming how teams test and monitor AI voice agents. Founded by IIT Bombay alumni with deep research and trading expertise, they’re solving the pain of manual voice testing by building automation tools that simulate thousands of real-world conversations.
Their platform uses dynamic personas, AI-generated datasets, and detailed workflows to uncover edge cases, identify failure points, and ensure that conversational agents perform reliably — before going live. Their mission is simple: help teams ship voice agents that work every time.
About the Role
As a Founding AI Engineer, you’ll work closely with the founders to design and build the core AI infrastructure behind their voice agent testing and monitoring platform. You’ll drive experimentation, build scalable systems, and collaborate directly with customers to incorporate feedback and improve performance.
This is a hands-on, high-impact role where you'll help define both product and technology direction from day one.
Key Responsibilities
- Build AI tools for testing, benchmarking, and monitoring conversational agents
- Develop scalable pipelines and evaluation methods for LLMs
- Implement systems for real-time monitoring and feedback
- Optimize deployments for high availability and performance
- Engage with customers to understand needs and shape product direction
- Continuously experiment and improve AI reliability and accuracy
What We’re Looking For
- 1 years of experience as an AI or ML Engineer
- Strong proficiency in Python
- Experience working with LLMs or conversational AI systems
- Familiarity with evaluation pipelines and production deployment
- Customer-focused mindset and startup experience a plus
