What are the responsibilities and job description for the Applied Scientist (Evaluation & Safety) position at Amigo?
About Amigo
Amigo builds trust and safety infrastructure for AI in mission-critical environments.
We partner with organizations in healthcare and other regulated sectors to deploy AI systems that operate reliably when the stakes are highest. Our infrastructure enables verification, monitoring, and real-time oversight—ensuring AI serves people safely at scale.
We've raised $6.5M from General Catalyst and GSV Ventures. Our team combines expertise in distributed systems, quantitative research, clinical operations, and regulatory environments to build AI that organizations can trust.
About This Role
As an Applied Scientist - Evaluation & Safety at Amigo, you'll build the evaluation frameworks that ensure our systems are safe and effective in healthcare environments. You'll design adversarial testing systems, create safety evaluation protocols, and build frameworks that catch potential failures before they reach patients. Your work is critical to maintaining the reliability and safety standards that healthcare organizations require.
What You'll Do
Health & Wellness
Amigo builds trust and safety infrastructure for AI in mission-critical environments.
We partner with organizations in healthcare and other regulated sectors to deploy AI systems that operate reliably when the stakes are highest. Our infrastructure enables verification, monitoring, and real-time oversight—ensuring AI serves people safely at scale.
We've raised $6.5M from General Catalyst and GSV Ventures. Our team combines expertise in distributed systems, quantitative research, clinical operations, and regulatory environments to build AI that organizations can trust.
About This Role
As an Applied Scientist - Evaluation & Safety at Amigo, you'll build the evaluation frameworks that ensure our systems are safe and effective in healthcare environments. You'll design adversarial testing systems, create safety evaluation protocols, and build frameworks that catch potential failures before they reach patients. Your work is critical to maintaining the reliability and safety standards that healthcare organizations require.
What You'll Do
- Design evaluation frameworks for healthcare systems
- Build adversarial testing systems that identify potential failure modes before deployment
- Create safety evaluation protocols that ensure systems meet healthcare standards
- Design test suites that cover edge cases, rare diseases, and complex medical scenarios
- Build automated evaluation pipelines that continuously assess system performance
- Collaborate with healthcare professionals to develop clinically relevant evaluation metrics
- Strong background in system safety and adversarial testing
- Experience with bias detection and fairness testing
- Knowledge of healthcare safety standards
- Understanding of statistical evaluation methods
- Track record of building evaluation systems for production software
- Healthcare or regulated industry experience
- Background in medical device testing or clinical trial design
- Experience with formal verification or safety-critical systems
- Knowledge of healthcare compliance and audit requirements
Health & Wellness
- Comprehensive health, dental, and vision insurance
- Mental health support and wellness coaching
- Flexible wellness stipend for fitness, therapy, or personal growth
- Daily catered lunch and dinner
- Annual learning budget for courses, books, or conferences
- Conference attendance budget for professional development
- Development setup of your choice
- Academic collaboration opportunities
Salary : $150,000 - $300,000
Applied Scientist, Amazon Connect
Amazon Development Center U.S., Inc. -
York, NY
Applied Scientist, Amazon Advertising
Amazon -
New York, NY
Staff Applied Scientist - Causal
Keystone AI -
New York, NY
