What are the responsibilities and job description for the Manager / Senior Manager, AI Safety Data and Evaluation Program position at Lila Sciences?
Your Impact at LILA
We're building a talent-dense, high-agency AI safety team at Lila that will engage all core teams within the organization (science, model training, lab integration, etc.), to identify and mitigate risks from scientific superintelligence while retaining the core scientific discovery capabilities of the stack. The initial focus of this team will be to build and implement a bespoke safety strategy for Lila, tailored to its specific goals and deployment strategies. This will involve technical safety strategy development, broader ecosystem engagement, safety-focused evaluations, safety systems to mitigate risks, and a research agenda that explores longer-term needs such as oversight of superintelligent scientific systems.
High-quality safety depends on the right expertise, consistent and repeatable workflows and high levels of interpretability. At Lila Sciences, you will turn scientific risk questions into well-run expert-data and evaluation programs to build this evidence-base.
You will bring a strong working understanding of AI safety and scientific misuse risks to connect internal risk mapping and threat modeling with expertise across Lila and external specialists. Your primary responsibilities will be program execution and data quality: organizing contributors, running red-teaming and evaluation campaigns, and ensuring their outputs meet agreed standards. You will also be responsible for gathering targeted high-impact feedback from partners in the AI Safety community on our different aspects of our approach to Safety. You will work closely with scientists who frame the research questions and interpret their significance, while creating the operational conditions for their expertise to have an impact. Ultimately, you will play a key role in generating the evidence that informs safety decisions made by senior leadership at Lila.
What You'll Be Building
Expected Base Salary Range: $180,000 USD - $256,000 USD
About LILA
Lila Sciences is building Scientific Superintelligence™ to solve humankind's greatest challenges. We believe science is the most inspiring frontier for AI. Rather than hard-coding expert knowledge into tools, LILA builds systems that can learn for themselves.
LILA combines advanced AI models with proprietary AI Science Factory™ instruments into an operating system for science that executes the entire scientific method autonomously, accelerating discovery at unprecedented speed, scale, and impact across medicine, materials, and energy. Learn more at www.lila.ai.
Guided by our core values of truth, trust, curiosity, grit, and velocity, we move with startup speed while tackling problems of historic importance. If this sounds like an environment you'd love to work in, even if you don't meet every qualification listed above, we encourage you to apply.
We’re All In
Lila Sciences is committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status.
Information you provide during your application process will be handled in accordance with our Candidate Privacy Policy.
A Note to Agencies
Lila Sciences does not accept unsolicited resumes from any source other than candidates. The submission of unsolicited resumes by recruitment or staffing agencies to Lila Sciences or its employees is strictly prohibited unless contacted directly by Lila Science’s internal Talent Acquisition team. Any resume submitted by an agency in the absence of a signed agreement will automatically become the property of Lila Sciences, and Lila Sciences will not owe any referral or other fees with respect thereto.
We're building a talent-dense, high-agency AI safety team at Lila that will engage all core teams within the organization (science, model training, lab integration, etc.), to identify and mitigate risks from scientific superintelligence while retaining the core scientific discovery capabilities of the stack. The initial focus of this team will be to build and implement a bespoke safety strategy for Lila, tailored to its specific goals and deployment strategies. This will involve technical safety strategy development, broader ecosystem engagement, safety-focused evaluations, safety systems to mitigate risks, and a research agenda that explores longer-term needs such as oversight of superintelligent scientific systems.
High-quality safety depends on the right expertise, consistent and repeatable workflows and high levels of interpretability. At Lila Sciences, you will turn scientific risk questions into well-run expert-data and evaluation programs to build this evidence-base.
You will bring a strong working understanding of AI safety and scientific misuse risks to connect internal risk mapping and threat modeling with expertise across Lila and external specialists. Your primary responsibilities will be program execution and data quality: organizing contributors, running red-teaming and evaluation campaigns, and ensuring their outputs meet agreed standards. You will also be responsible for gathering targeted high-impact feedback from partners in the AI Safety community on our different aspects of our approach to Safety. You will work closely with scientists who frame the research questions and interpret their significance, while creating the operational conditions for their expertise to have an impact. Ultimately, you will play a key role in generating the evidence that informs safety decisions made by senior leadership at Lila.
What You'll Be Building
- Translate agreed scientific briefs into executable plans covering tasks, contributor needs, instructions, milestones, quality standards and delivery expectations.
- Source and coordinate external experts with scientific colleagues, and organize access to relevant expertise within Lila.
- Manage contributor onboarding, task assignment, training and support so specialists can contribute effectively and consistently.
- Run expert-data collection, annotation, red-teaming and evaluation programs, tracking progress and resolving delivery problems.
- Establish quality controls covering completeness, consistency, provenance and adherence to rubrics; coordinate calibration, review, disagreement resolution and rework with scientific owners.
- Maintain clear records, versioned guidance and usable datasets, and improve workflows based on quality findings and researcher feedback.
- Identify questions that require scientific judgment and bring them to the appropriate specialists, ensuring decisions are reflected in program materials and subsequent work.
- Experience delivering expert-data, research-operations, evaluation or similarly demanding programs involving specialist contributors.
- A substantial working understanding of AI safety, including how threat models, capability evaluations, red teaming and safeguards inform one another. You should be able to interpret a safety research brief, recognize its assumptions and the consequences of data-quality failures, and identify when a question needs scientific escalation.
- Evidence of applying this safety understanding in research, evaluation or expert-data work, including how it shaped the scoping, execution or review of a program.
- Strong operational judgment and the ability to manage dependencies, changing requirements and several parallel workstreams.
- A track record of setting measurable quality standards, detecting recurring problems and improving the processes that produce data.
- Clear communication, careful documentation and the ability to build effective relationships with researchers, experts and vendors.
- Sufficient technical and scientific fluency to understand research briefs, clarify ambiguity and work productively with subject-matter experts.
- Experience contributing to scientific study design, assessing scientific validity or developing evaluation rubrics.
- Deeper scientific expertise or research experience in biology, chemistry or materials science.
- Data analysis, scripting, annotation platforms or tools for tracking research workflows and data provenance.
Expected Base Salary Range: $180,000 USD - $256,000 USD
About LILA
Lila Sciences is building Scientific Superintelligence™ to solve humankind's greatest challenges. We believe science is the most inspiring frontier for AI. Rather than hard-coding expert knowledge into tools, LILA builds systems that can learn for themselves.
LILA combines advanced AI models with proprietary AI Science Factory™ instruments into an operating system for science that executes the entire scientific method autonomously, accelerating discovery at unprecedented speed, scale, and impact across medicine, materials, and energy. Learn more at www.lila.ai.
Guided by our core values of truth, trust, curiosity, grit, and velocity, we move with startup speed while tackling problems of historic importance. If this sounds like an environment you'd love to work in, even if you don't meet every qualification listed above, we encourage you to apply.
We’re All In
Lila Sciences is committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status.
Information you provide during your application process will be handled in accordance with our Candidate Privacy Policy.
A Note to Agencies
Lila Sciences does not accept unsolicited resumes from any source other than candidates. The submission of unsolicited resumes by recruitment or staffing agencies to Lila Sciences or its employees is strictly prohibited unless contacted directly by Lila Science’s internal Talent Acquisition team. Any resume submitted by an agency in the absence of a signed agreement will automatically become the property of Lila Sciences, and Lila Sciences will not owe any referral or other fees with respect thereto.
Salary : $180,000