AI/ML Research Scientist, LLM Post-Training & Evaluation

Job Description

Job Description

Research Scientist, LLM Evaluation & Post-Training

Company: Centific

Location: Palo Alto, CA or Seattle, WA (Hybrid/Remote)

Type: Full-time

Salary: $150K - $160K Annually

About The Role

Centific is seeking a Research Scientist focused on LLM evaluation and post-training. This role involves defining and executing research agendas, developing evaluation frameworks, analyzing model behavior, and collaborating with cross-functional teams and customer stakeholders. The goal is to improve LLM evaluation methodologies and drive advancements in AI deployment.

Key Responsibilities

• Define and execute research on LLM evaluation and post-training.
• Develop and validate comprehensive evaluation frameworks for LLM and multimodal systems.
• Lead research in frontier evaluation domains (long-context, cross-modal, dynamic multi-turn).
• Analyze model behavior and provide recommendations for improvement.
• Collaborate with data scientists and ML engineers on evaluation and training pipelines.
• Engage with customer technical stakeholders to understand evaluation goals and provide recommendations.
• Contribute to knowledge creation through datasets, frameworks, reports, and publications.
• Promote thought leadership in LLM evaluation and post-training.
  
  

Required Qualifications

• MS or PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or related quantitative field (PhD preferred).
• 5 years of relevant experience in applied ML research, with substantial work in LLMs or foundation models.
• Demonstrated experience with LLM evaluation, benchmarking, alignment, post-training, or model quality research.
• Strong foundation in experimental design, statistical analysis, and scientific reasoning for ML systems.
• Strong Python coding skills for research, data processing, and ML frameworks (PyTorch, Hugging Face, JAX/TensorFlow).
• Ability to evaluate and compare human and automated evaluation methods.
• Strong written and verbal communication skills.
  
  

Preferred Qualifications

• Hands-on experience with fine-tuning or post-training experiments (SFT, preference optimization, RLHF/RLAIF).
• Experience with multimodal and long-context evaluation.
• Experience designing multi-turn, interactive, or agentic evaluation protocols.
• Publications or open-source contributions in LLM evaluation at top venues.
• Experience in customer-facing applied research or technical consulting.
• Familiarity with safety, trustworthiness, and governance in GenAI evaluation.