What are the responsibilities and job description for the LLM & RL Engineer | Agentic Engineering position at YC Bench?
About YC Bench
YC Bench is a live benchmark designed to forecast the top-performing Y Combinator startups at Demo Day. We combine real-world startup data with advanced AI to predict which early-stage companies will outperform their batch peers using short-term execution signals. Our mission is to make startup success measurable in months rather than years.
The Role
We are looking for a talented LLM & RL Engineer to help build and optimize the AI systems that power our forecasting platform. You will work at the intersection of large language models and reinforcement learning to create agentic systems capable of long-horizon reasoning, decision-making, and accurate prediction in uncertain environments.
Responsibilities
- Fine-tune, optimize, and deploy large language models (LLMs) for complex reasoning and forecasting tasks
- Design, implement, and scale reinforcement learning (RL) algorithms, including RLHF, RL from AI feedback, and agentic RL frameworks
- Build and improve LLM-based agents for simulation, planning, and multi-step decision making
- Develop robust machine learning pipelines for training, evaluation, and inference at scale
- Experiment with hybrid LLM RL approaches to enhance predictive accuracy and long-term performance
- Collaborate with the team to integrate models into the YC Bench platform and forecasting engine
- Stay up-to-date with the latest advancements in LLMs, RL, and agentic AI systems
Requirements
- Strong experience working with Large Language Models (fine-tuning, prompting, evaluation, and optimization)
- Solid background in Reinforcement Learning (policy optimization, value-based methods, actor-critic, RLHF, etc.)
- Proficiency in Python and modern ML frameworks (PyTorch, Hugging Face Transformers, vLLM, DeepSpeed, RL libraries such as TRL, Stable Baselines, or Ray RLlib)
- Experience building production-grade ML pipelines and handling large-scale training/inference
- Familiarity with agentic systems, long-horizon planning, or simulation environments is a big plus
- Passion for AI forecasting, decision-making under uncertainty, and real-world impact
Nice-to-Haves
- Experience with predictive modeling or time-series forecasting
- Background in startup analysis, venture capital, or early-stage company evaluation
- Publications or open-source contributions in LLMs or RL
- Comfort working in a fast-moving, early-stage environment
If you love pushing the boundaries of what LLMs and RL can do together — and want to apply cutting-edge AI to one of the most exciting prediction problems in tech — we'd love to hear from you.
YC Bench is a live benchmark designed to forecast the top-performing Y Combinator startups at Demo Day. We combine real-world startup data with advanced AI to predict which early-stage companies will outperform their batch peers using short-term execution signals. Our mission is to make startup success measurable in months rather than years.
The Role
We are looking for a talented LLM & RL Engineer to help build and optimize the AI systems that power our forecasting platform. You will work at the intersection of large language models and reinforcement learning to create agentic systems capable of long-horizon reasoning, decision-making, and accurate prediction in uncertain environments.
Responsibilities
- Fine-tune, optimize, and deploy large language models (LLMs) for complex reasoning and forecasting tasks
- Design, implement, and scale reinforcement learning (RL) algorithms, including RLHF, RL from AI feedback, and agentic RL frameworks
- Build and improve LLM-based agents for simulation, planning, and multi-step decision making
- Develop robust machine learning pipelines for training, evaluation, and inference at scale
- Experiment with hybrid LLM RL approaches to enhance predictive accuracy and long-term performance
- Collaborate with the team to integrate models into the YC Bench platform and forecasting engine
- Stay up-to-date with the latest advancements in LLMs, RL, and agentic AI systems
Requirements
- Strong experience working with Large Language Models (fine-tuning, prompting, evaluation, and optimization)
- Solid background in Reinforcement Learning (policy optimization, value-based methods, actor-critic, RLHF, etc.)
- Proficiency in Python and modern ML frameworks (PyTorch, Hugging Face Transformers, vLLM, DeepSpeed, RL libraries such as TRL, Stable Baselines, or Ray RLlib)
- Experience building production-grade ML pipelines and handling large-scale training/inference
- Familiarity with agentic systems, long-horizon planning, or simulation environments is a big plus
- Passion for AI forecasting, decision-making under uncertainty, and real-world impact
Nice-to-Haves
- Experience with predictive modeling or time-series forecasting
- Background in startup analysis, venture capital, or early-stage company evaluation
- Publications or open-source contributions in LLMs or RL
- Comfort working in a fast-moving, early-stage environment
If you love pushing the boundaries of what LLMs and RL can do together — and want to apply cutting-edge AI to one of the most exciting prediction problems in tech — we'd love to hear from you.