Staff Data Scientist: AI Evaluation & Context Systems

The Opportunity

Demand.io is a profitable, founder-led consumer AI commerce player driving $1B in annual GMV. We are the team behind SimplyCodes (the #1 AI-powered savings tool) and Product.ai (emerging Axiomatic Intelligence shopping assistant).

We are seeking a Staff Data Scientist to operate as the lead architect for our Evaluation & Context Systems. This is not a standard role; it is a high-leverage position for a builder who wants to define the mathematical standard of "Truth" for autonomous commerce agents.

Most Data Science today is probabilistic guessing. We are building the antidote: Deterministic Intelligence. We need you to architect the "Judge"—the rigorous evaluation system that measures the epistemic validity of our AI agents and ensures they adjudicate reality rather than hallucinate it.

What You Will Build

You will not be writing tickets for minor features. You will be architecting the strategic engines of the company.

1. The Evaluation Architecture (Benchmarking Truth): Standard benchmarks (MMLU) are useless for commerce. You will architect the Proprietary Evaluation Harness for our ecosystem. You will design the "Golden Sets" and adversarial loops that measure our AI's ability to distinguish a marketing claim from a verified fact. You define the metrics that prevent us from shipping hallucinations.
2. Neuro-Symbolic Retrieval (GraphRAG): Vector search finds "similar" text, not "true" facts. You will move us beyond simple RAG. You will architect a Neuro-Symbolic Retrieval System that fuses our massive Commerce Knowledge Graph with vector search. You ensure the Agent retrieves the logic of the entity, not just the semantics of the text.
3. The Economics of Intelligence: A 99% accurate model that costs $1.00 per query is a failure. You will own the Unit Economics of Intelligence. You will model the trade-offs between expensive "Neural Reading" and efficient "Symbolic Logic," optimizing our stack for Risk-Adjusted Value per Token.

Who You Are

We do not care about your pedigree; we care about your craft.

• You are a Scientific Engineer. You don't just "run experiments"; you design rigorous Protocols. You understand statistical significance, p-hacking, and the danger of averages.
• You obsess over "The Tails." You are not satisfied with 90% accuracy. You dig into the distribution tails to understand exactly why the 10% failed.
• You are an Anti-Academic. You do not want to publish papers. You want to ship systems that survive contact with 10 million users.
• You treat Evaluation as Code. You automate the generation of test cases. You don't write 1,000 unit tests; you write the Generator that creates them.

The Contract

We operate as a high-performance studio, not a typical corporation. We share the upside directly with the builders.

• Base Salary: $250,000 – $380,000. We target the 80th percentile to attract 1-of-1 talent. Your base covers your life; your equity builds your freedom.
• Profit Participation (PIUs): You aren't just an employee; you are a partner. We grant Profits Interest Units (PIUs), giving you true ownership and a share of the company's operating profit. You earn real cash flow distributions as we grow—not just at a theoretical exit—all within a structure designed for tax efficiency to maximize your take-home wealth.
• Benefits: 100% premium coverage for you and your family, daily catered lunches, and unlimited PTO that we actually expect you to use to recharge.

How to Apply

We don't use cover letters or ATS keyword bots. We measure logic, not keywords.

The Process

1. The Input: Instead of scheduling a 30-minute phone screen, you will answer 5 short questions via video (Async).
2. The Context: These are high-signal prompts designed to test how you prioritize and compress complexity. No rehearsal is required. We value clear thinking over production value.
3. The Trade: If your signal is clear, you skip the generic recruiter screen and move straight to a peer-level conversation with the team.