What are the responsibilities and job description for the AI Product Manager position at JSG (Johnson Service Group, Inc.)?
Our client is seeking an AI Product Manager to lead end-to-end execution of enterprise GenAI and RAG-based product capabilities, improving knowledge ingestion, retrieval quality, grounded answer generation, enterprise chat experiences, guardrails, observability, and agentic automation rollout. (Remote, US)
Must Have Skills
Must Have Skills
- Experience building and shipping enterprise-grade GenAI products, particularly RAG-based assistants or enterprise knowledge assistants.
- Strong understanding of the end-to-end RAG lifecycle, including data ingestion, document parsing, chunking, metadata enrichment, embeddings, indexing, retrieval, grounding, citations, and response synthesis.
- Ability to partner with engineering teams on retrieval tuning, hybrid search, re-ranking, query understanding, prompt structure, grounding, hallucination control, and answer-quality tradeoffs.
- Experience defining requirements, acceptance criteria, release scope, quality bars, and product validation processes for technically complex products.
- Strong execution and delivery management skills, with the ability to drive a defined roadmap across multiple technical teams and dependencies.
- Experience owning product testing and business validation before release, including scenario coverage, regression checks, readiness signoff, and definition of done.
- Understanding of guardrails and responsible AI for enterprise assistants, including content safety, PII handling, topic boundaries, confidence signaling, access control, and audit trails.
- Understanding of observability and monitoring for LLM products, including trace-level visibility, query lifecycle analysis, latency, token usage, drift, alerting, rollback, and user feedback instrumentation.
- Excellent stakeholder management skills across business users, end-user teams, engineering, platform, security, and leadership.
- Strong communication skills, including the ability to explain AI quality, trust, retrieval issues, and guardrails in business-friendly language.
- Familiarity with cloud-based GenAI architecture, including managed LLM services, vector/search indexes, object storage, workflow orchestration, serverless compute, containerized services, and enterprise integration patterns.
- Working knowledge of LLM capabilities and limitations, model tradeoffs, prompt architecture, and prompt vulnerabilities such as injection and jailbreak risks.
- Exposure to agentic AI concepts, including tool use, function calling, multi-step reasoning, planning, human-in-the-loop approvals, and safe autonomous execution.
- Experience running UAT, gathering nuanced user feedback, and converting findings into measurable product improvements.
- Experience supporting rollout of agentic automation use cases for lower-complexity workflows with appropriate safety controls, approval flows, and auditability.
- Own end-to-end product execution across knowledge ingestion, retrieval quality, grounded answer generation, enterprise chat experience, guardrails, observability, and agentic use case rollout.
- Work closely with end users to understand pain points, validate use cases, and ensure the product improves productivity and trust.
- Partner with engineering teams to clarify requirements, resolve functional gaps, and drive delivery against defined milestones.
- Own product testing and business validation before each release, including scenario coverage, acceptance criteria validation, regression checks, and readiness signoff.
- Partner with technical teams to improve RAG pipeline quality, including ingestion readiness, chunking and indexing implications, retrieval tuning, grounding, citation behavior, and answer quality.
- Define and own the evaluation framework, including metrics for retrieval relevance, answer correctness, faithfulness, hallucination rate, citation accuracy, fallback handling, and latency.
- Ensure each test run is measured against established baselines and quality expectations.
- Maintain product-level feedback loops using golden datasets, user feedback, UAT findings, and post-release learning.
- Drive observability requirements to monitor end-to-end query behavior, retrieval outcomes, response quality, latency, errors, drift, and user feedback signals.
- Participate in product decisions related to cloud-based GenAI architecture, including ingestion pipeline optimization and model usage / cost-quality balance.
- Support rollout of agentic automation use cases while ensuring safety controls, approval flows, and auditability are in place.
- Role focus includes enterprise GenAI, RAG-based assistants, enterprise knowledge assistants, LLM product quality, responsible AI, observability, and agentic automation.
- Key product areas include knowledge ingestion, retrieval, grounded generation, citations, guardrails, enterprise chat, evaluation frameworks, UAT, release readiness, and post-release improvement.
- The role requires close collaboration with business users, end-user teams, engineering, platform, security, and leadership stakeholders.