AI Experience Policy Specialist

About The Role

The GenAI Manual Evaluations team serve as Uber's human-in-the-loop quality, safety, and risk gate for AI deployments that will engage with our users. The team plays a critical role in ensuring AI meets a human bar and that our products can scale with consistent conformance, as well as prevent downstream impact by surfacing CX, policy, and financial risks before customer impact.

The GenAI Manual Evaluations team is responsible for conducting end-to-end reviews of customer and agent interactions with Uber's AI-powered support tools. We assess the full experience-from product behavior and agent responses to language quality, translations, policy accuracy, and knowledge base alignment.

Our goal is to establish a scalable, continuous review process that ensures the quality of AI-driven support solutions and enables the seamless integration of new features. By identifying experience gaps and areas for improvement, we deliver actionable insights that directly inform product, engineering, and operations roadmaps.

What You'll Do

• Execute rubric-based evaluations on real interactions between Uber's users (earners, riders, merchants) and our GenAI support products.
• Assess the accuracy, relevance, and overall customer experience of the responses provided by AI.
• Identify defects, edge cases, and gaps in the support experience and document findings into relevant and actionable insights for cross functional teams to action.
• Support the calibration and continuous improvement of various AI programs as we continue to scale to a larger audience.
• Become an SME of multiple tools & programs to provide comprehensive evaluations of varying interaction types.
• Communicate quickly and effectively to internal Uber members as needed
• Become an expert on processes and may help with sophisticated problem solving
• Follow process guides to complete tasks and provide support to various crucial initiatives
• Work with your Team Lead and the Manual Evals Program team to escalate suggestions and issues for timely resolution.
• Serve as the policy SME for AI support experiences-ensuring automated outputs align with current policies and support standards
  
  

Basic Qualifications

• Ability to distill complex technical processes into clear, usable content
• Strong understanding of support operations, AI tools, and agent workflows
• Strong expertise in policy operations or support policy frameworks
• A proactive, solutions-oriented mindset with attention to operational detail
• Strong technical aptitude and willingness to quickly learn and manage new internal tools
  
  

Preferred Qualifications

• Experience with escalation workflows, SOP development, or policy training programs.
• Experience coordinating cross-functional initiatives and working with project management tools like Jira.
• Familiarity with AI-driven tooling or similar automated support environments.
• Proven track record of improving communication flows in tech, support, or automation environments
  
  

For Phoenix, AZ-based roles: The base hourly rate amount for this role is USD$21.70 per hour. You will be eligible to participate in Uber's bonus program, and may be offered other types of comp. All full-time employees are eligible to participate in a 401(k) plan. You will also be eligible for various benefits. More details can be found at the following link https://jobs.uber.com/en/benefits.