AI Agent Performance Engineer

About Basata

Location: Tempe, AZ or San Francisco, CA (in-office)

At Basata, we’re rethinking the way healthcare practices get work done. Our AI-powered tools help automate the repetitive, time consuming tasks that bog teams down, so staff can spend less time on admin and more time with patients.

We’re a small, fast-moving team that loves building smart systems and simple, intuitive experiences. If you're someone who gets excited about early-stage energy, designing better ways to work, and fixing what’s broken in healthcare, we’d love to connect.

About The Role

We’re looking for an AI Agent Performance Engineer to own the quality, reliability, and performance of Basata’s conversational agents. You’ll build the observability, QA, and evaluation systems that ensure our agents consistently meet accuracy, speed, and reliability benchmarks.

This role sits at the intersection of engineering, analytics, and applied AI. You’ll own and track the performance metrics that matter, detect drift or degradation early, and partner closely with Product and AI/ML engineering to drive measurable improvements in agentic behavior.

Your work will directly shape how Basata delivers high-quality AI automation to every practice we support.

What You’ll Do

• Build and maintain observability and monitoring systems that track agent latency, accuracy, call outcomes, call-flow behavior, and failure modes in production.
• Develop rigorous QA and regression testing frameworks for Voice AI agents, including automated evaluation pipelines and scripted test scenarios.
• Define, instrument, and own key performance metrics and ensure agents consistently meet or exceed them.
• Analyze large volumes of agent interactions to identify trends, misclassifications, drift, prompt issues, or reliability gaps.
• Partner closely with Product, AI/ML, and Deployment teams to convert findings into concrete product, prompt, and architecture improvements.
• Build internal dashboards, alerts, and reports that make agent performance transparent for internal teams and customers.
• Conduct root-cause investigations for production issues and drive sustainable, long-term fixes.
• Document QA processes, performance baselines, and best practices to support fast scaling across specialties.
  
  

What We’re Looking For

• 3–5 years of experience in software engineering and Applied AI.
• Hands-on experience building, deploying, and maintaining LLM-based agentic applications in production is essential.
• Strong working knowledge of agentic AI frameworks
• Experience building observability, monitoring, or QA systems using logs, metrics, dashboards, or automated evaluation pipelines.
• Proficiency with Python, analytics tooling, and comfort working with large-scale data logs.
• Experience in fast-paced, zero-to-one environments and comfort operating with autonomy.
  
  

Why Join Basata?

• Drive real impact. Your work will help clinics run more smoothly and ensure patients get the care they need—faster and with fewer headaches.
• Shape something meaningful. From early product decisions to UI details, you'll play a big role in crafting both the code and the overall user experience.
• High ownership. Join a team where you’re trusted to lead, build, think critically, and bring ideas to life.
• Work with purpose. We’re not here to throw more tech at the wall, we're solving real problems in healthcare with tools that people rely on every day.
  
  

Our Core Values

• Find a way: We own outcomes—no excuses, no shortcuts. When obstacles appear, we remove them. When the path isn’t clear, we forge one. We take extreme ownership, stay scrappy, and do what it takes to get the job done. Whatever the challenge, we find a way.
• Dream big: We refuse to accept that healthcare has to be broken. We dream boldly and act deliberately to build a future where healthcare organizations are free to focus on what truly matters: caring for people. We challenge what’s possible, rethinking every mundane barrier standing in the way of better care. Our vision is ambitious—some might call it impossible—but we believe patients and providers deserve nothing less.
• Obsess over the customer: We put our customers at the center of everything we do. We anticipate their needs, solve their problems, and design experiences that make them say, “Wow, I love this.” Every decision starts with empathy: How will this help them? How will they use it? We make our customers the hero—while never compromising on ethics, privacy, or trust.
• Always be learning: We’re relentless about growth—our own and each other’s. We challenge each other to level up, not just in skill, but in mindset, awareness, and character. We attract and empower “A” players, raise the bar, and multiply the impact of those around us. We’re learning machines: always reading, asking, improving. Because when we grow as people, we become better teammates, leaders, and humans - that lifts the whole team with us.
• Move Fast. Deliver Excellence. We prioritize rapid execution without sacrificing quality. Speed unlocks momentum—but only when paired with precision. We test relentlessly, ship often, and iterate fast to drive the results we want. Whether it’s responding to a message or launching a product, we take bold action, measure impact, and refine quickly. Velocity is our competitive edge—excellence is our standard.