Staff Software Engineer, Machine Learning

Title of Role: Staff Software Engineer, Machine Learning
Location: Burlingame, CA (On-site, 4 days/week)
Company Stage of Funding: Venture-Backed (Growth Stage)
Office Type: On-site (4 days per week)
Salary: $145,000 – $250,000 base equity benefits

Company Description

Our client is a fast-growing AI company transforming how healthcare operations run by building production-grade AI systems that automate high-volume, mission-critical workflows. Their technology powers conversational AI agents that handle complex administrative processes — from insurance verifications to claims follow-ups — directly improving efficiency and outcomes for healthcare organizations nationwide.

The company combines deep AI expertise with healthcare domain knowledge, backed by top-tier investors and a rapidly scaling team. This is an opportunity to join a technical organization that’s applying cutting-edge LLM systems to real-world healthcare challenges — not just prototypes, but deployed systems that matter.

What You Will Do

As a Staff Software Engineer, Machine Learning, you will architect, build, and scale backend systems that power production LLM applications in healthcare. You’ll take ownership of core infrastructure that enables reliability, security, and performance for mission-critical AI workloads.

Key Responsibilities:

• Architect scalable APIs and backend services that wrap, orchestrate, and optimize LLM workflows for healthcare applications.

• Design and implement retrieval-augmented generation (RAG) pipelines and data ingestion systems to ground models in structured healthcare data.

• Develop observability and reliability frameworks — including monitoring, cost optimization, and safety guardrails for deployed LLMs.

• Optimize infrastructure performance through caching, batching, and distributed scaling across cloud and containerized environments.

• Ensure compliance and security by implementing HIPAA-ready infrastructure and robust data governance.

• Collaborate cross-functionally with ML engineers, product leaders, and healthcare domain experts to deliver reliable AI systems to production.

• Mentor peers and influence technical direction, establishing engineering best practices for high-scale AI applications.

Ideal Candidate Background

• 5 years of backend or full-stack software engineering experience.

• 3 years of hands-on experience building or scaling ML/LLM-enabled systems.

• Strong programming skills in Python, plus experience with one statically typed language (Go, Java, or TypeScript).

• Deep understanding of distributed systems, API design, and microservice architecture.

• Experience with LLM frameworks such as LangChain, Hugging Face, LlamaIndex, or OpenAI/Anthropic APIs.

• Cloud-native expertise with AWS, GCP, or Azure, and infrastructure tools like Kubernetes, Docker, and Terraform.

• Familiarity with MLOps or LLMOps — CI/CD for models, evaluation harnesses, and observability.

• Strong system design skills and an ability to align technical architecture with product and business goals.

Preferred Qualifications

• Experience deploying AI systems in healthcare or other regulated environments (HIPAA, FHIR, HL7).

• Hands-on experience with vector databases, RAG pipelines, or structured-output orchestration.

• Prior work on mission-critical, high-availability SaaS platforms.

• Knowledge of responsible AI practices, including model safety and data privacy.

Compensation, Benefits, and Other Details

• Base Salary: $145,000 – $250,000, depending on experience and qualifications.

• Equity: Competitive stock options package.

• Benefits: Comprehensive health, dental, and vision coverage; flexible PTO; and opportunities for professional growth and leadership.

• Work Environment: On-site in Burlingame, CA (4 days/week) with a highly collaborative, senior-only engineering team.

• Impact: Your systems will power real-world AI solutions that make healthcare more efficient, accurate, and human-centered.