What are the responsibilities and job description for the Senior Software Engineer, AI position at Jobgether?
This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Senior Software Engineer, AI in Canada.
This role sits at the core of next-generation AI system development, focusing on how large language model-powered products are evaluated, improved, and deployed at scale. You will design and build robust AI evaluation frameworks and production-grade agent architectures that power intelligent features across complex product surfaces. Working in a highly technical, collaborative engineering environment, you will help define how AI performance is measured, trusted, and continuously optimized. The role blends hands-on systems engineering with deep AI/ML thinking, including LLM orchestration, retrieval systems, and observability. You will contribute to production AI systems while shaping the methodologies that determine model quality and reliability. This is a high-impact position where your work directly influences how AI behaves in real-world, user-facing applications.
- Design and implement end-to-end AI evaluation frameworks, including offline evaluation systems, production tracing, and human-in-the-loop feedback loops.
- Define and operationalize key AI performance metrics such as task success rate, hallucination detection, response quality, and business impact indicators.
- Build and maintain evaluation datasets, automated test harnesses, and regression detection pipelines for AI systems.
- Architect reusable agent systems, including multi-step workflows, LLM DAGs, conversational agents, and orchestration patterns.
- Develop and scale retrieval-augmented generation (RAG) systems, including vector database management and retrieval optimization.
- Integrate observability and monitoring tools for LLM-based systems to improve debugging, reliability, and performance tracking.
- Make informed technical decisions across LLM providers, frameworks, and tooling based on trade-offs in cost, latency, and reliability.
- Lead end-to-end delivery of AI engineering projects, from scoping through execution and deployment.
- Collaborate with engineering leadership to define AI system strategy and evaluation methodologies.
- Elevate engineering standards through code reviews, documentation, mentoring, and technical leadership.
Requirements:
- 5 years of software engineering experience, with significant exposure to production AI/ML systems.
- Strong hands-on experience with LLM-based systems, including prompt engineering, RAG pipelines, and agent orchestration.
- Proven experience building and operating agentic AI systems in production environments.
- Deep understanding of AI evaluation frameworks and metrics, including the ability to distinguish meaningful signals from vanity metrics.
- Strong Python engineering skills with a focus on clean, testable, and production-grade code.
- Experience with tools such as LangGraph or similar agent orchestration frameworks.
- Familiarity with LLM observability tools (e.g., LangSmith or equivalents).
- Experience designing and working with vector databases and retrieval systems (e.g., Pinecone or similar).
- Strong understanding of cloud infrastructure (AWS, GCP, or equivalent), including serverless and event-driven architectures.
- Solid communication skills with the ability to discuss complex AI concepts across technical and non-technical audiences.
- Nice to have: experience with RLHF, LoRA, MLOps tooling, traditional ML, or open-source AI contributions.
Benefits:
- Competitive compensation with an estimated annual salary range of CAD $160,000–$180,000.
- Equity participation through stock options, subject to company plan terms.
- Comprehensive health benefits including medical, dental, life, disability, and AD&D coverage.
- Retirement savings plan and financial planning support.
- Paid time off, including holidays, sick leave, and generous parental leave.
- Remote-friendly work environment with home office and wellness stipends.
- Learning and development budget to support continuous skill growth.
- Wellness and mental health support programs and applications.
- Inclusive and accessible workplace with accommodations available upon request.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1