What are the responsibilities and job description for the Principal, AI Platform Engineering position at Scotiabank - Global Banking and Markets?
Requisition ID: 262692
Please note that the Salary Range shown is a guideline only. Salary offered may vary based on factors, including, but not limited to, the successful candidate’s relevant knowledge, skills, and experience.
Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.
Global Banking and Markets
Global Banking and Markets (GBM) is a leading Canadian Capital Markets and Investment Banking business with a growing platform in the US and Latin America, operating globally for over 100 years. Scotiabank’s strong U.S. presence provides our clients an important bridge to this key global market for trade and investment flows across the Americas and the world.
Global Banking & Markets provides a full range of investment banking, credit and risk management products and services relevant to the financing and strategic development needs of our clients. Our products include debt and equity financing, mergers & acquisitions, corporate banking, institutional equity sales, trading and research, fixed income products, derivatives, energy, foreign exchange and precious & metals. We also cross-sell the full range of wholesale products and services offered by the Scotiabank Group.
Be part of an innovative, Global Capital Markets and Investment Banking business with a unique geographic footprint that puts capital to work for our clients across industries! We work together to drive ambition for every future!
Purpose
The Principal AI Platform Engineer is a key contributor within AI Platform Engineering and serves as the technical authority and architectural steward for Scotiabank’s enterprise AI platform. Operating as the technical counterpart to the Vice President, AI Platform, this role defines and defends the core architectural decisions, engineering standards, and reference implementations that underpin all AI and agentic workloads across the Bank. In a highly regulated, Tier‑1 financial institution, the role balances deep hands‑on engineering with enterprise‑wide architectural leadership, ensuring the AI platform is secure, resilient, compliant, and scalable across multiple cloud providers. Architecture decisions made in this role establish the mandatory baseline for engineering teams across AI and Agentic Engineering, Cloud Platform, Security Engineering, Data Platform, and application teams that build on or integrate with the AI platform fabric. The role requires sustained hands‑on contribution to production systems, authorship of binding standards and reference implementations, and direct engagement with executive leadership, regulators, and strategic technology partners.
What You’ll Do
If your experience is closely related but doesn’t align perfectly with every qualification, we do encourage you to apply - you might be the right candidate for this or other roles at Scotiabank!
At Scotiabank, every employee is empowered to reach their fullest potential, respected for who they are and, embraced for their differences. That’s why we work to grow and diversify talent and engage employees in a performance-oriented culture.
What's in it for you?
Scotiabank wants you to be able to bring your best self to work – and life, every day. With a focus on holistic well-being, our many flexible benefit programs are designed to help support your unique family, financial, physical, mental, and social health needs.
#GBM
Location(s): United States : Texas : Dallas
Scotiabank is a leading bank in the Americas. Guided by our purpose: "for every future", we help our customers, their families and their communities achieve success through a broad range of advice, products and services, including personal and commercial banking, wealth management and private banking, corporate and investment banking, and capital markets.
At Scotiabank, we value the unique skills and experiences each individual brings to the Bank, and are committed to creating and maintaining an inclusive and accessible environment for everyone. If you require accommodation (including, but not limited to, an accessible interview site, alternate format documents, ASL Interpreter, or Assistive Technology) during the recruitment and selection process, please let our Recruitment team know. If you require technical assistance, please click here. Candidates must apply directly online to be considered for this role. We thank all applicants for their interest in a career at Scotiabank; however, only those candidates who are selected for an interview will be contacted.
Scotiabank is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other characteristic protected by federal, state, or local law.
Please note that the Salary Range shown is a guideline only. Salary offered may vary based on factors, including, but not limited to, the successful candidate’s relevant knowledge, skills, and experience.
Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.
Global Banking and Markets
Global Banking and Markets (GBM) is a leading Canadian Capital Markets and Investment Banking business with a growing platform in the US and Latin America, operating globally for over 100 years. Scotiabank’s strong U.S. presence provides our clients an important bridge to this key global market for trade and investment flows across the Americas and the world.
Global Banking & Markets provides a full range of investment banking, credit and risk management products and services relevant to the financing and strategic development needs of our clients. Our products include debt and equity financing, mergers & acquisitions, corporate banking, institutional equity sales, trading and research, fixed income products, derivatives, energy, foreign exchange and precious & metals. We also cross-sell the full range of wholesale products and services offered by the Scotiabank Group.
Be part of an innovative, Global Capital Markets and Investment Banking business with a unique geographic footprint that puts capital to work for our clients across industries! We work together to drive ambition for every future!
Purpose
The Principal AI Platform Engineer is a key contributor within AI Platform Engineering and serves as the technical authority and architectural steward for Scotiabank’s enterprise AI platform. Operating as the technical counterpart to the Vice President, AI Platform, this role defines and defends the core architectural decisions, engineering standards, and reference implementations that underpin all AI and agentic workloads across the Bank. In a highly regulated, Tier‑1 financial institution, the role balances deep hands‑on engineering with enterprise‑wide architectural leadership, ensuring the AI platform is secure, resilient, compliant, and scalable across multiple cloud providers. Architecture decisions made in this role establish the mandatory baseline for engineering teams across AI and Agentic Engineering, Cloud Platform, Security Engineering, Data Platform, and application teams that build on or integrate with the AI platform fabric. The role requires sustained hands‑on contribution to production systems, authorship of binding standards and reference implementations, and direct engagement with executive leadership, regulators, and strategic technology partners.
What You’ll Do
- Leads and drives a customer focused culture throughout their team to deepen client relationships and leverage broader Bank relationships, systems and knowledge.
- Make and defend architecture decisions for the AI Platform: the Apigee X LLM Router, the MCP Control Gateway, the Cross-Cloud Serving Bridge, model-serving topology across Vertex AI / Azure OpenAI (APIM) / Databricks Mosaic AI, the shared vector search platform (Vertex AI Vector Search, Cloud SQL pgvector), GPU compute fleet strategy on GKE, observability federated to Splunk, and the platform API contract. Author the binding ADRs; defend them at the Architecture Review Board, Cyber Architecture Council, Enterprise Architecture forums, and OSFI engagements with empirical evidence, benchmarks, proofs-of-concept, RFCs, and update or reverse positions in writing when the evidence changes. Hold the line on standards in the face of business-line pressure when the right answer requires it.
- Operate as the VP, AI Platform's technical counterpart. Provide the authoritative engineering position behind every executive decision the VP takes, multi-year platform thesis, gateway evolution, GPU and accelerator investment, vector platform selection, Cross-Cloud Bridge capacity, observability backend (Splunk Observability vs. Datadog), NHI tooling (CyberArk vs. SailPoint), Pillar-2 surface choice, build-vs-buy across the platform stack. Pair the VP's organisational authority with the technical credibility executives, examiners, and engineers trust without needing to verify.
- Stay hands-on. Continue to ship production code on platform services, Go control-plane and gateway services, Apigee policy chains, Envoy filters on the Cross-Cloud Bridge, MCP Control Gateway handlers, Terraform modules, Helm charts, OpenTelemetry instrumentation. Personally author the reference implementations that anchor the standards rather than delegating them. Conduct deep code reviews on the highest-consequence platform PRs. Spend time in production with on-call engineers during incidents, not just in postmortems. Engineering credibility is non-negotiable; this role loses authority the moment it stops shipping.
- Set enterprise-wide best practices, not just function-wide. Standards authored here are adopted by adjacent platform organisations across the bank, Cloud Platform Engineering, Security Engineering, Data Platform, Application Engineering, and business-line CIO platforms, through Architecture Review Board representation and through reference implementations these teams reuse rather than rebuild. Author and own the bank-wide playbooks that consume this role's authority: Go service standards for control-plane work, gateway design patterns, MCP server publication standards, Workload Identity Federation patterns, OpenTelemetry instrumentation conventions, FinOps tagging contracts, and the platform API design guide.
- Author and maintain the function-wide AI Platform ADRs and standards: gateway fabric (Apigee X / APIM AI Gateway / Mosaic AI Gateway / MCP Control Gateway), Cross-Cloud Bridge, compute infrastructure (GPU fleet sizing, GKE cluster architecture, node-pool strategy, sandboxed code-execution for agent-generated code), LLM serving (framework selection, caching layer design, fallback routing across Vertex / Azure OpenAI / Mosaic AI), vector search (index architecture, multi-tenancy, freshness guarantees), and AI security controls (gateway-level PII handling via DLP / Model Armor / Content Safety / Prompt Shields, audit logging for regulatory examination).
- Translate OSFI E-23 (model risk), B-13 (technology and cyber risk), B-10 (third-party risk), PIPEDA, AIDA, and Quebec Law 25 requirements into concrete platform infrastructure controls, implemented at the Apigee X LLM Router, the MCP Control Gateway, the Cross-Cloud Bridge, and the GKE walled garden, and defend that translation directly to OSFI examiners, Internal Audit, Cyber Risk, and Model Risk Management.
- Own the AI Platform security architecture: walled-garden default-deny VPC-SC perimeter, zero-trust network design with service mesh and mTLS by default for multi-tenant AI workloads on GKE, Workload Identity Federation (OAuth 2.1 / SPIFFE-style) between Microsoft Entra ID (with per-agent Entra Agent ID) and GCP IAM, secrets management strategy in Secret Manager, sandboxed agent code-execution boundaries, and the security review gate for new platform components.
- Lead enterprise vendor relationships on the technology side: design-partner engagement with Google (Vertex AI, Apigee X, GKE), Microsoft (Azure OpenAI, APIM, Entra), Databricks (Mosaic AI, Mosaic AI Gateway), Anthropic, NVIDIA, and the safety / observability / FinOps vendors. Make build-vs-buy and product-vs-roadmap calls that bind the platform for years.
- Lead evaluation and adoption decisions for new AI infrastructure technologies: gateway products, MCP server registries, serving frameworks, vector databases, observability platforms, GPU and accelerator architectures, confidential-compute options, and emerging GCP / Azure / Databricks AI services, designing the assessment methodology yourself and making the call.
- Mentor Staff and Senior AI Platform Engineers through architecture reviews, pairing on the hardest production problems, and deliberate development of their systems thinking and identify the next Principal-level talent in the function.
- Understand how the Bank’s risk appetite and risk culture should be considered in day-to-day activities and decisions.
- Creates an environment in which their team pursues effective and efficient operations of their respective areas in accordance with Scotiabank’s Values, its Code of Conduct and the Global Sales Principles, while ensuring the adequacy, adherence to and effectiveness of day-to-day business controls to meet obligations with respect to operational, compliance, AML/ATF/sanctions and conduct risk.
- Builds a high performance environment and implements a people strategy that attracts, retains, develops and motivates their team by fostering an inclusive work environment and using a coaching mindset and behaviours; communicating vison/values/business strategy; and, managing succession and development planning for the team.
- 10 years of experience in platform engineering, infrastructure architecture, or site reliability engineering at enterprise scale.
- Demonstrated experience as a senior technical authority setting AI, ML, or large‑scale platform architecture standards adopted across multiple engineering organizations.
- Current, hands‑on production engineering experience, including authorship of systems‑level services in Go or a comparable language.
- Deep expertise in multi‑cloud AI platform architecture, LLM gateways, GPU‑based infrastructure, and large‑scale distributed systems.
- Proven ability to translate regulatory, privacy, and technology risk requirements into concrete, auditable platform controls within a regulated financial institution.
- Exceptional written and verbal communication skills, with the ability to influence VP‑ and SVP‑level stakeholders through clear, evidence‑based technical positions.
- Experience operating within a Tier‑1 financial institution or similarly regulated environment
- Direct engagement with financial regulators and internal audit functions on technology and model risk matters.
- Experience influencing cloud service provider and AI vendor product roadmaps through strategic partnerships
If your experience is closely related but doesn’t align perfectly with every qualification, we do encourage you to apply - you might be the right candidate for this or other roles at Scotiabank!
At Scotiabank, every employee is empowered to reach their fullest potential, respected for who they are and, embraced for their differences. That’s why we work to grow and diversify talent and engage employees in a performance-oriented culture.
What's in it for you?
Scotiabank wants you to be able to bring your best self to work – and life, every day. With a focus on holistic well-being, our many flexible benefit programs are designed to help support your unique family, financial, physical, mental, and social health needs.
#GBM
Location(s): United States : Texas : Dallas
Scotiabank is a leading bank in the Americas. Guided by our purpose: "for every future", we help our customers, their families and their communities achieve success through a broad range of advice, products and services, including personal and commercial banking, wealth management and private banking, corporate and investment banking, and capital markets.
At Scotiabank, we value the unique skills and experiences each individual brings to the Bank, and are committed to creating and maintaining an inclusive and accessible environment for everyone. If you require accommodation (including, but not limited to, an accessible interview site, alternate format documents, ASL Interpreter, or Assistive Technology) during the recruitment and selection process, please let our Recruitment team know. If you require technical assistance, please click here. Candidates must apply directly online to be considered for this role. We thank all applicants for their interest in a career at Scotiabank; however, only those candidates who are selected for an interview will be contacted.
Scotiabank is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other characteristic protected by federal, state, or local law.