Engineering Manager, Inference Platform

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. Our novel wafer-scale architecture provides the AI compute power of dozens of GPUs on a single chip, with the programming simplicity of a single device. This approach allows Cerebras to deliver industry-leading training and inference speeds and empowers machine learning users to effortlessly run large-scale ML applications, without the hassle of managing hundreds of GPUs or TPUs.  

Cerebras' current customers include global corporations across multiple industries, national labs, and top-tier healthcare systems. In January, we announced a multi-year, multi-million-dollar partnership with Mayo Clinic, underscoring our commitment to transforming AI applications across various fields. In August, we launched Cerebras Inference, the fastest Generative AI inference solution in the world, over 10 times faster than GPU-based hyperscale cloud inference services.

Location: Toronto / Sunnyvale 

We're looking for a deeply technical, hands-on engineering leader for our Inference Service Platform. You will lead a high performing team to tackle a critical challenge: scaling LLM inference on Cerebras’ advanced compute clusters and delivering a world-class, on-prem solution for enterprise customers. In this role, you’ll set the technical vision while staying close to the code, architecting highly reliable, low latency distributed systems. If you have proven expertise in distributed systems and scaling modern model-serving frameworks, we want to hear from you. 

Responsibilities 

• Provide hands-on technical leadership, owning the technical vision and roadmap for the Cerebras Inference Platform, from internal scaling to on-prem customer solutions. 

• Lead the end-to-end development of distributed inference systems, including request routing, autoscaling, and resource orchestration on Cerebras' unique hardware. 

• Drive a culture of operational excellence, guaranteeing platform reliability (>99.9% uptime), performance, and efficiency. 

• Lead, mentor, and grow a high-caliber team of engineers, fostering a culture of technical excellence and rapid execution. 

• Productize the platform into an enterprise-ready, on-prem solution, collaborating closely with product, ops, and customer teams to ensure successful deployments. 

Skills & Qualifications 

• Technical Leadership: 6 years in high-scale software engineering, with 3 years leading distributed systems or ML infra teams; strong coding and review skills. 

• Inference Expertise: Proven track record scaling LLM inference: optimizing latency (<100ms P99), throughput, batching, memory/IO efficiency and resources utilization. 

• ML Systems Knowledge: Expertise in distributed inference/training for modern LLMs; understanding of AI/ML ecosystems, including public clouds (AWS/GCP/Azure). 

• Frameworks & Tools: Hands-on with model-serving frameworks (e.g. vLLM, TensorRT-LLM, Triton or similar) and ML stacks (PyTorch, Hugging Face, SageMaker).  

• Infrastructure: Deep experience with orchestration (Kubernetes/EKS, Slurm), large clusters, and low-latency networking. 

• Operations & Monitoring: Strong background in monitoring and reliability engineering (Prometheus/Grafana, incident response, post-mortems). 

• Leadership & Collaboration: Demonstrated ability to recruit and retain high-performing teams, mentor engineers, and partner cross-functionally to deliver customer-facing products. 

Preferred Skills 

• Experience with on-prem/private cloud deployments. 

• Background in edge or streaming inference, multi-region systems, or security/privacy in AI. 

• Customer-facing experience with enterprise deployments. 

Why Join Cerebras

People who are serious about software make their own hardware. At Cerebras we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection  point in our business. Members of our team tell us there are five main reasons they joined Cerebras:

1. Build a breakthrough AI platform beyond the constraints of the GPU.
2. Publish and open source their cutting-edge AI research.
3. Work on one of the fastest AI supercomputers in the world.
4. Enjoy job stability with startup vitality.
5. Our simple, non-corporate work culture that respects individual beliefs.

Read our blog: Five Reasons to Join Cerebras in 2025.

Apply today and become part of the forefront of groundbreaking advancements in AI!

Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.

This website or its third-party tools process personal data. For more details, click here to review our CCPA disclosure notice.