What are the responsibilities and job description for the Machine Learning Inference Engineer position at Oscar Technology?
Title: Machine Learning Inference Engineer
Location: San Francisco, CA - Hybrid
Salary: $200K-$250K Equity
A rapidly growing AI startup is hiring a Machine Learning Inference Engineer to help optimize and scale production machine learning systems. The company is building next-generation AI applications and is looking for an engineer who enjoys solving complex performance challenges across model serving and inference infrastructure. This is a startup environment where engineers take ownership over their work, collaborate closely with leadership, and have a direct impact on the performance and scalability of production AI systems.
This role is focused on productionizing machine learning models and optimizing inference performance at scale. You will build high-performance model-serving infrastructure, improve latency, throughput, and compute efficiency, and work across GPU optimization, distributed inference, and production deployment. This is a hands-on engineering role ideal for someone passionate about building scalable AI infrastructure and making machine learning systems faster, more efficient, and production-ready.
Experience:
* 2 years of experience in Machine Learning Engineering or AI Infrastructure
* Strong Python and PyTorch experience building production ML systems
* Experience with inference frameworks such as Triton, TensorRT, vLLM, or similar
* Experience optimizing model latency, throughput, and infrastructure cost
* Experience with model optimization techniques such as pruning, distillation, or KV caching
* Understanding of GPU infrastructure and deploying ML models at scale
* Experience with diffusion models, multimodal AI, or large-scale vision systems is a plus
Benefits:
* Equity
* Paid Time Off
* Medical, dental, and vision coverage
* Opportunity to create lasting impact within a growing organization
* Hybrid working schedule
Oscar Associates Limited (US) is acting as an Employment Agency in relation to this vacancy.
Salary : $200,000 - $250,000