Accountabilities

You will lead end-to-end inference optimization, ensuring that advanced machine learning models perform efficiently and reliably in production environments. Your responsibilities will include:

Own the optimization process from research models to production deployment.

Improve latency and throughput for real-time voice and LLM inference at consumer scale.

Work directly with inference engines such as vLLM, TensorRT-LLM (TRT-LLM), and SGLang to enhance performance.

Implement model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.

Develop high-performance GPU workloads using C++, CUDA, Rust, or optimized Python.

Build and scale distributed multi-GPU and multi-node inference systems using Kubernetes, Ray, and custom load-balancing solutions.

Collaborate with research and product teams to bring new models into production.

Requirements

The ideal candidate is a senior-level machine learning inference or systems engineer with strong hands-on experience optimizing production inference engines and building scalable GPU infrastructure. You should bring:

At least 5 years of experience in ML inference or systems engineering, at a senior level equivalent to Google L5 or Meta E5.

Practical experience optimizing inference engines such as vLLM or TRT-LLM at the engine level, beyond basic model deployment.

A proven background in developing backend and machine learning systems and serving models in production.

Strong systems programming and GPU optimization skills using C++, CUDA, Rust, or optimized Python.

Hands-on knowledge of model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.

Experience with distributed systems, Kubernetes, Ray, load balancing, multi-GPU and multi-node inference, and high-concurrency environments.

Excellent English communication skills.

Residence in the United States, Canada, the United Kingdom, Germany, Switzerland, or Serbia.

Additional advantages include a track record of rapid career progression, contributions to open-source inference engines or technical publications, and a PhD in Computer Science, Physics, or Mathematics, or a degree from a leading computer science program. The ability to commute to Mountain View, California, is also a plus.

Benefits

Salary: $250,000–$300,000 annual base salary, plus competitive equity.

Employment: Full-time, permanent position.

Work arrangement: Onsite in Mountain View, California, preferred; remote opportunities available in the United States, Canada, the United Kingdom, Germany, Switzerland, and Serbia.

Visa support: H-1B and OPT transfers supported.

Relocation assistance: Available for candidates relocating to the San Francisco Bay Area.

Language: English-speaking working environment.

How Jobgether works

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1