Staff Software Engineer (GenAI)

TL;DR

Staff Software Engineer (GenAI): Leading the architecture, development, and optimization of the inference engine for Databricks Foundation Model API with an accent on high throughput, low latency, and robust scaling. Focus on collaborating with researchers and optimizing performance across GPUs and accelerators.

Company

Databricks is the data and AI company, providing a unified platform for data, analytics, and AI.

What you will do

• Own and drive the architecture, design, and implementation of the inference engine.
• Partner closely with researchers to integrate new model architectures.
• Lead optimization for latency, throughput, and memory efficiency.
• Define standards for instrumentation and profiling tools.
• Architect scalable routing and memory management for inference workloads.
• Ensure reliability and fault tolerance in inference pipelines.

Requirements

• BS/MS/PhD in Computer Science or related field.
• 6+ years of experience in performance-critical systems.
• Deep understanding of ML inference internals.
• Hands-on experience with CUDA and GPU programming.
• Strong background in distributed systems design.
• Excellent communication and leadership skills.