Software Engineer (AI Engineering)

TL;DR

Software Engineer (AI Engineering): Design, develop, and optimize the inference engine that powers Databricks’ Foundation Model API with an accent on fast, scalable, and efficient LLM serving systems. Focus on optimizing for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.

Location: Remote

Salary: $142,200—$204,600 USD

Company

Databricks is the data and AI company.

What you will do

• Contribute to the design and implementation of the inference engine.
• Collaborate with researchers to bring new model architectures or features into the engine.
• Optimize for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
• Build and maintain instrumentation, profiling, and tracing tooling to uncover bottlenecks and guide optimizations.
• Develop and enhance scalable routing, batching, scheduling, memory management, and dynamic loading mechanisms for inference workloads.
• Support reliability, reproducibility, and fault tolerance in the inference pipelines.

Requirements

• BS/MS/PhD in Computer Science, or a related field
• Strong software engineering background (3+ years or equivalent) in performance-critical systems
• Solid understanding of ML inference internals.
• Hands-on experience with CUDA, GPU programming, and key libraries (cuBLAS, cuDNN, NCCL, etc.)
• Comfortable designing and operating distributed systems.
• Demonstrated ability to uncover and solve performance bottlenecks across layers.

Nice to have

• Published research or open-source contributions in ML systems, inference optimization, or model serving.

Culture & Benefits

• Comprehensive benefits and perks that meet the needs of all employees.
• Headquartered in San Francisco, with offices around the globe.