Book a call

Backed by Combinator

We tune inference stacks for AI teams, down to the kernel.

vLLM and SGLang ship what works on average. No deployment is average. We profile yours against the hardware limit and close the gap.

You only pay if we meet the performance target.

Book a call

FlashAttention has no kernel for Gemma 4’s largest attention heads on the RTX PRO 6000. We wrote one.

Global attention, Gemma 4 31B, one RTX PRO 6000
vLLM default
1,770 ms
Our kernel
464 ms
floor 294 ms
Read how

The Inference Engineering Blog

  1. A custom attention kernel for Gemma 4 on the RTX PRO 6000
    1,770 → 464 ms
  2. vLLM’s Triton fallback underuses the tensor cores on Gemma 4 attentionfinding
    17% of measured peak
  3. Three launch settings speed up Gemma 4 attention with identical outputfinding
    −41% per call
  4. The case for inference optimization
    essay
  5. Compounding GPU and CPU work on a small model
    6.4 → 3.4 ms
  6. A fused MLP kernel paid off only after whole-step CUDA graphsfinding
    −9.3% per step
  7. Caching less to serve more requests
    2.6× requests per GPU
  8. On Qwen3.5 35B-A3B, state copies fill most of each request’s memoryfinding
    85% of memory
  9. TCP for server-to-GPU communication
    325 → 72 ms
  10. Multimodal processor rewrite: 5× lower latency
    428 → 81 ms

Read more