Step 6 of 15Falsified

Persist the Triton and FlashInfer kernel caches

Keeping both JIT caches on a persistent volume changed nothing, and Triton compiled one kernel during inference, so the 40 s wasn't Triton or FlashInfer compiling.

Time to Ready
130 → 130 s
First request, no graphs
44.7 → 44.7 s
Step 05 · Measured with the first answer
All graphs
135 s
Piecewise only
134 s
Piecewise, fewer sizes
124 s
No graphs
126 s
Step 06 · Framework kernel caches kept
All graphs, caches kept
135 s
No graphs, caches kept
125 s

Hover or tap a segment for its time. The twin's readiness probe runs every 10 s, so totals land on 10-second steps and the last segment absorbs the wait.

Keeping the framework kernel caches changed neither the start nor the first request.

Hypothesis

The 40 seconds looked like kernel compilation repeated on every pod start, because Triton's and FlashInfer's caches lived in the container's temporary home directory.

Change

Both caches moved to the persistent volume.

Result

Both caches were written and read on later runs, with no change in the start or the first request. vLLM warns on every Triton compile during inference, and only one kernel compiled. The engine logged nothing for about 40 seconds, then ran the warm-up request in about 5. A neighbouring pod on the same image held a 6.5 MB cache from the CUDA driver itself, also in a temporary directory.

Decision

Falsified: the 40 seconds wasn't Triton or FlashInfer compiling. The driver's own cache became the next lead.