---
title: "Inference"
slug: "inference"
updated: 2026-10-28T15:31:13Z
published: 2026-10-28T15:31:13Z
canonical: "docs.vastdata.com/inference"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference

As long-context LLM inference scales, GPU High Bandwidth Memory (HBM) quickly becomes the primary bottleneck because the Key-Value (KV) cache grows linearly. Integrating vLLM’s native OffloadingConnector, host DRAM, and VAST Data creates a disaggregated path that spills KV from GPU → CPU → NFS, bypassing GPU memory bounds and cutting warm Time-To-First-Token (TTFT) when refixes hit the CPU/FS tiers.

### KB Articles:

- [**File Prerequisites**](/kb/docs/file-prerequisites)
- [**Dynamo**](/kb/docs/dynamo)
  - [**Dynamo VAST CSI Integration (Optional)**](/kb/docs/dynamo-vast-csi-integration-optional)
  - [**Dynamo using vLLM**](/kb/docs/dynamo-using-vllm)
- [**vLLM**](/kb/docs/vllm)
  - [**vLLM (AMD)**](/kb/docs/vllm-amd)
  - [**vLLM (NVIDIA)**](/kb/docs/vllm-nvidia)
  - [**vLLM & LMCache In-Process (AMD)**](/kb/docs/vllm-lmcache-in-process-amd)
  - [**vLLM & LMCache In-Process (NVIDIA)**](/kb/docs/vllm-lmcache-in-process-nvidia)
  - [**vLLM & LMCache Multi-Process (AMD)**](/kb/docs/vllm-lmcache-multi-process-amd)
  - [**vLLM & LMCache Multi-Process (NVIDIA)**](/kb/docs/vllm-lmcache-multi-process-nvidia)
  - [**ModelExpress with vLLM (NVIDIA)**](/kb/docs/modelexpress-with-vllm-nvidia)
- [**SGLang**](/kb/docs/sglang)
  - [**SGLang HiCache using NIXL (NVIDIA)**](/kb/docs/sglang-hicache-using-nixl-nvidia)

### Use Case:

**VAST Data for KV Cache** [**Scale Context Windows, not GPU Costs**](https://www.vastdata.com/usecase/kv-cache)