LLM Inference Estimator

Pick a model, a quantization, a context length and your GPU (or CPU-only). We estimate whether the model fits in memory, how that memory splits between weights and KV cache, and how many tokens per second you can expect - all computed right here in your browser.

Runs entirely in your browser - nothing you select leaves your tab.

Your setup

Honesty note

Estimate - confirm with your own run.

Fit uses 90% of your memory headroom; speed uses your memory bandwidth with a 0.45-0.65 efficiency factor. Numbers are estimates, not measurements.

Use cases

Common situations where you'd check fit and speed before running a local LLM.

  • Before you download a model

    See whether a model at a given quantization actually fits in your VRAM before spending the bandwidth to download it.

  • Choosing a quantization

    Find the largest quantization that still fits your card at your target context, so you keep as much quality as you can.

  • Context length planning

    Check how much of your VRAM a long context eats through the KV cache before it pushes the weights into offload.

  • Comparing candidate GPUs

    Estimate how many tokens/sec a model would produce on the GPU you own versus one you are considering buying.

  • Deciding between GPU and CPU

    Get a realistic sense of CPU-only speed so you know when a GPU is worth the cost.

Frequently asked questions

How is "will it fit" decided?
We estimate the weight memory (parameter count times the quant's bytes per weight) and the KV-cache memory at your context length, add them, and compare to 90% of your GPU's VRAM. If it fits inside that it "fits"; if it only fits once some weights spill to system RAM it "fits with offload"; otherwise it "won't fit".
What is the KV-cache line?
While generating, the model keeps a key/value cache for every position in your context, and it grows with context length. We show it separately from the weights so you can see how much of your memory the context is consuming.
How is the tokens-per-second estimate computed?
We divide your GPU's memory bandwidth (GB/s) by the bytes streamed per token (the weights plus the KV cache read at your context), then apply an efficiency factor of 0.45-0.65 because real inference uses 45-65% of peak bandwidth. We always show the range, never a single number.
Why a range instead of one speed number?
Because it is an estimate, not a measurement. Efficiency varies with the model, quantization, runtime (llama.cpp, Ollama and friends), drivers and your exact context length. Confirm with your own run before relying on any number.
Is my data uploaded anywhere?
No. The model and GPU catalogs and all of the math run inside your browser - nothing you pick is sent to a server.