Glyd keeps a bf16 model's weights and KV cache in fewer bits in GPU memory and rebuilds the exact values as the model runs. It is the same model in about a third less memory, so it fits a smaller GPU or serves more requests on the same one.
curl -LsSf https://getglyd.com/install.sh | sh glyd run Qwen/Qwen3.5-9B # settings for your GPU, then a chat in the terminal glyd run Qwen/Qwen3.8-27B:swift # fewer bits, not lossless: :kestrel 6.5 bits per weight, :swift 5.5
With vLLM, or in Python:
pip install "glyd[vllm]"
vllm serve Qwen/Qwen3.5-9B --quantization glydimport glyd model = glyd.from_pretrained("Qwen/Qwen3.5-9B")
Linux (x86_64 or aarch64) with an NVIDIA GPU: Ampere, Ada, Hopper or Blackwell, driver 580 or newer.
| Measured | bf16 | Glyd |
|---|---|---|
| Qwen3.5-9B under vLLM on an RTX 4090: weights | 17.66 GiB | 12.68 GiB (−28.2%) |
| Qwen3.5-9B under vLLM on an RTX 5090: weights | 17.66 GiB | 13.89 GiB (−21.3%) |
| Qwen3.5-9B on an RTX 4090 (24 GB): context | out of memory at start (even at 4,096 tokens) | 189,440 tokens |
| Qwen3.8-27B under vLLM on an H100: weights | 50.22 GiB | 38.77 GiB (−22.8%) |
| Qwen3.8-27B: its layers' matrices, read back bit for bit | 49.52 GB | 33.28 GB (−32.8%) |
The freed memory becomes KV cache. Logs: rtx4090-5090-qwen35-9b-v029-2026-10-06
(Qwen3.5-9B, vLLM 0.30.0; CHANGELOG v0.29.0),
h100-qwen38-v028-2026-10-03 and
open-models-a10-2026-09-27 (sizes.txt). Every result:
getglyd.com/benchmarks.
What lossless means here: every weight, and every key and value in the KV cache, reads back as the
exact bf16 value. Outputs can still differ from bf16's in the last bits, because Glyd's kernels add the
same products in a different order, as any two GPU kernels do. exact=True gives bf16's logits exactly.
Underneath the GPU work, Glyd is a lossless compression library and CLI in Rust with a C ABI: a drop-in
alternative to LZ4, Snappy and zstd that also turns logs and table dumps into typed columns (-r) and
compresses a new version of a file against the old one (--base).
brew install surya-koritala/glyd/glyd # or a release's tarball glyd --max events.json -o events.glyd # fewer bytes than zstd -3, faster reads glyd -d events.glyd -o events.json
- getglyd.com/docs: getting started, vLLM, the GPU package, the codec.
- DETAILS.md: everything on one page, with the results and their logs (as text for LLMs).
- CHANGELOG.md, the roadmap, SECURITY.md.
From v0.29.0 all of Glyd is under the Business Source License 1.1: the glyd crate and CLI, the C ABI, the
Python and Go bindings, the store (glyd-store) and the GPU package (glyd-gpu).
Free forever for personal and non-commercial use (including education, research and nonprofits) on computers you own or rent for yourself, and for anyone to try, test and develop with. Commercial use needs a license: suryakoritala@getglyd.com. Each version becomes Apache-2.0 four years after its release.
Releases up to v0.28 keep the licenses they shipped with, in those releases' tags: the codec (the glyd crate, CLI, C ABI
and bindings) under BSD-3-Clause OR GPL-2.0, the store and the GPU package under BUSL-1.1. Code in
third_party/ keeps its own license.
