Danh mục

surya-koritala/Glyd

Lossless AI compression: 33% less GPU memory, bit for bit.

snapshot · 26/10/2026
SG

surya-koritala/Glyd

Rust · danh tính provisional_slug_only

Lossless AI compression: 33% less GPU memory, bit for bit.

compressiondata-compressionentropy-codinglossless-compressionlz4object-storagerustsimd
Sao319lịch sử sao
Hạng#12651snapshot toàn cục
Fork3bản ghi nguồn
Số ngày trending0lịch sử xu hướng
Snapshot chỉ có lịch sử sao đầy đủ của star-history/star-history. Lịch sử thiếu thì không vẽ.

README

Từ GitHub · lưu tạm tối đa 7 ngày

Glyd

Lossless AI compression: 33% less GPU memory, bit for bit.

CI Release License: BUSL-1.1

Glyd keeps a bf16 model's weights and KV cache in fewer bits in GPU memory and rebuilds the exact values as the model runs. It is the same model in about a third less memory, so it fits a smaller GPU or serves more requests on the same one.

Run it

curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3.5-9B                       # settings for your GPU, then a chat in the terminal
glyd run Qwen/Qwen3.8-27B:swift                # fewer bits, not lossless: :kestrel 6.5 bits per weight, :swift 5.5

With vLLM, or in Python:

pip install "glyd[vllm]"
vllm serve Qwen/Qwen3.5-9B --quantization glyd
import glyd
model = glyd.from_pretrained("Qwen/Qwen3.5-9B")

Linux (x86_64 or aarch64) with an NVIDIA GPU: Ampere, Ada, Hopper or Blackwell, driver 580 or newer.

Results

Measured bf16 Glyd
Qwen3.5-9B under vLLM on an RTX 4090: weights 17.66 GiB 12.68 GiB (−28.2%)
Qwen3.5-9B under vLLM on an RTX 5090: weights 17.66 GiB 13.89 GiB (−21.3%)
Qwen3.5-9B on an RTX 4090 (24 GB): context out of memory at start (even at 4,096 tokens) 189,440 tokens
Qwen3.8-27B under vLLM on an H100: weights 50.22 GiB 38.77 GiB (−22.8%)
Qwen3.8-27B: its layers' matrices, read back bit for bit 49.52 GB 33.28 GB (−32.8%)

The freed memory becomes KV cache. Logs: rtx4090-5090-qwen35-9b-v029-2026-10-06 (Qwen3.5-9B, vLLM 0.30.0; CHANGELOG v0.29.0), h100-qwen38-v028-2026-10-03 and open-models-a10-2026-09-27 (sizes.txt). Every result: getglyd.com/benchmarks.

What lossless means here: every weight, and every key and value in the KV cache, reads back as the exact bf16 value. Outputs can still differ from bf16's in the last bits, because Glyd's kernels add the same products in a different order, as any two GPU kernels do. exact=True gives bf16's logits exactly.

The codec

Underneath the GPU work, Glyd is a lossless compression library and CLI in Rust with a C ABI: a drop-in alternative to LZ4, Snappy and zstd that also turns logs and table dumps into typed columns (-r) and compresses a new version of a file against the old one (--base).

brew install surya-koritala/glyd/glyd          # or a release's tarball
glyd --max events.json -o events.glyd          # fewer bytes than zstd -3, faster reads
glyd -d events.glyd -o events.json

Docs

License

From v0.29.0 all of Glyd is under the Business Source License 1.1: the glyd crate and CLI, the C ABI, the Python and Go bindings, the store (glyd-store) and the GPU package (glyd-gpu).

Free forever for personal and non-commercial use (including education, research and nonprofits) on computers you own or rent for yourself, and for anyone to try, test and develop with. Commercial use needs a license: suryakoritala@getglyd.com. Each version becomes Apache-2.0 four years after its release.

Releases up to v0.28 keep the licenses they shipped with, in those releases' tags: the codec (the glyd crate, CLI, C ABI and bindings) under BSD-3-Clause OR GPL-2.0, the store and the GPU package under BUSL-1.1. Code in third_party/ keeps its own license.