thinletter

thinletter is a small research group. We make existing models lighter to run and measure them against the originals. thinletter.io · info@thinletter.io

ThinCtx

ThinCtx-small: the quality of Qwen3.5 in a small GPU memory. Qwen3.5 converted so that the GPU memory for the context stays small and grows about 15 to 20 times more slowly than the original's cache, with the weights stored at 5.4 to 6.4 bits per weight and the context kept compressed in system memory. At 128k tokens the 4B-small needs 4.0 GB of GPU memory instead of 12.0 GB for the original, and at 540k tokens 4.7 GB instead of 24.5 GB.

• ThinCtx-Qwen3.5-4B-small · full precision: ThinCtx-Qwen3.5-4B
• ThinCtx-Qwen3.5-2B-small · full precision: ThinCtx-Qwen3.5-2B
• ThinCtx-Qwen3.5-0.8B-small · full precision: ThinCtx-Qwen3.5-0.8B (not recommended for tool calling)
• Runtime: pip install thinctx · results: thinletter.io/thinctx

ThinEmbed

The query side of an embedding model, compressed into a small file that runs in the browser on WebGPU, against your unchanged document index. Open source, Apache-2.0.

• harrier-0.6b (English): WebGPU clients · GGUF clients
• Qwen3-Embedding-0.6B (Czech): WebGPU client · GGUF client
• bge-m3 (Czech, multilingual): GGUF clients
• Demo · code · results: thinletter.io/query-encoders