inference-optimization/gemma-4-1B-0.8B-tiny
1B • Updated • 1.54k • 2
Tiny Models
Tiny models used for testing
-
-
inference-optimization/DSV4-tiny-empty
3B • Updated • 3.45k • 1 -
inference-optimization/Qwen3-1.6B-A0.9B
Text Generation • 2B • Updated • 10.9k • 3 -
inference-optimization/Llama-3.2-0.5B-Instruct
Text Generation • 0.5B • Updated • 13.9k • 1 -
inference-optimization/GLM-5.2-0.8B-A0.8B
Text Generation • 0.8B • Updated • 3.57k • 4 -
inference-optimization/Phi-3.5-MoE-0.8B-A0.2B
Text Generation • 0.8B • Updated • 1.02k • 1 -
inference-optimization/Llama-4-Scout-1.7B-0.4B-Instruct
Image-Text-to-Text • 2B • Updated • 988 -
inference-optimization/gpt-oss-2.5B-A1.3B
3B • Updated • 947 -
inference-optimization/Qwen3-VL-1.0B-A0.4B-Instruct
Image-Text-to-Text • 1B • Updated • 1.01k • 1 -
inference-optimization/Inkling-0.6B-A0.6B
Image-Text-to-Text • 0.6B • Updated • 949 • 1 -
inference-optimization/DeepSeek-V4-Pro-0.5B-A0.37B
Text Generation • 0.5B • Updated • 976 -
inference-optimization/Kimi-K3-0.40B
Feature Extraction • 0.4B • Updated • 833 • 83 -
inference-optimization/Kimi-K3-0.40B-MXFP4
0.4B • Updated • 1.18k • 7 -
inference-optimization/Kimi-K3-0.40B-NVFP4
0.4B • Updated • 61 • 5 -
inference-optimization/gemma-4-unified-0.8B-tiny
0.8B • Updated • 931 -
inference-optimization/granite-vision-4.1-0.2B-tiny
0.2B • Updated • 916 -
inference-optimization/Qwen3-VL-Reranker-0.1B-tiny
98.7M • Updated • 937 -
inference-optimization/Qwen3.8-1.0B-A0.6B
Text Generation • 1.0B • Updated • 3.96k • 6 -
inference-optimization/GLM-5.3-Flash-0.1B-A0.1B
Image-Text-to-Text • 84.4M • Updated • 2.8k • 3 -
inference-optimization/Qwen3.8-Flash-Next-0.2B-A0.2B
Image-Text-to-Text • 0.2B • Updated • 1.22k • 2 -
inference-optimization/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP
Text Generation • 1B • Updated • 167 • 1 -
inference-optimization/Qwen3.6-8B-A1.6B
Image-Text-to-Text • 8B • Updated • 940 -
inference-optimization/NemotronH-0.3B-A0.3B
Text Generation • 0.3B • Updated • 3.48k • 1 -
inference-optimization/GLM-5.3-0.6B-A0.4B
Text Generation • 0.6B • Updated • 1.02k • 1
inference-optimization/GLM-4.7-Flash-0.82B-MTP
Text Generation • 0.8B • Updated • 19Note PR #3225. BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP
Text Generation • 0.9B • Updated • 20Note PR #3225. BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-5-0.88B-MTP
Text Generation • 0.9B • Updated • 18Note PR #3225. BF16 source; backbone validated, MTP execution unverified.
inference-optimization/Inkling-0.92B-MTP
Image-Text-to-Text • 0.9B • Updated • 38Note PR #3225. BF16 multimodal fixture; text backbone validated. Two-rank pipeline and MTP serving are not validated.