test-models - a inference-optimization Collection

inference-optimization 's Collections

Granite 4 Small and Tiny Quantized Models

NVIDIA-Nemotron-3-Nano-30B-A3B Quantized Models

Qwen3-Next-80B-A3B Quantized Models

KV Cache Quantization

test-models

updated 26 days ago

inference-optimization/test_tencentbac_fastmtp

Updated Mar 4 • 1
inference-optimization/test_qwen3_next_mtp

Updated Mar 4 • 2
inference-optimization/Qwen3-Next-80B-A3B-Instruct_mtp_speculator

Text Generation • 2B • Updated Mar 18 • 23
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch3

2B • Updated Mar 19 • 9
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch1

2B • Updated Mar 19 • 3
inference-optimization/Qwen3-Next-80B-A3B-Instruct-MTP-ultrachat-epoch2

2B • Updated Mar 19 • 6
inference-optimization/Qwen3-Next-80B-A3B-Instruct-GSM8K-MTP-finetuned

81B • Updated 26 days ago • 62

Note Qwen3-Next-80B with GSM8K-finetuned MTP head (+22.5% acceptance rate). Generated using speculators.