CCCCyx commited on
Commit
2805586
·
verified ·
1 Parent(s): 46a87a2

Add bilingual quantization benchmark comparison

Browse files
.gitattributes CHANGED
@@ -37,3 +37,5 @@ tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  assets/benchmark-streaming.png filter=lfs diff=lfs merge=lfs -text
38
  assets/architecture.png filter=lfs diff=lfs merge=lfs -text
39
  assets/logo.png filter=lfs diff=lfs merge=lfs -text
 
 
 
37
  assets/benchmark-streaming.png filter=lfs diff=lfs merge=lfs -text
38
  assets/architecture.png filter=lfs diff=lfs merge=lfs -text
39
  assets/logo.png filter=lfs diff=lfs merge=lfs -text
40
+ assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
41
+ assets/mossvl_quantization_benchmark_comparison_zh_4k.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -41,6 +41,24 @@ The checkpoint carries its bitsandbytes configuration, HQQ cache configuration,
41
  and MOSS-VL remote modeling code. Load the directory directly; do not add a
42
  second runtime quantization configuration.
43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  ## Hardware requirements
45
 
46
  The model is designed to run on a single NVIDIA GPU with 24 GB of VRAM. Use
 
41
  and MOSS-VL remote modeling code. Load the directory directly; do not add a
42
  second runtime quantization configuration.
43
 
44
+ ## Quantization benchmark
45
+
46
+ The following comparison covers the offline FP8 + KV8 deployment profile and
47
+ the realtime W4NF4 + KV8 deployment profile against their non-quantized BF16
48
+ checkpoints.
49
+
50
+ ### English
51
+
52
+ <p align="center">
53
+ <img src="assets/mossvl_quantization_benchmark_comparison_en_4k.png" alt="MOSS-VL quantization benchmark comparison" width="100%"/>
54
+ </p>
55
+
56
+ ### Chinese
57
+
58
+ <p align="center">
59
+ <img src="assets/mossvl_quantization_benchmark_comparison_zh_4k.png" alt="MOSS-VL quantization benchmark comparison in Chinese" width="100%"/>
60
+ </p>
61
+
62
  ## Hardware requirements
63
 
64
  The model is designed to run on a single NVIDIA GPU with 24 GB of VRAM. Use
assets/mossvl_quantization_benchmark_comparison_en_4k.png ADDED

Git LFS Details

  • SHA256: 75ed545c603986ffe7e8ce419f3bc49796a427c5f288a919b7d2bd805f44e5e4
  • Pointer size: 132 Bytes
  • Size of remote file: 2.8 MB
assets/mossvl_quantization_benchmark_comparison_zh_4k.png ADDED

Git LFS Details

  • SHA256: 8fda6adf7c96f7ce07e4dfc354882b321c1ff9ccf729b5584f816d10cf209631
  • Pointer size: 132 Bytes
  • Size of remote file: 2.68 MB