INC4AI commited on
Commit
88aa273
·
verified ·
1 Parent(s): e3fc5a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -8
README.md CHANGED
@@ -5,8 +5,8 @@ base_model:
5
  ---
6
  ## Model Details
7
 
8
- This model is an mxfp8 quantized version of [unsloth/DeepSeek-R1-BF16](https://huggingface.co/unsloth/DeepSeek-R1-BF16) generated by [intel/auto-round](https://github.com/intel/auto-round).
9
- Please follow the license of the original model.
10
 
11
  ## How to Use
12
 
@@ -14,12 +14,14 @@ The step-by-step README of quantization and evaluation can be found in [Intel Ne
14
 
15
  ## Evaluate Results
16
 
17
- | Task | backend | BF16 | MXFP8 |
18
- |:---------:|:-------:|:--------:|:-------:|
19
- | hellaswag | vllm | 0.6903 | 0.6956 |
20
- | piqa | vllm | 0.8319 | 0.8324 |
21
- | mmlu | vllm | 0.8489 | 0.8532 |
22
- | gsm8k | vllm | 0.9568 | 0.9583 |
 
 
23
 
24
 
25
  ## Ethical Considerations and Limitations
 
5
  ---
6
  ## Model Details
7
 
8
+ This model card is for mxfp8/mxfp4/nvfp4 quantization of [unsloth/DeepSeek-R1-BF16](https://huggingface.co/unsloth/DeepSeek-R1-BF16) based on [intel/auto-round](https://github.com/intel/auto-round).
9
+ The models are not able to be published due to the storage limitation. Please follow the INC example README to generate and evaluate the low precision models.
10
 
11
  ## How to Use
12
 
 
14
 
15
  ## Evaluate Results
16
 
17
+
18
+ | Task | backend | BF16 | MXFP8 | MXFP4 | NVFP4 |
19
+ |:-----------:|:-------:|:----------:|:----------:|:----------:|:----------:|
20
+ | hellaswag | vllm | 0.6903 | 0.6956 | 0.6898 | 0.6953 |
21
+ | piqa | vllm | 0.8319 | 0.8324 | 0.8297 | 0.8303 |
22
+ | mmlu | vllm | 0.8489 | 0.8532 | 0.8426 | 0.8495 |
23
+ | gsm8k | vllm | 0.9568 | 0.9583 | 0.9553 | 0.9606 |
24
+ | **average** | vllm | **0.8320** | **0.8349** | **0.8294** | **0.8339** |
25
 
26
 
27
  ## Ethical Considerations and Limitations