unsloth/Qwen3.8-Flash-Next-GGUF
Image-Text-to-Text • 177B • Updated • 320
👋 Open to Work
Jerome Coste
datayoda
AI & ML interests
None yet
Recent Activity
liked a model about 8 hours ago
unsloth/Qwen3.8-Flash-Next-GGUF liked a model about 19 hours ago
Audio8/Audio8-TTS-Preview-0.1b liked a model 1 day ago
peculiar-ragdoll/Qwen-Sharp-Chat-TemplatesOrganizations
None yet
liked a
model about 8 hours ago
liked a
model about 19 hours ago
liked 8
models 1 day ago
peculiar-ragdoll/Qwen-Sharp-Chat-Templates
Updated • 254
peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF-MTP
Image-Text-to-Text • 0.4B • Updated • 15.8k • 46
Lightricks/LTX-2.5
Image-to-Video • Updated • 894k • 1.86k
Audio8/Audio8-TTS-Preview-0.6b
Text-to-Speech • 0.6B • Updated • 22.5k • • 376
Qwen/Qwen3.8-Flash-Next
Image-Text-to-Text • 180B • Updated • 2.55k • 3.6k
cafonez/Qwen3.8-27B-ROCmI4-MTP-GGUF
Image-Text-to-Text • 27B • Updated • 1.22k • 17
Kwaipilot/KAT-Dev
Text Generation • 33B • Updated • 377 • • 220
jcbtc/Ling-3.0-Flash-CIRU-IU4
Text Generation • 127B • Updated • 580 • 5
MTP in the name but the card says no MTP
6
#6 opened 2 days ago
by datayoda
liked a
model 2 days ago
liked a
model 3 days ago
liked 2
models 6 days ago
reacted to nightmedia's post with 🔥 6 days ago
Post
4080
Qwen3.8-27B metrics
It's hard to track all model cards where I post these, so I figured people would get more value out of seeing these in the open.
The performance is as measured on a M4 MBP 128GB, speed may vary depending on your platform.
These are all instruct metrics, generated by including this line in the jinja template:
Then run the test suite to generate the metrics:
This will generate the file:
This is a JSON containing all gathered metrics; for example the q4-hi:
I use the value of acc_norm for metrics, rounded to 3 decimals.
As I get more quants tested, I will add them here.
A complete test run for a single quant takes 7-9 hours depending on quant size, 10-12 hours for BF16 depending on perplexity: this is why you see on my model cards that I usually post the first three, that only take 2-3 hours :)
-G
It's hard to track all model cards where I post these, so I figured people would get more value out of seeing these in the open.
quant arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.591,0.782,0.896,0.746,0.448,0.801,0.711
q8-hi 0.602,0.779,0.896,0.747,0.446,0.793,0.703
q6-hi 0.602,0.775,0.895,0.748,0.448,0.795,0.710
q4-hi 0.604,0.780,0.898,0.744,0.454,0.795,0.708
mxfp4 0.581,0.771,0.889,0.738,0.442,0.798,0.713
1M
mxfp8 0.590,0.787,0.897,0.744,0.446,0.801,0.709
Quant Perplexity Peak Memory Tokens/sec
mxfp8 6.090 ± 0.054 34.74 GB 138
mxfp4 5.952 ± 0.051 21.30 GB 148The performance is as measured on a M4 MBP 128GB, speed may vary depending on your platform.
These are all instruct metrics, generated by including this line in the jinja template:
{%- set enable_thinking = false %}Then run the test suite to generate the metrics:
mlx_lm.evaluate --model MODEL --tasks winogrande boolq arc_challenge arc_easy hellaswag openbookqa piqaThis will generate the file:
eval_MODEL_0.4.9_winogrande_boolq_arc_challenge_arc_easy_hellaswag_openbookqa_piqaThis is a JSON containing all gathered metrics; for example the q4-hi:
"arc_challenge": {
"alias": "arc_challenge",
"acc,none": 0.5819112627986348,
"acc_stderr,none": 0.014413988396996116,
"acc_norm,none": 0.6040955631399317,
"acc_norm_stderr,none": 0.01429122839353657
},I use the value of acc_norm for metrics, rounded to 3 decimals.
As I get more quants tested, I will add them here.
A complete test run for a single quant takes 7-9 hours depending on quant size, 10-12 hours for BF16 depending on perplexity: this is why you see on my model cards that I usually post the first three, that only take 2-3 hours :)
-G
- 18 replies
liked a
model 6 days ago
any chance for a Dirk heretic?
👍 3
1
#3 opened 8 days ago
by datayoda
liked a
model 11 days ago
liked a
model 12 days ago