int4 convrot text encoder?
Will there be int4 convrot text encoder?
Second this - nvfp4 is for blackwell only, isn't it? Would it be possible to make 4-bit for those with Ada?
yes please! a pruned int4 would get it to a more usable point for those in Ada
also, I wonder how much can text enconder and video VAE be quantized without messing the quality too much
The nvfp4 is not for Blackwell only, you can just use it.
Kijai, Are you willing to build a Lightx lora for the Minimax H3 model?
Kijai, Are you willing to build a Lightx lora for the Minimax H3 model?
Everybody think about distillation of this BigDaddy)
Everything in its own time
Stay tuned and F5 huggingface
P.S. I think distillation may take 3~5 days
Will there be int4 convrot text encoder?
side note for you guys, my repo contains some gguf encoders, they arent much smaller than the nvfp4 encoder in this repo but it should help A LITTLE i guess. also 2 quants of the model and a mixed precision gguf of a q2 with q6 layering i havent tested yet
https://huggingface.co/realrebelai/MiniMax-H3_GGUFs/tree/main
The nvfp4 is not for Blackwell only, you can just use it.
"NVFP4 is specifically tied to NVIDIA's hardware-accelerated 4-bit floating-point format, requiring dedicated 5th-generation Tensor Cores found in Blackwell and newer architectures for native execution. While older or non-Blackwell GPUs can theoretically parse or upcast these weights through software emulation, they lack native silicon support, leading to severe performance and efficiency penalties." According to Google.
The nvfp4 is not for Blackwell only, you can just use it.
"NVFP4 is specifically tied to NVIDIA's hardware-accelerated 4-bit floating-point format, requiring dedicated 5th-generation Tensor Cores found in Blackwell and newer architectures for native execution. While older or non-Blackwell GPUs can theoretically parse or upcast these weights through software emulation, they lack native silicon support, leading to severe performance and efficiency penalties." According to Google.
It's more nuanced than that, in this case it's not really different than using something like GGUF, which also has to be upcast to be used, it's just used as storage format here, there's nothing hardware specific about that.
Yep, did not notice any benefits from NVFP4 on 3090 but it was not any worse than int8 either. So no point to use on 30 series. NVFP4 is a smaller file, but no reason to sacrifice possible detail quality for no speed gain on RTX 30.
Let me clarify a couple of things for some users. As Kijai mentioned above, nvfp4 is used here only for the text encoder and not as a diffusion model. So, there is no quality loss or hardware benefit for users with older RTX cards. In the past, I ignored all nvfp4 models since I have an older card. But I was surprised today that it is possible to use nvfp4 as a text encoder, thanks to Kijai for mentioning that. So, I did some quick tests with my RTX 3060 12GB and sure enough, you can use the nvfp4 model as a text encoder without any problem and save some extra space (15 GB vs. 24 GB), but that's pretty much all. nvfp4 is, as Kijai mentioned, used as a storage container format . Btw, here is my quick test with the default ComfyUI preset and prompt. MiniMax-H3 (960x544x20steps) with ref2va pruned int8 convrot on my RTX 3060 12GB/32GB and it works without any problem (with nvfp4 txt). Cheers!
The text encoding seems to be almost instant since the text model isn't actually generating anything, it's just encoding the text once as if it were about to encode the first token of a response, from my understanding. So while it would be nice to have more optimizations, it doesn't seem like there's much to gain in speed (though maybe Int4's accuracy would be better?)
FOR EVERYONE ASKING for the INT4 convrot text encoder:
works well on a 5060 16GB
Video generated at 768x576 - 10 seconds long = 10 minutes rendering time
The nvfp4 is not for Blackwell only, you can just use it.
NVFP4 does run on non‑50‑series GPUs, but its speed is extremely slow compared to INT4. I think INT4 quantization is necessary.