OmniVoice quantization question
#1
by nikich340 - opened
Greetings, and thank you for such an incredible project. I managed to test so many models in an hour, without managin all their dependencies and pitfalls, because I have unified engine!
As for OmniVoice, there are bf16 and f16 versions in quite equal filesize. Is there any difference between them, and which to prefer? (if I use CUDA tts generation on sm_120 blackwell gpu).
@nikich340 Thank you for your kind words! I don’t have a strong preference between them. Both should work on an SM120 Blackwell GPU, and I would expect any performance difference to be within the noise level. Some of the GGUFs in the repo are leftovers from earlier experiments. For example, org may be identical to fp16. Sorry for the confusion.