ONNX / WebGPU release for Chatterbox Nano?
Hi, Resemble AI team! Chatterbox Nano looks extremely promising, especially for lightweight local and browser-based voice applications.
Chatterbox Turbo already has an official ONNX release with FP16, Q8, Q4, and Q4F16 variants, but Nano currently only appears to have the original PyTorch/Safetensors checkpoints. Since Nano is much smaller and designed for edge deployment, it seems like an especially strong candidate for ONNX Runtime, ONNX Runtime Web, and WebGPU support.
Would you consider releasing an official ONNX export for Chatterbox Nano, ideally split into components similarly to Chatterbox Turbo?
@owensong Hi, thank for your interest!
Unfortunately, we don't have any plan to release onnx or other quantized our models in the future.
Why not? I’m honestly disappointed and frustrated by this decision.
Nano seems like the model in your lineup that would benefit most from ONNX Runtime, ONNX Runtime Web, and WebGPU support. The point of a model this large is so that it can run on WebGPU quickly. Without an ONNX-compatible deployment path, that potential feels unnecessarily limited.
The README and demo samples appear to barely have any changes, and some were left unchanged at all. This release feels so unpolished and looked over. I'm not saying that this model is bad, and I don't want to sound overly disrespectful - but this model has so much potential while the release and advertising on every end is just completely horrible. Also, there is very little guidance for users who want to deploy Nano outside the original PyTorch environment.
Could you clarify whether the lack of an ONNX release is due to technical limitations, licensing concerns, or engineering priorities? Also, does your response apply specifically to Nano, or are you saying Resemble AI does not plan to release ONNX or quantized versions of any future models?
I genuinely think Nano has more potential than the current release reflects, so I hope the team will reconsider or at least provide a path for the community to help.
@leo-lennox I just ended up doing it myself in the end, I guess. Released it on my profile if you wanna use it.