Something between BLOOM-176B and BLOOM-7B1?
bloom-176b requirea about 300G of VRAM
bloom-7b1 requires about 30G of VRAM
Can anyone create a version that uses 48G of VRAM, so it can be fine tuned on two 3090s?
Or 86G of VRAM, so it can be fine tuned on four 3090s?
Do you expect noticable performance increase going from bloom-7b1 to a bloom-14b?
I cannot know for sure, but, to the best of my knowledge, it is unlikely that the bigscience will train another model just like BLOOM, since the compute resources used for training the original bloom have been released. Then again, some other training projects have related models in the range you have specified.
If you need local compute, but you're okay with english-only models, there are several 20B variants (flan-T5 or gpt-neox and many other awesome projects).
If you want to inference/train this specific version of bloom with your 3090s, there's https://github.com/bigscience-workshop/petals