25 1 26

Lucas Silva PRO

YuLexuan30

AI & ML interests

None yet

Recent Activity

reacted to marksverdhei's post with 🤗 about 13 hours ago

Dear Hugging Face team, can we please have a way to archive hf repositories / spaces? I have a bunch of spaces that used to work but don't any more due to the hf space implementations changing and i think it would be good if I could archive those like in GitHub. React to this post if you want to see this feature! 💡

liked a model 6 days ago

lodestones/Chroma1-Radiance

liked a model 6 days ago

lodestones/Chroma2-Kaleidoscope

View all activity

Organizations

reacted to marksverdhei's post with 🤗 about 13 hours ago

Post

617

Dear Hugging Face team, can we please have a way to archive hf repositories / spaces? I have a bunch of spaces that used to work but don't any more due to the hf space implementations changing and i think it would be good if I could archive those like in GitHub.

React to this post if you want to see this feature! 💡

liked 2 models 6 days ago

lodestones/Chroma1-Radiance

Updated 16 minutes ago • 140

lodestones/Chroma2-Kaleidoscope

Text-to-Image • Updated 30 minutes ago • 54

liked a Space 15 days ago

FLUX.2 [Klein] 9B

💻

393

Generate or edit images from text prompts or existing photos

liked a model 15 days ago

Lightricks/LTX-2

Image-to-Video • Updated 12 days ago • 2.5M • • 1.39k

liked a model 16 days ago

black-forest-labs/FLUX.2-klein-4B

Image-to-Image • Updated 16 days ago • 80.7k • • 395

liked a Space 19 days ago

LTX-2 Video Fast

🎥

191

Fast high quality video with audio generation

liked a model 21 days ago

well9472/Nanosaur

Updated 22 days ago • 13

liked a model 25 days ago

lodestones/Zeta-Chroma

Text-to-Image • Updated about 1 hour ago • 105

liked a Space about 2 months ago

AI Image Editor

⚡

Edit photos instantly using simple text prompts

New activity in black-forest-labs/FLUX.2-dev about 2 months ago

Space no longer working

#10 opened about 2 months ago by

YuLexuan30

reacted to martinsu's post with 🔥 about 2 months ago

Post

2031

I wasted days on a GPU node on a bug that shouldn't exist

So I was fine-tuning TildeOPEN-30B and the outputs were... weird. Token ID 179 (<0x00>) kept appearing between almost every token pair. Took me a bit to figure out what was going on.

Turns out I used the fast tokenizer for training, but the model was trained on the slow one. Silent failure.

Well... long story short—TGI uses (forces) the fast tokenizer, no questions asked. And you'll have agile's kryptonite: silent failure. If the model was trained on slow, it's a silent disaster.

I got curious and wrote a quick script to check how common this is. Ran it on 6,014 LLM HF models overnight.

Roughly 10% of HF model downloads have mismatched tokenizers. Not all mismatches are catastrophic, but some are brutal — like chat template markers inflating from 1 token to 3, silently wrecking context windows and causing model act weird.

This wasn't rigorous research, but the drift is real. And the worst part? 968 models(out of 500+ downloads) have both fast and slow tokenizers present, but they still produce different outputs. No missing files, no errors — just silent degradation.

TGI defaults to the fast tokenizer, as does AutoTokenizer.from_pretrained(). If a fast tokenizer doesn't exist, it auto-generates one. If your model was trained on slow, you get silent degradation. Output looks fine; the model just performs worse. Sometimes really worse. You'd never know.

If model was trained on fast tokenizer, its fine, but how do You know?

The root cause? Either model authors run HF conversion and upload both without verifying, or users run TGI, which always forces(converts to) fast .

The result of this fight with tokenizers is martinsu/tildeopen-30b-mu-instruct

It's based on TildeOPEN-30B (a solid EU HPC multilingual base). Nothing fancy—just a proper instruction fine-tune where I didn't mess up the tokenizer this time.

Full article: https://github.com/martins-u/tokenmagedon

1 reply

liked a Space about 2 months ago

FLUX.2 [dev]

💻

647

Create images from text and optional images

New activity in multimodalart/ai-toolkit 2 months ago

Please update space to add Z-Image lora training

#15 opened 2 months ago by

YuLexuan30

reacted to multimodalart's post with 🤗❤️ 2 months ago

Post

35582

New feature 🔥
Image models and LoRAs now have little previews 🤏

If you don't know where to start to find them, I invite you to browse cool LoRAs in the profile of some amazing fine-tuners: @artificialguybr , @alvdansen , @DoctorDiffusion , @e-n-v-y , @KappaNeuro @ostris