Hello everyone, yesterday there were minor problems that prevented the usage of the Embedding model. Mainly because of the Processor Class.
Posting here that the team has already solved the bugs.
If there is any problem with your usage, first delete the cached model (.cache folder in Hugging Face), redownload it, and if the issue persists, post a thread on the model page.
Yuriy Perezhohin PRO
yuriyvnv
AI & ML interests
Automatic Speech Recognition, Embeddings, Code Generation, Synthetic Data Generation and Filtering
Recent Activity
liked
a model
6 days ago
zai-org/GLM-OCR
updated
a model
7 days ago
yuriyvnv/3layers_wt_alignment_PT_100_epochs
published
a model
7 days ago
yuriyvnv/3layers_wt_alignment_PT_100_epochs
Organizations
replied to
their
post
9 days ago
posted
an
update
9 days ago
Post
2160
🎯 WAVe: 1B Multimodal Embedding Model for Word-Level Speech Quality
Multimodal embeddings for speech + transcript that verify quality at the word level, not just sentence level. Catches mispronunciations, timing errors, and prosody issues that sentence-level filters miss.
📊 Impact on Portuguese ASR:
• 34% reduction in training steps
• 50% better cross-domain generalization
• 30% less synthetic data needed
• Word-aligned attention finds errors other methods miss
🏗️ Architecture:
• Text: XLM-RoBERTa (278M params)
• Audio: Wav2Vec2-BERT 2.0 (581M params)
• Word Alignment: Multi-head attention + GLU (14M params)
• Total: 1B parameters
# Assess speech-transcript alignment
Perfect for filtering synthetic speech datasets before ASR training.
Model: yuriyvnv/WAVe-1B-Multimodal-PT
Code to create WAVe : https://github.com/yuriyvnv/WAVe
#multimodal #speech #embeddings #asr
#syntheticdata #qualityassessment
Multimodal embeddings for speech + transcript that verify quality at the word level, not just sentence level. Catches mispronunciations, timing errors, and prosody issues that sentence-level filters miss.
📊 Impact on Portuguese ASR:
• 34% reduction in training steps
• 50% better cross-domain generalization
• 30% less synthetic data needed
• Word-aligned attention finds errors other methods miss
🏗️ Architecture:
• Text: XLM-RoBERTa (278M params)
• Audio: Wav2Vec2-BERT 2.0 (581M params)
• Word Alignment: Multi-head attention + GLU (14M params)
• Total: 1B parameters
from transformers import AutoModel, AutoProcessor
processor = AutoProcessor.from_pretrained(
"yuriyvnv/WAVe-1B-Multimodal-PT",
trust_remote_code=True
)
model = AutoModel.from_pretrained(
"yuriyvnv/WAVe-1B-Multimodal-PT",
trust_remote_code=True
)# Assess speech-transcript alignment
inputs = processor(text="Olá, como está?", audio=audio_array, sampling_rate=16000, return_tensors="pt")
quality = model(**inputs).quality_score.item()Perfect for filtering synthetic speech datasets before ASR training.
Model: yuriyvnv/WAVe-1B-Multimodal-PT
Code to create WAVe : https://github.com/yuriyvnv/WAVe
#multimodal #speech #embeddings #asr
#syntheticdata #qualityassessment
-
- 1 reply