Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -12,17 +12,20 @@ tags:
|
|
| 12 |
Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter),
|
| 13 |
a masked-span speech editor.
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
-
| edit type | metric | Ren et al. | wavepainter
|
| 19 |
-
|---|---|---|---|
|
| 20 |
-
| substitution | WER | 4.41 | **3.
|
| 21 |
-
| insertion | WER | 4.97 |
|
| 22 |
-
| deletion | WER | **6.88** | 9.
|
| 23 |
-
| substitution | SIM | 0.78 | **0.943** |
|
| 24 |
-
| insertion | SIM | 0.82 | **0.961** |
|
| 25 |
-
| deletion | SIM | 0.78 | **0.918** |
|
| 26 |
|
| 27 |
Five of the six compared numbers improve on theirs: substitution and insertion
|
| 28 |
WER, and speaker similarity on all three edit types. We do not beat them on
|
|
@@ -31,26 +34,34 @@ gap with a ground-truth waveform splice.
|
|
| 31 |
|
| 32 |
Signal quality is reported full-reference, since a no-reference MOS estimator
|
| 33 |
rates the untouched original recordings below synthesised output on this
|
| 34 |
-
benchmark. Over the audio the edit does not touch, PESQ is 4.
|
| 35 |
for substitution / insertion / deletion, against 4.351 for copy-synthesis with no
|
| 36 |
edit at all.
|
| 37 |
|
| 38 |
-
###
|
| 39 |
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
## Files
|
| 49 |
|
| 50 |
-
- `phase1-base/`
|
| 51 |
-
- `phase2-hubert-
|
| 52 |
-
- `phase2-hubert-
|
| 53 |
-
|
| 54 |
|
| 55 |
## Use
|
| 56 |
|
|
|
|
| 12 |
Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter),
|
| 13 |
a masked-span speech editor.
|
| 14 |
|
| 15 |
+
`phase2-hubert-child` is the released model and the artifact of record: phase two
|
| 16 |
+
as trained, and the checkpoint every number in the paper comes from. On the
|
| 17 |
+
English `full` split of Ming-Freeform-Audio-Edit, against Ren et al.
|
| 18 |
+
([arXiv:2602.00560](https://arxiv.org/abs/2602.00560), the row with GRPO), scored
|
| 19 |
+
with oracle edit spans taken from the transcript diff:
|
| 20 |
|
| 21 |
+
| edit type | metric | Ren et al. | wavepainter |
|
| 22 |
+
|---|---|---|---|
|
| 23 |
+
| substitution | WER | 4.41 | **3.479** |
|
| 24 |
+
| insertion | WER | 4.97 | **4.400** |
|
| 25 |
+
| deletion | WER | **6.88** | 9.697 |
|
| 26 |
+
| substitution | SIM | 0.78 | **0.943** |
|
| 27 |
+
| insertion | SIM | 0.82 | **0.961** |
|
| 28 |
+
| deletion | SIM | 0.78 | **0.918** |
|
| 29 |
|
| 30 |
Five of the six compared numbers improve on theirs: substitution and insertion
|
| 31 |
WER, and speaker similarity on all three edit types. We do not beat them on
|
|
|
|
| 34 |
|
| 35 |
Signal quality is reported full-reference, since a no-reference MOS estimator
|
| 36 |
rates the untouched original recordings below synthesised output on this
|
| 37 |
+
benchmark. Over the audio the edit does not touch, PESQ is 4.090 / 4.083 / 4.079
|
| 38 |
for substitution / insertion / deletion, against 4.351 for copy-synthesis with no
|
| 39 |
edit at all.
|
| 40 |
|
| 41 |
+
### The interpolated variants
|
| 42 |
|
| 43 |
+
Two earlier releases interpolate the phase-two child back toward the phase-1 base,
|
| 44 |
+
which moves the diffusion denoiser only. They are published because the paper's
|
| 45 |
+
first version reported them and anyone who fetched them should keep working; the
|
| 46 |
+
paper no longer reports either.
|
| 47 |
+
|
| 48 |
+
| | substitution | insertion | deletion |
|
| 49 |
+
|---|---|---|---|
|
| 50 |
+
| `phase2-hubert-child` (released) | **3.479** | 4.400 | **9.697** |
|
| 51 |
+
| `phase2-hubert-a085` | 3.063 | 4.128 | 9.136 |
|
| 52 |
+
| `phase2-hubert-a070` | 3.381 | **3.817** | 9.525 |
|
| 53 |
+
|
| 54 |
+
WER, lower is better. The coefficient has no single optimum across edit types, so
|
| 55 |
+
both variants are kept rather than one being presented as strictly better.
|
| 56 |
+
Speaker similarity is 0.943 / 0.961 / 0.918 for all three except insertion at
|
| 57 |
+
a=0.70, which is 0.960.
|
| 58 |
|
| 59 |
## Files
|
| 60 |
|
| 61 |
+
- `phase1-base/` -- the phase-1 editor. Start here to re-run phase 2 (~1 GPU-hour).
|
| 62 |
+
- `phase2-hubert-child/` -- the released model, and the artifact of record.
|
| 63 |
+
- `phase2-hubert-a085/` -- an earlier release, interpolated at 0.85.
|
| 64 |
+
- `phase2-hubert-a070/` -- the same, at 0.70.
|
| 65 |
|
| 66 |
## Use
|
| 67 |
|