RootAccess4Life commited on
Commit
84aa227
·
verified ·
1 Parent(s): 79347cf

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +34 -23
README.md CHANGED
@@ -12,17 +12,20 @@ tags:
12
  Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter),
13
  a masked-span speech editor.
14
 
15
- On the English `full` split of Ming-Freeform-Audio-Edit, against Ren et al.
16
- ([arXiv:2602.00560](https://arxiv.org/abs/2602.00560), the row with GRPO):
 
 
 
17
 
18
- | edit type | metric | Ren et al. | wavepainter (a=0.85) | wavepainter (a=0.70) |
19
- |---|---|---|---|---|
20
- | substitution | WER | 4.41 | **3.063** | 3.381 |
21
- | insertion | WER | 4.97 | 4.128 | **3.817** |
22
- | deletion | WER | **6.88** | 9.136 | 9.525 |
23
- | substitution | SIM | 0.78 | **0.943** | **0.943** |
24
- | insertion | SIM | 0.82 | **0.961** | 0.960 |
25
- | deletion | SIM | 0.78 | **0.918** | **0.918** |
26
 
27
  Five of the six compared numbers improve on theirs: substitution and insertion
28
  WER, and speaker similarity on all three edit types. We do not beat them on
@@ -31,26 +34,34 @@ gap with a ground-truth waveform splice.
31
 
32
  Signal quality is reported full-reference, since a no-reference MOS estimator
33
  rates the untouched original recordings below synthesised output on this
34
- benchmark. Over the audio the edit does not touch, PESQ is 4.089 / 4.083 / 4.076
35
  for substitution / insertion / deletion, against 4.351 for copy-synthesis with no
36
  edit at all.
37
 
38
- ### Which checkpoint to use
39
 
40
- `phase2-hubert-a085` is the released model and the artifact of record.
41
- `phase2-hubert-a070` is the same phase-2 child interpolated back toward the
42
- phase-1 base at a lower coefficient: it is better on insertion (3.82 vs 4.13
43
- WER) and worse on substitution and deletion. The interpolation coefficient has
44
- no single optimum across edit types, so both are published rather than one
45
- being presented as strictly better. Speaker similarity is identical between
46
- them -- the coefficient moves only the diffusion denoiser.
 
 
 
 
 
 
 
 
47
 
48
  ## Files
49
 
50
- - `phase1-base/` the phase-1 editor. Start here to re-run phase 2 (~1 GPU-hour).
51
- - `phase2-hubert-a085/` the released model, and the artifact of record.
52
- - `phase2-hubert-a070/` the same child at a lower interpolation coefficient;
53
- better insertion, worse substitution and deletion.
54
 
55
  ## Use
56
 
 
12
  Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter),
13
  a masked-span speech editor.
14
 
15
+ `phase2-hubert-child` is the released model and the artifact of record: phase two
16
+ as trained, and the checkpoint every number in the paper comes from. On the
17
+ English `full` split of Ming-Freeform-Audio-Edit, against Ren et al.
18
+ ([arXiv:2602.00560](https://arxiv.org/abs/2602.00560), the row with GRPO), scored
19
+ with oracle edit spans taken from the transcript diff:
20
 
21
+ | edit type | metric | Ren et al. | wavepainter |
22
+ |---|---|---|---|
23
+ | substitution | WER | 4.41 | **3.479** |
24
+ | insertion | WER | 4.97 | **4.400** |
25
+ | deletion | WER | **6.88** | 9.697 |
26
+ | substitution | SIM | 0.78 | **0.943** |
27
+ | insertion | SIM | 0.82 | **0.961** |
28
+ | deletion | SIM | 0.78 | **0.918** |
29
 
30
  Five of the six compared numbers improve on theirs: substitution and insertion
31
  WER, and speaker similarity on all three edit types. We do not beat them on
 
34
 
35
  Signal quality is reported full-reference, since a no-reference MOS estimator
36
  rates the untouched original recordings below synthesised output on this
37
+ benchmark. Over the audio the edit does not touch, PESQ is 4.090 / 4.083 / 4.079
38
  for substitution / insertion / deletion, against 4.351 for copy-synthesis with no
39
  edit at all.
40
 
41
+ ### The interpolated variants
42
 
43
+ Two earlier releases interpolate the phase-two child back toward the phase-1 base,
44
+ which moves the diffusion denoiser only. They are published because the paper's
45
+ first version reported them and anyone who fetched them should keep working; the
46
+ paper no longer reports either.
47
+
48
+ | | substitution | insertion | deletion |
49
+ |---|---|---|---|
50
+ | `phase2-hubert-child` (released) | **3.479** | 4.400 | **9.697** |
51
+ | `phase2-hubert-a085` | 3.063 | 4.128 | 9.136 |
52
+ | `phase2-hubert-a070` | 3.381 | **3.817** | 9.525 |
53
+
54
+ WER, lower is better. The coefficient has no single optimum across edit types, so
55
+ both variants are kept rather than one being presented as strictly better.
56
+ Speaker similarity is 0.943 / 0.961 / 0.918 for all three except insertion at
57
+ a=0.70, which is 0.960.
58
 
59
  ## Files
60
 
61
+ - `phase1-base/` -- the phase-1 editor. Start here to re-run phase 2 (~1 GPU-hour).
62
+ - `phase2-hubert-child/` -- the released model, and the artifact of record.
63
+ - `phase2-hubert-a085/` -- an earlier release, interpolated at 0.85.
64
+ - `phase2-hubert-a070/` -- the same, at 0.70.
65
 
66
  ## Use
67