---
license: mit
library_name: pytorch
tags:
  - speech-editing
  - speech-synthesis
  - text-to-speech
---

# wavepainter

Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter),
a masked-span speech editor.

`phase2-hubert-child` is the released model and the artifact of record: phase two
as trained, and the checkpoint every number in the paper comes from. On the
English `full` split of Ming-Freeform-Audio-Edit, against Ren et al.
([arXiv:2602.00560](https://arxiv.org/abs/2602.00560), the row with GRPO), scored
with oracle edit spans taken from the transcript diff:

| edit type | metric | Ren et al. | wavepainter |
|---|---|---|---|
| substitution | WER | 4.41 | **3.479** |
| insertion | WER | 4.97 | **4.400** |
| deletion | WER | **6.88** | 9.697 |
| substitution | SIM | 0.78 | **0.943** |
| insertion | SIM | 0.82 | **0.961** |
| deletion | SIM | 0.78 | **0.918** |

Five of the six compared numbers improve on theirs: substitution and insertion
WER, and speaker similarity on all three edit types. We do not beat them on
deletion WER -- see the repository's Limitations, which also locates most of that
gap with a ground-truth waveform splice.

Signal quality is reported full-reference, since a no-reference MOS estimator
rates the untouched original recordings below synthesised output on this
benchmark. Over the audio the edit does not touch, PESQ is 4.090 / 4.083 / 4.079
for substitution / insertion / deletion, against 4.351 for copy-synthesis with no
edit at all.

### The interpolated variants

Two earlier releases interpolate the phase-two child back toward the phase-1 base,
which moves the diffusion denoiser only. They are published because the paper's
first version reported them and anyone who fetched them should keep working; the
paper no longer reports either.

| | substitution | insertion | deletion |
|---|---|---|---|
| `phase2-hubert-child` (released) | 3.479 | 4.400 | 9.697 |
| `phase2-hubert-a085` | 3.063 | 4.128 | 9.136 |
| `phase2-hubert-a070` | 3.381 | 3.817 | 9.525 |

WER, lower is better. Nothing is bolded because the released checkpoint is not
the best row: interpolating toward the phase-1 base improves every WER column,
a=0.85 most on substitution and deletion and a=0.70 on insertion. We publish the
child as the artifact of record anyway, because it is the model phase two
produced and the coefficient was chosen by looking at these scores. The
coefficient also has no single optimum across edit types, which is why both
variants are kept rather than one being presented as strictly better. Speaker
similarity is 0.943 / 0.961 / 0.918 for all three except insertion at a=0.70,
which is 0.960.

## Files

- `phase1-base/` -- the phase-1 editor. Start here to re-run phase 2 (~1 GPU-hour).
- `phase2-hubert-child/` -- the released model, and the artifact of record.
- `phase2-hubert-a085/` -- an earlier release, interpolated at 0.85.
- `phase2-hubert-a070/` -- the same, at 0.70.

## Use

```bash
git clone https://github.com/pujariaditya/wavepainter && cd wavepainter
pip install -e . && ./setup.sh
python scripts/download_weights.py
./scripts/verify_benchmark.sh
```

Checkpoints are sha256-pinned in `scripts/download_weights.py`. Licence and
attribution, including for the components these weights derive from, are in the
repository's `ATTRIBUTION.md`.