---
title: Phoneme Aligner
emoji: 🗣️
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 6.26.0
python_version: "3.12"
app_file: app.py
pinned: false
license: mit
short_description: Unofficial third-party demo - align speech with its phonemes
preload_from_hub:
  - yky-h/japanese-hubert-base
  - facebook/wav2vec2-large-xlsr-53
---

# Phoneme and speech aligner

> **Unofficial demo.** Built by a third party, with no connection to the
> paper's author or to the `CyberAgentAILab/vae_speech_align` repository.
> It uses their published code and pretrained models.
>
> **非公式デモです。** 本デモは論文著者および
> `CyberAgentAILab/vae_speech_align` リポジトリとは無関係の、第三者が作成した
> ものです。公開されているコードと学習済みモデルを利用しています。

An interactive demo of
[vae_speech_align](https://github.com/CyberAgentAILab/vae_speech_align):
give it speech and the words spoken in it, and it shows where every
phoneme starts and ends. Nothing to download — the result is a playable
ribbon under the spectrogram, where each phoneme can be clicked and
heard on its own.

This directory *is* the Hugging Face Space: `app.py`,
`requirements.txt`, `assets/` and this README are everything it needs.

## Running it locally

From the repository root:

```bash
uv run --with gradio python demo/app.py
```

Configs, phoneme inventories and the sample recordings are bundled in
`assets/`, so they are served from the application directory and the
demo needs no repository checkout. Only the pretrained checkpoints are
downloaded, from
[GitHub Releases](https://github.com/CyberAgentAILab/vae_speech_align/releases/tag/models-v3),
into `~/.cache/vae_speech_align_demo` (set `VSA_DEMO_CACHE` to move
that) — unless `example/inference/{ja,en}/model.safetensors` already
exist in a checkout.

`assets/` is copied from `example/inference/` in the upstream
repository; refresh it from there if those samples or configs change.

## Deploying to a Space

Create the Space (SDK: Gradio, hardware: CPU basic is enough), then push
this directory to it as the repository root:

```bash
git remote add space https://huggingface.co/spaces/<owner>/<space-name>
git subtree push --prefix demo space main
```

`.github/workflows/space.yml` does the same on every push to `main`
once the repository has an `HF_TOKEN` secret and an `HF_SPACE`
variable (`<owner>/<space-name>`); without them it does nothing.

Notes for the Space:

- `preload_from_hub` bakes the two SSL feature extractors into the
  image so the first alignment does not wait on a 1.2 GB download.
  Drop those lines if the build rejects them.
- The app preloads both alignment models in a background thread at
  startup, so the page is usable while they load.
- `requirements.txt` installs the package from `main`. Pin it to a tag
  or commit if you want the Space frozen to a known revision.
- CPU basic (2 vCPU, 16 GB) is enough: both models together need about
  2 GB of RAM, and a ten-second recording aligns in roughly a second.