VocalTwin

Personalised voice cloning + TTS
Train on your own recordings – then turn any text into speech that sounds like you.

---

Table of Contents

1. Features
2. Quick Start
3. Installation
4. Usage
   • Train only
   • Synthesize only
   • Both in one pass
5. Directory Layout
6. Troubleshooting & Tips
7. Contributing
8. License
9. Acknowledgements

---

Features

• OpenVoice V2 tone-colour converter
• MeloTTS multilingual base TTS (EN · ES · FR · ZH · JA · KO …)
• Simple CLI – train, synthesize, train_and_synthesize
• Works with one or many MP3 samples (more = better)
• Pure-Python, GPU-accelerated (falls back to CPU)
• Clean, class-based code; easy to extend

---

Quick Start

Five-minute demo (Linux / macOS / WSL)

git clone https://github.com/shamspias/VocalTwin.git
cd VocalTwin
python -m venv .venv && source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python -m unidic download                                # one-time

# download OpenVoice V2 checkpoints (~600 MB)
curl -L https://myshell-public-repo-host.s3.amazonaws.com/openvoice/checkpoints_v2_0417.zip -o ckpt.zip
unzip ckpt.zip -d checkpoints_v2 && rm ckpt.zip

# drop a few clean MP3s into audio_samples/  (≈30-60 s total audio)
# create texts/hello.txt with some text

python main.py train_and_synthesize --language EN
# → outputs/hello.wav  (your voice!)

---

Installation

Prerequisites

• Python 3.9 – 3.10
• ffmpeg in your PATH
• NVIDIA GPU with CUDA 11+ recommended (CPU works, just slower)

# clone + create venv
git clone https://github.com/shamspias/VocalTwin.git
cd VocalTwin
python -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\activate

# install deps
pip install --upgrade pip
pip install -r requirements.txt
python -m unidic download            # for Japanese tokeniser

# fetch OpenVoice V2 converter checkpoints
mkdir -p checkpoints_v2
curl -L https://myshell-public-repo-host.s3.amazonaws.com/openvoice/checkpoints_v2_0417.zip -o ckpt.zip
unzip ckpt.zip -d checkpoints_v2 && rm ckpt.zip

---

Usage

1. Train on your voice

Put MP3s in audio_samples/ (more = better):

python main.py train

Creates checkpoints/target_se.pth – your speaker embedding.

---

2. Synthesize text

Add one or more .txt files to texts/ and run:

python main.py synthesize --language EN   # EN / ES / FR / ZH / JA / KO

Each .txt → .wav with your voice in outputs/.

---

3. All-in-one

python main.py train_and_synthesize --language EN

---

CLI reference

python main.py --help

usage: VocalTwin [-h] [--audio_dir AUDIO_DIR] [--text_dir TEXT_DIR]
                 [--checkpoint_dir CHECKPOINT_DIR] [--output_dir OUTPUT_DIR]
                 [--language LANGUAGE]
                 {train,synthesize,train_and_synthesize}

Positional arguments:
  {train,synthesize,train_and_synthesize}
                        Action to perform

Optional arguments:
  --audio_dir AUDIO_DIR         MP3 training recordings           [audio_samples]
  --text_dir TEXT_DIR           Input .txt files                  [texts]
  --checkpoint_dir CHECKPOINT_DIR
                                Model checkpoints + target SE     [checkpoints]
  --output_dir OUTPUT_DIR       Generated WAVs                    [outputs]
  --language LANGUAGE           TTS language code (MeloTTS)       [EN]

---

Directory Layout

VocalTwin/
├── audio_samples/         # your MP3 recordings
├── texts/                 # input .txt
├── outputs/               # generated WAV
├── checkpoints/           # target_se.pth lives here
├── checkpoints_v2/        # OpenVoice converter checkpoints
├── src/
│   ├── trainer.py         # extracts speaker embedding
│   ├── synthesizer.py     # TTS + tone-colour conversion
│   └── utils.py           # helpers (placeholder)
├── main.py                # CLI
└── requirements.txt

---

Troubleshooting & Tips

• Noise matters – recordings should be clear, 16 kHz+ preferred.
• Short texts (< 3 s) sometimes clip; add punctuation or line-breaks.
• On CPU the conversion step is slow; expect ~1 × RT or worse.
• “target_se.pth not found”? Run the train step first or check paths.

---

Contributing

Bug reports & PRs are welcome!

1. Fork → feature branch → PR to main
2. Follow the existing code style (black + isort).
3. Add/update docstrings and a short demo if introducing a new feature.

---

License

VocalTwin is released under the MIT License – see LICENSE.

---

Acknowledgements

• OpenVoice V2 – tone-colour converter
• MeloTTS – multilingual neural TTS
• Demo recordings courtesy of the open-source speech community

---

Made with Frustration 🫤 by @shamspias