Dub a video in your own voice

# Assemble Decosa consented creator dubbing on this machine

You are setting up consented dubbing. It takes a creator's video and gives back an extra Spanish audio track in a
voice whose owner consented to this use. The track is cut to exactly the video's length, and it comes with subtitles, a
subtitle QA report, a hand-off script for human dubbers, and a signed record plus a C2PA label once a person approves.
Work step by step. Show me each command before you run anything with `sudo`, and stop to ask if a check fails.

## Step 0: set up with a coding agent, rehearse on mock data, then go private

This prompt is for a coding agent running on the machine that will host the service. We recommend Claude Code with
Claude Opus 5.5; any capable coding agent works. Work in this order:

1. Set up on mock data only. During the whole setup you (the agent) work with the synthetic sample bundle below and
   nothing else. Do not ask me for real data, and do not open, read, list or copy files that hold real data, even to
   "test with something realistic".
2. Rehearse. When the steps below are done and the service is healthy, fetch the mock-data bundle for this tool,
   https://decosa.ai/samples/consented-dubbing.zip (756 KB, 17 checks, synthetic or openly licensed: see `licence` in expected.json),
   show me what is in it, and run the rehearsal against the local API:
   `docker compose exec api python scripts/rehearse.py consented-dubbing` (the api image carries the same bundle under /app/rehearsal/consented-dubbing/;
   with no key set, the script asks the local API for a short demo token). From a decosa-api checkout instead:
   `python scripts/rehearse.py consented-dubbing --bundle consented-dubbing.zip --base-url http://127.0.0.1:<PORT>`.
   It sends the mock inputs to the local API and prints PASS or FAIL for each expected property (for example: "the consent gate allows Mara Quill's own voice for her channel", "a dub in Theo Marsh's voice is refused because he revoked his consent", "the refusal is a receipted consent-ledger decision and no voice was rendered"). Show me
   the full output. Every check must pass. If one fails, fix the install and run it again; never edit `expected.json`
   to make a check pass.
3. Stop there. Once the rehearsal passes, tell me, and I will run my own data against the local API myself, on this
   machine.

For the person running this: a coding agent that runs in the cloud sees everything in its context, including files it
reads, command output and anything pasted into the chat. Keep real data out of the chat and out of anything the agent
can read. Switch to your own data only after the rehearsal has passed and the agent's work is done.

## 0. Ground rules and licences
- Models: **Voxtral Mini 4B Realtime** (ASR, Apache-2.0) and **Qwen3.8-27B** (translation, Apache-2.0).
  **Chatterbox Multilingual** (voice, MIT, `ResembleAI/chatterbox` @ `5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18`, the
  `t3_23lang` weights that `chatterbox-tts==0.1.4` loads). Keep its Perth watermark on.
- **Every voice render needs consent.** This stack refuses to render any voice without an active entry in the Decosa
  consent ledger. The entry must cover this project, the purpose (dubbing, narration or accessibility), the territory
  and today's date. Don't bypass the gate. A voice-clone tool without one is the kind of tool Tennessee's ELVIS Act
  (in force 1 Jul 2024) makes its distributor liable for.
- Enrol real people only with their recorded consent. A living performer licensing a replica to someone else needs
  counsel or union representation in the entry (California AB 2602 and New York S7676B test for it). The consent
  ledger's own assemble prompt explains the entry fields.
- Uploaded video is only transcribed. Its audio is never used as a voice. The voice reference is the consent clip you
  enrolled.
- Nothing is published until a person presses Approve. Never replace the original track.
- Bind every port to 127.0.0.1. Logs carry ids, counts and timings, never transcript or translation text. Keep it that
  way.

## 1. Check the machine
1. `nvidia-smi`. Qwen3.8-27B NVFP4 needs about 57 GB and Voxtral about 24 GB; a 96 GB card holds both. If you already
   run them, just point at them. The voice model runs on CPU at about 3.3x real time (a 57 s video took about 4 min on
   our box) or on a GPU with at least 8 GB free at about 0.25x real time.
2. `docker --version` and `docker compose version`. If Docker or the NVIDIA container toolkit is missing, ask me, then
   install them from the official repositories.
3. Disk: about 10 GB for the voice runtime and weights (CPU torch plus 3 GB of Chatterbox files), plus the model servers.

## 2. Model servers
Use the Decosa ASR and LLM images (`${DECOSA_REGISTRY}/decosa-asr:<tag>` and `decosa-llm:<tag>`, **publishing
soon**), or any vLLM that serves:
- Voxtral as a realtime WebSocket at `ws://127.0.0.1:8090/v1/realtime` (served name `voxtral-realtime`);
- Qwen3.8-27B as an OpenAI endpoint at `http://127.0.0.1:8114/v1`. Use its served name below; ours is `qwen3.8-27b`.

## 3. Build the api image with the voice runtime
`${DECOSA_REGISTRY}/decosa-api:<tag>` is **publishing soon**. Until then, build from source:
`git clone <decosa-api source: on request at https://decosa.ai/contact?topic=self-host>` (access required). Check out a release that contains
`decosa_api/verticals/dubbing/` and `decosa_api/verticals/consent/`, then run
`docker build -f docker/api/Dockerfile -t decosa-api:local .`. Then add the voice runtime in its own venv. Its
dependencies pin older numpy and torch than the api's, so it runs as a separate process:
```dockerfile
# ./api-dub/Dockerfile
FROM decosa-api:local
USER root
# chatterbox-tts pins numpy<1.26, which has no wheels for the image's Python 3.12: the voice venv uses Python 3.11 via uv
RUN pip install --no-cache-dir "c2pa-python>=0.37" "pillow>=10" uv && \
    UV_PYTHON_INSTALL_DIR=/opt/uv-python uv venv -p 3.11 /opt/chatterbox && \
    uv pip install --python /opt/chatterbox/bin/python --no-cache --index-strategy unsafe-best-match \
      --extra-index-url https://download.pytorch.org/whl/cpu "torch==2.6.0+cpu" "torchaudio==2.6.0+cpu" \
      "numpy>=1.24,<1.26" librosa==0.11.0 s3tokenizer transformers==4.46.3 diffusers==0.29.0 resemble-perth==1.0.1 \
      conformer==0.3.2 safetensors==0.5.3 pykakasi==2.3.0 soundfile huggingface_hub "setuptools<70" && \
    uv pip install --python /opt/chatterbox/bin/python --no-cache --no-deps chatterbox-tts==0.1.4 && \
    chmod -R a+rX /opt/uv-python /opt/chatterbox && mkdir -p /hf && chown decosa /hf
USER decosa
ENV DECOSA_DUB_TTS_PYTHON=/opt/chatterbox/bin/python HF_HOME=/hf
```
`docker build -t decosa-api:dub ./api-dub`. Why these pins: `pkuseg` (Chinese only) doesn't build, so the package goes
in with `--no-deps`. The Perth watermarker imports `pkg_resources`, hence `setuptools<70`. For a GPU voice, build a
second venv with CUDA torch (we use `torch==2.7.1+cu128` on Blackwell), set `DECOSA_DUB_TTS_PYTHON_GPU` to it and
`DECOSA_DUB_TTS_DEVICE=auto`. The model then loads per job and only when the GPU has 8 GB free.

## 4. docker-compose.yml
Write this in `~/decosa/dubbing/`. `network_mode: host` lets the api reach your model servers on 127.0.0.1.
```yaml
services:
  api:
    image: decosa-api:dub
    network_mode: host
    environment:
      DECOSA_HOST: 127.0.0.1
      DECOSA_PORT: "8445"
      DECOSA_DATA_DIR: /data
      DECOSA_ASR_WS: ws://127.0.0.1:8090/v1/realtime
      DECOSA_ASR_MODEL: voxtral-realtime
      DECOSA_LLM_ROUTE: direct
      DECOSA_LLM_URL: http://127.0.0.1:8114/v1
      DECOSA_LLM_MODEL: qwen3.8-27b
      DECOSA_DUB_TTS_DEVICE: cpu
      DECOSA_PROVENANCE_DIR: /provenance
    volumes: ["decosa-data:/data", "decosa-provenance:/provenance", "hf-cache:/hf"]
    healthcheck: { test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8445/dubbing/info', timeout=4)"], interval: 30s, retries: 10 }
volumes:
  decosa-data:
  decosa-provenance:
  hf-cache:
```
Use named volumes, never host folders: the api runs as uid 10001, and a host folder Docker creates belongs to root.
`docker compose up -d`, wait for the health check, then create the C2PA signing material once:
`docker compose exec api python scripts/provenance_devcert.py && docker compose restart api`. It is a development CA,
so public validators show the signature as valid and the issuer as untrusted. On first start the api creates this box's
Ed25519 key in `/data/attest/`; the ledger, receipts and records are signed with it. Back it up with
`docker compose cp api:/data/attest ./attest-backup` and never print it.

## 5. Smoke test
```bash
B=http://127.0.0.1:8445
curl -s $B/dubbing/info | jq '{tts: .models.tts.licence, ledger: .consent.ledger}'          # "MIT", true
T=$(curl -s -XPOST $B/demo/session -H 'content-type: application/json' -d '{"vertical":"consented-dubbing"}' | jq -r .token)
curl -s -XPOST $B/dubbing/jobs -H "authorization: Bearer $T" -H 'content-type: application/json' -d '{"sample_id":"revoked"}' | jq '.consent.code'   # "revoked"
J=$(curl -s -XPOST $B/dubbing/jobs -H "authorization: Bearer $T" -H 'content-type: application/json' -d '{"sample_id":"own-voice"}' | jq -r .job_id)
until curl -s $B/dubbing/jobs/$J -H "authorization: Bearer $T" | jq -e '.status!="queued" and .status!="running"' >/dev/null; do sleep 10; done
curl -s $B/dubbing/jobs/$J -H "authorization: Bearer $T" | jq '{status, exact: .track.exact, samples: .track.samples, glossary: .translation.glossary.rate, qa: .qa.target.verdict}'
H=$(curl -s $B/dubbing/jobs/$J -H "authorization: Bearer $T" | jq -r .review_hash)
curl -s -XPOST $B/dubbing/jobs/$J/approve -H "authorization: Bearer $T" -H 'content-type: application/json' \
  -d "{\"approver\":\"Self-host test\",\"review_hash\":\"$H\",\"confirm\":true}" | jq '.release | {label, exact, c2pa: .c2pa.validation_state}'
curl -s $B/dubbing/releases/$J/record.json | jq '{record: .}' | curl -s -XPOST $B/record/verify -H 'content-type: application/json' -d @- | jq .ok
```
Expect: the revoked voice refused before any work; the dub in `review` with `exact: true` and 2,736,000 samples (57 s
at 48 kHz); the release labelled "AI-dubbed, voice consented by Mara Quill (consent entry ce_…)" with the C2PA state
`Valid`; the record verifying `true`. On CPU the first dub took 337 s in our clean-room test (including the 3 GB weight download). Report what you measure.

## 6. Your own voices and videos
Enrol each voice in the consent ledger (`POST /consent/entries`, with the consent clip). Then list it in
`DECOSA_DUB_VOICES_DIR/voices.json` (same shape as `decosa_api/verticals/dubbing/data/voices.json`, with the consent
clip next to it) and restart. Upload a video with `POST /dubbing/uploads` (at most 60 MB and 3 minutes). Then
`POST /dubbing/jobs` with `upload_id`, `voice`, `project`, `territory` and your `glossary`.

## 7. Point the app at the local API
Set `NEXT_PUBLIC_DECOSA_API=http://127.0.0.1:8445` in the site's `.env.local`. The contract is in `API_CONTRACT.md`,
section "Consented creator dubbing".