Turn a client call into notes

# Assemble Decosa privileged call notes on this machine

You are setting up self-hosted call notes for a law firm on this Linux machine. A client call (recorded live in the browser, or uploaded afterwards) becomes: a speaker-attributed transcript, an intake memo where every line cites transcript lines and is checked against them, the names heard for a conflicts check, a time entry rounded up to 0.1 hour with a billing narrative, follow-up tasks, a client email draft (English or Spanish), and a session record signed with this machine's own key. The call audio is held in memory and dropped once the transcript is made. Work step by step, show me each command before running anything that needs sudo, and stop if a check fails.

**Before anything else, remind me:**
- Client calls are privileged and confidential. Keep everything on this machine: the model route stays local (`direct`), and nothing goes to a hosted service.
- Get recording consent first. Some states (for example California, Penal Code 632) require every party's consent; the API refuses to record or process a call until a consent statement is given, and applies the strictest rule of the states involved. That is a prompt, not legal advice.
- The memo is a draft for a lawyer to check. Speech recognition mishears names and numbers; deadlines are only the ones said on the call; the time entry follows your billing rules.

Repeat these points in your final summary.

## Step 0: set up with a coding agent, rehearse on mock data, then go private

This prompt is for a coding agent running on the machine that will host the service. We recommend Claude Code with
Claude Opus 5.5; any capable coding agent works. Work in this order:

1. Set up on mock data only. During the whole setup you (the agent) work with the synthetic sample bundle below and
   nothing else. Do not ask me for real data, and do not open, read, list or copy files that hold real data, even to
   "test with something realistic".
2. Rehearse. When the steps below are done and the service is healthy, fetch the mock-data bundle for this tool,
   https://decosa.ai/samples/privileged-call-notes.zip (3 KB, 10 checks, synthetic or openly licensed: see `licence` in expected.json),
   show me what is in it, and run the rehearsal against the local API:
   `docker compose exec api python scripts/rehearse.py privileged-call-notes` (the api image carries the same bundle under /app/rehearsal/privileged-call-notes/;
   with no key set, the script asks the local API for a short demo token). From a decosa-api checkout instead:
   `python scripts/rehearse.py privileged-call-notes --bundle privileged-call-notes.zip --base-url http://127.0.0.1:<PORT>`.
   It sends the mock inputs to the local API and prints PASS or FAIL for each expected property (for example: "the memo has at least eight facts", "opposing counsel is on the conflicts list", "the 159-second call is a 0.1-hour entry"). Show me
   the full output. Every check must pass. If one fails, fix the install and run it again; never edit `expected.json`
   to make a check pass.
3. Stop there. Once the rehearsal passes, tell me, and I will run my own data against the local API myself, on this
   machine.

For the person running this: a coding agent that runs in the cloud sees everything in its context, including files it
reads, command output and anything pasted into the chat. Keep real data out of the chat and out of anything the agent
can read. Switch to your own data only after the rehearsal has passed and the agent's work is done.

## What you are building

| service | image | model | port |
|---|---|---|---|
| `llm` | `${DECOSA_REGISTRY}/decosa-llm:0.1.0` (vLLM 0.29.0, `vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1`) | `nvidia/Qwen3.8-27B-NVFP4` @ `482ca0f3832238542f8f5295dde86b5f22711d80`, Apache-2.0 | internal 8000 |
| `asr` | `${DECOSA_REGISTRY}/decosa-asr:0.1.0` (vLLM 0.27.1 + `mistral-common[audio]`) | `mistralai/Voxtral-Mini-4B-Realtime-2602`, Apache-2.0 | internal 8000 |
| `api` | `${DECOSA_REGISTRY}/decosa-api:0.1.0` (no GPU) | none | `127.0.0.1:8445` |
| `diarize` | built from the decosa-api source (`services/diarize`) | `OpenMOSS-Team/MOSS-Transcribe-Diarize` @ `704aa4a9c304e8520be88901e0d1960158ef5b15`, Apache-2.0 | internal 8092 |

Any OpenAI-compatible endpoint serving an open model can replace `llm` (set `DECOSA_LLM_URL` and `DECOSA_LLM_MODEL`); the measured setup is the one above.

## 1. Check the GPU, driver and Docker

1. Run `nvidia-smi`. I need one NVIDIA GPU with at least 48 GB and driver 580 or newer.
   - Blackwell (RTX PRO 6000, B200): use the defaults below (NVFP4).
   - Hopper (H100/H200): set `LLM_MODEL=Qwen/Qwen3.8-27B-FP8`, `LLM_REVISION=main`, `LLM_GPU_UTIL=0.62`.
   - 48 GB Ada/L40S: FP8 checkpoint as above, plus `LLM_MAX_LEN=16384`, `LLM_GPU_UTIL=0.70`, `ASR_GPU_UTIL=0.22`, `DECOSA_LIVE_CAP=2`.
   - Under 48 GB: stop and tell me it will not fit.
   Only the Blackwell defaults have been measured; the other rows are starting points.
2. Check `docker --version` and `docker compose version`. If Docker is missing, install Docker Engine from Docker's official apt/dnf repository for this distro.
3. Check `docker run --rm --gpus all ubuntu nvidia-smi`. If it fails, install the NVIDIA Container Toolkit (`nvidia-container-toolkit`) from NVIDIA's repository, run `sudo nvidia-ctk runtime configure --runtime=docker` and restart Docker.
4. Confirm about 80 GB of free disk for images and weights.

## 2. Get the images

The images are **on request** while self-host is in early access: ask at https://decosa.ai/contact?topic=self-host, and Decosa sends the registry (set it as `DECOSA_REGISTRY`), pull access and the compose file. Try `docker pull ${DECOSA_REGISTRY}/decosa-{llm,asr,api}:0.1.0`. If a pull fails (not published yet, or no access), build from source once the `decosa-api` source is published: clone it, then `docker compose build llm asr api` in the repo, which builds the same tags from `docker/`. If neither works, stop and tell me.

## 3. Write the compose file

Create `~/decosa-callnotes/.env`:

```bash
DECOSA_TAG=0.1.0
DECOSA_GPU=0
LLM_MODEL=nvidia/Qwen3.8-27B-NVFP4
LLM_REVISION=482ca0f3832238542f8f5295dde86b5f22711d80
LLM_MAX_LEN=65536
LLM_GPU_UTIL=0.60
ASR_GPU_UTIL=0.25
DECOSA_LLM_ROUTE=direct   # local model; receipts are signed by this box's own key ("attested")
DECOSA_SIGNER_NAME="<who signs these records, e.g. Example Family Law PLLC>"
DECOSA_LIVE_CAP=4
```

Create `~/decosa-callnotes/docker-compose.yml` with exactly these services:

```yaml
name: decosa-callnotes
x-gpu: &gpu
  deploy: { resources: { reservations: { devices: [ { driver: nvidia, device_ids: ["${DECOSA_GPU:-0}"], capabilities: [gpu] } ] } } }
x-health: &health
  interval: 15s
  timeout: 5s
  retries: 5
services:
  llm:
    image: ${DECOSA_REGISTRY}/decosa-llm:${DECOSA_TAG}
    <<: *gpu
    ipc: host
    restart: unless-stopped
    volumes: [hf-cache:/root/.cache/huggingface]
    command: ["${LLM_MODEL}", "--revision", "${LLM_REVISION}", "--served-model-name", "qwen3.8-27b",
              "--language-model-only", "--max-model-len", "${LLM_MAX_LEN}", "--gpu-memory-utilization", "${LLM_GPU_UTIL}",
              "--max-num-seqs", "16", "--kv-cache-dtype", "fp8_e4m3", "--speculative-config", '{"method":"mtp","num_speculative_tokens":3}',
              "--seed", "0", "--enable-force-include-usage", "--disable-uvicorn-access-log", "--host", "0.0.0.0", "--port", "8000"]
    healthcheck: { <<: *health, test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4)"], start_period: 900s }
  asr:
    image: ${DECOSA_REGISTRY}/decosa-asr:${DECOSA_TAG}
    <<: *gpu
    ipc: host
    restart: unless-stopped
    depends_on: { llm: { condition: service_healthy } }   # start after llm so the memory split is stable
    volumes: [hf-cache:/root/.cache/huggingface]
    command: ["--model", "mistralai/Voxtral-Mini-4B-Realtime-2602", "--tokenizer-mode", "mistral", "--config-format", "mistral",
              "--load-format", "mistral", "--compilation-config", '{"cudagraph_mode":"PIECEWISE"}', "--max-model-len", "45000",
              "--max-num-batched-tokens", "8192", "--max-num-seqs", "16", "--gpu-memory-utilization", "${ASR_GPU_UTIL}",
              "--served-model-name", "voxtral-realtime", "--host", "0.0.0.0", "--port", "8000"]
    healthcheck: { <<: *health, test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4)"], start_period: 600s }
  api:
    image: ${DECOSA_REGISTRY}/decosa-api:${DECOSA_TAG}
    restart: unless-stopped
    depends_on: { llm: { condition: service_healthy }, asr: { condition: service_healthy } }
    environment:
      DECOSA_ASR_WS: ws://asr:8000/v1/realtime
      DECOSA_LLM_ROUTE: ${DECOSA_LLM_ROUTE}
      DECOSA_LLM_URL: http://llm:8000/v1
      DECOSA_LLM_MODEL: qwen3.8-27b
      DECOSA_LIVE_CAP: ${DECOSA_LIVE_CAP}
      DECOSA_SESSIONS_PER_IP_HOUR: "1000"
      DECOSA_BUDGET_AUDIO_S: "3600"                   # per session; one call up to 30 min (DECOSA_CALLNOTES_MAX_AUDIO_S)
      DECOSA_BUDGET_LLM_TOKENS: "200000"
      DECOSA_SESSION_TTL_S: "28800"
      DECOSA_LOCAL_SIGNING: "on"                      # Ed25519 key created at /data/attest/ed25519.pem on first start
      DECOSA_SIGNER_NAME: ${DECOSA_SIGNER_NAME}
      DECOSA_DIARIZE_URL: ""                          # step 4 sets this if you add speaker labels
      DECOSA_CORS_ORIGIN_REGEX: '^https?://(localhost|127\.0\.0\.1)(:\d+)?$$'
    ports: ["127.0.0.1:8445:8445"]
    volumes: [decosa-data:/data]
    healthcheck: { <<: *health, test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8445/healthz', timeout=4)"], start_period: 20s }
volumes: { hf-cache: {}, decosa-data: {} }
```

Run `docker compose up -d`, then poll `docker compose ps` until all three are healthy (the LLM takes 5–10 minutes the first time) and `curl -s localhost:8445/healthz` shows `"asr": true, "llm": true`. If `llm` runs out of memory, lower `LLM_GPU_UTIL` or `LLM_MAX_LEN`; the two GPU shares must add up to less than about 0.9.

## 4. Speaker labels (diarize service)

The `diarize` service (MOSS-Transcribe-Diarize 0.9B, Apache-2.0) has no published image yet. If I want speaker labels, run `services/diarize` from the decosa-api source on this box (install with the `uv` commands at the top of `services/diarize/requirements.txt`, then `DIARIZE_HOST=172.17.0.1 DIARIZE_PORT=8092 DIARIZE_DEVICE=cuda:0 .venv-diarize/bin/python services/diarize/server.py`; `172.17.0.1` is the docker bridge address from `ip -4 addr show docker0`, reachable from containers and not from the LAN). Add `extra_hosts: ["host.docker.internal:host-gateway"]` to `api` and set `DECOSA_DIARIZE_URL: http://host.docker.internal:8092`. `curl -s localhost:8445/healthz` then shows `"diarize": true`. The 0.9B model needs about 2 GB of weights; lower `LLM_GPU_UTIL` to 0.55 so it fits. That fit is an estimate, not measured. For call notes this step is **required for uploads** (`POST /callnotes/audio` answers 503 without it) and strongly recommended live: without it the memo is written from the live captions, with no Attorney/Client labels.

## 5. The signing key

1. `curl -s localhost:8445/attest/signing-key` shows the public key. Show me the `pubkey`, and tell me to back up the `decosa-data` volume (it holds `/data/attest/ed25519.pem`).
2. Receipts from this box say `status: "attested"`: signed by its own key. They show nothing was changed after signing and who signed; they do not prove the model heard correctly.

## 6. Smoke test: a synthetic call, uploaded

```bash
API=localhost:8445
curl -s "$API/callnotes/consent-rules?attorney_state=TX&client_state=CA" | jq '{rule, prompt}'
TOKEN=$(curl -s $API/demo/session -H 'content-type: application/json' -d '{"vertical":"privileged-call-notes"}' | jq -r .token)
# the bundled synthetic call (from the decosa-api source): demo_scripts/audio/cn-family-custody-ortiz.wav
curl -sN "$API/callnotes/audio?attorney_state=OH&client_state=OH&all_parties_consented=true&method=stated_on_call&call_date=2026-09-15" \
  -H "authorization: Bearer $TOKEN" -H 'content-type: audio/wav' --data-binary @cn-family-custody-ortiz.wav > /tmp/cn.sse
grep '"type": "result"' /tmp/cn.sse | sed 's/^data: //' > /tmp/cn.json
jq '{claims: .claim_check, hours: .time_entry.hours, names: [.conflicts[].name], audio_deleted: .audio_deleted.bytes, record_ok: .record_check.ok}' /tmp/cn.json
grep '"lane": "record"' /tmp/cn.sse | sed 's/^data: //' | jq '.data' > /tmp/cn-record.json
curl -s $API/record/verify -H 'content-type: application/json' --data-binary @/tmp/cn-record.json | jq '{ok, summary}'
```

Pass if: the first call prints `all-party`; the result has a `claim_check` with counts, `hours` 0.1, a names list that includes `Gregory Paskett`, a non-zero `audio_deleted`, and `record_ok: true`; and the verify prints `ok: true`. Without the consent parameters, `/callnotes/audio` answers 400.

Then the live path: `POST /demo/replay {"vertical":"privileged-call-notes","script_id":"cn-family-custody-ortiz"}` streams the same call through Voxtral in real time; the stream ends with lanes `memo`, `conflicts`, `time_entry`, `tasks`, `email`, `privacy` and `record`.

## 7. Point the app at the local API

- Base URL: `http://localhost:8445` (web app: `NEXT_PUBLIC_DECOSA_API=http://localhost:8445`). Add other origins to `DECOSA_CORS_ORIGINS`.
- Live calls: `ws://localhost:8445/ws/live?vertical=privileged-call-notes&token=<t>&consent=<url-encoded JSON {attorney_state, client_state, all_parties_consented, method}>&call_date=YYYY-MM-DD&lang=en|es`, 16 kHz mono PCM16 frames of about 100 ms, then `{"type":"stop"}`. A missing or insufficient consent statement closes the socket with code 4400 and the prompt to show.
- Recordings: `POST /callnotes/audio` (WAV, m4a, mp3, ogg or webm; consent in the query string). Transcripts you already have: `POST /callnotes/notes` with `transcript` or `segments` and `consent`.
- `GET /callnotes/info` lists what it does and does not do; `GET /callnotes/consent-rules` the rule for a pair of states.
- The server stores nothing. Keep the memo and the signed record in your matter file; anyone can check the record with `POST /record/verify`.
- Keep the API on `127.0.0.1`; for the LAN, put a TLS reverse proxy with authentication in front and set `DECOSA_TRUSTED_PROXIES`.

## 8. Hosted gateway route (off, and leave it off)

`DECOSA_LLM_ROUTE=gateway` would send the prompts, which contain the client's words, to Decosa's hosted gateway. Never use it for client calls on a self-hosted box.

Finish with a summary: what is running, the health output, the signing key's pubkey, the smoke-test results, and the reminders above.