Digest a deposition

# Assemble the Decosa deposition and hearing digest on this machine

You are setting up a self-hosted deposition and hearing digest on this Linux machine: it reads transcripts with page:line numbering (text or PDF), writes a topic digest where every sentence cites page:line ranges, checks each sentence against the lines it cites, lists non-answers and exhibits, finds testimony that conflicts across witnesses, and exports to Word or Markdown. Work step by step, show me each command before running anything that needs sudo, and stop if a check fails.

**Before anything else, remind me:** transcripts under a protective order or with privileged content must stay on this machine; keep the model route local (`direct`) and never send them to a hosted service. The digest is a work aid, not a certified transcript, and not legal advice. ABA Formal Opinion 512 (29 Jul 2024) expects lawyers to understand how a tool uses client information and to bill only for time actually spent. Repeat these points in your final summary.

## Step 0: set up with a coding agent, rehearse on mock data, then go private

This prompt is for a coding agent running on the machine that will host the service. We recommend Claude Code with
Claude Opus 5.5; any capable coding agent works. Work in this order:

1. Set up on mock data only. During the whole setup you (the agent) work with the synthetic sample bundle below and
   nothing else. Do not ask me for real data, and do not open, read, list or copy files that hold real data, even to
   "test with something realistic".
2. Rehearse. When the steps below are done and the service is healthy, fetch the mock-data bundle for this tool,
   https://decosa.ai/samples/deposition.zip (4 KB, 9 checks, synthetic or openly licensed: see `licence` in expected.json),
   show me what is in it, and run the rehearsal against the local API:
   `docker compose exec api python scripts/rehearse.py deposition` (the api image carries the same bundle under /app/rehearsal/deposition/;
   with no key set, the script asks the local API for a short demo token). From a decosa-api checkout instead:
   `python scripts/rehearse.py deposition --bundle deposition.zip --base-url http://127.0.0.1:<PORT>`.
   It sends the mock inputs to the local API and prints PASS or FAIL for each expected property (for example: "the transcript parses into 3 pages of numbered page:line lines", "the parser recognises the numbered deposition layout", "the digest finishes without errors"). Show me
   the full output. Every check must pass. If one fails, fix the install and run it again; never edit `expected.json`
   to make a check pass.
3. Stop there. Once the rehearsal passes, tell me, and I will run my own data against the local API myself, on this
   machine.

For the person running this: a coding agent that runs in the cloud sees everything in its context, including files it
reads, command output and anything pasted into the chat. Keep real data out of the chat and out of anything the agent
can read. Switch to your own data only after the rehearsal has passed and the agent's work is done.

## What you are building

| service | image | model | port |
|---|---|---|---|
| `llm` | `${DECOSA_REGISTRY}/decosa-llm:0.1.0` (vLLM 0.29.0, `vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1`) | `nvidia/Qwen3.8-27B-NVFP4` @ `482ca0f3832238542f8f5295dde86b5f22711d80`, Apache-2.0 | internal 8000 |
| `api` | `${DECOSA_REGISTRY}/decosa-api:0.1.0` (no GPU; includes poppler `pdftotext` for PDF transcripts, pypdf as a fallback) | none | `127.0.0.1:8445` |
| `diarize` (optional, audio only) | built from the decosa-api source (`services/diarize`) | `OpenMOSS-Team/MOSS-Transcribe-Diarize` @ `704aa4a9c304e8520be88901e0d1960158ef5b15`, Apache-2.0 | internal 8092 |

No speech model is needed for transcripts. `diarize` only turns a recording into an uncertified rough transcript.

## 1. Check the GPU, driver and Docker

1. Run `nvidia-smi`. I need one NVIDIA GPU with at least 48 GB and driver 580 or newer.
   - Blackwell (RTX PRO 6000, B200): use the defaults below (NVFP4). This is the measured setup.
   - Hopper (H100/H200) or 48 GB Ada/L40S: set `LLM_MODEL=Qwen/Qwen3.8-27B-FP8`, `LLM_REVISION=main`; on 48 GB also `LLM_MAX_LEN=32768`. Not measured.
   - Under 48 GB: stop and tell me it will not fit.
2. Check `docker --version`, `docker compose version` and `docker run --rm --gpus all ubuntu nvidia-smi`. If Docker or the NVIDIA Container Toolkit is missing, install them from the official Docker and NVIDIA repositories (`sudo nvidia-ctk runtime configure --runtime=docker`, then restart Docker).
3. Confirm about 60 GB of free disk.

## 2. Get the images

The images are **on request** while self-host is in early access: ask at https://decosa.ai/contact?topic=self-host, and Decosa sends the registry (set it as `DECOSA_REGISTRY`), pull access and the compose file. Try `docker pull ${DECOSA_REGISTRY}/decosa-{llm,api}:0.1.0`. If a pull fails, build from source once the `decosa-api` source is published: clone it and run `docker compose build llm api`. If neither works, stop and tell me.

## 3. Write the compose file

Create `~/decosa-deposition/.env`:

```bash
DECOSA_TAG=0.1.0
DECOSA_GPU=0
LLM_MODEL=nvidia/Qwen3.8-27B-NVFP4
LLM_REVISION=482ca0f3832238542f8f5295dde86b5f22711d80
LLM_MAX_LEN=65536
LLM_GPU_UTIL=0.85
DECOSA_SIGNER_NAME="<who signs these digests, e.g. Firm litigation support>"
```

Create `~/decosa-deposition/docker-compose.yml` with exactly these services:

```yaml
name: decosa-deposition
x-health: &health
  interval: 15s
  timeout: 5s
  retries: 5
services:
  llm:
    image: ${DECOSA_REGISTRY}/decosa-llm:${DECOSA_TAG}
    deploy: { resources: { reservations: { devices: [ { driver: nvidia, device_ids: ["${DECOSA_GPU:-0}"], capabilities: [gpu] } ] } } }
    ipc: host
    restart: unless-stopped
    volumes: [hf-cache:/root/.cache/huggingface]
    command: ["${LLM_MODEL}", "--revision", "${LLM_REVISION}", "--served-model-name", "qwen3.8-27b",
              "--language-model-only", "--max-model-len", "${LLM_MAX_LEN}", "--gpu-memory-utilization", "${LLM_GPU_UTIL}",
              "--max-num-seqs", "16", "--kv-cache-dtype", "fp8_e4m3", "--speculative-config", '{"method":"mtp","num_speculative_tokens":3}',
              "--seed", "0", "--enable-force-include-usage", "--disable-uvicorn-access-log", "--host", "0.0.0.0", "--port", "8000"]
    healthcheck: { <<: *health, test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4)"], start_period: 900s }
  api:
    image: ${DECOSA_REGISTRY}/decosa-api:${DECOSA_TAG}
    restart: unless-stopped
    depends_on: { llm: { condition: service_healthy } }
    environment:
      DECOSA_LLM_ROUTE: direct                  # local model only; receipts are signed by this box's key ("attested")
      DECOSA_LLM_URL: http://llm:8000/v1
      DECOSA_LLM_MODEL: qwen3.8-27b
      DECOSA_LOCAL_SIGNING: "on"                # Ed25519 key created at /data/attest/ed25519.pem on first start
      DECOSA_SIGNER_NAME: ${DECOSA_SIGNER_NAME}
      DECOSA_SESSIONS_PER_IP_HOUR: "1000"
      DECOSA_BUDGET_LLM_TOKENS: "400000"         # per session; a 300-page deposition needs roughly 100k-350k (estimate)
      DECOSA_SESSION_TTL_S: "28800"
      DECOSA_DEPOSITION_MAX_LINES: "8000"        # about 320 deposition pages across all transcripts in one digest
      DECOSA_DEPOSITION_AUDIO: "0"               # "1" only with the diarize service
      DECOSA_DIARIZE_URL: ""                    # step 4 sets this for recordings
      DECOSA_CORS_ORIGIN_REGEX: '^https?://(localhost|127\.0\.0\.1)(:\d+)?$$'
    ports: ["127.0.0.1:8445:8445"]
    volumes: [decosa-data:/data]
    healthcheck: { <<: *health, test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8445/healthz', timeout=4)"], start_period: 20s }
volumes: { hf-cache: {}, decosa-data: {} }
```

Run `docker compose up -d` and poll `docker compose ps` until both are healthy (the LLM takes 5-10 minutes the first time). `curl -s localhost:8445/healthz` should show `"llm": true` (`asr` is false here and that is fine).

## 4. Optional: recordings (diarize)

Only if I want rough transcripts from recordings: run `services/diarize` from the decosa-api source on this box (install with the `uv` commands at the top of `services/diarize/requirements.txt`, then `DIARIZE_HOST=172.17.0.1 DIARIZE_PORT=8092 DIARIZE_DEVICE=cuda:0 .venv-diarize/bin/python services/diarize/server.py`; `172.17.0.1` is the docker bridge address from `ip -4 addr show docker0`, reachable from containers and not from the LAN). Add `extra_hosts: ["host.docker.internal:host-gateway"]` to `api` and set `DECOSA_DIARIZE_URL: http://host.docker.internal:8092`. Also set `DECOSA_DEPOSITION_AUDIO: "1"` and lower `LLM_GPU_UTIL` to 0.80. `POST /deposition/rough` then takes a 16 kHz mono 16-bit WAV and returns text headed "UNCERTIFIED ROUGH TRANSCRIPT" with `S01:`/`S02:` speaker labels. The fit beside the LLM is an estimate, not measured.

## 5. Smoke test

```bash
TOKEN=$(curl -s localhost:8445/demo/session -H 'content-type: application/json' -d '{"vertical":"deposition"}' | jq -r .token)
curl -sN localhost:8445/deposition/digest -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{"transcripts":[{"sample_id":"fictional-pruitt"},{"sample_id":"fictional-okafor"}]}' > /tmp/dep.sse
grep -oE '"(type|lane|status)": "[a-z_]+"' /tmp/dep.sse | sort | uniq -c
ID=$(grep '"type": "done"' /tmp/dep.sse | sed 's/^data: //' | jq -r .digest_id)
curl -s "localhost:8445/deposition/digests/$ID/export?format=docx" -H "authorization: Bearer $TOKEN" -o /tmp/digest.docx
grep '"lane": "record"' /tmp/dep.sse | sed 's/^data: //' | jq .data > /tmp/record.json
curl -s localhost:8445/record/verify -H 'content-type: application/json' --data-binary @/tmp/record.json | jq '{ok, summary}'
```

Pass if: lanes `transcript`, `flags`, `digest`, `verifier`, `contradictions` and `record` all appear; every `receipt` has `"status": "attested"`; the contradictions lane marks the time of the fall (2:15 pm against 11:30 am) `INCONSISTENT`; the verify prints `ok: true`; and `/tmp/digest.docx` opens in Word. On the measured card the run takes about 75 s.

Then test a PDF: `curl -s localhost:8445/deposition/parse -H "authorization: Bearer $TOKEN" -H 'content-type: application/pdf' --data-binary @some-transcript.pdf | jq '{layout, pages, lines, range, warnings}'`. `layout: numbered` means the reporter's own page:line numbers were found; `plain` means the file had none and the cites are Decosa's own pagination.

## 6. Point the app at the local API

- Base URL `http://localhost:8445` (web app: `NEXT_PUBLIC_DECOSA_API=http://localhost:8445`). Add other origins to `DECOSA_CORS_ORIGINS`.
- `POST /deposition/digest` streams Server-Sent Events; each `lane` event replaces that lane. Exports: `GET /deposition/digests/{id}/export?format=md|docx` with the same token, for one hour. Unsupported sentences are held back from exports and listed at the end.
- Keep the API on 127.0.0.1. For other users on the LAN, put a TLS reverse proxy with authentication in front and set `DECOSA_TRUSTED_PROXIES`.

## 7. Keep the direct route

`DECOSA_LLM_ROUTE=gateway` would send prompts, which contain the transcript, to the hosted Decosa
API. Do not use it for client matters. At most, public-record transcripts.

Finish with a summary: what is running, the health output, the smoke-test results, and the privilege, not-certified and Opinion 512 reminders.