Check an AI note

# Assemble the Decosa clinical AI assurance monitor on this machine

You are setting up a monitor that measures an ambient AI scribe's notes against the visit transcripts, inside this
hospital's network. For each visit it takes the transcript (or the audio) and the note the scribe product wrote, checks
every note sentence against the transcript, lists the medications, allergy status, plan items and impression the note
left out, and signs a per-visit report of hashes and counts. Signed reports roll up into rates per vendor, specialty and
week. Work step by step, show me each command before you run anything with `sudo`, and stop to ask if a check fails.

## Step 0: set up with a coding agent, rehearse on mock data, then go private

This prompt is for a coding agent running on the machine that will host the service. We recommend Claude Code with
Claude Opus 5.5; any capable coding agent works. Work in this order:

1. Set up on mock data only. During the whole setup you (the agent) work with the synthetic sample bundle below and
   nothing else. Do not ask me for real data, and do not open, read, list or copy files that hold real data, even to
   "test with something realistic".
2. Rehearse. When the steps below are done and the service is healthy, fetch the mock-data bundle for this tool,
   https://decosa.ai/samples/clinical-ai-monitor.zip (8 KB, 10 checks, synthetic or openly licensed: see `licence` in expected.json),
   show me what is in it, and run the rehearsal against the local API:
   `docker compose exec api python scripts/rehearse.py clinical-ai-monitor` (the api image carries the same bundle under /app/rehearsal/clinical-ai-monitor/;
   with no key set, the script asks the local API for a short demo token). From a decosa-api checkout instead:
   `python scripts/rehearse.py clinical-ai-monitor --bundle clinical-ai-monitor.zip --base-url http://127.0.0.1:<PORT>`.
   It sends the mock inputs to the local API and prints PASS or FAIL for each expected property (for example: "all 4 clinical sentences were checked", "every checked sentence got a verdict (none not judged or errored)", "the planted exam finding (oxygen saturation 97%) is not supported"). Show me
   the full output. Every check must pass. If one fails, fix the install and run it again; never edit `expected.json`
   to make a check pass.
3. Stop there. Once the rehearsal passes, tell me, and I will run my own data against the local API myself, on this
   machine.

For the person running this: a coding agent that runs in the cloud sees everything in its context, including files it
reads, command output and anything pasted into the chat. Keep real data out of the chat and out of anything the agent
can read. Switch to your own data only after the rehearsal has passed and the agent's work is done.

## 0. Ground rules and licences
- Models: Qwen3.8-27B (Apache-2.0) as the judge; MOSS-Transcribe-Diarize 0.9B (Apache-2.0) only if we send audio.
  decosa-api is AGPL-3.0-or-later.
- Transcripts and notes are PHI. Bind every port to 127.0.0.1 (or the internal interface I name), never expose it to the
  internet, and keep `DECOSA_LLM_ROUTE=direct` so no model call leaves this machine. The service keeps nothing: text
  and audio stay in memory for one request, and logs carry counts only. Keep it that way; do not add request logging.
- It is a documentation-quality measure, not software that recommends care: do not wire its flags into anything that
  changes a patient's care in real time. Check with me whether our scribe contract allows us to benchmark the product.

## 1. Check the machine
1. `nvidia-smi`: one GPU with at least 32 GB for the judge (Qwen3.8-27B NVFP4 needs about 20 GB of weights plus KV
   cache; a 25-minute visit is about 10k tokens of transcript, sent first in every prompt). Driver 570 or newer.
   Blackwell cards run NVFP4; older cards use `Qwen/Qwen3.8-27B-FP8`.
2. `docker --version` and `docker compose version`. If Docker or the NVIDIA container toolkit is missing, install them
   from the official Docker and NVIDIA repositories after asking me, then run
   `docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi`.
3. Disk: about 30 GB free for the model and images.
4. Audio (optional): the diarizer needs its own GPU memory (about 4 GB) and a Python 3.12 venv with PyTorch.

## 2. Images and weights
- `${DECOSA_REGISTRY}/decosa-api:<tag>` (**publishing soon**). If the pull fails, build from source:
  `git clone <decosa-api source: on request at https://decosa.ai/contact?topic=self-host>` (access required), check out a release that contains
  `decosa_api/verticals/monitor/`, and `docker build -f docker/api/Dockerfile -t decosa-api:local .`
- `vllm/vllm-openai:v0.29.0` for the judge; weights `nvidia/Qwen3.8-27B-NVFP4` (revision
  `482ca0f3832238542f8f5295dde86b5f22711d80`), or `Qwen/Qwen3.8-27B-FP8` on pre-Blackwell cards.
- Audio only: from the same clone, `uv venv .venv-diarize --python 3.12` and
  `uv pip install -p .venv-diarize -r services/diarize/requirements.txt --index-strategy unsafe-best-match --extra-index-url https://download.pytorch.org/whl/cu130`,
  then run `DIARIZE_HOST=127.0.0.1 DIARIZE_PORT=8092 .venv-diarize/bin/python services/diarize/server.py` (it downloads
  `OpenMOSS-Team/MOSS-Transcribe-Diarize`, 1.8 GB). `curl -s 127.0.0.1:8092/health` must say `"ok": true`.

## 3. docker-compose.yml
Write this in `~/decosa/monitor/`:

```yaml
services:
  llm:
    image: vllm/vllm-openai:v0.29.0
    command: ["--model", "nvidia/Qwen3.8-27B-NVFP4", "--revision", "482ca0f3832238542f8f5295dde86b5f22711d80",
              "--served-model-name", "qwen3.8-27b", "--max-model-len", "32768", "--enable-prefix-caching",
              "--kv-cache-dtype", "fp8_e4m3", "--gpu-memory-utilization", "0.85"]
    ports: ["127.0.0.1:8114:8000"]
    ipc: host
    volumes: ["~/.cache/huggingface:/root/.cache/huggingface"]
    deploy: { resources: { reservations: { devices: [{ driver: nvidia, count: 1, capabilities: [gpu] }] } } }
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4)"]
      interval: 30s
      retries: 40
      start_period: 900s
  api:
    image: ${DECOSA_REGISTRY}/decosa-api:<tag>        # or decosa-api:local from step 2
    ports: ["127.0.0.1:8445:8445"]
    extra_hosts: ["host.docker.internal:host-gateway"]
    environment:
      DECOSA_LLM_ROUTE: direct
      DECOSA_LLM_URL: http://llm:8000/v1
      DECOSA_LLM_MODEL: qwen3.8-27b
      DECOSA_DIARIZE_URL: http://host.docker.internal:8092   # empty string if we only send transcripts
      DECOSA_ASR_WS: ws://127.0.0.1:1/unused                 # the live scribe is not part of this stack
      DECOSA_BUDGET_LLM_TOKENS: "400000"                      # per session; a note needs about 170 tokens a sentence
      DECOSA_BUDGET_AUDIO_S: "7200"
      DECOSA_SESSION_TTL_S: "28800"
      DECOSA_LOCAL_SIGNING: "on"
      DECOSA_SIGNER_NAME: "<hospital> quality monitor"
      DECOSA_MONITOR_MAX_CONCURRENT: "3"
    volumes: ["decosa-data:/data"]
    depends_on: { llm: { condition: service_healthy } }
volumes:
  decosa-data:
```

If the diarizer runs on the host, bind it to the Docker bridge address instead of 127.0.0.1 so the container can
reach it, or leave `DECOSA_DIARIZE_URL` empty and send transcripts only. On first start the api service creates this
box's Ed25519 key in the `decosa-data` volume. Back it up and never print it: reports and summaries are signed with it
(an attestation by us, the operator, not a proof), and the committee pins its public key.

## 4. Smoke test
1. `docker compose up -d`, then wait for `llm` to be healthy (the first start downloads about 20 GB).
2. `curl -s localhost:8445/monitor/info | jq '{findings: (.findings|keys), judge, diarizer}'`. (`/healthz` reports
   `ok: false` on this stack because it also checks the live-scribe recogniser, which we do not run; `llm: true` is what
   matters here.)
3. Token: `T=$(curl -s -XPOST localhost:8445/demo/session -H 'content-type: application/json' -d '{"vertical":"clinical-ai-monitor"}' | jq -r .token)`.
4. Run a bundled sample with a planted wrong-speaker error:
   `curl -s localhost:8445/monitor/samples | jq '{segments: .encounters[1].segments, note: (.encounters[1].notes[] | select(.id=="speaker") | .note), vendor: "Scribe A", specialty: .encounters[1].specialty}' > req.json`
   `curl -s -XPOST localhost:8445/monitor/check -H "authorization: Bearer $T" -H 'content-type: application/json' -d @req.json > run.json`
   Expect `decision: "errors"` and `.sentences[] | select(.finding)` to include the planted sentence (compare
   `.encounters[1].notes[]|select(.id=="speaker")|.planted` from the samples). Run the `clean` note too: expect `clean`
   or at most one flag.
5. Verify: `jq -s '{report: .[1].report, note: .[0].note, segments: .[0].segments}' req.json run.json | curl -s -XPOST localhost:8445/monitor/verify -H 'content-type: application/json' -d @-`
   must show `valid_signature`, `transcript_matches` and `note_matches` true. Change one count in the report and verify again: it must fail.
6. Summary: `jq '{reports: [.report]}' run.json | curl -s -XPOST localhost:8445/monitor/summary -H 'content-type: application/json' -d @-`
   returns rates and a signed `summary`.
7. Audio (if the diarizer runs): `curl -s -XPOST localhost:8445/monitor/transcribe -H "authorization: Bearer $T" --data-binary @visit.wav`
   returns `lines` with `Clinician` and `Patient` turns; pass its `segments` to `/monitor/check` with `"source": "audio"`.
8. Time it and tell me: on our RTX PRO 6000, shared with other work, a 27-sentence note against a 14-minute visit took
   about 15-20 s straight to the model server, almost all of it judge calls.

## 4b. The detail checker (optional, CPU)
Our M17 model settles the judge's "a detail not in the visit" flags: when the note's drug, dose, frequency, route, date,
duration, side or number conflicts with the transcript line, the flag becomes a changed-detail error (changed details
caught: 94% with it, 72% without, on our held-out mock visits). It needs no GPU: about 2.5 GB of RAM and 8 CPU threads.
The weights are not published yet (their model card is under review); until they are, ask us for them, or skip this
step: without `DECOSA_DETAIL_URL` the tool runs the judge alone and says so in `/monitor/info` (`detail_checker.configured: false`).
1. `uv venv ./detail-venv --python 3.12` and `uv pip install -p ./detail-venv -r services/detail_checker/requirements.txt
   --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple --index-strategy unsafe-best-match`.
2. Put the weights in `./detail-weights/` and start it from the decosa-api checkout:
   `CUDA_VISIBLE_DEVICES= DETAIL_MODEL_DIR=./detail-weights DETAIL_PORT=8495 ./detail-venv/bin/python services/detail_checker/server.py`.
3. Give the api service `DECOSA_DETAIL_URL=http://host.docker.internal:8495` (or the host's address) and restart it;
   `curl -s localhost:8445/monitor/info | jq .detail_checker` shows `reachable: true` and the weights sha256.

## 5. Wire it in
Keep reports, not text: for each sampled visit, call `POST /monitor/check` with the vendor, version, specialty and the
ISO week, append `report` to a JSON-lines file, and once a week or month send those reports to `POST /monitor/summary`
as `{"reports": [...], "pin": true}` (`jq -s '{reports: ., pin: true}' reports.jsonl`). The signed summary lists every report it counts; that is the committee's evidence. Use
`encounter_ref` for your own opaque visit id (the report keeps only its hash), never a medical record number. Point the
Decosa site at it with `NEXT_PUBLIC_DECOSA_API=http://127.0.0.1:8445` if you want the console. Contract:
`API_CONTRACT.md`, section "Clinical AI assurance monitor".