Medicare sales-call record

Eval results

Scored on a held-out or test splitRun 26 Sep 2026Eval write-up (decosa-api, access required)

• Per-rule accuracy, test B run 1 (never used to change anything)40/40held outn = 40Run 2: 40/40
• Per-rule accuracy, test run 4101/101test splitn = 101Run 1 (before the one code change): 101/101
• Planted typed answers found, test run 49/9test splitn = 9Test B: 2/2; audio: 9/9 (also on the re-voiced audio)
• Call-sheet items, test run 4 / test B run 179/79 / 31/32test splitn = 111Test run 1, before the needs change: 77/79
• Benefit claims graded right (good ok, bad flagged), test run 425/25 / 4/4test splitn = 29Test B: 8/8 / 2/2; audio (re-voiced 26 Sep 2026 with Decosa house voices) runs 1 and 2: 18/19 / 2/3, both misses on t9 where the diarizer split a line and the scorer paired claims with the wrong gold lines (first build: 19/19 / 3/3 and 18/19 / 3/3)
• False alarms on clean calls, test run 4 / test B run 13 of 87 / 0 of 42test splitn = 129All 3 are claims the plan facts do not mention
• Cited time inside the spoken gold line, audio run 128/29test splitn = 29Audio re-voiced 26 Sep 2026 with Decosa house voices (Kokoro-82M) and re-run: 21/29 within 3 s of the line's start; run 2: 29/29. First build (macOS voices): 29/29, 16/29 within 3 s

Nightly smoke check

Loading the nightly status…

Result

pass

Run

26 Sep 2026

Latency, this run

n/a

p50 over passed runs

7.5 s

Receipts

15

Model calls

n/a

Tokens

n/a

Cost per run

$0.006

Self-host verification

Verified on 26 Sep 2026: fresh clone of decosa-api into a clean directory on our server, api image built from docker/api/Dockerfile, compose api service with a named data volume, direct route, local signing; torn down after

The assembly prompt's smoke tests ran against the already-running local Qwen3.8-27B vLLM (127.0.0.1:8114, network_mode host instead of the compose llm service): tb1-showcase flagged the disclaimer timing, "free", the final-expense pitch and the $3,000 dental claim (contradicted), 15 attested receipts, record verified, retention 2029-11-05 / 2032-11-05, 3.8 s; the pasted cold call flagged Medicare, free, unsolicited, no SOA and the dental claim; no consent gave 400.

Models and licences

Standard tier. Licence posture: permissive (Apache, MIT or BSD).

• Two extractions (products, first benefit discussion, enrollment steps; the agent's benefit claims), one typed yes/no/unclear check per question, and one grounding judgment per benefit claim against the plan factsQwen3.8-27B (NVIDIA NVFP4)Apache-2.0
• Recording to a timed, speaker-labelled transcript (POST /medicare/transcribe, self-host; the demo's audio samples were transcribed with it)MOSS-Transcribe-Diarize 0.9BApache-2.0

All quality evidence

Every sourced number on the tool’s Stack tab, by tier. Some are proxies from another task; their labels say so.