155 · Finance and insurance · Legal · preview
Injury demand reader
Eval results
Scored on a held-out or test splitRun 29 Sep 2026Eval write-up (decosa-api, access required)
- Respond-by date exact7 of 7test splitn = 7every time-limited letter in the test set; the one ordinary demand was read as not time-limited (8 of 8)
- Statute's minimum exact7 of 7test splitn = 7
- Conditions of acceptance found31 of 31test splitn = 31precision 33 of 41
- Billed totals within $18 of 8test splitn = 80 invented bill lines
- Value or payment language in the tool's own words0test splitn = 8
Dataset
24 synthetic injury demand packages (letters, bills and records; 406 pages; CA, GA, MO, MT, UT, FL, TX, OH) written blind by a separate author with planted deadlines, conditions, missing elements, specials gaps and citation problems; dev 16, test 8.
Caveats
- Synthetic packages from one author; real ones are longer, scanned and messier.
- Small test set (8 packages): wide confidence intervals.
- Prompts and rules were tuned on the 16 dev packages.
- Six states encoded; the rest are not.
Nightly smoke check
Loading the nightly status…
- Result
- pass
- Run
- 29 Sep 2026
- Latency, this run
- n/a
- p50 over passed runs
- 161 s
- Receipts
- 23
- Model calls
- n/a
- Tokens
- n/a
- Cost per run
- $0.020
Self-host verification
Not yet verified on a fresh self-host setup.
Rehearsal bundle: demand-reader.zip (4 KB, 8 checks). Mock inputs plus the expected results, so you can prove your own setup works before any real data touches it.
Known limits
- Hosted verification ran on our pre-release server through the production gateway, before these routes reached the production API.
- Measured on 24 synthetic packages written by one author; real packages are longer, scanned and messier.
- Typed or pasted page text only in this version; scans go through the document reader first.
- Six states encoded; the rest return 'not encoded: check with counsel'.
Models and licences
Standard tier. Licence posture: permissive (Apache, MIT or BSD).
- Reads the letter's terms and every condition with quotes, each bill page's charge lines, and each record page's visits (the medical chronology's page extractor); the grounding judge checks every sentence it writesQwen3.8-27B (NVIDIA NVFP4)Apache-2.0
All quality evidence
Every sourced number on the tool’s Stack tab, by tier. Some are proxies from another task; their labels say so.
Lite · one 48 GB card (1)
- accuracy on this task: not measured yet
Standard · the hosted demo, one 96 GB card (2)
- respond-by date exact, held-out test (8 packages, gateway, run once): 7 of 7decosa-api docs/evals/demand-reader.md, measured on our server 2026-09-29, gateway route
- conditions of acceptance found / billed totals within $1 (test): 31 of 31 / 8 of 8decosa-api docs/evals/demand-reader.md, measured on our server 2026-09-29, gateway route
Best · DeepSeek-V4-Flash on two more cards (1)
- accuracy on this task: not measured yet
Wanted · two large judges from different families (1)
- accuracy on this task: not measured yet