Skip to content
decosa

73 · Legal · Healthcare · live

Medical chronology with page cites

Open the toolJSON

Eval results

Scored on a held-out or test splitRun 27 Sep 2026Eval write-up (decosa-api, access required)

  • Planted events found189 / 198test splitn = 1986 held-out synthetic packets, 96 pages; prompts and thresholds frozen after the dev split
  • Date right, of events found189 / 189test splitn = 189
  • Cite on the right page and box, of events found189 / 189test splitn = 189the cited box overlaps the planted line by at least half its height
  • Lines that are not planted events2 / 224test splitn = 224both were therapy techniques in a PT plan listed as procedures
  • Planted date conflicts flagged6 / 6test splitn = 60 false conflicts
  • Planted gaps in treatment found6 / 6test splitn = 60 false gaps
  • Pre-existing conditions flagged (related or not said right)12 / 12 (12 / 12)test splitn = 120 false flags
  • Events on faxed copies cited on the copy too57 / 66test splitn = 66
  • Seconds per 100 pages537.7 stest splitn = 96end to end on the shared hosted gateway, one packet at a time
  • Planted events found, dev98 / 99dev (tuned on)n = 993 packets used while building; 1 of 114 lines not a planted event

Dataset

Nine synthetic record packets for fictional patients (3 dev, 6 held-out test), 16 pages each from five providers: typed PDFs with a text layer, scans with rotation, speckle and JPEG noise, handwritten forms and flow sheets, and faxed copies, with planted events, copies, one date conflict, one gap and two pre-existing conditions each (decosa_api/verticals/chronology/synth.py).

Caveats

  • Same author wrote the generator, the answer key and the prompts; one template family, tidier than real records.
  • Matching an entry to a planted event uses its type and key words; a few defensible extra lines (an operative note listed as a visit, orders as referrals) are counted as neither right nor wrong.
  • No real records and no nurse reviewer's chronology to compare with.
  • Timing is on a gateway shared with other workloads; a private GPU runs faster.

Nightly smoke check

Loading the nightly status…

Result
pass
Run
27 Sep 2026
Latency, this run
n/a
p50 over passed runs
25 s
Receipts
8
Model calls
n/a
Tokens
n/a
Cost per run
$0.004

Self-host verification

Verified on 27 Sep 2026: fresh clone into a clean directory, api image from docker/api/Dockerfile, compose api with a named volume, direct route to the local Qwen3.8-27B and the running document reader and retrieval services, local signing; torn down after

The rehearsal bundle passed 12/12 in 68 s (the 16-page packet uploaded as files: conflict, gap, pre-existing, copies, cites in the index, reader receipts, record verified and failed when changed; every receipt attested) and the smoke module passed in 17.9 s. The reader and retrieval services were not built from the compose file here: the running ones were used.

Rehearsal bundle: medical-chronology.zip (943 KB, 12 checks). Mock inputs plus the expected results, so you can prove your own setup works before any real data touches it.

Known limits

  • Hosted verification ran on the pre-release server (decosa-api the pre-release branch on our server, gateway route): the smoke test on one 4-page file, and the full 16-page sample from the console. The production API gets this tool when the branch merges.
  • Measured on synthetic packets written by the same author as the prompts, one template family; not on real records or against a nurse reviewer's chronology.
  • When the reader skips a line on a scan, the event on it is missed: 9 of 198 planted events were missed, mostly this way.
  • Hosted runs are capped at 40 pages; a page takes about 5 s on the shared gateway.
  • Billing records and charges are not extracted.

Models and licences

Standard tier. Licence posture: permissive (Apache, MIT or BSD).

  • The event schema and page checks, copies, merging, conflicts, gaps, pre-existing, cites to chunks and bytes, the Markdown chronology and the signed record (no model; CPU)decosa-api medical chronology (decosa_api/verticals/chronology), importing the document reader, evidence retrieval (scans into retrieval), typed judgment and dates blocksAGPL-3.0-or-later
  • One call per page (the page's elements, plus the page image for a scan) listing typed events with quote and date; typed judgments for duplicate and conflict pairs and for pre-existing conditions; re-reads of regions the parser was unsure ofQwen3.8-27B (NVFP4)Apache-2.0
  • Finds and orders the regions of every page (text, headings, tables, checkboxes) with their boxesDocling 2.130 with the Heron layout model (document reader block)MIT (Docling) + Apache-2.0 (weights)
  • Reads each region of a scanned page (text, handwriting, tables as cells)PaddleOCR-VL-1.6 (0.9B, document reader block)Apache-2.0
  • Scans into retrieval: each page's elements as chunks with page and box, the index and layout hashes, every cite located to a chunk and byte span, and a reranked search per procedure for other pages that give another dateEvidence retrieval block: Qwen3-Embedding-0.6B + Qwen3-Reranker-4B (decosa-retrieval service)Apache-2.0

All quality evidence

Every sourced number on the tool’s Stack tab, by tier. Some are proxies from another task; their labels say so.

Lite · document reader and the model, no retrieval service (1)
  • Planted events found, conflicts, gaps with BM25 only: not measured yetthe eval runner supports it (scripts/chronology_eval.py --bm25); not run on the test split
Standard · reader, retrieval service and the model (hosted demo) (5)
  • Planted events found (held-out, 6 packets, 96 pages): 189 / 198 (95.5%)decosa-api docs/evals/medical-chronology.md, test split, 27 Sep 2026
  • Date right / cite on the right page and box, of those found: dates 189 / 189, cites 189 / 189decosa-api docs/evals/medical-chronology.md, test split
  • Lines that are not planted events: 2 / 224 (0.9%)decosa-api docs/evals/medical-chronology.md, test split
  • Planted date conflicts / gaps / pre-existing flagged: 6 / 6, 6 / 6, 12 / 12; 0 false flags of each kinddecosa-api docs/evals/medical-chronology.md, test split
  • Events on faxed copies carrying the copy's cite: 57 / 66decosa-api docs/evals/medical-chronology.md, test split

How we measure · All tools