{"schema_version":"1","site":"https://decosa.ai","id":"foia-desk","num":"34","name":"Public-records desk","tool_name":"Redact a public-records release","short":"Records desk","blurb":"For a records officer answering a FOIA or state public-records request: turns the request into a search scope, finds the responsive records, and proposes each redaction with its exemption and a reason that names the harm without repeating the withheld words. The release PDF is built without the withheld text and read back to prove nothing under a box can be copied out. It drafts the index and the response letter, and seals every call and every officer decision in a signed record.","status":"live","labels":{"industry":["public-sector","legal"],"job":["review","attest"],"input":["text"],"deploy":["selfhost"],"status":"live","output":["record","data"],"data":["pii"],"hardware":"gpu-96","licence":"permissive"},"industries":["public-sector","legal"],"runs_in":["selfhost"],"part_of":[],"built_from":["typed-judgment","grounding","signed-record"],"models":"Qwen3.8-27B","where":"Self-host for real requests; hosted for the demo sets only","hardware":"1× RTX PRO 6000 (96 GB) or 1× RTX 5090 (32 GB) for the model; pattern finders, the release PDF and its check run on CPU","final_artifact":"A release PDF with a passed integrity check, an index of withheld information (CSV), a response letter, and a signed record and decision ledger.","self_host_first":true,"verification":{"receipt_coverage":"full","summary":"Receipt per call; release check; signed record and decision ledger","manual_qa":{"hosted":{"date":"2026-09-25","result":"pass","p50_ms":40000,"p95_ms":null,"runs":null,"receipts_per_run":41,"cost_per_run_usd":0.012},"selfhost":{"date":"2026-09-25","result":"pass","method":"Fresh clone of the branch into a clean directory, docker build of docker/api/Dockerfile, the assemble prompt's api service with a named volume, pointed at the already-running local vLLM (Qwen3.8-27B NVFP4 on 127.0.0.1:8114) through host networking; then torn down.","notes":"Verified on 2026-09-25: the image builds, the service starts healthy, info reports logprobs and both exemption lists, scoping works, the coastal-permits sample passes end to end (17 s, 41 calls, every disposition as expected, release check ok on 28 spans), the signed record verifies and fails at the changed entry, the PDF export carries no withheld text, one officer decision rebuilds the release and the ledger verifies. The model server's own startup was not re-verified (no new GPU load)."},"known_limits":["Labels are one AI reviewer's (Claude's), not a records officer's; the held-out sets are small and synthetic.","Medical details are the weakest kind: both held-out misses were medical.","It over-redacts company officials' names that the model reads as private, and agency staff named only in a record's text (not in its mail headers); they go to review or to the officer.","Runs are not identical: the same records can come back with a few redactions more or fewer from one run to the next (temperature 0 on a shared server is not bit-for-bit repeatable). The pattern finders, the date filter and the last contact check are code and do not vary; the officer decides every proposal.","The release is re-typeset from text: no original layout, no PDF, scan or email-archive intake, no audio or video.","The hosted route slows when the shared gateway is loaded."],"nightly_covers":null},"nightly":"https://api.decosa.ai/verify/status"},"eval_summary":{"metrics":[{"name":"Personal-data spans redacted (PII recall), final pipeline","value":"59 of 60 (98.3%)","unit":null,"n":60,"split":"test","note":"Test run 5; the test sets were run five times with fixes in between."},{"name":"Planted spans redacted, final pipeline","value":"70 of 71 (98.6%)","unit":null,"n":71,"split":"test","note":"Pattern finders alone: 33 of 71 (46.5%)."},{"name":"Redaction precision / keep violations","value":"83.1% (69 of 83) / 3","unit":null,"n":83,"split":"test","note":"Most over-redactions are company officials' names and business emails."},{"name":"Responsiveness precision / recall","value":"96.2% / 100.0%","unit":null,"n":77,"split":"test","note":null},{"name":"Withheld spans recoverable from the release PDF","value":"0 of 95","unit":null,"n":95,"split":"test","note":"All readers; an earlier run (test run 2) failed this check."},{"name":"PII recall on a set new to the pipeline (bus-contract, run 3)","value":"5/8","unit":null,"n":8,"split":"heldout","note":"Responsiveness 7/8; the cleanest held-out number, before the medical-span fix."},{"name":"Cost at list price","value":"$0.77 per 1,000 records","unit":null,"n":77,"split":"test","note":null}],"dataset":"Synthetic agency records (3 dev sets, 4 held-out sets, 48 held-out records) labelled by Claude, plus 29 public FERC-released Enron emails labelled by hand for responsiveness.","held_out":true,"caveats":["The test sets were run four times and three design faults were fixed after seeing test results; the final numbers are not clean held-out numbers.","Labels were written by one AI reviewer (Claude), not a records officer.","Small sample: 48 synthetic held-out records and 29 public emails.","Synthetic records are cleaner than real ones: no OCR noise, attachments or long threads.","Medical details are the weakest kind (5/7); an officer still has to read each released record.","Only the hosted route was measured; the self-host route was smoke-tested."],"date":"2026-09-25","doc_url":"https://decosa.ai/metrics/evals/foia-desk"},"stack":{"summary":"Paste the request and send the agency's records. An open model turns the request into a search scope the officer edits; code drops records dated outside it; a typed judgment calls each remaining record responsive or not, with a calibrated probability. For responsive records, pattern finders catch identifiers (SSNs, dates of birth, licence, account and card numbers; phones, emails and home addresses unless they are official contacts), and the model proposes the rest: names of private individuals, medical details, staff recommendations written before a decision, advice from agency counsel. Code maps each proposal's category to the exemption of the chosen list (federal FOIA or California CPRA), so the same kind of text always gets the same label, and checks that no reason repeats the withheld words. Records with agency counsel on them go through the privilege engine. A detail redacted anywhere is redacted everywhere. The release PDF is typeset without the withheld text, with a box and the exemption code where each deletion was, and read back by three readers to prove nothing can be recovered. It drafts the index of withheld information and the response letter, and every call and officer decision is sealed in a signed record. It proposes; the records officer decides.","tagline":"Scope a FOIA or public-records request, find the responsive records, and propose each redaction with its exemption and a reason; release a PDF with the withheld text really removed, checked, indexed and signed.","deployment":"self-host-first","regulatory_note":"A drafting aid for records officers, not legal advice and not a determination under any public-records law. Federal: 5 U.S.C. § 552(b)(1)-(9) lists the exemptions; an agency may withhold only where it reasonably foresees harm to an interest an exemption protects or where law prohibits disclosure (§ 552(a)(8)(A), FOIA Improvement Act of 2016); it must release reasonably segregable portions and mark the exemption at the place of each deletion where technically feasible (§ 552(b)); the deliberative process privilege does not apply to records created 25 years or more before the request (§ 552(b)(5)); an adverse determination must give at least 90 days to appeal and point to the FOIA Public Liaison and OGIS (§ 552(a)(6)(A)). California: the Public Records Act, Gov. Code § 7920.000 et seq. (recodified by AB 473, operative 1 Jan 2023), with § 7927.700 (personnel, medical or similar files), § 7927.705 (records exempt under other law, including Evidence Code privileges), § 7922.000 (the public-interest balancing test, under which California courts recognise the deliberative process privilege), § 7922.525(b) (segregable portions) and § 7922.540 (a written denial naming each person responsible). A custom list for another state is the agency's to check. Vaughn indexes (Vaughn v. Rosen, 484 F.2d 820 (D.C. Cir. 1973)) are filed in litigation; the index here is a draft for that use. Real requests involve personal data: run them on the agency's own machine; the hosted demo is for the fictional and public sample sets only. Model licence: Apache-2.0 (Qwen3.8-27B). The Enron emails are public records released by FERC (2003), used as a small research sample. Statutes checked against the official texts on 25 Sep 2026.","components":[{"id":"desk","role":"Records desk: date filter, pattern finders, exemption lists and reasons, reason leak check, consistency, release PDF writer and its integrity check, index, letter, signed record and ledger (no model; CPU)","name":"decosa-api foia module (decosa_api/verticals/foia)","hf_repo":null,"license":"AGPL-3.0-or-later","params":null,"quant":null,"vram_gb":0,"memory_gb_estimate":null,"engine":"Python 3.12; the typed-judgment engine (vertical 24), the privilege engine (vertical 30) and the filing-preflight finders and black-box check (vertical 21); pdfplumber (MIT) as the independent PDF reader","receipt_coverage":"partial","in_hosted_demo":null,"tiers":["lite","standard"],"alternative_to":null},{"id":"judge","role":"Model: the request scope, the responsiveness and deliberative calls, the redaction spans with their category, description and harm, and the privilege engine's calls","name":"Qwen3.8-27B (NVFP4)","hf_repo":"nvidia/Qwen3.8-27B-NVFP4","license":"Apache-2.0","params":"27.8B","quant":"NVFP4 (MLP NVFP4, GDN/attention FP8) + FP8 KV cache; MTP head, 3 draft tokens","vram_gb":20,"memory_gb_estimate":null,"engine":"vLLM 0.29.0, temperature 0, thinking off, prefix caching, top-20 logprobs on the direct route","receipt_coverage":"strong","in_hosted_demo":true,"tiers":["lite","standard"],"alternative_to":null}],"tiers":[{"id":"lite","label":"Lite · one 32 GB card, self-hosted","summary":"The same model and prompts on a single RTX 5090, one call per question with log-probabilities. The privilege engine's reason check can be switched off (DECOSA_FOIA_GROUNDING=0) to save a call per record with counsel on it.","components":["desk","judge"],"hardware":"1x RTX 5090 32 GB","quality_evidence":[{"metric":"Held-out test sets","value":"not measured separately: the same weights and prompts as the standard tier; speed on a 5090 not measured","source":"not measured yet"}],"latency_note":"estimate: not timed on a 5090.","in_hosted_demo":false,"receipt_coverage":"partial","receipt_note":"Self-hosted: calls and records are signed by your own box, not countersigned by the gateway.","hosting":null},{"id":"standard","label":"Standard · one 96 GB card (measured; hosted demo)","summary":"Qwen3.8-27B on an RTX PRO 6000, through our gateway with one call per question on the hosted demo (log-probabilities when self-hosted).","components":["desk","judge"],"hardware":"1x RTX PRO 6000 96 GB","quality_evidence":[{"metric":"Planted personal details redacted, 48 held-out synthetic records (60 labelled: names, phones, addresses, emails, SSN, dates of birth, licence and account numbers, medical details)","value":"59 of 60 (98.3%); the pattern finders alone would catch 46.5% of all planted spans","source":"docs/evals/foia-desk.md, test run 5 (30 Sep 2026, current pipeline); labels written by Claude (an AI agent), not a records officer"},{"metric":"What it missed","value":"The misses are listed per record in the eval write-up; medical details are the weakest kind.","source":"docs/evals/foia-desk.md"},{"metric":"Responsiveness, 77 held-out records (48 synthetic, 29 public FERC-released Enron emails): precision / recall","value":"96.2% / 100.0%","source":"docs/evals/foia-desk.md, test run 5 (30 Sep 2026, current pipeline)"},{"metric":"Exemption label on redacted spans / records with agency counsel withheld in full with the right exemption","value":"70 of 70 / 6 of 6 (federal (b)(5), (b)(6), (b)(4); CPRA § 7927.700, § 7927.705, § 7922.000)","source":"docs/evals/foia-desk.md, test run 5 (30 Sep 2026, current pipeline)"},{"metric":"Redaction precision (redactions that overlap a labelled span) / text that had to be released but was boxed","value":"83.1% / 3 spans: mostly company officials' names and emails, boxed everywhere by the privacy-protective consistency step and sent to review","source":"docs/evals/foia-desk.md, test run 5 (30 Sep 2026, current pipeline)"},{"metric":"Release check: withheld spans recoverable from the PDF (built-in reader, pdfplumber, filing-preflight black-box check)","value":"0 of 95 on the held-out releases; 0 characters under any box","source":"docs/evals/foia-desk.md, test run 5 (30 Sep 2026, current pipeline)"},{"metric":"How the numbers moved as design faults were fixed (the test sets were run five times; no threshold tuned on them)","value":"personal details redacted 81.0% → 78.8% → 91.7% → 96.7% → 98.3%; responsiveness recall 91.4% → 86.0% → 98.0% → 100.0% → 100.0%","source":"docs/evals/foia-desk.md, History; two fresh held-out sets were written before runs 2 and 3"}],"latency_note":"measured on our server, shared card and gateway; the run time and the model cost per 1,000 records are in the measured results below.","in_hosted_demo":true,"receipt_coverage":"strong","receipt_note":"Hosted: every call has its own gateway-signed receipt, listed in the signed record.","hosting":null}],"alternates":[],"services":[{"name":"decosa-api","port":8445,"image":"${DECOSA_REGISTRY}/decosa-api:<tag>","purpose":"GET /foia/info, /foia/samples; POST /foia/scope, /foia/review (SSE or JSON); POST /foia/runs/{id}/decisions; GET /foia/runs/{id}/export?format=pdf|csv|letter|md|record|ledger. Keeps records in memory for the request and the run (one hour), never on disk; logs counts only."},{"name":"vLLM","port":8114,"image":"vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1","purpose":"Qwen3.8-27B NVFP4 behind our gateway (hosted) or called directly with logprobs (self-host)."}],"tools":[{"name":"Synthetic request sets (6 sets, 69 records)","url":null,"license":"Synthetic, written for Decosa (CC0); fictional agencies, people, numbers (555-01xx phones) and .example domains","purpose":"Two development sets (the console samples, 21 records: a federal permit office and a California city) and four held-out test sets (48 records) (a federal grants office, a California water district, a federal parks office, a California school district), with planted personal data, medical details, deliberative text, advice from agency counsel, official contacts that must be released, copies and out-of-scope records."},{"name":"Enron email corpus (CMU copy, via the Hugging Face dataset corbt/enron-emails)","url":"https://www.cs.cmu.edu/~enron/","license":"Public record released by FERC in 2003; distributed by CMU as a research resource, with a request to respect the privacy of the people in it","purpose":"A stand-in for agency email: 39 messages from May-August 2001 labelled for a hypothetical request about price caps (10 in the demo sample, 29 held out). Responsiveness only; no data planted."},{"name":"scripts/foia_eval.py and docs/evals/foia-desk.md","url":null,"license":"Apache-2.0","purpose":"Runs the sets through the API and scores responsiveness, planted-span recall by kind, exemption labels, redaction precision, the release check and cost."},{"name":"POST /record/verify","url":null,"license":"Apache-2.0","purpose":"Checks the signed record or ledger and names the first entry that was changed. The console also verifies it in your browser."}],"hardware":[{"tier":"1x RTX 5090 32 GB","fits":true,"notes":"Qwen3.8-27B NVFP4 needs about 20 GB of weights plus KV cache for a 12,000-character record. Estimate: same model and prompts as the measured card, not run here on a 5090."},{"tier":"1x RTX PRO 6000 Blackwell 96 GB","fits":true,"notes":"Measured on our server: the eval and the hosted demo ran on this card, shared with other services the whole time."}],"latency":[{"lane":"12-record demo sample, hosted gateway route (about 42 calls)","typical_ms":40000,"source":"measured on our server 2026-09-25: 13-76 s over six runs (41-42 calls each), on a gateway shared with other evaluation jobs; 17 s self-hosted on the direct route"},{"lane":"one record, hosted gateway route, 3 records in flight","typical_ms":4000,"source":"measured on our server 2026-09-25: 77 held-out records in 310 s (about 2.8 calls per record)"}],"benchmark":null,"notes":["Responsiveness uses a typed yes/no question: responsive at p ≥ 0.7, not responsive at ≤ 0.3, the officer decides in between (such records still get redaction proposals). Records dated outside the scope's range are dropped by code, with the reason. The question says outright that an exempt or privileged record is still responsive; before that line the model dropped counsel's emails as 'exempt'.","Kinds of records the request lists ('including emails and memos') are kept as examples, never as a limit, unless the officer sets them as one: the model read them as limits and dropped press releases and letters from the public.","A model mistake on contact details can only over-redact: phones, emails and addresses are redacted unless the model names them as official or they are on the agency's own mail domain.","Names of agency staff and counsel from the mail headers are not redacted even if the model proposes it, and words the request itself uses are not withheld as commercial or other non-privacy information.","A personal detail redacted in one record is redacted wherever it appears in the release; when one record released it as official and another redacted it, both go to the officer and the redaction wins until they decide.","The release PDF is re-typeset from the records' text: box widths are rounded to 4 characters so a box does not give away a name's length; the original layout is not kept, and scanned images are not read."]},"buyer_facts":[{"label":"Data retention","value":"Records are held in memory for the request and the run (one hour, so the release can be rebuilt after the officer's decisions), for the token or key that made it. Nothing is written to disk; logs carry counts only."},{"label":"What leaves the box","value":"Self-hosted: nothing. Hosted demo: the records go to the model through our gateway, which is why the hosted demo is for the sample sets only."},{"label":"Model cost per 1,000 records","value":"Measured on the held-out test run at the gateway's list price (see Cost at list price in the measured results); GPU time only when self-hosted."},{"label":"Exemption lists","value":"Federal FOIA (5 U.S.C. § 552(b)(1)-(9)) and the California Public Records Act, checked against the statutes on 25 Sep 2026; any other state as a custom list the agency checks."},{"label":"Output","value":"Per record: responsive or not, disposition, each redaction with its exemption, category and reason. A release PDF with a passed integrity check, the index as CSV, a response letter draft, a review memo, a signed record and a signed ledger of each officer decision."},{"label":"Input","value":"The request text and the records as JSON text (id, date, from, to, cc, subject, custodian, body). Up to 25 records per request hosted; set DECOSA_FOIA_MAX_DOCS on your own box. No PDF, scan or email-archive intake in this version."}],"data_handling":{"page":"/data#foia-desk","self_host":{"level":"confidential","leaves":"nothing","summary":"Runs on your machine; nothing is sent to Decosa or a third party by default."},"hosted":{"level":"operator-processed","demo_only":true,"summary":"Hosted demo on sample or public data only; self-host for real data.","gpus":"operator-contracted","third_parties":[],"retention":"Records are held in memory for the request and the run (one hour, so the release can be rebuilt after the officer's decisions), for the token or key that made it. Nothing is written to disk; logs carry counts only.","used_for_training":false,"encrypted_while_processed":false},"sealed_tier":{"applies":false,"note":"The sealed tier (raw chat only, never use-case pipelines) is paused at launch (/docs/sealed-tier)."},"external_calls":[]},"console":{"href":"/legal/foia-desk","input":"foia","lanes":[{"id":"scope","title":"Scope","kind":"json"},{"id":"records","title":"Records","kind":"list"},{"id":"redactions","title":"Redactions","kind":"list"},{"id":"release","title":"Release","kind":"list"},{"id":"record","title":"Signed record","kind":"json"}],"samples":[{"n":1,"id":"coastal-permits","title":"Coastal permits","deep_link":"/legal/foia-desk?sample=1&autorun=0"},{"n":2,"id":"alder-point","title":"Alder point","deep_link":"/legal/foia-desk?sample=2&autorun=0"},{"n":3,"id":"ferc-enron","title":"Ferc enron","deep_link":"/legal/foia-desk?sample=3&autorun=0"}],"deep_link_params":{"sample":"1-based index into samples, or a sample id","autorun":"1 = start the run once the sample is loaded; 0 (default) = only preselect","reduce-motion":"1 = turn off animations"}},"api":{"base":"https://api.decosa.ai","contract":"/api/contract.json","contract_markdown":"/api/contract.md","reference":"/docs/api","keys":"/account/keys"},"prompts":{"hosted":"/prompts/foia-desk-hosted.md","selfhost":"/prompts/foia-desk-selfhost.md","assemble":"/prompts/foia-desk-assemble.md","mac":"/prompts/foia-desk-mac.md"},"rehearsal":{"bundle":"/samples/foia-desk.zip","bundle_url":"https://decosa.ai/samples/foia-desk.zip","folder":"/samples/foia-desk/","expected":"/samples/foia-desk/expected.json","files":["/samples/foia-desk/expected.json","/samples/foia-desk/inputs/people.json","/samples/foia-desk/inputs/records.json","/samples/foia-desk/inputs/request.json","/samples/foia-desk/inputs/scope.json"],"bytes":3746,"checks":["the resident's comment (CP-002) is released in part","her phone number and email address are redacted under (b)(6)","both Social Security numbers on the dive roster (CP-007) are redacted","both dates of birth on the dive roster are redacted","the record from before the requested range (CP-010) is not responsive, decided by code with no model call","nothing redacted can be recovered from the release PDF","the Vaughn index (CSV) cites (b)(6) for CP-002","the Vaughn index does not repeat a redacted SSN","the signed record verifies","a record with its redaction count changed no longer verifies","every model call has a signed receipt"],"licence":"Synthetic, written for Decosa (CC0): the Bureau of Coastal Permits, Tidewater Dredge Co. and every person, address, phone number and SSN are fictional.","about":"A federal FOIA request about a dredging permit, its confirmed search scope and three synthetic records from a fictional Bureau of Coastal Permits: a resident's comment with her phone and email, a dive-team roster with two Social Security numbers, and an email from before the requested date range. The comment must be released in part with the phone and email redacted under (b)(6), both SSNs redacted, the early record put out of scope by code, the release PDF must pass its recoverable-text check and the signed record must verify.","run":{"containers":"docker compose exec api python scripts/rehearse.py foia-desk","checkout":"python scripts/rehearse.py foia-desk --bundle foia-desk.zip --base-url http://127.0.0.1:8445","mac":".venv/bin/python scripts/rehearse.py foia-desk"},"guidance":"Set up with a coding agent (we recommend Claude Code with Claude Opus 5.5; any capable coding agent works) on mock data only, run the rehearsal until every check passes, then run your own data locally yourself. Never give the agent real data during setup."},"hardware_fit":{"check":"/self-host/hardware?use=foia-desk","data":"/api/hardware.json","tiers":[{"id":"lite","gpu_gb":57.6,"basis":"stack","unknown":[]},{"id":"standard","gpu_gb":57.6,"basis":"stack","unknown":[]}],"mac":{"fit":"full","memory_gb":32}},"links":{"page":"/legal/foia-desk","json":"/use-cases/foia-desk.json","metrics":"/metrics/foia-desk","console":"/legal/foia-desk","console_sample":"/legal/foia-desk?sample=1&autorun=0","stack":"/legal/foia-desk#stack","try_live":"/legal/foia-desk","watch":"/legal/foia-desk","build":"/legal/foia-desk#build","self_host":"/legal/foia-desk#self-host","prompts":{"hosted":"/prompts/foia-desk-hosted.md","selfhost":"/prompts/foia-desk-selfhost.md","assemble":"/prompts/foia-desk-assemble.md","mac":"/prompts/foia-desk-mac.md"}}}