Classify with a confidence you can act on

# Decosa Typed-judgment API: use the hosted API

You are wiring Decosa's typed-judgment API into this project. It answers narrow questions about a context with typed
answers: `yesno`, `choice` (one of N options), `score` (ordered levels, 1 to 5 by default) or `labels` (any of N). Each
answer comes with a probability, the full distribution, a short reason and receipt ids, and each request ends with a
record signed by the server. The model is Qwen3.8-27B on pinned open weights, so the same request with the same seed can be
re-run later against the same model (confident verdicts reproduce; see the limits). An eval mode grades an output against a rubric, or compares two outputs. Use only what is listed
below. If you need something else, stop and ask me.

- Base URL: `https://api.decosa.ai`
- Health check: `GET https://api.decosa.ai/healthz`.
- Keep policy in code: the API returns probabilities; thresholds, weights and "what happens next" belong to this project.

## Auth: API key (or a demo session)
1. Preferred: an API key (`dk_…`) from "Get an API key" on the tool page. Keep it in an environment variable,
   `DECOSA_API_KEY`, never in code. Send `Authorization: Bearer $DECOSA_API_KEY`.
2. Without a key: `POST https://api.decosa.ai/demo/session` with `{"vertical": "typed-judgment"}` returns `{"token", "expires_at", "budget"}`.
   a limited number of sessions per network per hour (the current limits are in `demo_sessions` of GET /healthz); each session has a token allowance (its `budget`). Over a limit: HTTP 429 with `Retry-After`.
3. Budget per request: about 90 generated tokens per question (16 with `"reasons": false`) plus 8 per sample. 402 if it
   would not fit. One request at a time per demo token; a key allows 2 at once and 60 requests a minute.

## Endpoints
- `POST /judgment/judge` (token). Body:
  `{"context"?: "text or any JSON", "questions": {"<id>": {"type": "yesno"|"choice"|"score"|"labels", "question": "...", "options"?: {"key": "description"}, "levels"?: ["lowest", "...", "highest"] | 5, "labels"?: {"key": "description"}, "threshold"?: 0.5}}, "method"?: "auto"|"samples"|"single"|"logprobs", "samples"?: 2-8, "seed"?: 0, "reasons"?: true}`
  - Limits: 16 questions, 26 options, 2-9 levels, 12 labels (each label is one model call), context 24,000 characters.
  - `method`: `samples` (the hosted default: the temperature-0 answer plus k seeded samples, 4 by default), `single` (one
    call, the model's stated confidence mapped through a table fit on a dev set), `logprobs` (self-host only until the
    gateway passes log-probabilities through; 400 on the hosted API).
  - JSON by default: `{answers: {<id>: {type, answer, probability, distribution, reason, method, near_tie, receipt_ids, expected? (score), option? (choice), probabilities? (labels)}}, record, usage: {calls, prompt_tokens, completion_tokens, cost_usd}, receipts, budget, note}`.
    A question that could not be judged has `answer: null` and an `error`; the others still come back.
  - With `Accept: text/event-stream` (or `"stream": true`): `ready`, then a `receipt` per model call and an `answer` per
    question as each finishes, then `result` (answers, record, usage), `budget`, `done`.
- `POST /judgment/eval` (token), eval-judge mode, same output shape:
  - rubric: `{"task"?: "...", "reference"?: "...", "candidate": "...", "rubric": [{"id": "faithful", "question": "...", "type"?: "score"}]}`
    Criteria are 1-5 scores (very poor to excellent) unless they say otherwise.
  - pairwise: `{"task"?: "...", "candidates": ["answer A", "answer B"]}` → `answers.preference` with `answer` `a`, `b` or
    `tie`, the averaged distribution over both orders, and `position_consistent`.
- `POST /judgment/batch` (token): `{"items": [{"id": "row-1", "context": ..., "questions": {...}}], "method"?, "samples"?, "seed"?, "reasons"?}`.
  Up to 50 items and 400 model calls per batch. JSON `{results: [{id, answers, record, usage}], usage: {items, calls, cost_usd}}`,
  or NDJSON lines (`item` ..., `usage`, `budget`, `done`) with `"stream": true`.
- `POST /judgment/v1/systemone` (token): the local decision server's shape (`{state, questions: {id: {type: noul|choice|score, instructions, criteria}}, options: {seed, samples}}`),
  answered in that shape (`noul`, `choice`, `score`, `probabilities`, `confidence`) plus `record`. Score criteria are zero-based there.
- `POST /judgment/verify` (no token) `{"record": {...}, "context"?: ...}` → `{valid_signature, signed_by_this_server, context_matches?}`.
- `GET /judgment/info` (methods, limits, prices, the calibration in use and the held-out numbers), `GET /judgment/samples`,
  `GET /attest/signing-key` (no token).

## Example: route support tickets (Python, `pip install httpx`)
```python
import httpx, os
API = "https://api.decosa.ai"
H = {"Authorization": f"Bearer {os.environ['DECOSA_API_KEY']}"}
body = {
    "context": ticket_text,
    "questions": {
        "refund": {"type": "yesno", "question": "Does the customer ask for their money back?"},
        "team": {"type": "choice", "question": "Which team should handle it?",
                 "options": {"returns": "damaged or unwanted items", "billing": "charges", "shipping": "late or lost parcels"}},
    },
    "method": "samples", "samples": 4, "seed": 0,
}
r = httpx.post(f"{API}/judgment/judge", json=body, headers=H, timeout=120)
r.raise_for_status()
out = r.json()
team = out["answers"]["team"]
if team["probability"] < 0.7:
    send_to_human(ticket_id, reason=team["reason"])          # the threshold is yours
else:
    route(ticket_id, team["answer"])
store(ticket_id, out["record"])                                # keep the signed record with the decision
```

## Same thing in TypeScript (Node 18+ fetch)
```ts
const res = await fetch("https://api.decosa.ai/judgment/judge", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.DECOSA_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({ context: ticket, questions: { refund: { type: "yesno", question: "Does the customer ask for their money back?" } } }),
});
if (!res.ok) throw new Error(`${res.status}: ${(await res.json()).error}`);
const { answers, record, usage } = await res.json();
console.log(answers.refund.answer, answers.refund.probability, record.verdicts_sha256, usage.cost_usd);
```

## And curl
```bash
curl -sS https://api.decosa.ai/judgment/judge -H "Authorization: Bearer $DECOSA_API_KEY" -H "Content-Type: application/json" \
  -d '{"context":"The parcel arrived two weeks late.","questions":{"late":{"type":"yesno","question":"Was the delivery late?"}}}'
```

## Re-run and compare (the reproducibility check)
Send the same request with the same `seed` again and compare `record.verdicts_sha256` (answers only) and
`record.answers_sha256` (answers and probabilities). Also check `record.method.model_root` (the weights) and
`record.method.prompt_sha256`: if either changed, the model or the prompt changed, and scores are not comparable.

## Verify a record yourself (`pip install cryptography`)
```python
import json, urllib.request
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
record = out["record"]                                         # from the request above
pub = json.load(urllib.request.urlopen("https://api.decosa.ai/attest/signing-key"))["pubkey"]
assert record["signer"] == pub
body = {k: v for k, v in record.items() if k != "sig"}
msg = record["v"].encode() + b"\n" + json.dumps(body, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
Ed25519PublicKey.from_public_bytes(bytes.fromhex(pub)).verify(bytes.fromhex(record["sig"]), msg)
```
Probabilities in the record are integers in basis points (10000 = 1.0). Each id in `record["receipts"]` resolves at
`GET https://api.decosa.ai/receipts/{id}` to the gateway-signed receipt for that model call.

## Honest limits
- Re-runs are not bit-identical on a busy shared server (batching changes the arithmetic). In our eval, 98-99% of yes/no
  answers and 96-98% of four-option answers held on a re-run, and no answer with probability 0.9 or more changed. Answers
  under 0.75 come back with `near_tie: true` (labels: a list); treat them as uncertain, not as verdicts.
- Cost at the gateway's list price, measured: about $0.12 per 1,000 one-call judgments without a reason, $0.17 with one,
  and about $0.60 per 1,000 with the default 4 samples. `usage.cost_usd` gives the exact figure per request.
- Probabilities were calibrated on public dev sets (BoolQ for yes/no, MMLU for choices) and measured on held-out test
  sets: see the Stack tab. Scores (1-5) and labels use the yes/no and default settings and are not separately calibrated.
  Calibration varies by domain: log your own outcomes and check before a threshold decides anything that matters.
- With `samples`, probabilities are coarse (k+1 votes, smoothed). More samples cost more prompt tokens.
- The model trusts the context you send. Put facts in the context; do not ask it to know things it cannot see.

On request. The container images and the compose file aren’t public yet. Ask for self-host access and Decosa sends the registry (DECOSA_REGISTRY) and the compose file’s URL (DECOSA_COMPOSE_URL) these steps use. They are the steps we tested end to end on a fresh machine.

# Decosa Typed-judgment API: run it yourself (containers)

You are setting up the Decosa typed-judgment API on this machine, so contexts never leave it and the model weights stay
pinned. It answers typed questions (yes/no, one of N, scores, labels) with calibrated probabilities and signs a record
per request. Nothing is sent to Decosa's hosted API.

Status: the container images (${DECOSA_REGISTRY}/decosa-*) and the compose file are on request while self-host is in early access (not on a public registry yet): ask at https://decosa.ai/contact?topic=self-host, and Decosa sends the registry as DECOSA_REGISTRY, the compose file URL as DECOSA_COMPOSE_URL, and pull access. If a pull fails with
"not found", "denied" or "unauthorized", stop and tell me. Do not substitute other images.

Ask me before any command that needs sudo, and show me the command first.

## Step 0: set up with a coding agent, rehearse on mock data, then go private

This prompt is for a coding agent running on the machine that will host the service. We recommend Claude Code with
Claude Opus 5.5; any capable coding agent works. Work in this order:

1. Set up on mock data only. During the whole setup you (the agent) work with the synthetic sample bundle below and
   nothing else. Do not ask me for real data, and do not open, read, list or copy files that hold real data, even to
   "test with something realistic".
2. Rehearse. When the steps below are done and the service is healthy, fetch the mock-data bundle for this tool,
   https://decosa.ai/samples/typed-judgment.zip (2 KB, 10 checks, synthetic or openly licensed: see `licence` in expected.json),
   show me what is in it, and run the rehearsal against the local API:
   `docker compose exec api python scripts/rehearse.py typed-judgment` (the api image carries the same bundle under /app/rehearsal/typed-judgment/;
   with no key set, the script asks the local API for a short demo token). From a decosa-api checkout instead:
   `python scripts/rehearse.py typed-judgment --bundle typed-judgment.zip --base-url http://127.0.0.1:<PORT>`.
   It sends the mock inputs to the local API and prints PASS or FAIL for each expected property (for example: "the customer asks for a refund: yes", "the refund answer carries a probability of at least 0.8", "the returns team handles it first"). Show me
   the full output. Every check must pass. If one fails, fix the install and run it again; never edit `expected.json`
   to make a check pass.
3. Stop there. Once the rehearsal passes, tell me, and I will run my own data against the local API myself, on this
   machine.

For the person running this: a coding agent that runs in the cloud sees everything in its context, including files it
reads, command output and anything pasted into the chat. Keep real data out of the chat and out of anything the agent
can read. Switch to your own data only after the rehearsal has passed and the agent's work is done.

## Steps
1. Docker: if `docker compose version` fails, install Docker Engine and the compose plugin using Docker's official
   instructions for this distribution (docs.docker.com/engine/install). Also install the NVIDIA container toolkit and
   check `docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi`.
2. Fetch the compose file:
   `mkdir -p ~/decosa && cd ~/decosa && curl -fsSL "${DECOSA_COMPOSE_URL}" -o compose.yaml`
   Read it. Keep the `llm` service (Qwen3.8-27B on vLLM, weights revision `482ca0f3832238542f8f5295dde86b5f22711d80`)
   and the `api` service. For the `api` service set `DECOSA_LLM_ROUTE=direct`, `DECOSA_LLM_URL=http://llm:8000/v1`,
   `DECOSA_LLM_MODEL=qwen3.8-27b` and bind every port to 127.0.0.1. Keep `--enable-prefix-caching` on the `llm` service.
3. Pull and start: `docker compose pull && docker compose up -d`. Wait for the `llm` health check (the first start
   downloads about 20 GB of weights).
4. Check: `curl -fsS http://127.0.0.1:<PORT>/judgment/info` must show `methods.auto: "logprobs"` (the direct route reads
   token log-probabilities: one call per question, the best-calibrated method). `GET /attest/signing-key` shows this box's
   public key. Show me the key: it is what others pin to verify my records.
5. Smoke test: get a token with `POST /demo/session {"vertical":"typed-judgment"}`, fetch `GET /judgment/samples`, and
   send the `support-triage` sample's `request` (without `method` and `samples`) to `POST /judgment/judge`. Expect
   `refund: yes` and `team: returns`. Send it again and compare `record.verdicts_sha256`: it must match. Then
   `POST /judgment/verify` with the record: `valid_signature` and `signed_by_this_server` must both be true.
6. Report back: the public key and key id, the answers with their probabilities, whether the re-run matched, and how
   long each request took.

Off by default. Joining as a provider serves other people's requests on this GPU; never do it on a box that holds
confidential contexts. If I ask for it later, follow the Provide page instead of improvising.

No NVIDIA GPU? This tool also runs entirely on an Apple Silicon Mac (MLX, 32 GB of unified memory or more): use https://decosa.ai/prompts/typed-judgment-mac.md instead.