61 · Creative and media · Compliance and trust · live
Made-for-kids content pre-flight
Eval results
Scored on a held-out or test splitRun 26 Sep 2026Eval write-up (decosa-api, access required)
- Made for kids or not (full check)39 / 40test splitn = 40Dev: 24 / 24. The miss was a close call (p = 0.59).
- Audience, 4-way (primary / mixed / general / mature)37 / 40test splitn = 40Two mixed-audience uploads called primary
- Setting mismatches found15 / 15, 0 falsetest splitn = 15
- Made for kids or not, audience-only audit39 / 40test splitn = 405 calls per upload instead of 15
- Planted requests for children's details flagged3 / 3test splitn = 3
- Frame check: picture-only stills, vision on10 / 12held outn = 126 / 12 with vision off. Public-domain images labelled by the building agent; never used for tuning
Dataset
64 synthetic uploads (24 dev, 40 test) written from the FTC's factors, YouTube's made-for-kids guidance and the Disney complaint, with the uploader's setting and planted data requests and sales pitches; 12 public-domain stills (6 children's-book illustrations, 6 not for children) for the frame check.
Caveats
- The same author wrote the uploads, the labels and the prompts; one prompt change was made on dev before the test split ran.
- The test split ran twice: the first run overlapped a gateway outage (one upload unanswered; otherwise the same scores) and was re-run unchanged; the complete run is reported.
- Short synthetic uploads that are on the nose; real channels, long vlogs and non-English videos will do worse.
- The frame check used historical illustrations, not modern kids' video. Not a COPPA compliance determination.
Nightly smoke check
Loading the nightly status…
- Result
- pass
- Run
- 26 Sep 2026
- Latency, this run
- n/a
- p50 over passed runs
- 7.2 s
- Receipts
- 15
- Model calls
- n/a
- Tokens
- n/a
- Cost per run
- $0.004
Self-host verification
Verified on 26 Sep 2026: Fresh clone into a clean directory, docker build of the api image (85 s), the api with a named volume on host networking against the running local vLLM (Qwen3.8-27B) and diarizer; then torn down.
The puppet sample gave made for kids, a high mismatch and the expected flags in 3.6 s (11 attested calls); the review record verified and failed once the designation was changed; the video sample was transcribed (6 lines, signed ASR receipt) and OCR'd in the container; the rehearsal bundle passed 13 of 13; no titles or transcript text in the logs. The model server's own startup was not re-verified (no new GPU load).
Rehearsal bundle: kids-content-preflight.zip (4 KB, 13 checks). Mock inputs plus the expected results, so you can prove your own setup works before any real data touches it.
Known limits
- Hosted verification ran on the pre-release server (decosa-api the pre-release branch on our server, gateway route); production gets this tool when the branch merges.
- Measured on 64 synthetic uploads written and labelled by the building agent; not on real channel metadata or with an independent reviewer's labels.
- Mixed-audience uploads with a stated child age range are usually called primary; the made-for-kids designation is still right.
- The audience probability comes from sampling and moves between runs; one sample trailer landed at 0.41 on one run and 0.93 on another.
- The frame check was measured on 12 public-domain stills, not on real kids' video.
Models and licences
Standard tier. Licence posture: permissive (Apache, MIT or BSD).
- Evidence lines per FTC factor (quotes), a typed audience choice with probabilities, a typed yes/no per factor (typed-judgment, vertical 24); with vision on, a description of four framesQwen3.8-27B (NVIDIA NVFP4)Apache-2.0
- Speech to timed lines for a video sent without a transcript, with a speech receipt signed by the instanceMOSS-Transcribe-Diarize 0.9BApache-2.0
- Reads on-screen text from six sampled frames of a videoTesseract OCR 5 (English)Apache-2.0
- The pre-flight: intake, quote checks, word-list cues, the setting check, the kids-directed flags, the batch audit, sign-off and the signed review record (no model; CPU)decosa-api kids module (decosa_api/verticals/kids)AGPL-3.0-or-later
All quality evidence
Every sourced number on the tool’s Stack tab, by tier. Some are proxies from another task; their labels say so.
How often is the made-for-kids call right, and are mismatches caught?
- Made for kids or not (held-out 40): 39 of 40 (the miss came back as a close call for a reviewer)
- Setting mismatches found: 15 of 15 (no false mismatch)
- Audience, 4-way: 37 of 40 (two mixed-audience uploads called primary)
- Audience-only audit: 39 of 40 (a third of the calls)
- Frame check, picture only: 10 of 12 (6 of 12 without it)
Source: decosa-api docs/evals/kids-content-preflight.md, 2026-09-26
Lite · audience only (the catalogue audit) (2)
- held-out test, 40 synthetic uploads: made for kids or not / setting mismatches found: 39 / 40 / 15 of 15, 0 falsedecosa-api docs/evals/kids-content-preflight.md, measured on our server 2026-09-26, gateway route
- planted requests for children's details / sales pitches flagged (word lists only): 2 of 3 / 0 of 3decosa-api docs/evals/kids-content-preflight.md, 2026-09-26
Standard · full check on text (hosted demo) (4)
- held-out test, 40 synthetic uploads: made for kids or not / 4-way audience: 39 / 40 / 37 / 40decosa-api docs/evals/kids-content-preflight.md, measured on our server 2026-09-26, gateway route; prompts frozen on a separate 24-upload dev split
- setting mismatches found / false mismatches / close calls on correct uploads: 15 of 15 / 0 / 2decosa-api docs/evals/kids-content-preflight.md, 2026-09-26
- planted requests for children's details / sales pitches flagged: 3 of 3 / 2 of 3decosa-api docs/evals/kids-content-preflight.md, 2026-09-26
- demo samples: audience and setting check as planted: 7 of 7decosa-api docs/evals/kids-content-preflight.md, 2026-09-26
Best · adds the frame check (vision) (2)
- 12 public-domain stills, picture only: made-for-kids call agrees with the label, vision off / on: 6 of 12 / 10 of 12decosa-api docs/evals/kids-content-preflight.md, measured on our server 2026-09-26, gateway route (children's-book illustrations 0 of 6 / 4 of 6; other stills 6 of 6 both ways)
- frame check on real kids' video: not measured yet