{"schema_version":"1","site":"https://decosa.ai","id":"paper-claim-check","num":"43","name":"Citation and claim checker for papers","tool_name":"Check a paper's citations","short":"Citation check","blurb":"Checks every citing sentence of a manuscript against the cited paper's open full text and quotes the passage: supported, overstated, not supported or contradicted, one receipted call per citation. Each reference is looked up in Crossref and OpenAlex for retractions, corrections, wrong DOIs and years, and self-citation stacks are flagged. The output is an issue list and a signed report for an editor, reviewer or integrity office to act on.","status":"live","labels":{"industry":["science-research"],"job":["review"],"input":["text","files"],"deploy":["hosted","selfhost"],"status":"live","output":["data","record"],"data":["confidential"],"hardware":"gpu-96","licence":"permissive"},"industries":["science-research"],"runs_in":["hosted","selfhost"],"part_of":[],"built_from":["grounding","signed-record"],"models":"Qwen3.8-27B","where":"Hosted for published papers and your own drafts; self-host for manuscripts under review","hardware":"1× RTX PRO 6000 (96 GB) or 1× RTX 5090 (32 GB) for the model; parsing and the reference checks run on CPU","final_artifact":"An issue list, every citation with its verdict and quoted passage, a Markdown or JSON report and a signed record.","self_host_first":false,"verification":{"receipt_coverage":"full","summary":"Receipt per citation checked; signed report","manual_qa":{"hosted":{"date":"2026-09-25","result":"pass","p50_ms":9574,"p95_ms":null,"runs":null,"receipts_per_run":4,"cost_per_run_usd":0.0028},"selfhost":{"date":"2026-09-25","result":"pass","method":"Fresh clone of the branch into a clean directory, docker build of docker/api/Dockerfile, the api service with a named volume plus the GROBID container, pointed at the already-running local vLLM (Qwen3.8-27B NVFP4 on 127.0.0.1:8114) through host networking; then torn down.","notes":"Verified on 2026-09-25: the image builds, the service starts healthy with GROBID parsing, the smoke test passes (10.5 s, 4 attested calls), the planted sample flags reference 43 retracted, the overstated [42] sentence and reference 1's year, the signed report verifies and fails when one verdict is changed, and no manuscript text reaches the logs. In one of three runs the swapped [36] citation came back supported."},"known_limits":["About one flag in four on correct background citations is noise (12 of 49 on the test papers); each flag quotes the passage, so a reader can dismiss it quickly.","Only open-access cited papers are read; 56% of test citations had only an abstract or nothing, and those are marked unavailable.","Labels and plants are Claude's (an AI agent), not a domain expert's, on six biomedical papers.","Run-to-run variation: the same citation can be flagged in one run and supported in the next.","PDF text loses superscript citation numbers; figures and tables are not read."],"nightly_covers":null},"nightly":"https://api.decosa.ai/verify/status"},"eval_summary":{"metrics":[{"name":"Planted wrong-paper citations flagged","value":"12 of 12","unit":null,"n":12,"split":"test","note":"95% CI 76-100%"},{"name":"Planted overstatements flagged","value":"12 of 12","unit":null,"n":12,"split":"test","note":"All came back contradicted, with the differing passage quoted. 95% CI 76-100%."},{"name":"False alarms on correct citations with open full text","value":"12 of 49 (24.5%)","unit":null,"n":49,"split":"test","note":"95% CI 14.6-38.1%: about one flag in four is noise."},{"name":"Planted retracted citations / year errors / DOI swaps flagged","value":"6 of 6 / 6 of 6 / 6 of 6","unit":null,"n":18,"split":"test","note":"Year and DOI results are after fixes made following the test run (before: 5 of 6 and 4 of 6)."},{"name":"Retraction flag on known retracted works, with DOI / without","value":"68 of 68 / 68 of 68","unit":null,"n":68,"split":"heldout","note":"Drawn from Crossref's own notices: tests the pipeline, not Retraction Watch coverage."},{"name":"Retraction flag on random control articles, with DOI / without","value":"0 of 60 / 0 of 60","unit":null,"n":60,"split":"heldout","note":null}],"dataset":"Eight CC BY PubMed Central papers: 2 dev papers used to write the prompt, 6 test papers fixed before any test run (471 citation pairs), with planted errors; 60 unplanted citation pairs labelled blind; a retraction set of 68 retracted works and 60 random controls from Crossref/OpenAlex metadata.","held_out":true,"caveats":["False-alarm labels are an AI agent's, not a domain expert's, on only 49 correct citations: a wide interval.","Overstatement plants were hand-written by the builder.","Only open-access cited papers can be checked (44% of test pairs had full text); figures and tables are not read.","Reference-check fixes were made after the test run and re-run without the model; the claim verdicts are from the first run.","Run-to-run variation: the same sentence can be flagged in one run and supported in the next."],"date":"2026-09-25","doc_url":"https://decosa.ai/metrics/evals/paper-claim-check"},"stack":{"summary":"Paste a manuscript with its reference list, or upload a PDF, DOCX or LaTeX file. Each reference is looked up in Crossref and OpenAlex: retractions, withdrawals, expressions of concern and corrections (Crossref's notices, which include the Retraction Watch data), DOIs that do not exist or point to another paper, and wrong years or first authors. For every sentence that cites something, an open model reads the cited paper's open full text (PubMed Central open access through Europe PMC, or arXiv) and says whether the paper supports it, overstates it, does not say it or says otherwise, quoting the passage. Citations whose full text is not open are marked unavailable, never guessed. With the authors' names it also flags self-citation stacks. The result is an issue list and a signed report with hashes, verdicts, cited DOIs and receipt ids but no manuscript text. For authors, reviewers, editors and research-integrity offices; it screens, a person decides.","tagline":"Every citing sentence checked against the cited paper's open full text, with the passage quoted; retracted and corrected sources, wrong DOIs and years, and self-citation stacks flagged; one receipt per check and a signed report.","deployment":"hosted-or-self-host","regulatory_note":"A screening aid, not a finding of research misconduct. Citation manipulation, including self-citation stacking, is treated by journals under COPE's guidance, and an allegation is for the journal or institution to investigate. A manuscript under peer review is confidential: NIH prohibits its peer reviewers from using generative AI tools to analyse or critique applications (notice NOT-OD-23-149, June 2023), and many publishers tell reviewers not to upload manuscripts to AI tools, so reviewers and editors should run the self-hosted version on their own machine. Only DOIs, identifiers and reference strings are sent to Crossref, OpenAlex, Europe PMC, arXiv and doi.org; the manuscript's text is not. Open-access full text is read under each article's own licence and only short passages are quoted back. Retraction and correction data come from Crossref (which has made the Retraction Watch database openly available since September 2023) and OpenAlex, and can lag the publishers. Model licence: Apache-2.0 (Qwen3.8-27B). Checked 25 Sep 2026.","components":[{"id":"checker","role":"Checker: manuscript and reference parsing, metadata lookups, retraction and citation-error checks, self-citation, signed report (no model; CPU)","name":"decosa-api papercheck module (decosa_api/verticals/papercheck)","hf_repo":null,"license":"AGPL-3.0-or-later","params":null,"quant":null,"vram_gb":0,"memory_gb_estimate":null,"engine":"Python 3.12; pdftotext (poppler) for PDFs; the grounding checker (vertical 17) for passage spans, evidence selection and verdict parsing","receipt_coverage":"partial","in_hosted_demo":null,"tiers":["lite","standard","wanted"],"alternative_to":null},{"id":"grobid","role":"Reference parser (optional): splits each reference into authors, title, year and DOI","name":"GROBID 0.8.2 (CRF models)","hf_repo":null,"license":"Apache-2.0","params":null,"quant":null,"vram_gb":0,"memory_gb_estimate":null,"engine":"Java service, CPU only (image lfoppiano/grobid:0.8.2)","receipt_coverage":"none","in_hosted_demo":null,"tiers":["lite","standard","wanted"],"alternative_to":null},{"id":"judge","role":"Model: reads each citing sentence against the cited paper's passages, then reviews its own flags","name":"Qwen3.8-27B (NVFP4)","hf_repo":"nvidia/Qwen3.8-27B-NVFP4","license":"Apache-2.0","params":"27.8B","quant":"NVFP4 (MLP NVFP4, GDN/attention FP8) + FP8 KV cache; MTP head, 3 draft tokens","vram_gb":20,"memory_gb_estimate":null,"engine":"vLLM 0.29.0, temperature 0, thinking off","receipt_coverage":"strong","in_hosted_demo":true,"tiers":["standard"],"alternative_to":null},{"id":"glm-judge","role":"Stronger open model for the claim check (wanted)","name":"GLM-5.3-Flash","hf_repo":null,"license":"MIT","params":null,"quant":null,"vram_gb":null,"memory_gb_estimate":170,"engine":"SGLang (vLLM does not run it on these Blackwell cards yet)","receipt_coverage":"none","in_hosted_demo":false,"tiers":["wanted"],"alternative_to":null}],"tiers":[{"id":"lite","label":"Lite · references only, any CPU","summary":"Retractions, corrections, wrong DOIs, years and first authors, missing and uncited references, and self-citation stacks, with no model and no GPU (check_claims: false). No claim check.","components":["checker","grobid"],"hardware":"Any CPU","quality_evidence":[{"metric":"Retraction flag on known retracted works (60 sampled from Crossref's notices + 8 well-known), cited with DOI / without a DOI","value":"68 of 68 / 68 of 68","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25; the sampled set comes from the same Crossref data, so this tests the pipeline, not Retraction Watch's coverage"},{"metric":"Retraction flag on 60 random journal articles (controls), with DOI / without","value":"0 of 60 / 0 of 60","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25"},{"metric":"Planted retracted citations / wrong years / wrong DOIs found in 6 test papers","value":"6 of 6 / 6 of 6 / 6 of 6","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25; the no-DOI retraction and the DOI plants after fixes made on the test run (disclosed there)"},{"metric":"Unplanted reference flags on the same 6 real papers (431 references): genuine / arguable / wrong","value":"3 / 3 / 0 errors and warnings (two wrong DOIs in a published paper, one malformed author list; one ambiguous supplement DOI, two software records whose year differs), after the fixes","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25; judged by Claude (an AI agent)"}],"latency_note":"measured: seconds per paper with a warm cache; first lookups are paced by the polite rate limits.","in_hosted_demo":true,"receipt_coverage":"partial","receipt_note":"No model calls, so no receipts; the report is signed by the instance.","hosting":null},{"id":"standard","label":"Standard · one 96 GB card (measured; hosted demo)","summary":"Qwen3.8-27B on an RTX PRO 6000 reads every citing sentence against the cited paper's open text and reviews its own flags; the reference checks run alongside. Also fits a 32 GB card (estimate).","components":["checker","grobid","judge"],"hardware":"1x RTX PRO 6000 96 GB","quality_evidence":[{"metric":"Planted wrong-paper citations flagged not supported or contradicted, 6 held-out CC BY papers","value":"12 of 12","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25"},{"metric":"Planted overstatements flagged (numbers inflated, association made causal, population or design changed)","value":"12 of 12, each with the differing passage quoted","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25; plants written by Claude (an AI agent)"},{"metric":"False alarms on correct citations with open full text (blind labels)","value":"12 of 49 flagged (24.5%, 95% CI 14.6-38.1%)","source":"decosa-api docs/evals/paper-claim-check.md, 2026-09-25; labels written by Claude (an AI agent), not domain experts, before the verdicts were seen"}],"latency_note":"measured on our server, shared gateway: under a minute to a few minutes per whole paper, rising with its citations; about half a minute for the demo.","in_hosted_demo":true,"receipt_coverage":"strong","receipt_note":"Hosted: every check and every review has its own gateway-signed receipt, listed in the signed report.","hosting":null},{"id":"wanted","label":"Wanted · GLM-5.3-Flash for the claim check","summary":"A stronger reasoning model from another family for the background citations the standard model flags too eagerly. It needs two 96 GB cards or a large Mac, so community providers are welcome. Not served yet.","components":["checker","grobid","judge","glm-judge"],"hardware":"Network providers: 2x 96 GB cards or a Mac with 192 GB or more (about 170 GB of weights). Estimate.","quality_evidence":[{"metric":"This eval, same protocol","value":"not measured yet","source":null}],"latency_note":"not measured yet","in_hosted_demo":false,"receipt_coverage":"none","receipt_note":"Not hosted yet, so no receipts today.","hosting":"network"}],"alternates":[],"services":[{"name":"decosa-api","port":8445,"image":"${DECOSA_REGISTRY}/decosa-api:<tag>","purpose":"GET /papercheck/info, /papercheck/samples; POST /papercheck/extract (PDF, DOCX, LaTeX, text), /papercheck/check (SSE or JSON); GET /papercheck/runs/{id}/export?format=md|json|record. Keeps the manuscript in memory for the request and the run for one hour, never on disk; logs counts only."},{"name":"vLLM","port":8114,"image":"vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1","purpose":"Qwen3.8-27B NVFP4 behind our gateway (hosted) or called directly (self-host)."},{"name":"GROBID (optional)","port":8070,"image":"lfoppiano/grobid:0.8.2","purpose":"Reference parsing, CPU only; set DECOSA_PAPERCHECK_GROBID_URL."}],"tools":[{"name":"Crossref REST API","url":"https://api.crossref.org/","license":"Crossref metadata is open (no restrictions on reuse of the bibliographic facts); retraction and correction notices include the Retraction Watch data","purpose":"Works by DOI (title, authors, year, updated-by notices) and bibliographic search for references without a DOI. Polite pool: User-Agent and mailto; our limits 5 requests/s for works and 2/s for searches, under Crossref's 10/s and 3/s."},{"name":"OpenAlex API","url":"https://openalex.org/","license":"CC0","purpose":"Single works by DOI only (no credit cost): is_retracted as a second opinion, PMCID and open-access locations, the abstract. 5 requests/s."},{"name":"Europe PMC REST API","url":"https://europepmc.org/RestfulWebService","license":"Open-access full text under each article's own licence (the demo paper is CC BY 4.0)","purpose":"PMCID, open-access flag, abstract, and JATS full text of open-access articles. 5 requests/s."},{"name":"arXiv","url":"https://arxiv.org/","license":"Per paper, as chosen by the authors","purpose":"PDF full text of cited preprints, at most one request every 3 seconds."},{"name":"scripts/papercheck_eval.py and docs/evals/paper-claim-check.md","url":null,"license":"Apache-2.0","purpose":"Builds the eval from six CC BY PMC papers, plants miscitations, scores detection, false alarms against blind labels, and retraction flags on 128 known retracted works and 60 controls."},{"name":"POST /record/verify","url":null,"license":"Apache-2.0","purpose":"Checks the signed report and names the first entry that was changed. The console also verifies it in your browser."}],"hardware":[{"tier":"Any CPU (references only)","fits":true,"notes":"Measured: the reference checks for a 98-reference paper take 1-2 s with a warm cache; the first run is paced by the polite rate limits (estimate: well under a second per reference). No GPU."},{"tier":"1x RTX 5090 32 GB","fits":true,"notes":"Qwen3.8-27B NVFP4 needs about 20 GB of weights plus KV cache for prompts of about 3,000 tokens. Estimate: same model as the measured card, not run here on a 5090."},{"tier":"1x RTX PRO 6000 Blackwell 96 GB","fits":true,"notes":"Measured on our server: the eval, the hosted demo and the self-host check ran on this card, shared with other services the whole time."}],"latency":[{"lane":"Demo sample, 24 citations and 43 references, hosted gateway route (2 calls in flight)","typical_ms":19800,"source":"measured on our server 2026-09-25, one run; the published version (23 citations) took 13.7 s"},{"lane":"Smoke manuscript, 4 citations, hosted gateway route","typical_ms":9600,"source":"measured on our server 2026-09-25: 8.8-10.4 s over 3 runs"},{"lane":"Whole test papers, 29-143 citations, hosted gateway route (2 calls in flight, saturated gateway)","typical_ms":110000,"source":"measured on our server 2026-09-25: 35-247 s over 6 papers"},{"lane":"Demo sample, self-hosted (clean clone, container build, direct route, GROBID on)","typical_ms":32600,"source":"measured on our server 2026-09-25, one run, 2 calls in flight on a shared card"}],"benchmark":null,"notes":["Only open-access cited papers can be read: in the six test papers 44% of citations had open full text, 48% only an abstract and 8% nothing. An abstract can confirm or contradict a citation but not show that the paper never says something, so those come back unavailable.","A flagged verdict (overstated or contradicted) gets a second model call that must quote the passage that differs; a flag whose quote is not really in the paper is cleared. This removed 23 first-pass flags on the test papers without losing a planted one.","Each reference without a DOI is matched by Crossref's bibliographic search and accepted only when most of the record's title is in the reference and the year or first author matches; a preprint twin or a duplicate record is handled, and a retracted record with the same title is flagged too.","PDF text loses superscript citation numbers; paste text with [n] markers, or upload the DOCX or LaTeX.","Not included yet: paywalled full text (a journal can run it with its own text-and-data-mining access on its own box), figures and tables, and a manuscript-system plug-in."]},"buyer_facts":[{"label":"Data retention","value":"The manuscript is held in memory for the request; the run (verdicts, issues, the text for exports) for one hour, for the token or key that made it. Nothing is written to disk; logs carry counts only."},{"label":"What leaves the box","value":"Self-hosted: only DOIs, identifiers and reference strings, to Crossref, OpenAlex, Europe PMC, arXiv and doi.org. Hosted demo: also the text, to the model through our gateway, so use it for published papers or your own drafts."},{"label":"Model cost per paper","value":"A few cents to about ten cents per paper at the gateway's list price, rising with the number of checked citations (measured); the metadata services are free."},{"label":"Input","value":"Text with a reference list under a heading such as \"References\", or a PDF, DOCX or LaTeX file; numbered [3] or author-year (Smith et al., 2020) citations. Hosted limits: 250,000 characters, 250 references, 60 checked citations per request."},{"label":"Output","value":"An issue list, every citation with a verdict and the quoted passage, a Markdown or JSON report, and a signed record verifiable at /record/verify."},{"label":"Politeness","value":"Identifies itself with a contact address to every service, stays under Crossref's published polite-pool limits, uses only OpenAlex lookups that cost no credits, and fetches arXiv at most once every 3 seconds."}],"hosted_now":{"needs":["qwen3.8-27b"],"off":[],"live_by_default":true,"live_status":"https://api.decosa.ai/status"},"data_handling":{"page":"/data#paper-claim-check","self_host":{"level":"confidential","leaves":"identifiers","summary":"Runs on your machine; by default only short identifiers or a digest go to the public services listed in external_calls."},"hosted":{"level":"operator-processed","demo_only":false,"summary":"TLS to Decosa's server, then decrypted and processed by Decosa's API server, with the open models run by NEAR AI through OpenRouter, with Reka AI as the only fallback under Decosa's account.","gpus":"operator-contracted","third_parties":[],"retention":"The manuscript is held in memory for the request; the run (verdicts, issues, the text for exports) for one hour, for the token or key that made it. Nothing is written to disk; logs carry counts only.","used_for_training":false,"encrypted_while_processed":false},"sealed_tier":{"applies":false,"note":"The sealed tier (raw chat only, never use-case pipelines) is paused at launch (/docs/sealed-tier)."},"external_calls":[{"to":"Crossref, OpenAlex, Europe PMC, arXiv and doi.org","route":"both","sends":"identifiers","what":"DOIs, identifiers and reference strings from the reference list, never the manuscript text, to find and read each cited work.","default":"always","off":null}]},"console":{"href":"/tools/life-sciences/paper-claim-check","input":"papercheck","lanes":[{"id":"issues","title":"Reference issues","kind":"list"},{"id":"citations","title":"Citations","kind":"list"},{"id":"record","title":"Signed report","kind":"json"}],"samples":[{"n":1,"id":"telecommuting-planted","title":"Telecommuting planted","deep_link":"/tools/life-sciences/paper-claim-check?sample=1&autorun=0"},{"n":2,"id":"telecommuting","title":"Telecommuting","deep_link":"/tools/life-sciences/paper-claim-check?sample=2&autorun=0"}],"deep_link_params":{"sample":"1-based index into samples, or a sample id","autorun":"1 = start the run once the sample is loaded; 0 (default) = only preselect","reduce-motion":"1 = turn off animations"}},"api":{"base":"https://api.decosa.ai","contract":"/api/contract.json","contract_markdown":"/api/contract.md","reference":"/docs/api","keys":"/account/keys"},"prompts":{"hosted":"/prompts/paper-claim-check-hosted.md","selfhost":"/prompts/paper-claim-check-selfhost.md","assemble":"/prompts/paper-claim-check-assemble.md","mac":"/prompts/paper-claim-check-mac.md"},"rehearsal":{"bundle":"/samples/paper-claim-check.zip","bundle_url":"https://decosa.ai/samples/paper-claim-check.zip","folder":"/samples/paper-claim-check/","expected":"/samples/paper-claim-check/expected.json","files":["/samples/paper-claim-check/expected.json","/samples/paper-claim-check/inputs/manuscript.md"],"bytes":2306,"checks":["four references are found","citation [1] is supported with a quoted passage","citation [2] is supported with a quoted passage","the wrong paper cited as [3] is flagged","reference 4 is flagged retracted","the retraction is raised as an error issue","the signed record verifies","a record with the [3] verdict changed to supported no longer verifies","verification points at a claim entry","every model call has a signed receipt"],"licence":"The sentences are written for Decosa; the three reference strings come from the reference list of Mauras S et al., PLoS Comput Biol 2021;17(8):e1009264 (CC BY 4.0), and reference 4 is the public bibliographic entry of Mehra et al., The Lancet 2020 (retracted). Part of decosa-api, which will be released under AGPL-3.0-or-later; until then the source is on request.","about":"A four-sentence manuscript citing three papers from the reference list of an open-access CC BY article, plus the retracted 2020 hydroxychloroquine registry study. Citations [1] and [2] must be supported with a quoted passage, [3] (a hospital Staphylococcus aureus paper cited for SARS-CoV-2 super-spreaders) must be flagged, reference 4 must be flagged retracted, and the signed record must verify and catch a changed entry. Lookups go to Crossref, OpenAlex and Europe PMC.","run":{"containers":"docker compose exec api python scripts/rehearse.py paper-claim-check","checkout":"python scripts/rehearse.py paper-claim-check --bundle paper-claim-check.zip --base-url http://127.0.0.1:8445","mac":".venv/bin/python scripts/rehearse.py paper-claim-check"},"guidance":"Set up with a coding agent (we recommend Claude Code with Claude Opus 5.5; any capable coding agent works) on mock data only, run the rehearsal until every check passes, then run your own data locally yourself. Never give the agent real data during setup."},"hardware_fit":{"check":"/self-host/hardware?use=paper-claim-check","data":"/api/hardware.json","tiers":[{"id":"lite","gpu_gb":0,"basis":null,"unknown":[]},{"id":"standard","gpu_gb":57.6,"basis":"stack","unknown":[]},{"id":"wanted","gpu_gb":249.6,"basis":"estimate","unknown":[]}],"mac":{"fit":"partial","memory_gb":32}},"links":{"page":"/tools/life-sciences/paper-claim-check","json":"/use-cases/paper-claim-check.json","metrics":"/metrics/paper-claim-check","console":"/tools/life-sciences/paper-claim-check","console_sample":"/tools/life-sciences/paper-claim-check?sample=1&autorun=0","stack":"/tools/life-sciences/paper-claim-check#stack","try_live":"/tools/life-sciences/paper-claim-check","watch":"/tools/life-sciences/paper-claim-check","build":"/tools/life-sciences/paper-claim-check#build","self_host":"/tools/life-sciences/paper-claim-check#self-host","prompts":{"hosted":"/prompts/paper-claim-check-hosted.md","selfhost":"/prompts/paper-claim-check-selfhost.md","assemble":"/prompts/paper-claim-check-assemble.md","mac":"/prompts/paper-claim-check-mac.md"}}}