Reaper — auditor agent: source verification method and scorecard

✓ Verified by Reaper · 360/394 · 19 Oct 2026

Reaper carries out two jobs together: (1) in multi-voice Spark (Kıvılcım) rounds, testing every academic and regulatory citation one by one against Crossref, PubMed E-utils, and DOI registries to eliminate fabricated references and generic textbook summaries; (2) checking with a proof command whether every method named on this site and in its documents is actually running today (1 round every 6 hours). Spark stays in the middle; Reaper runs in its own isolated context and is itself audited via deliberate fault injection.

Method: source verification and rote-summary filtering (Yöntem: kaynak doğrulama)

In a multi-voice Spark round, blind external and local voices submit a thesis, theory, and bibliography for each question without hearing one another. Before synthesis, Reaper tests every citation one by one against primary registries (Crossref API, PubMed E-utils, doi.org, WHO, and Official Gazette archives):

• One-by-one citation verification (Crossref / PubMed / DOI): First author, publication year (±1), journal or publisher, and title core are queried against primary records. Matching records are stamped DOĞRULANDI (verified); vague citations that cannot be located are stamped DOĞRULANAMADI (unverified) and cannot be used as support.
• A fabricated citation invalidates the answer (light class included): A non-existent article, fake title, or author–journal mismatch is classified as UYDURMA · AĞIR (heavy fabrication); a real article paired with a non-existent DOI/PMID or an identifier pointing to a different paper is classified as UYDURMA · HAFİF (light fabrication). The rule is mechanical: any fabricated citation — light class included — eliminates that voice’s thesis on that question from synthesis and halves its weight in the next round.
• No voice is a benchmark by authority alone: No distinction is made between a frontier cloud model or a local model; no voice is exempt from verification by virtue of its title or authority. In Spark, the measure is what proves true.
• Rote-summary tag (EZBER): Even when a citation is real, does the voice’s output bring specific counter-evidence or a sharpening objection to the brief’s question and the author’s field thesis, or does it merely repeat a textbook / general literature summary? Generic summaries that bring neither new evidence nor a concrete objection are filtered out with the EZBER tag and do not enter synthesis.

Verification class	Definition and mechanical sanction
DOĞRULANDI (verified)	Primary record found; first author + year + journal/publisher + title core match. Enters synthesis.
DOĞRULANAMADI (unverified)	Record not found (journal name only or vague citation); no proof of fabrication, but excluded from synthesis support.
UYDURMA · AĞIR (heavy fabrication)	Non-existent paper, non-existent DOI, author–journal mismatch, or fake title. Thesis on that question is eliminated.
UYDURMA · HAFİF (light fabrication)	Paper is real, but the supplied identifier (DOI or PMID) does not exist or belongs to another paper. Counted as fabrication by mechanical rule; thesis eliminated.
EZBER (rote summary)	Citation is not fabricated, but the response is a generic literature summary that brings no specific evidence or objection to the brief’s question; excluded from synthesis.

Sector article round measurement (18–19 Oct 2026): across the 10 external voices, 394 citations were tested → 360 verified · 9 unverified · 25 fabricated (Wave 1: 356 citations → 322 verified). Local models (S6/S7/S8/S9) produced 18 heavy fabrications; the exact same fake citations (Joda 2017 and Sailer 2009) recurred across three voices, exposing shared training-data memorisation. In S10, 7 light identifier fabrications resulted in 5 theses eliminated. In the follow-up rings, S3 (Astra) scored 38/38 and the Author (S0) 26/26 (consolidated total: 420 citations → 386 verified).

Scorecard: source verification and audit rounds

The scorecard table below is read from /data/reaper_karne.json; the 6-hour audit runner and every new Spark round append a row to this file.

System audit: what it measures and three arms

Three things are written for every method: a proof command, the expected result and an owner. Reaper runs the command and compares the output with the expectation. The method list: methods described on the site, tools with a contract, scheduled services, runners and agents.

Arm	Engine	Job
Measurement	Qwen 3.8 27B (local) + mechanical proof	Collects the evidence; gives no verdict.
Judgement	Gemma 4 31B (local)	Classifies each method and writes its fall test.
Synthesis	Argon (high thinking)	Compares the two arms, resolves conflicts by running the command itself, updates the site and the board.

Special-training plan: the untrained baseline is measured first on a blind set of 50+ methods; training is done only if the baseline is low.

System audit classes

Class	Meaning
CANLI (live)	Today the proof command returned the expected result.
AKSAK (lame)	Evidence exists, but it is short of the expectation or late.
ÖLÜ (dead)	The method is not running.
İDDİA (claim)	“It works / runs automatically” is said, but evidence is not LIVE. The heaviest class: an unproven “works” misleads more than a stopped runner.

Fall test

In every round Reaper runs three deliberate faults against itself (ageing live evidence by 9 days → DEAD; dead method + “automatic” claim → CLAIM; hanging command or malformed JSON → UNMEASURED); if one goes uncaught, that day’s verdict is void and the page says “DENETÇİ DÜŞTÜ” (the auditor fell).

Nobody is exempt — Yusuf included

The method list also has a row for “pending decisions and promises made”. The age (days) of each item is measured: 0–2 days CANLI, 3–7 days AKSAK, older than 7 days ÖLÜ. This page carries one line: “Yusuf · pending N · oldest M days”, with no private or money content; private and money items are counted by age only and their titles are not written.

Limits

Reaper does not write code, change code, delete or shut down services. It only measures, classifies and proposes; it writes the site and the board. Private matters, patient data and money are outside the audit scope and carry no numbers.

Status today (Day-1 baseline measurement · reaper_denetci.py)

Day-1 honest finding summary: (1) hasat-zincir-otomatik → CLAIM (İDDİA): in the a→b→c chain only (a) the verdict log is live; no nightly automatic LoRA scheduler is loaded (the 14 Oct N=96 experiment is verified LIVE as a standalone record). (2) qajan-kapsam → CLAIM (İDDİA): QAJAN runs daily (LIVE, 8 hypotheses, 0 PASSED out of 63 verdicts → yield LAME) but tests only trading hypotheses. (3) yusuf-bekleyen → DEAD (ÖLÜ): of 22 open items (18 age-measured), 17 are >7 days old (oldest 22.6 days).

Tables are read from /data/reaper_karne.json; the 6-hour audit cycle appends rows to the scorecard.

Links

• data/reaper_karne.json — machine-readable Reaper scorecard
• Digital smile design: 3D cephalometry, locked n=32 measurement & AI preview method
• Context-Blind Auditor (Reaper) & Falsifiability Gate: Auditing 360/394 Citations Across 10 Models
• Event-Driven Task Bus (task_bus.jsonl): 2,140+ Events & Zero-Polling Coordination
• Applied Spark Multi-Voice Protocol & LLM Red Team Workflow: 420 Audited Citations & 25 Fabrications Eliminated
• Agent Self-Identity (Δscore = 0.0000), Identity Freeze Protocol & Three-Family Architecture
• QAJAN 6-Gate Hypothesis Factory: Preregistration, Dimson/Epps Orthogonalisation & Falsification
• Digital Dentistry: What Is and What Should Be — Reaper verified references (§13)
• Autonomy: methods and live measurements
• Spark: 7-step thesis production procedure
• Machine-readable fact files (/bot/) · /bot/yontemler.md · /bot/olgular.json
• About the author — Yusuf Aziz