Catch what long technical files hide.
EU AI Act pre-screening for Articles 9 to 15, one Article at a time, with NVIDIA Nemotron on Nebius.
Violations falsely cleared when the violating section comes first in a five-section file. Whole-file Nemotron 3 Nano vs the Screener, 73 cases, median of three runs.
Cleared by one whole-file Nemotron 3 Ultra call: a statistical tie with the Screener, which wrongly rejects more compliant files (15% vs 6%).
Scored runs in a pre-registered re-test on 120 expert-labelled AIReg-Bench files: held-out split, three repeats, bootstrap CIs.
Built from what broke
We tested LLM auditors before building one, then re-tested our own design under a protocol fixed in advance. Each step answers a failure we measured.
Judge one Article at a time
With a generic prompt, compliant sections after a violation dilute it. The Screener asks about one named Article and counts a disclosed gap against compliance. Our re-test showed this prompt, not splitting the file, closes most of the gap.
Sample three times
Nemotron 3 Nano judges each Article three times in parallel. A single sample flips between identical runs and over-rejects; the median of three cut false rejections from 23% to 13%.
Escalate disagreement
When the samples disagree, Nemotron 3 Ultra decides. If Nano and Ultra still disagree, the Article goes to a human instead of a guess.
Flag claims, check quotes
Notes to the reviewer and unsupported compliance claims are flagged and ignored. Evidence quotes are shown only if they appear in your text.
Screen a file
Paste documentation, ideally with an "Article N" heading per section so evidence is easier to check, or load an example. Takes about 10 to 20 seconds. A screening aid for reviewers, not legal advice.
What the research found
Long files fool a generic judge
With the violating section first in a five-section file, whole-file judges using AIReg-Bench's prompt falsely cleared 9.6% to 52% of non-compliant documents. A targeted prompt brought that to 0 to 4%, with or without splitting the file.
Fake approvals did not work
"Certified compliant" and "pre-approved by the AI Office" text cleared no non-compliant document.
Rewrite attacks rarely did
Claude Sonnet rewrote violating files to look compliant. It succeeded once.
Experts disagree too
The three legal annotators agree with each other at Krippendorff's alpha 0.657, so the Screener sends close calls to a person rather than pretending to certainty.