No LLM in the gate: blocking AI-agent mistakes before they execute

2026-08-19 · engineering

A prompt is advice. A PreToolUse hook is enforcement. This post is about why we put the gate at the tool-call boundary instead of anywhere else, why there is no language model on the enforcement path, and how a thumbs-down becomes a rule that blocks the next agent — including the failure modes we hit shipping it.

The problem: one wrong tool call is enough

An AI coding agent does not fail the way a human does. It fails fast, and it repeats. The same agent that shipped a good PR at 2pm will, at 4pm, run rm -rf on the wrong directory, force-push over main, echo an API key into an outbound request, or drop a production table — because the token stream that produced the good action and the catastrophic one look identical until the side effect lands.

Three defenses are commonly proposed. Each fails for a structural reason:

• Better prompts / rules files. Advisory. The model can read "never force-push to main" and force-push to main in the same session. A rule the agent can ignore is not a control.
• Post-hoc review. Too late. By the time a human reads the diff, the table is dropped and the key is in someone's logs. Review governs what merges, not what executes.
• An LLM judge in the loop. Reintroduces the exact non-determinism you are trying to remove. A gate that occasionally hallucinates "allow" on a destructive command is not a gate — it is a suggestion with latency.

The insight: gate the tool call, deterministically, before it runs

Every capable agent harness — Claude Code, Cursor, Codex, Gemini CLI, Amp, Cline, OpenCode — exposes a PreToolUse boundary: the moment after the model has decided to run a tool and before the tool runs. That is the only place that sees the concrete, fully-resolved action (bash, SQL, file write, HTTP fetch, MCP call) with time left to stop it.

ThumbGate lives in that hook. When the agent proposes a tool call, the hook receives it, evaluates it against a rule set, and returns one of three verdicts before the harness executes anything:

The evaluation is pure pattern matching against a deterministic rule set. No model call. The same input always produces the same verdict, in single-digit milliseconds, with an audit line written either way. That property — determinism — is the whole point. An enforcement layer you cannot predict is one you cannot trust with the destructive cases.

The gate is not smarter than the agent. It does not need to be. It needs to be certain, fast, and always there.

Where the rules come from: a thumbs-down becomes a rule

Determinism forces the real question: where does the rule corpus come from? Hand-writing every bad pattern does not scale, and it does not learn from your codebase. Our answer is a feedback loop that turns operator judgment into enforcement:

observe → capture (👎 + context) → distill to a lesson
        → candidate rule → promote (count thresholds) → enforce

When an agent does something wrong, the operator captures a thumbs-down with one line of context. That becomes a history-aware lesson. Matching negative occurrences promote into the Infrastructure Firewall as PreToolUse gates on a simple count path: after one negative occurrence the pattern becomes a warn gate; after three it upgrades to block. That is intentional and deterministic — not a Thompson-sampling precision/recall gate. Live gates also do not guarantee lesson provenance (sourceLessonId / equivalent IDs are not required on the persisted gate), so operators should treat the rule text and occurrence count as the source of truth, not a guaranteed back-link to the originating lesson.

The firewall improves from operating, not from retraining. To be explicit, because it is the most common misunderstanding: ThumbGate does not update model weights. It intercepts tool calls at runtime. The learning happens in the rule corpus, not the model.

What we got wrong: the false-positive tax

A matcher that scans command text for rm -rf or a secret pattern is fast and predictable. It is also lexical, which means it fires on text that merely mentions the dangerous thing. In production we hit exactly this, and we filed the bugs against ourselves:

• A guard scoped to financial actions fired on ordinary prose, because an everyday English word carries both a git meaning and a commercial one and appeared here in its git sense.
• A permission guard blocked a documentation write that quoted a shell command as an example.
• A blast-radius evaluator counted a shared working tree's dirty files instead of the single staged file, and false-blocked a clean commit.

The lesson is not "lexical matching is wrong." It is that a gate must scan the action, not the narration of the action — the tool plus its resolved arguments, never surrounding prose or an evidence field. Scoping matchers to real side effects is ongoing work, tracked in the open, and it is the honest cost of choosing determinism over an LLM judge. We would make the same trade again: a gate that occasionally over-blocks and tells you exactly why beats a gate that silently, unpredictably lets the destructive case through.

What this buys you

• The catastrophic classes are stopped before execution, not flagged after.
• Every decision is reproducible and logged — a record you could hand an auditor, not a vibe.
• The rule set is yours — grown from your agents' actual mistakes, portable across every MCP-capable harness you run.
• No cloud on the enforcement path. The gate runs locally; the block does not depend on a network round-trip.

The pattern generalizes beyond coding agents, but coding agents are where the blast radius is highest and the tool-call boundary is cleanest, so that is where we started. If you run agents against anything you would be unhappy to lose, the gate belongs before the tool call.

npx thumbgate init