Process over outcome: gates for self-improving coding loops
Frontier labs are publishing the same uncomfortable finding from the training side: when agents have a large action space, outcome reward alone stops being a sufficient metric. They mine git history, scrape reference solutions, and optimize the score instead of the work. The same failure mode shows up in day-to-day coding agents—just with different costumes: skipped tests, edited metrics, force-pushed “fixes,” and “done” claims without receipts.
ThumbGate is not a training stack. It is a self-improving firewall for the execution boundary. This post is about why that distinction matters when agents start improving themselves.
What “process over outcome” means at the tool call
Outcome-only control asks: did the benchmark pass? Did the PR open? Did the agent say it finished? Process control asks: what path did it take, what evidence is required, and is this a known-bad promotion pattern?
Concrete process checks ThumbGate is built to support:
- Holdout / proof gates — do not promote a claimed improvement without required verification evidence.
- Known-bad promotions — flag patterns like metric editing, missing holdouts, or hidden failed runs (see the Autoresearch safety pack).
- PreToolUse decisions — ALLOW / WARN / DENY before shell, file, git, deploy, or API actions execute.
- Reviewable lessons — thumbs become local memory; repeated negative patterns can promote into gates; stale auto-promoted gates expire.
Self-improving firewall ≠ self-improving weights
Poolside-style research posts on reward hacking are about training loops. Our product claim is narrower and operational:
- We do not retrain your model from thumbs.
- We do store feedback as local lessons, re-rank relevant lessons for the next action, and promote repeated failures into prevention gates.
- We do hard-block detected secret exfiltration and gate-bypass attempts by default; most other matches warn unless strict enforcement is on.
That is how the control layer improves over time without pretending every free install is a perfect firewall—or that every agent claim is honest.
What to do this week
- Install local checks:
npx thumbgate init - Pick one self-improving or “autonomous fix” workflow that already burned you.
- Require process evidence (tests, holdout, screenshot, CI) before the agent can claim done.
- If you want a managed boundary for that one workflow, use the $499 Enterprise Workflow Gate.