3 Verification Patterns That Make AI-Generated Code Trustworthy

The hardest part of autonomous coding isn't generating the code. Any agent can generate code.

The hardest part is knowing, at 2 AM when reviewing the overnight PR, that the code is correct — not just syntactically valid, but behaviorally correct, properly tested, and safe to merge.

This is the verification problem. And most AI coding tools handwave it.

Here are three concrete verification patterns that turn AI-generated code into something you can actually trust — each one a real engineering strategy, not a prompt-engineering trick.

Pattern 1: Executable Contracts

The simplest verification pattern is also the most neglected: write a contract before you write the implementation, and make the agent verify against it.

A contract isn't a spec. A spec says "build a user authentication system." A contract says:

# contract_test.py
def test_user_can_register():
    auth = AuthSystem(db_url=TEST_DB_URL)
    user = auth.register("alice", "secure-password-123")
    assert user.id is not None
    assert user.username == "alice"
    assert auth.authenticate("alice", "secure-password-123") is True
    assert auth.authenticate("alice", "wrong-password") is False

def test_duplicate_registration_fails():
    auth = AuthSystem(db_url=TEST_DB_URL)
    auth.register("bob", "password1")
    with pytest.raises(DuplicateUserError):
        auth.register("bob", "password2")

This is the specification. The agent's job is to make these tests pass — not to interpret what "build authentication" means. Distinction: specification by executable behavior, not prose.

Why this works: Tests are the ground truth. The agent can run them. The agent can iterate on failures. You don't need to read the agent's code to know whether it worked — the green bar tells you.

Ralph Workflow makes this the default pattern: the ralph run command accepts a spec file that can include test stubs, and the verify phase runs pytest against them. If tests fail, the fix loop continues. If they pass, you get a green PR.

Pattern 2: Diff-Against-Baseline

Most AI coding failures aren't wrong code — they're missing something that used to work. The agent adds a feature, passes its own tests, but breaks three other things.

The diff-against-baseline pattern catches this:

1. Run the full test suite before the agent starts (baseline)
2. Run it again after the agent finishes (candidate)
3. Compare: any test that passed in baseline but failed in candidate is a regression
4. If regressions exist, revert and report — don't merge

# Baseline
pytest --json-report --json-report-file=baseline.json

# Agent runs its work...

# Candidate
pytest --json-report --json-report-file=candidate.json

# Diff
python compare_results.py baseline.json candidate.json

Why this works: It catches regressions that the agent's own scope doesn't cover. The agent tested the new feature — it didn't test the payment processor it didn't touch. The diff catches that.

Ralph Workflow's verify phase runs exactly this comparison. The checkpoint file tracks the baseline hash. If the candidate suite has regressions, the fix loop activates — the agent gets the failure output and tries again.

Pattern 3: Property-Based Gate

Unit tests check specific inputs against specific outputs. Property-based tests check invariants — properties that must hold for any input.

Property-based testing is especially useful for AI-generated code because the invariants are usually simple to state and hard to implement correctly:

# Property: sorting should be idempotent
@given(lists(integers()))
def test_sort_is_idempotent(xs):
    sorted_once = sorted(xs)
    sorted_twice = sorted(sorted_once)
    assert sorted_once == sorted_twice

# Property: round-trip encode/decode
@given(dictionaries(keys=text(), values=integers()))
def test_json_roundtrip(data):
    encoded = json.dumps(data)
    decoded = json.loads(encoded)
    assert data == decoded

# Property: no operation should lose data
@given(lists(integers(), min_size=1))
def test_filter_never_makes_things_appear(xs):
    result = my_custom_filter(xs)
    for item in result:
        assert item in xs  # filtered output is subset of input

Properties are the contracts of the system — the mathematical truths that must hold. A codebase with property tests is dramatically harder for an AI agent to silently break.

Ralph Workflow integration: The verify phase can be configured to run Hypothesis property tests as a gating step. The agent doesn't need to understand property-based testing — it just needs to make the gate green.

Why Verification Beats Review

Code review of AI output is exhausting. The agent writes 400 lines, you read 400 lines, you catch 3 issues, it fixes them, you read 380 new lines. This is not scaleable.

Verification flips the model: instead of reviewing code, you review evidence. The tests passed. The diff is zero. The property invariants hold. The contract is satisfied. You're reviewing a red/green report, not a diff.

One Thing You Can Do Tonight

Pick one module in your codebase that has solid existing tests. Write a 10-line spec for a small feature. Run it through an agent with Pattern 1 (contract tests) and Pattern 2 (diff-against-baseline). Review the verification output, not the code. See if you trust it.

Ralph Workflow is a free and open-source AI agent orchestrator that builds verification into every phase of the loop. It runs on your machine with the agents you already use — Claude Code, Codex, OpenCode. Star it on Codeberg (the primary repo), or install with pipx and run your first overnight task tonight.

Start here: your first overnight task →