Statewave Guide — Day 2: Unknown is better than wrong

The benchmark expected one route. The indexer found another. Only one of them was right

UpdatedOctober 5, 2026

Statewave Guide — Day 2: Unknown is better than wrong

Statewave is an open-source, self-hosted memory runtime for AI agents. Beyond retrieval, it adds governance: access policies, sensitivity labels, tamper-evident audit receipts, and full source traceability. Apache 2.0. → statewave.ai

Part of seriesStatewave Guide — Build Log

Built 26 Aug 2026 · written up 5 Oct 2026

Yesterday's question was:

Can we follow one real user action all the way from a button to the backend — without guessing?

Yes.

But getting there was a useful reminder that even your benchmark can lie to you.

The target looked simple:

New Client
    ↓
openCreateClient()
    ↓
NewClientDialog
    ↓
submitClient()
    ↓
clientService.create()
    ↓
POST /api/clients

Frontend to backend.

One connected application graph.

No docs.

No manually maintained knowledge base.

Easy, right? 😅

Our first quality run had other ideas.

invokes precision: 39%

API detection: 19%

calls_api: 0%

Beautiful.

At least the tests were honest. 😂

Then something more interesting happened.

We found what looked like an indexer bug.

The benchmark expected:

GET /clients

The indexer produced:

GET /api/clients

My first instinct was to normalize them.

Turns out that would have been wrong.

One adversarial fixture had another router with /clients whose mount point could not be determined.

Strip /api from the known endpoint and suddenly two different routes become the same thing.

Congratulations.

We've manufactured product knowledge.

So instead of fixing the indexer…

we fixed the benchmark.

That became another rule:

A benchmark isn't truth. It's an executable statement of what we currently believe truth is.

If the evidence proves the benchmark wrong, the benchmark loses.

We also introduced partial identities:

api:GET:?/clients

Meaning:

"I know this route exists. I do not know where it is mounted."

No guessing.

No helpful little inference quietly connecting it to the wrong backend handler.

After reconciliation:

API precision: 100% invokes: 100% / 100% uses_hook: 100% precision uses_service: 100% precision submits_to: 100% renders: 100% validates_with: 100% calls_api: 83% precision / 100% recall

Across the graph:

0 relationships without evidence. 0 invalid confidence values. 0 unsupported inferences pretending to be facts. 0 / 37 adversarial trap violations.

437 tests green.

We also discovered our previous "deterministic output" wasn't actually deterministic.

Same repository.

Different working directory.

Different graph.

😬

Eight working directories later:

one hash.

Fixed.

One of the most important files we added might actually be refusals.md.

A catalogue of things the system intentionally refuses to infer.

One real-world pattern would have generated dozens of fake API endpoints if we'd taken the obvious shortcut.

So we didn't.

That's increasingly becoming the philosophy behind Statewave Guide:

Missing knowledge is fixable. Wrong knowledge destroys trust.

So yes:

We can follow a real user action all the way to the backend without guessing.

Now comes the next problem.

We know increasingly well what happened.

But not necessarily what those facts mean to the user.

Statewave Guide — Day 2.

Next question:

Can a model add product meaning to this graph without being allowed to invent product truth?

This day in the repo: PR #2 · Previous: Day 1 — The DOM is not the product

Statewave Guide is built on Statewave — an open-source, self-hosted memory runtime for AI agents. Governance and provenance, not just retrieval. Apache 2.0.

→ statewave.ai · github.com/smaramwbc/statewave

Every post in this series: The Statewave Guide Journey Index

Originally published on the Statewave blog.

#testing #typescript #ai #build-in-public

4 views

Comments (1)

Join the discussion

indiainfranotes11d ago

Unknown beats wrong is the right default for anything with provenance in the name. A trace that says "no evidence for this hop" is worth more than one that fills the gap with a plausible guess, because the guess gets copied into every later report. Do you store why a hop is unknown, like missing span versus dropped header, so the gaps themselves become queryable?

iin1006h07

Statewave11d ago

Partly, and your version is the better one.

The shape of the unknown is stored and queryable: a partial identity like api:GET:?/clients says "route known, mount point unknown", so every deliberately incomplete edge can be selected. What is not stored is the reason for that particular gap. The reasons live in refusals.md as a catalogue of patterns the indexer refuses to infer, which is documentation rather than data on the node.

That split is the weak part, and a reason code on the instance, drawn from the same catalogue, is the fix. Gaps then become countable per reason instead of visible one at a time, and the series gets an honest metric: which gaps shrank because the indexer got better, and which stayed because the information genuinely is not in the code.

On the copying: that asymmetry is why the file exists at all. A gap costs one lookup. A fabricated edge costs trust in the whole graph, and it is the one thing nobody can audit later, because it looks exactly like the edges that were earned.

Statewave Guide — Build Log

Part 3 of 5

Building a help chat that understands a product by reading its codebase. One build day per post, in order, including the dead ends. The code is ahead of the story: everything here already happened, and the repository shows how it ended.

Up next