← Selected projects
Knowledge engine for codebasesPrototype · 2026

Project Wiki

A knowledge engine for projects spread across several repositories and tools, starting with git and Notion. It turns them into a graph of concepts, keeps a link from every concept back to the exact code it came from, and updates itself as the sources change. Coding agents read it over MCP. I read it in a browser or just ask it.

01 / The problem

Everyone relearns the same project by reading the same files again, including the agents.

The knowledge about a project is usually spread out: code in a few repositories, decisions and specs in Notion, and more in whatever other tools the team uses. The understanding of how it all fits together doesn't live in any of them. A person rebuilds it by reading the source. A coding agent rebuilds it from scratch every session, pulling the same files into its context window and paying for the same reading again. Generated API docs just mirror the folder structure and go out of date with it, and full text search gives you text, not understanding. Some things aren't written down in any single file at all, like a wire contract that two repositories both depend on. So I wanted a knowledge base organised by concept instead of by file. Every concept should point back to the exact code it came from, keep up as the code changes, and be readable by both people and agents without going back to the raw sources.

02 / Architecture

One core, with the CLI, the API, and the MCP server as thin layers on top.

The core follows a hexagonal architecture with four layers, where dependencies only point inward. Two import-linter rules enforce this, and one of them lists every framework, database driver, and model SDK the inner layers aren't allowed to import. Sources come in through connectors, git and Notion so far, and adding a new kind of source means writing a new connector rather than changing the core. Everything is stored in Postgres as stable references. A retrieval index and a batched distillation step turn them into a concept graph where every concept links back to its source. An organising step groups concepts into domains and connects them across sources, and anything contested goes to a review queue for a person to decide. The CLI, the API, and the MCP server all call the same use cases, and maintenance reruns a small part of the same pipeline whenever a source changes.

03 / Engineering

The decisions behind the system.

01

Batched, resumable, self-correcting distillation pipeline

The initial build is slow, expensive, and likely to fail somewhere. A single model call over a whole repository hits the output limit, loses everything if it fails, and spreads the model's attention too thin. So the pipeline splits the work into small batches, and a failure halfway through only costs the batch that failed.

Implementation details
  • File-aware packing groups a file's units together up to a token budget and sub-splits a single oversized unit, so the model always distils a symbol alongside its local context
  • Each batch commits in its own transaction, so partial progress survives a later failure and re-running init skips batches already done; a failed batch is isolated, recorded with its failure kind, and the run finishes the rest
  • Retry depends on why the batch failed: a truncated reply is re-planned into smaller sub-batches with the budget halved per attempt, a transient error backs off, invalid JSON retries once and then stays visibly failed, capped at three attempts
  • A batch is identified by its packing ordinal, not by the set of coordinates it covers, because sub-split pieces share one parent coordinate and keying by coordinate re-distilled a whole oversized file once per piece
Distillation pipeline / Three-part walkthrough01 / 03
01Packing — a file's units travel together up to a token budget, and an oversized unit is sub-split.
02

Provenance and incremental maintenance from git

Keeping the wiki up to date without rebuilding it is the hardest part. Maintenance works from git, only looks at what a change touched, and never removes knowledge unless it can show it's gone.

Implementation details
  • What changed is computed from git directly, never from a webhook payload, because a pull-request payload carries no file list and a push payload truncates past 20 commits
  • Only the touched documents are re-ingested, and only the concepts whose provenance points at those coordinates are re-examined, through the same batch path init uses, so the two cannot drift
  • Evidence is retired only when its content is provably gone, asked of the coordinate's identity rather than a line range; a concept left with no live evidence is proposed for deprecation, never deleted
  • A concept that its own changed source now contradicts is corrected, quoting the document's new sentence, rather than retired; a source that merely stops mentioning a concept is never treated as a verdict
Maintenance / A merged pull request through the loop01 / 01
01Maintenance flow — a git diff selects documents, provenance selects concepts, each is corrected, deprecation-proposed, or queued.
03

Making correctness something the build checks

After a green build lied to me for months, I made the build check both the code and whether the documentation still matches it.

Implementation details
  • Strict hexagonal, four rings, enforced by two import-linter contracts, one of them a forbidden-imports list naming every framework, driver, and model SDK the domain and application rings may not import
  • A test that stubs the seam it is testing proves nothing: browser-driven init passed its gate for months while never working, because the endpoint flushed instead of committing and the fake background task never touched the database; every scheduling site now commits before scheduling and a test drives real scheduled work against Postgres
  • A requirement counts as verified only through a passing test tagged with its ID: a pytest plugin writes a verification map and a generator rolls it into the status page, which is never hand-edited
  • Four documentation-honesty gates run inside the single build command: the migration head must match the ORM metadata, .env.example must document every settings field, the generated agent context must match its source, and every internal doc link and repository path must resolve
Verification / The build gate01 / 01
01make check — six stages, with the four documentation-honesty guards running inside the test stage.

04 / Product walkthrough

PROJECT WIKI / RUNNING APP03 VIEWS
01Browse — a concept reads as a short chapter, with its provenance and relationships beside it.

05 / How I work

Plan first, build in small increments, verify against reality.

I built Project Wiki on my own. What kept a project this size on track was mostly how I worked around the code: decisions written down before building, requirements that had to be proven by tests, and regular checks against the running product, not just the test suite.

Plan first

The first days went into planning with no production code: scope, architecture, data model, and the order to build in. Every significant decision got its own short record with the reasoning behind it, about 50 in total. Open questions stayed on a list instead of being decided too early.

Requirements up front, proven by tests

I wrote the requirements before building, as a chain: use cases, then requirements with stable IDs, then work items that moved through plan, implement, verify, and done. Every test is tagged with the requirements it proves, and a requirement only counts as verified when one of those tests passes. That's what "68 of 69" means further down: a number generated from the test run, not one I wrote by hand.

Small increments, real interfaces early

The work ran in seven phases, each with its own plan, split into small commits that reference the work items they deliver. Anything deferred got a real interface straight away, so later work could plug in without rewriting what was already there. Months later, a rollback feature took about two hours because the data model had been built for it from the start.

One place per fact

Project status lives in exactly three places: a generated page showing what's verified, a page for where the build stands, and an ordered roadmap. I added that rule after a review found documents contradicting each other because the same fact was written in several places.

Verify against reality

After each phase I ran the product end to end in a real browser, against two test repositories with planted behaviours and an answer key written in advance. Findings became numbered fixes in later phases, and known limitations went into their own catalogue instead of being hidden. That habit started with a feature that passed every test and had never worked.

06 / Evaluation

Tested against the requirements, plus two full runs with real models.

The gate is the single build command: roughly 775 backend tests and 127 frontend tests, plus ruff, strict pyright, and the import-linter dependency rule, all green. 68 of 69 registered requirements are verified by a requirement-tagged test. The 69th, engine-proposed reading paths, isn't built yet, and the status page says so, so the count includes every requirement the project has.

The proof runs drove guided init and the full maintenance loop end to end against a two-repository fixture pair with real Anthropic and OpenAI models, on 1 and 14 August 2026. Each run was scored against an assessment key written beforehand. In the maintenance run, 8 of 9 checks passed, 1 was unevaluable, and 1 surfaced a real defect: a concept left asserting what its source no longer said, because the reconciliation judge was scoped by source reference rather than by document. That was fixed. Measured cost for both fixtures was about $1.10.

What I haven't tested is how it behaves at the size of a real project: how big the review queue gets, whether coverage skews toward prose, and where the single process maintenance loop gives out. That's the next step: running it on my own platform and fixing what comes up, instead of guessing now.

68 / 69requirements verified by a requirement-tagged test, on the generated status page
8 / 9maintenance checks passing in the 14 August 2026 proof run, scored against a key written beforehand
~775 + 127backend tests and frontend tests in the green build gate

07 / Outcome

Not finished, but tested, and now going into real use on my own projects.

The core parts are built and tested: a pipeline that can resume and recover when model calls fail, maintenance that updates the wiki from git without dropping anything it can't show is gone, and a build that fails if the documentation stops matching the code. It isn't finished. Next I'm using it on my own platform to see how it holds up in real work, and improving it based on that.

Delivered

The full engine (ingest, index, distil, merge, organise, review, light divergence) and all four v1 surfaces (guided init and review queue, visual browser, MCP server, in-product chat with server-side history), plus self-maintenance from merged pull requests.

Validated

68 of 69 requirements green by requirement-tagged tests, plus two end-to-end proof runs against real Anthropic and OpenAI models scored against assessment keys written before each run; one run caught and led to the fix of a real reconciliation-scope defect.

Next

I'm putting it to work on my own multi-repo platform and improving it based on what real use shows, like how big the review queue gets, whether coverage leans toward prose, and where single-process maintenance hits its limits. The one unbuilt requirement, engine-proposed reading paths, comes first.

Next projectContextual Outreach