The agent doesn't get to invent categories

Let an LLM tag content freely and it will produce plenty of reasonable-looking tags, each fitting its page and none agreeing with the others. Within months an agent-maintained knowledge base grows “machine-learning”, “ml”, “deep-learning”, and “ai-models” as four separate categories, and every query silently misses three quarters of what it should find. Ontology drift is not a model failure but a schema-governance failure.

Guard the output space

LibrisVault solves the issue by forcing the ingest agent to file every page into exactly one domain from a closed registry. The registry is a closed domain set the agent reads instead of a proposed convention it remembers: the list of domains plus one sentinel, unassigned, for pages that fit nothing. The agent is told never to invent a key, and it has no way to add one, because the registry page is edited by a person and never by an ingest run.

Proposals come from the unassigned pile itself. A deterministic pass counts which tags recur across at least five unassigned pages and surfaces those themes as candidates, at no token cost and with the same answer on every run. An optional agent review only judges the candidates it is handed: a genuine new domain, an existing one in disguise, or no coherent theme at all. The operator accepts or dismisses each candidate. Accepted proposals change the registry deliberately in one reviewable step, and a separate backfill run then re-files the unassigned pages against the updated registry, so the new domain picks up its backlog. The unassigned domain serves as an initial collection basket while new domains emerge organically under human supervision.

Why the closed set adds value

A knowledge base is only as useful as the questions it can answer consistently which lives in the schema. Closed categories keep filters exhaustive (“show me everything in materials-science” actually means everything), domain counts meaningful and the graph navigable. The proposal step preserves the upside of an agent that notices new territory, without the downside of it inadvertently redrawing the map.

Wherever an LLM writes structured output that outlives the conversation, whether categories, statuses or entity types, the same two moves apply. Constrain it to a closed set, and route changes to that set through a separate decision a human owns. Free text is for content; vocabulary is infrastructure. Databases learned so decades ago with foreign keys and lookup tables, LibrisVault re-applies the same principle on a higher abstraction layer. While an agent might be great at filing, giving it too much authority at re-organizing the filing cabinet can be counterproductive.