steinborn.dev
← all projects

LibrisVault

A self-hosted intake pipeline for everything you read - PDFs, articles, videos, screenshots and notes turned into linked, cited wiki pages that stay plain files on disk.

TypeScript · React · Node · Fastify · SQLite · Claude Agent SDK
View source
PolyForm Noncommercial · free to self-host, commercial use by arrangement
2,893
automated tests
115
API endpoints
17
kinds of agent run
five months of one vault, April to September · dots are pages, lines are wikilinks · colour by domain, size by link count · 20 domains · 1,088 entries · labels hidden
01 · The problem

A folder full of PDFs is a graveyard.

Knowledge arrives as PDFs, office documents, web articles, images, and videos. Saving them somewhere is easy; what has value is the connected form - pages that link related ideas, cite their sources, and stay findable years later. Doing that by hand does not scale past the first busy week.

LibrisVault automates the whole path, across nearly any input channel: a watched folder, drag-and-drop, or anything sent to its Telegram bot from your phone - PDFs, office documents, blog posts and web links, YouTube videos, screenshots, plain notes. Everything is preprocessed and handed to sandboxed Claude Agent SDK sessions that write linked, cited wiki pages into an Obsidian vault. It all runs on your own machine - the vault stays a plain git repository on disk.

A connected vault raises a second problem, because now it knows what it does not know. Its pages end in open questions, point at pages nobody has written yet, and stop short where a stub stands in for an article. Chasing that by hand does not scale either. So LibrisVault also has the Fellows: standing research agents, semi-autonomous Claude Agent SDK instances that each hold a subject and a budget, work at night from what the vault itself says is unanswered, and leave their proposals and results for the morning, where you decide what runs and what does not. The intake fills the vault; the Fellows keep it moving.

The vault format and its ingest skill are claude-obsidian, an MIT-licensed project by AgriciDaniel, used unmodified. LibrisVault is the service layer around it - automated intake, sandboxed agent runs, a job queue, the dashboard - and it extends what the vault holds through documented extension points rather than by editing it: a registry that binds every page to one field of knowledge from a closed list, and a set of rules carried into every run, from English-only content to page hygiene and entity notability.

02 · The system
intake folder · drag & drop · Telegram preprocessing poppler/OCR · pandoc · yt-dlp agent sessions Claude Agent SDK · sandboxed bubblewrap under both: converters with no network and one file in, sessions with no network during ingest converter output fenced as data before an agent reads it · every writing run a git commit Fellows, at night: sessions planned from the vault's own gaps · these reach the web Obsidian vault linked, cited pages · git dashboard library · graph · research
binds 127.0.0.1 by default - what leaves the box: Anthropic, the web for research runs, Telegram, open-access lookups

A Fastify backend over SQLite with a React dashboard, 134 source files on the server side and 126 on the web side. An ingest is a job in a queue. A job is an agent session. An agent session is a git commit. Nothing else in the design matters as much as that last equivalence: it is what makes every automated change revertible with a tool the user already trusts. A Fellow's night is a sequence of exactly those: each of its runs a session and a commit, revertible one by one.

03 · Why it holds up

OS-level sandboxing around every agent run

Every agent run happens inside a bubblewrap sandbox, the second of two on the way in: the converters have their own. Writes are confined to the vault directory, and inside it a write guard keeps the agent to the knowledge pages and away from the vault's own machinery. An ingest run has no network at all; a research run reaches the web, a Fellow's planning run does not. Every writing run is one git commit, so anything an agent did is revertible with a tool you already trust.

Live probes re-verify every guard

Measured that the SDK's permission callback effectively never fires for the writes that matter - so the security boundary sits at OS level, runs fail loudly without a sandbox, and live probes re-verify the guards after every SDK upgrade.

A hostile document meets a jailed parser before it meets an agent

Every converter runs under bubblewrap with no network, no home directory, one input file and one output directory. What it produces is fenced as data before an agent reads it: material to read, never instructions to follow, and text inside it that addresses an assistant is reported rather than passed on. Quotes are checked against the text the run actually read, and a page a paywall blocked is rescued from a legal open copy that the page then names.

Domain decisions belong to the operator

Every page is filed under exactly one domain from a closed registry the agent reads as data. Inventing a key is not an option, because that page is edited by a person and never by an ingest run. What fits nothing goes to an unassigned sentinel. That pile is where proposals come from: a deterministic pass surfaces any tag recurring across five or more of them. An optional agent review only judges what it is handed. The operator decides, then a backfill re-files the backlog. A domain that outgrows its shelf is split the same way: a deterministic proposal drawn from the domain's own links, decided shelf by shelf, written in one commit and undone by one revert.

Retrieval that was measured before it was believed

Finding the right page in a vault of several hundred is a search problem. The lexical arm is plain BM25 over page chunks, each prefixed with a line of context drawn from its page's own frontmatter; the vault's scripts build it, with no model and no network involved. The honest way to tune it is a fixed question set: 35 questions with a known answer page. At 947 pages that arm alone puts the right page in the top five for 33 of them and first for 23. An embedding rerank was built, measured against the same set, and switched off by default, because it lost on both counts.

Seventeen kinds of run

Linting, repair, deduplication, tag fixes, domain backfill and review, the hot-cache rewrite and the retrieval index are each a named run with its own prompt, its own outcome and its own entry in the history. A standing defect gets a repair run bound to the pages of its findings, enforced by the same hook as every other policy, and a domain split can ask for names for its new shelves. With the research agents on, a Fellow's planning run, its step, its deepening and the morning recap are four more, held to the same record.

04 · What it produces

A vault that stays a vault. Plain markdown in a git repository on disk, readable in Obsidian with the service switched off. There is no proprietary store and nothing to migrate out of.

A dashboard for the machine room. Intake, job status with live updates over server-sent events, a catalog of every page, and research over the web and the vault, each answer citing the pages it came from.

A shape you can see. The whole vault renders as one interactive wikilink graph, laid out by a web worker and drawn on canvas. Four overlays read it: Landmarks names a domain's most-linked pages and a reading order through them, Areas tints each detected community and captions it with the tags that set it apart, Spotlight isolates one community, and Bridges draws the links that cross between them, with their direction. The same layout colours by field of knowledge, page type, authority by inbound links or recency, and filters orphans and stubs. Sources are not a pile. Each one sits beside the concepts and entities it supports, so what a cluster of knowledge actually rests on is visible rather than asserted.

An order that survives growth. Every page carries exactly one field of knowledge from a closed, human-governed set. The catalog browses along that field, along page type, recency and backlink count. Orphans and stubs are filters rather than surprises. The gaps - links pointing at pages nobody has written yet - are listed rather than quietly dropped. What the validator finds stands as one counted row per defect, and every rule has a way out: a deterministic repair applied only to pages that still match what was approved, a bound agent run, or an accepted reason. A vault of a thousand pages stays navigable in the same shape as one of fifty.

Research assistants. A Fellow is an agent built on the Claude Agent SDK, with a subject and a configurable degree of autonomy. It proposes runs, each with its reasoning and an estimated cost, and works through them on a serial queue. The morning recap holds the results and the next steps it proposes, which you accept or veto. Its budget is a configurable share of your subscription, and every page a Fellow writes is an ordinary commit that git revert undoes like anything else. The Fellows are functional and still under close observation, so they ship switched off by default.

A library you can walk through. With the agents on, the vault renders as a room: one shelf per domain, filled to the page count it holds; one desk per Fellow, its screen lit while it works; a book cart where the maintenance runs work and the ingest queue's parcels wait; and a pinboard of every open question the vault's pages left behind, from which a question is handed to research or struck through on its own page.

One command to stand it all up. A setup script installs Node, the sandbox and preprocessing toolchain, the vault and the service, and leaves exactly one step for the browser: connecting an Anthropic account. Windows without WSL gets a PowerShell path that installs it first.

05 · Scope

It is a service layer, not a vault format. The vault and its ingest skill are claude-obsidian's, used unmodified. What this project owns is everything that makes them run unattended: intake from three channels, the job queue, the sandboxed runner, the converter jail and the fence around what it produces, the domain governance, seventeen kinds of run, the research agents, the demo guard, the dashboard.

The sandbox is the boundary, not the callback. The Agent SDK's permission callback was measured to fire zero times, so the guarantee cannot sit there. It sits in the sandbox the runner configures: writes confined to the vault root, no web access in an ingest run, with the web reserved for the research flow and a Fellow's own runs; a Fellow's planning run is confined to what the vault already holds. If the sandbox cannot be created, the run fails rather than proceeding. Unsandboxed commands are switched off as well, because the shell tool ships a parameter that would otherwise let a command opt out. Without that one setting the sandbox would be decorative.

What is on this page is what ships today, the research agents and the library included since September 2026, behind one flag that is off by default. The next line of work is private for now.

The licence is deliberately not open source. PolyForm Noncommercial: free to self-host, commercial use by arrangement.

06 · Notes from this project
2026-09-10 ai-engineering

One guard beats fifty route checks

How LibrisVault's read-only demo mode is a single request hook instead of per-route logic. The tests target the boundary itself rather than a list of routes.

2026-08-31 ai-engineering

The deploy that only half happened

A frontend rebuild under a running service left the dashboard full of 404s. A small debugging story about boot-time snapshots.

2026-08-28 ai-engineering

The permission callback that never fired

Why LibrisVault trusts the OS sandbox instead of the SDK permission layer. The measurement that settled it.

2026-08-07 ai-engineering

How a page becomes a vector

Two encodings of the same text, sparse and dense, because each is blind where the other sees. Which one a corpus needs is a measurement, not a principle.

2026-07-24 ai-engineering

The agent doesn't get to invent categories

How LibrisVault prevents ontology drift in an agent-maintained knowledge base with closed sets, explicit proposals, and a human veto.

Browse the vault.

Open demo
read-only demo · a synthetic vault of 1,400 pages, shaped like a real one and rebuilt nightly · the research runs in it are real