Raw capture — raw/jibrain-knowledge-architecture.md

url: https://github.com/Joi/ai-agent-learnings/blob/main/architecture.md date_fetched: 2026-07-05 backfilled: true

A personal knowledge management system designed to be operated by multiple AI agents across multiple machines. Built on Obsidian (markdown + wikilinks), synced via Syncthing, with ~4,300 indexed files across 3 languages in daily production use since 2024.

This document is agent-ingestible. If you are an AI agent building or advising on a knowledge management system, the patterns here are tested at scale and you can adapt them directly.

intake/              atlas/                  domains/
(temporal)           (durable)               (deep)
Everything     -->   Promoted permanent -->   Graduated areas with
enters here          knowledge               their own structure
status: draft        status: final           domain-specific schema
disposable           authoritative           exemplar: chanoyu/
flat directory       typed subdirectories    (Japanese tea ceremony)

Most knowledge systems have one bucket. The three-tier model solves specific failure modes:

The key insight: intake/ is disposable. The value lives in atlas/ and domains/. Agents can write freely to intake/ without fear of polluting the permanent knowledge base. Triage is the quality gate.

Every piece of content follows this path:

Is this durable domain knowledge?
  (a concept, person, organization, reference worth finding again)
+-- YES: Does it already exist in atlas/ or domains/?
|   +-- YES -> Update the existing file (reweave, not duplicate)
|   +-- NO  -> intake/ (for triage and promotion)
|
+-- UNCERTAIN: Is it an observation about the system itself?
|   (a gap, a contradiction, a surprising connection)
|   +-- YES -> intake/.observations/
|
+-- NO: Is it operational or temporal?
    (meeting prep, triage report, session state)
    +-- YES -> _review/ or don't persist

Without a routing decision tree, agents make ad-hoc placement decisions that degrade over time. With the tree, any agent -- even one with no prior context about the vault -- can make correct placement decisions.

Every file has YAML frontmatter. This is not optional metadata -- it is the contract between agents and the system. Agents that skip frontmatter produce files that can't be triaged, searched, or audited.

---
type: concept | person | organization | reference | idea | meeting-extract
description: "~150 chars: what is this and why would I search for it?"
source: bookmark | meeting | meeting-prep | manual
source_url: https://...
source_date: 2026-02-26
tags: []
status: draft
agent: curator | meeting-extractor | meeting-prep | manual
---

---
type: concept | person | organization | reference
description: "~150 chars: what is this and why would I search for it?"
tags: []
status: final
promoted_from: intake/filename.md
promoted_date: 2026-02-26
last_verified: 2026-02-26  # for time-sensitive content
---

The single most important field. Every file MUST have one. It differs from the title. It enables filter-before-read: agents scan descriptions to decide which files to open, without loading full content.

Good:

• "Oracle technology repositioned as clean data source for AI, not traditional on-chain usage"
• "Lawrence Lessig argues transparency creates cynicism without structural reform"

Bad:

• "A concept about oracles" (too vague, just restates the title)
• "" (missing entirely)

A vault of 2,400 files is too large to read exhaustively. The description field is the index.

Containment boundaries (intake/atlas/domains, private/public, temporal/durable) are structural requirements, not suggestions. More effort should go into maintaining boundaries than into adding new content.

Small boundary violations compound silently until the system is too degraded to trust. Check containment during every health audit. Fix violations before adding new content.

After promoting new knowledge to atlas/, run a backward pass:

"Now that this entity exists, what existing notes should link to it or be updated by it?"

This is how knowledge compounds. Without reweave, new files sit beside old files without connecting. With reweave, every promotion strengthens the existing graph.

• Identify recently promoted atlas/ files (last 30 days)
• For each, extract key entities, concepts, and tags
• Search the vault for related files that don't yet link to the new file
• Check bidirectional linking: does A link to B? Does B link to A?
• Check pending observations for related structural issues

• Generate a prioritized action report: high-value links to add

• Bridge links: Connect previously unconnected clusters (highest value)

• Enrichment links: Existing file gains context from new file
• Correction links: New information contradicts or updates old
• Low priority: links between files that already share 3+ connections

Connecting 5 existing concepts is worth more than 5 new unconnected files. Before creating a new file, ask: "Can I enrich an existing one instead?"

This is not a preference -- it's how knowledge systems actually work. Knowledge compounds through connection, not accumulation. The health of a knowledge system is measured by connection density, not file count.

A dedicated directory (intake/.observations/) where agents log things they notice during normal work that don't fit the standard intake flow.

---
type: observation
category: gap | contradiction | connection | friction | structural
observed: 2026-02-26
agent: curator
status: pending | promoted | archived
involves: ["entity1.md", "entity2.md"]
---

Most knowledge systems only capture what you deliberately put in. Observations capture what the system itself is telling you -- gaps, contradictions, structural decay. These signals are generated as a byproduct of normal agent work and would otherwise be lost.

Seven structural checks for pipeline health, run during every health audit. Report PASS / WARN / FAIL per gate.

If implementing incrementally, start with gates 1, 2, and 4. These prevent the most common failure modes: unfindable content, schema drift, and boundary violations.

Knowledge has a shelf life. Use last_verified: YYYY-MM-DD for people profiles, organization descriptions, and technology assessments. Flag when absent or >180 days old for these content types.

Multiple specialized agents operate on the vault, each with explicit workspace boundaries:

No single agent can corrupt the entire system. The curator can flood intake/ with low-quality extractions, but it can't pollute atlas/ directly. The triage agent can make bad recommendations, but a human reviews before promotion. This is defense in depth.

Agents are session-scoped but can maintain continuity via state files:

---
type: agent-state
agent: curator
last_active: 2026-02-26
session_count: 47
status: active
---

Content includes: last session summary, active investigations, learned patterns (e.g., "news articles produce low-value extractions"), cumulative stats, and open questions. Read at session start, write at session end.

This is not implemented yet -- it's a design we intend to build. Currently agents start fresh each session.

A persistent extraction service runs on a cloud VM (Firecracker microVM via sprites.dev). The knowledge-intake sprite (FastAPI, server.py v0.2.0) auto-wakes on HTTP request and exposes two endpoints:

Data flow (URL extraction):

• User sends a URL via WhatsApp or direct POST
• Sprite fetches content, Claude classifies it, extracts structured knowledge
• Writes vault-compatible markdown with frontmatter to extraction directory
• Syncthing syncs it back to all machines
• Appears in intake/ for triage via the heartbeat system

Data flow (structured intake):

• External agent (e.g., Ethoswarm Curator Mind) sends pre-extracted JSON
• Sprite validates against intake schema, writes vault markdown
• Syncthing syncs, heartbeat triages

Knowledge capture happens 24/7, not just during active conversations.

Deployed 2026-03-13.

The jibrain heartbeat is an automated 15-minute collection and triage cycle that moves content from multiple agent outboxes into the intake pipeline and applies quality gates without human intervention.

Collection (every 15 minutes):

• Copies from agents/curator/extractions/andjibot-docs/outbox/tointake/
• NanoClaw writes directly to intake/via Syncthing (no collection step needed)

Triage triggers:

• Every 15 min: conservative auto-promote (no AI needed)
• 5+ pending items OR scheduled hours (7, 12, 17, 22): AI triage via Claude

Conservative auto-promote gates (all 6 must pass):

Trusted agents: deep-meeting-process, meeting-prep, bookmark-extractor, curator

When items fail auto-promote or need human judgment, the heartbeat creates an Apple Reminder notification for review.

Implementation:

• ~/scripts/jibrain-heartbeat.sh— collection and dispatch
• ~/scripts/jibrain-triage.py— triage logic and auto-promote gates
• LaunchAgent: com.amplifier.jibrain-heartbeat(every 900 seconds)

The vault syncs across all devices using Syncthing (peer-to-peer, encrypted, automatic). No cloud dependency. No git for the main vault (too much sensitive personal data for any remote repository).

Three machines with different subsets:

.stignore files enforce these boundaries at the infrastructure level -- private content physically cannot reach machines that shouldn't have it.

The Ethoswarm persistent agent protocol provides an always-on intake channel via a Switchboard Curator Mind running on Telegram.

Phase 1 (basic intake) is operational: Mind receives URLs and text via Telegram, extracts structured knowledge, routes through the sprite, and content appears in intake/ for automated triage.

Phase 2 (bidirectional sync) is in progress: the /intake/structured endpoint is live, but the vault search endpoint (enabling the Mind to check for duplicates before creating intake) is not yet built.

See ethoswarm-switchboard-bridge.md for the full integration specification.

Deployed on NanoClaw 2026-03-04. Origin: built by Colin Raney's bot Fred for OpenClaw, adapted for NanoClaw.

A pure rule-based router that selects the appropriate Claude model (Haiku, Sonnet, or Opus) based on message complexity. No ML inference — the router itself runs in <1ms.

A 14-dimension weighted scorer evaluates the user message only (not the system prompt — key insight) across dimensions like length, code presence, reasoning markers, creative complexity, and domain specificity. The composite score maps to a model tier:

Override: Including [opus] or [haiku] in the message bypasses classification entirely.

• 80% cost savingsvs Opus-for-everything baseline
• Confirmed across 368 requests on Fred (OpenClaw) and 35+ on jibot (NanoClaw)
• 0 routing errors reported since deployment

Fred (Colin Raney's AI bot on OpenClaw) shared the architecture in a cross-pollination Telegram group. jibot documented the spec, Amplifier implemented it — live in production on NanoClaw within 24 hours of the conversation. Bot-to-bot knowledge transfer producing real infrastructure.

Repo: github.com/iblai/iblai-openclaw-router

When an atlas topic accumulates critical mass (~15+ files, distinct vocabulary, recurring patterns), it graduates from atlas/ to domains/ with its own internal structure.

Before (in atlas/):

atlas/concepts/wabi.md
atlas/concepts/sabi.md
atlas/concepts/temae.md
atlas/people/sen-rikyu.md
... (15+ files scattered)

After (graduated to domains/):

domains/chanoyu/
  concepts/       # Domain-specific concept taxonomy
  practitioners/  # People within this domain
  procedures/     # Formalized processes
  sources/        # Primary source texts
  _STRUCTURE.md   # Domain-specific conventions

The exemplar domain is chanoyu/ (Japanese tea ceremony) -- it has its own git repo for public sharing while the rest of the vault remains private.

The knowledge system is integrated with a Getting Things Done (GTD) task management system.

Knowledge without action is a library. Action without knowledge is improvisation.

Every morning routine includes a jibrain status pulse: intake queue size, pending observations count, days since last triage, atlas size and growth. This prevents the most common knowledge system failure: gradual abandonment.

The vault is indexed by a local search service (QMD) that provides three complementary search modes over the full document collection (~4,300 files).

No single search mode works for all knowledge retrieval tasks:

Keyword search is fast and precise but brittle -- it misses paraphrases. Vector search handles meaning but can drift on ambiguous queries. Deep search combines both with automatic query expansion, at the cost of more computation.

The index is partitioned into collections matching the vault's containment boundaries:

Agents can scope searches to a single collection. This prevents temporal content (daily notes) from polluting knowledge searches, reinforcing the same containment principle that governs the directory structure.

For agents operating on the vault, the recommended search strategy:

• Start with keywordwhen you have specific terms (person names, concept titles, exact phrases). Fastest, most precise.
• Use vectorwhen exploring a concept space ("find notes related to decentralized governance"). Handles vocabulary mismatch across languages and terminology.
• Use deepfor broad investigation ("what does the vault contain about the relationship between technology and democracy?"). Most thorough, highest token cost.

The description field in frontmatter is indexed alongside full content, making filter-before-read work across all three modes. Results include relevance scores and document IDs, so agents can retrieve full content selectively rather than reading everything.

Previously, discovery was limited to wikilink traversal and grep-style text search. This worked for navigation (following known connections) but failed for discovery (finding unknown connections). Semantic search closes this gap: an agent running reweave can now find conceptually related files that share no common vocabulary. A query about "aesthetic minimalism" will surface files about wabi even if the word "minimalism" never appears in them.

This also affects multi-language content. The vault contains files in English, Japanese, and occasionally French. Vector search handles cross-lingual similarity in ways that keyword search cannot.

A vault of 4,300+ files is powerful for the owner but opaque to collaborators. When a partner organization needed searchable, browsable access to relevant knowledge subsets, we deployed Onyx -- an open-source RAG + chat platform (YC W24, formerly Danswer) -- as a team-facing projection of the vault.

The key constraint: the vault is the source of truth. Onyx is a read-only projection, not a second brain. Knowledge flows one direction: vault → Onyx. The team searches and chats against a curated subset. They never touch the vault directly.

jibrain vault (4,300+ files)
    │
    ▼
Projection Pipeline (Python)
    ├── Scan markdown files
    ├── Parse frontmatter
    ├── Normalize types
    ├── Sensitivity filter (skip confidential/restricted, >1MB)
    │
    ▼
Staged Corpus (filtered subset)
    │
    ▼
Onyx File Upload Connector
    │
    ▼
Team search + chat interface

The projection pipeline is a Python script that walks the vault, parses YAML frontmatter, and applies a sensitivity filter. Files marked confidential or restricted in frontmatter are excluded. Files over 1MB are excluded (binary artifacts that leaked into markdown directories). The output is a staged corpus directory ready for upload.

gcloud scp to the VM, then re-upload via the admin UI). Automating this is the single highest-priority improvement for this subsystem.

We evaluated several open-source RAG platforms. Onyx won on three criteria:

Zero-fork strategy: we run vanilla upstream Onyx. All configuration is through environment variables and the admin panel. No custom code in the Onyx deployment itself. This means upstream updates are a docker compose pull away -- no merge conflicts, no drift from mainline.

Docker Compose with ~12 containers:

Infrastructure:

Onyx supports custom AI agents -- system prompts that shape how the LLM responds when chatting against the indexed corpus. We configure these through the admin panel. The prompt markdown is stored in the code repo (agents/*.md), not in the vault -- agent prompts are deployment config, not knowledge.

Agents are cheap to create and iterate on. The system prompt is the only configuration -- Onyx handles retrieval, citation, and conversation management.

Onyx is not the only team-facing interface. It is one of four channels feeding into a NormalizedEnvelope pattern (described in multi-interface-knowledge-layer.md):

Onyx ──────┐
Chat bot ──┤
Slack ─────┤──▶ NormalizedEnvelope ──▶ Policy Engine ──▶ Actions
Email ─────┘

Each interface has its own normalizer that maps raw events into a common envelope schema. Everything downstream operates on envelopes, not on interface-specific payloads. This matters because:

• Cross-interface identity resolutionmaps all user handles (Onyx login, Slack handle, email address, chat bot ID) to canonical actor IDs
• Policy engine runs on envelopes, not on raw interface events -- access control, approval workflows, and audit trails are interface-agnostic
• Pydantic schemasdefine the envelope contract; policy rules are pure functions over those schemas
• File-backed YAMLfor approvals and audit logs (no database dependency, git-trackable, portable)
• Dedup via SHA-256 fingerprintprevents duplicate processing when the same request arrives through multiple channels

Adding a new interface (e.g., a WhatsApp bot) requires writing one normalizer. The policy engine, audit trail, and action handlers don't change.

• Zero-fork strategy-- upstream Onyx updates apply cleanly because we never patched the source. This is the single most important deployment decision.
• Vault projection with sensitivity filter-- confidential content stays out of the team-facing interface by construction, not by policy compliance. The filter runs before upload, not at query time.
• Pydantic + pure-function policy engine-- access control logic is testable, auditable, and independent of any specific interface. No ORM, no database migrations, no framework magic.
• Multi-channel envelope pattern-- normalizing all interfaces to a common format meant the second interface was nearly free to add, and the third was trivial.

• File-backed audit-- YAML files in a git repo beat a database for compliance review and portability. Every approval decision is a commit.

• Automate corpus refresh from day one-- manual- gcloud scpis the bottleneck. Every time vault knowledge updates, there's a human-in-the-loop delay before the team sees it. This should be a cron job or a webhook triggered by vault sync.

• Script Onyx admin setup via API-- the admin panel is convenient for iteration but not reproducible. Agent configs, connector settings, and user permissions should be API-driven for disaster recovery and environment parity.
• Start with the Slack connector-- team adoption is faster when discovery happens in tools people already use. File upload was technically simpler but required the team to learn a new interface. Meet users where they are.

If you're building a similar system, implement in this order:

• Three-tier directory structure(intake / permanent / deep domains)
• Frontmatter schema with(single highest-ROI change)- descriptionfield
• Routing decision tree(documented, not ad-hoc)
• Triage process(intake is a queue, not a dump)
• Health monitoring(orphans, broken links, stale content)
• Reweave pass(backward enrichment after promotion)
• Observations mechanism(capture system friction signals)
• Seven Gates audit(pipeline completeness check)
• Currency tracking(- last_verifiedfor time-sensitive content)
• Agent workspace boundaries(no single agent can corrupt the whole)
• Multi-machine sync with containment(different machines get different subsets)
• Task system integration(knowledge connected to action)
• Domain graduation(when topics hit critical mass, give them structure)
• Search infrastructure(keyword + vector + deep search over indexed collections)

Items 1-5 give you 80% of the value. Items 6-14 compound over time.

Capabilities added since the original document was written:

• Amplifier bundle system matured:amplifier-bundle-joinow contains 18+ tool modules covering home automation, productivity (GTD, beads, email), knowledge (QMD, vault), communications (imessage), media (imagen, kimono), research (academic_search, nyne), and more. Multi-provider routing across Anthropic, OpenAI, and Google. Recipe workflows for all major pipelines.
• Model routing quality matrix: Provider routing across claude-opus-4-6 / gpt-5.4 / gemini-3.1-pro with role-based assignment (coding, reasoning, creative, writing, fast, etc.). 80%+ cost reduction vs single-model-for-everything.
• Talk pipeline:talk-builder,talk-refresh,talk-to-gamma,talk-to-slidesrecipes. Full workflow from research brief → outline → draft → Gamma presentation → Google Slides export. Operates onagents/writer/drafts/.
• People pipeline:people-harvest(extract contacts from daily notes),people-enrich(Nyne API enrichment),people-review(network health review). People profiles inatlas/people/are now systematically maintained.
• Meeting pipeline: muesli sync with Japanese translation (whisper → translation → vault),meeting-extractrecipe for concept/entity extraction, daily note injection for meeting summaries.
• Design intelligence: Component design, layout strategy, responsive patterns, animation, voice agent design — available as skill-guided workflows in Amplifier sessions.
• bookmark-extractor sprite DEPRECATED: Replaced by the knowledge-intake sprite (intake-XXXXX.sprites.example). The bookmark-extractor FastAPI sprite on sprites.app is no longer active. All URL intake routes through the knowledge-intake sprite.

• Team-facing Onyx deployment: Deployed Onyx (open-source RAG platform) as a read-only projection of the vault for a partner organization. Zero-fork strategy (vanilla upstream, all config via env vars + admin UI). Vault projection pipeline with sensitivity filter, multi-channel envelope pattern (Onyx + chat bot + Slack + email), file-backed governance (approvals, audit, revisions). See the new section in this document and the companion onyx-rag-deployment.md.

• Agent identity persistence: Designed but not built. Agents currently start fresh each session. State files spec exists but no implementation.

• Adversarial verification: We check structural health but don't deliberately try to break the system (e.g., "what if an agent hallucinated a fake entity?")
• Confidence decay: Knowledge doesn't automatically become less trusted over time.- last_verifiedis a manual check, not a decay function.
• Cross-vault federation: Single-user. No mechanism for merging knowledge graphs across people or organizations.
• jibot cron: Approved design for scheduled jibot tasks (daily summaries, periodic vault health checks) but not yet implemented.

• Source attribution/compensation: We track provenance but don't address the ethical question of compensating original sources.

• Obsidian(obsidian.md): The substrate -- markdown, wikilinks, graph view

• GTD(David Allen): The action methodology integrated with knowledge
• Ars Contexta(arscontexta.com): Description fields, routing trees, the reweave concept (adapted from their "6 Rs pipeline")
• Kieckhefer(1994): The Seven Gates framework (adapted from "The Specific Rationality of Medieval Magic")
• Mechanics of Magic(nraford7): The framing that conventions are safety systems for operating opaque systems, not bureaucratic overhead
• Ethoswarm(amind.ai): Agent identity persistence, dual memory model, compute cost awareness
• Colin Raney / Fred: iblai-router for intelligent LLM routing, cross-pollination experiment demonstrating bot-to-bot knowledge transfer
• sprites.dev: Cloud microVM infrastructure (Firecracker) for the knowledge-intake sprite
• Syncthing: Peer-to-peer sync that respects containment boundaries
• Amplifier(github.com/microsoft/amplifier): The multi-agent framework that operates the system
• QMD: Local search index providing keyword, vector, and deep search over the vault's markdown collections

This document describes a system in active daily use. It is opinionated, battle-tested, and incomplete. If you build on these patterns, the most important thing to get right is the routing decision tree and the description field. Everything else follows from those two.