Garage v1.5 — plan

v1.0 made the corpus retrievable: walk → extract → attribute → chunk → embed → hybrid search → MCP. #7/#12 added a first derived layer — LangExtract facts, one chunks row per fact so they embed for free.

v1.5 makes the corpus understood. Six themes, one shared spine:

The spine is a Postgres job queue drained by a cancellable dream RPC. Every LLM-per-document pass in v1.5 is a job kind on that queue; ingest only enqueues. That keeps ingest fast, makes every pass resumable, and gives the Mac app one thing to start when idle and one thing to cancel when the user comes back.

Everything here runs locally (Ollama / LlamaXPC). Nothing in v1.5 adds a cloud path, and the communication egress guarantee is extended to the new derived tables rather than weakened.

0. Where we start (grounding)

What already exists and shapes the design:

• facts (006_facts.sql) is per-document, replaced wholesale on re-extraction, grounded by char_start/char_end, and cascades from documents. 007_chunk_fact_link.sql gives each fact a chunks row (chunks.fact_id) so backfill_model’s anti-join embeds it with no fact-specific code. Keep this trick; it’s the model for memories too.
• The extraction prompt asks for every fact, verbatim, no salience judgment (enrich/facts.py:PROMPT). gemma2:2b via Ollama is the only real backend; llama_xpc is a transport that works but has no inference engine behind it yet.
• Providers are hard-coded strings in two unrelated places: embed/factory.py (ollama, lmstudio, llama_xpc, one class each, hosts from ollama_host/lmstudio_host) and enrich/facts.py (FACT_DISTIL_PROVIDERS, LangExtract-shaped). You can have one Ollama and one LM Studio, and a chat-capable provider for facts is a different concept from an embedding provider even when it’s the same server.
• Search (search/hybrid.py) has no notion of fact vs. content chunk: SearchHit carries no fact_id, so a fact hit is indistinguishable from a chunk hit to the MCP client.
• classify.py decides corpus_class by shape only. There is no topic/category concept and no per-document summary.
• MCP tools are all read-only. There is no write path from an agent into the corpus — neither for a distilled memory nor for verbatim text an agent wants kept.
• The Mac app has interval maintenance (AppState.runScheduledMaintenance: scan → ingest → backfill every N minutes, guarded only by “nothing else running”). Nothing checks user idleness, thermal state, or power source. IngestService uses ProcessInfo.beginActivity to prevent idle sleep — the opposite of what dreaming needs.
• Cross-cutting rules that every theme must honour: data/sql/00*.sql is schema truth and db/models.py mirrors it; migrations are idempotent and re-applied; new settings go through config.SECTIONS and garage.schema.json must be regenerated; each new test file needs a py_test in garage_python/tests/BUILD.bazel; test_egress_block.py keeps every outbound client inside net/egress.py (the egress guard: destination allowlist, communications only to loopback) and every cloud AI SDK out of the code and the lockfile.

1. Shared spine: the enrichment queue + dream worker

Schema — 008_enrich_queue.sql

CREATE TYPE enrich_job_kind AS ENUM (
    'summarize', 'contextualize', 'categorize', 'extract_facts', 'link_documents',
    'compress_facts', 'canonicalize_entities', 'anticipate_questions', 'summarize_communities',
    'consolidate_memories', 'synthesize_insights', 'deep_thoughts'
);
CREATE TYPE enrich_job_state AS ENUM ('queued', 'running', 'done', 'failed', 'deferred');

CREATE TABLE enrich_jobs (
    id            bigserial PRIMARY KEY,
    kind          enrich_job_kind NOT NULL,
    document_id   bigint REFERENCES documents(id) ON DELETE CASCADE,  -- null for corpus-wide kinds
    scope         text,              -- e.g. category slug for synthesize_insights
    priority      int  NOT NULL DEFAULT 0,
    state         enrich_job_state NOT NULL DEFAULT 'queued',
    attempts      int  NOT NULL DEFAULT 0,
    -- content_sha256 the job was enqueued against, so a stale job for a
    -- since-rebuilt document is dropped rather than run.
    content_sha256 bytea,
    scheduled_at  timestamptz NOT NULL DEFAULT now(),
    started_at    timestamptz,
    finished_at   timestamptz,
    error         text,
    CONSTRAINT enrich_jobs_one_pending UNIQUE (kind, document_id, scope)
        -- partial: WHERE state IN ('queued','running','deferred')
);

• Enqueue is cheap and happens in ingest: when pipeline.py creates a document or its content_sha256 changes, it upserts summarize (priority by recency × authored). Each job enqueues its successor on completion: summarize → categorize (consumes the summary) → extract_facts (consumes the category’s distillation profile, §4) → link_documents (consumes facts + entities, §4b). Content-rebuild cascades delete facts today; the re-enqueue closes that gap.
• Drain is garage dream (CLI) / Dream(DreamRequest) returns (stream DreamStatus) (gRPC). Claims one job at a time with FOR UPDATE SKIP LOCKED, one transaction per job, honours a wall-clock budget (--budget 30m) and a stop flag checked between jobs so gRPC cancellation is prompt. Ollama unreachable → job deferred with backoff, not failed.
• Dependency order inside a session: summaries → chunk contexts → categories → facts → fact-tree compression (per document) → document linking → entity canonicalization → anticipated questions → community summaries → fact-tree compression (per community) → memory consolidation → insights → deep thoughts, but within a budget, and skipping later passes when the earlier ones still have backlog.
• Ordering within a kind is by heat (research A7): heat = α·recall_hits + β·linked_memories + γ·exp(-age/μ) computed in SQL over documents/memories (recall_hits from the query log below), so the items the user actually touches get the expensive passes first; corpus-wide passes run only when per-document backlog is below a threshold.
• Two phases by cost (research SCM): “NREM” jobs are SQL-only (retention decay, heat refresh, strengthening co-recalled memories) and run on any idle minute; “REM” jobs call the model and run only on idle stretches ≥ dream.idle_minutes.
• Not a new process: it’s a Typer command plus an executor.py entry, like enrich-facts today. enrich-facts becomes a thin “enqueue + drain now” wrapper and is kept for scripting.

Queries drive the queue

Every rag_search / rag_recall / gRPC Search is a signal about where the derived layer is thin. A local-only query log turns that signal into work:

CREATE TABLE search_events (
    id             bigserial PRIMARY KEY,
    at             timestamptz NOT NULL DEFAULT now(),
    surface        text NOT NULL,            -- mcp_stdio | mcp_http | grpc | cli
    client         text,                     -- MCP client name when known
    query          text NOT NULL,
    mode           text NOT NULL,
    filters        jsonb NOT NULL DEFAULT '{}'::jsonb,
    hit_count      int  NOT NULL,
    low_confidence boolean NOT NULL DEFAULT false,
    top_score      real
);
CREATE TABLE search_event_hits (
    event_id    bigint NOT NULL REFERENCES search_events(id) ON DELETE CASCADE,
    rank        int    NOT NULL,
    kind        text   NOT NULL,             -- chunk | fact | memory | summary
    chunk_id    bigint REFERENCES chunks(id) ON DELETE CASCADE,
    document_id bigint REFERENCES documents(id) ON DELETE CASCADE,
    PRIMARY KEY (event_id, rank)
);

Writing the log is part of the search call (one insert, same transaction as the recall-count bumps in §3). What it drives, cheapest first:

1. Heat. recall_hits in the heat score (§1 above) is a count over search_event_hits with time decay, so the documents people actually retrieve rise to the front of every per-document queue. A document hit today gets its contextualize/extract_facts backlog before one nobody has touched.
2. Re-distillation of hit documents. After each search the server enqueues, at high priority, extract_facts for any hit document whose facts are missing, dormant, extracted under an older profile version or model, or extracted before the document’s last content_sha256 change — and contextualize for any hit chunk with context_source='heuristic'. Dreaming revisits exactly what retrieval keeps landing on, and re-distills with the current profile and the larger dream.model.
3. Chunk-hit-without-fact-hit. A document whose chunks rank in the top-10 while none of its facts do is a document whose salient content the extractor missed. It is enqueued for extract_facts with gleanings + 1 and a raised max_per_document, and the query is passed to the job as a hint (“a reader asked about …”) so the profile prompt can steer toward it.
4. Low-confidence queries seed anticipation. A low_confidence query is, by definition, a question the corpus should answer better. It becomes an anticipate_questions seed for its top-3 documents (the query is the anticipated question), so the next dream session precomputes a cited answer if one exists — Sleep-time Compute’s “predictable queries” (research A2), with the user’s own queries as the predictor.
5. Reformulations → lessons. Two queries from the same client within 2 minutes with overlapping hits are a reformulation pair; dreaming turns repeated pairs into kind=lesson memories (“queries about X should also match Y”) used for query expansion — the cheap half of Reflexion (research A9), pulled forward from 1.6 because the log now exists.
6. Rehearsal. Hits already bump last_recalled_at on facts and memories (§3), so retrieval is also what keeps useful things from being forgotten.

Bounded: enqueueing from a search is a single INSERT … ON CONFLICT DO NOTHING per hit document against enrich_jobs_one_pending, never a model call in the search path; the log is capped by search.log_retention_days (default 90) and search.log_enabled (default true), lives only in the local database, and is never included in any egress path. The Mac app’s Status page shows “revisit queue: N documents, driven by M queries this week”.

Tests (test_search_events.py): a search inserts one event + hits; hit document with stale facts is enqueued once; chunk-hit-without-fact-hit raises gleanings; low-confidence query enqueues anticipate_questions with the query as seed; reformulation detection window; retention purge; no enqueue when log_enabled=false.

Config — new dream section

enabled (default true), model (a larger local model than facts.model; idle time affords it), provider, budget_minutes per session, max_jobs_per_session, idle_minutes (Mac-side threshold, stored here so CLI and app agree), require_ac_power (default true), pause_on_thermal_pressure (default true).

Tests

test_enrich_queue.py: enqueue-on-ingest and on-content-change; stale-content_sha256 job is dropped; claim is exclusive; budget stops cleanly between jobs; cancellation flag honoured; Ollama-down → deferred not failed.

2. Salient facts

Goal: the facts table stops being a verbatim inventory and becomes a compressed, deduplicated set of claims worth retrieving. Fewer rows, fewer vectors, higher hit quality.

Extraction changes (enrich/facts.py)

1. Prompt is assembled from the document’s distillation profile (§4), not a single generic string. The profile supplies the fact kinds wanted, a category-specific instruction, and category-specific few-shot examples; the generic part requests only facts a reader would want to remember about the document (decisions, commitments, entities and their relationships, dates and amounts, preferences, conclusions) and to skip boilerplate, restatements, and anything derivable from the title/path. Keep the grounding rule (exact wording, no paraphrase) because char_interval grounding is what makes a fact verifiable and ungrounded ones are dropped.
2. Structured attributes on every extraction, via LangExtract attributes: kind ∈ {entity, relationship, event, decision, preference, quantity, claim}, salience ∈ 1–10 (the Generative Agents “mundane=1, poignant=10” rating, shared with memories so scores are comparable — research A1), optional subject. Facts are decontextualized propositions (pronouns resolved, entities named — research B1) so they stand alone as FTS and embedding targets; each chunk’s prompt carries the document’s known entities so far (authors, mail participants, entities from earlier chunks and same_thread siblings — research C6) so mentions resolve to one name. A relationship extraction additionally carries subject/predicate/object and is what feeds the triple store (§4b). Few-shot examples updated to show low-salience facts being omitted.
3. Post-filter (deterministic, tested without a model):
   • drop salience < profile.min_salience (profile default 3);
   • drop by shape, extending extract/quality.py heuristics: < 4 words, all-caps headings, pure dates/URLs, lines that are ≥ 80% of the document title;
   • cap per document at profile.max_per_document and profile.per_kchars (whichever is smaller), keeping highest salience then earliest position; profile.gleanings (default 1) maps onto LangExtract extraction_passes for recall on small models (research C1);
   • collapse in-document near-duplicates by normalized text (fact_sha256 over lower/strip/punct-collapsed text).

4. Cross-document dedup → canonical facts. New columns:

   ALTER TABLE facts ADD COLUMN salience   smallint NOT NULL DEFAULT 3;
   ALTER TABLE facts ADD COLUMN kind       text     NOT NULL DEFAULT 'claim';
   ALTER TABLE facts ADD COLUMN fact_sha256 bytea;
   ALTER TABLE facts ADD COLUMN canonical_fact_id bigint REFERENCES facts(id) ON DELETE SET NULL;
   ALTER TABLE facts ADD COLUMN status text NOT NULL DEFAULT 'kept';  -- kept | merged | compressed | dormant | dropped
   ALTER TABLE facts ADD COLUMN last_recalled_at timestamptz;          -- forgetting curve (§3)
   ALTER TABLE facts ADD COLUMN recall_count int NOT NULL DEFAULT 0;
   CREATE INDEX facts_sha ON facts (fact_sha256);
   

   After a document’s facts are stored, the job embeds them (the existing chunk path) and, in the same dream session, retrieves the 10 nearest kept facts from other documents and asks the model for one of ADD / UPDATE / DELETE / NOOP per candidate (research A3 — Mem0’s consolidation step, one structured-output call); cosine ≥ facts.dedup_threshold (0.94) short-circuits to NOOP without a model call. A merge sets canonical_fact_id, status='merged', and deletes the merged fact’s chunk so only canonical facts carry vectors; a DELETE is a supersede, never a row delete. Provenance is kept: the merged row still has its document_id and span, so rag_get_document can show “this fact is also stated in N other documents”. canonical_fact_id is SET NULL on delete so removing the canonical document promotes the next-oldest sibling (a small reconcile step in the same job kind).

Search / MCP surface

• SearchHit and MCP Hit gain kind: "chunk" | "fact" | "memory" | "summary" (derived from chunks.fact_id / chunks.memory_id), and facts gain evidence_count. A fact hit returns its parent passage (the chunk containing char_start) as text, with the fact as snippet — the proposition-retrieval gain in research B1 comes from the passage, not the proposition alone.
• rag_search(include: ["chunks","facts","memories"]) filter; default includes all.
• rag_get_document returns facts with salience/kind, and a also_stated_in list for canonicals.

2b. Fact-tree compression

Salience filtering bounds how many facts a chunk produces; it does nothing about the same fact appearing across 40 chunks, or about a document whose 160 leaf facts are individually true and collectively noise. Compression is the answer, and it is structural rather than ad hoc: a tree with a fixed fan-in per level, four operators applied from lossless to lossy, and leaves that are never deleted — every compressed fact is a parent node with evidence links to the children it replaced. Retrieval searches all levels collapsed-tree style (research B4), so a specific question hits a leaf and a general one hits a compressed node.

Fan-in of roughly 8:1 per level is what bounds the vector count: which facts carry vectors is L0 facts above the salience threshold that survived compression, plus every L1+ node. That ratio is the compression the emb_* tables actually feel.

Operators, applied in this order at each level:

1. Duplicate collapse — lossless. fact_sha256 over normalized text, then cosine ≥ 0.94. SQL + KNN, no model. (Already the first step of §2’s canonical dedup; it now also produces the L1 node.)
2. Aggregation over triples — near-lossless and SQL only. Facts sharing (subject, predicate) with different objects or times become one aggregate: three Rick —paid_to→ Acme on the 3rd of three months become one node with {count: 3, amount: 50, cadence: monthly, first, last} in attributes. This is the pay-off of §4b for compression: the group-by is free and the model only writes the sentence.
3. Subsumption — mildly lossy. If B entails A (a small NLI cross-encoder — the reranker family from §2’s search improvements can double here — or a batched model yes/no), keep B and demote A to evidence of B. “Acme is headquartered in Austin, Texas” subsumes “Acme is in Texas”.
4. Abstraction — lossy, model-written, and the only step that may invent wording. A cluster of related survivors (KNN cluster within the scope, or same community) becomes one generalized statement that must cite ≥ 2 children; citations that don’t resolve discard the abstraction — the same rule as insights (§5).
5. Pruning to budget — last resort. If a level is still over budget, keep by salience × retention × specificity with MMR for diversity, so survivors aren’t five variants of one topic.

Steps 1–2 run in extract_facts itself (they’re cheap); 3–5 are the compress_facts dream job, per document after extraction and per community after summarize_communities.

Schema (012_triples.sql gains these; facts is the node table for every level):

ALTER TABLE facts ADD COLUMN level    smallint NOT NULL DEFAULT 0;   -- 0 chunk · 1 document · 2 community · 3 corpus
ALTER TABLE facts ADD COLUMN scope_id bigint;                        -- document_id / community_id at that level
ALTER TABLE facts ADD COLUMN operator text;                          -- dedup | aggregate | subsume | abstract | prune
ALTER TABLE facts ADD COLUMN children_hash bytea;                    -- hash of the child-id set; dirty when it changes
CREATE TABLE fact_children (
    parent_id bigint NOT NULL REFERENCES facts(id) ON DELETE CASCADE,
    child_id  bigint NOT NULL REFERENCES facts(id) ON DELETE CASCADE,
    PRIMARY KEY (parent_id, child_id)
);
CREATE INDEX facts_level_scope ON facts (level, scope_id);

A compressed leaf goes status='compressed' and loses its chunk (so its vector), but keeps its row, span and document_id, and stays reachable through fact_children — for provenance, for rag_get_document’s “stated in N places”, and for un-compressing if a parent is later invalidated. facts.status is therefore kept | merged | compressed | dormant | dropped.

Incremental, never global. A parent stores children_hash; it is recomputed only when its child set changes by more than facts.recompress_threshold (20%), flagged dirty by ingest or re-extraction and picked up by dreaming in heat order. A single new document builds its own L1 and marks its L2 parent dirty; the corpus tree is never rebuilt wholesale.

Two guards:

• Answerability floor (research C4, MINE-style). For a scope, sample questions generated from its L0 facts and check they are answerable from its L1 facts alone; if answerability drops below facts.answerability_floor (0.9), the scope’s budget is raised rather than compressing harder. This is the test that keeps “compression” from becoming “loss”, and it runs as part of compress_facts with a model, so it costs idle time not query time.
• Forgetting by level. The forgetting curve (§3) applies per node, but a parent’s last_recalled_at is the max over its live children, so a compressed fact still being used through its leaves never goes dormant while its leaves are.

Config (facts section): community_budget (50), recompress_threshold (0.2), answerability_floor (0.9), abstraction_min_children (2).

Tests (test_fact_tree.py): operator order and idempotence on re-run; aggregation groups only same (subject, predicate); subsumption keeps the more specific fact; an abstraction with < 2 resolvable citations is discarded; budgets enforced per level with MMR diversity; compressed leaf loses its chunk but not its row; children_hash dirtiness threshold; answerability floor raises the budget; parent last_recalled_at follows children.

Search improvements that ride along (search/hybrid.py)

Independent of facts, three retrieval changes the literature makes a strong case for, all local and all precompute-friendly (research B2, B3, B4, B5, B8, B9):

• Chunk context. chunks.context text + context_source ('heuristic'|'llm'): a heuristic context (title + heading path + first sentence of the summary) at ingest, upgraded by the contextualize dream job to a 50–100-token model-written situating sentence; embed context || text and include context in the generated tsvector. Anthropic reports −49% top-20 retrieval failures with contextual embeddings + BM25, −67% with reranking on top.
• Summaries as chunks. documents.summary is embedded as a kind='summary' chunk in the same emb_* tables (RAPTOR collapsed-tree, level 1) — zero query-time change.
• Weighted, multi-list RRF. Σ w_e/(k + rank) with per-corpus_class engine weights (code → FTS heavier), extra engines as extra lists (facts, summaries, entities/PPR, recency), and a weakest-link guard that drops an engine’s list when it returns < 3 hits.
• Optional local reranker. A cross-encoder (bge-reranker-v2-m3 or mxbai-rerank-base-v2, downloaded via ModelDownloadXPCService, run in the embed XPC service) over the RRF top-50, rerank: bool on MCP/gRPC search, default on when present; reranks the parent passage after dedup by document. Its top score is the low_confidence signal on results.
• Client self-routing. Results carry rank, score, low_confidence and document_length_tokens; a rag_read_document(id, range) tool lets a client pull the whole document when chunks don’t suffice (Self-Route), and multi-entity “how/why” queries are split into 2–3 sub-queries fused by the same RRF (RAG-Fusion).

Config — facts section

enabled, model (default gemma2:2b), provider (a provider name, §6; default the llama_xpc entry), dedup_threshold (0.94), and the profile defaults min_salience (3), max_per_document (40), per_kchars (4) that a category profile may override. enrich_facts_on_ingest stays false: facts are a dream job.

Tests

test_facts.py extended: salience filter, shape filter, per-doc cap ordering, in-doc dedup, canonical merge deletes the merged chunk but not the fact row, canonical promotion on delete. A prompt-regression fixture (a few paragraphs + expected kept/dropped set) run under pytest -m model only when Ollama is reachable, so CI stays hermetic.

3. Generic memory

Goal: a durable, grounded, searchable store of things worth remembering — regardless of whether the user said it, an agent learned it during a session, or the dreamer inferred it — with recall that favours important and recently-useful memories.

Model: a memory is a document

Follow the conversations pattern (structured table + a synthesized documents row). Each memory gets a documents row in a synthetic source (sources.kind = 'memory', added to the CHECK), with uri = memory://<uuid>; that row’s chunk(s) are embedded and searched by every existing path with zero new embedding code, and every existing filter (source, class, trust, author) just works.

CREATE TYPE memory_kind AS ENUM ('fact', 'preference', 'decision', 'note', 'insight', 'thought', 'qa', 'lesson', 'todo');
CREATE TYPE memory_origin AS ENUM ('user', 'agent', 'dream');

CREATE TABLE memories (
    id               bigserial PRIMARY KEY,
    document_id      bigint NOT NULL UNIQUE REFERENCES documents(id) ON DELETE CASCADE,
    kind             memory_kind NOT NULL,
    origin           memory_origin NOT NULL,
    origin_detail    text,          -- MCP client name, dream job id, ...
    text             text NOT NULL,
    subjects         text[] NOT NULL DEFAULT '{}',
    importance       real NOT NULL DEFAULT 0.5 CHECK (importance BETWEEN 0 AND 1),
    confidence       real NOT NULL DEFAULT 1.0 CHECK (confidence BETWEEN 0 AND 1),
    status           text NOT NULL DEFAULT 'active',   -- active | archived | superseded
    superseded_by    bigint REFERENCES memories(id) ON DELETE SET NULL,
    created_at       timestamptz NOT NULL DEFAULT now(),
    updated_at       timestamptz NOT NULL DEFAULT now(),
    last_recalled_at timestamptz,
    recall_count     int NOT NULL DEFAULT 0,   -- with created_at/last_recalled_at drives the forgetting curve
    expires_at       timestamptz
);

-- Grounding. An insight must cite what it was inferred from.
CREATE TABLE memory_evidence (
    memory_id   bigint NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
    document_id bigint NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
    chunk_id    bigint REFERENCES chunks(id) ON DELETE SET NULL,
    fact_id     bigint REFERENCES facts(id)  ON DELETE SET NULL,
    PRIMARY KEY (memory_id, document_id, COALESCE(chunk_id,0), COALESCE(fact_id,0))
);

Privacy rule (enforced in code + test): a memory’s corpus_class is the most restrictive of its evidence. Any evidence with communication makes the memory communication, which the egress guard already refuses to ship: memory inputs go through net.egress.check_destination with their corpus_class, like every other content. trust_tier: authored for origin=user, reference for agent/dream (the owner didn’t write it; a later UI “confirm” flips it to authored).

Recall ranking

search/recall.py: hybrid RRF as today over the memory source, then re-rank with score × (0.5 + importance) × retention, where retention is the forgetting curve below. Every recall bumps last_recalled_at/recall_count and re-computes stability.

Forgetting curve (shared by memories and facts)

One mechanism, in enrich/forgetting.py, simulating Ebbinghaus with the spacing effect (research A4 MemoryBank, A1 Generative Agents, FadeMem):

t          = now − GREATEST(created_at, last_recalled_at)      -- time since store OR last retrieval
retention  = exp(−t / stability)                               -- Ebbinghaus: R = e^(−t/S)
stability  = S0 · growth^recall_count                          -- each retrieval makes it decay slower

• S0 (memory.stability_days, default 7 for memories, 30 for facts — facts are grounded in a document and should not evaporate as fast as an agent’s note) and growth (memory.stability_growth, default 1.6, so the 1st recall stretches decay to ~11 days, the 5th to ~70). This is the spacing effect: something recalled repeatedly is effectively permanent, something stored once and never used fades. origin=user memories use S0 = ∞ (never decay) unless the user opts in.
• Computed in SQL as an expression over created_at, last_recalled_at, recall_count (no stored score to go stale), and exposed as a view column retention on memories and facts. The NREM phase (§1) refreshes an index-friendly retention_bucket nightly for cheap filtering.
• Where it applies: the memory re-rank above; the fact lane in RRF (facts are one list whose ranks are pre-weighted by retention, so a stale fact yields to a fresh one at equal relevance); and rag_recall’s low-confidence rejection, which uses score × retention.
• Forgetting = archiving, never deletion. retention < memory.archive_below (0.05) and importance < 0.3 → status='archived' for memories; for facts, status='dormant', which removes the fact’s chunk from search but keeps the row and its evidence. Archived/dormant items are excluded from recall by default and included with include_archived=true. A recall of a dormant fact (via its parent document) restores it.
• Rehearsal. The consolidate_memories pass may rehearse — bump last_recalled_at without a user recall — for memories with importance ≥ 0.8 that are about to cross the threshold, and for anything cited as evidence by a live insight, so the graph never archives something an insight depends on. Rehearsal is logged (recall_source='rehearsal') so the counts stay honest.
• Documents are not forgotten (episodic sources are immutable, research A10), but a recency list 1/(k + rank_by_GREATEST(mtime, last_hit_at)) is one of the weighted RRF engines (§2), so retrieval favours what the user is currently working on without hiding anything.

Tests (test_forgetting.py): monotonic in t, monotonic in recall_count; user memories never decay; archive threshold is a pure function of the columns; rehearsal never touches origin=user; dormant fact restored by a parent-document hit. rag_recall applies a low-confidence rejection on score × retention (research MemX) and returns nothing rather than noise, and accepts as_of to answer against memories/triples valid at a date (research A5). Memories carry LLM-generated keywords/tags indexed on the FTS side (research A8).

Always-loaded tier (research A6): a bounded core memory (≤ 4 KB, labelled blocks persona, user, current_projects) exposed as an MCP resource so clients read it every session without a search; dreaming rewrites it, and rag_remember(block=...) can edit a block within the size limit. Alongside it, a dream-maintained user portrait memory and day/week digests (research A4).

Tiers by heat (research A7): agent-written memories land in a short-term tier that is only embedded and deduped; dreaming promotes them into long-term recall when their heat crosses τ, so a noisy client cannot pollute recall.

Surfaces

• MCP (first write tools in the project): rag_remember(text, kind, importance, subjects, evidence_document_ids), rag_recall(query, kinds, limit), rag_forget(memory_id), rag_list_memories(kind, status, limit), and rag_store (below). All return dataclasses. Writes are bounded (rag_remember text ≤ 4 KiB, rag_store ≤ 256 KiB, rate-limited per process), tagged origin=agent with the client name, and gated by a new mcp.allow_memory_writes setting (default true for stdio, false for the HTTP transport unless explicitly enabled — HTTP is the shared local server).

rag_store — verbatim input, distilled like everything else

rag_remember is for something already reduced to a sentence or two. rag_store is the other door: an agent (or the user, via garage store / paste in the app) hands over verbatim text — a conversation excerpt, a pasted email, meeting notes, a web page’s text, a decision record — and Garage keeps it as a first-class document and runs the same passes the rest of the corpus gets.

rag_store(
    text,                      # verbatim; stored untouched as documents.content
    title=None,
    kind="note",               # note | conversation | clipping | decision | reference
    corpus_class="document",   # 'communication' allowed and honoured by the egress guard
    subjects=[],               # hints only; the model still decides
    category=None,             # optional slug; a user/agent pin, never overwritten (§4)
    distill="queue",           # queue | now | none
    remember=False,            # also create a memory pointing at this document
) -> StoreResult(document_id, uri, summary?, categories?, facts?, memory_id?, queued_jobs)

• The text becomes a documents row in a synthetic inbox source (sources.kind='inbox', uri = inbox://<uuid>), trust_tier = authored when the MCP client says the user wrote it (kind='note'|'decision') and received otherwise, chunked and embedded exactly like a file. content_sha256 dedups an identical re-store to the same document.
• distill="queue" (default) enqueues summarize → categorize → extract_facts → link_documents (§1) at high priority, so the next dream session picks it up first. distill="now" runs those passes synchronously on facts.provider/model and returns the summary, categories and kept facts in the response — bounded by a per-call budget (mcp.store_distill_timeout_seconds, default 60) after which the remainder is queued and the response says so. "none" stores only.
• remember=True additionally creates a memories row (kind mapped from the store kind, origin=agent) whose evidence is the new document, so a stored decision is both searchable as a document and recallable as a memory.
• corpus_class='communication' is accepted and honoured: the document, its facts, and any memory derived from it are communication-class and never reach a remote-allowed provider (§6). The egress guard’s content rule (net.egress.check_destination with the corpus_class) is what enforces it; the tool does not get a bypass.
• CLI: garage store [--title] [--kind] [--class] [--distill now|queue|none] [FILE|-]; gRPC StoreText; the app’s Memories view gets a “Store…” sheet for paste-in.

Tests (test_store.py): verbatim content round-trips byte-for-byte; identical re-store dedups; queue enqueues the four kinds at high priority; now returns facts and falls back to queueing past the timeout; remember=True links evidence; communication-class store refuses a remote provider; size and rate limits enforced.

• CLI: garage memory add|list|recall|forget|confirm.
• gRPC: Remember, Recall, ListMemories, ForgetMemory for the app.
• Mac app: a Memories view (filter by kind/origin, confirm/archive, see evidence) — the “dream journal” in §5 is this view filtered to origin=dream.

Tests

test_memory.py: create → document+chunk exist; class inherits most restrictive evidence; egress refuses communication-class memory; recall ranking monotonic in importance/recency; recall bumps counters; MCP write gate honoured per transport; forget cascades to chunk/vectors.

4. Document categorization

Goal: a topic axis (finance, health, legal, project:<x>, travel, receipts, …) next to the existing “what is it” (corpus_class) and “how trusted” (trust_tier) axes — cheap at ingest, refined while dreaming, filterable in search — and the switch that decides how a document is distilled. A receipt and a design doc should not be asked the same questions.

Schema — 010_categories.sql

CREATE TABLE categories (
    id          smallserial PRIMARY KEY,
    slug        text NOT NULL UNIQUE,
    parent_id   smallint REFERENCES categories(id) ON DELETE CASCADE,
    name        text NOT NULL,
    description text,                -- fed to the model as the zero-shot definition
    builtin     boolean NOT NULL DEFAULT false,
    -- Distillation profile (§4a). Null inherits the parent's, then the global default.
    profile     jsonb
);
CREATE TABLE document_categories (
    document_id bigint   NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
    category_id smallint NOT NULL REFERENCES categories(id) ON DELETE CASCADE,
    confidence  real NOT NULL DEFAULT 1.0,
    assigned_by text NOT NULL,       -- 'path' | 'model' | 'user'
    PRIMARY KEY (document_id, category_id)
);
ALTER TABLE documents ADD COLUMN summary text;
ALTER TABLE documents ADD COLUMN summary_model text;

Assignment, two tiers

1. Path prior at ingest (ingest/categorize.py, data-driven table in the style of attribute/pathrules.py): ~/Documents/Taxes/** → finance/tax, receipts in the path → finance/receipts, repo roots → project:<repo-name> (auto-created leaf categories under project). Instant, confidence 0.6, assigned_by='path'. Never enqueues a model.
2. Model at dream time (categorize job, consumes documents.summary): zero-shot over the taxonomy’s descriptions, JSON output {"categories":[{"slug":..,"confidence":..}]}, multi-label, top-3 with confidence ≥ 0.5. A user assignment (assigned_by='user') is never overwritten.
3. Classifier tier (research C8, TnT-LLM): the LLM labels only a seed sample per category; a per-category logistic regression over the existing emb_* vectors (assigned_by='classifier') assigns the bulk, with the LLM as fallback on low margin. Refreshed by dreaming whenever the taxonomy changes — orders of magnitude fewer model calls on a 20k-document corpus. The summary itself (1–3 sentences, summarize job) is stored and returned by rag_get_document/ListDocuments — it’s what makes the Mac app’s document list legible and is the input to facts and insights too.

Taxonomy: a seeded builtin set (~20 leaves), extensible in config (categories.extra: [{slug, parent, name, description, profile}]) and by CLI (garage category add). Emergent/clustered categories are out of scope for 1.5 (see §8).

4a. Distillation profiles — categories drive distillation

A profile is what a category tells the downstream passes to look for. Stored on the category (categories.profile, inheriting up the tree), it is the input to extract_facts (§2), link_documents (§4b) and synthesize_insights (§5):

{
  "fact_kinds":      ["quantity", "entity", "event"],
  "instruction":     "This is a receipt or invoice. Extract vendor, total, currency, date, line items over $50, and payment method.",
  "examples":        [{"text": "...", "extractions": [...]}],   // LangExtract ExampleData: steer by demonstration (research C7)
  "min_salience":    2,
  "max_per_document": 12,
  "predicates":      ["paid_to", "purchased", "dated"],
  "entity_kinds":    ["org", "product", "money", "date"],
  "insight_prompt":  "Summarize spending patterns by vendor and month."
}

Builtin profiles ship for every builtin category (receipts, legal/contracts: parties, obligations, dates, amounts; health: providers, diagnoses, medications, dates; project:*: decisions, APIs, open questions, owners; meeting notes: attendees, decisions, action items; communication threads: commitments, requests, dates). A document with several categories uses the union of fact kinds and the strictest caps; a document with none uses the global facts defaults, which is exactly today’s behaviour, so nothing regresses for uncategorized documents.

This is why categorize runs before extract_facts in the queue (§1): the category is a dependency of distillation, not a sibling of it. The path prior means most documents have a category before the model ever runs, so the first dream session already distills with the right profile.

test_categorize.py additionally covers: profile inheritance (leaf → parent → global), multi-label union/strictest-cap merge, and that an uncategorized document’s assembled prompt equals the global one.

4b. Semantic triples — inter-relating documents

Goal: documents relate to each other through what they are about (shared entities) and how they relate (one revises, cites, replies to, or is an attachment of another), stored as (subject, predicate, object) triples that search can expand along and the app can show.

CREATE TABLE entities (
    id          bigserial PRIMARY KEY,
    kind        text NOT NULL,           -- person | org | project | product | place | topic | money | date | other
    name        text NOT NULL,           -- canonical display form
    norm_name   text NOT NULL,           -- lower/strip/diacritics, for exact matching
    author_id   bigint REFERENCES authors(id) ON DELETE SET NULL,  -- when the entity is a known person
    document_id bigint UNIQUE REFERENCES documents(id) ON DELETE CASCADE, -- synthetic doc, for embedding
    meta        jsonb NOT NULL DEFAULT '{}'::jsonb,
    created_at  timestamptz NOT NULL DEFAULT now(),
    CONSTRAINT entities_norm_unique UNIQUE (kind, norm_name)
);
CREATE TABLE entity_aliases (
    entity_id bigint NOT NULL REFERENCES entities(id) ON DELETE CASCADE,
    alias     text   NOT NULL,
    PRIMARY KEY (entity_id, alias)
);

-- A node is either an entity or a document; exactly one of each pair is set.
CREATE TABLE triples (
    id                  bigserial PRIMARY KEY,
    subject_entity_id   bigint REFERENCES entities(id)  ON DELETE CASCADE,
    subject_document_id bigint REFERENCES documents(id) ON DELETE CASCADE,
    predicate           text   NOT NULL,
    object_entity_id    bigint REFERENCES entities(id)  ON DELETE CASCADE,
    object_document_id  bigint REFERENCES documents(id) ON DELETE CASCADE,
    object_literal      text,
    confidence          real   NOT NULL DEFAULT 1.0,
    assigned_by         text   NOT NULL,      -- 'rule' | 'model' | 'user' | 'dream'
    -- Grounding: where this triple was stated or inferred from.
    document_id         bigint REFERENCES documents(id) ON DELETE CASCADE,
    fact_id             bigint REFERENCES facts(id)     ON DELETE SET NULL,
    -- Bi-temporal (research A5/C5): when it was true in the world vs. when we learned/retired it.
    valid_from          timestamptz,
    valid_to            timestamptz,
    created_at          timestamptz NOT NULL DEFAULT now(),
    invalidated_at      timestamptz,
    invalidated_by      bigint REFERENCES triples(id) ON DELETE SET NULL,
    CHECK (num_nonnulls(subject_entity_id, subject_document_id) = 1),
    CHECK (num_nonnulls(object_entity_id, object_document_id, object_literal) = 1)
);
CREATE INDEX triples_subject_entity ON triples (subject_entity_id, predicate);
CREATE INDEX triples_object_entity  ON triples (object_entity_id, predicate);
CREATE INDEX triples_subject_doc    ON triples (subject_document_id, predicate);
CREATE INDEX triples_object_doc     ON triples (object_document_id, predicate);
CREATE UNIQUE INDEX triples_dedup ON triples (
    COALESCE(subject_entity_id,0), COALESCE(subject_document_id,0), predicate,
    COALESCE(object_entity_id,0), COALESCE(object_document_id,0), COALESCE(object_literal,''),
    COALESCE(document_id,0)
) WHERE invalidated_at IS NULL;   -- a superseding works_at may coexist with its predecessor

-- Builtin + user predicates with one-sentence definitions, embedded once (research C2).
CREATE TABLE predicate_definitions (
    predicate   text PRIMARY KEY,
    definition  text NOT NULL,
    functional  boolean NOT NULL DEFAULT false,   -- one live object per subject (works_at, lives_in)
    builtin     boolean NOT NULL DEFAULT false
);
-- User-asserted "these are different" pairs, consulted before any merge (research C3).
CREATE TABLE entity_must_not_link (a bigint, b bigint, PRIMARY KEY (a, b));
ALTER TABLE entities ADD COLUMN specificity real;      -- 1 / distinct grounding documents (research B6)
ALTER TABLE entity_aliases ADD COLUMN source text;     -- which resolver rule produced the alias

Contradiction is a rule, not a judge. A new live triple on a functional predicate with the same subject and a different object invalidates the older one (invalidated_at, invalidated_by) rather than deleting it; the model is asked only when two live objects survive. valid_from defaults to the grounding document’s date (mail Date, mtime, commit date) passed to the extractor as observation_time, so “last year” resolves (research C5). rag_related and rag_recall take as_of.

Predicates are open vocabulary from the model, normalized to a builtin set where one matches (about, mentions, authored_by, works_at, part_of, depends_on, decided, paid_to, revises, cites, replies_to, attachment_of, same_thread, same_project) with the raw form kept in meta. Normalization is EDC-style (research C2): the model writes a one-line definition for an open predicate, its embedding is matched against predicate_definitions; below a threshold the predicate stays open with normalized=false. Profiles (§4a) list the predicates each category expects (with definitions) in the prompt, which both steers the model and lets the normalizer be strict. An owner entity linked to the is_self author anchors first-person facts from authored documents (I decided… → owner —decided→ …), which is what makes this a personal knowledge graph (research C10); rag_assert(subject, predicate, object) records user-stated triples that are never overwritten.

Three producers, cheapest first:

1. Rules at ingest / link time (assigned_by='rule', deterministic, tested without a model): document —authored_by→ entity(author) from document_authors (authors are entities: entities.author_id); document —same_project→ document for the same git root; document —revises→ document for version-numbered siblings (Proposal v2.docx / Proposal v3.docx, -final, (1)), replies_to/same_thread from mail In-Reply-To and conversations; attachment_of from mail attachments.
2. Facts → triples (extract_facts job): every kept relationship/entity fact resolves its subject/object to entities side-information first (research C3): author_id/email → norm_name token-set equality (“Mark, Rick” = “Rick Mark”) → alias → KNN over entity synthetic documents ≥ 0.92 → create, never across a must_not_link pair, recording the rule in entity_aliases.source; then inserts a triple grounded on fact_id. document —about→ entity for the top-N entities by salience × specificity (so “email” and “meeting” don’t win). The [0.80, 0.92) band is left for the canonicalize_entities dream job — batched yes/no merge judgments (research C4) — and gets a soft similar_to edge meanwhile (research B6).
3. link_documents dream job (assigned_by='dream'): for a document, KNN over summaries (top 10, ≥ 0.85) then a single model call given both summaries asking whether and how they relate, restricted to the predicate set; anything below confidence 0.6 is dropped. This is what finds “this design doc and that thread are about the same decision” across sources.

Surfaces: rag_related(document_id, predicates?, hops=1) → documents grouped by predicate; rag_entity(name) → entity, aliases, top documents, triples; rag_search(expand_related=true) runs Personalized PageRank over entities + documents (research B6: damping 0.5, document node weight 0.05, seeds = query’s entity hits weighted by specificity, adjacency cached in-process from triples and rebuilt by dreaming) and adds the top documents as one more RRF list (off by default; it’s a recall lever, not a ranking one). HippoRAG 2’s LLM triple filter is skipped — its ablation shows +0.7. garage triples / garage entity CLI; gRPC GetRelated, GetEntity; the app’s document detail gains a Related section. Memory evidence (§3) may reference triples.

Privacy: a triple grounded on a communication document is itself communication-class (document_id is the grounding), and rag_related from a non-communication document into a communication one is allowed locally — everything here is local — but an insight (§5) built from it inherits the class as already specified.

Tests: test_triples.py: node check constraints; entity resolution order (exact → alias → KNN → create) is idempotent on re-run; rule producers (revises sibling detection, thread linking); predicate normalization keeps raw form; rag_related hop expansion and dedup.

Surfaces

rag_search(category=...) and rag_list_categories() (with counts); garage categorize --source; gRPC ListCategories/SetDocumentCategory; Sources/Documents views show category chips with a filter.

Tests

test_categorize.py: path table precedence; project leaf auto-creation is idempotent; model assignment respects user overrides and the confidence floor; summary presence gates the job.

5. Dreaming mode

Goal: the Mac spends its idle time turning documents into understanding — without the user noticing fans, battery drain, or a stalled UI.

Mac app owns idleness; Python owns work

Idle detection needs macOS APIs, so it lives in GarageApp as IdleMonitor (new Services/IdleMonitor.swift), publishing isIdle:

• user idle: CGEventSource.secondsSinceLastEventType(.combinedSessionState, .null) ≥ dream.idle_minutes (default 10);
• power: on AC unless require_ac_power is off (IOPSCopyPowerSourcesInfo), and not isLowPowerModeEnabled;
• thermal: ProcessInfo.thermalState ≤ .fair;
• app state: Postgres running, no ingest/backfill/scan running;
• optional: screen locked or display asleep counts as idle immediately.

Scheduling uses NSBackgroundActivityScheduler (qualityOfService = .background, repeats = true, interval = 15 min, generous tolerance) so the OS coalesces us with other background work; when it fires and isIdle, AppState.startDreaming() opens the streaming Dream RPC. The Python process runs at background QoS (setpriority/nice 15), single worker.

Wake-up is fast: IdleMonitor polls every 5 s while dreaming; the first user event cancels the RPC. Python checks the stop flag between jobs (each job is one document, so typically < 30 s at the model sizes below); the current job finishes or rolls back, and the model is unloaded from Ollama (keep_alive: 0) so the user’s own LLM use isn’t fighting for memory.

Existing interval maintenance stays as-is; dreaming is an additional trigger with stricter gates. runScheduledMaintenance runs dream only if idle, so a user who never leaves the machine still gets ingest/backfill but not the heavy passes.

The passes (job kinds, in order)

Corpus-wide passes (consolidate, synthesize) are scheduled once per session at the end, and only when per-document backlog is below a threshold, so a first night on an 18k-document corpus spends itself on summaries rather than on insights over an incomplete picture.

Deep thoughts — opinions about the user

An insight is about the corpus. A deep thought is about you: a distilled, second-person opinion drawn from facts gathered across documents — “you have numerous repositories cloned with no changes of your own”, “most of your notes on the Garage project are decisions with no follow-up”, “you receive far more from Acme than you send them”. It is the one place the system is allowed to have an opinion, so it is built to be checkable, sparse, and easy to dismiss.

Probes, then synthesis. The example isn’t found by reading text; it’s found by a query over what ingest already knows (repos in sources, document_authors with is_self). So a thought starts as a probe — a named, data-driven observer in enrich/probes/ (the same table-driven style as attribute/pathrules.py), each returning a small result set and a boolean “notable”:

Probes are SQL (NREM-cheap) and run every dream session; the deep_thoughts job (REM) takes only the notable results, adds the top facts/triples behind them, and asks dream.model for a second-person, one-paragraph opinion with the probe’s rows as evidence and a confidence. Users and builtins can add probes; each carries a description that goes into the prompt so the model knows what the numbers mean.

Storage and hygiene. A thought is memories(kind='thought', origin='dream') with memory_evidence pointing at the documents the probe surfaced, plus attributes.probe, attributes.probe_hash (hash of the notable rows) and attributes.stance (observation | suggestion). Rules that keep it from becoming noise:

• Sparse: at most dream.max_thoughts_per_session (default 3) per night, highest probe-weight first; a probe that produced a thought waits probe.cooldown_days (default 30) before it can again.
• Never repeated: a new thought whose probe_hash matches a live one is a NOOP; one whose rows changed by < 20% updates the existing thought (same Mem0 ADD/UPDATE/NOOP decision as everything else).
• Self-resolving: the probe re-runs each session; when its condition no longer holds the thought is archived with status='resolved' — the journal shows “you cleaned up 12 of those clones”, which is the payoff for having the probe be data, not prose.
• Dismissible with teeth: “not useful” in the app archives the thought and lowers that probe’s weight; three dismissals disable the probe until re-enabled in settings. “Useful” bumps the probe and rehearses the thought.
• Never egresses: a thought whose probe touched communication data is communication-class, like any derived memory (§3). Probes over messages are opt-in (dream.thoughts_over_communications, default false) because “you ignore X” is a sharper thing to say than “you have unrevised drafts”.
• Recallable: rag_recall(kinds=['thought']) and a rag_thoughts() MCP tool that returns the live ones with their probe evidence, so an agent can say “Garage notes that you…” with a citation instead of guessing.

Tests (test_deep_thoughts.py): each builtin probe on a fixture DB (notable / not notable); cooldown and per-session cap; probe_hash NOOP and < 20% UPDATE; resolution archives when the probe clears; dismissals lower weight and disable at three; communication-class inheritance; prompt receives probe description and rows.

Models

Two settings, deliberately separate: facts.model (small, e.g. gemma2:2b, used by enrich-facts on demand) and dream.model (e.g. gemma3:12b/qwen3:8b, used for summaries, categories, linking, insights). Each is a (provider name, model ref) pair against the registry in §6, so the two can live on different providers — small model in-process on llama_xpc, large one on an Ollama box on the LAN — and the dream worker asks the provider for chat capability at start rather than failing per job.

UI

Status page gains a “Dreaming” state (current pass, jobs done / remaining, “Dream now” / “Pause dreaming” buttons); the Memories view (§3) filtered to origin=dream is the Dream journal — what Garage concluded overnight, each with its evidence links and confirm/dismiss. Deep thoughts get their own card at the top of the journal with the probe’s numbers, useful / not-useful, and a “resolved” history so the user can see what they acted on.

Tests

Python: test_dream.py (pass ordering, budget, cancel, insight citation validation drops < 2-citation output, insight of communication evidence is communication class). Swift: IdleMonitorTests with injected clock/power/thermal providers; scheduler start/cancel state machine.

6. N providers

Goal: any number of named inference providers, each usable for whichever roles it supports, with llama_xpc (the in-app engine) as the default and Ollama / LM Studio / any OpenAI-compatible local server as alternatives — configured once, referenced by name everywhere.

Config — providers section

"providers": [
  { "name": "local",    "kind": "llama_xpc", "default": true },
  { "name": "ollama",   "kind": "ollama",    "host": "http://localhost:11434" },
  { "name": "studio",   "kind": "openai",    "host": "http://localhost:1234/v1", "token_file": "~/.lmstudio-token" },
  { "name": "workshop", "kind": "ollama",    "host": "http://workshop.local:11434", "allow_remote": true }
]

• kind ∈ llama_xpc | ollama | openai (LM Studio, llama.cpp server, vLLM — anything OpenAI-compatible; lmstudio accepted as an alias). Kinds are the implementations; names are what everything else references. Several entries of one kind are fine.
• The llama_xpc entry is implicit if absent, so a config with no providers key behaves as today. Existing ollama_host, lmstudio_host, lmstudio_api_token_file keep loading and materialize as entries named ollama / lmstudio (deprecated aliases, warned once; removed in 1.6) — config raises on unknown keys, so this is an explicit alias path in SECTIONS, not leniency.
• allow_remote defaults false: a non-loopback host is refused at load. When true, that provider is never handed communication-class content — a fifth line in the egress guard, enforced in the provider registry (which is the one place every model call goes through) and covered by test_egress_block.py. The registry builds its clients through net/egress.py, whose allowlist then comes from the registry’s hosts instead of ollama_host / lmstudio_host. (The cloud OCR path has been removed; OCR is Tesseract only.)

Code — garage_rag/providers/

Replace the two hard-coded sets with one registry:

providers/base.py      Provider ABC: name, kind, capabilities() -> {embed, chat, json}
                       embedder(model_ref) -> Embedder          (today's embed/base.Embedder)
                       chat(model_ref, messages, *, json_schema=None) -> ChatResult
                       langextract_model(model_ref) -> BaseLanguageModel
                       probe() -> ProviderHealth (reachable, loaded models)
providers/llama_xpc.py wraps LlamaXPCClient (embed + chat)
providers/ollama.py    wraps the ollama SDK (embed + chat) and enrich/ollama_provider.py (facts)
providers/openai.py    OpenAI-compatible /v1 over httpx from net/egress (embed + chat), no SDK; LM Studio is this with a token
providers/registry.py  get_provider(name) / default_provider() / for_role("facts") ...

• embed/factory.get_embedder(provider, model_ref) becomes a shim over the registry; embedding_models.provider is reinterpreted as a provider name (existing values ollama, lmstudio, llama_xpc coincide with the alias names, so no data migration).
• enrich/facts.py drops FACT_DISTIL_PROVIDERS and takes a provider name; LlamaXPCLanguageModel moves under providers/llama_xpc.py alongside an OpenAILanguageModel for LangExtract so facts work on LM Studio too.
• Every role setting is (provider, model): default_embedding_model unchanged (the model row names its provider), facts.provider/model, dream.provider/model. garage providers list probes each (reachable, models loaded, capabilities) and `garage register-model –provider ` validates the name against the registry. The Mac app's Models view lists providers from `ListProviders` and lets the user add an Ollama/OpenAI-compatible endpoint.

Tests

test_providers.py: registry builds from config; implicit llama_xpc default; legacy host keys alias into entries and warn once; duplicate names rejected; non-loopback host refused without allow_remote; remote provider refuses communication content; get_embedder shim resolves registered models by provider name; capability probe gates roles.

7. Cross-cutting work

• Migrations 008–014 (enrich_queue, search_events, facts_salience, chunk_context, memories, categories, triples), each idempotent; db/models.py mirrors; test_migrate.py applies twice.
• Config: providers, dream (incl. max_thoughts_per_session, thoughts_over_communications, probes), search (log_enabled, log_retention_days), facts, memory (incl. stability_days, stability_growth, archive_below), categories sections in SECTIONS; regenerate garage.schema.json; every field documented (existing test enforces).
• Proto: Dream, Remember, Recall, ListMemories, ForgetMemory, StoreText, ListCategories, SetDocumentCategory, GetRelated, GetEntity, AssertTriple, ReadDocument, ListProviders; DocumentDetail gains summary, categories; DocumentFactInfo gains salience, kind, evidence_count. Regenerate *_pb2*.
• Egress: test_egress_block.py unchanged in mechanism; add assertions that enrich/dream.py, enrich/memory.py, enrich/categorize.py import no HTTP client other than the Ollama/LlamaXPC ones, and that EgressRequest refuses memory inputs of communication class.
• Docs: docs/memory.md, docs/dreaming.md, docs/categories.md, docs/providers.md, docs/triples.md; update architecture.md (new derived layer + queue diagram) and schema.md; CLAUDE.md architecture section.
• BUILD: py_test entries for the twelve new test files; aspect gazelle for new modules.

8. Milestones

Each milestone is independently shippable and leaves aspect test //... green.

M3/M4 are parallelizable once M1 lands; M2 needs M4’s profiles. Suggested order if serial is M0 → M1 → M4 → M2 → M3 → M4b → M5 (categories before facts because profiles drive distillation; memory before triples because canonical-fact dedup, memory consolidation and entity resolution share one KNN-merge helper).

9. Risks

• Throughput. One LLM call per document over a ~20k-document corpus is many idle hours at 8–12B parameters. Mitigations are built in: priority by recency×authored, per-session budget, summaries before insights, and the small facts.model for the on-demand path. Ship with the expectation that the first full pass takes several nights and say so in the UI.
• Salience quality at 2B. gemma2:2b judging salience may be noisy. The deterministic post-filter is the backstop; the -m model regression fixture is how we measure prompt changes. If it isn’t good enough, facts moves entirely to dream.model.
• JSON-output fragility. All model output goes through LangExtract (facts) or a strict Pydantic parse with one retry (summaries/categories/insights); a parse failure is a failed job with the raw output in error, never a partial write.
• Agent writes. rag_remember is the first way an MCP client mutates the corpus. Bounds, origin tagging, the HTTP-transport gate, and trust_tier=reference until confirmed keep a misbehaving client from polluting authored.
• Idle heuristics. False “idle” (user watching a video) is annoying; false “busy” is merely slow. Bias towards not dreaming: AC-only by default, thermal .fair ceiling, instant cancel.

10. Decisions to confirm

Recommendations are stated; these change scope if the answer differs.

1. Canonical facts in 1.5? Recommended yes (M2): without cross-document dedup, “salient” only halves the row count rather than collapsing repeated claims. Alternative is per-document salience only, deferring dedup to 1.6.
2. Taxonomy: fixed builtin + user-extensible (recommended) vs. emergent clustering. Emergent categories are a 1.6 candidate once summaries exist for the whole corpus.
3. MCP memory writes default on for stdio, off for HTTP. Confirm, or default off everywhere.
4. Dream on battery: off by default. Confirm.
5. dream.model default. Recommend gemma3:12b when ≥ 32 GB unified memory, else qwen3:8b; the app can pick at first run based on ProcessInfo.physicalMemory.
6. Forgetting. Recommend archive (never delete) for agent/dream memories, and never auto-touch user memories. Confirm the 180-day / importance < 0.3 defaults.
7. Legacy provider keys. Recommend keeping ollama_host/lmstudio_host as deprecated aliases for 1.5 (warn once) and removing in 1.6, rather than a hard break now.
8. Predicate vocabulary. Recommend the closed builtin set with raw form retained, over fully open predicates; open predicates make rag_related grouping and profile steering much weaker.
9. Entities as synthetic documents (for KNN alias resolution) — same trick as memories. Confirm you’re fine with entities and memories both adding rows to documents.

11. Adopted from the literature

v1.5-research.md surveys ~45 papers across agent memory, retrieval architecture and knowledge-graph construction and triages each as ADOPT / 1.6 / REF. The ADOPT items are already folded into the sections above; this is the index:

Parked for 1.6 (see the research doc’s deferred list): A-MEM neighbour evolution, late chunking via llama_xpc, LightRAG theme keys, RAPTOR level 2 / TnT-LLM taxonomy generation / BERTopic emergent categories, two-stage quantized KNN, hierarchical digests, conformal category thresholds, OpenIE6 fallback, MemoRAG clues. (Reflexion-style lessons moved into 1.5 once the query log existed — §1.)