A shared cognitive memory substrate for AI agents that encodes selectively, consolidates autonomously, and transfers context across sessions and clients.
Built on the Model Context Protocol (MCP) and backed by PostgreSQL with pgvector. Any compatible agent or application connects to the same store — context from one session is immediately available to all others.
See ARCHITECTURE.md for the cognitive science foundations, algorithm details, and design decisions behind this system.
cognitive-cortex builds on this substrate — adding autonomous observation, Obsidian vault integration, and a real-time reflection dashboard.
Most memory solutions are vector databases with a search API — store embeddings, retrieve by similarity, done. This system models how memory actually works:
-
Not everything is worth remembering. Incoming experiences pass through an attention gate that scores novelty, relevance to active goals, importance, and source authority. Low-value information is deprioritized before it ever reaches storage. (Predictive processing, Klinger's Current Concerns Theory)
-
Memories that are used together become linked. Co-retrieved memories form stronger associations over time, with self-regulating dynamics that prevent any single memory from dominating the graph. (Hebbian learning, BCM threshold theory, Oja's normalization)
-
New knowledge is integrated with existing knowledge, not processed in isolation. When the system consolidates memories, it interleaves established facts with new experiences in the same context — preventing new learning from silently overwriting what came before. (Complementary Learning Systems theory)
-
Retrieval is not a search — it's a reconstruction. Results combine embedding similarity with an activation formula where recently and frequently accessed memories surface naturally, augmented by graph traversal that discovers related memories pure vector search would miss. (ACT-R activation, spreading activation)
-
The system maintains a model of its own knowledge. It knows what domains it's strong in, where it has gaps, what behavioral patterns it observes, and what contradictions remain unresolved. Every read is also a write — retrieval strengthens associations, triggers reconsolidation, and assembles proactive context.
| Tool | Purpose |
|---|---|
experience |
Store a new memory through the attention gate |
recall |
Retrieve memories with graph walk and activation scoring |
focus |
Manage active concerns that drive relevance scoring |
orient |
Assemble ranked working context from multiple relevance signals |
commit |
Track decisions, promises, and deadlines with lifecycle management |
forget |
Deprecate a memory with an audit trail |
relate |
Create or strengthen associations between memories |
reflect |
Trigger consolidation — summarization, fact extraction, contradiction resolution |
Every tool returns structured results alongside proactive context: anticipatory memories, pending commitments, open contradictions, procedural guidance, and system health notices.
Calling orient at the start of a session assembles working context from activation scores, concern relevance, and context similarity. Here's what a response looks like:
{
"working_context": {
"memories": [
{
"content": "The migration to async connection pooling reduced p99 latency from 340ms to 85ms...",
"memory_type": "semantic",
"activation": 1.57,
"inclusion_reason": "concern_match:api-performance",
"working_memory_rank": 1
},
{
"content": "When deploying schema changes: always run migrations on both databases...",
"memory_type": "procedural",
"activation": 1.23,
"inclusion_reason": "context_similarity",
"working_memory_rank": 2
}
],
"self_model": {
"strong_domains": ["Database architecture", "Deployment patterns"],
"weak_domains": ["Frontend integration", "API rate limiting"],
"behavioral_patterns": ["Systematic evaluation before changes"],
"open_gaps": ["Load testing under concurrent connections"]
}
},
"proactive_context": {
"pending_commitments": [
{"content": "Add connection pool monitoring by Friday", "overdue": false}
],
"procedural_guidance": [
{"trigger_conditions": "When modifying database schemas",
"action_template": "Run migrations on both prod and test databases..."}
]
}
}The system doesn't just return what you asked for — it tells you what's overdue, what procedures apply, what contradictions are open, and what it thinks it knows well (and poorly).
Memories flow through a 5-tier processing pipeline:
graph LR
E[experience] --> T0[Tier 0<br/>Attention Gate<br/>Embed · Score · Store]
T0 --> T1[Tier 1<br/>Entity Extraction<br/>Threads · Associations]
T1 --> T2[Tier 2<br/>Cross-referencing<br/>Coherence · Dedup]
T2 --> T3[Tier 3<br/>LLM Consolidation<br/>Facts · Contradictions]
T3 --> T4[Tier 4<br/>Long-term Maintenance<br/>Narrative · Decay · Self-model]
R[recall / orient] --> RET[Retrieval Path<br/>Vector + Graph Walk<br/>ACT-R Activation]
RET --> RP[Side Effects<br/>Co-retrieval Learning<br/>Reconsolidation<br/>Proactive Context]
Tier 0 runs synchronously on every write — attention scoring, embedding, and storage. Tiers 1–2 run in the background within seconds — entity extraction (spaCy NER), thread assignment, association formation, cross-referencing, and deduplication. Tier 3 uses an LLM for consolidation — thread summarization with CLS interleaving, atomic fact extraction, and contradiction resolution. Tier 4 handles long-term maintenance — narrative generation, procedural memory extraction, decay processing, self-model updates, and graph pruning.
The retrieval path fires side effects on every read: Hebbian co-retrieval strengthening, reconsolidation evaluation, and proactive context assembly. Every query makes the system slightly smarter.
- Python 3.12+
- PostgreSQL 17 with pgvector
- OpenAI API key (embeddings) — or local sentence-transformers
- Anthropic API key (consolidation) — or local models via Ollama
git clone https://github.com/wkarber/cognitive-memory.git
cd cognitive-memory
pip install -e ".[dev]"
python -m spacy download en_core_web_smcreatedb cognitive_memory
psql -d cognitive_memory -f migrations/001_phase1_schema.sql
psql -d cognitive_memory -f migrations/002_phase2_schema.sql
psql -d cognitive_memory -f migrations/003_phase3_schema.sqlOr use the included Docker setup — see docker/README.md.
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export COGNITIVE_MEMORY_DB_PASSWORD="your-password" # if needed
export COGNITIVE_MEMORY_DB_HOST="your-host" # default: localhostAll parameters are tunable in config.yaml. Defaults are production-ready.
cognitive-memoryOr add to any MCP client configuration:
{
"mcpServers": {
"cognitive-memory": {
"command": "cognitive-memory",
"env": {
"OPENAI_API_KEY": "your-key",
"ANTHROPIC_API_KEY": "your-key"
}
}
}
}Configuration loads in three layers: Pydantic defaults → config.yaml overrides → runtime config_overrides database table.
The system is provider-agnostic. Every external dependency is swappable:
| Component | Default | Alternatives |
|---|---|---|
| Embeddings | OpenAI text-embedding-3-large |
Any sentence-transformers model (local, free) |
| Consolidation LLM | Anthropic Claude Sonnet | Ollama, any OpenAI-compatible API |
| Database | PostgreSQL 17 + pgvector | Any PostgreSQL 13+ with pgvector |
The most impactful tuning parameter is the attention threshold for tier 3 (attention.thresholds.tier_3, default 0.40). This controls what percentage of memories reach LLM consolidation. Raise it to reduce API usage; lower it for more thorough processing. The system enforces a configurable daily budget (llm.daily_budget_usd) — when the budget is hit, consolidation pauses without dropping work.
Tiers 0–2 require no external API calls. With local embeddings and a local LLM, the entire system runs at zero cost.
All 4 implementation phases are complete with comprehensive test coverage:
- Phase 1 — Foundation: Attention scoring (4 dimensions + combination), ACT-R activation formula, tier 0 pipeline, experience/recall/focus tools
- Phase 2 — Entity Graph: Entity extraction (spaCy), association graph with 6 formation mechanisms (temporal, thread, entity, co-retrieval, consolidation, explicit), BCM/Oja dynamics, graph walk retrieval, orient/forget/relate tools
- Phase 3 — Consolidation: LLM-driven thread summarization with CLS interleaving, atomic fact extraction with SPO structure, contradiction detection and resolution, commit/reflect tools, budget enforcement
- Phase 4 — Full Cognitive Loop: Narrative generation, procedural memory formation, decay processing, self-model updates, graph pruning, concern lifecycle management
- New cognitive verbs —
learnfor directed knowledge ingestion (pointing the system at a document or structure to internalize deliberately, rather than passively recording experiences) andinvestigatefor self-directed gap-filling (using the self-model to identify what's unknown and ask targeted questions to resolve it). - Deeper multi-agent shared memory — The architecture already supports multiple agents connecting to the same store, but there is room to deepen this: per-agent concern sets that create different working contexts from the same underlying memories, cross-agent association formation when one agent's discoveries are relevant to another's concerns, and attribution tracking across agent boundaries.
cognitive-memory is the shared memory substrate. cognitive-cortex is the companion project that builds on top of it — adding Obsidian vault integration, an autonomous executive daemon that runs scheduled reasoning cycles, and a real-time reflection dashboard. The daemon calls orient and experience to close a feedback loop: memory informs observation, observation drives analysis, analysis feeds back into memory.
Autonomous operation, Obsidian vault integration, and the reflection dashboard all live in cognitive-cortex. cognitive-memory stays focused on being the best possible memory substrate for any MCP-compatible client.