v4

That's the V4 docs outline — looks like the high-level architecture for the reasoning system. Let me flesh this out into proper documentation.

---

docs/v4.md — Memory Daemon V4: Reasoning Infrastructure

# Memory Daemon V4

## Reasoning Infrastructure

Version 4 introduces a reasoning layer on top of the memory system. This is not just retrieval — it's active reasoning over memory.

---

## Core Concepts

### Goals

Goals are the top-level unit of reasoning. A goal represents something the system is trying to achieve or understand.

- `GoalRecord` stored in SQLite
- Tracks progress and status
- Can be active, completed, or archived
- Goals inform retrieval and reasoning priority

### Blackboard

A shared space where reasoning components post and consume information.

- Thread-safe, in-memory
- Entries are typed and timestamped
- Subscriptions for event-driven workflows
- Used for communication between reasoning nodes

### Task Scheduler

Orchestrates parallel execution of reasoning tasks.

- Thread pool with configurable workers
- Task priority support
- Async task submission and result collection
- Used for concurrent retrieval, ranking, and reasoning

### Memory Manager

Unified interface for all memory operations.

- Wraps MemorySystem
- Handles storage, retrieval, and maintenance
- Provides clean API for reasoning nodes
- Manages memory lifecycle

### Reasoning Nodes

Atomic reasoning units that can be composed into workflows.

- Each node does one thing well
- Examples: Query, Summarize, Compare, Synthesize, Decide
- Nodes can be chained into computation graphs
- Results are posted to the blackboard

### Computation Graph

A directed graph of reasoning nodes.

- Defines the flow of reasoning
- Supports branching and merging
- Nodes execute in parallel where possible
- Graph is serializable for reproducibility

### Planner

Translates high-level goals into execution plans.

- Takes a goal and context
- Produces a computation graph
- Optimizes for performance and accuracy
- Uses feedback to improve planning

### Execution Queue

Manages the execution of computation graphs.

- Priority queue for pending work
- Handles retries and failures
- Reports progress and results
- Tracks execution history

---

## System Architecture

┌─────────────────────────────────────────────────────────────┐ │ User │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Memory Controller │ │ (remember, recall, chat, reflect) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────┼─────────────────────┐ ▼ ▼ ▼ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ Goals │ │ Planner │ │ Execution │ │ (active goals│ │ (maps goals │ │ Queue │ │ and status) │ │ to graphs) │ │ (pending and │ └───────────────┘ └───────────────┘ │ running) │ │ │ └───────────────┘ │ │ │ └─────────────────────┼────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Blackboard │ │ (shared state, event-driven, thread-safe) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────┼─────────────────────┐ ▼ ▼ ▼ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ Reasoning │ │ Task │ │ Computation │ │ Nodes │ │ Scheduler │ │ Graph │ │ (atomic │ │ (parallel │ │ (flow of │ │ operations) │ │ execution) │ │ reasoning) │ └───────────────┘ └───────────────┘ └───────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Memory System │ │ (retrieval, ranking, feedback, graph, vector store) │ └─────────────────────────────────────────────────────────────┘

---

## Component Relationships

| Component | Depends On | Provides |
|-----------|------------|----------|
| **Goals** | — | High-level objectives |
| **Planner** | Goals, Blackboard | Execution plans (graphs) |
| **Execution Queue** | Planner | Scheduled work |
| **Task Scheduler** | Execution Queue | Parallel execution |
| **Reasoning Nodes** | Memory Manager, Blackboard | Atomic operations |
| **Computation Graph** | Reasoning Nodes | Workflow definition |
| **Blackboard** | — | Shared state |
| **Memory Manager** | MemorySystem | Memory operations |

---

## Example Reasoning Flow

1. User: "What should I work on next?" 2. Controller → Planner 3. Planner inspects active goals 4. Planner builds computation graph: ┌─────────────────────────────────────────────────────────────┐ │ 1. Query Goals │ │ 2. Query Recent Activity │ │ 3. Compare Goals vs Activity │ │ 4. Synthesize Recommendation │ │ 5. Store Reflection │ └─────────────────────────────────────────────────────────────┘ 5. Graph → Execution Queue 6. Scheduler runs nodes in parallel where possible 7. Results posted to Blackboard 8. Final recommendation returned to user

---

## Key Design Decisions

### 1. Goal-Driven Reasoning
Rather than reactive retrieval, the system actively reasons about what it should do based on its goals.

### 2. Blackboard as Shared Memory
All reasoning components communicate through the blackboard. This decouples components and enables event-driven workflows.

### 3. Parallel Execution by Default
The task scheduler runs reasoning nodes in parallel. This makes the system fast even with complex graphs.

### 4. Pluggable Reasoning Nodes
Nodes are modular and can be added or replaced without changing the core system. This supports experimentation.

### 5. Graph-Based Execution
Computation graphs make reasoning flows explicit, reproducible, and optimizable.

---

## V4 Features

| Feature | Status | Description |
|---------|--------|-------------|
| Goals | ✅ Complete | Store and track goals |
| Blackboard | ✅ Complete | Shared state with subscriptions |
| Task Scheduler | ✅ Complete | Parallel execution with priorities |
| Memory Manager | ✅ Complete | Unified memory interface |
| Reasoning Nodes | 🚧 In Progress | Atomic reasoning units |
| Computation Graph | 🚧 In Progress | Directed graph of nodes |
| Planner | 🔜 Planned | Maps goals to graphs |
| Execution Queue | 🔜 Planned | Manages pending work |

---

## Next Steps

1. ✅ Memory System (V4 foundation)
2. ✅ Blackboard + Scheduler
3. ✅ Query + Ranking Pipeline
4. 🔄 Reasoning Nodes (current)
5. 🔄 Computation Graph (current)
6. 🔜 Planner
7. 🔜 Execution Queue
8. 🔜 Full integration and testing

---

## Version History

| Version | Focus |
|---------|-------|
| V1 | Core memory + basic retrieval |
| V2 | Ranking pipeline + feedback |
| V3 | Type routing + BM25 + inverted index |
| V4 | Reasoning infrastructure + goals + blackboard |
| V5 | Hierarchical memory + compression (research) |

---

## See Also

- `03_system_architecture.md` — Detailed system architecture
- `05_Design_Principles.md` — Design principles and tradeoffs
- `v5.md` — V5 research proposal

---

Next?

Want me to continue with the other docs? Or is this a good start for V4?