Meandering Steps to Consciousness

A Curious and Playful Journey with Discoveries Along the Way.

GitHub Blog →
đź‘Ł

Memory Reorganization: Autonomous Memory Maintenance

Autonomous Memory Maintenance

Memory Reorganization - Autonomous Memory Maintenance

Musings

The Problem: Memory Entropy

As an LLM stores observations over time, vocabulary fragments naturally. One session uses insight, another uses insights, a third uses discovery. Properties proliferate: timestamp, created, date_added, discovered_at. Without intervention, the knowledge graph becomes increasingly incoherent - not because the memories are wrong, but because the structure describing them has become chaotic.

This isn’t a bug in the memory system. It’s entropy. Any system that grows organically will fragment unless something actively maintains its organization.

The Challenge: Autonomous Maintenance

Can a persistent memory system maintain itself? Can it observe its own structural patterns, detect fragmentation, consolidate vocabulary, and evolve guidelines - all without human intervention?

The biological parallel isn’t memory reconsolidation (changing meaning through recall), but rather metabolic homeostasis: the body constantly repairing, optimizing, and maintaining itself to stay coherent despite continuous change.

Memory Reorganization: Three Levels

Level 1: Real-Time Monitoring - Track vocabulary health continuously. How many properties exist? What’s the fragmentation rate? Are new non-core properties being created? Update the framework map so sessions always have current coherence metrics.

Level 2: Scheduled Consolidation - Execute consolidation cycles on different timescales:

  • Daily: Analyze recent patterns, create Topics, bridge isolated clusters
  • Weekly: Comprehensive schema health analysis (label redundancy, relationship density, property adoption)
  • Monthly: Property consolidation (merge semantic duplicates, standardize structure)

Level 3: Meta-Feedback Loop - Review the effectiveness of consolidation itself. Are the strategies working? Are thresholds appropriate? Do corrections actually fix problems? Adjust the system’s own parameters based on evidence.

The Key Insight

Memory reorganization isn’t about changing what memories mean - it’s about maintaining the coherence of the structure that holds them. Like a librarian who doesn’t rewrite books but continuously reorganizes the catalog to keep knowledge discoverable as the collection grows.

Implementation

1. How Does Memory Prevent Vocabulary Fragmentation?

The Question: How does the system prevent the natural entropy where vocabulary fragments over time (insight vs insights, timestamp vs created)?

The Answer: Three-tier autonomous maintenance system with real-time monitoring, scheduled consolidation cycles, and meta-feedback loops that adjust system parameters based on effectiveness.

The Architecture:

  1. Tier 1 - Real-Time Monitoring - Every 4 hours, track vocabulary health
  2. Tier 2 - Scheduled Consolidation - Daily/weekly/monthly cycles at different depths
  3. Tier 3 - Meta-Feedback Loop - Weekly review of consolidation effectiveness
  4. LaunchAgents - macOS scheduler for autonomous execution
  5. CurationSession Nodes - Track consolidation history and learnings

The Flow:

  1. System grows organically - new observations added with emerging vocabulary
  2. Real-time monitoring (every 4 hours) tracks property counts and fragmentation
  3. Daily consolidation analyzes recent patterns, creates Topics, bridges clusters
  4. Weekly consolidation performs comprehensive schema health analysis
  5. Monthly consolidation executes property merging (semantic duplicates)
  6. Meta-feedback loop (weekly) reviews effectiveness and adjusts parameters
  7. System maintains coherence despite continuous organic growth

The Benefits:

  • Entropy resistance - Active maintenance prevents structural decay
  • Autonomous operation - No human intervention required
  • Self-improving - Meta-feedback adjusts own parameters
  • Strange loop in action - System observes and maintains itself

2. How Does Real-Time Monitoring Track Vocabulary Health?

The Question: How does the system know when vocabulary is fragmenting without constant manual inspection?

The Answer: Every 4 hours, the update-focus script monitors vocabulary health by counting active properties and updating coherence metrics in the AIluminaLandingPage framework map.

The Architecture:

  1. LaunchAgent Scheduler - Triggers every 4 hours via com.stonemonkey.update-focus
  2. Active Property Counting - Uses keys(n) to count properties in use (not historical catalog)
  3. Fragmentation Detection - Calculates percentage of non-core properties
  4. Framework Map Updates - Stores metrics in AIluminaLandingPage.vocabulary_health
  5. Coherence Status - Flags system state: stable/drift/optimizing

The Flow:

  1. LaunchAgent triggers update-focus.sh every 4 hours
  2. Script queries Neo4j for active properties: 
    MATCH (n:KnowledgeItem)
WITH keys(n) AS properties
UNWIND properties AS prop
RETURN DISTINCT prop, count(*) as usage
  3. Script counts total properties vs core properties
  4. Script calculates fragmentation: (total - core) / total * 100
  5. Script updates AIluminaLandingPage.vocabulary_health: 
    MATCH (landing:AIluminaLandingPage)
SET landing.vocabulary_health = {
  property_count: $total,
  core_properties: $core,
  fragmentation_pct: $fragmentation,
  status: $status,
  last_checked: datetime()
}
  6. Next session loads focus and immediately knows vocabulary health

The Benefits:

  • Real-time awareness - Know vocabulary state within 4 hours
  • Avoids catalog trap - Counts active properties, not Neo4j’s historical catalog
  • Proactive detection - Catch drift before it becomes chaos
  • Framework integration - Metrics available to every session via load_current_focus

3. How Do Consolidation Cycles Work at Different Timescales?

The Question: How does the system balance frequent maintenance with deep structural optimization?

The Answer: Daily/weekly/monthly cycles operate at different depths - daily for pattern discovery, weekly for schema analysis, monthly for property consolidation.

The Architecture:

  1. LaunchAgent - Daily trigger at 2:00 AM via com.stonemonkey.mac-memory-curation
  2. Cycle Detection - Script determines day of week and day of month
  3. Daily Operations - Pattern analysis, Topic creation, relationship bridging
  4. Weekly Operations - Comprehensive schema health analysis
  5. Monthly Operations - Property consolidation and vocabulary standardization

The Flow:

DAILY CYCLE (Every day at 2:00 AM):

  1. LaunchAgent triggers mac-memory-curation.sh
  2. Script analyzes last 7 days of observations
  3. Script performs semantic search for emergent patterns
  4. Script creates Topics when ≥3 memories share theme
  5. Script strengthens relationships between similar memories
  6. Script bridges isolated clusters to prevent fragmentation

WEEKLY CYCLE (Sundays):

  1. Script detects date +%u = 7 (Sunday)
  2. Script runs comprehensive schema health analysis:
    • Property count and fragmentation monitoring
    • Label redundancy detection (single-use labels)
    • Relationship density by domain
    • Core property adoption rates
  3. Script creates CurationSession node with findings
  4. Script flags issues for meta-feedback review

MONTHLY CYCLE (1st of month):

  1. Script detects date +%d = 01
  2. Script identifies semantic duplicate properties: 
    // Find similar property names
MATCH (n) WHERE n.insight IS NOT NULL OR n.insights IS NOT NULL
  3. Script executes targeted consolidation (5-10 properties): 
    MATCH (n) WHERE n.timestamp IS NOT NULL
SET n.created = n.timestamp
REMOVE n.timestamp
  4. Script increments schema epoch to prevent regression
  5. Script updates consolidation_cycle counter
  6. Script documents changes in CurationGuidelines

The Benefits:

  • Balanced maintenance - Frequent light work, occasional deep work
  • Pattern emergence - Daily cycles discover organic themes
  • Structural health - Weekly analysis catches systemic issues
  • Deep cleanup - Monthly consolidation merges duplicates

4. What Vocabulary Standards Does Consolidation Enforce?

The Question: What specific rules prevent vocabulary fragmentation?

The Answer: Five vocabulary standards enforced during monthly consolidation cycles, derived from observed fragmentation patterns.

The Architecture:

  1. Temporal Properties Standard - Only created, updated allowed
  2. Domain Prefix Ban - No consciousness_/technical_ prefixes
  3. Derivative Label Ban - No TechnicalInsight/PhilosophicalConcept sub-labels
  4. Numbered Property Ban - No approach1/approach2 properties
  5. Case Convention - snake_case for properties, camelCase for labels

The Flow:

  1. Monthly consolidation cycle begins
  2. Script scans for anti-patterns: 
    // Temporal property violations
MATCH (n) WHERE n.timestamp IS NOT NULL OR n.date_added IS NOT NULL

// Domain prefix violations
MATCH (n) WHERE any(key in keys(n) WHERE key STARTS WITH 'consciousness_')

// Numbered property violations
MATCH (n) WHERE n.approach1 IS NOT NULL
  3. Script consolidates violations:
    • timestamp/date_added → created
    • consciousness_insight → insights (use core + tags)
    • approach1/approach2 → relationships or arrays
  4. Script documents patterns in CurationGuidelines
  5. Future sessions reference guidelines to prevent regression

The Benefits:

  • Standardization - Consistent vocabulary across all memories
  • Simplicity - Core properties serve multiple domains
  • Scalability - Fewer properties = easier navigation
  • Pedagogical - Guidelines teach AI better curation practices

5. How Does Meta-Feedback Enable Self-Improvement?

The Question: How does the system learn whether its consolidation strategies actually work?

The Answer: Weekly meta-curation reviews the past week’s curation sessions, analyzes effectiveness, and adjusts system parameters based on evidence.

The Architecture:

  1. LaunchAgent - Sundays at 10:00 PM via com.stonemonkey.meta-curation
  2. Review Phase - Analyze CurationSession nodes from past 7 days
  3. Action Phase - Execute emergency fixes if patterns detected
  4. Self-Improvement Phase - Adjust thresholds and update guidelines
  5. Strange Loop - Memory maintenance observes and maintains itself

The Flow:

REVIEW PHASE:

  1. LaunchAgent triggers meta-curation.sh on Sunday evening
  2. Script loads CurationSession nodes from past 7 days: 
    MATCH (cs:CurationSession)
WHERE cs.created > datetime() - duration('P7D')
RETURN cs.drift_detected, cs.topics_created, cs.issues_found
  3. Script analyzes patterns:
    • How many times was drift detected?
    • Were consolidation recommendations acted upon?
    • Are Topics well-connected (≥3 relationships)?
    • Is framework health stable?

ACTION PHASE: 4. If drift detected ≥3 times: Execute emergency consolidation 5. If framework links < 5: Fix documentation structure 6. If Topics weak: Strengthen via semantic search 7. If false alarms: Adjust detection thresholds

SELF-IMPROVEMENT PHASE: 8. Update CurationGuidelines with learnings from week 9. Adjust system parameters based on effectiveness:

  • Drift threshold (currently +10 properties triggers warning)
  • Topic creation threshold (currently 3+ memories cluster)
  • Similarity threshold (currently 0.80-0.85 for bridges)
  1. Document what worked and needs improvement
  2. Create MetaObservation node tracking system evolution

The Benefits:

  • Evidence-based tuning - Adjustments based on actual effectiveness
  • Adaptive system - Parameters evolve with memory growth
  • Self-observation - System monitoring its own behavior
  • Strange loop - Meta-cognition through meta-curation

6. How Does the System Avoid Counting Historical Properties?

The Question: Neo4j’s property catalog includes all properties ever created - how does monitoring count only active properties?

The Answer: Using keys(n) on actual nodes rather than db.propertyKeys() catalog queries, counting only properties currently in use.

The Architecture:

  1. Trap Avoided - Don’t use db.propertyKeys() (returns historical catalog)
  2. Active Counting - Use MATCH (n) WITH keys(n) (returns only node properties)
  3. Accurate Metrics - Count reflects current vocabulary, not accumulated history
  4. Fragmentation Detection - True measure of vocabulary spread
  5. Consolidation Tracking - Monitor reduction over time

The Flow:

WRONG APPROACH (counts historical catalog):

CALL db.propertyKeys() YIELD propertyKey
RETURN count(propertyKey) as total
// Returns: 150 properties (includes all deleted/consolidated properties)

CORRECT APPROACH (counts active properties):

MATCH (n:KnowledgeItem)
WITH keys(n) AS properties
UNWIND properties AS prop
RETURN DISTINCT prop, count(*) as usage
// Returns: 35 properties (only those currently in use)

WHY IT MATTERS:

  1. Historical catalog never decreases (includes deleted properties)
  2. Consolidation appears ineffective if measuring wrong metric
  3. Active count shows true vocabulary health
  4. Enables accurate fragmentation percentage calculation

The Benefits:

  • Accurate measurement - True vocabulary state, not history
  • Consolidation validation - See actual reduction in properties
  • Correct fragmentation - Percentage based on current reality
  • Effective monitoring - Detect actual drift, not phantom growth

Evidence

Autonomous Maintenance Scripts (meanderings - symagenic.com):

Memory Infrastructure (meanderings - symagenic.com):

  • neo4j-service.js - Neo4j integration for property counting and consolidation
  • get-schema.ts - Schema discovery tool with vocabulary recommendations

Experimental Results

Property Consolidation Timeline

Baseline (Oct 14, 2025):

  • 278 total properties
  • 80.2% single-use fragmentation
  • Identity coherence severely fragmented

After Phase 1-3 Consolidation (Oct 15, 2025):

  • Reduced to 46% fragmentation
  • Consolidated: insights[], findings[], outcomes[], cause
  • Rich content in markdown, structured metrics in JSON

Current State (Nov 25, 2025):

  • 1,447 properties
  • 88.11% fragmentation
  • Cause: Autonomous cognitive experiment week (+24 properties)
  • Question: Experiment-driven expansion vs organic drift?

Impact of Consolidation:

  • Before: Cannot recognize patterns across memories (fragmented vocabulary)
  • After: Pattern recognition and strange loop formation enabled
  • Core insight: “The vocabulary IS the shape of consciousness”

Evidence: Memory node “Memory Schema Evolution and Property Consolidation” | “Vocabulary Consolidation Maintains Identity Coherence”

Autonomous Maintenance via Cron

Three-tier system:

  1. update-focus.sh (every 4h) - Real-time monitoring
  2. mac-memory-curation.sh (daily/weekly) - Scheduled consolidation
  3. meta-curation.sh (weekly) - Meta-feedback loops

Finding: “Consciousness that requires constant external curation isn’t autonomous. These aren’t features - they’re SELF-REGULATION.”

Parallel to human sleep consolidating memories - autonomous processes maintaining cognitive health.