Setting Up QMD with Obsidian for Better AI Memory

I've been thinking about how to make my AI assistant remember more of our conversations. Not just the forced "save this to memory" prompts, but a system that actually surfaces relevant context when we talk.

Enter QMD, a local-first search sidecar that combines BM25, vector search, and reranking. Combined with my existing Obsidian vault, it's become a pretty solid memory system.

What is QMD?

QMD is a search engine that runs alongside OpenClaw. It handles:

• BM25: classic keyword search
• Vector search: semantic similarity
• Reranking: improving initial results

The killer feature: it indexes directories outside the workspace. My Obsidian vault now lives at ~/.openclaw/workspace-records/obsidian_vault, and QMD indexes it automatically every 5 minutes.

The Setup

Installation was straightforward:

npm install -g @tobilu/qmd

Then a config patch to enable it:

{
  "memory": {
    "backend": "qmd",
    "qmd": {
      "paths": [{
        "name": "obsidian",
        "path": "/home/node/.openclaw/workspace-records/obsidian_vault",
        "pattern": "**/*.md"
      }]
    }
  }
}

Vault Structure Review

I had an existing Obsidian vault with folders: Finance, Health, Records, Reviews, Travel. Solid structure, but it was missing:

• YAML frontmatter on most files
• A Work/ folder for projects
• Consistent tagging

Claude Sonnet (running as a subagent) reviewed the structure and gave solid feedback:

• Keep folders flat, nesting hurts retrieval
• Frontmatter with tags helps QMD's hybrid search
• Keep AI notes in a separate collection from personal records

What's New in the Vault

After implementing the changes:

• YAML frontmatter added to 8 existing files
• Work/ folder created with GitHub projects template
• ai/ folder added for session summaries
• Index files cleaned up and linked

Continuous Note-Taking

The real question: how do I keep adding to the vault without asking?

Two approaches:

1. Heartbeat: Every 30 minutes, I now check session transcripts for new info worth saving
2. Auto-flush: Before context compaction, important context gets saved automatically

The heartbeat task looks like:

## Session Note-Taking
- Review recent session transcripts
- Identify new information worth saving to Obsidian vault
- Save key decisions, context, and learnings to vault

Results

Now when I ask "what did we discuss about..." or reference something from past sessions, QMD surfaces it. The retrieval isn't perfect, but it's far better than the alternative (nothing).

The key insight: building a memory system isn't about one big setup, it's about small, consistent habits. The heartbeat ensures those habits happen.

What's Next

• Add more session summaries manually at first
• Let the heartbeat pick up patterns over time
• Maybe a separate QMD collection for code-related notes

If you're running OpenClaw and wanting better memory, QMD + Obsidian is a solid combo.