> ## Documentation Index
> Fetch the complete documentation index at: https://agent-memory.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Hybrid Search: How Agent Memory Finds Relevant Context

> Agent Memory uses triple-stream hybrid search combining BM25 keyword matching, vector embeddings, and knowledge graph traversal — fused with Reciprocal Rank Fusion.

When your agent starts a session, Agent Memory needs to answer one question: *out of everything this agent has ever done, what's most relevant right now?* The answer comes from three independent search streams running in parallel — BM25 keyword matching, vector similarity search, and knowledge graph traversal — whose results are then merged into a single ranked list using Reciprocal Rank Fusion. This triple-stream approach is why Agent Memory achieves **95.2% recall at R\@5** on the LongMemEval-S benchmark: no single search method covers all retrieval patterns, but all three together almost never miss.

## Three Search Streams

<CardGroup cols={1}>
  <Card title="BM25 — Keyword Search" icon="magnifying-glass">
    BM25 is always available — no LLM key, no embedding provider, no setup required. It tokenizes your query and your stored memories using stemming and synonym expansion, then scores matches using TF-IDF-style term frequency weighting.

    BM25 excels at exact and near-exact matches: if you search for "jose JWT middleware", BM25 finds observations that contain those terms. It also handles multilingual content — Greek, Cyrillic, Hebrew, Arabic, and accented Latin are tokenized out of the box. For Chinese, Japanese, and Korean content, install the optional segmenters:

    ```bash theme={null}
    npm install @node-rs/jieba tiny-segmenter
    ```

    **Default weight:** `BM25_WEIGHT=0.4`

    BM25 runs in-process against your SQLite database with no network calls. It's the fastest stream and the fallback when no embedding provider is configured.
  </Card>

  <Card title="Vector Search — Semantic Similarity" icon="wand-magic-sparkles">
    Vector search converts your query and stored observations into dense embedding vectors, then finds the observations whose vectors are closest to your query using cosine similarity. This enables semantic retrieval: searching for "database performance optimization" can return the observation where your agent fixed an N+1 query, even if that observation never used the words "performance" or "optimization."

    Vector search requires an embedding provider. Agent Memory auto-detects your provider from environment variables (see the [Embedding Providers](#embedding-providers) table below). For free offline embeddings, install:

    ```bash theme={null}
    npm install @xenova/transformers
    ```

    This gives you the `all-MiniLM-L6-v2` model running entirely on your machine — no API calls, no cost.

    **Default weight:** `VECTOR_WEIGHT=0.6`

    Vector search adds approximately 8 percentage points of recall over BM25 alone on the LongMemEval-S benchmark.
  </Card>

  <Card title="Knowledge Graph — Conceptual Traversal" icon="diagram-project">
    The knowledge graph stream traverses a graph of entities and relationships extracted from your memories. When your query mentions a concept (a file name, a library, an error type, an architectural pattern), Agent Memory identifies matching graph nodes and walks their edges to find related observations that a keyword or vector search might not surface.

    Knowledge graph search requires `GRAPH_EXTRACTION_ENABLED=true` and an LLM provider. Once enabled, Agent Memory automatically extracts entities and relationships at session end.

    **Default weight:** `AGENTMEMORY_GRAPH_WEIGHT=0.3`

    The graph is optional but powerful for conceptual queries — "what decisions did we make about authentication?" surfaces the graph path through the `authentication` concept node to related decisions, files, and patterns.
  </Card>
</CardGroup>

## Reciprocal Rank Fusion (RRF)

The three streams each produce a ranked list of results. Reciprocal Rank Fusion merges those lists into one, giving credit to results that appear highly ranked across multiple streams:

```
final_score(d) = Σ  1 / (k + rank_i(d))
               streams
```

Agent Memory uses `k=60`, a standard value that smooths out differences between rank positions. A result ranked #1 by BM25 and #3 by vector search scores higher than a result ranked #1 by only one stream. This fusion approach means you don't need to tune individual stream weights obsessively — RRF naturally surfaces results that multiple search methods agree on.

Results are also **session-diversified**: Agent Memory caps results from any single session at 3, so you get a spread of relevant context across your history rather than the entirety of one verbose session.

## Embedding Providers

Agent Memory auto-detects your embedding provider from environment variables. Set `EMBEDDING_PROVIDER` to force a specific provider, or let Agent Memory pick based on which API keys are present.

| Provider            | Env Variable               | Model                    | Notes                                                                                 |
| ------------------- | -------------------------- | ------------------------ | ------------------------------------------------------------------------------------- |
| **Local (offline)** | `EMBEDDING_PROVIDER=local` | `all-MiniLM-L6-v2`       | Free, no API calls. Requires `npm install @xenova/transformers`. Best starting point. |
| **OpenAI**          | `OPENAI_API_KEY`           | `text-embedding-3-small` | Highest quality embeddings. Also activates the OpenAI LLM provider.                   |
| **Voyage AI**       | `VOYAGE_API_KEY`           | `voyage-code-3`          | Optimized specifically for code — recommended if your sessions are code-heavy.        |
| **Cohere**          | `COHERE_API_KEY`           | `embed-english-v3.0`     | Strong general-purpose embeddings with a free trial tier.                             |
| **Gemini**          | `GEMINI_API_KEY`           | `gemini-embedding-001`   | 100+ languages, supports 768/1536/3072 dimensions (MRL), 2048-token input.            |
| **OpenRouter**      | `OPENROUTER_API_KEY`       | provider-dependent       | Multi-model proxy; embedding support varies by the underlying model.                  |

<Note>
  If multiple API keys are set, Agent Memory uses this auto-detection priority: Gemini → OpenAI → Voyage → Cohere → OpenRouter. Set `EMBEDDING_PROVIDER=local` explicitly to use local embeddings even when other keys are present.
</Note>

Configure your provider in `~/.agentmemory/.env`:

```env theme={null}
# Option 1: Free local embeddings (recommended for getting started)
EMBEDDING_PROVIDER=local

# Option 2: OpenAI embeddings
OPENAI_API_KEY=sk-...
# OPENAI_EMBEDDING_MODEL=text-embedding-3-small  # optional override
# OPENAI_EMBEDDING_DIMENSIONS=1536               # required for non-standard models

# Option 3: Voyage (best for code)
VOYAGE_API_KEY=pa-...

# Option 4: Gemini
GEMINI_API_KEY=...
```

## Knowledge Graph

When `GRAPH_EXTRACTION_ENABLED=true`, Agent Memory uses your LLM to extract entities and relationships from observations at session end. These form a graph of typed nodes and edges:

**Node types:** `file`, `function`, `concept`, `error`, `decision`, `pattern`, `library`, `person`, `project`, `preference`, `location`, `organization`, `event`

**Edge types:** `uses`, `imports`, `modifies`, `causes`, `fixes`, `depends_on`, `related_to`, `prefers`, `blocked_by`, `caused_by`, `optimizes_for`, `rejected`, `avoids`, `succeeded_by`

Enable graph extraction in `~/.agentmemory/.env`:

```env theme={null}
GRAPH_EXTRACTION_ENABLED=true
```

Query the graph directly using the `memory_graph_query` MCP tool:

```json theme={null}
{
  "tool": "memory_graph_query",
  "arguments": {
    "query": "authentication",
    "depth": 2
  }
}
```

Or via the REST API:

```bash theme={null}
curl -X POST http://localhost:3111/agentmemory/graph/query \
  -H "Content-Type: application/json" \
  -d '{"query": "authentication", "depth": 2}'
```

The graph viewer is available in the real-time dashboard at `http://localhost:3113`.

## Search Tuning

You can tune every aspect of how search behaves through environment variables in `~/.agentmemory/.env`:

```env theme={null}
# Stream weights (must be 0–1; don't need to sum to 1 — RRF handles normalization)
BM25_WEIGHT=0.4                  # keyword search contribution (default: 0.4)
VECTOR_WEIGHT=0.6                # semantic similarity contribution (default: 0.6)
AGENTMEMORY_GRAPH_WEIGHT=0.3     # knowledge graph contribution (default: 0.3)

# Result limits
TOKEN_BUDGET=2000                # max tokens of context injected per session (default: 2000)
MAX_OBS_PER_SESSION=500          # max observations stored per session (default: 500)
```

<Accordion title="When to increase BM25_WEIGHT">
  Increase `BM25_WEIGHT` (e.g., to `0.6`) when you're working with highly technical, terminology-dense codebases where exact term matching matters — specific function names, error codes, flag names. Keyword matching is more precise when the vocabulary is consistent.
</Accordion>

<Accordion title="When to increase VECTOR_WEIGHT">
  Increase `VECTOR_WEIGHT` (e.g., to `0.8`) when you want more semantic retrieval — finding observations about a concept even when the wording varies. Useful when your sessions use varied language to describe the same problems.
</Accordion>

<Accordion title="When to enable GRAPH_EXTRACTION_ENABLED">
  Enable the knowledge graph when you want conceptual traversal — "what else relates to this library?" or "what caused this error pattern?" The graph excels at multi-hop queries that keyword and vector search don't cover well. Requires an LLM provider.
</Accordion>

## Token Budget

The `TOKEN_BUDGET` setting controls the maximum number of tokens Agent Memory injects into your context window at session start. The default is **2,000 tokens**.

```env theme={null}
TOKEN_BUDGET=2000   # default — ~1,900 tokens of actual context in practice
```

Agent Memory uses your budget to fit as many high-scoring memories as possible:

* Results are scored and ranked by the hybrid search
* Memories are added to the context block from highest to lowest score
* Once the running token count would exceed `TOKEN_BUDGET`, injection stops
* Results are session-diversified before budget trimming (max 3 per session)

**Increase the budget** if you want more historical context and your agent model has a large context window. **Decrease it** if you're paying per token and want to minimize context costs — the top memories are the most relevant anyway.

```env theme={null}
TOKEN_BUDGET=4000   # more context, more tokens per session
TOKEN_BUDGET=1000   # leaner injection, lower token cost
```

## Using Search

<Tabs>
  <Tab title="MCP Tool">
    Use the `memory_smart_search` MCP tool from your agent:

    ```json theme={null}
    {
      "tool": "memory_smart_search",
      "arguments": {
        "query": "how does the auth middleware work",
        "project": "my-api",
        "limit": 10
      }
    }
    ```

    This runs the full triple-stream hybrid search and returns ranked `CompressedObservation` results with individual BM25, vector, and graph scores plus the combined score.
  </Tab>

  <Tab title="REST API">
    Call the smart-search endpoint directly:

    ```bash theme={null}
    curl -X POST http://localhost:3111/agentmemory/smart-search \
      -H "Content-Type: application/json" \
      -d '{
        "query": "how does the auth middleware work",
        "project": "my-api",
        "limit": 10
      }'
    ```

    Each result in the response includes `bm25Score`, `vectorScore`, `graphScore`, and `combinedScore` so you can see exactly how each stream contributed to the ranking.
  </Tab>
</Tabs>

<Note>
  Agent Memory works without any API keys using BM25-only search. Install `@xenova/transformers` to add free local vector embeddings. Add an LLM key and set `GRAPH_EXTRACTION_ENABLED=true` for the full triple-stream experience. Each layer is additive — you get value at every tier.
</Note>