NVIDIA NeMo Agent Toolkit + Memanto

NVIDIA NeMo Agent Toolkit builds agent workflows from YAML. Its memory module accepts pluggable backends. nemo-agent-toolkit-memanto registers Memanto as one of those backends, _type: memanto. It works with the toolkit’s automatic memory wrapper and with its built-in memory tools, with no other code changes.

Automatic memory

With auto_memory_agent, every user message is saved, relevant memories are recalled before the agent answers, and replies are saved.

Memory tools

add_memory, get_memory and delete_memory let the agent decide when to save, recall or forget.

One agent per user

Each user’s memories live in their own Memanto agent, so recall never crosses users.

Facts, not transcripts

Conversation turns are distilled into typed facts, preferences and decisions. Small talk stores nothing.

How It Works

User message  →  auto_memory_agent  →  save turn  →  extract typed memories  →  Memanto agent for this user
                        │
                        ├──  recall top_k memories  →  "Relevant context from memory" before the inner agent
                        │
                        └──  inner agent answers  →  save reply

Memory call	What Memanto does
Item with `memory` text, which the `add_memory` tool produces	Stores the text as written, with the item’s `tags`.
Item with only a `conversation`, which `auto_memory_agent` produces	Runs conversation extraction and stores what it finds. Small talk stores nothing.
`search(query, top_k, user_id=...)`	Recalls the user’s most relevant memories. Also accepts the Memanto filters `type`, `tags` and `min_similarity`.
`remove_items(user_id=...)`	Deletes the user’s agent and all of its memories.
`remove_items(user_id=..., memory_id=...)`	Deletes one memory.

Each user maps to the agent <agent_prefix>_<first 32 hex characters of sha256(user_id)>, so user IDs never appear in agent names. Searching or deleting for a user who has no memories returns nothing and creates no agent.

Prerequisites

Python 3.11, 3.12 or 3.13
NeMo Agent Toolkit (nvidia-nat-core) 1.8 or later
Memanto 0.2.28+, installed automatically, and a Moorcheh API key, or an on-prem Memanto backend
nvidia-nat-langchain for auto_memory_agent and the ReAct agent

Install and configure

Install

Install into the same environment as nvidia-nat-core:

pip install nemo-agent-toolkit-memanto nvidia-nat-langchain

Check that the toolkit finds it:

nat info components -t memory -q memanto

Set your key

export MOORCHEH_API_KEY=your_key_xxxxxxxxxxxxxxxxxx

If it isn’t set, the key saved by memanto setup is used. The on-prem backend needs no key.

Add the memory backend

memory:
  user_memory:
    _type: memanto

Configuration

Field	Default	Description
`api_key`	`MOORCHEH_API_KEY`	Moorcheh API key.
`agent_prefix`	`nat`	Prefix of the per-user agent IDs. Must match `[A-Za-z0-9_-]{1,32}`. Use a different prefix for each application so they never share memories.
`max_memories`	`20`	Maximum memories extracted from one conversation, from 1 to 100.

Automatic memory

Wrap any agent in auto_memory_agent. The user identity comes from the toolkit runtime, so each user gets their own memory without passing an ID.

config.yml

memory:
  user_memory:
    _type: memanto

llms:
  nim_llm:
    _type: nim
    model_name: meta/llama-3.3-70b-instruct

functions:
  current_datetime:
    _type: current_datetime
  my_react_agent:
    _type: react_agent
    llm_name: nim_llm
    tool_names: [current_datetime]

workflow:
  _type: auto_memory_agent
  description: Assistant that remembers each user
  inner_agent_name: my_react_agent
  memory_name: user_memory
  llm_name: nim_llm
  search_params:
    top_k: 5

This example uses an NVIDIA NIM model, so also set NVIDIA_API_KEY:

nat run --config_file config.yml --input "I'm vegetarian and I live in Toronto."
nat run --config_file config.yml --input "Suggest a restaurant for me."

The first turn stores memories such as “User is vegetarian.” and “User lives in Toronto.”. The second turn recalls them before the agent answers.

auto_memory_agent requires a description, and the ReAct agent requires at least one tool. Leaving either out makes the config fail validation.

Memory tools

To let the agent choose what to remember, give it the toolkit’s memory tools:

memory:
  user_memory:
    _type: memanto

functions:
  add_memory:
    _type: add_memory
    memory: user_memory
    user_id: alex
  get_memory:
    _type: get_memory
    memory: user_memory
    user_id: alex
  delete_memory:
    _type: delete_memory
    memory: user_memory
    user_id: alex

workflow:
  _type: react_agent
  llm_name: nim_llm
  tool_names: [add_memory, get_memory, delete_memory]

A fixed user_id suits a single-user deployment. For multiple authenticated users, set a trusted user_id_resolver instead.

Tool-level user_id and user_id_resolver need NeMo Agent Toolkit 1.9 or later. On 1.8, leave them out: the agent passes user_id in each tool call.

Inspect what a user’s agent knows

Memories are ordinary Memanto memories, so you can read, correct and audit them with the CLI or the web UI. To find a user’s agent:

from nat.plugins.memanto.memanto_editor import MemantoEditor

print(MemantoEditor(agent_prefix="nat").agent_id_for("alex"))

Behavior and limits

Behavior	Detail
Search results	`memory` holds the content and `similarity_score` the relevance, where higher is more similar. `metadata` carries `memory_id`, `type`, `title`, `confidence`, `provenance`, `source`, `created_at` and `updated_at`.
Metadata is not stored	`MemoryItem.metadata` passed to `add_items` is ignored. Use `tags` instead.
Extraction cost	With `auto_memory_agent`, each saved turn makes one extraction LLM call on the Memanto backend, and the agent’s reply waits for it.
Delete means delete	`delete_memory` removes the user’s agent and every memory in it permanently.
One session per agent	Memanto keeps one active session per agent. Running another client against the same user’s agent signs this one out until its next call, which re-establishes the session.

In minimal container images, install the system CA certificates, for example apt-get install ca-certificates. Without them, calls to the Moorcheh API fail with CERTIFICATE_VERIFY_FAILED.

Integrations

NVIDIA NeMo Agent Toolkit

NVIDIA NeMo Agent Toolkit + Memanto

Automatic memory

Memory tools

One agent per user

Facts, not transcripts

How It Works

Prerequisites

Install and configure

Configuration

Automatic memory

Memory tools

Inspect what a user’s agent knows

Behavior and limits

Next Steps

​NVIDIA NeMo Agent Toolkit + Memanto

Automatic memory

Memory tools

One agent per user

Facts, not transcripts

​How It Works

​Prerequisites

​Install and configure

​Configuration

​Automatic memory

​Memory tools

​Inspect what a user’s agent knows

​Behavior and limits

​Next Steps

NVIDIA NeMo Agent Toolkit + Memanto

How It Works

Prerequisites

Install and configure

Configuration

Automatic memory

Memory tools

Inspect what a user’s agent knows

Behavior and limits

Next Steps