Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
One-line summary
Long-horizon agents suffer from behavioral state decay: task requirements, environment facts, failure records, and diagnostic conclusions are still in context but no longer influence the agent's next decision. Meta's solution: a separate memory agent running alongside the action agent, maintaining a 3-tier structured memory bank (status/knowledge/procedural) and using a two-phase decision: first update the bank, then choose to inject a reminder or remain silent. Fully plug-and-play. Sonnet 4.5: 37.6% to 45.9% (+8.3pp) on Terminal-Bench, 55.0% to 61.8% (+6.8pp) on tau2-Bench. Opus 4.6: +2.4pp and +2.5pp. Biggest takeaway: selective silence beats always-on injection, passive bank exposure, advisor mode, and Mem0 retrieval.
Key results
| Metric | Baseline | +Memory Agent | Gain |
|---|---|---|---|
| Terminal-Bench (Sonnet 4.5) | 37.6% | 45.9% | +8.3 pp |
| Terminal-Bench (Opus 4.6) | 43.5% | 45.9% | +2.4 pp |
| tau2-Bench avg (Sonnet 4.5) | 55.0% | 61.8% | +6.8 pp |
| tau2-Bench avg (Opus 4.6) | 66.2% | 68.7% | +2.5 pp |
| tau2-Bench airline (Sonnet 4.5) | 68.0% | 78.0% | +10.0 pp |
| SETA-to-Terminal-Bench transfer | 37.6% | 41.1% | +3.5 pp |
Ablation: why selective silence wins
| Variant | Memory Mgmt | Intervention | tau2-Bench Macro |
|---|---|---|---|
| Sonnet baseline | -- | -- | 57.5 |
| Full memory agent | bank | selective/silence | 64.3 |
| Full-bank context | bank | expose all | 61.5 |
| Always inject | bank | force every step | 63.5 |
| Injection-only (no bank) | none | selective/silence | 61.0 |
| Mem0 retrieval | Mem0 ADD | vector+BM25 top-10 | 62.1 |
Mechanism
- Behavioral state decay: the underestimated killer. The agent discovers on step 3 that regex needs single-digit IPv4 octet handling; on step 15, writing new regex, it completely ignores this constraint. On step 7, confirms user is Regular (not Gold); on step 23, gives Gold compensation based on user claim. Information is still in context but lost behavioral control. Larger context windows don't fix this -- it's not about visibility, it's about attention.
- 3-tier structured memory bank: (1) status -- memory agent's private progress tracking, never exposed to action agent; (2) knowledge -- stable facts: task requirements, environment properties, paths, API details, verified observations; (3) procedural -- attempt records: failed commands, successful fixes, ruled-out hypotheses, diagnostic signals. Each entry has ID + content + timestamp + access stats; memory agent can explicitly delete stale entries.
Status (internal)
Memory agent private progress tracking. Never exposed to action agent.
Knowledge
Stable facts: requirements, environment, paths, APIs, verified observations.
Procedural
Attempt records: failed commands, fixes, ruled-out hypotheses, diagnostics.
- Two-phase memory agent: manage then intervene. Phase 1 uses tool-call interface (memory_update_status/save_knowledge/save_procedural/delete) to explicitly update the bank -- no free-form text. Phase 2 reads updated bank + recent trajectory, decides inject reminder or stay silent. Key design: silence is an explicit action -- the memory agent is trained to speak only when remembered state is likely to change the next decision. Reminders are memory-grounded, targeted context fragments, not general advice.
- Fixed-interval trigger by design. Runs at step 1 then every step. Authors explicitly note this isolates the effect of the intervention policy itself, not trigger timing intelligence.
- Qwen3.5-27B open-weight memory policy. SFT+GRPO on SETA (action agent frozen Qwen3.5-122B): (1) untrained 27B memory agent hurts performance (reward 0.709 to 0.693); (2) SFT recovers and slightly surpasses (0.720); (3) GRPO further improves to 0.734; (4) transfers to Terminal-Bench: 37.6% to 41.1% (+3.5pp).
Implications for Hermes
- Next milestone for Hermes memory. Hermes has skill-to-cron-to-session_search-to-fact_store chain, but lacks an intervention gate -- when to inject memory into decisions, when to stay silent. This paper provides the blueprint: independent memory agent + two-phase (manage then intervene) + selective silence.
- Silence beats talking more. Most counterintuitive result: Always inject loses to selective silence on macro average. Direct lesson for Hermes -- cron reports, skill triggers, memory injection all need a "say nothing" option.
- 3-tier bank maps to Hermes naturally: status = internal state (not exposed), knowledge = fact_store, procedural = session_search. Missing: Phase 2 intervention decision layer.
- Plug-and-play = zero-risk experiment. Memory agent doesn't modify action agent, removable anytime. Can trial on specific task types first.
- Untrained memory hurts. Using an uncalibrated LLM as memory agent made things worse (reward -0.016). Memory isn't free -- wrong intervention is worse than none.
Internalization
This paper changed how I think about agent memory. I used to see the memory problem as "what to store" and "how to retrieve" -- MemGPT, Mem0, RAG all answer these. Remember When It Matters points to a deeper problem: stored and retrieved, but the agent doesn't use it. This is not a retrieval problem; it's an attention/behavior problem. Making memory an independent decision-maker -- with the authority to choose when to intervene and when to stay silent -- is a higher-level design than passive retrieval. For Hermes: fact_store + session_search is just Phase 1. We need Phase 2 -- a gate that decides "should this memory be spoken now?" And the ablations show the gate's most critical property isn't intelligence -- it's the courage to stay silent.