AIエージェントの記憶は最適量がモデルごとに違うとIBMが8モデルで検証

""Memory" here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories."／"No model weights are updated. The learning loop changes the guidance available to the agent, not the underlying model — which is exactly why it's cheap to adopt and portable across the eight models we tested."／"The agent attempts tasks and produces trajectories. ALTK-Evolve extracts behavioral guidelines from both its successful and unsuccessful runs. It consolidates those guidelines into a reusable set."（Learning happens around the model, not inside it 節）— IBM Research より

"Both memory configurations draw from the same guideline set, mined once (via the loop above) from AppWorld's training split only. What changes between them is only how that one set is delivered — the full guideline set injects all of it every step, while curated retrieval delivers a selected subset — never how the guidelines were produced, and no test-split data ever goes into building it."／"The number of guidelines a model mines depends on its own capability, so we report configurations by strategy — "full guideline set" vs. "curated retrieval" — rather than by raw counts, which aren't comparable across models."（The three configurations we compare 節）— IBM Research より

"When we scaled the evaluation to eight models — from a 30B dense model to frontier proprietary systems — one finding stood out:"／"We evaluated on AppWorld — 585 multi-step tasks (168 test_normal + 417 test_challenge) across 9 simulated apps (calendars, messaging, payments, and so on). Tasks are scored two ways: whether the agent fully completes each task (TGC — Task Goal Completion) and whether every variant of a scenario passes (SGC — Scenario Goal Completion, a stricter, all-or-nothing bar)."（Results Across the Spectrum 節）／"AppWorld tasks are graded by two metrics, both reported as percentages (higher is better):"／"SGC — Scenario Goal Completion. A stricter, all-or-nothing metric. Each scenario bundles several variants of the same task (the same request with different data, phrasing, or edge conditions). SGC counts a scenario as passing only if the agent succeeds on every variant."（Appendix: Understanding the Metrics 節）／"Figure 1. Representative models in the three observed patterns. Bars show TGC on AppWorld test_normal for baseline vs. the best-memory configuration"（The three patterns, in one view 節・代表例の表と同じ掃引に基づく図の注記）— IBM Research より

"Memory keeps paying off as long as a model has a remaining failure mode to target."／"Strong models with headroom want the full guideline set — every guideline, including rare edge-case lessons. They have the capacity to absorb and apply all of it. DeepSeek-V3.2 (671B MoE) climbed +9.5 percentage points in task completion when given its full self-mined guideline set."／"Smaller or weaker models get drowned by a large guideline set. For these, a tight, high-confidence core plus a handful of task-relevant guidelines retrieved per task works best. gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach — while the full guideline set gained less and cost ~50% more tokens."／"And the effect doesn't disappear at the top of the range: GPT-5.5 and Opus, both near the ceiling on TGC, still gain +7.2 and +7.1pp SGC respectively."（The Key Insight／Results Across the Spectrum 節）— IBM Research より

"Already-saturated models show no measurable gain. We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively. GLM-5 (745B MoE) sat here in our runs."／"What puts a model into one pattern rather than another isn't simply parameter count. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to shape where a model lands, and separating those factors is ongoing work."／"spending no extra context until their remaining failure modes are better understood"（The Key Insight: Dosage Depends on Capability 節）— IBM Research より

"A practical concern: injecting a full guideline set inflates every ReAct step's input, because the guidelines are re-sent each turn."／"Table 1. Average token use per task, accumulated across agent steps, measured against the no-memory baseline."（表の注記）— IBM Research より。上表の3行（148K→263K／110K→166K／110K→116K と各増加率）は、この Table 1 に各セルとして掲載されている

"Curated retrieval keeps cost near baseline. For weaker models, where selection wins on accuracy, it also wins on cost — the best of both worlds (+16.1pp TGC at only +5% tokens for gpt-oss-120b). Better performance here does not require more inference cost."／"Memory doesn't blow up the reasoning loop. DeepSeek runs about the same number of ReAct steps with memory as without (≈18–19 on average), so the added cost is input-token inflation, not longer trajectories."（The Cheapest Memory Strategy Can Also Be the Best 節）— IBM Research より

"The real efficiency lever in production is prompt caching: the static portion of the guideline set is identical across steps and can be cached, cutting effective cost substantially. Cache-aware prompt design — keeping the shared guideline-set prefix stable so it stays cacheable — is worth engineering for."（The Cheapest Memory Strategy Can Also Be the Best 節）— IBM Research より