NeurIPS · Personalized, Aligned, Long-Term Memory for AI Systems (PALM) Workshop · First posted
Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents
Accepted · Workshop · Agentic AI
Can an agent reach the same decision with fewer memories? On 6,912 exact benchmark conditions, JointCore found a lowest-cost set that preserved the full-memory decision every time.
Abstract
LLM-agent memory selection often uses retrieval rankings based on record relevance or individual influence. Such rankings cannot identify sufficient subsets under redundant or jointly necessary evidence. We study per-decision memory sufficiency: selecting a small subset that preserves full-memory behavior. To separate search error from model error, we introduce JointCoreBench, a benchmark with nine interaction families, six workflow domains, natural-language decisions, and exact sufficient-set oracles, and JointCore, a relevance/cost-ordered group-deletion baseline that tests blocks before individual records. On 6,912 exact conditions, cost-ordered JointCore recovers a minimum-cost sufficient set in 100.0% of cases, reduces subset evaluations from 30.00 to 11.13 (62.9%), and achieves 87.8% active-memory savings. Across four 12B–70B model families and 432 tasks per model, its unweighted mean is 10.28 versus 16.00 model queries and 50.6% versus 43.1% minimum-cost recovery relative to cost-aware singleton deletion. Both methods preserve full-memory behavior under the deterministic primary interface. Tests with physical deletion, UNKNOWN masks, PENDING sentinels, counterfactual values, and alternative labels identify interface-dependent behavior. JointCore is an efficient benchmark baseline; the study does not establish deployment-ready memory compression. Code is available at https://github.com/dr-pandit-69/jointcore-neurips.
Findings
- On 6,912 exact conditions, cost-ordered JointCore recovered a minimum-cost sufficient set in every case.
- Subset evaluations fell from 30.00 to 11.13, a 62.9% reduction, with 87.8% active-memory savings in the exact benchmark.
- Across four model families and 432 tasks per model, average minimum-cost recovery was 50.6% for JointCore and 43.1% for deletion one record at a time.
What the evaluation covers
The workflows were generated from a common grammar. Preserving the full-memory decision can also preserve a wrong answer. Recovery varied when the researchers changed how a missing memory or a decision label was presented.
My contribution
Led the research and experimental development of this project.
Sources and citation
View BibTeX citation
@misc{memory-selection-2026,
title = {{Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents}},
author = {BHVSP Subrahmanyam and Neeraj Choudhary},
year = {2026},
eprint = {2610.23266},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.23266},
note = {Accepted at NeurIPS · Personalized, Aligned, Long-Term Memory for AI Systems (PALM) Workshop}
}