The Half That Shipped

TL;DR

• The paper: MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (Jiang, Li, Li and Li, ACL 2026) represents every remembered event across four orthogonal graphs — semantic, temporal, causal, entity — and treats retrieval as a policy-guided walk over them rather than a vector lookup.
• The build: fully coded from scratch against the published spec — algorithms, equations, hyperparameter table, appendix prompts — as a dependency-free Python library with 71 deterministic offline tests, in a single day from first commit to running in production. The authors released their own implementation; I never opened it, so nothing here is copied or adapted from it.
• The product problem: the evo.ehs assistant condensed each conversation's history into a standalone question and then forgot everything. "The permit I asked about last week" was unanswerable.
• The integration: memory ships as a retrieval-only sidecar to the existing RAG — zero extra model calls per turn, appended after the relevance gate so it can never widen what the assistant will answer, and keyed per user rather than per site.
• The honest accounting: the representation and retrieval halves shipped whole. The quality engine — an LLM doing extraction and causal inference — was swapped for deterministic rules. The paper's answering layer never shipped at all.

What the paper proposes

Most systems that give a language model long-term memory store past interactions in one pile and retrieve by embedding similarity. That works until the question isn't about similarity. "What happened after the May inspection?" is a question about time. "Why was that permit suspended?" is about causation. "What has this person told me?" is about identity. A single similarity score flattens all three into one number and hopes.

MAGMA — Jiang, Li, Li and Li, ACL 2026 — claims that those relations deserve separate structure. Every remembered event becomes a node, and the edges between nodes are partitioned into four subspaces that are maintained independently: a temporal backbone chaining events in order, a causal graph of inferred cause-and-effect, a semantic graph linking conceptually similar events, and an entity graph tying every mention of a person or thing back to one abstract node — which is how a fact from six months ago stays reachable from a name mentioned today.

Retrieval then becomes a walk rather than a lookup. A lightweight classifier reads the query's intent, entry points are found by fusing dense, keyword, and time-window rankings, and a beam search expands from there — scoring each hop by combining how well the edge type matches the query's intent with how well the neighbour matches its meaning. A "why" question is biased toward causal edges; a "when" question toward the temporal backbone. The retrieved subgraph is then flattened into a prompt with each fact carrying its timestamp and a reference back to its source.

It's a good paper, and the part I found most useful is unglamorous: relative dates are resolved at write time. "We finished it yesterday," said in a conversation on the 20th, is stored carrying 2026-08-19. The model never has to do calendar arithmetic at read time, because the arithmetic already happened.

Building it from the paper

The authors published their reference implementation under a permissive licence, so copying it would have been legal and fast. I built from the paper instead — the algorithms, the equations, the hyperparameter table, the appendix prompts — and never opened their source. Two reasons, one principled and one practical.

The principled one: a paper is a specification, and implementing from a specification is the only way to find out whether the specification is complete. It wasn't, quite — beam width, the score-decay factor, and how many anchors to keep are all unstated, so I chose values, documented every choice as a choice, and left them configurable. That list of gaps is worth more than a working copy of someone else's code.

The practical one: research code is written to produce a number in a table. Product code has to be tested, swapped, and operated. Building it myself meant I could make the expensive parts pluggable from the start — which turned out to be the entire basis of the integration below.

Here is the paper's transition score, the heart of the traversal, as it exists in the library. The intent-weighted structural term and the semantic term combine exactly as the paper specifies; the decay and the pruning threshold are mine:

# traversal.py — the per-hop transition score (paper Eq. 5)
structural = weights.get(edge.type, 1.0)   # intent-specific edge-type weight
sim = self._sim_to_query(v_id, plan)       # neighbour vs. query embedding
s_uv = math.exp(cfg.lambda1 * structural + cfg.lambda2 * sim)
score_v = u_score * cfg.decay_gamma + s_uv # cumulative, decayed by depth

The test suite is 71 cases and runs entirely offline with no API key and no network — which matters more than it sounds like it should, and I'll come back to why. The one I care about most replays the worked example from the paper's own appendix: a conversation on 20 October where someone mentions hiking "yesterday," and the question "when did she hike?" The library answers 19 October 2023, which is the paper's published answer. Reproducing someone else's stated result is the cheapest confidence you will ever buy.

The product problem

evo.ehs is an Environmental Health and Safety platform — permits, incidents, chemical inventories, training records. Its assistant answers questions about a site's own data through evo-ai, a multi-tenant retrieval service with hybrid dense-plus-keyword search, a live SQL layer for counting questions, and a relevance gate that refuses anything the records don't cover.

What it could not do was remember. Each conversation carried its own history, which was condensed into a standalone question and then discarded. Ask about a permit on Monday, come back Thursday, and the assistant had never met you. For a compliance tool that people use in short, repeated visits, that's a real gap: the follow-up a week later is the normal shape of the work.

Four integration decisions

1. A sidecar, not a replacement

MAGMA is positioned in the paper as the memory system, benchmarked against other memory systems. It would have been easy to read that as an invitation to replace the retrieval stack. That would have been wrong: the existing hybrid search already covers what MAGMA's anchor stage does, and the SQL layer answers questions neither could. So memory joins the evidence pool as one more contributor, and the documents keep primacy. In practice, on a question about your records, MAGMA contributes context; on a question about your conversations, it contributes the answer.

2. Zero model calls per turn

The paper's memory is built by a language model: an extractor pulls entities and facts out of each utterance, and a consolidation pass reasons over each event's neighbourhood to infer causal links. That's the quality engine — and at one or two model calls per remembered turn, on every question from every user, it's also a bill that scales with engagement.

Because I'd built the library with the model behind an interface, there was a second option: ship the deterministic implementations instead. Entities come from pattern rules, causal edges from cue words, embeddings from a hashing bag-of-words. Remembering and recalling now cost microseconds and a small file per user — and nothing per token. It is unambiguously lower fidelity, and it is documented as lower fidelity. The upgrade is a constructor argument if the trade ever stops making sense.

3. Memory may not open a door the gate closed

This was the decision I thought hardest about. The relevance gate exists so an off-topic question gets refused before any model is invoked — it's a cost control and a guardrail at once. Memory is a new pile of text that could be made to clear that gate, which would quietly turn the guardrail into a suggestion.

So memory is appended after the gate has already made its decision, where it can enrich an answer but never cause one. With one deliberate exception: when the documents refuse but the user's own prior conversations match the question strongly, memory alone may answer — because "what did I ask you last week?" is a question no document was ever going to satisfy, and refusing it would be its own kind of wrong.

# rag.py — memory never inflates a retrieval score
if below_relevance_threshold(nodes):
    if analytics_nodes:
        nodes = analytics_nodes          # SQL result still answers
    elif memory_node is not None and memory_strong:
        nodes = []                       # the user's own history answers
    else:
        return {"answer": GATED_ANSWER, "gated": True, ...}

if memory_node is not None:
    # appended last: documents and SQL rows stay the primary evidence
    nodes = nodes + [memory_node]

"Strongly" is deliberately conservative — a real vocabulary overlap, not a semantic near-miss — because the failure mode of a loose threshold is a gate that no longer means anything.

4. The identity problem, which was nearly a privacy bug

Memory is only useful if it's yours. The wrinkle is how the two services authenticate: evo.ehs talks to the retrieval service as a trusted backend holding one service key, asserting which site's data it may read on each request. One key, one identity, many humans behind it.

Which means that switching memory on without further work would have keyed every user of a site to the same memory. Ask "what did I ask last week?" and you'd get your colleague's questions. Not a leak across companies — the tenant boundary held — but a leak across desks, which in a compliance product is quite bad enough.

The fix follows the pattern already established for tenancy: if the caller is trusted to assert which site it's reading, it can be trusted to assert which person is asking. The site travels in a header, the person in the request body, and memory keys on the pair.

# the EHS-side client: the site in a header, the person in the body
json={
    "question": question,
    "history": history or [],
    "user_id": str(user_id) if user_id is not None else None,
}

Worth being precise about what caught this: not a test and not a review, but writing the documentation. Describing who the memory belonged to forced the question of how the service could possibly know — and the answer was that it couldn't yet. Docs written honestly are a design review that happens to produce docs.

What survived, what didn't

It's tempting to describe an integration like this as "implemented the paper." Here is the accurate version.

Shipped whole. The four-graph substrate, with all four edge types maintained and traversed. The full retrieval pipeline — intent routing, fused anchors, intent-weighted beam search, and the provenance-carrying prompt format. And the write-time temporal grounding, which is the single piece I'd port into any memory system regardless of its architecture.

Shipped in shape, not in spirit. The paper splits memory writing into a fast path and an asynchronous slow path, because the slow path's model calls would otherwise sit on the critical path of a user's request. That split exists in the code — and then runs synchronously, because deterministic consolidation takes microseconds and there is nothing left to hide. The background worker is written and unused. Architecture without its motivation is just structure.

Deliberately downgraded. Model-driven extraction and causal inference, replaced by rules; the sentence-transformer embedder, replaced by feature hashing. This is the paper's quality engine, and removing it is the largest fidelity loss in the whole exercise.

Never shipped. The paper's answering layer — its query-adaptive prompt, its judging methodology, its benchmark results. In production, MAGMA retrieves and the host system answers. Which is why nothing in the product claims the paper's accuracy numbers: those were measured on a configuration this deployment doesn't run.

The irony worth noting is that the paper's central thesis is what made the subsetting possible. Decoupling the memory representation from the retrieval policy is exactly what lets you take the representation, take the policy, and leave the answering behind. A more tightly integrated design would have been all-or-nothing.

Verifying it in production

A memory feature has an awkward property: it is invisible until it isn't. Nothing renders differently, no new page appears, and the failure mode — silently remembering nothing, or silently remembering the wrong person's history — looks identical to working correctly from the outside.

So the offline suite carries the load. 71 deterministic tests on the library, plus integration tests covering every way memory meets the gate: that a strong match can answer a refused question, that a weak one cannot, that memory appends last when documents already answer, and that a memory failure can never take a query down. That the whole engine runs without a network is what makes those tests possible — the cost decision and the testability decision turned out to be the same decision.

Then I checked it on the running system, which is a different question from checking the code. A throwaway user asked two ordinary questions about a site's records, in separate conversations. Then, in a fresh conversation with no history attached:

Q: What did I ask you about flammable chemicals?
   sources: chemical, chemical, docs, chemical, chemical, chat_memory
   A: You asked, "Are any of our chemicals flammable?"

Q: What did I ask you earlier?
   sources: docs, docs, docs, docs, docs, chat_memory
   A: You asked the following questions earlier:
      - First question: How many permits are there?
      - Second question: Are any of our chemicals flammable?
      - Third question: What did I ask you about flammable chemicals?

Note chat_memory sitting in the source list beside the record types. Remembered turns are cited like any other evidence, which means a user can always see when the assistant is drawing on their history rather than their data. Then the test user's memory was deleted through the same endpoint any user's erasure uses, and the store was confirmed empty — a verification that doesn't clean up after itself isn't finished.

One finding came out of that run that no unit test would have produced. The second question — the bare "what did I ask you earlier?" — shares no vocabulary with anything it needed to recall, so the conservative strong-match rule should not have fired. It worked anyway: the product's own documentation chunks cleared the gate, and memory rode in on the enrichment path. The guardrail held, the answer was still correct, and I only know that because I asked the real system rather than the mock.

What I'd tell the next person

Implement the paper, then negotiate with it. A research architecture is optimised for a benchmark; your system has a cost ceiling, a latency budget, and existing guardrails that already work. The useful question is never "did I implement it faithfully" but "which parts earn their place here." That negotiation is easier if you build the expensive pieces behind interfaces before you know which ones you'll want to swap.

Say which configuration you're running. A paper's numbers belong to a specific setup. Ship a cheaper one and those numbers are no longer yours to quote — so the docs say plainly that this deployment trades fidelity for cost, and where the dial is if that changes. An honest limitation is worth more than an inherited claim.

Determinism is a testing strategy. The rule-based backend was chosen to make memory free. Its second effect was larger: the entire pipeline — ingestion, graph construction, traversal, prompt assembly — runs identically on every machine with no key and no network. For a stochastic system, being able to assert exact behaviour is worth a great deal, and I'd now reach for a deterministic fallback implementation even in systems where cost wasn't the driver.

Write the documentation before you're done. The per-user identity gap surfaced while writing who the memory belongs to. Enabling the feature also falsified an existing sentence in the product docs — that nothing in the system stores what anyone asked — which was true the day it was written and wasn't afterwards. A configuration change can turn a true sentence into a false one somewhere nobody is looking; when you flip a flag, go and find the claims it just broke.

← Back to Blog