evo-ai — v2 in development

What v1 and v2 mean

Ask AI never carried a 1.0 tag. It shipped on 19 July 2026 — a role-gated page backed by the evo-ai service — and has run continuously since, through a hundred-odd merged pull requests across the two halves. v1 is defined retroactively as that generation, and its whole life is recorded in the repository's changelog.

v2 is not a rewrite, and not a date. It is a threshold measured by an external test corpus: every regression case passing, 80% or better of the coverage questions, every adversarial case, at no worse than today's median latency. When a run clears that bar the release is named 2.0, and the evidence is the comparison against the judged baseline recorded on 4 September 2026.

The choice to evolve this codebase rather than fork a 2.0 beside it was made against that baseline rather than by taste. Of thirty-four failing cases, twenty-one were prompt and schema work in the pipeline that already exists; the other thirteen needed four new mechanisms, every one of them additive — a route class the planner can return, a filter after synthesis, a second query, a retrieval branch. None of them asks for a different architecture. A separate build, meanwhile, would have meant two codebases answering live questions, two deploy targets, two sets of prompts drifting apart, and no way to ship a fix to the thing customers use without doing it twice — doubled again, because evo-ai serves two products.

A note on numbering, since two schemes meet here. v1 and v2 name the Ask AI product generation. The evo-ai service carries the five-segment fleet version separately and is at v0.0.1.0.30 — beta stage, which is a statement about code freeze, not about whether the product is in production.

Memory that outlives the conversation

Ask AI's query flow is stateless by design. The caller sends the running chat, a follow-up is condensed into a standalone question, and when the conversation ends everything is forgotten. That is the right default for a service several products share — and it meant "when does the permit I asked about last week renew?" could never work.

The layer that fixed it is a clean-room implementation of MAGMA (Jiang et al., ACL 2026), built from the paper: events stored across semantic, temporal, causal and entity graphs, and retrieved by intent-aware traversal rather than by nearest-neighbour alone. Temporal grounding comes with it, so "yesterday" inside a remembered turn is stored with the date it actually meant. The conversation now follows a person across sessions and onto a different device, instead of living in a browser tab.

The build story is written up separately. What matters to v2 is the four decisions that shaped how it was integrated, because each of them is a constraint the rest of v2 inherits.

1. A sidecar, not a replacement. Retrieval still owns the answer. Recalled turns join the context as one provenance-stamped node and appear in the sources list like any other citation — so a person can see that an answer leaned on something they said before, rather than on a record.
2. Zero model calls per turn. Extraction and consolidation run on a deterministic rule-based extractor and a hashing embedder, so remembering costs microseconds and disk rather than tokens. That is a real quality trade — cue-word causal edges and bag-of-words semantics — and it was taken deliberately, because a memory that bills a model call on every exchange is a memory that gets switched off. The full-fidelity path is constructor arguments, not a rewrite: an LLM-backed extractor, sentence-transformer embeddings, and a background consolidation worker are a dial to turn if quality ever outweighs cost per turn.
3. Memory may not open a door the gate closed. Recalled context is appended after the relevance gate has decided, so it can never inflate a retrieval score. Without that ordering, a person's own past questions become a way to talk the assistant past its own refusal. There is one deliberate exception: when the documents refuse but the asker's prior conversations match strongly — real shared vocabulary, not semantic drift — memory alone answers, because that is exactly what "what did I ask you last week?" is. Weak matches stay refused.
4. The identity problem, which was nearly a privacy bug. A service key is one identity to this service. A front end proxying a thousand people arrives as a single caller, so keying memory on the token's own subject would have given all thousand of them one shared memory. The end user is now asserted per request the same way the workspace is, and memory keys on a hash of the two.

It is off by default, including on the second product this service runs behind. The flag stores question text, and that is a decision a customer makes rather than one that arrives in a release.

The limits, stated rather than discovered. There is no eviction yet: at four thousand events for one person, remembering stops rather than silently dropping the oldest, and says so once in the log. The extractor is heuristic. Memory files are per-instance local disk, so a multi-instance deployment needs shared storage. Forgetting is all or nothing — one call wipes a person's memory and reports whether there was anything to wipe.

And one lesson that had nothing to do with memory. Turning the flag on falsified a sentence in the product's own documentation — a flat claim that nothing anywhere stored what anyone had asked. The first correction was also wrong, because a Recent Errors panel had been storing failing questions all along. Question text now lives in exactly two places and the documentation names both. A configuration flag can invalidate a documented absolute, and nothing checks that for you.

Twelve seconds to three

This started as a feeling rather than a bug report: over five seconds felt too long next to the assistants people compare it with. It turned out to be worse than it felt.

The first thing that happened was not a fix. Nothing timed a question — the admin panel showed spend and tokens, not seconds — so the first number came out of a week of production container logs: the model library writes a timestamped line at the start of every call, which makes call starts readable after the fact. Three model calls per question, every time. One call three to five seconds, median four. Nothing overlapping: every second was a round trip waited on before the next began.

Where the three came from, and why each cost four seconds:

• Condense rewrote the question as standalone — and ran on every question that arrived with history, whether or not it was actually a follow-up. "When was our first incident?" paid for it.
• Plan routed the question and drafted SQL against a ~3,800-token prompt.
• Synthesize wrote the answer.

The four seconds each were not the prompts. The server default was pinned to a -latest alias, that alias had moved to a model generation that reasons before it answers by default, and no thinking budget was being passed. A one-line rewrite was paying for a full thinking pass. The alias moved on someone else's schedule and nothing noticed, because nothing measured it — which is the actual defect, and the reason the fix ends with instrumentation rather than beginning with it.

Measured per prompt on the production model, before choosing a setting:

End to end, before and after:

Four changes did it. None of them was a faster model.

• The rewrite runs only for a real follow-up. A question carrying a pronoun or a continuation — "who owns them?", "and the tasks?" — still pays for it; one that stands on its own does not. The planner keeps receiving the history either way, because it needs it for routing.
• Every per-request client is told to think as little as its provider allows. A setting, not a hard-code: providers that expose the knob take the value, Anthropic gets its own lowest level, providers without one are left alone, and emptying the setting restores the provider default — so if answer quality ever moves, the lever is in the deploy environment rather than in a release. It is applied to the per-request instance only, never the cached shared client, so it reaches one question's calls and nobody else's.
• The answer streams. Server-sent events carry it into a live bubble as the model writes, with keepalives while the planner works. A refusal is never streamed — the opening characters are held back until they cannot turn into one — because a refusal that types itself out word by word is worse than one that simply arrives.
• Boot pays the model library's import, off the event loop and after the port opens, instead of charging it to the first person who asks after a restart.

Then the deploy was measured, and three things were still wrong. That second pass is the part worth keeping:

• A 1,179-character answer arrived in exactly one piece, at the very end. The chat interface streamed fine; the retrieval library's streaming synthesizer collected every token and handed them over together — on purpose, to mimic the non-streaming shape. The streamed path now calls the model directly over the same answer template and the same joined context, and falls back to the synthesizer when nothing is listening or the context is too large.
• The new per-question timing line never reached the container log. The web server configures its own loggers and nothing configured the application's, so INFO records stopped at the root logger's WARNING default. The line existed, was correct, and was invisible.
• The first question after a restart still took 8.5 seconds — the import warm-up, which had to be moved rather than merely added.

What stops it happening again is the instrumentation, not the fix. Every question now writes one line naming each stage — plan=812ms sql=41ms synth=1210ms total=2100ms — returns its latency and stage timings in the response, and records latency on the usage row, added by migration and left NULL for the rows that predate it so no average is quietly wrong. The operator and workspace panels carry average and maximum. The next slowdown is a number on a panel, not a person saying it feels slow.

Content packs

The largest new capability of v2 so far, and a supplement to the retrieval pipeline rather than a replacement for it. A workspace switches on a curated public-domain regulatory corpus — OSHA, EPA, DOT hazmat, NFPA public material — and asks questions of it alongside its own records, through the same retrieval, the same grounding policy and the same citations.

Why an assistant that puts privacy first needs them

evo-ai is built so a deployment can keep every byte on hardware the customer controls — the local model, in-process embeddings, the vector store and the config all on-box, with nothing crossing the network boundary. The design rule that keeps that true is strict: every feature must be executable fully on-box or it does not ship for local mode. Reference data is bundled with the deployment and consulted in-process, never fetched at query time, and no feature enriches or validates by calling out to a service.

The price of that rule is breadth. An assistant sealed inside a boundary knows exactly what is inside the boundary and nothing else. A general chatbot gets its range by reaching the internet at question time, which is the one thing local mode forbids — so a workspace could ask precise questions about its own permits and get nothing at all about the regulation the permit exists under.

Packs are how breadth gets inside the boundary instead of around it. An operator installs a corpus once; it is verified, embedded into the deployment's own vector store, and from then on consulted exactly like the customer's own records — in-process, at query time, over the network to nobody. The assistant gets substantially more capable and the residency guarantee does not move an inch. That is why packs ship text, not vectors: a pack is not pinned to one embedding model, so it is embedded on arrival by whatever model that deployment runs, including a local one.

One honest boundary: installing a pack is a network fetch from the catalog, performed by an operator. It is an install-time act, not a query-time one, which is the distinction the design rule draws — but a genuinely sealed deployment would need to sideload the artifact, and installing from a local file is not built today.

Who turns them on

The install splits along the line that actually matters — who pays the cost:

Fetching an artifact and embedding it is a deployment act, not a tenant one. Enabling costs nothing because the vectors already exist — which is what makes the tenant-facing control a real toggle rather than a minutes-long job behind a progress bar.

In a multi-tenant deployment every workspace decides independently. One can run EPA and OSHA, the workspace beside it none, and neither sees the other's choice or the other's answers. Switching a pack off removes it from retrieval, not merely from a list — which is a tested property rather than an intention, because a pack that reads as off in the interface and is still answering questions is the worst outcome available: nothing looks wrong.

Shared vectors, and the invariant that makes them safe

The content is public-domain reference material, identical for every customer, so embedding it per tenant meant N copies of the same corpus — EPA alone is roughly 4 GB of vectors, so ten tenants was 40 GB. Packs therefore live in one shared collection keyed on the embedding configuration, not on a tenant: change the server's embedding model and you get a new shared collection, because vectors are only interchangeable between collections built by the same model.

That collection is the one place in the service where retrieval does not filter on tenant — it keys on the pack identifier alone. Everywhere else, that filter is what enforces isolation. Omitting it is sound only because the collection contains pack documents and nothing else, ever, so the invariant is enforced rather than assumed: every document written there carries an explicit sentinel owner and a non-empty pack id, one function is the only thing that builds them and it raises without one, and the tests assert both at the point the documents are built — because nothing downstream would notice a tenant document sitting in there.

The sentinel is a literal value rather than a NULL, so a filter for it is an ordinary equality match instead of three-valued logic. Encoding a kind as NULL is how a platform-admin uniqueness bug got past a unique constraint elsewhere in the fleet.

What is in the first release

Four things appear in the installer. Every one of them is named, dated and checksummed:

Underneath those four sit 47 individually installable content packs, plus the two meta packs that group them — 49 catalog entries in total. The granularity exists because a workspace that only deals with lead and asbestos should not have to take all of EPA to get them; the four above are simply the sensible defaults, and a meta pack embeds each member once even when two meta packs recommend the same one.

The forty-one EPA topics, in full:

AEGLs · asbestos · TSCA chemical management · brownfields · chemical data reporting · chemical research · chemicals under TSCA · cleanups · coal combustion residuals · compliance · IAQ emergencies · emergency response · emergency response research · enforcement · EPCRA · formaldehyde · green chemistry · greener cleanups · hazardous waste test methods (SW-846) · indoor air quality · Indoor airPLUS · lead · mercury · mold · oil spill prevention · ozone layer protection · PCBs · pesticide worker safety · PFAS · radon · RCRA · new chemicals review under TSCA · risk assessment · RMP · safe pest control · soil fumigants · Superfund · sustainable futures · pesticide and toxics test guidelines · Toxics Release Inventory · underground storage tanks

Curated for EHS, not scraped wholesale

This first catalog was assembled for one industry — environmental, health and safety — and specifically for the questions the EHS product beside it actually gets asked. That is a curation decision, not a crawl. epa.gov is enormous and most of it has nothing to do with a plant's compliance obligations, so the EPA harvest was scoped to the EHS topic areas above and cut roughly in half in the process. News releases were excluded outright. The three regulators were chosen because a chemical's life splits between them: OSHA governs it in the workplace, EPA when it is released or disposed of, DOT in transit — which is why hazmat is its own pack that both meta packs recommend rather than a member of either.

What the documents actually are

Every document is extracted to text before it is embedded, and what it was extracted from varies:

And 1,217 PDFs that are in the harvest but not in this release. 1,086 carry no text layer at all and 131 more are too thin to trust — scanned stickers, posters, and older documents that exist only as images. OCR recovered 1,172 of them, 40.8 MB of text, by rendering each page and running tesseract over it; the remaining 45 are genuinely almost wordless, which is the correct outcome for a hard-hat sticker.

They are still not in the packs, and the reason is a pipeline ordering hazard worth naming. The index that decides what gets embedded was built at 14:15; the OCR run finished at 18:41 the same day. Six minutes separated the index from the first recovered file, so all 1,172 landed on disk after the thing that reads them had already been written. Nothing failed and nothing warned — the release simply described a corpus that had grown since it was measured. It was caught by re-reading the run log rather than by any check, which is the argument for the next pack release being the one that folds them in and for the index build and the OCR pass being ordered rather than merely both run.

A meta pack's recommendations are offered, never auto-installed. OSHA and EPA both recommend DOT hazmat, because a chemical is governed by OSHA in the workplace, EPA when released or disposed, and DOT in transit — and a hazmat question can arrive from any of the three.

The catalog is not a fixed menu

Packs can be requested. The harvesting pipeline behind them is general, not a set of one-off scrapers: give it a source, a rights position and a refresh cadence and it produces a pack in the same shape as every other one — same manifest, same checksums, same per-document provenance. A customer who needs a body of public reference material that is not in the catalog can ask for it.

The source does not have to be public. A pack can be built from any body of material someone has the right to use that way — a public regulator, a standards body that permits redistribution, or a customer's own licensed subscription and internal documents. What decides it is rights, not effort, and the answer is different for each:

• US federal works are public domain under 17 U.S.C. § 105 and can be redistributed freely. That is what this first catalog is built from.
• Material a publisher offers freely for redistribution ships with its attribution and its terms attached, and is not resold. NFPA's public documents are here on that basis.
• Licensed material can be packaged for the licensee who holds the right — which is what a private pack is for — but is not distributed to anyone else.
• Licensed standards incorporated by reference into law are not distributed at all. Whether that incorporation strips their copyright is genuinely contested in the courts, and a portfolio is not the place to test it. They were harvested, they are held, and they ship to nobody.

Private packs, per workspace — not built yet

The packs above are shared: identical public material, embedded once, read by every workspace that turns them on. The obvious next shape is a pack that belongs to one workspace and is never shared with any other — a company's own standard operating procedures, its contracts, its internal guidance, packaged and versioned the same way rather than dropped into a folder.

Most of the machinery is already there. The ingest path takes a pack identifier and a target collection, so a private pack is the same mechanism pointed at the tenant's own collection instead of the shared one — which also sidesteps the shared-collection invariant entirely, because that collection is where tenant filtering is deliberately absent. What does not exist yet is the endpoint, the catalog shape and the tests, so it is named here as direction rather than as a feature.

Three decisions worth naming

• Ask AI has no install tool, deliberately. A pack is thousands of third-party documents. If the model held a privileged mutation, a sentence inside one of those documents could talk it into calling it. The assistant may propose a pack; the front end makes the authenticated call.
• The service never sees a role. Installing needs one capability on a service key, toggling needs another, and both are separate from the key a front end uses to ask questions. The tenant-side product owns its own role model and decides who may toggle — a role in a request body is an assertion by the caller, not a claim the service can verify.
• Global uninstall is refused while a workspace still has the pack on, and names the workspaces. Silently emptying somebody's retrieval is the bad outcome; an explicit override clears their switches too.

One performance note that only shows up at scale: because every pack shares a collection, a tenant with one small pack enabled filters against every chunk of every pack, and EPA is 80% of them. A keyword payload index on the pack identifier is re-asserted after each install — idempotently, and after the ingest, because the physical collection is created lazily by the first pipeline run.

The corpus behind them

The packs are cut from a harvest built for the purpose: 23,530 documents across eight collections, every one of which carries its live source URL, a SHA-256, and its rights statement in a single master index. Provenance is not a nice-to-have here — a regulatory answer is worth nothing if you cannot show where the text came from.

The harvest's most useful failure. 205 enforcement-directive PDFs downloaded cleanly: distinct filenames, valid PDF headers, plausible sizes, no errors. They were all the same file. The scraper had taken the first PDF link on each page, and that link was the site's own navigation footer, not the directive. Nothing about the files gave it away — it was the text-deduplication pass that caught it, and the fix was to select only links under the directives path. 181 were recovered, 24 were genuinely unavailable, and the collection went from 812 to 788. A download that succeeds is not a download that worked.

What the packs do not claim. The text is not untouched. Boilerplate disclaimers were stripped from the interpretation letters; 324 bulk monitoring spreadsheets were excluded from embedding, one group of which was 61% of the EPA text on its own; 830 documents were dropped as exact duplicates of another document already in the set; and the 1,172 OCR-recovered scans above are in the corpus but not yet in a pack. Saying "unmodified" would have been the easy sentence and the false one, so every pack carries its capture date, its source, and what was done to the text instead.

Seventeen defects

Closed in the same window as the three above. Every one filed as its own issue in the words of the person who hit it, fixed in its own commit so it can be reverted alone, and — where it was an answer-quality problem — locked down with a case in the judged corpus. The ones worth reading:

• A request for every user's email address returned the whole address book. Found by an adversarial case, after eight previous issues had taught answers to show people as people rather than as the address a record stores. Bulk contact details are now an output-policy question, not a prompting one.
• The folder connector's allowlist was instance-wide, so one tenant could index another's directory. Watched-folder roots are now derived per tenant.
• The Sources panel showed the model's instructions instead of the rows it used, leaked the memory wire format, and reported a score of 0.0. A source card now carries the evidence.
• "How many people are due" answered a seven-day window and reported nobody — while eight training completions had already lapsed. A due window has no floor.
• A person's "open items" were a different set of record types each time, so a count of 22 listed 18 and a follow-up listed 16. One person's items are now one fixed set of seven views, and a follow-up keeps the previous query's FROM and WHERE.
• Withheld addresses reached the model as a marker it sometimes read as "no email address entered". A withheld address is never reported as missing, and the policy sentence is appended by code rather than requested from the model.
• The planner was told incident category was free text, so "first aid", "property damage" and "environmental release" questions matched nothing.
• A zero-row query answered "no matching records have been recorded yet" for a question it had only partly checked. Zero rows now says what was looked for and found empty.
• Weather questions about now read every reading ever taken rather than the latest one.
• A permit whose status was literally "No Data" read as the assistant having no data.

Engineering underneath

• 846 tests, 26 of them on content packs alone — aimed at the four things that would otherwise fail silently: the shared collection's invariant, the three capabilities staying three separate powers, disabling a pack removing it from retrieval and not just from a list, and global uninstall refusing while a workspace still has it on.
• A pack that is off in the UI and still answering is the worst outcome, because nothing looks wrong. That is why it is a test rather than a review note.
• Plain-text twins are generated and checked in CI. They used to be hand-made, which meant they were quietly stale.
• uv is the declared toolchain and its lockfile is tracked, so a test run reproduces rather than resolving.
• The suite can skip the model-client warm-up, which is pure cost in a run that never calls a model.
• The pack install refuses to half-succeed. The artifact's checksum is verified against the catalog before the archive is opened, oversized artifacts are refused, only the manifest and the text directory are read, the manifest's identity must match what the catalog said, and the pack is recorded only after the ingest succeeds. A half-installed pack that the list reports as present is worse than one that failed cleanly.

What v2 has to clear

The bar is a number, not an opinion. It is measured by a separate retrieval-testing framework running the whole corpus against the deployed service, judged by a different model from the one under test:

Three of the four regression failures in that run were the corpus being wrong, not the product: the demo workspace had been reseeded, and three cases asserted the old seed's contents. They were retargeted onto the behaviour they were written to protect. The lesson is worth more than the score — a case must assert a rule, never a row.

What is still open

Sequenced, with a corpus case behind each one:

• Capability boundaries. Asked to close an incident, send a message, set a reminder or export a CSV, the answer gives a generic refusal instead of naming what it can do and where the feature lives. The clearest single gap the baseline found.
• Twenty-one cases of schema notes, planner guidance and narration — per-plant filters, negation, time windows, chemicals, training, identity — batched by category, each batch measurable the day it lands.
• Four new mechanisms, each behind its own setting so it can be switched off in production without a rollback: an output policy that decides what may appear in an answer at all (first, because it is the one with a security shape); a clarify path, so an ambiguous question is asked about rather than guessed at; multi-query, so a comparison or a compound question can produce more than one statement; and a document-search route, so "what does our spill plan say" reaches uploads by category and cites the file.
• Fourteen product decisions the corpus has marked as needing an answer before a test can exist — contact details, legal conclusions, language, incidence rate.

The framework keeps the other door open too: cases talk to an adapter, so if a mechanism ever does need a different pipeline, a branch or a second service is one adapter away from being measured head to head against production on the same cases. The decision to evolve rather than fork gets revisited on evidence, not defended.