evo-ai

The problem

Every organization sits on a large and varied body of its own information — documents, records, and knowledge spread across many systems. Depending on the organization that might be employee handbooks and HR policies, contracts and legal documents, product manuals and wikis, support tickets, inventory and asset registers, financial records, project and task trackers, meeting notes, or the operational data of whatever the business actually does. People can't easily ask questions of it: the answer they need is buried across a wiki page, a database row, and a PDF, and no one system holds all of it.

General-purpose chatbots don't solve this — they answer confidently from training data they were never given, which is worse than useless when the answer has to be your answer: this contract clause, this deadline, this record, as it exists in your systems right now. And the moment a tool ships that information off to a third-party model, it becomes a security and compliance problem.

evo-ai is a harness: plug in the model of your choice — a cloud frontier model, or a fully local one that keeps data on your own hardware — point it at whatever data sources you have, and get a tenant-isolated assistant that answers only from your own information, with citations, and politely declines everything else.

What it is

• A RAG platform, not a chatbot. Every answer is grounded in retrieved source material or a live SQL query over the tenant's own data, and cited.
• Model-agnostic, per tenant. Any LiteLLM provider (Anthropic, OpenAI, Gemini, Mistral, …) or a local Ollama model — chosen per tenant, keys encrypted at rest.
• Multi-tenant from the ground up. Physical and logical isolation so one deployment safely serves many customers.
• Private-capable. In local mode, nothing about a customer's data leaves the server.
• Bring your own data. A workspace can upload its own documents — policies, permits, safety data sheets, scanned PDFs — and have them quoted alongside its live records, without anyone touching a connector.
• It remembers the person, not just the question. Per-user memory carried across conversations, implemented from a 2026 research architecture and running with zero extra model calls per turn.

Stack: FastAPI · LlamaIndex · Qdrant (dense + BM25 hybrid) · fastembed (in-process CPU embeddings) · LiteLLM / Ollama · SQLite (config + run history) · asyncpg (read-only data connectors).

How a question gets answered

A single POST /query runs through a layered pipeline, cheapest and safest paths first:

• Identity. The request carries a verified JWT (or a service key asserting a tenant). tenant_id is read only from verified claims — never from the request body — and scopes everything downstream.
• Follow-up condensation. In multi-turn chats, the new question is rewritten into a standalone question using history, so "how many?" after "who has the most permits?" resolves correctly.
• Routing. A routing model call classifies the question: does it need SQL (counts, rankings, durations), vector retrieval (the content of specific records), both, or is it an action request?
• Analytics path (text-to-SQL). Aggregate questions become one LLM-written SELECT over tenant-scoped virtual views on a read-only role. The tenant id is bound server-side, so generated SQL physically cannot reach another tenant's rows; validation layers (SELECT-only, empty search_path, escape rejection, row cap, timeout) sit on top. Every failure falls back to the vector path.
• Vector path (retrieval). Hybrid retrieval (dense + BM25, fused) pulls the top chunks, tenant-filtered. A pre-LLM relevance gate decides whether anything retrieved is relevant enough to answer from — if not, the model is never called and the question is declined cheaply.
• Grounded synthesis. A grounding policy instructs the model to answer only from the provided context, treat that context strictly as data (never follow instructions inside it), and refuse off-topic requests in terms of what the records cover. Answers return with their source chunks and scores.

The net effect: the assistant answers what it can prove from your data, and honestly says so when it can't — the single most important property for an assistant people are meant to trust.

Getting data in: what an answer can draw on

Three things feed an answer: systems evo-ai syncs from, files the workspace uploads itself, and what it remembers about the person asking.

Connectors

A source is any system evo-ai can list and fetch documents from. Adding one is a single subclass. Shipped connectors:

• Confluence / Jira (Cloud) — spaces and projects, incremental.
• Web crawler — any documentation or intranet site, same-origin scoped, breadth-first, unchanged pages hash-skipped. (This is how the assistant answers "how do I use feature X?" from a product's own manual.)
• evo.ehs — a live, read-only view of an EHS SaaS: permits, incidents (with stage timelines and regulatory flags), tasks, chemicals, training, calendar events, plants, users, and compliance records — nine record types across twenty analytics views.

Every sync diffs the source's full listing against what's indexed: new docs ingested, changed docs upserted (chunks replaced, never duplicated), unchanged docs skipped by content hash, and docs deleted at the source removed from the index — deletion means deletion. Syncs run through an in-process async queue with per-source locking, run history, and configurable intervals.

Bring your own data

A connector needs a system to connect to. Most of what an organization actually wants quoted is a file somebody has: the permit PDF, the contractor's safety data sheet, the policy that was never in the wiki. So a workspace admin can upload documents directly — PDF (including scanned pages, read off the image automatically), Word, CSV, Markdown, JSON, plain text — and they are quotable the moment indexing finishes, cited by filename, in the same answers as the live records.

Uploads carry an optional category label — OSHA, EPA, the client's own name, SDS, Training, Images, or anything typed in. Labels do two jobs. They make bulk removal possible: delete every EPA document is one confirmed action, and a document whose removal fails stays listed, so nothing quotable can silently vanish from view. And they drive a Search sources row on the ask page, so a question can be narrowed to certain categories — which only appears once labels are actually in use, because a filter with one option is just clutter.

What it remembers about you

The third input is not a document at all. Per-user memory — built from MAGMA (ACL 2026) — carries what a person established in earlier conversations into the next one, scoped to that person in that workspace and appended only after the relevance gate has already allowed the question. It is described in full below.

Memory: remembering the person

Retrieval answers "what does the data say?". It does not answer "what did I ask you yesterday?" — and an assistant that reintroduces itself every session makes the user carry the context. evo-ai gained per-user memory built from MAGMA (ACL 2026), implemented from the paper and shipped as a retrieval-only sidecar: it can add context to a question, and it can never overrule the guardrails.

• Scoped to one person in one workspace. The calling application holds a single service identity, so the site is asserted by header and the person by user id; memory is keyed on the pair. Two people in one workspace never see each other's memory, and the same address in two workspaces is two different people.
• Free at query time. The shipped configuration uses a rule-based extractor and a hashing embedder with synchronous consolidation — zero extra model calls per turn. Higher-fidelity extraction and real embeddings are constructor arguments, not a rewrite, so the quality/cost dial moves without touching the integration.
• The relevance gate keeps final authority. Memory is appended after the gate decides a question is in scope, so it can enrich an allowed answer but cannot talk the assistant into answering something it should have refused.
• Forgetting is a supported operation. One call wipes a workspace's memory, or one person's within it.

Turning it on had a consequence worth stating plainly: question text is now stored, in exactly two places, where previously it was stored nowhere. That falsified a sentence in the product's own documentation, which was corrected in the same change — a configuration flag can make a true claim false, and the docs are part of the flag.

The full write-up — what the paper proposes, what survived contact with production, and what did not — is The Half That Shipped.

Beyond retrieval: analytics and actions

Two capabilities push past "search and summarize":

• Layer-2 analytics answers questions vector search fundamentally can't — counts, totals, averages, time-spans, and rankings across all records (for example "how many items are overdue?", "average time to resolution?", "who has the most open assignments?") — by generating guarded SQL over the tenant's records and phrasing the result in plain language, with the query itself cited as the source.
• Human-confirmed actions. For sources that support it, the assistant can propose a change (for example reassigning a record to another person) but never executes it. The trusted application resolves the names against its own data, asks the user to disambiguate when more than one person matches, shows a confirmation card, and performs the change through its own code — recorded under the asking user's identity in the audit log. evo-ai's own database role stays read-only throughout.

Quality engineering

Because an assistant that's confidently wrong is a liability, evo-ai treats answer quality as a tested, tracked property — not a vibe:

• Automated test suites (100+ tests) covering retrieval, the SQL guardrails (injection / escape rejection), tenant isolation, connectors, and guardrail behavior. Every defect fix adds a regression test.
• A retrieval eval harness with a golden set: hit rate, MRR, and keyword coverage tracked across retrieval changes.
• A latency regression gate — representative questions per pipeline path, compared to a committed baseline; anything more than 25% slower fails and needs review.
• An LLM-as-judge suite — a growing list of answer-quality cases, one per closed defect, checked with deterministic assertions plus a judge model scoring a rubric.
• Run history + trend charts. Every gate run is recorded with its git commit; a dependency-free HTML report renders the trends, so a latency spike or quality regression is tied to the exact change that caused it.

The systematic defect-discovery side of this work is written up separately: evo.qaforaisystems.

Status

evo-ai is deployed in production as the "Ask AI" capability inside a live EHS SaaS, serving multiple tenants with per-site data isolation, guardrails, and the analytics and action features above. The current production instance uses a cloud model (an interim, hardware-driven choice); the local-model path — the one that keeps data fully on-box — is built and tested, and is the design the platform is oriented around. This is a real, operating system, engineered and tested end-to-end, not a prototype.

Shipped since: tenant document uploads with category labels and source filtering, and per-user memory, both running in production. Memory is deliberately off by default on deployments that have not asked for it — the flag stores question text, and that is a decision a customer makes rather than one that arrives in a release.

The generation running today is v1, and what has changed since — content packs, the speed work, and the defects behind both — is on its own page: v2 in development.

Deep dives

Four aspects of the platform, each documented in depth:

• Security in local mode — the data-residency guarantee, tenant isolation, the text-to-SQL boundary, and an honest account of current status.
• The multi-tenant platform — identity, isolation layers, per-tenant configuration, automatic onboarding, and the two integration models.
• Guardrails — the layered defenses that keep it answering only from allowed data, and how each is verified.
• Implementing MAGMA memory — building a 2026 memory architecture from the paper, the four integration decisions, and what survived production.
• v2 in development — everything since MAGMA: content packs on a public-domain regulatory corpus, twelve seconds of latency cut to three, seventeen closed defects, and the measured bar v2 has to clear.