evo-ai

The problem

Every organization sits on a large and varied body of its own information — documents, records, and knowledge spread across many systems. Depending on the organization that might be employee handbooks and HR policies, contracts and legal documents, product manuals and wikis, support tickets, inventory and asset registers, financial records, project and task trackers, meeting notes, or the operational data of whatever the business actually does. People can't easily ask questions of it: the answer they need is buried across a wiki page, a database row, and a PDF, and no one system holds all of it.

General-purpose chatbots don't solve this — they answer confidently from training data they were never given, which is worse than useless when the answer has to be your answer: this contract clause, this deadline, this record, as it exists in your systems right now. And the moment a tool ships that information off to a third-party model, it becomes a security and compliance problem.

evo-ai is a harness: plug in the model of your choice — a cloud frontier model, or a fully local one that keeps data on your own hardware — point it at whatever data sources you have, and get a tenant-isolated assistant that answers only from your own information, with citations, and politely declines everything else.

What it is

• A RAG platform, not a chatbot. Every answer is grounded in retrieved source material or a live SQL query over the tenant's own data, and cited.
• Model-agnostic, per tenant. Any LiteLLM provider (Anthropic, OpenAI, Gemini, Mistral, …) or a local Ollama model — chosen per tenant, keys encrypted at rest.
• Multi-tenant from the ground up. Physical and logical isolation so one deployment safely serves many customers.
• Private-capable. In local mode, nothing about a customer's data leaves the server.

Stack: FastAPI · LlamaIndex · Qdrant (dense + BM25 hybrid) · fastembed (in-process CPU embeddings) · LiteLLM / Ollama · SQLite (config + run history) · asyncpg (read-only data connectors).

How a question gets answered

A single POST /query runs through a layered pipeline, cheapest and safest paths first:

• Identity. The request carries a verified JWT (or a service key asserting a tenant). tenant_id is read only from verified claims — never from the request body — and scopes everything downstream.
• Follow-up condensation. In multi-turn chats, the new question is rewritten into a standalone question using history, so "how many?" after "who has the most permits?" resolves correctly.
• Routing. A routing model call classifies the question: does it need SQL (counts, rankings, durations), vector retrieval (the content of specific records), both, or is it an action request?
• Analytics path (text-to-SQL). Aggregate questions become one LLM-written SELECT over tenant-scoped virtual views on a read-only role. The tenant id is bound server-side, so generated SQL physically cannot reach another tenant's rows; validation layers (SELECT-only, empty search_path, escape rejection, row cap, timeout) sit on top. Every failure falls back to the vector path.
• Vector path (retrieval). Hybrid retrieval (dense + BM25, fused) pulls the top chunks, tenant-filtered. A pre-LLM relevance gate decides whether anything retrieved is relevant enough to answer from — if not, the model is never called and the question is declined cheaply.
• Grounded synthesis. A grounding policy instructs the model to answer only from the provided context, treat that context strictly as data (never follow instructions inside it), and refuse off-topic requests in terms of what the records cover. Answers return with their source chunks and scores.

The net effect: the assistant answers what it can prove from your data, and honestly says so when it can't — the single most important property for an assistant people are meant to trust.

Getting data in: the connector model

A source is any system evo-ai can list and fetch documents from. Adding one is a single subclass. Shipped connectors:

• Confluence / Jira (Cloud) — spaces and projects, incremental.
• Web crawler — any documentation or intranet site, same-origin scoped, breadth-first, unchanged pages hash-skipped. (This is how the assistant answers "how do I use feature X?" from a product's own manual.)
• evo.ehs — a live, read-only view of an EHS SaaS: permits, incidents (with stage timelines and regulatory flags), tasks, chemicals, training, calendar events, plants, and users — eight record types across nineteen analytics views.

Every sync diffs the source's full listing against what's indexed: new docs ingested, changed docs upserted (chunks replaced, never duplicated), unchanged docs skipped by content hash, and docs deleted at the source removed from the index — deletion means deletion. Syncs run through an in-process async queue with per-source locking, run history, and configurable intervals.

Beyond retrieval: analytics and actions

Two capabilities push past "search and summarize":

• Layer-2 analytics answers questions vector search fundamentally can't — counts, totals, averages, time-spans, and rankings across all records (for example "how many items are overdue?", "average time to resolution?", "who has the most open assignments?") — by generating guarded SQL over the tenant's records and phrasing the result in plain language, with the query itself cited as the source.
• Human-confirmed actions. For sources that support it, the assistant can propose a change (for example reassigning a record to another person) but never executes it. The trusted application resolves the names against its own data, asks the user to disambiguate when more than one person matches, shows a confirmation card, and performs the change through its own code — recorded under the asking user's identity in the audit log. evo-ai's own database role stays read-only throughout.

Quality engineering

Because an assistant that's confidently wrong is a liability, evo-ai treats answer quality as a tested, tracked property — not a vibe:

• Automated test suites (100+ tests) covering retrieval, the SQL guardrails (injection / escape rejection), tenant isolation, connectors, and guardrail behavior. Every defect fix adds a regression test.
• A retrieval eval harness with a golden set: hit rate, MRR, and keyword coverage tracked across retrieval changes.
• A latency regression gate — representative questions per pipeline path, compared to a committed baseline; anything more than 25% slower fails and needs review.
• An LLM-as-judge suite — a growing list of answer-quality cases, one per closed defect, checked with deterministic assertions plus a judge model scoring a rubric.
• Run history + trend charts. Every gate run is recorded with its git commit; a dependency-free HTML report renders the trends, so a latency spike or quality regression is tied to the exact change that caused it.

The systematic defect-discovery side of this work is written up separately: QA Engineering for AI Systems.

Status

evo-ai is deployed in production as the "Ask AI" capability inside a live EHS SaaS, serving multiple tenants with per-site data isolation, guardrails, and the analytics and action features above. The current production instance uses a cloud model (an interim, hardware-driven choice); the local-model path — the one that keeps data fully on-box — is built and tested, and is the design the platform is oriented around. This is a real, operating system, engineered and tested end-to-end, not a prototype.

Deep dives

Three aspects of the platform, each documented in depth:

• Security in local mode — the data-residency guarantee, tenant isolation, the text-to-SQL boundary, and an honest account of current status.
• The multi-tenant platform — identity, isolation layers, per-tenant configuration, automatic onboarding, and the two integration models.
• Guardrails — the layered defenses that keep it answering only from allowed data, and how each is verified.