QA Engineering for AI Systems

The system under test

The platform is a multi-tenant retrieval-augmented (RAG) system wired into a production SaaS as its "Ask AI" feature. The architecture is my design: document retrieval, text-to-SQL analytics over live application data, and human-confirmed write actions, all under strict per-tenant isolation and guardrails. A user can ask a plain-language question and get an answer grounded in their own data — or ask the system to take an action, which it stages for explicit confirmation and records in an audit trail attributed to the asking user.

That authorship is what makes the QA work here meaningful rather than incidental. I know exactly how the system is supposed to behave, which is precisely why I test it as an adversary: the interesting defects in an LLM system are not crashes but confident, plausible wrong answers, and catching those takes someone who can establish what the right answer actually is.

The defect workflow

Every defect runs the same loop — the discovery process I built and run against the live system. It is an ordinary rigorous SDET cycle, adapted to a system whose outputs are probabilistic:

• Intake — a defect observed in production use (often a screenshot) is logged immediately as a tracked bug: issue in the owning repository.
• Reproduce & isolate — replicated on a production-parity stack, capturing the generated SQL, the prompts, and the retrieval scores, plus ground-truth queries run directly against the database to establish the correct answer.
• Root-cause — the actual mechanism, not the symptom. (For example: the planner joined an email column to a display-name column because the schema documentation never stated the column's semantics.)
• Fix + regression test — the minimal fix, paired with a test that would have caught the defect. The suites run on every change.
• Verify & close — the exact failing scenario is re-run against parity and production, numbers compared to database ground truth, and the issue closed with the fixing commit referenced.
• Class analysis — when defects share a cause, I fix the class, not the instance. Three related defects triggered a column-parity audit of all 19 SQL views against the document pipeline, closing the whole category at once.

Exemplar defects

Each is a documented issue with a reproduction, a root cause, and a fix. The pattern to notice: most are not code crashes but correctness and communication failures unique to LLM systems.

Wrong-join hallucination

A follow-up question — "how many?" — returned 0. The planner had joined an assignment column holding an email address to a display-name column; the schema documentation never declared the semantics, so the model guessed. Fix: semantic annotations added to the planner schema. Verified count went from 0 to the true 14.

// class: undocumented schema semantics feeding text-to-SQL

Misleading empty-result phrasing

A zero-row result was phrased as "the records don't cover it," implying missing capability, when weather logging had simply just been enabled and had nothing recorded yet. Fix: zero rows now answers "nothing recorded yet," distinguishing no data from not supported.

// class: LLM phrasing of empty evidence

Stale-document contact answers

A plant's real phone number was denied because contact fields lived only in embedded documents — refreshed every 30 minutes — and not in the live SQL views the query path reads. Fix: expose the fields for live lookup so current data is answered from current data.

// class: retrieval staleness vs. live queries

Identity collision on AI-executed actions

Two users shared a name; auto-generated usernames had no uniqueness handling at four code sites and no database constraint. Fix: a suffixing helper, a data backfill in development and production, and a partial unique index. Found because ambiguous identity resolution is exactly what breaks an AI that can execute actions on a person's behalf.

// class: identity resolution & disambiguation for AI-initiated actions

Internal identifiers leaking into answers

Answers leaked internal UUIDs and rendered literal markdown asterisks. Fixed on both sides: the planner is instructed never to select id columns, and list output is normalized before rendering.

// class: LLM output hygiene

Refusals that exposed internals

Refusal copy exposed implementation terms ("indexed data") to end users. Fixed with a structured gated flag so the product renders audience-appropriate wording that lists what the asker is actually able to query.

// class: audience-appropriate refusal / guardrail UX

AI-specific QA classes

Testing an LLM system means testing failure modes a conventional test plan never has to. The classes this work covers:

• Prompt / schema semantic gaps — undeclared column meaning leading the planner astray.
• Text-to-SQL correctness vs. ground truth — generated queries checked against direct database answers, not eyeballed.
• Retrieval staleness vs. live data — embedded-document answers reconciled against live views.
• Relevance-gate calibration — score-fusion behavior is dataset-dependent and has to be tuned and regression-tested, not assumed.
• Empty-evidence phrasing — distinguishing "no data yet" from "not supported."
• LLM output hygiene — no leaked ids, no raw markdown, no internal jargon.
• Identity resolution for AI-initiated actions — lettered choices with a cancel option, human confirmation, and audit-log attribution to the asking user.
• Guardrail regression tests — grounding, tenant isolation, prompt-injection posture, and SQL injection / escape rejection suites, all run on every change.

Evidence

Everything below is real and verifiable in the repositories:

• Automated test suites — 110 tests in the LLM platform and 174 in the host SaaS at the time of writing; every defect fix added or updated a test.
• Retrieval evaluation harness — a golden set held at a 100% hit rate with mean reciprocal rank around 0.82, maintained through guardrail changes so tuning never silently regressed retrieval quality.
• Ground-truth verification — every answer checked against direct SQL during defect work, with counts matching exactly (51 permits, 13 users, 4 flammable chemicals).
• Full audit trail — AI-executed actions recorded under the asking user and tagged as originating from the assistant, so every automated action is attributable.