ADR-0006: Three-Layer Search with In-Database Index and Pluggable LLM¶
- Status: accepted
- Refined by: ADR-0044 (layer 1 realized: FTS baseline)
- Date: 2026-06-04
- Deciders: Nicolas Burri
Context¶
Search is an important system capability and shall be backed by an LLM. The data is highly sensitive, which mandates a two-layer access: a local RAG mechanism plus an LLM that may be cloud-based or on-prem (project_overview.md). Mandator separation (ADR-0002) must extend to search: retrieval must never cross scope permissions. Customer contracts will differ on whether any external service may see their data.
Options Considered¶
Considered along three axes — index store (PostgreSQL/pgvector vs. external vector DB), embedding computation (local vs. cloud API), and LLM policy granularity (global vs. per mandator). Key rejected alternatives:
- External vector DB (Qdrant, Weaviate, …): richer retrieval features and scale headroom, but permission filtering becomes application code — a second separation mechanism beside RLS that can be wrong independently. A QMS holds thousands of documents, not millions; the scale headroom is not needed.
- Cloud embedding APIs: better model choice, but full document text would leave the system at indexing time, bypassing retrieval-side minimal disclosure entirely.
- Global LLM setting: simpler, but the first customer forbidding cloud LLMs would block the feature for everyone.
Decision¶
Search consists of three layers:
- Full-text search (PostgreSQL FTS): the baseline "find the document" capability — no LLM involved, works offline, covers the majority of daily queries.
- Local RAG: on every content-freezing life-cycle transition (ADR-0003), text is extracted from the blob, chunked, embedded by a locally running embedding model (ONNX/Ollama; never a cloud API), and stored in PostgreSQL with pgvector. Retrieval is fully local.
- LLM layer ("ask the QMS"): the user's question plus the retrieved, permission-filtered chunks go to a pluggable LLM provider — cloud or on-prem behind a single interface.
Security properties by construction:
- RLS covers the index: vector and FTS rows carry the same scope key as all content tables (ADR-0002); the separation test suite (RISK-001) covers index tables too.
- Minimal disclosure: only retrieved, permission-filtered chunks ever reach an LLM — never whole documents, never unfiltered content.
- Per-mandator LLM policy from v1: each mandator configures cloud LLM / on-prem LLM / no LLM for content in its scopes. Queries spanning scopes with different policies use the most restrictive applicable policy.
- Auditability: every LLM call is audit-logged with provider and the exact chunks sent.
Consequences¶
- The
searchmodule (ADR-0005) contains: extraction/chunking/embedding pipeline, pgvector + FTS retrieval, the LLM provider interface, and the policy enforcement point. It remains an extraction candidate; the in-DB index keeps even an extracted search service stateless. - A local embedding model becomes a deployment dependency (model choice TBD at implementation; behind an interface like the LLM provider).
- The index is derived data: backups may skip it; restore triggers a rebuild.
- Re-indexing strategy on embedding-model change must be designed (versioned embeddings).
- Text extraction quality for binary formats (PDF, docx) bounds RAG quality — extractor choice TBD at implementation.
- pgvector joins the SOUP list once scaffolding exists.