LQMS Deployment Guide¶
Operator-facing guide for deploying LQMS (Kotlin/Quarkus backend, PostgreSQL/jOOQ, OIDC auth, Quinoa-bundled Angular SPA). It covers the production container, database roles, the security-critical BYPASSRLS requirement, every configuration key, IdP integration, the bootstrap admin flow, email enablement, backup retention, sweep tuning, and upgrades.
For a ready-to-run local stack (app + PostgreSQL + Keycloak) see ../../deploy/:
deploy/docker-compose.yml, deploy/README.md, and the deploy/smoke.sh end-to-end check.
Scope note. The bundled
deploy/compose stack is a demo/colleague-feedback topology with demo secrets and a single BYPASSRLS datasource. Everything in this guide marked hardening is what you additionally do for a real deployment.
1. Prerequisites¶
- Runtime: Docker (or any OCI runtime / Kubernetes). The image is a Quarkus JVM fast-jar on
eclipse-temurin:25-jre-noble; no JDK/Gradle/Node needed at runtime. - PostgreSQL 18 reachable from the app, with a role that can bypass RLS for migrations (§4).
- An OIDC provider (a corporate IdP, or the bundled Keycloak) — see §6.
- Persistent storage for the content blob store (§5,
lqms.storage.root) and the database. - Build-time only (to produce the image): Docker + JDK 21 + Node, plus outbound HTTPS to Maven Central, the Gradle distribution service, and the npm registry. See §2.
2. Building the production image¶
The reference server deployment does not use this section. Since 2026-08-06 CI publishes the image to
ghcr.io/nburri/lqmson every release tag and the server pulls it — seeserver-deployment.md§4. What follows is the mechanics of building the image by hand: the CI job does exactly this, and it is still the path for an air-gapped target or a registry outage.
deploy/Dockerfile is multi-stage:
- build — JDK 21 + Gradle (via the wrapper) + Node 22. Runs
./gradlew :backend:api:quarkusBuild -Dquarkus.quinoa=true, which compiles the backend, builds the Angular SPA (Quinoa, ADR-0032), and assembles the Quarkus fast-jar with the SPA bundled underMETA-INF/resources. - runtime —
eclipse-temurin:25-jre-noble, runsjava -jar /app/quarkus-run.jaras a non-root user.
# from the repo root
./gradlew :backend:persistence:generateJooq # one-time / after migration changes — see below
docker build -f deploy/Dockerfile --build-arg GIT_SHA="$(git rev-parse --short HEAD)" -t lqms:latest .
Build/version stamp (GIT_SHA, stale-deployment detector)¶
The image carries a build stamp — the git short SHA + build timestamp — surfaced by
GET /api/version and shown in the SPA's user menu, so an operator (or a colleague on a demo)
can see at a glance whether the running build is the expected one. Because .git is excluded from
the image build context, the SHA is passed as the GIT_SHA build ARG (deploy/Dockerfile
exports it so both the Gradle backend stamp and the Quinoa-spawned npm SPA stamp pick it up). Omit
it and both stamps fall back to "unknown"; the build still succeeds. With the bundled compose
stack, wire it on the up/build command:
The jOOQ build-time Docker dependency (important)¶
jOOQ classes are generated from the real, Flyway-migrated schema by starting a throwaway
PostgreSQL with Testcontainers — which needs a Docker socket. A hermetic docker build has no
Docker socket, so the image build cannot run jOOQ codegen itself. The build therefore consumes
pre-generated jOOQ sources via -Plqms.jooq.provided (wired in deploy/Dockerfile, shipped
into the build context by a .dockerignore exception).
Operator obligation: run ./gradlew :backend:persistence:generateJooq (on a machine with
Docker) before building the image, and again whenever the migrations under
backend/persistence/src/main/resources/db/migration change. If you skip it on a fresh checkout,
the image build fails at Kotlin compile with unresolved ch.lqms.persistence.jooq.* symbols.
(A proxied build network is supported: docker build --build-arg https_proxy=… --build-arg
http_proxy=… --build-arg no_proxy=… …; the args are no-ops when unset.)
3. Deployment topology¶
browser ──TLS──> [ reverse proxy / TLS term ] ──> LQMS app (:8080)
│ ├── PostgreSQL 18 (data + RLS)
│ └── content store (filesystem volume)
──OIDC──> IdP (corporate or Keycloak) <─────┘ (discovery, JWKS, token, login redirect)
- The app serves the SPA and the API same-origin under one port (
8080);/api/*is the authenticated API,/triggers OIDC login, static SPA assets are public (ADR-0032). - Terminate TLS in front of the app. The session cookie is issued
Secure(ADR-0028), so the app must be reached over HTTPS in production, and the IdP redirect URIs must behttps://….
4. Database roles and the BYPASSRLS requirement (READ THIS)¶
Why it matters (ADR-0035 §7, ADR-0002/0009)¶
Mandator/project separation is the top quality goal and is enforced in the database with
row-level security: every content table is FORCE RLS. The cross-tenant fan-out that creates
notifications for derived tenants runs through a SECURITY DEFINER function
(lqms_fanout_source_event). A DEFINER function bypasses RLS only if its owner — the role that
ran the migrations — is a superuser or has BYPASSRLS.
If migrations run as a plain owner, the function still "succeeds" but the caller's own RLS filters the derived scopes away: it returns 0 and derived tenants are silently never notified. That fails closed for separation but silent for compliance — the worst kind of bug, because nothing errors at runtime.
Enforcement: the app runs FanoutDeploymentCheck at startup and refuses to boot if the
function owner cannot bypass RLS. You will see this line in a healthy boot:
and this failure if the migration role lacks the privilege:
the owner of lqms_fanout_source_event is neither superuser nor BYPASSRLS: … Run migrations as a
BYPASSRLS role or grant it: ALTER ROLE <migration_role> BYPASSRLS.
Minimum setup — the migration role¶
Create the migration role with LOGIN + BYPASSRLS and let it own the schema so every migrated
object (including the DEFINER function) is owned by a BYPASSRLS role. deploy/postgres-init.sql
does exactly this:
CREATE ROLE lqms_migration WITH LOGIN PASSWORD '…' BYPASSRLS;
ALTER SCHEMA public OWNER TO lqms_migration;
GRANT ALL PRIVILEGES ON DATABASE lqms TO lqms_migration;
GRANT ALL ON SCHEMA public TO lqms_migration;
Point the app datasource at this role. Flyway migrates at startup (§10).
Current single-datasource reality¶
The application uses one datasource, so it connects with the same BYPASSRLS migration role
it migrates with — but runtime RLS is still fully enforced for content access: every content
transaction runs SET LOCAL ROLE lqms_app (a NOBYPASSRLS role) before touching data
(RlsScopeContext, ADR-0026), so the FORCE-RLS policies apply and the database backstop is
active. This is the test-proven RISK-002 closure (RlsScopeContextTest demonstrates RLS holding
even over a superuser datasource). Code paths that deliberately do NOT switch roles (startup
bootstrap, the scheduler sweep, notification delivery — system-context by design, ADR-0037/0039)
run with the connection's privileges.
The residual the two-role hardening below addresses is defense-in-depth, not a live gap: a bug or compromise that skipped the role switch would run privileged. Connecting with a NOBYPASSRLS runtime role would make that class of bug fail closed.
Two-role hardening (recommended for production)¶
The design (ADR-0002/0009, V002__row_level_security.sql) intends two roles:
| Role | Privilege | Used for |
|---|---|---|
lqms_migration |
LOGIN, BYPASSRLS |
Flyway migrations (and the DEFINER function owner) |
lqms_runtime |
LOGIN, NOBYPASSRLS, member of lqms_app |
the application's runtime datasource |
The runtime role must be a member of the lqms_app group role (created by migration V002) and
must not bypass RLS, so the database enforces separation even if an application query forgets
its scope filter. It cannot be created in postgres-init.sql because lqms_app does not exist
until migrations run; create it after first migration:
Splitting the app onto lqms_runtime (a second datasource used for everything except Flyway) is a
code change beyond this packaging work and is staged for review, not implemented here. Until
then, run the single-datasource stack behind the same separation discipline you would apply to any
app-tier-enforced tenant boundary.
5. Configuration reference¶
All keys are standard MicroProfile/Quarkus config: set them as environment variables (shown in
ENV_VAR form) or -D system properties. Env mapping uppercases and replaces ./- with _
(e.g. lqms.storage.root → LQMS_STORAGE_ROOT, quarkus.oidc.auth-server-url →
QUARKUS_OIDC_AUTH_SERVER_URL). Defaults below are from backend/api/src/main/resources/application.properties.
5.1 Datasource¶
| Key | Default | When to change |
|---|---|---|
quarkus.datasource.db-kind |
postgresql |
fixed |
quarkus.datasource.jdbc.url |
(dev services in dev/test) | required in prod, e.g. jdbc:postgresql://db:5432/lqms |
quarkus.datasource.username |
— | required in prod: the migration role (§4), e.g. lqms_migration |
quarkus.datasource.password |
— | required in prod; use a secret, not a literal |
quarkus.datasource.devservices.image-name |
postgres:18-alpine |
dev/test only (Testcontainers) |
5.2 Flyway (schema migrations)¶
| Key | Default | When to change |
|---|---|---|
quarkus.flyway.migrate-at-start |
true |
leave on unless you gate migrations externally (§10) |
5.3 HTTP / REST boundary¶
| Key | Default | Notes |
|---|---|---|
quarkus.rest.path |
/api |
API prefix; SPA lives at / (ADR-0032) |
quarkus.http.auth.permission.* |
see below | authorization boundary, leave as shipped |
quarkus.http.auth.proactive |
true |
proactive auth |
Boundary rules (most-specific path wins): / authenticated (SPA entry → OIDC login);
/api/health permit (liveness); /api/version permit (public build stamp — carries no tenant
data, §5.12); /api/* authenticated; /logout authenticated (RP-initiated logout, §5.5);
/q/* authenticated in prod (permitted in dev); /* permit (SPA static assets). Do not loosen
these.
5.4 Frontend (Quinoa, ADR-0032)¶
| Key | Default | Notes |
|---|---|---|
quarkus.quinoa |
false |
must be true at build time to bundle the SPA (the Dockerfile passes -Dquarkus.quinoa=true); irrelevant at runtime |
quarkus.quinoa.build-dir |
dist/webui/browser |
Angular output dir |
quarkus.quinoa.enable-spa-routing |
true |
client-side routing fallback |
5.5 Authentication / OIDC (ADR-0011/0028/0033, REQ-AUTH-*)¶
| Key | Default | When to change |
|---|---|---|
quarkus.oidc.auth-server-url |
(dev services in dev) | required in prod: the issuer URL, e.g. https://idp.example.com/realms/lqms |
quarkus.oidc.client-id |
(dev services) | required in prod: the LQMS client id at the IdP |
quarkus.oidc.credentials.secret |
— | required for a confidential client (BFF); use a secret |
quarkus.oidc.application-type |
hybrid |
browser BFF + machine bearer in one; leave as-is |
quarkus.oidc.authentication.pkce-required |
true |
keep on; the IdP client must allow PKCE (S256) |
quarkus.oidc.authentication.cookie-same-site |
lax |
CSRF posture (ADR-0033); leave as-is |
quarkus.oidc.token.refresh-expired |
true |
silent refresh (REQ-AUTH-006); leave on |
quarkus.oidc.token.audience |
${quarkus.oidc.client-id} |
audience hardening (REQ-AUTH-011); the IdP must add the LQMS client to the access-token aud (see §6) |
quarkus.oidc.authentication.allow-multiple-code-flows |
false |
leave off (finding #18): reuse the single state-cookie name q_auth so repeated code-flow challenges overwrite instead of accumulating q_auth_* cookies into an HTTP 431 (quarkusio/quarkus#40268). Trade-off: no two concurrent interactive logins in separate tabs at the same instant |
quarkus.oidc.authentication.java-script-auto-redirect |
false |
leave off (finding #18): an XHR the SPA marks X-Requested-With: JavaScript gets a 499 (not a 302 the background call can't complete), so no surplus state cookie is minted and the SPA reloads to a clean login. Do not treat the 431 by raising quarkus.http.limits.max-header-size |
quarkus.oidc.logout.path |
/logout |
RP-initiated logout: hitting this ends the IdP session and clears the BFF cookie. Register the post-logout URI at the IdP (§6) |
quarkus.oidc.logout.post-logout-path |
/ |
where the IdP redirects after logout (back to the SPA entry → fresh login) |
After a restore, existing browser sessions are invalid — re-login is expected (and now fails clean).
A restore that replaces the Keycloak realm (e.g. restore.sh … --force) rotates the realm signing
keys, so any BFF session cookie a browser still holds becomes unrefreshable. With the two knobs above
this fails clean: the SPA's next request gets a 499, reloads, and lands on a fresh login — the
session cookies are reset server-side. Without them (the pre-fix behaviour, finding #18) the same
situation wedged the profile: each background XHR minted a new q_auth_* state cookie until the
Cookie header exceeded the server limit and every request returned HTTP 431. If a user reports a
persistent 431 after a restore, the immediate remedy is to clear the site's cookies (or use a fresh
profile); the structural fix is these two settings. Never work around it by enlarging the header limit.
5.6 Keycloak Dev Services (test/dev only — ignore in prod)¶
quarkus.keycloak.devservices.web-client-timeout (60s) and quarkus.devservices.timeout
(180s) only affect the Dev Services containers started during ./gradlew test/dev mode. They
have no effect on a production deployment that supplies its own quarkus.oidc.auth-server-url.
5.7 System bootstrap (ADR-0012) — see §7¶
| Key | Default | Notes |
|---|---|---|
lqms.bootstrap.admin.issuer |
(unset → bootstrap skipped) | the admin identity's OIDC issuer |
lqms.bootstrap.admin.subject |
(unset) | the admin's stable OIDC subject (sub) |
lqms.bootstrap.admin.email |
(unset) | the admin's email |
lqms.bootstrap.admin.display-name |
(unset, optional) | display name |
issuer, subject, and email must all be set for the bootstrap to run; otherwise it is
skipped (logged as "No bootstrap admin identity configured").
5.8 Content store (ADR-0001)¶
| Key | Default | Notes |
|---|---|---|
lqms.storage.root |
data/content |
set to a persistent, backed-up path/volume (the image defaults LQMS_STORAGE_ROOT=/app/data/content); document/version blobs live here |
lqms.content.attachment.max-bytes |
10485760 (10 MB) |
max size of a single image attachment (ADR-0045); rejected above this |
lqms.content.attachment.max-count |
25 |
max number of image attachments per draft version (ADR-0045) |
Image attachments (ADR-0045, rich content) are PNG/JPEG only — magic-byte-verified on upload,
not by extension — and share the lqms.storage.root blob store with document/version content;
no separate storage config.
5.9 Overdue-task sweep (ADR-0037, REQ-NOT-004) — see §9¶
| Key | Default | Notes |
|---|---|---|
lqms.tasks.sweep-interval |
5m |
how often the scheduler tick runs (also drives email delivery, §8) |
lqms.tasks.due-after |
P14D |
ISO-8601 period after which an open task is due |
lqms.tasks.escalate-after |
P7D |
grace period after due before escalation |
5.10 Notification email (ADR-0041) — see §8¶
| Key | Default | Notes |
|---|---|---|
lqms.notifications.email.enabled |
false |
enable to deliver notifications by email |
lqms.notifications.default-locale |
en |
language for recipients with no app_user.locale |
quarkus.mailer.from |
lqms@example.com |
sender; set a real address when email is enabled |
quarkus.mailer.host / .port / .username / .password / .tls |
— | standard quarkus-mailer SMTP settings, supplied per deployment when email is on |
5.11 Runtime PostgreSQL session variables (not config keys)¶
lqms.authorized_scope_ids and lqms.current_user_id are PostgreSQL session GUCs set by the
app per transaction (RlsScopeContext, ADR-0026) — they are not operator configuration and must
not be set as app config.
5.12 SPA cache headers (stale-bundle prevention)¶
The app sets Cache-Control on the served SPA so a browser never runs an old entry point against a
freshly deployed backend (a real bug caught in the demo — a cached index.html served the previous
build):
| Key | Value | Effect |
|---|---|---|
quarkus.http.filter.spa-entry-nocache.matches / .header."Cache-Control" |
/\|/index.html → no-cache |
the SPA entry point is revalidated every load, so a new deploy is picked up immediately |
quarkus.http.filter.spa-hashed-assets.matches / .header."Cache-Control" |
content-hashed *.js/*.css → public, max-age=31536000, immutable |
fingerprinted bundles cache for a year (safe — a new build emits new filenames) |
Leave these as shipped; they are the counterpart to the GIT_SHA build stamp (§2) for keeping a
deployment demonstrably current.
6. IdP integration (real deployment)¶
LQMS authenticates browsers via the OIDC Authorization Code flow with PKCE and a server-side (BFF) session; machine/MCP clients present bearer tokens validated against the same provider (ADR-0011). To integrate a corporate IdP, register an LQMS confidential client and ensure it provides:
- Authorization Code + PKCE (S256).
pkce-required=trueis on; the client must permit PKCE. - A confidential client secret →
quarkus.oidc.credentials.secret. - Redirect URIs for the app's public HTTPS origin (e.g.
https://lqms.example.com/*), and post-logout redirect URIs as needed. - An audience mapper that adds the LQMS client id to the access token
audclaim (REQ-AUTH-011 /quarkus.oidc.token.audience). Without it, valid tokens are rejected. In Keycloak this is anoidc-audience-mapperon the client (seedeploy/lqms-realm.jsonand the test realmbackend/api/src/test/resources/quarkus-realm.json). - An
emailclaim (and ideallygiven_name/family_name). The user is JIT-provisioned from token claims; email is used for notification delivery and is refreshable from the IdP (REQ-DPR-009). - A stable
subper user — this is the identity key LQMS stores (issuer + subject). It must match the bootstrap adminsubjectfor the break-glass admin (§7).
Set quarkus.oidc.auth-server-url to the realm/issuer URL and quarkus.oidc.client-id to the
registered client. The URL must be reachable from the app (discovery/JWKS/token) and resolve to
the same issuer for the browser (login redirect). On a single host with the bundled Keycloak, use a
hostname reachable from both sides (see deploy/README.md, "Browser vs server url").
7. Bootstrap admin flow (ADR-0012)¶
On startup, if lqms.bootstrap.admin.{issuer,subject,email} are all set, BootstrapRunner
idempotently brings up the base hierarchy and grants that identity system administration. It runs
on every boot (safe, idempotent) — for initial setup and break-glass recovery.
- Set
subjectto the admin user's stable OIDCsubat your IdP andissuerto the IdP issuer URL — they must match the token the admin will present, or the grant won't apply to the right account. - Leave all three unset to skip bootstrap entirely (e.g. once admins are managed in-app).
- In the bundled stack these are preset to the demo user (fixed
subin the realm), so the stack self-bootstraps on first boot.
8. Email enablement (ADR-0041) + backlog note (ADR-0043)¶
Notifications are an outbox: the notification row is the source of truth, and a delivery pass on the scheduler tick (§9) sends unstamped rows via the channel. Email is disabled by default so a deployment without SMTP still delivers in-app notifications (REQ-NOT-007).
To enable email:
lqms.notifications.email.enabled=true.- Configure quarkus-mailer:
quarkus.mailer.host,.port,.from(a real address),.username/.password,.tls/.sslas your SMTP relay requires. - Optionally set
lqms.notifications.default-localefor recipients without a saved locale.
Backlog semantics you must know (ADR-0043): while email is disabled, notification rows accrue
with emailed_at unstamped by design. There is no age cap — when you later enable the
channel, the entire accumulated backlog is delivered on the next ticks. If you enable email on a
system that has been running a while, expect a burst of catch-up mail. If that is undesirable,
stamp or archive the backlog before enabling (operational step; no config flag exists for it).
9. Sweep tuning (ADR-0037)¶
The scheduler tick (lqms.tasks.sweep-interval, default 5m, advisory-locked so only one instance
sweeps) marks overdue tasks, escalates, and — when email is on — runs the delivery pass.
lqms.tasks.due-after(P14D): how long an open task may sit before it is due.lqms.tasks.escalate-after(P7D): grace after due before escalation.- Tune the interval to your delivery latency vs. load trade-off; the due/escalate periods to your
QMS SLA. Values are ISO-8601 durations (
PnD,PTnH, …).
10. Upgrade path¶
- Schema: Flyway runs at startup (
quarkus.flyway.migrate-at-start=true) and applies any newV0xx__*.sqlmigrations automatically. The current baseline is V001…V066 (latest: V065blob_quarantine, V066 calendar obligation schedules); a healthy boot log shows Flyway "Successfully applied" / "up to date" and the deployment-check line from §4. - Run migrations as the BYPASSRLS migration role (§4) — the whole point of that role. If you ever regenerate jOOQ or add migrations, rebuild the image with fresh generated sources (§2).
- Rollout: deploy the new image; on boot it migrates then runs the deployment/bootstrap checks. Because the sweep is advisory-locked and the session is a stateless cookie (ADR-0028), a rolling restart is safe. Keep only one schema-migrating instance starting at a time if you scale out (Flyway takes a lock, but avoid concurrent first-boot races).
- Backups: see §11 — take a database + content-store backup before a migration you cannot cheaply roll back (there is no automated down-migration).
11. Backup & retention window (ADR-0025 / ADR-0016)¶
Disposal and erasure are forward-only with respect to backups: erasing a record removes it from the live system immediately, but copies persist in existing backups until those backups age out. ADR-0025 therefore makes a bounded, configurable backup-retention window an operational requirement: erasure is "effective immediately in the live system, and in backups within the backup-retention window."
Operator obligation: define and enforce a backup-retention window per your data-protection policy, and ensure backups older than that window are actually deleted, so disposed/erased data does not linger indefinitely.
There is no application config key for this window today — it is an operational property of your
backup tooling/schedule (e.g. your pg_dump/PITR retention and content-store snapshot retention),
not something LQMS enforces. Document your chosen window and the deletion mechanism in your
operational runbook. Back up both the PostgreSQL database and the content store
(lqms.storage.root) together, since a document's metadata and its blobs must be restored
consistently.
12. Verifying a deployment¶
A healthy boot log contains:
- Flyway applying/confirming migrations V001…V066;
Deployment checks passed (fan-out DEFINER owner can bypass RLS).(§4);System bootstrap complete for admin subject '…'.when a bootstrap admin is configured (§7).
Then:
GET /api/health→200 {"status":"UP"}.GET /api/version→200 {"sha":"…","timestamp":"…"}— confirms the running build (public, no auth;shaisunknownif the image was built without theGIT_SHAbuild ARG, §2).GET /(browser) → redirect to the IdP login.- An authenticated
GET /api/mewith a valid bearer token →200with the provisioned user.
deploy/smoke.sh automates these against the bundled compose stack.
Deploy-image render guard (FWK005). The smoke's opt-in Stage 5 renders a merged audit-pack on
the real Temurin 25-jre runtime and asserts genuine PDF bytes — the only gate that exercises the
deployment JVM with content. The FOP FWK005 render fault is a runtime-JVM-only failure that no CI
job catches (all CI runs JDK 21; and while the tag-gated publish-image job now builds the deploy
image, nothing in CI ever runs it), so this smoke is its durable guard — run it before cutting a
release tag. Because it needs seeded released content it is
opt-in; deploy/nightly-smoke.sh runs it end-to-end on an isolated stack (build → up → seed →
render → teardown) as a single command for the nightly smoke watch.