← Docs home

Architecture

What runs where, how a score gets from a contestant’s PR to the leaderboard, how .env and the admin panel configure the app, and what the security model actually rests on. For why these choices were made instead of alternatives, see docs/decisions.md. For the contract a new CTF vertical must satisfy, see docs/modules.md. For day-to-day operation, see docs/operations.md.

Platform and modules

OWASP CTF in a Box is a control plane with modules plugged into it. The split is deliberate: the platform never knows what a challenge is, only how a score arrives and how a leaderboard renders; a module never re-implements org provisioning, teams, ingestion, or ranking. More than one module can be enabled at once — secure-development (targets, GitHub-mediated scoring, the worked example throughout this doc), quiz (a self-paced single/multi-select question bank, scored entirely inside the app — see Quiz data flow below), classic (a jeopardy-style flag board, also scored entirely inside the app — see Jeopardy data flow below), and ai (externally hosted AI/LLM challenges: the box mints a contestant’s identity for the outside site and grades or accepts a solve back, also scored entirely inside the app — see AI data flow below). See docs/modules.md §5 for what the three app-side modules’ UI contract still leaves open. An id outside the registry still fails the build loudly; the boundary is the module contract.

The platform (control plane) owns A module provides
The disposable per-event GitHub org and its lifecycle (setup/ctf-setup.sh). The targets it forks/provisions per event, and its teardown equivalents (contract §7).
Auth (GitHub OAuth sign-in) and the admins allowlist. — (uses the platform’s identity).
Team registration, roster, join codes, the dedupe rollup, and the requirement that a contestant be on a team before anything scores (apps/web, ctf:team:*). — (scores are per author; the platform maps authors to teams).
The scoring pipeline: the single audited writer POST /score, the poll transport, the github-actions[bot] trust filter (sync/, scorer/). Its scoring workflow and the score payloads it submits through that one writer (contract §2–3, §6).
Leaderboard ranking, points aggregation, the score-over-time series, and rendering (scorer/src/serve.js, apps/web). Its challenge catalogue — stable target/challenge IDs with totals — plus display metadata and progress semantics (contract §4–5).
The admin panel runtime overrides (freeze, hints, registration, module enablement, per-module display name/blurb) (ctf:admin:settings). — (inherits the controls; its registry displayName/description are the defaults an organizer’s moduleTitle:<id>/moduleBlurb:<id> override).
The two config planes (runtime config flow below): the .env bootstrap keys every container reads at start, and the ctf:admin:settings hash /admin writes and every reader re-reads live — including the event’s identity, the enabled module set and the Secure Development target list (#386). Its entry in the static module registry, plus whatever runtime settings it adds to that same hash through the validated settings path (contract §1).

Everything below — the services, the score data flow, the security model — is the platform. Where secure-development fills a module slot (its targets, its pull_request_target scoring workflow, its catalogue), it is called out as the worked example, exactly as the module contract does.

System overview

Everything runs as one docker-compose.yml stack (see decisions.md #2). Two independent things happen in parallel: contestants browsing the app, and scores flowing in from GitHub.

Animated diagram. One docker-compose box at runtime. The contestant browser reaches caddy over HTTPS; caddy proxies to the app; the app reads teams and hints from srh and the leaderboard from scorer; scorer is the one writer for secure-development score state, landing it in redis via srh; secure-development scores arrive one way, by sync polling GitHub and posting to scorer directly, the push tile where a scoring Action posted to /score having been removed in v0.6 per issue 377. Quiz, Jeopardy and AI score entirely app-side and never touch scorer.

The plain-text shape, for anything that can’t render the animation above:

                         contestant browser
                                 |
                                 | HTTPS
                                 v
                          +-------------+
                          |    caddy    |   reverse proxy; one Caddyfile,
                          +------+------+   whose only route is "/"
                                 |
                                 v
                          +-------------+
                          |     app     |   Next.js contestant UI
                          |  apps/web/  |
                          +--+-------+--+
           UPSTASH_REDIS_REST_URL   LEADERBOARD_API_URL
                          |               |
                          v               v
                    +-----------+   +-----------+
                    |    srh    |<--|  scorer   |   judges PRs;
                    | (Upstash- |   | (bearer-  |   GET /leaderboard, POST /score
                    |  REST     |   |  token    |
                    |  proxy;   |   |  authed)  |
                    |  POST-cmd |   +-----+-----+
                    |  subset)  |         ^
                    +-----+-----+         | POST /score (bearer token)
                          v               |
                    +-----------+    +----+-----------------------+
                    |   redis   |    | sync polls the event org's |
                    +-----------+    | forked repos' issue        |
                                     | comments via a GitHub App  |
                                     | token, then POSTs to       |
                                     | scorer directly — nothing  |
                                     | reaches the box inbound    |
                                     +----------------------------+

app reaches srh directly (team/hint data plus the leaderboard read adapter can go through it) and reaches scorer directly for LEADERBOARD_API_URL reads (docker-compose.yml’s app service sets both UPSTASH_REDIS_REST_URL: http://srh:80 and LEADERBOARD_API_URL: http://scorer:4000). scorer is the only writer to Redis-backed score state; everything else that touches scores goes through it.

Components

Service Source Responsibility
caddy caddy:2-alpine image (digest-pinned, ADR 51); the one Caddyfile, caddy/Caddyfile.poll, mounted unconditionally Reverse proxy in front of app, and that is its whole job: there is one Caddyfile, it has no /score route to scorer, and no setting adds one — so the box has zero inbound scoring surface.
app apps/web/ (vendored Next.js app, built from local source via apps/web/Dockerfile) Contestant-facing UI: GitHub sign-in, challenge browser, leaderboard, rules/FAQ/how-to-play pages. It reads nothing at build time: ADMIN_LOGINS and GITHUB_ORG come from the environment at container start, and the event’s dates, name and other identity fields, which modules run and which Secure Development targets run are all runtime /admin settings (see below).
scorer ${SCORE_IMAGE:-…} — your own build from the in-repo engine scorer/, which bakes the public vendored rubric by default (see docs/scorer.md); setup/ctf-setup.sh org mirrors whatever SCORE_IMAGE names into the event org. The compose fallback ghcr.io/owasp-ctf/score:latest is a private upstream image the kit does not assume access to. Judges submitted PRs against the baked rubric; exposes POST /score (bearer-token authed write) and GET /leaderboard. The one score writer in the system. Part of the secure-development module, so it carries profiles: ["secdev"], the same profile as sync: the two come up together or not at all. SCORE_IMAGE decides availability: non-empty is what adds a Secure Development profile at up (so this container exists at all) and what seeds the default module set, and with it empty the scorer cannot run. What is live is enabledModules in ctf:admin:settings, set from /admin — so an available Secure Development module can still be switched off, and a running scorer is not by itself an enabled module. See ADR 26, superseded by ADR 55.
srh hiett/serverless-redis-http Upstash-REST-compatible HTTP proxy in front of redis, so the app’s @upstash/redis client works unchanged against local Redis. Implements only the POST-command-array subset of Upstash’s REST API (no path-style GET /get/<key> shortcut — see scripts/smoke.sh).
redis redis:8-alpine (digest-pinned, ADR 51), --appendonly yes Durable state: scores, team/hint data. Named volume redis-data survives box reboots.
sync sync/ (Node, sync/src/*.js) The score transport (profiles: ["secdev"]). Polls the event org’s forked target repos’ issue comments with a GitHub App installation token, validates them, and forwards trusted score payloads to scorer. Also reads the organizer’s pause flag, master-reset epoch and Secure Development target list every tick and writes a heartbeat (see “Organizer admin panel” below). It no longer decides whether it is enabled — the compose profile does, so if the container is up it polls; a missing GITHUB_ORG is therefore a genuine misconfiguration and sync refuses to start, exiting non-zero and naming the key. restart: on-failure restarts exactly that, so the container comes back and logs the same refusal until GITHUB_ORG is set — which is what an organizer wants, rather than one message scrolling away in a container nothing brings back.

Data flow for a score

Animated diagram. A contestant opens a PR; a pull_request_target Action judges the patch in the base repo; it posts a PR comment carrying the score marker, the one score transport since push ingest was removed in v0.6 per issue 377; sync's tick then filters comments by author, parses and validates, then POSTs to /score itself; scorer is the one writer, landing the score in redis via srh monotonically; the app then reads GET /leaderboard and composes the overlay pipeline (module-contributions, then team-standings, then hint-penalties, then score adjustments last) before rendering. The score marker is trust-authoritative and only ever comes from the judge's own output, never from the PR checkout.

  1. A contestant forks a target repo in the event org, patches a vulnerability, and opens a PR back to the org’s copy.
  2. A pull_request_target GitHub Action (rendered per target from the in-repo template scorer/consumer-workflow.example.yml by setup/ctf-setup.sh org, which commits it to each fork automatically, see docs/modules.md §6.1) runs in the base repo’s context — where org secrets live — and scores the patch using the private scorer image, while the contestant’s PR code runs sandboxed with no access to those secrets.
  3. The Action reports the result the only way there is: a PR comment authored as github-actions[bot] carrying a machine-readable marker, <!-- ctf-score: {...} --> (sync/src/parse.js’s MARKER). Nothing in the workflow talks to the box at all — push ingest, where the Action POSTed the score straight at a public /score, was removed in v0.6 (#377, ADR 56) together with the judge’s SCORE_API/SCORE_TOKEN hook and the LEADERBOARD_URL/LEADERBOARD_TOKEN org secrets that addressed it. An event org still holding those secrets should delete them: nothing reads them, and every run a contestant’s PR triggers can read them (Status and upstream dependencies).
  4. sync’s next tick (sync/src/index.js’s tick()) calls fetchNewScoreComments (sync/src/github.js), which fetches issue comments since the last cursor and filters by comment author (cfg.commentAuthor, default github-actions[bot]) before anything else runs — a forged comment from any other login is dropped at this step, never reaching JSON parsing.
  5. parseScoreComment (sync/src/parse.js) extracts the JSON block and validates author against the GitHub-login grammar (GITHUB_LOGIN), target against the configured target list, and solved as a string array, before returning a payload. The poller’s seen set is keyed by comment revision — its id AND its updated_at — not by id alone. The scoring workflow posts ONE comment per target and EDITS it (a “scoring in progress” placeholder, then the result), so an id-only key meant a PR whose first run produced no score burned its id on the placeholder and could never be scored afterwards: the edit carrying the real result was skipped, silently, and the cursor advanced past it. Re-presenting a revision is safe because the scorer’s write is monotonic and idempotent on replay (step 7).
  6. submitScore (sync/src/submit.js) POSTs the validated payload, plus at — the comment’s own GitHub updated_at, never a field of the marker — to POST /score on scorer with a bearer token (Authorization: Bearer ${cfg.scorerToken}), over the compose network — that endpoint is reachable from inside the box and nowhere else. The scorer compares the token in constant time: both sides are SHA-256’d and passed to timingSafeEqual, so neither the token’s bytes nor its length are recoverable from how long a rejection takes. A 2xx is success; a 422 is the scorer’s final verdict on when the score was posted (logged as refused, never retried, not counted as dropped); any other 4xx is treated as a permanent rejection (dropped, logged); anything else throws and the poller un-marks the comment as seen so it retries next tick. The poller does not hold its cursor while scoring is frozen or outside its window: it reads the effective freeze only for its heartbeat.
  7. scorer judges the score at at (ADR 67): an unparseable at is a 400, one more than five minutes in the future is clamped to now, and one before the master-reset epoch, outside the scoring window (outsideScoringWindow) or inside a freeze (pausedAt, over pausedSince and the ctf:admin:pauses interval log the app writes with the flag) is refused with a final 422. A freeze with no recorded start answers 503, which the poller retries. A payload with no at (an older sync) keeps the arrival-time rule. The scorer then writes the score to Redis (via srh), timed at, as a monotonic, idempotent-on-replay update — the write model is described in docs/decisions.md #5. Because the comment is edited on every run, at is the time of its last run: a solve first read from a later re-run takes the re-run’s time.
  8. app reads GET ${LEADERBOARD_API_URL}/leaderboard and renders the result on the contestant-facing leaderboard page. Alongside the ranked leaderboard/teams standings, the payload carries a top-level catalog (per target, each challenge’s id/name/points/owasp, derived from the rubric — owasp is carried only by exec-grammar catalogues, null for declarative YAML targets) and a solvedIds array on every entry’s and team’s apps.<target>. The app joins the two to show which flags are solved — the collapsible per-target list under a contestant’s breakdown, and a team’s per-target flags (solved by its members’ union, plus the ones still open), both drawn by the same components/progress/ row the profile uses. Where a challenge carries an owasp code the list groups by it, most winnable group first; a target whose catalogue reports owasp: null throughout (a declarative YAML target — see above) has nothing to group by and renders one unheaded, unbadged list instead. Both fields are additive; an older scorer that omits them simply falls back to the solved/total counts.
  9. Before rendering, the app composes the fetched LeaderboardData through a fixed pipeline (src/lib/leaderboard/folded.ts, memoized across requests for 10 s and shared by every concurrent viewer — the result has no per-viewer input; a fold that throws is never cached): withModuleContributions → withTeamStandings → withModuleSeries → withHintPenalties → withScoreAdjustments (src/lib/leaderboard/{module-contributions,team-standings,module-series,hint-penalties,score-adjustments}.ts). withModuleSeries sits third because it charts each team’s roster and needs the team rows the standings stage has just built; it reads the app-side modules’ per-item timestamps to put their points on the chart and leaves points untouched, so the penalty stage still nets the full module total. withScoreAdjustments runs after it: it applies the organizer’s per-team adjustments from ctf:adjustments to each team’s netted total, floors it at 0, and re-ranks the teams. Individual rows are not adjusted, and the chart stays gross. withModuleContributions attributes each row’s points into a per-module ModuleProgress for every enabled module — secure-development is attributed, not added, since its points already came from the scorer above; quiz, classic and ai each score entirely app-side, so none of those modules’ points are ever inside entry.points to begin with and all three are added on top instead (entry.points += quizTotal.points + classicTotal.points + aiTotal.points) — see Quiz data flow, Jeopardy data flow, and AI data flow below. Hint penalties fold after every module stage, netting the final all-module total exactly once — module blocks everywhere show their gross contribution and the row’s −N hints marker is what reconciles them against the netted header. Netting earlier, before the app-side modules’ points are added, would make hints free for any row whose points arrive later: a classic- or quiz-only contestant. Ranking itself (src/lib/leaderboard/rank.ts’s compareStanding) is: combined points descending, then items completed across modules descending, then earliest last-activity ascending (#522 — until v0.7 items came first), with a patched/lastSolveAt fallback for sources that carry no per-module data. Teams rank by the same three keys (compareTeamStanding): the activity time is the latest of every module block’s and the scorer’s team lastSolveAt, so a points tie goes to the team that earned its last points first in any module, not to the scorer’s Secure-Development-only order. A Secure Development time is when its score comment was posted on the PR (step 7), not when the box read it. An expanded leaderboard row then renders each enabled module’s own detail block (components/module-detail.tsx switches on moduleId — a secure-development row shows the existing per-target breakdown, and a module with a different progress shape defines its own). An expanded team row instead renders the shared progress tree read-only (components/leaderboard-team-row.tsx → components/progress/progress-row.tsx): one row per module carrying that module’s points and its done/total in the module’s own unit word, secure-development opening into the same per-target rows the profile draws. Every enabled module is on an expanded row from the start (#585): a row that has completed nothing in a module gets a zero block, 0 / N and 0 pts, rather than no block — but only when the module’s totals read succeeded and its catalogue is non-empty, so a failed read still degrades to absence and nothing reads 0 / 0. A team with targets gets its secure-development block at zero too, so its per-target rows stay on the row beside the others. A zero block adds no points, items or activity time, so totals and order are unchanged, and a row’s per-item lists load only once something outside Secure Development is completed. The team’s own hint spend is not shown there — it is that team’s business — so the header total is labelled net pts instead of carrying a −N hints marker.

Leaderboard with no scoring backend

Steps 1–8 above assume secure-development is enabled — there is a scorer, and LEADERBOARD_API_URL/LEADERBOARD_SOURCE name a real backend to read. When it’s disabled (a quiz-only, classic-only or ai-only event, or any event with no scored module), none of that pipeline runs at all: getLeaderboardSourceMode (src/lib/leaderboard/source.ts) checks isModuleLive("secure-development") before looking at LEADERBOARD_SOURCE, and — not overridably by that env var — resolves to "empty" instead, serving emptySource (src/lib/leaderboard/empty.ts): no entries, no teams, every capability false. This is deliberately not the mock source; placeholder data would be indistinguishable from real standings on a board that also carries real quiz, classic or ai points.

Everything a contestant sees on such a board is then built by the overlay pipeline itself, on top of nothing. withModuleContributions creates a row for any login that holds module (quiz, classic and/or ai) points and has no entry from the source — the board’s login set is the union of the source’s logins and the logins holding module points, matched case-insensitively, so a contestant with quiz, classic or ai points but no scored PR gets a row instead of staying invisible until one exists — and a login holding more than one module’s points gets ONE row carrying every held block, never one row per module. A created row has every scorer-supplied field (patched/failed/total/apps) genuinely zero — there is no scoring entry behind it — and its only points are the modules’, added rather than attributed (see Quiz data flow, Jeopardy data flow and AI data flow below for why those are different verbs). withTeamStandings also gives every team member an Individual row: a member with no points yet gets a 0 row, ranked after everyone who scored and by name among the zeros, so a player who can find their team on the Teams view can find themselves too. The rosters are the team store’s and the source’s own; signing in without a team creates no row, though a player who scored before leaving their team keeps theirs. It does the same one step later for teams: every team row — the source’s own, and the membership-only rows it synthesises from live team records the source does not know — gets quiz, classic and ai points added, once, via withTeamQuizPoints, withTeamClassicPoints and withTeamAiPoints, each deduped by question/flag/challenge across members, so a quiz-only, classic-only or ai-only event’s default view — the teams board, whenever teams exist — doesn’t open on every team tied at zero. See decisions.md #25 for why the board is built this way.

Quiz data flow

Animated diagram. A contestant submits an answer; a cheap JS gate pre-check runs first; the real authority is one atomic Lua script that rechecks the cap and cooldown against current state, compares the answer, and on a match writes the answer row and bumps the aggregates; quiz points are then ADDED to the leaderboard, never attributed, and a team's total is the union of its members' correct answers, never their sum.

The quiz module never touches scorer, sync, or GitHub — it’s the app’s own, entirely separate scoring path, running inside apps/web against Redis keys it owns outright. apps/web/src/lib/quiz-store.ts is the only writer during normal contestant and authoring activity — answering, grading, question authoring/deletion all go through it — but two admin-store.ts bulk-maintenance paths touch ctf:quiz:* directly rather than calling into quiz-store.ts: seedDemoData() (HSETs the questions key, the answer key, a per-login answers hash, both aggregate hashes and ctf:quiz:lastAt when seeding demo data) and the master reset’s scanDelByPrefix() (SCAN+DELs ctf:quiz:answers:*/ctf:quiz:attempts:*/ctf:quiz:points/ ctf:quiz:answered/ctf:quiz:lastAt — see “Master reset” below). Both reuse quiz-keys.ts’s shared key constants and canonicalizeChoices recipe rather than re-deriving them, so the two writers can’t silently disagree on key names or answer-set format even though they’re separate code paths:

Grading is one atomic Lua script, not a sequence of round trips: reading the current attempt count and cooldown, re-checking the cap and cooldown against the current admin settings, bumping the attempt counter, comparing the submission against the stored key, and — on a match — writing the answer row, incrementing both aggregate counters and stamping ctf:quiz:lastAt, all happen inside a single script execution. The JS-side quizGate pre-check that runs before the script is only a cheap early-out over its own separate, non-atomic read; the script is what actually closes the race, because Redis runs it to completion before starting the next one, so a burst of near-simultaneous submissions on the same question can’t collectively spend more attempts than the cap allows.

Fail-closed — deliberately the opposite of the scoring freeze. If the gate’s attempt/answer lookup itself errors, it refuses the answer (a distinct "unavailable" reason) rather than guessing. This is the inverse of effectivePaused’s fail-open behavior below (a Redis blip must never silently drop a live, already-judged PR submission): for the quiz, the safe failure on an unverifiable lookup is “don’t grade it,” not “grade a possibly-replayed submission.”

Quiz points are ADDED to the board, not attributed from it. withModuleContributions (src/lib/leaderboard/module-contributions.ts) handles all four enabled modules, but splits them by how their points arrive: secure-development’s points are already inside entry.points (the scorer computed them), so the overlay only attributes that existing figure into a ModuleProgress block, while quiz, classic and ai are each added by the same rule the rest of this section describes for the quiz. The quiz never submits anything through scorer’s POST /score — the web app holds no score-writing token for that endpoint at all, so there is nothing for it to authenticate as a writer with — its points are computed and stored entirely by the app, so they must be added onto the scorer-sourced total (entry.points += quizTotal.points) before the combined board re-ranks.

A team’s quiz total is the union of its members’ correctly-answered questions (getTeamQuizTotalsBatch), never the sum of their individual aggregates — summing would double-count a question two teammates both answered, exactly like a shared flag would double-count under naive summation. Individual rows read the cheap per-login aggregate counters instead (getQuizTotals); only a team standing pays the per-member HGETALL cost. That happens in exactly one place: withTeamStandings calls withTeamQuizPoints (and its classic and ai counterparts) once, over the union of the source’s own deduped team rows (scorer, mock or lambda, with rosters merged from the team store) and the membership-only rows it synthesises for teams the source does not know — the only point in the pipeline where every team exists. withModuleContributions, one stage earlier, stamps a source team’s secure-development block and nothing else. It used to add the app-side modules to the source’s teams too, so on a scorer-sourced board every team counted its quiz, Jeopardy and AI points twice (issue #520). Those per-member reads for every team on the board go out in a single pipeline (one HGETALL per distinct member, not one round trip per team), because /leaderboard is dynamic and fetched no-store — a per-team round trip would bill an event one REST call per team on every page view.

The overlay’s two quiz reads are settled independently: getQuizTotals supplies the points, listQuestions only the “answered / total” denominator. A failed question-list read degrades to a missing denominator (clamped to at least the answered count, so the ratio can never read “1 / 0”), never to lost points — points and the ranking they drive must not hinge on a cosmetic read.

The master reset (below) wipes ctf:quiz:answers:*, ctf:quiz:attempts:*, ctf:quiz:points, ctf:quiz:answered and ctf:quiz:lastAt — contestant progress — but deliberately leaves ctf:quiz:questions and ctf:quiz:key untouched, the same way it leaves ctf:admin:settings untouched: both are organizer- authored content, not event-run state a reset should ever destroy.

Jeopardy data flow

Animated diagram. A contestant submits a flag; a cheap JS pre-check runs first, failing open on a paused or out-of-window read but closed on a cooldown-lookup error; the real authority is one atomic SUBMIT_SCRIPT that rechecks the already-solved guard and cooldown against fresh state, compares the flag's normalized form, and on a match writes the solve row and bumps the aggregates; Jeopardy points are ADDED to the leaderboard, never attributed, and a team's total is the union of its members' solved challenges, never their sum.

The classic module is the jeopardy-style flag board: an organizer authors a set of challenges, each hiding a flag under a description; a contestant reads the description, finds the flag by whatever means the challenge calls for, and submits the string for points, graded instantly. Like quiz, it never touches scorer, sync, or GitHub — it is a second, entirely separate app-side scoring path, running inside apps/web against its own Redis keys. apps/web/src/lib/classic-store.ts is the only writer during normal contestant and authoring activity (submitting, grading, challenge authoring/deletion all go through it); admin-store.ts’s bulk-maintenance paths (demo seed, master reset) are the one documented exception, reusing classic-keys.ts’s shared key constants directly rather than calling into classic-store.ts — the same deliberate exception quiz-store.ts documents.

Authoring produces TWO flag hashes, not one, keyed by challenge id:

Both are written together in one Upstash pipeline call inside upsertChallenge, so they can never observably disagree — a challenge can never be live with a flagnorm belonging to a previous version of its flag. See decisions.md’s ADR on two flag hashes for why the store keeps both rather than one.

Normalization happens in JS, on both the authoring and submission paths, and deliberately NEVER in Lua. normalizeFlag is the one function either side may use, precisely so they can never independently drift; Lua’s string.lower is ASCII-only, so a Lua-side re-normalization of any non-ASCII flag would disagree with the JS side and produce a challenge nobody could solve. SUBMIT_SCRIPT (below) receives an already-normalized value and compares whole strings with Lua’s == — a flag can contain braces, quotes, and backslashes, so it is never pattern-matched out of a JSON blob the way a points value is.

The full key layout is thirteen ctf:classic:* keys, all named in classic-keys.ts: challenges (the public-safe hash contestants see — no field on it could carry a flag even by accident), flag and flagnorm (above), hints (paid-hint text per challenge — written by the admin form, SECRET until purchased through hint-store’s reveal, exactly the flag hashes’ rule), categories (one JSON array, the organizer’s chosen display order), stories (one JSON list of {id, title, intro, steps} story arcs, authored content like the categories, so a master reset keeps it), opened:<login> (challenge id to the time that contestant first opened its page, written once with HSETNX; Insights measures time to solve from it, and a master reset wipes it), solves:<login> (a contestant’s banked solves — {points, at}, points captured at solve time so a later re-price never rewrites history), attempts:<login> (every submission, right or wrong — {attempts, firstAt, lastAt, lastAtMs}, the cooldown’s own read; Insights falls back to firstAt for time to solve when no first-open time exists), and four running aggregates: points and solved (per-login totals the leaderboard overlay reads with flat HGETALLs regardless of board size), lastAt (each login’s latest award time, the leaderboard’s tiebreak, #522) and solvecount (the per-challenge distinct-solver count, distinct by construction because the already-solved guard runs before any write; the admin list shows it as the players column). The contestant board shows a different figure, teams that solved each challenge (#595, lib/team-solve-counts.ts), folded from the members’ solves hashes on read. The same read names each challenge’s first blood (#188, teamSolveBoard and lib/first-blood.ts): the member with the earliest solve row at, an exact tie going to the case-insensitively first login. It is display only and has no key of its own, so no grading script writes it and every reset that removes a solve row or a membership removes it.

Submission: POST /api/classic/submit derives login from the session (never the request body) and calls submitFlag(login, challengeId, flag). A cheap, non-atomic JS pre-check (evaluateGate) short-circuits on, in order: scoring paused/outside the scheduled window (fails open — a Redis blip must never silently drop a submission a contestant is entitled to make), already solved, or still inside the cooldown (fails closed, with its own "unavailable" reason, if the lookup itself errors). Past the pre-check, one atomic Lua script — SUBMIT_SCRIPT, not the pre-check — is the actual authority: it re-reads the already-solved guard and the cooldown against state read fresh at script-execution time (never a value the caller read earlier), so a race that slips past the pre-check is still caught, atomically. On a correct submission it reads the challenge’s current price off the challenge hash, writes the solve row, bumps the three counters (points, solved, solvecount) and moves lastAt forward, all in the same script execution.

There is no attempt cap anywhere in this gate — only a cooldown, in SECONDS. classicCooldownSec (organizer-configurable, default 5, capped at 3600 — CLASSIC_COOLDOWN_SEC_MAX in admin-store.ts) is the only knob; 0 disables it. This is worth stating explicitly because every neighbouring retry-gate setting on this platform (quizRetryAfterMin, hintsUnlockAfterMin) is expressed in minutes — classic’s is seconds, because the job is blunting scripted brute force on a short timescale, not rationing genuine tries the way quiz’s attempt cap does. See decisions.md’s ADR on no attempt cap.

Points are static. SUBMIT_SCRIPT reads a challenge’s price off ctf:classic:challenges at the moment of a correct solve; nothing anywhere lowers it as more contestants solve it, and there is no first-blood bonus. Re-pricing a challenge later never changes what was already banked, because solves:<login> captures points at solve time.

Descriptions render through a hand-rolled Markdown subset (apps/web/src/lib/markdown.ts): bold, italics, inline code, fenced code blocks, ordered/unordered lists, and links restricted to an http:/ https:/mailto: scheme allowlist (control characters and whitespace stripped before parsing, scheme-relative //host rejected outright). The parser produces a typed node tree, never an HTML string, and components/markdown.tsx renders that tree into React elements — dangerouslySetInnerHTML is never called anywhere in the pipeline, so injected markup is structurally impossible rather than filtered out. See decisions.md’s ADR on the hand-rolled renderer.

Jeopardy points are ADDED to the leaderboard, never attributed — the scorer never sees a flag, so there is nothing of classic’s to attribute from (same reasoning as quiz’s points; see Quiz data flow above). A team’s classic total is the union of its members’ solved challenges (getTeamClassicTotalsBatch), never the sum of their individual aggregates, for the same double-counting reason a shared flag or a shared quiz answer would otherwise double count. That union-by-item fold is not classic’s own logic: it is leaderboard/team-fold.ts’s foldTeamItems, the identical function quiz-store.ts calls for its own team total — one shared dedupe rule (earliest-record-wins on a tie, latest timestamp for “last activity”) rather than two copies that could silently diverge. See decisions.md’s ADR on the shared fold.

Secrecy is a contestant boundary, not an absolute one — mirroring ctf:quiz:key. listChallenges (the contestant path — /flags, and the leaderboard’s read of the catalogue) never issues a command against ctf:classic:flag or ctf:classic:flagnorm, and the Challenge shape it returns has no field that could carry a flag. listChallengesForAdmin (the GET /api/admin/classic surface, behind requireAdmin) DOES return a challenge’s flag, in a separate AdminChallenge shape ({challenge, flag}) deliberately not assignable to Challenge — reaching the public half takes an explicit .challenge, so handing an admin record to a contestant-facing component is a compile error, not a leak someone has to notice in review. A flag is genuinely stored in plaintext and visible to anyone with /admin access — see docs/operations.md’s “Jeopardy” section for the organizer-facing statement of that trade-off.

Deleting a challenge retires it — contestant history and banked points are untouched. deleteChallenge removes the challenge and both flag rows, but deliberately leaves solves:<login>/attempts:<login> rows and the three aggregate counters alone, mirroring deleteQuestion. Points already banked for a deleted challenge stay on the leaderboard; only the master reset clears them.

Visibility is a field on the public record, decided where it is read. A challenge stored with enabled: false is hidden from contestants: it does not exist on their side, unless its releaseAt instant has come (now >= releaseAt, the instant itself released). The store keeps a release time only while it is in the future, always with enabled: false, next to releaseAtMs, its epoch-ms mirror, because the Lua scripts compare it with Redis’s TIME and parse no ISO dates. Nothing is scheduled or stored per reader; pages read with the server’s clock, the scripts with Redis’s, and a page can lag the grader by at most the leaderboard fold’s 10-second memo. Every reader asks one pure predicate, isChallengeVisible in lib/classic-release.ts, and filters before anything is built. listChallenges returns hidden challenges too, so each contestant caller filters through visibleNow (the board, the board-items API, the profile, the landing count, the leaderboard and display-mode denominators, the /api/hints texts) or asks classicVisibility (the challenge page, its metadata, the attachment download); classic-release-coverage.test.ts fails a caller that does neither. Story positions are built over the visible challenges, so a hidden step is skipped like a deleted one. The two scripts that write decide for themselves from the record they read: SUBMIT_SCRIPT reads the challenge’s record first and answers a hidden one missing before the already-solved guard or any write, and the hint REVEAL_SCRIPT does the same from KEYS[8] before the story lock, the text or any charge (CLASSIC_HIDDEN_LUA in classic-keys.ts, which reads TIME only for a disabled record with a release time). The JS pre-check leaves a hidden challenge’s “solved” and “cooldown” to the script, so neither confirms it exists. An admin preview’s dry run skips the check, and so does an admin looking at a hidden challenge after launch: classicVisibility opens its page to them with a “Hidden from contestants” banner, and the submit and reveal routes grade their flag or hint as a dry run (adminSeesHidden). The admin list (/api/admin/classic) returns every challenge with its field, for the status chip and the form. See ADR 68.

AI data flow

Animated diagram. The challenge page mints an Ed25519 launch token after four gates and embeds it in the launcher href. A solve can arrive three ways: an in-box Server Action that re-runs the gate order, an external submit endpoint authenticated by the token alone, or an external event endpoint authenticated by an HMAC signature then the token then a replay nonce. All three funnel into one shared atomic AWARD_SCRIPT. AI points are then ADDED to the leaderboard, never attributed, and a team's total is the union of its members' solved challenges, never their sum.

The ai module is externally hosted AI/LLM challenges: an organizer authors each challenge in /admin (mode flag/event/both, a launch URL template, categories, an optional paid hint, a submission cooldown), and a contestant plays it on the outside site or types a flag back on /ai/[id]. Like quiz and classic, it never touches scorer, sync, or GitHub — a third, entirely separate app-side scoring path, running inside apps/web against its own Redis keys. apps/web/src/lib/ai-store.ts is the only writer during normal contestant and authoring activity; admin-store.ts’s bulk-maintenance paths (demo seed, master reset) are the one documented exception, reusing ai-keys.ts’s shared key constants directly — the same deliberate exception quiz-store.ts and classic-store.ts document. The one thing ai needs that neither sibling does is an identity to hand the outside world, which is why the module also owns a launch-token mint and an external-event intake, both described below.

Identity out: the launch mint. /ai/[id]’s Server Component (apps/web/src/app/(site)/ai/[id]/page.tsx) is the ONE place in the app that mints a launch token, via mintLaunchUrl/buildLaunchClaims (lib/ai-launch.ts). This is gate-at-mint: the render checks the module is live, then reads the session, then runs redirectIfNotLaunched() (the launch lock, #464 — before launch a non-admin is sent to /), then redirects a teamless contestant away — all four before the mint is ever reached, so before the event is launched only an admin previewing the page can hold a token (and nothing it reports scores until the scoring window opens), and there is no code path above the mint that calls it without a login in hand. The token is Ed25519 (ADR 53), signed with the module-wide keypair in ctf:ai:launchkey, minted lazily on first use; its claims carry the player’s login (sub), the one challenge it is scoped to (aud), and a capped progress snapshot across the whole board. It rides in exactly one place — the launcher <a>’s href on the challenge page — and nowhere else in the app renders a token-bearing URL. The public half is what an external backend or a pure static SPA verifies against, served by the one unauthenticated, cacheable route GET /api/ai/launch-key.

Flags in: three surfaces, two store functions, one script. A solve can arrive three ways, and every one folds into the same atomic Lua script:

All three surfaces funnel into submitAiFlag/awardAiEvent, which share one AWARD_SCRIPT (ai-store.ts) — sharing is deliberate, because two scripts would eventually disagree about the already-solved guard that makes the solve counter distinct-by-construction. The script itself refuses an event assertion against a challenge authored as mode: "flag", so a missed mode-check in the route cannot turn every flag-only challenge into something any signing-key holder can assert.

The key layout is fifteen ctf:ai:* keys, split by secrecy class:

Grading is one atomic Lua script, exactly like quiz’s and classic’s: the already-solved guard, the cooldown (graded path only — a signed event has no wrong answer to rate-limit), the flag comparison, the solve row, the three aggregate counters and the award time are read and written inside one script execution, against state read fresh at that instant rather than a value either caller read earlier. The JS-side pre-check (evaluateGate) that runs before it is only a cheap early-out; the script is what actually closes the race.

Fail directions, and they don’t all point the same way. The launch lock is closed — a settings read that fails counts as not launched, the same direction quiz’s own gate lookup takes on an unverifiable read, because a mint or a solve is exactly the kind of write a gate exists to hold back. Team membership is open — hasTeam’s own catch resolves to true, so a team-store blip never drops a submission an already-teamed contestant is entitled to make. The pause/schedule settings read is open for the same reason effectivePaused is elsewhere: a Redis blip must not silently freeze a live award. GET /api/ai/launch-key is the one closed route here in a different sense — wrapped in aiRoute, a thrown store error answers 503 {error:"unavailable"} rather than an empty or partial key, because handing back a bad key would have every integrator cache something that verifies nothing.

AI points are ADDED to the leaderboard, never attributed — the scorer never sees a launch token or a flag, so there is nothing of ai’s to attribute from (same reasoning as quiz’s and classic’s points; see Quiz data flow above). withModuleContributions (src/lib/leaderboard/module-contributions.ts) creates a row for any login that holds ai points and has no entry from the scoring source, exactly as it does for quiz and classic — a login reported by more than one app-side module still gets exactly ONE created row, carrying every reported block. Gross module blocks render everywhere, with hint penalties folding after every module stage at the row level, same as every other module.

A team’s ai total is the union of its members’ solved challenges (getTeamAiTotalsBatch), never the sum of their individual aggregates — summing would double-count a challenge two teammates both solved. It reads every member’s solve hash in one pipeline for the whole board and folds them through the same shared foldTeamItems (leaderboard/team-fold.ts) that quiz’s and classic’s team totals use — one dedupe rule, not three copies that could silently diverge.

Master reset clears progress, nonces, and the launch key — never the catalogue. resetEvent’s RESET_PREFIXES wipe ai’s solve/attempt rows, the three aggregate hashes, the award-time hash, and every spent replay nonce, but deliberately leave ctf:ai:challenges/ctf:ai:flag/ctf:ai:flagnorm/ctf:ai:hints/ ctf:ai:signkey/ctf:ai:categories untouched — organizer-authored content, the same rule quiz’s and classic’s questions/challenges get. Unlike those two siblings, the reset also deletes ctf:ai:launchkey outright: a master reset starts the event over, so no live launch token should survive it, and the next launch mints a fresh keypair — every already-issued token stops verifying, and any deployed external verifier has to re-fetch GET /api/ai/launch-key on its next check. clearAiChallenges (ai-store.ts), used by a whole-event archive import rather than the reset, is the opposite on both counts: it wipes the catalogue (challenges, both flag hashes, hints, signing keys, categories, and the per-challenge solvecount) while leaving contestant history and the launch keypair alone, because rotating identity on every archive import would break every deployed integration for a wipe that was only ever meant to replace the challenge list. See docs/ai-module.md §9 for the integrator-facing statement of the same rotation contract.

Contestant and team state

A team is required to score (ADR 47). POST /api/quiz/answer and POST /api/classic/submit refuse a teamless login with 403 { error: "no-team" } — after the launch lock, before the store call, and (on the classic route) before the body is even parsed, so the refusal cannot become an oracle for whether a flag was correct. The page-level redirect to /profile#team is signposting on top of that, not the boundary. The check fails open: a Redis blip lets the submission through rather than dropping a correct answer.

Key builders for all of the above live in apps/web/src/lib/team-keys.ts — a dependency-free module, the same pattern as quiz-keys.ts/classic-keys.ts, so readers that must not import the server-only store can still name the keys without open-coding the strings.

Organizer admin panel (runtime overrides)

.env’s ADMIN_LOGINS (checked case-insensitively against the signed-in GitHub login, apps/web/src/lib/admin-auth.ts’s requireAdmin) gates the runtime-override layer described above — the plane that is readable and writable while the stack is running, without a restart:

Support operations (ADR 48)

POST/DELETE /api/admin/ops/user and /api/admin/ops/team, behind requireAdmin, act on one contestant or one team: look up, reset progress, delete, remove from team, transfer captaincy, disband. They exist because the master reset was previously the only destructive control, so a single stuck contestant mid-event meant choosing between doing nothing and wiping the event.

The GET is gated as hard as the writes — one named contestant’s team, points, attempts and hint spend is precisely the read a non-admin must never have. The team overrides drop team-store’s captain guard (an organizer acts on a team they are not on) but keep the existence and membership checks inside the Lua, so an admin path is not the one that races a contestant clicking Leave.

Secure Development solves can be deleted but not kept deleted. The scorer writes them with HSETNX so replays no-op, and the poller re-submits from PR comments — so a per-contestant reset clears them and the next re-score writes them back. A sync restart does not: the poller’s cursor is in Redis (ctf:sync:state, below), so a restarted poller does not re-read the comments it already consumed. resetEvent solves this globally by freezing and bumping resetAt; there is no per-login equivalent, so the API returns a warning instead of pretending. Quiz, classic and ai writes originate in the app, so those deletes are final.

Engagement metrics (ADR 50)

GET /api/admin/metrics (JSON, or ?format=csv for the per-challenge table) folds the funnel, per-challenge difficulty, solves-over-time, module split and hint usage entirely out of keys the modules already maintain. There is no collection step and no new write path, and nothing is fetched from a fork — authenticating a fork means a credential every contestant can read, so fork-reported engagement would be forgeable by the contestants it measures.

Admin-only permanently: the aggregates are harmless, but the payload is computed from per-contestant rows, so every field added later is one edit away from carrying a login. The response ships its own caveats array, because a metric whose limits travel separately from it gets quoted without them.

What it reads. Four aggregate reads in one pipeline (ctf:quiz:points, ctf:classic:points, ctf:ai:points, ctf:hints:spent), a SCAN of ctf:solves:* for Secure Development, listTeams(), and then ten reads per contestant (PER_LOGIN) — their quiz answers, classic solves and ai solves, the three matching attempt hashes, firstTeamAt off ctf:user:<login>, their hint purchase times, and the classic and ai first-opened hashes — batched 200 commands to a round trip. Around those it also reads the three catalogue hashes (ctf:quiz:questions, ctf:classic:challenges, ctf:ai:challenges) for challenge titles only, the leaderboard source for Secure Development points, and the folded leaderboard for team points, so a team’s total here matches the board’s. The module contract for those row shapes is docs/modules.md §10.

Who counts as a contestant is the union of everyone on a team, everyone with points in any module, and everyone with a user record (ctf:user:<login>, found by SCAN; the ctf:user:<login>:hints sets the same pattern matches are skipped). Team membership is what makes stuck measurable: someone who attempted everything and solved nothing has no points row, so only their team knows they exist. The user records are what make “ever on a team” honest: a player who joined a team, left it and scored nothing is on no roster and in no points hash, so without them the figure missed exactly the people it exists to count. If the scan fails, the panel says so in caveats rather than serving the short count as a measurement.

Logins are read under every spelling the box holds. Per-login hashes are keyed on the login as the session spelled it (ctf:classic:solves:ChrisZoc); only the joins between logins lowercase. So each contestant’s rows are read under every stored spelling (rosters, the points and hint-spend hashes, the scorer’s entries, user records) and merged. Reading under the lowercased login alone hid 12 of 42 RTS contestants from every figure on the panel.

Team points are the leaderboard’s. The per-team figure comes from the same folded leaderboard the board shows (getFoldedLeaderboard({ fresh: true })), so the two cannot disagree. When the board cannot be read, the panel falls back to summing members’ totals and says so in caveats.

The fold is capped at 2000 contestants (MAX_CONTESTANTS), far beyond what the kit targets — the cap exists so a runaway key space cannot turn an admin click into an unbounded read. When it bites it says so in caveats, because a silently truncated metric reads as a complete one.

On demand, never cached. The fold is O(contestants), so it runs on the button rather than on arrival, and the button doubles as the refresh: an organizer re-reading it mid-event wants the current number, not one from a minute ago.

Aggregate counters are deliberately not read. ctf:classic:solvecount and ctf:ai:solvecount would be free per-module shortcuts for per-challenge solves, but folding each contestant’s own rows produces the same figure for all three app-scored modules from one source. Reading both would invite the two to disagree with no way to tell which was right.

The solve-rate denominator has a floor. It is max(people with an attempt row, people who solved it), not the attempt-row count alone: an earned row can exist without an attempt row, because the demo seed writes answers directly and anything predating the attempts hash has the same shape. Dividing by attempt rows alone produced solve rates of 200% and 300% on a seeded event — nonsense on its face rather than a subtle inaccuracy — so the larger of the two is both the correct denominator and the floor that keeps the rate inside 0..1.

Freeze = judged at the comment’s time, not stop execution. Setting paused does not touch fork Actions or GitHub — PRs keep getting judged and commented on. The poller does not hold its cursor either: sync/src/index.js’s tick() reads redis.isPaused() only to write the paused heartbeat, and forwards every score with its comment’s updated_at. The scorer reads the freeze, the window and the reset epoch on every POST /score that carries at (store.readScoringPolicy()) and refuses with a final 422 a score posted while any of them closed scoring (scorer/src/serve.js’s judgeScoreTime, ADR 67). So a PR scored before the freeze still counts when it is read during it, and one scored during it does not count after it lifts. A payload with no at (an older sync) keeps the arrival-time check: 503 while effectively paused, retried. Both reads fail open on a Redis error — a Redis blip must never drop a live score by accident (sync/src/redis.js’s isPaused() catches and returns false; scorer/src/store.js’s isPaused() and readScoringPolicy() do the same, the latter returning “no policy”, so the score is recorded).

The poller’s state lives in Redis, next to the scores. The per-repo since/ETag cursors, the seen cache, ingested/dropped/lastDrop and the last reset epoch applied are one JSON string at ctf:sync:state, read once at startup and written after every tick (sync/src/index.js, through sync/src/redis.js’s readPollState/writePollState). It used to be a file on the sync-state volume, and a file does not survive a container that restarts without its volume — every Fargate task restart (ADR 64). A poller that loses its cursor re-reads every score comment: nothing is double-counted (the scorer’s HSETNX), but the Secure Development solves an organizer removed with a per-contestant reset come back, and /admin’s counters restart at 0. Keeping the cursor with the scores also makes the two fail together: if Redis is wiped, the key is gone with the leaderboard and the poller re-reads everything, which is what rebuilds the board.

The fail directions follow from that:

The master reset deliberately does not delete ctf:sync:state; the resetAt epoch below is how it clears the cursor. A poller with no Redis client (a dev run without UPSTASH_REDIS_REST_URL) still keeps its state in the STATE_PATH file.

Stored state is repaired, never trusted. It is JSON this service wrote (sync/src/state.js’s parseState), which makes its shape tempting to assume once it parses — and that was a real outage: a bare {} is valid JSON, so a partial write or a hand edit during a reset produced a file that loaded fine and then threw on state.repos[repo] for every repo, on every tick. Nothing contains that throw — tick()’s per-repo try wraps only the fetch — so it reached the fatal handler, exited 1, and compose restarted straight back into the same file. Ingestion stayed down for the whole event.

parseState (used for both the Redis value and the file) now validates the shape it parsed and repairs what is unusable, field by field rather than all-or-nothing: a damaged repos is reset while ingested and resetAt survive, because re-zeroing them would misreport the event’s totals and re-apply a master reset already performed. repoState does the same one level down, since a per-repo entry can be damaged on its own and markSeen dereferences seen immediately. Every repair is logged; a missing file or key is not, because that is every event’s first boot.

Master reset + the reset epoch. resetEvent() (admin-store.ts, behind POST /api/admin/reset, requireAdmin + server-side type-to-confirm) wipes all event data — SCAN+DEL of ctf:solves:*, ctf:team:*, ctf:user:*, ctf:joincode:*, ctf:hints:*, ctf:quiz:answers:*/ctf:quiz:attempts:*/ctf:quiz:points/ ctf:quiz:answered/ctf:quiz:lastAt, and ctf:classic:solves:*/ctf:classic:attempts:*/ctf:classic:points/ ctf:classic:solved/ctf:classic:solvecount/ctf:classic:lastAt/ ctf:classic:opened:*, ctf:ai:solves:*/ctf:ai:attempts:*/ctf:ai:points/ctf:ai:solved/ ctf:ai:solvecount/ctf:ai:lastAt/ctf:ai:opened:* (the first-open times Insights measures from, #616), the spent replay nonces ctf:ai:nonce:*, the module-wide ctf:ai:launchkey (so no launch token issued before the reset survives it — see “AI data flow” above), the activity log (ctf:activity:log), both sponsor hashes (ctf:sponsors and ctf:sponsors:logo — a sponsor list belongs to one event run, and deleting it is the only off switch) and the organizer’s score adjustments (ctf:adjustments) — keeps ctf:admin:settings and (deliberately) the organizer’s authored content, ctf:quiz:questions/ctf:quiz:key, ctf:classic:challenges/ ctf:classic:flag/ctf:classic:flagnorm/ctf:classic:categories and ctf:ai:challenges/ctf:ai:flag/ctf:ai:flagnorm/ctf:ai:hints/ ctf:ai:signkey/ctf:ai:categories, and appends a reset audit line. The prefix list is walked unconditionally: the reset does not check which modules are enabled, so keys a since-disabled module left behind are cleared too. On its own that isn’t enough in poll mode: sync would re-ingest the same PR comments within a cycle and undo the wipe. So the reset also freezes scoring and bumps a resetAt epoch field in the settings hash. sync/src/index.js’s tick() reads it (redis.getResetAt()) every tick and, when it advances, drops its per-repo cursor/seen state, so the next poll re-reads from scratch. On its own that re-poll would re-bank every pre-reset score (the seen-set that would have deduped them was just cleared), so the epoch is also the ingestion watermark (#551): a bot comment whose updated_at predates resetAt is marked seen and skipped (tallied once as preReset in the poll summary), while a re-run that edits the comment past the epoch is a new revision and lands as normal. The scorer applies the same watermark to each score’s at, and the reset clears the freeze log and opens its own freeze, so a PR scored between the reset and the un-freeze is refused too. The epoch is epoch-milliseconds from resetEvent (an ISO string from an older state file is read too); an unparseable value is no watermark — skipping on junk would silently lose every score, the same fail-open direction getResetAt takes on a read error. This resetAt signal is the app→sync coordination that lets a wipe cross the container boundary without the app writing sync’s own poll state (ctf:sync:state). Every disruptive control prompts for confirmation (type-to-confirm for the reset; one-click for the freeze/registration toggles).

Demo seed. seedDemoData() + POST /api/admin/seed populate a demo leaderboard (teams, demo sponsors, and timestamps spread for a rising graph) with data for every module the event has enabled, read from the live settings: Secure Development solves on a bundled fixture of real challenge-ids so the scorer scores them, a small quiz question bank (DEMO_QUESTIONS) with some already answered, and demo Jeopardy and AI challenges with solves and unsolved attempts. The board then shows a genuinely combined score across modules. A disabled module gets no demo rows at all, and a settings read that fails aborts the seed rather than seeding every module. Its inverse, clearDemoData() + DELETE /api/admin/seed, removes exactly the run-state rows seeding added (fake contestants/teams/solves, sponsors) but leaves the demo questions/challenges/flags/categories as authored content, same as a master reset already treats real ones. Neither route sits behind a DEMO_MODE env var (issue #419 removed that gate everywhere): admin auth plus a type-to-confirm body is the whole safety net, the same pattern /api/admin/reset already used.

Known limitation: the hint toggle is only live at the reveal boundary. resolveHintConfig() is the single answer to “are hints on right now”, and every hint read path goes through it: revealHint/hintGate (the purchase boundary), getHintNotice and getClassicHintIds/getAiHintIds (the boards’ paid-hint notice and 💡 purchase affordance), getViewerHints (the profile tile and /api/hints), and getHintPenalties (the read-time leaderboard penalty). Flipping hintsEnabled in /admin therefore changes all of them on the next request, with no rebuild and no restart.

The affordability gate reads the folded leaderboard. hintGate refuses a priced hint when the contestant’s net score does not cover it (#553): hint-balance.ts takes gross from the contestant’s row on the folded board (points + hintPenalty — the one place every module’s points are already summed per login) and the spend fresh from ctf:hints:spent, as the case-folded sum of the hash (a case-only login rename splits one person’s spend across two fields), because the fold is memoized for ~10 s and a second purchase must not pass on the row’s stale penalty. The gross side is folded fresh for this read (fresh: true), never from the ~10 s memo: the memo is process-local and the AWS module runs two app tasks, so a score-lowering write on one task could leave the other serving a pre-write gross for a TTL. The read is rare — it sits behind the module, enabled, time and progress gates, and the route is rate-limited per login — so it pays for its own fold. Even a fresh fold can finish after a write on the other task lowered the score, so the gross travels with a score revision (ctf:admin:score-rev, read before the fold): the reveal script compares it to the current one before reading the spend or charging, answers stale when it moved, and the store re-reads and retries once. The revision alone cannot cover the middle of a multi-step wipe, so every admin operation that lowers a score (the master and per-player resets and the delete, the demo clear, any settings write — a module switched off takes its points out of the fold) is bracketed through leaderboard/fold-cache.ts (a leaf, since admin-store sits upstream of the fold): beginScoreLowering() before its first write raises a shared in-progress counter (ctf:admin:score-lowering, with a TTL stuck-guard) and bumps the revision — and throws if it cannot, in which case nothing is written — endScoreLowering() in a finally after its last write lowers the counter and bumps again (a failure midway leaves the earlier deletes standing). While the counter is up, the balance read refuses without folding (the gate answers busy: “Scores are being updated. Try again in a moment”) and the script refuses stale. Both calls also drop the process-local memo, so the board on the writing instance does not show wiped scores for a TTL (a fold already running when the invalidation came is discarded, not memoized). It fails closed like the progress gate, and exempts an already-owned hint (a re-view charges nothing). The gate’s read and the charge are still two round-trips, so the reveal script makes the limit atomic: it takes the gate’s gross as an argument, re-reads the spend inside the script, and refuses before its SADD when gross − spend < cost — two parallel reveals against a balance that covers one land exactly one. Every verdict returns the post-call, case-folded spend total, and the reveal reports balance = gross − that total — not the gate’s net − cost, which a reveal landing in between would have outdated — and the challenge page shows it beside the deduction as “N pts left”. It is a lower bound: gross is the fold’s figure, and a solve landing between the fold and the charge is not in it (score awards do not bump the revision — every solve would otherwise force concurrent buyers to retry); awards only add, so it is never overstated, and the page refresh after the reveal shows the live score. That read is why the policy helpers the fold needs — HINTS_AVAILABLE, resolveHintConfig, getHintPenalties — live in hint-config.ts: the fold’s hint stage imports them, the store imports the fold, and one module cannot sit on both sides of that without a cycle. hint-store.ts re-exports them.

Two things stay separate from that override on purpose. HINTS_AVAILABLE is a capability check — Upstash credentials present — since hint text lives only there and no organizer setting can conjure it; the read paths test it first because a credential-less deployment need not read settings to learn hints are off. And turning hints off does not rewrite history: ctf:hints:spent keeps its rows, so the penalties return intact when hints come back on. See docs/decisions.md #31, which supersedes the v1 limitation recorded in #19.

Runtime config flow

Animated diagram of the two configuration planes an event is built from, with no config file between them. Bootstrap plane: the wizard or the organizer writes GITHUB_ORG, ADMIN_LOGINS, SCORE_IMAGE and EVENT_URL into .env; docker compose up reads them, adding the secdev profile only when SCORE_IMAGE is non-empty; the app, sync and scorer containers read those keys from their process environment once, at start, so changing one is a restart and never a rebuild. Runtime plane: the organizer changes everything else live in /admin — which modules run, which Secure Development targets run, the event's identity, the scoring schedule, hints and team caps — and each change is written to the single ctf:admin:settings hash in Redis, which the app re-reads on every request and the sync poller re-reads on every tick. The app image itself takes no configuration build-arg at all, so the same image runs every event and a build can no longer ship an empty admins list.

No image in this kit takes a configuration build-arg (config v2, #386). Every fact about an event reaches the running system one of exactly two ways.

Bootstrap — .env, read once at container start. Four keys, plus the secrets: ADMIN_LOGINS (who may reach /admin), GITHUB_ORG (the org whose forks are linked and polled), SCORE_IMAGE (which scorer image to run, and by its non-emptiness whether this deployment can provide Secure Development at all — it adds the compose profile that gives the module its containers and seeds the first-boot default module set; enabledModules in ctf:admin:settings stays the live /admin selector), and EVENT_URL (ADR 43). The app reads the first two through src/lib/bootstrap-env.ts, a server-only module — no process.env read ever reaches a client bundle, which is what keeps the admin list off the wire. Changing any of them is an .env edit and a container recreate; there is no rebuild, and nothing is compiled in.

Runtime — the ctf:admin:settings hash, re-read per request. Everything an organizer changes during an event: the live module set (lib/enabled-modules.ts → lib/resolved-modules.ts over the static modules.ts registry), the live Secure Development target list (lib/enabled-apps.ts, a per-request filter over apps.ts’s static catalogue of all six), the event’s identity — name, location, time zone, contact e-mail, Discord invite, logo link — through lib/site.ts’s request-cached getSite(), the scoring schedule and the freeze, the hint policy, and the team caps. The dates line and the countdown are derived from the scoring window rather than stored separately (lib/event-dates.ts). sync re-reads the target list on every tick through its own Redis client, and scorer reads the same pause and window fields — the three-reader lockstep the rest of this document describes.

Two fail directions, deliberately opposite and worth knowing apart:

GITHUB_ORG disagrees between two readers on purpose: the app degrades gracefully (a bare repo name where a fork link would go), while sync’s loadConfig throws at startup with the key named, because for the poller a missing org is a genuine misconfiguration, not “nothing to poll”.

Which modules and targets exist at all is still static code — the ModuleId union and registry in apps/web/src/lib/modules.ts, the AppId union in apps.ts, sync/src/config.js’s TARGETS and scorer/src/targets.js, with scripts/check-module-registries.mjs failing if the three target lists disagree. Registration is deliberate and duplicated (ADR 10, ADR 13); only selection moved to runtime (ADR 55).

Pages are dynamic (ƒ in the build output, not ○) precisely because of this: / resolves the module nav through a build-time-unreachable Redis read, so it must never be statically prerendered — CI asserts that apps/web/.next/server/app/index.html does not exist after a production build.

Security model

Testing strategy

Layer Where What it proves
Unit (sync) sync/test/*.test.js, run via npm test (Node’s built-in test runner) Config loading/validation, comment parsing and the author grammar, cursor/ETag handling, submit retry semantics, state persistence — in isolation, no network or Docker.
Unit (scorer) scorer/test/*.test.js, run via npm test (Node’s built-in test runner) Rubric loading/validation, probe grammar + evaluation, the judge’s report format (the score-action regexes and the sync marker, pinned verbatim), serve auth/validation/monotonic-replay semantics, leaderboard aggregation, and both solve stores (memory, and Redis-via-SRH against a mocked endpoint) — in isolation, no network or Docker.
Unit (app) apps/web/src/lib/__tests__/*, run via vitest run Bootstrap-env parsing (ADMIN_LOGINS case/whitespace/empties, the SCORE_IMAGE-derived default module set), timezone-independent date formatting from the scoring window, module/app enablement filtering (enabled-apps.ts’s live, per-request target filter, defaulting to all six), site config derivation, and — apps/web/src/lib/leaderboard/__tests__/{module-contributions,rank,pipeline}.test.ts — the module-contribution overlay’s attribution (secure-development attributed not added, no double counting; pipeline.test.ts shows a penalised row’s module blocks gross and nets the hint penalty once, on the row total; with the quiz module disabled a source’s teams pass through untouched and no quiz block is read at all; with it enabled, quiz points are added to an entry’s and a deduped team’s totals, a quiz-less entry gets a zero quiz block at 0 / N that moves no total, rank or order (#585), and a quiz-active row on an upstash-shaped board keeps its patches in the items tiebreak — added quiz points can move a row either way in the points order, whatever its patch count) and the points/cross-module-completion/earliest-activity ranking — including the regression that ordering is already correct with hints disabled, since withHintPenalties no-ops in that case and must not be the thing doing the re-rank, and the pinned points order of an Upstash-shaped board with its ties broken on patched. The quiz store itself (src/lib/__tests__/quiz-store*.test.ts) covers all-or-nothing set comparison, the attempt cap and cooldown (including the atomic grading script’s authority over the JS-side pre-check, and its fail-closed behavior on a lookup error), and question authoring validation; components/__tests__/{admin-quiz-controls,quiz-board}.test.tsx cover the authoring form and the contestant answer UI. The derived-plumbing rules get their own direct coverage, since neither is observable in a static render: src/lib/__tests__/quiz-id.test.ts pins that generateQuestionId always emits an id QUIZ_ID_RE accepts (across a corpus of punctuation-only, non-Latin, emoji and over-long prompts) and that two identical prompts never collide, and admin-quiz-controls.test.tsx pins that payloadFromEditor submits an existing question’s stored id no matter how the draft was rewritten, plus reorderQuestions’s recomputed order values. The drag handlers themselves are deliberately NOT unit-tested — this repo has no testing-library and does not want one — which is why every decision they make lives in those two pure functions instead. The answer-key boundary is pinned from both sides: listQuestions never issues a command against ctf:quiz:key while listQuestionsForAdmin returns the set paired by question id (quiz-store.test.ts), GET /api/admin/quiz returns it for an admin and returns a body with no answer data at all for a 401/403 (app/api/quiz/__tests__/routes.test.ts), the admin edit draft prefills it (admin-quiz-controls.test.tsx) while the collapsed question list doesn’t paint it, and /quiz’s page-level view model strips it even when the store hands one over (app/(site)/quiz/__tests__/page-view-model.test.tsx, with quiz-board.test.tsx’s markup check as the independent second guard).
Live Lua (app) Every apps/web/src/lib/__tests__/*.upstash.test.ts suite — the grading scripts’ {classic-store,quiz-store,ai-store}.lua.upstash.test.ts and the settings, hint, team and other store suites beside them — all describe.skipIf-gated through live-redis.ts’s liveConfigured on UPSTASH_REDIS_REST_URL/_TOKEN; the app CI job brings up redis + srh (digest-pinned like docker-compose.yml), sets CTF_LUA_SUITES_REQUIRED=1 so a skip fails the job, and runs every *.upstash.test.ts file serially (vitest run upstash --no-file-parallelism — admin-store and hint-store share the fixed ctf:admin:settings hash) The three older suites, one store each: admin-store pins the settings write + audit append as one atomic step and the audit cap; hint-store the hint reveal under a policy the suite seeds itself (a player with no solves on the target is refused with forbidden and charged nothing, the first reveal charges the organizer’s configured price — not the baked default — once a solve is seeded, the second is free, and a missing hint charges nothing); team-store the team scripts (the four-player cap, one team per player, a populated team’s captain refused with Transfer or disband before leaving as a no-op, then transfer → leave → the last member’s leave deleting the team hash, member set and join-code index while every user hash survives). Each was checked against a one-line store mutation (the gate condition flipped; the key deletion dropped; the captain rule weakened) and went red (#235). The newer suites close the rest of that gap (#497): team-captain runs the captain-only scripts (remove, rename, transfer, disband, regenerate) through the store, refusals asserted as no-ops, plus concurrent regenerations and a disband racing one (no join code may survive resolving to the team or to a deleted one); admin-scripts runs the organizer overrides FORCE_REMOVE/FORCE_TRANSFER and the runtime-admin ADMINS grant; rate-limit-store runs CONSUME on a two-second window, observing the expiry for real. Each was checked against a one-line mutant (window extended on every call, captain guard dropped, transfer to a non-member, another team’s member record cleared, audit cap dropped) and went red. The three grading scripts — the scoring authority — EXECUTED against a real Redis, on run-unique keys: missing/already/incorrect/correct/cooldown/exhausted/mode verdicts, the exact attempts and solve rows written, that a refused submission writes nothing, the cooldown boundary (refused at now < lastAtMs + cooldownMs, graded at equality), the >= cap, maxAttempts = 0 as uncapped, a first-ever submission with a cooldown set (no attempts row, so lastAtMs is nil), solvecount keyed by the challenge and the two totals by the login, the case-sensitive form chosen only for a caseSensitive record, and ai’s event path (no flag compare, no attempts row, source recorded, a mode: "flag" challenge refused). Each of the six single-line Lua mutations the 2026-08-25 review found survivable (already-solved polarity, and lastAtMs, login-keyed solvecount, > cap, mode refusal, case-sensitive branch) fails at least one of these. The mocked *.grade.test.ts suites still pin what the stores hand the scripts (key and argument order); together the two layers cover the chain.
Shell (bats) setup/test/, scripts/test/, deploy/fly/test/ and deploy/aws-terraform/test/, each run by the shell job in ci.yml ctf-setup.sh’s subcommands against fixture .env files: dry-run fork/workflow/mirror/teardown plans, secrets generation, the wizard’s answers, and the .env reader’s edge cases (trailing comments, blank values, a missing file refused rather than read as “no Secure Development”) — no real gh/docker calls needed. The other directories cover the repo’s scripts (the shared .env reader against docker compose config, the CI path filter, the changelog and comment-policy checks, image pins) the Fly module’s deploy.sh and render-compose.sh, and the AWS module’s deploy.sh with stubbed docker/aws/terraform.
Offline smoke scripts/smoke.sh The full poll pipeline against fixture services (test/fixtures/mock-github.mjs, test/fixtures/mock-scorer.mjs, docker-compose.smoke.yml): Redis and the srh REST proxy work, sync ingests fixture score comments, scores match the fixtures, a forged comment is dropped by the trust filter, an unauthenticated POST /score is rejected, each score carries its own comment’s updated_at as at, and — the poller’s half of the freeze — setting ctf:admin:settings paused directly on Redis (the same key the app’s settings route writes) shows in ctf:sync:status while the poller keeps forwarding a queued fixture score, with its comment time, for the scorer to judge. The scorer’s half (refusing a score posted before the launch, during a freeze or before a reset, against a real Redis) is scripts/acceptance-scorer.sh’s freeze stage. This is what CI’s smoke job runs, and needs no live GitHub org, Action runs, or scorer image access.
Docker acceptance scripts/acceptance-app.sh Builds the real apps/web/Dockerfile once, with no config build-arg, then runs that same image as four containers with different runtime environments and asserts what each renders. web-acceptance has GITHUB_ORG and SCORE_IMAGE set but no Redis at all: /challenges redirects to the landing page (the launch lock fails closed on an unreadable settings read), the page <title> is the identity default “OWASP CTF in a Box” (identity fails open to its default rather than being read from anywhere baked), Secure Development is the only default board, /admin renders its shell, /health carries the build stamp the image was built with, and /health/deep answers 503 degraded with redis down and launched: null, leaking no host or error detail. web-launched is its twin on a real Redis behind srh with scoringStartsAt written, so /challenges renders all six targets with every fork link following GITHUB_ORG. web-noscorer has SCORE_IMAGE empty: nothing is enabled, the landing page renders its no-boards copy pointing at /admin, and /challenges 404s. web-default has no GITHUB_ORG: /challenges renders bare repo names and no fork link. This is the layer that proves the runtime config flow actually reaches rendered HTML.
Docker acceptance (scorer) scripts/acceptance-scorer.sh Builds the scorer image from scorer/ with the example rubric and closes the scoring loop offline: judge runs against a fake target that passes some probes and fails others, and the script asserts the report’s score-action regexes, that no probe internals leak into the comment, that the sync marker parses via the real sync/src/parse.js, and that the marker POSTed the way sync does lands on GET /leaderboard with rubric-derived points/totals. A second, stock fake app fails every probe and must score 0 and gain no leaderboard entry. A second serve instance backed by real Redis (via srh) then covers the freeze and comment-time rules: an unlaunched event answers 503, a hand-set paused answers 503 until cleared, and a score sent with its comment’s at is recorded at that time and judged by it — refused for good (422) when posted before the launch, inside a recorded freeze (still refused after the freeze lifts) or before the master-reset epoch, scored when posted before the freeze, and held with a 503 under a freeze with no recorded start. That SCORE_API/SCORE_TOKEN are inert in the judge is pinned in scorer/test/judge.test.js, not here.
Docker acceptance (quiz-only) scripts/acceptance-quiz-only.sh Boots the real app image with SCORE_IMAGE empty and quiz enabled through the admin settings route (no config file exists to bind it to), seeds one question and one contestant’s answer straight into Redis (no OAuth app in CI to drive real authoring/answering), and asserts against the running app: /quiz shows the seeded question by name, /challenges 404s, and /leaderboard shows the contestant by login with their quiz points — the one assertion a vacuously-up-but-broken app can’t fake, since a quiz-only event’s leaderboard source is emptySource and carries no rows of its own. It also asserts the DOCUMENTED bring-up structurally: --profile app must resolve to a line-up with no scorer and no sync (a quiz-only organizer cannot pull the private scorer image), while --profile secdev --profile app — what a non-empty SCORE_IMAGE derives — must still contain both. Separately brings sync up under secdev with no GITHUB_ORG and asserts it refuses at start-up, naming the key, with a non-zero exit.
Docker acceptance (classic-only) scripts/acceptance-classic-only.sh The classic module’s sibling of the quiz-only script, following every one of its design decisions: boots the real app image with SCORE_IMAGE empty and classic enabled at runtime, seeds a challenge and a solve straight into Redis, and asserts /flags shows the challenge by title, /challenges 404s, /leaderboard shows the contestant’s classic points by login, and the --profile app line-up contains no secure-development service. Like the quiz-only script it then brings sync up under secdev with no GITHUB_ORG and asserts it refuses at start-up, naming the key, with a non-zero exit.
Docker acceptance (ai-only) scripts/acceptance-ai-only.sh The ai module’s sibling of the quiz-only/classic-only scripts, following the same design decisions: boots the real app image with SCORE_IMAGE empty and ai enabled at runtime, seeds a challenge and a contestant’s solve straight into Redis, and asserts /ai shows the challenge by title without leaking its flag, /ai/<id> 200s while /ai/<bad-id> 404s, /challenges, /flags and /quiz all 404, /leaderboard shows the contestant’s ai points by login, GET /api/ai/launch-key mints the keypair internally and serves its public key with no OAuth/cookie/session available, and the --profile app line-up contains no secure-development service. It too brings sync up under secdev with no GITHUB_ORG and asserts the same start-up refusal.
Vacuous-pass sweep scorer/tools/vacuous-sweep.mjs Points every target’s rubric at an in-process HTTP stub that is UP but USELESS (three personalities: empty-200, not-found, server-error) and fails if any challenge passes — a challenge that “blocks the exploit” against a stub proves nothing. Must report 0; wired into CI only once the count reached 0/321.

CI (.github/workflows/ci.yml) carries a changes gate (native git diff, no third-party action) plus eleven gated jobs — sync-tests, scorer (node --test + acceptance-scorer.sh), vacuous (the sweep above), shell (shellcheck + bats, including deploy/fly/’s scripts and bats suite), smoke, app (vitest + next build + the / never-prerendered assertion + acceptance-app.sh), quiz-only, classic-only, ai-only, registries (the three duplicated target lists still agree), and docs (Jekyll build + link/meta checks). The gate runs only the jobs whose area a PR touches; a push to main runs all eleven. The heavier stock-scores-zero / patched-scores-right workflows are scoped to judge-relevant scorer inputs, so a leaderboard-only change doesn’t spin up the per-target Maven/gradle builds.

Names

Several names orbit “the project” and they are not interchangeable — the table moved to the glossary, which also defines the terms (target, module, rubric, probe, marker, …) the rest of this doc uses.