Build a CI platform: a developer pushes code, and within seconds a pipeline of builds, tests, and checks runs across a fleet of isolated runners - in parallel where the pipeline allows, skipped where caching proves nothing changed - and reports pass or fail back to the pull request while the developer waits.
A pipeline is the recipe a repo declares for validating a change: a set of jobs (build the code, run unit tests, run the linter, run integration tests, package the artifact), each running in its own container, with dependencies between them - integration tests need the build's output; the linter needs nothing but the source. Those dependencies make the pipeline a DAG - a directed acyclic graph. Directed: arrows point from a job to the jobs that need it. Acyclic: no loops, because "A needs B needs A" can never start. The shape is the whole point: everything not connected by an arrow can run at the same time. A pipeline of build → [unit tests, lint in parallel] → integration tests finishes in the time of its longest path (the critical path), not the sum of all jobs - a 25-minute pile of jobs can finish in 12. The two levers this design obsesses over both come from that observation: parallelism shrinks the wall-clock toward the critical path, and caching removes jobs from the path entirely when their inputs did not change. Everything else - queues, runners, schedulers - is machinery in service of those two levers.
Most systems in this practice set serve users who glance and move on. CI serves a developer who is synchronously waiting - they pushed, and until the checks go green they cannot merge, and often cannot mentally move on. Do the tax math: a developer triggers ~20 builds a day. If the platform wastes one minute per build - slow queue, cold runner, un-cached install - that is 20 minutes per developer per day, times every developer in every customer org. A CI platform's latency is multiplied by everyone's merge frequency, which is why "p95 < 30s to first job" is a hard budget and not a nice-to-have, and why the two big levers (parallelism and caching) are worth real engineering: they buy back human minutes at organizational scale. The budget also decomposes usefully, and each term gets its own fix: webhook processing (milliseconds), fetching config at the commit (a second, cached), compiling the DAG (milliseconds), waiting for a runner (THE variable term - bounded by pre-warmed pools and autoscaling), container start (bounded by pre-pulled images). When an interviewer asks where the 30 seconds goes, walking that decomposition is the answer.
The numbers put this system in the same family as Design CodeSignal - a compute platform, not a data platform. The database tier is deliberately boring (PostgreSQL holding build/job state at trivial rates); the engineering lives in the pipeline between "push" and "green check." Two derived numbers shape everything. First, Little's Law sizes the fleet: concurrency equals arrival rate times duration, so ~60 builds/sec peaking at ~15 minutes each means ~50K concurrent builds and ~70K concurrent jobs - which is why the runner fleet autoscales on queue depth and why the daily business-hours curve (known in advance, like CodeSignal's hiring seasons) justifies pre-provisioning. Second, the same law shows the leverage of the two big levers. Cut average build duration from 15 minutes to 8 - via DAG parallelism and cache hits - and concurrency (and therefore fleet cost) nearly halves AT THE SAME TIME as every developer waits half as long. Speed and cost point the same direction here, which is rare and worth saying in an interview: caching is not an optimization garnish, it is simultaneously the biggest cost lever and the biggest experience lever in the system. Storage splits by temperature like everything in this set: hot little state rows in PostgreSQL; a petabyte of content-addressed cache and half a petabyte of logs in object storage, both safely evictable/expirable because neither is the truth about anything - the repo is the truth, and every byte we hold can be regenerated from it.
TRIGGER
[Dev pushes] -> [GitHub / GitLab] --webhook--> [Webhook Service]
| verify HMAC, dedupe
v
[Pipeline Compiler]
fetch .ci.yml at that commit
validate -> compile to DAG
|
v
[Scheduler]
the DAG: [build] ---> [unit tests] ---+
| +--> [integration]
+------> [lint] ---------+
enqueue every job whose dependencies are all green
|
v
[Job Queues (per resource class,
per-org fairness)]
|
v
[Runner Fleet (autoscaling)]
per job: fresh container (pre-warmed)
restore cache by content key
run steps -> upload artifacts
| |
[Cache Store] [Artifact Store]
(content- (outputs flow to
addressed) dependent jobs)
|
logs stream --> [Log Pipeline] --> live tail + 90d retention
|
v
[Status Reporter] --> PR checks
(pass / fail, blocks merge)Strip the word "orchestration" down and the scheduler is a state machine over job rows plus one loop. Every job moves through: pending (dependencies not yet met) → ready (all dependencies succeeded) → queued → running → succeeded / failed / skipped. The loop is: whenever any job finishes, re-evaluate - which pending jobs now have every dependency green? Enqueue exactly those. That single rule produces all the behavior: fan-out (build succeeds → unit tests AND lint both become ready and run in parallel) and fan-in (integration stays pending until BOTH unit tests and lint are green) fall out for free from "all dependencies satisfied." Failure is the same rule inverted: a failed job marks everything downstream of it skipped (there is no point testing a build that did not compile), and the build's final status aggregates the leaves. The important operational choice: the job states live in PostgreSQL, and scheduler workers are stateless processors of completion events. Any scheduler instance can handle any event, so the "brain" of the system has no memory of its own to lose - restart it, scale it, kill one mid-event; the truth is in the database and the loop resumes. The same truth-versus-worker split as everywhere else in this set, applied to the control plane itself.
It would be easy to store each repo's pipeline configuration in our own database with a settings UI. Every serious CI system refuses, and the reasons compound. Versioned together: the pipeline is code that builds code, and they change together. A PR that adds a dependency AND updates the install step is one atomic change - reviewable in one diff, revertible in one revert. With config in our database, every such change becomes two changes in two systems that can disagree. History stays honest: rebuilding an old commit uses the config as it was AT that commit, so last month's release rebuilds the way it built last month. A database holds only "the current config," which silently rewrites history. Branches get experiments for free: a branch can overhaul its pipeline without touching anyone else's builds, because the config rides the branch. The division of truth this creates is clean: git owns WHAT TO DO (the recipe, versioned with the code), our database owns WHAT HAPPENED (builds and jobs as immutable facts). The compiler bridges them by fetching the config at the exact SHA being built. This is the same "pin the data to its schema version" discipline as Design SurveyMonkey's survey versioning - the recipe that produced a result must remain attached to the result.
The pipeline config (lives in the REPO, read at the commit being built)
# .ci.yml
jobs:
build:
image: node:20
steps: [npm ci, npm run build]
cache:
key: deps-{{ hash("package-lock.json") }} # content-addressed
path: node_modules
artifacts: [dist/]
lint:
image: node:20
steps: [npm run lint]
allowed_failure: true # red lint warns, does not block
unit_tests:
needs: [build] # DAG edges
image: node:20
steps: [npm test -- --reporter junit]
integration:
needs: [unit_tests, lint] # fan-in: waits for BOTH
image: node:20
services: [postgres:16]
steps: [npm run test:integration]
Database Schemas (PostgreSQL - the estimates say it is enough)
- builds
build_id UUID PRIMARY KEY,
repo_id UUID NOT NULL,
commit_sha CHAR(40) NOT NULL,
branch VARCHAR(255),
trigger trigger_enum, -- push | pull_request | manual
status build_status_enum, -- running | succeeded | failed | canceled
created_at TIMESTAMP DEFAULT NOW(),
started_at TIMESTAMP, finished_at TIMESTAMP,
INDEX (repo_id, created_at DESC)
- jobs (the scheduler's state machine IS this table)
job_id UUID PRIMARY KEY,
build_id UUID NOT NULL,
name VARCHAR(255) NOT NULL,
needs TEXT[], -- dependency names within the build
state job_state_enum, -- pending|ready|queued|running|
-- succeeded|failed|skipped
allowed_failure BOOLEAN DEFAULT FALSE,
runner_id VARCHAR(100), exit_code INT,
retry_count INT DEFAULT 0,
queued_at TIMESTAMP, started_at TIMESTAMP, finished_at TIMESTAMP,
logs_ref VARCHAR(512),
INDEX (build_id), INDEX (state, queued_at)
- test_results
job_id UUID, suite VARCHAR(255),
passed INT, failed INT, skipped INT,
failures JSONB -- [{test, message, trace_ref}]
- cache_index
cache_key VARCHAR(128) PRIMARY KEY, -- "deps-sha256:..." content hash
repo_id UUID NOT NULL, -- caches are namespaced per repo
storage_ref VARCHAR(512), size_bytes BIGINT,
last_hit_at TIMESTAMP, -- LRU eviction reads this
INDEX (repo_id), INDEX (last_hit_at)
Queues & storage
- Job queues: Redis Streams, one per resource class (small/large/GPU),
consumer groups + leases; delivery not truth - jobs still 'ready'
in PostgreSQL re-enqueue if a stream is lost (per Design CodeSignal)
- Object storage: /cache/{repo}/{key}, /artifacts/{build}/{job}/...,
/logs/{build}/{job}.log -- lifecycle rules expire logs at 90 days{
"delivery_id": "gh_72dfe1a0",
"event": "push",
"repository": "acme/backend-api",
"commit_sha": "a1b2c3d4e5f6...",
"branch": "feature/payment-retry",
"pusher": "dev@acme.com"
}{
"build_id": "b_8f3e2d1c",
"status": "created"
}
// The HMAC signature on the raw payload was verified BEFORE parsing -
// an unverified webhook endpoint would let anyone trigger builds of
// arbitrary commits. delivery_id dedupes provider redeliveries: same
// id, same build, no duplicate pipeline.{
"build_id": "b_8f3e2d1c",
"repo": "acme/backend-api",
"commit_sha": "a1b2c3d4",
"status": "running",
"jobs": [
{ "name": "build", "state": "succeeded", "duration_seconds": 141,
"cache": { "key": "deps-sha256:7d86...", "hit": true,
"restored_in_seconds": 18 } },
{ "name": "lint", "state": "succeeded", "allowed_failure": true,
"duration_seconds": 44 },
{ "name": "unit_tests", "state": "running", "started_seconds_ago": 95 },
{ "name": "integration", "state": "pending",
"waiting_on": ["unit_tests"] }
],
"critical_path": ["build", "unit_tests", "integration"]
}
// The cache hit tells the story: npm ci would have taken ~10 minutes;
// restoring node_modules by content key took 18 seconds. lint ran in
// parallel with unit_tests (fan-out); integration waits for both
// (fan-in) - and since lint is done, it waits only on unit_tests.{ "job": "unit_tests", "line": 812,
"ts": "2026-07-06T18:04:11.312Z",
"text": "PASS src/payments/retry.test.ts (4.2s)" }
{ "job": "unit_tests", "line": 813,
"ts": "2026-07-06T18:04:12.008Z",
"text": "Tests: 214 passed, 214 total" }
// Streamed live within 1-2s of being written; the same lines append to
// object storage for the 90-day record. Secret values are pattern-masked
// before the stream ever leaves the runner.{
"failed_only": true
}{
"build_id": "b_8f3e2d1c",
"retried_jobs": ["integration"],
"reused_jobs": ["build", "lint", "unit_tests"],
"note": "Successful jobs and their artifacts are reused; only failed jobs re-run."
}
// Infrastructure failures never need this - they auto-retried. This
// button exists for code-shaped failures, and a pass-on-retry at the
// same commit gets flagged as a flake candidate rather than hidden.POST /repos/acme/backend-api/statuses/a1b2c3d4
{
"context": "ci/integration",
"state": "success",
"description": "214 tests passed in 6m 12s",
"target_url": "https://ci.example.com/builds/b_8f3e2d1c"
}
// One status per check, per commit - this is the green tick, and with
// branch protection enabled it is also the merge gate. The whole
// platform's output, reduced to its essence: a boolean with a URL.Critical path: the longest dependency chain through the DAG, and therefore the minimum possible wall-clock time for the build no matter how many runners you own. Adding parallelism off the critical path is free speed; shortening the path itself (caching a step on it, splitting a long job) is the only way to beat it. Pipeline optimization is graph optimization, and the platform should show authors their path. Workspace passing: how outputs travel along DAG edges. A job declares artifacts (dist/, a compiled binary); the runner uploads them at job end, and each dependent job downloads them into its workspace before starting. This is what makes "test the thing build produced" literal - the integration job tests the exact bytes, not a re-build. Restore keys: the pragmatic fallback when an exact cache key misses. The lockfile changed, so "deps-{new hash}" has no entry - but the most recent entry under the "deps-" prefix is probably 95% right, so restore it and let the install step reconcile the difference incrementally. Exact hits skip work entirely; prefix hits shrink it. Both are safe for the same reason all of this is safe: the final install step is authoritative, and the cache only ever changes where work starts, never what is correct.