Design SurveyMonkey: System Design & Architecture

Build a survey platform: creators design forms and share a link, respondents answer by the millions, and results aggregate into a live dashboard. It is a system with two personalities - a write-optimized ingestion pipe and a read-optimized analytics engine - joined in the middle.

Scope

In Scope

  • Survey builder: question types (multiple choice, text, rating), required flags, validation rules, and basic show/hide logic (show question B if question A meets one condition)
  • Publishing and distribution: a shareable link and an embed snippet, with source tagging (email vs social vs embed)
  • Response collection at scale: validated, deduplicated, durable - including partial progress that survives a refresh
  • Live analytics: response counts, per-question distributions, and completion rates, updating within seconds
  • Exports: CSV/Excel of filtered responses, running as background jobs with progress and a download link
  • Editing a live survey safely: versioned definitions so existing responses are never corrupted

Out of Scope

  • Complex branching engines (multi-condition skip patterns, question piping): we draw the line at single-condition show/hide - past that, survey logic becomes a rules-engine design of its own
  • Sending the emails and reminders: distribution generates links and tags sources; the delivery machinery is a notification system design of its own
  • ML-powered insights, sentiment analysis, and trend prediction
  • A template marketplace and question-bank ecosystem
  • Billing, plan limits, and enterprise SSO/admin

💡 Why "collecting form answers" is harder than it sounds

A survey platform looks like a CRUD app until you notice it is two systems wearing one product. The first system is an ingestion pipe: accept millions of responses, some days a hundred million, with spikes when a survey goes viral - and never lose one. A lost response is invisible (the creator never knows it existed) and a duplicated one silently corrupts the results, so the write path needs durability and deduplication as first-class features, not afterthoughts. The second system is an analytics engine: a creator opens a dashboard on a survey with two million responses and expects per-question distributions in under two seconds. That is not "read a row" - it is "aggregate thirty million answers, grouped and filtered," a completely different shape of work. These two personalities want opposite storage designs - one wants fast, safe, row-at-a-time writes; the other wants massive columnar scans - and the heart of this design is refusing to make one store do both jobs. Everything else on this page hangs off that split.

Functional Requirements

  • A creator can build a survey from standard question types, with validation and basic show/hide logic
  • A creator can publish and get a shareable link and embed code; responses are tagged with their source
  • A respondent can answer on any device without an account; partial progress survives a refresh
  • Submitting is reliable: exactly one response is recorded even through retries, double-clicks, and flaky networks
  • A creator sees a live dashboard: total responses, completion rate, and per-question distributions, fresh within seconds
  • A creator can export filtered responses; large exports run in the background with progress and a signed download link
  • A creator can edit a live survey; existing responses keep their meaning (versioned definitions)

Non-functional Requirements

  • Submission: p95 < 500ms, and an accepted response is never lost - durability of responses IS the product
  • Form loads: < 800ms globally (static shell and definition served from CDN)
  • Dashboard: < 2s p95 even for surveys with millions of responses
  • Scale: ~10B responses/year; peak days reach ~100M; one viral survey alone may take thousands of responses/sec
  • Exports: 100K rows in seconds, 10M rows in minutes - always as background jobs with progress
  • Isolation and privacy: strict data separation between creators; respondent anonymity as a survey-level option

💡 What does "exactly one response" mean when respondents have no accounts?

The dedupe requirement has a twist the login-based designs in this set never face: respondents are anonymous. There is no user_id to hang uniqueness on - just a browser that loaded a form. So we mint one: on first load, the form gets a respondent token (stored in the browser), and the final submit carries an idempotency key derived from it. Retries, double-clicks, and back-button resubmits all carry the same key and collapse into one stored response - the accidental duplicates, which are the overwhelming majority, are dead. Be honest about the limit, because interviewers probe it: a token identifies a browser, not a human. Incognito mode, a second device, or clearing storage yields a fresh token and a second response. True one-response-per-person is impossible without authentication (or invasive fingerprinting, which is a privacy decision, not a technical one). What the product actually promises is: no duplicates from accidents and retries, plus optional friction (one-per-email, one-per-invite-link) for surveys that need more. Scoping the guarantee honestly - accidents prevented, determined adversaries not - is the senior answer here, and the same "what does the promise actually cover" discipline as the delivery guarantees in Design WhatsApp.

Back-of-the-Envelope Estimates

  • Assume ~50M surveys created/year (≈1.6/sec - creation is negligible load) and ~10B responses/year
  • A write is a response submission or partial save; a read is a form load, a dashboard view, or an export
  • Responses: 10B/year ≈ 27M/day average ≈ 320/sec. Peak days (elections, big campaign sends) hit ~100M/day - about 4x average - and within-day spikes push bursts to ~10K/sec. A single viral survey can take thousands/sec on its own
  • Response size: ~2KB (answers JSON + metadata) → 27M/day x 2KB ≈ ~55GB/day, ~20TB/year - storage is modest; the challenge is spike shape and durability, not volume
  • Dashboards: ~50M views/day ≈ 600/sec - but the cost is not the request rate, it is the query: one dashboard on a 2M-response survey aggregates ~30M individual answers. The read problem is measured in rows scanned per query, not queries per second
  • Exports: ~1M/day, ranging from a thousand rows to ten million - minutes of work at the top end, which is why they are jobs, not requests
  • Form loads: ~50M/day ≈ 600/sec average, static shell + cached definition from CDN - the cheapest traffic in the system

💡 What the estimates mean: one dataset, two query shapes, two stores

Look at what the two halves of the system ask of the same responses table. Ingestion asks row-shaped questions: write THIS response, check THIS idempotency key, fetch THIS response. Row-oriented transactional databases (PostgreSQL) are built for exactly this - constraints for dedupe, transactions for durability, and 320/sec average (10K/sec bursts) of 2KB writes is comfortable territory. Analytics asks column-shaped questions: the average of question 5 across two million responses; the distribution of question 2 filtered by date. A row store answers that by reading every full row and throwing away 95% of each - millions of rows, all columns, to aggregate one field. A columnar store (ClickHouse) lays data out the other way: all values of one column stored together, so the same query reads only the needed columns, and survey data compresses brutally well besides (a column holding "Yes/No/Maybe" two million times shrinks 10-100x). That is where the dashboard's 100x speedup comes from - not magic, just layout matched to the question. So responses live twice, on purpose: PostgreSQL as the source of truth (row-shaped, constrained, durable) and ClickHouse as a derived, rebuildable copy (column-shaped, aggregation-fast), connected by an event stream that lags by seconds. In front of both, Redis holds pre-aggregated live counters for the dashboard tiles everyone actually stares at, updated as responses flow - the Design a Newsfeed buffered-counter pattern working a different job. And the judgment flip, as always: at small scale - thousands of responses per survey - PostgreSQL with a few indexes does both jobs fine, and the second store plus its sync pipeline would be pure operational cost. The split earns its keep only when dashboards start scanning millions of rows. Know which side of that line the numbers put you on.

High-level Architecture

  • Survey Builder Service (PostgreSQL): survey definitions as versioned, immutable-once-published documents
  • CDN: the static form shell plus each survey's published definition JSON, cached by version - respondent traffic mostly never touches our servers
  • Response Ingestion Service (stateless, scale-out): validates answers against the definition version, enforces idempotency, writes the truth row, and emits an event - the one path that must never fail
  • Event Stream (Kafka): the bridge from truth to everything derived, fed via a transactional outbox so a response and its event can never disagree
  • Analytics Loader → ClickHouse: consumes events, flattens responses to one row per answer, loads the columnar store that powers filtered and historical queries
  • Live-Stats Aggregator → Redis: consumes the same events into per-question counters and completion funnels - the always-hot numbers dashboard tiles read
  • Dashboard Service: tiles from Redis (instant), deep filtered queries from ClickHouse (seconds)
  • Export Service: a job queue and workers that stream query results from ClickHouse into files in object storage, with progress tracking and signed download links

Data/User Flow Diagram

BUILD + PUBLISH
[Creator] --> [Builder Service] --> [PostgreSQL: survey definitions,
                     |               versioned, immutable once published]
                     +--publish--> definition JSON cached at [CDN] (by version)

COLLECT - the write path (protect this at all costs)
[Respondent] --form shell + definition--> [CDN]
      | partial saves (respondent token)        | final submit
      v                                         v   (+ idempotency key)
[Progress buffer (Redis, TTL)]        [Ingestion Service]
                                            | validate against definition
                                            | dedupe (unique constraint)
                                            v
                              [PostgreSQL: responses = TRUTH]
                                            |  (same transaction: outbox row)
                                            v
                                   [Event Stream (Kafka)]
                                      |                  |
                                      v                  v
                            [ClickHouse loader]   [Live-stats aggregator]
                                      |                  |
                                      v                  v
                          [ClickHouse (columnar,   [Redis: per-question
                           derived, rebuildable)]   counters, funnels]

ANALYZE - the read path
[Creator dashboard] --live tiles-----------> [Redis pre-aggregates]
                    --filtered deep queries-> [ClickHouse]

EXPORT
[Creator] --> [Export Service] --> [job queue] --> worker streams from
              ClickHouse --> file in [Object Storage] --> signed URL

💡 What is columnar storage, and where does the 100x actually come from?

A row store keeps each response together on disk: all forty answers of response #1, then all of response #2. Perfect for "fetch response #1" - one seek, everything there. Terrible for "average of question 5 across two million responses": the database must read all two million full rows and discard 39/40ths of every one. A columnar store flips the layout: all two million answers to question 5 sit together, followed by all answers to question 6. Now the same aggregation reads exactly one column and skips the rest of the table entirely - that alone is most of the speedup. Compression stacks on top. A column is millions of values of the same kind, often from a tiny set ("Yes", "No", "Maybe" over and over), which compresses 10-100x - so even the column we do read is a fraction of its raw bytes. Add partition pruning (a date filter skips whole months of data without touching them) and vectorized execution (aggregating values in CPU-friendly batches), and a 30-second row-store scan becomes a 300ms columnar query. The 100x is not one trick; it is four multiplying. The cost, and why the row store stays: columnar layouts are miserable at row-shaped work - writing one response means touching forty column files, and "give me response #1 in full" reassembles it from forty places. Neither layout is better; each is optimal for one query shape. That is the whole argument for running both.

Data Model & Storage

  • Surveys + Survey Versions (PostgreSQL): the survey row points at its current version; each version's definition (questions, validation, show/hide rules) is an immutable JSONB document once published. Edits create version N+1 - nothing published is ever mutated.
  • Responses (PostgreSQL - the truth store): one row per response with the answers as JSONB, the survey_version it was written against, the respondent token, source tag, and timing. A unique constraint on (survey_id, idempotency_key) is the dedupe referee. Partitioned by survey hash and month for manageable maintenance.
  • Outbox (PostgreSQL): the event row written in the same transaction as the response, relayed to Kafka - so a crash between "stored" and "announced" can neither lose nor invent an event.
  • Answers (ClickHouse - the derived copy): flattened to one row per (response, question) with typed answer columns, partitioned by month, ordered by (survey_id, question_id). Rebuildable from truth at any time; its loss is downtime for dashboards, never data loss.
  • Live stats (Redis): per-(survey, question, option) counters and completion funnels, updated by the stream consumer with buffered increments - the Design a Newsfeed counter pattern.
  • Partial progress (Redis, TTL 24h): respondent token → answers so far, so a refresh restores the form; promoted into a truth row only on final submit.
  • Export jobs (PostgreSQL) + files (Object Storage): job status and progress in the database, the generated CSV/Excel behind signed, expiring URLs.

💡 Truth versus derived: why it matters which copy is which

Responses exist in three places - PostgreSQL, ClickHouse, Redis - and the design only stays sane because exactly one of them is the truth. PostgreSQL is the truth: the copy with the constraints, the transactions, the backups, and the durability promise. Every engineering hour spent on "never lose data" is spent here and only here. ClickHouse and Redis are derived: computed FROM the truth via the event stream, and rebuildable from it. If the ClickHouse cluster burned down, dashboards would go dark for the hours a reload takes - painful, but zero responses lost. If Redis vanished, live tiles would rebuild from the stream. Because derived copies are disposable, they are allowed to be fast and loose: no backups to manage, seconds of lag tolerated, schema reshaped freely for query speed. This asymmetry is a load-bearing design tool, the same one as Design a Newsfeed's feed store (derived pointers, rebuildable) versus its Posts table (truth): decide which copy is the truth, put all the durability engineering there, and let every other copy be cheap. Systems that treat three copies as three truths end up reconciling forever. One honest wrinkle: deletion. When a creator (or GDPR) demands a response gone, the truth row deletes cleanly - but columnar stores hate point deletes, so ClickHouse takes a tombstone-and-rewrite path that runs batched and lagged. "Delete everywhere, eventually, provably" needs its own small pipeline, and pretending otherwise is how compliance incidents happen.

API Design

POST /v1/surveys
Create a survey (draft version 1).
PUT /v1/surveys/:id
Edit - creates a new draft version; published versions are immutable.
POST /v1/surveys/:id/publish
Freeze the current version, activate the link, push the definition to CDN.
GET /v1/forms/:slug
The respondent-facing definition (cached by version at the CDN).
PUT /v1/responses/progress
Autosave partial answers under the respondent token (TTL-buffered).
POST /v1/responses
Final submit with idempotency key - retries return the original response.
GET /v1/surveys/:id/analytics
Live tiles (Redis) and filtered aggregations (ClickHouse).
POST /v1/surveys/:id/exports
Start a background export; returns a job id.
GET /v1/exports/:id
Job progress and, when done, a signed download URL.

Low-level Design

Builder + publish - definitions as immutable versions (step by step):

  • 1) Editing always targets a draft; publishing freezes it as version N, forever.
  • 2) The published definition JSON is pushed to the CDN keyed by version - so when the creator later publishes N+1, the URL changes and caches roll over naturally, with no invalidation scramble.
  • 3) Validation rules ride inside the definition and run twice: client-side for instant feedback, server-side as the gate. The client copy is a courtesy; the server copy is the truth - the same trust-boundary discipline as hidden tests in Design CodeSignal.
  • 4) Show/hide logic (single-condition, per scope) is data in the definition, interpreted by the renderer - no code deploys per survey.

Submitting a response - exactly-once out of anonymous traffic (the genuinely tricky part):

  • 1) On first form load the respondent gets a token, kept in the browser.
  • 2) Partial answers autosave to Redis under that token (24h TTL) - a refresh or a dropped connection restores the form, and nothing is in the truth store yet.
  • 3) Final submit carries an idempotency key derived from (survey_id, respondent_token).
  • 4) Ingestion validates the answers against the exact definition version the form was rendered from, then INSERTs with the unique constraint on the key. A retry, double-click, or back-button resubmit hits the constraint and gets back the ORIGINAL response_id with a 200 - the caller cannot tell it was a duplicate, which is the point. Retry on silence, dedupe by key on arrival: the Design WhatsApp invariant, here guarding opinions instead of messages.
  • 5) In the same transaction, an outbox row is written; a relay ships it to Kafka afterwards. The same-transaction trick is Design Calendly's sync-jobs move: the response and its announcement commit or vanish together, so the derived stores can never learn of a response that does not exist, or miss one that does.

The live dashboard - two tiers of reads (step by step):

  • 1) The tiles everyone stares at - total responses, completion rate, each question's distribution - come from Redis counters that the stream consumer updates within seconds of each submission, using buffered increments (the Design a Newsfeed counter pattern; a viral survey is a hot key here too).
  • 2) Anything filtered or historical - "ratings by week, US respondents only, completed sessions" - is a live ClickHouse query: partition pruning plus columnar scan returns in seconds even over millions of responses.
  • 3) Why not ClickHouse for the tiles too: 600 dashboard views/sec repeating the same handful of aggregations would burn OLAP capacity recomputing answers that Redis simply keeps running totals of. Pre-aggregate what everyone asks; compute on demand what few ask.
  • 4) Tiles are labeled fresh-within-seconds, not real-time - honest lag beats fake precision.

The viral survey - protecting the write path (step by step):

  • A survey shared by a celebrity takes thousands of submissions/sec while its creator refreshes the dashboard obsessively.
  • 1) Form loads are already the CDN's problem, not ours.
  • 2) Ingestion is stateless and scales horizontally; the truth write is a cheap constrained INSERT - PostgreSQL absorbs a hot survey's appends comfortably because there is no read-modify-write anywhere on the path.
  • 3) The counters take the hot-key pressure, and buffered increments flatten it.
  • 4) If the stream or loaders fall behind, dashboards go stale by minutes - and that is the designed degradation order: derived data lags, truth never blocks. The creator sees slightly old numbers; not one response is dropped or delayed.
  • Say the priority out loud in the interview: when the two personalities compete under load, ingestion wins, always - analytics is recoverable, a lost response is not.

Exports - minutes of work behind an honest job (step by step):

  • 1) POST /exports writes a job row (queued) and returns immediately - a 10M-row export is minutes of streaming, and holding an HTTP request open for minutes is a failure designed in.
  • 2) A worker claims the job, streams the filtered result set from ClickHouse in chunks, and appends to a file in object storage - never materializing 10M rows in memory.
  • 3) Progress updates land on the job row; the UI polls and shows a moving bar (the honest-feedback pattern from Design CodeSignal's queue position).
  • 4) Completion flips the job to done with a signed, expiring download URL - exports are bulk PII, so links die.
  • 5) Failures retry idempotently by job id (partial files are overwritten); per-tenant concurrency caps keep Monday-morning export storms fair.

Editing a live survey - schema versioning (step by step):

  • The dangerous moment: 10K responses collected, and the creator edits question 3. What do old answers mean now?
  • 1) Published versions are immutable; the edit creates version N+1, and every response permanently records the version it answered.
  • 2) Additive changes (new question, new option) keep question identity: dashboards aggregate across versions by stable question_id, and older responses simply show no data for new questions.
  • 3) Destructive changes (removing options, rewording the question's meaning) get a new question_id - the analytics deliberately split into before/after cohorts rather than silently blending answers to two different questions under one label.
  • 4) The builder warns creators before destructive mid-collection edits, because the split is real and no storage trick removes it.
  • The principle underneath: data outlives schema - version the schema, pin every row to the version it was born under, and refuse to reinterpret history.

Database Types and Schemas

Database Schemas

PostgreSQL - the truth side

- surveys
  survey_id UUID PRIMARY KEY,
  creator_id UUID NOT NULL,
  slug VARCHAR(80) UNIQUE,           -- the public link
  current_version INT NOT NULL DEFAULT 1,
  status survey_status_enum,         -- draft | live | closed
  created_at TIMESTAMP DEFAULT NOW()

- survey_versions (immutable once published)
  survey_id UUID, version INT,
  definition JSONB NOT NULL,
    -- { questions: [{ id, type, title, required, options[],
    --    validation{}, show_if{question_id, equals} }], settings{} }
  published_at TIMESTAMP,
  PRIMARY KEY (survey_id, version)

- responses (partitioned by hash(survey_id), month)
  response_id UUID PRIMARY KEY,
  survey_id UUID NOT NULL,
  survey_version INT NOT NULL,       -- pins meaning: data outlives schema
  respondent_token VARCHAR(64),
  idempotency_key VARCHAR(128) NOT NULL,
  answers JSONB NOT NULL,            -- { "q_nps": 9, "q_feedback": "..." }
  source VARCHAR(30),                -- link | email | embed
  completion_seconds INT,
  submitted_at TIMESTAMP DEFAULT NOW(),
  UNIQUE (survey_id, idempotency_key)   -- THE dedupe referee

- outbox (same-transaction event relay -> Kafka)
  event_id BIGSERIAL PRIMARY KEY,
  response_id UUID, survey_id UUID,
  payload JSONB, relayed BOOLEAN DEFAULT FALSE

- export_jobs
  job_id UUID PRIMARY KEY, survey_id UUID, creator_id UUID,
  filters JSONB, format export_format_enum,
  status job_status_enum,            -- queued | running | done | failed
  progress_pct INT DEFAULT 0,
  file_ref VARCHAR(512), expires_at TIMESTAMP

ClickHouse - the derived side (rebuildable from truth)

- answers  -- one row per (response, question): the flattened analytics copy
  CREATE TABLE answers (
    survey_id UUID, question_id String, question_version Int32,
    response_id UUID, submitted_at DateTime,
    source LowCardinality(String),
    answer_text Nullable(String),
    answer_number Nullable(Float64),
    answer_choice LowCardinality(Nullable(String))
  ) ENGINE = MergeTree()
  PARTITION BY toYYYYMM(submitted_at)     -- date filters prune whole months
  ORDER BY (survey_id, question_id, submitted_at);
  -- LowCardinality: "Yes/No/Maybe" columns compress 10-100x

Redis

- Live tiles:      "stats:{survey_id}:{question_id}" -> {option: count, ...}
- Funnel:          "funnel:{survey_id}" -> {loaded, started, completed}
- Partial saves:   "progress:{respondent_token}" -> answers-so-far, TTL 24h
- Counter buffers: flushed to tiles every few seconds (Newsfeed pattern)

API Request/Response Payloads

POST /v1/responses
Request
{
  "survey_id": "s_8f3e2d1c",
  "survey_version": 3,
  "idempotency_key": "s_8f3e2d1c:rt_7a2b8f4e",
  "answers": {
    "q_nps": 9,
    "q_feedback": "Loved the onboarding flow."
  },
  "source": "email",
  "completion_seconds": 74
}
Response
{
  "response_id": "r_4c8e6f2a",
  "status": "recorded"
}

// A retried or double-clicked submit carries the same idempotency_key,
// hits the unique constraint, and receives THIS same response_id with a
// 200 - the caller cannot tell it was a duplicate, which is the point.
// The outbox event was written in the same transaction, so analytics
// can never disagree with the truth store about this response existing.
GET /v1/surveys/:id/analytics (tiles + a filtered query)
Response
{
  "live": {
    "responses": 184203,
    "completion_rate": 81.4,
    "freshness": "updated 3s ago",
    "questions": {
      "q_nps": { "avg": 7.9,
                 "distribution": { "9": 41230, "10": 38854, "...": "..." } }
    }
  },
  "filtered_query": {
    "filters": { "date_from": "2026-06-01", "source": "email",
                 "completed": true },
    "q_nps_avg": 8.3,
    "rows_scanned": 2941000,
    "query_ms": 340
  }
}

// Two tiers on one screen: "live" comes from Redis pre-aggregates
// (seconds fresh, zero query cost); "filtered_query" ran on ClickHouse -
// ~3M rows scanned in 340ms, which is the columnar layout earning its keep.
POST /v1/surveys/:id/exports → GET /v1/exports/:id
Request
{
  "format": "csv",
  "filters": {
    "date_range": { "start": "2026-06-01", "end": "2026-06-30" },
    "completed_only": true
  }
}
Response
// immediately:
{ "job_id": "exp_9d4f1a2b", "status": "queued" }

// GET /v1/exports/exp_9d4f1a2b, a minute later:
{
  "job_id": "exp_9d4f1a2b",
  "status": "running",
  "progress_pct": 62,
  "rows_so_far": 6200000
}

// ...and when done:
{
  "job_id": "exp_9d4f1a2b",
  "status": "done",
  "row_count": 10014312,
  "file_size_mb": 812,
  "download_url": "https://exports.example.com/exp_9d4f1a2b.csv?sig=...&exp=...",
  "url_expires_at": "2026-07-13T18:04:11Z"
}

// Minutes of work behind an honest job: progress instead of a hung
// request, and a signed expiring link because an export is bulk PII.
Error responses (examples)
Response
// 422 - server-side validation is the gate (client validation
//       was only a courtesy)
{
  "error": "validation_failed",
  "message": "Question q_nps requires a number between 0 and 10",
  "question_id": "q_nps"
}

// 410 - survey closed by its creator or past its end date
{
  "error": "survey_closed",
  "message": "This survey is no longer accepting responses"
}

// 404 - survey does not exist or belongs to another creator: identical
//       on purpose, so survey ids cannot be probed across tenants
{
  "error": "not_found",
  "message": "Survey not found"
}

💡 Three terms from the flows: transactional outbox, respondent token, pre-aggregation

Transactional outbox: the pattern for "write to my database AND tell the event stream" without a gap. Writing to two systems separately means a crash between them either loses the event (analytics never hears about a real response) or invents one (analytics counts a response that rolled back). The outbox fixes it by writing the event as a ROW in the same database transaction as the response; a relay process then ships outbox rows to Kafka. The response and its announcement commit atomically or not at all. Design Calendly used the identical move for calendar sync jobs. Respondent token: the anonymous identity substitute - a random id minted on first form load and kept in the browser. It anchors partial-progress saves and the idempotency key. It deliberately identifies a browser, not a person; the requirements note covers what that does and does not promise. Pre-aggregation: keeping the answer to a known query up to date as data arrives, instead of recomputing it per request. The live tiles are pre-aggregation (Redis counters updated per response, read 50M times/day for free); the filtered queries are its opposite - computed on demand, because the filter combinations are unbounded. The dividing rule: pre-aggregate what everyone asks constantly; compute on demand what anyone might ask once.

Concluding Discussion

Potential Bottlenecks

  • Issue: A viral survey concentrates thousands of writes/sec plus obsessive dashboard refreshes on one survey.
    Mitigation: stateless ingestion scales out and the truth write is a cheap constrained append; counters absorb the hot key via buffered increments; the creator's dashboard reads pre-aggregates, not queries. Degradation order is explicit: derived data may lag minutes, the write path never blocks.
  • Issue: Dashboard queries over 10M+ response surveys get slow or expensive.
    Mitigation: the columnar layout (read one column, not every row), month partitioning (date filters prune untouched data), LowCardinality compression - and the tiles everyone actually watches never query ClickHouse at all.
  • Issue: Monday-morning export storms - thousands of large exports queued at once.
    Mitigation: background jobs with per-tenant concurrency caps and priority tiers, chunked streaming (no export ever materializes in memory), and honest progress so nobody resubmits a "stuck" job.

Single Points of Failure

  • Kafka / the stream consumers: if they stop, dashboards and exports go stale - but the outbox keeps accumulating events inside PostgreSQL, so recovery replays everything and no response is ever missing from analytics. Degrade derived, protect truth.
  • ClickHouse: its loss is dashboard downtime, not data loss - it is a derived copy, rebuildable from the truth store. Run it replicated to make that downtime rare rather than to protect data.
  • PostgreSQL primary: the one component whose failure matters. Hot standby with automatic failover; in-flight respondents lose nothing because partial progress sits in Redis and the browser retries the submit - idempotently - once the standby is up.

Security Improvements

  • Tenant isolation on every query: a creator's survey_id scope rides every read and write, and cross-tenant probes get the same 404 as nonexistent surveys.
  • Anonymity mode severs the link: when enabled, the respondent token is dropped at ingestion (kept only transiently for dedupe) and IP/user-agent are never stored - anonymity is enforced by not recording, not by promising.
  • Deletion must reach the derived copies: truth-row deletes are clean; ClickHouse deletion runs as a batched tombstone-and-rewrite pipeline - GDPR compliance is a pipeline, not a DELETE statement.
  • Public forms are abuse surfaces: rate limiting per token and IP, CAPTCHA hooks for suspicious traffic, and server-side validation as the only gate that counts. Export links are signed and expiring - an export file is bulk PII.

Critical Knowledge

If the interviewer asks "why keep responses in two databases?":

  • Because the two halves of the product ask opposite-shaped questions of the same data: ingestion asks row-shaped ones (write this response, check this key), analytics asks column-shaped ones (aggregate one field across two million rows).
  • A row store answers aggregations by reading every full row and discarding 95% of each; a columnar store reads just the needed column, compressed 10-100x, with partitions pruned - that stack is the dashboard's 100x.
  • Keep the roles straight: PostgreSQL is the truth (constraints, transactions, backups); ClickHouse is a derived, rebuildable copy fed by an event stream - its loss is downtime, never data loss.
  • Then flip it for judgment: below millions of rows per survey, PostgreSQL with indexes does both jobs and the second store is pure operational drag. The split is earned by scan sizes, not by fashion.

If the interviewer asks "how do you stop duplicate responses when respondents have no accounts?":

  • Mint identity: a respondent token on first form load, and an idempotency key derived from (survey, token) on submit; a unique constraint makes the retry, the double-click, and the back-button resubmit all collapse into the original response - returned with a 200, indistinguishable from success.
  • Name the invariant: retry on silence, dedupe by key on arrival - Design WhatsApp's delivery discipline guarding opinions instead of messages.
  • Then be honest about the boundary: a token is a browser, not a human - incognito or a second device defeats it, and true one-per-person requires authentication.
  • The senior move is scoping the promise out loud: accidents and retries are prevented by architecture; determined adversaries are handled by product friction (one-per-email invites), not by pretending the token is stronger than it is.

If the interviewer asks "what happens when a survey goes viral?":

  • Walk the pressure points in order. Form loads: already the CDN's problem. Writes: stateless ingestion scales out, and the truth write is a cheap constrained append with no read-modify-write - thousands/sec on one survey is absorbable. Counters: the hot key, flattened by buffered increments (the Newsfeed pattern).
  • Then give the degradation order, because that is the real answer: if the stream or loaders lag, dashboards go stale by minutes while ingestion stays untouched - derived data lags, truth never blocks.
  • State the priority as a principle: when the system's two personalities compete under load, ingestion wins always, because analytics is recoverable and a lost response is invisible and gone forever.

If the interviewer asks "why are exports background jobs instead of a download button?":

  • Because the top end is 10M rows - minutes of streaming - and holding an HTTP request open for minutes is a designed failure: proxies time out, users refresh, work restarts.
  • The job pattern: return a job id instantly, stream from ClickHouse in chunks to object storage (never materializing the file in memory), publish progress the UI can poll, finish with a signed expiring URL - expiring because an export is bulk PII.
  • Retries are idempotent by job id; per-tenant concurrency caps keep one enterprise's Monday export storm from starving everyone (the fairness lesson from Design CodeSignal's queue).
  • The general rule: any user-triggered work measured in minutes belongs behind a job with progress, not a request - honesty about latency is a design feature.

If the interviewer asks "what happens when a creator edits a survey that already has responses?":

  • The trap: reinterpreting old answers under a new question definition silently corrupts data - answers to "rate our support" must not be blended into a reworded "rate our product".
  • The mechanism: published definitions are immutable; every edit creates version N+1, and every response permanently pins the version it answered - data outlives schema, so version the schema and pin the data.
  • The policy riding on it: additive changes (new question, new option) keep question identity and aggregate seamlessly across versions; destructive changes get a new question_id, deliberately splitting analytics into before/after cohorts instead of blending two meanings under one label.
  • And the product half: warn the creator before a destructive mid-collection edit - no storage trick can merge cohorts that mean different things, and pretending otherwise is how dashboards lie.