A Two-Rail Analytics Workflow That Refuses to Lie
Two rails for turning Amplitude data into a deployed analytics page. A daily snapshot for recurring numbers. A question-driven build for meeting-driven investigations. Both share a data gate that refuses to ship a misleading result, and an independent critique step that has to find at least one thing wrong.
The problem
Alef ran on a healthy stack of analytics tools. Amplitude was capturing event-level behaviour from 1.8 million learners. Usersnap was catching qualitative feedback in-product. CSV exports were flowing into spreadsheets. And nobody could answer the simplest question that mattered to product decisions: what is the user actually trying to do, and where does the platform fail them?
Each tool was working. The integration was the gap. Product managers were citing Amplitude charts in standups. Designers were quoting Usersnap comments in critiques. Neither group was connecting the two. Decisions were being made on partial evidence, dressed up as data-driven.
The deeper problem surfaced once the integration started shipping. A fast pipeline produces fast pages. Fast pages produce fast errors. In one week of post-deploy commits, the team fixed mislabelled axes, percentages without denominators, footers in the wrong date format, and finding bullets that overstated what the data actually showed. Every error was caught by a human reading the live page. Every error should have been caught before the page was live.
What I built
An automated workflow that turns an Amplitude question into a deployed analytics page in 10 to 20 minutes. Not a dashboard. A workflow with two rails, three checks, and an independent critique step before anything ships.
Rail 1: the daily snapshot
One command. Same recurring charts, same shape, every day. The system reads a canonical chart registry, queries Amplitude, builds an HTML page, runs the data gate, and deploys to Vercel. End-to-end. No question, no review, because the questions are already encoded in the registry.
Rail 2: the question-driven build
Pass it a hypothesis, a design tension, or a quote from the room. It matches the question to a topic in the registry, runs discovery if no topic matches, builds a page that answers that specific question, and then stops. The build hands back a file path. I read it. Then I deploy. The stop is intentional. It was added in direct response to the rework week.
The data gate
Both rails run the same three checks before deploy. Sourcing: every number on the page has a chart ID, metric, date range, cohort, and supporting sample size. Mixed windows wear an amber badge so they cannot be silently averaged. Hallucination: at least three headline numbers are spot-checked against the raw Amplitude API responses from the build session. If the checks run on a page from a previous session with no tool history, the result is “cannot verify”, not “pass”. Privacy: no PII, no cohorts under five users, no internal URLs, no Jira or Notion IDs, no tokens, noindex, nofollow confirmed in the head. The gate refuses to deploy on any failure.
The cold-reader sub-agent
An independent critique step on Rail 2. It reads the file with no context from the build session and returns three to five bullets covering: the ten-second misread, the misleading number, the trace from question to finding, the conclusion test, and the weakest label. It must list at least one thing wrong. If genuinely nothing is wrong, it has to justify in one line.
The cold-reader exists because the build session and the review session share too much context. A reader who has watched the page being built has already accepted its framing. A reader without that context catches what the builder missed.
What mattered
The technical work was the easy part. The harder work was deciding what not to ship. Most analytics dashboards fail because they try to answer every question. Each page from this workflow answers one. Three questions, well-instrumented, beat thirty questions half-answered.
The real lesson was about trust. The data gate and cold-reader pattern came from a specific week of rework: labels that misread at a glance, percentages without denominators, calculated values that nobody could trace back to a chart ID. Each fix became a rule. Each rule became a check. The workflow now encodes those failure modes as automated guards, which means the team can move fast without sacrificing the discipline that earns design a seat at the product table.
The third lesson: AI-assisted tooling makes this kind of integration cheap. A workflow that would have taken a quarter of engineering time five years ago took a working week with Claude Code as the build partner. That changes what design operations can credibly own.
Results
Speed-to-insight reduced from days to 10 to 20 minutes per analysis.
Workflow adopted internally and now serves as the template for further analytics pages across the product organisation.
Documented as a P-ops playbook with command reference, design rules, and a labelling taxonomy derived from observed rework.
Design questions now arrive at sprint planning with data attached, rather than data being requested after a decision is already drafted.
What I learned
Most analytics work goes into building more dashboards. The leverage was in the discipline layer that sits above the dashboards: a gate that has to pass, a critique that has to find a flaw, a registry that makes the same question return the same answer two days running. The artefact is the page. The actual product is the trust the team places in the page.
The other thing this work taught me is that AI-augmented design operations is not a slogan. It is a real change in what a small design and product ops team can credibly own. A pipeline like this used to require a data engineer. It now requires a design leader who can specify the guardrails clearly enough to delegate the build to an AI partner. The role of the design leader shifts from doing the work to specifying the work in language that produces the right work.