Build the workflow,
not a description of it.
TikTok posted a Product Designer role focused on design engineering and AI workflow. The core ask: turn a design system into something an AI agent can actually generate from — and apply design judgment to curate and elevate the output.
Most AI-generated UI work looks like it. Generic spacing, arbitrary colors, decoration standing in for decisions. The underlying problem is usually the same: no real system for the AI to check its work against. Just a vague instruction to "match the brand."
Rules specific
enough to fail.
Eight reference components were hand-built to establish ground truth. The token file uses the W3C Design Tokens Community Group format — the same structure tools like Style Dictionary consume in production. Every color, type scale, spacing value, radius, and motion timing lives in that format.
Alongside the tokens, a separate rules file defines checks a component must pass. The most load-bearing rule: card containers use a signature asymmetric radius — 28px 2px 28px 2px. It distinguishes the design language from a generic dark card and is the rule the AI most consistently fails.
A rule you can't fail is documentation, not a constraint. Every rule in the set was written to be checkable.
Generate → Critique
→ Revise.
A Python script drives the full pipeline. Step 1 generates a self-contained HTML component from the token file and rules. Step 2 sends a Playwright screenshot of that component to the model with the critique prompt — check each rule, report pass/fail with specific evidence. Step 3 feeds the critique back in and asks for a revised component that addresses the failures.
The component: a Trending Sounds panel, the canonical surface for TikTok's audio discovery feature. Sounds are audio clips attached to user-generated videos — not music tracks. Sound thumbnails show the original creator's circular profile avatar, not album art. The metric is number of videos made with a sound, not play count.
This intervention is part of the case study, not a footnote. Knowing where AI generation breaks — and why — is as useful as knowing where it works.
Left: raw generation. Correct TikTok-native content (circular creator avatars, video use counts, "Use Sound" CTAs) but rule violations — Inter on the heading, uniform border-radius, three equal primary action buttons. Right: post-revision. Clash Display heading fixed, buttons differentiated (one filled primary, two outlined). Signature-cut radius still not applied.
Accurate critique.
Honest limits.
The critique step correctly identified all rule violations in the raw output — off-token avatar colors, uniform border-radius where signature-cut was required, Inter font on a heading that needed Clash Display, three equal primary buttons violating the action hierarchy rule.
The most telling result: the critique correctly flagged R6 (motion timing) as unverifiable from a static screenshot. It didn't guess. It said it couldn't check and moved on. Across every run, it never claimed to verify a rule it couldn't — which is what you want from an automated QA step.
The revision also fixed R7 meaningfully: it demoted two of the three filled "Use Sound" buttons to outlined variants, correctly establishing one primary action and two secondary ones.
Content regression
on every revision pass.
In every revision, the model improved rule conformance and dropped content simultaneously. R2 — the signature-cut radius — was called out explicitly in every critique and never applied in any revision. The model defaulted to uniform rounding every time, regardless of how specifically the rule was stated.
The revision also consistently shed elements that weren't the subject of any rule violation. If the critique said "fix the font," the revision fixed the font and dropped a data row. The model treated revision as an opportunity to simplify, not just correct.
A reliable QA tool.
An unreliable finisher.
The generate-critique-revise loop is a strong mechanism for surfacing rule violations. The critique step is accurate, epistemologically honest about its limits, and produces actionable output. That part works.
The revision step is not the final output. It fixes some things and breaks others. The loop is most useful as a structured QA pass that surfaces decisions for a human designer — not as a fully autonomous generator.
The practical workflow: generate a component, run the critique, use the critique output as a structured fix list, apply the fixes with human judgment. The AI surfaces the violations. The designer decides how to resolve them.
Grade the rule set
before grading the model.
While preparing a rebuild of the pipeline, an audit of the constraint files found something uncomfortable: the rules file stated the signature radius as 20px 4px 20px 4px, while the token file said 28px 2px 28px 2px. The rules file was never updated when the radius was pushed further during the system build. The model had been fed two contradictory values for the most distinctive rule in the system — every run.
Correcting the record mattered more than protecting the earlier narrative — the correction is documented in the case study itself, not silently patched.
Generate a product.
Then drive it like a user.
Version one asked for a component and got a picture. Version two treats every documented failure as an engineering requirement, and changes the ask: generate a complete, functional single-page app — snap-scroll video feed, engagement rail with working like state, comments bottom sheet with a gated send button, five-tab navigation, profile view — in one self-contained HTML document with real JavaScript.
Output streams at up to 64K tokens, and truncation aborts loudly instead of rendering garbage (v1's black-screen failure). Every control carries a test hook, because the pipeline no longer just screenshots the result: Playwright renders the app at a mobile viewport, taps the like button, opens the comments sheet, and switches to the profile tab — screenshotting each state. A control the harness can't click is an automatic failure.
The critique step now receives all four screenshots and the full source. That closed v1's biggest gap: the 120ms motion rule went from "cannot verify from a static screenshot" to verified in the CSS, and a functional checklist is checked against the JavaScript itself.
All four scripted interactions succeeded on the first generation — the like toggled to its filled accent state, the sheet opened with a disabled-until-typed send button, the nav switched views. Same model as v1; the difference was the ask.
Reviewing the product,
not the pixels.
With source access, the automated review produced findings a screenshot-only pass could not: arbitrary hex backgrounds with no token backing, raw pixel values dodging the spacing scale, tertiary text below AA contrast at 12px, a decorative accent glow violating the action-color rule — and two dead controls: buttons with polished hover states and no click handlers. A screenshot review would have praised them.
One finding showed genuine care: the like count appeared unchanged between the before/after screenshots. Instead of failing it, the critique traced the JavaScript, confirmed the increment fires (243,000 → 243,001), and identified the count formatter as the reason the change is invisible at "243K".
V1's most consistent failure was revision-as-simplification: every fix pass shed content. V2's revise step carries a hard constraint — fix only what's flagged, remove nothing. In this run it held: every card, count, control, and test hook survived, and all four interactions passed again on the revised build. One clean run is one data point, not proof — the writeup says so. Open the V2 build →
Same loop.
Real components.
No product team ships a monolithic HTML file. V3 changes the output contract to what a team actually consumes: React function components — props-driven, composed by an App that owns state, with every color, spacing, radius, and timing read from a TOKENS object. The critique gains a third grading dimension alongside design rules and functional checks: code quality — components must be props-driven, no magic values, content flows as data, state lives in one place.
It also raised its own bar. V2's critique noticed that liking a 243K-like video produces no visible count change — and excused it after confirming the logic. V3's critique graded the same phenomenon as a specification failure: a demo seeded at 48.2K cannot prove "the count visibly changes."
The revision fixed the architecture (magic values 14 → 1, all content as props, comment state lifted to App — the vanishing-comments bug is structurally gone) and reseeded the like counts to small values so the increment is literally visible: 482 → 483. It didn't just fix code; it chose data that makes the requirement provable.
Don't take the
screenshots' word for it.
Below is the revised output of the pipeline — every component, style, and interaction exactly as the model produced them. The only processing is mechanical: the JSX is compiled ahead of time and React is bundled into the file so the embed can't be blanked by a CDN outage (raw artifact, byte for byte). Scroll the feed, double-tap a video, tap the heart, open the comments, switch tabs.
Interactions are real, data is hardcoded, nothing persists. Tap the heart, open the comments, switch tabs.
Audit the harness,
not just the output.
The pipeline logged warnings that the nav test hooks were missing from both builds — yet every automated nav click succeeded and the profile screenshots prove it. The warning was a false positive in my own harness: it greps for literal data-testid strings, but React constructs them dynamically. The static check was written for V2's output format and never updated for V3's.
Part three's weak link was the constraint files; here it was the test harness. Same lesson, one level up: every time the thing being verified gets more sophisticated, the verifier has to be re-audited too.
The ask determines
the output.
Across three pipeline versions the same model produced a static card, a working app, and a componentized codebase with its architecture reviewed and corrected. The constant was the loop; the variable was the contract — what the generation was asked to produce and what the critique was equipped to check. A rule set has to be audited before it can grade anything; my own contradiction quietly corrupted a documented conclusion. And verification has to touch the artifact: driving the app with real automated interactions surfaced dead controls and confirmed working state in a way no amount of screenshot critique could.