token-forge
One design-token source (W3C DTCG JSON) built with Style Dictionary into CSS variables, a Tailwind v4 theme and typed JS; four Lit web components that use only semantic tokens; a GitHub Action that turns every token PR into a semver release plan; and an LLM migration agent that moves legacy CSS onto the tokens, measured against two codemods and a whole-file LLM rewrite. This page is styled with the generated tokens: the button above switches the theme by changing one attribute.
Open the component Storybook Code on GitHub
Token-migration benchmark
30 legacy stylesheets (20 CSS, 10 Lit), 438 graded declarations. Several tokens share a light-theme value (white is page, card and secondary-button background), so a value-only codemod can look right in light mode and still paint the wrong colour in dark mode. "Dark-theme paint errors" counts exactly that. Model: Qwen3-Coder 30B (Q4_K_M) on one NVIDIA L4 via Ollama, temperature 0.3, 3 seeds. The corpus and its labels are synthetic and were written by me; see the README for limits.
Held-out test split: 292 graded declarations
| Method | Correct | Dark-theme paint errors | Collateral edits | LLM calls / file |
|---|---|---|---|---|
| Codemod, exact value, first match | 76.4 % | 63.0 | 0.0 | 0.0 |
| Codemod + near-miss + selector heuristics | 94.2 % | 12.0 | 0.0 | 0.0 |
| LLM rewrites whole file | 90.6 % ± 1.1 | 15.0 | 0.0 | 0.0 |
| Agent, LLM sees only property + value | 87.0 % ± 0.0 | 31.0 | 0.0 | 3.5 |
| Agent, LLM sees file + rule context | 96.0 % ± 0.2 | 7.7 | 0.0 | 3.5 |
| By site type | exact | ambiguous | near | format | deprecated | dim | keep |
|---|---|---|---|---|---|---|---|
| Codemod, exact value, first match | 89.0/89 | 23.0/49 | 1.0/40 | 20.0/20 | 17.0/17 | 50.0/50 | 23.0/27 |
| Codemod + near-miss + selector heuristics | 89.0/89 | 38.0/49 | 38.0/40 | 20.0/20 | 17.0/17 | 50.0/50 | 23.0/27 |
| LLM rewrites whole file | 86.3/89 | 34.7/49 | 35.0/40 | 20.0/20 | 15.0/17 | 48.0/50 | 25.7/27 |
| Agent, LLM sees only property + value | 89.0/89 | 23.0/49 | 32.0/40 | 20.0/20 | 17.0/17 | 50.0/50 | 23.0/27 |
| Agent, LLM sees file + rule context | 89.0/89 | 39.3/49 | 38.0/40 | 20.0/20 | 17.0/17 | 50.0/50 | 27.0/27 |
Dev split (used for tuning): 146 graded declarations
| Method | Correct | Dark-theme paint errors | Collateral edits | LLM calls / file |
|---|---|---|---|---|
| Codemod, exact value, first match | 69.2 % | 44.0 | 0.0 | 0.0 |
| Codemod + near-miss + selector heuristics | 90.4 % | 12.0 | 0.0 | 0.0 |
| LLM rewrites whole file | 85.6 % | 20.0 | 0.0 | 0.0 |
| Agent, LLM sees file + rule context | 94.5 % | 6.0 | 0.0 | 4.2 |
| By site type | exact | ambiguous | near | format | deprecated | dim | keep |
|---|---|---|---|---|---|---|---|
| Codemod, exact value, first match | 47.0/47 | 12.0/28 | 0.0/26 | 4.0/4 | 3.0/3 | 24.0/24 | 11.0/14 |
| Codemod + near-miss + selector heuristics | 47.0/47 | 21.0/28 | 22.0/26 | 4.0/4 | 3.0/3 | 24.0/24 | 11.0/14 |
| LLM rewrites whole file | 47.0/47 | 19.0/28 | 17.0/26 | 4.0/4 | 3.0/3 | 21.0/24 | 14.0/14 |
| Agent, LLM sees file + rule context | 47.0/47 | 23.0/28 | 23.0/26 | 4.0/4 | 3.0/3 | 24.0/24 | 14.0/14 |
Token checks
32 WCAG 2.2 contrast pairs checked in both themes on every build; the lowest margin is 6 % above its minimum. Components are tested with axe (WCAG 2.2 AA) in both themes, with keyboard and form-association tests, and with visual snapshots in a pinned Playwright container.