Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,10 @@ jobs:
landing-next-e2e:
runs-on: ubuntu-latest
timeout-minutes: 60
env:
# Render the frozen e2e/fixtures/benchmark-results.json, so a benchmark
# refresh on main never changes a screenshot. This build is never deployed.
LANDING_BENCHMARK_DATA: fixture
steps:
- name: Checkout
uses: actions/checkout@v7
Expand Down Expand Up @@ -885,9 +889,13 @@ jobs:
# changepacks compares the change against a local main, which a pull request checkout does not have
git fetch --no-tags origin main:refs/heads/main
bunx @changepacks/cli@latest check --format json
# The e2e builds render the frozen e2e/fixtures/benchmark-results.json, so
# a benchmark refresh on main never changes a screenshot. The site is
# deployed from a separate build of the live benchmark-results.json below.
- name: Build Landing
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
LANDING_BENCHMARK_DATA: fixture
run: |
bun run --filter @devup-ui/components build-storybook
mv ./packages/components/storybook-static ./apps/landing/public/storybook
Expand Down Expand Up @@ -921,6 +929,7 @@ jobs:
- name: Build Landing (singleCss)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
LANDING_BENCHMARK_DATA: fixture
run: |
rm -rf apps/landing/dist apps/landing/.next apps/landing/.vinext apps/landing/df
DEVUP_SINGLE_CSS=1 bun run --filter landing build
Expand All @@ -933,6 +942,15 @@ jobs:
name: playwright-report-singlecss
path: playwright-report/
retention-days: 1
# The same singleCss build as above, with the live benchmark numbers.
- name: Build Landing for deployment
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
rm -rf apps/landing/dist apps/landing/.next apps/landing/.vinext apps/landing/df
DEVUP_SINGLE_CSS=1 bun run --filter landing build
grep -q 'data-benchmark-source="live"' apps/landing/dist/client/index.html
- name: Upload the site for deployment
uses: actions/upload-pages-artifact@v5
with:
Expand Down Expand Up @@ -1053,3 +1071,101 @@ jobs:
NATIVE_RELEASE_BUNDLE: ${{ runner.temp }}/native-release-bundle
with:
publish: true

# Publishes the numbers `benchmark` measured on a push to main: rewrites
# benchmark-results.json (the README sections and the landing page read it),
# appends the run to benchmark/history.json, and pushes that as one commit to
# main, so every merge leaves its numbers next to it and a slower build can be
# traced to the merge that caused it (`bun benchmark-refresh.js --trend <row>`).
#
# It cannot release or loop: it waits for `release` to finish so it never
# races changepacks, it runs no build, publish or changepacks step, and the
# commit it pushes is made with GITHUB_TOKEN (whose pushes start no workflow)
# and carries [skip ci]. It records a failed gate too, so the regression is in
# the history, and skips a run whose artifact is missing.
#
# A commit older than the published results is only added to the history
# (benchmark-refresh.js checks the ancestry). If main moves during the job it
# rebuilds the commit on the new HEAD; if pushing to main keeps failing (for
# example once a ruleset protects main without a bypass for GitHub Actions),
# it pushes the commit to `chore/benchmark-refresh` and keeps one pull request
# open for it instead.
benchmark-refresh:
needs: [benchmark, release]
if: ${{ !cancelled() && github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.benchmark.result != 'skipped' }}
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: benchmark-refresh-main
cancel-in-progress: false
env:
GIT_CONFIG_COUNT: "1"
GIT_CONFIG_KEY_0: core.autocrlf
GIT_CONFIG_VALUE_0: "false"
permissions:
contents: write
pull-requests: write
steps:
- name: Download the measured run
id: measured
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: benchmark-run
path: ${{ runner.temp }}/benchmark-run
- name: Checkout main
if: steps.measured.outcome == 'success'
uses: actions/checkout@v7
with:
ref: main
# benchmark-refresh.js compares commits and reads their baselines
fetch-depth: 0
- uses: oven-sh/setup-bun@v2
if: steps.measured.outcome == 'success'
with:
bun-version: latest
name: Install bun
- name: Record the run and push it to main
if: steps.measured.outcome == 'success'
env:
GH_TOKEN: ${{ github.token }}
MEASURED: ${{ runner.temp }}/benchmark-run/benchmark-run.json
MESSAGE: "chore(benchmark): record run ${{ github.run_id }} of ${{ github.sha }} [skip ci]"
BRANCH: chore/benchmark-refresh
shell: bash
run: |
set -euo pipefail
git config user.name 'github-actions[bot]'
git config user.email '41898282+github-actions[bot]@users.noreply.github.com'

# The scripts need no dependencies; only the files they write are staged.
record() {
git fetch -q origin main
git reset -q --hard origin/main
cp "$MEASURED" benchmark-run.json
bun benchmark-refresh.js
bun render-benchmark-readme.js
rm benchmark-run.json
git add -- benchmark-results.json benchmark/history.json README.md README_ko.md 'packages/*/README.md'
}

for attempt in 1 2 3; do
record
if git diff --cached --quiet; then
echo "Nothing to record for ${GITHUB_SHA}"
exit 0
fi
git commit -q -m "$MESSAGE"
if git push origin HEAD:refs/heads/main; then
exit 0
fi
echo "Push to main failed (attempt ${attempt}); rebuilding on the new HEAD"
done

echo "::warning::Could not push the benchmark refresh to main; updating ${BRANCH} instead"
git push --force origin "HEAD:refs/heads/${BRANCH}"
if [ -z "$(gh pr list --head "$BRANCH" --state open --json number --jq '.[].number')" ]; then
gh pr create --base main --head "$BRANCH" \
--title 'chore(benchmark): record the latest main benchmark' \
--body 'Written by the `benchmark-refresh` job because it could not push to main. It carries the latest main run only; merging it publishes the numbers and its history entry.'
fi
38 changes: 19 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,33 +81,33 @@ npm install @devup-ui/webpack-plugin

<!-- benchmark:start -->

[Latest CI benchmark](https://github.com/dev-five-git/devup-ui/actions/runs/37758611477) (commit `467a379a`, Devup UI 1.0.44) on `ubuntu-24.04` with Next.js 16.3.3. Every row is the median of 3 cold builds that ran in rotating order, shown with the range of the samples. All Next.js builds use the native TypeScript 7 CLI for type checking. The English and Korean tables are generated from the same checked result file, `benchmark-results.json`.
[Latest CI benchmark](https://github.com/dev-five-git/devup-ui/actions/runs/43770976836) (commit `c750d33c`, Devup UI 1.0.44, 2026-11-15) on `ubuntu-26.04` with Next.js 16.4.6. Every row is the median of 3 cold builds that ran in rotating order, shown with the range of the samples. All Next.js builds use the native TypeScript 7 CLI for type checking. The English and Korean tables are generated from the same checked result file, `benchmark-results.json`, which CI refreshes after every merge to main and records in `benchmark/history.json`, one entry per commit, so a slower build can be traced to the merge that caused it.

Build Size is every byte of the build output directory (`.next`, or `dist` for vinext), including caches and server output. The checked table retains its original historical CSS measurements, which recursively counted `.css` files throughout that output. New runs count only browser-shipped CSS: `.next/static/**/*.css` (Webpack `static/css` and Turbopack `static/chunks`) or vinext `dist/client/**/*.css`, excluding caches and server CSS. A missing browser-assets directory is an error; an existing directory with no CSS counts zero. Approved CSS measurement corrections are limited to explicitly named `devup-ui`, `devup-ui-single` and `vinext-devup-ui` rows. `bun benchmark-check.js --write-css-baseline devup-ui` retains the per-file mode; `bun benchmark-check.js --write-css-baseline devup-ui-single,vinext-devup-ui` selects only the other two rows in one comma-separated argument. Only selected existing rows' CSS bytes change; unlike `--write-baseline`, this preserves sample count, build bytes, ratios, tolerances, all unselected rows and calibration history. Missing or unsupported names and conflicting writer modes are rejected.
Client Size is the JavaScript and CSS shipped to the browser, uncompressed: `.next/static/**/*.{js,css}` or vinext `dist/client/**/*.{js,css}`. Build Size (incl. caches) is every byte of the build output directory (`.next`, or `dist` for vinext); most of it is the bundler cache and server output, and it moves by a few kilobytes between identical builds, so it is kept for continuity rather than as what users download. CSS Size counts only browser-shipped CSS: `.next/static/**/*.css` (Webpack `static/css` and Turbopack `static/chunks`) or vinext `dist/client/**/*.css`, excluding caches and server CSS. A missing browser-assets directory is an error; an existing directory with no CSS counts zero. Every size is measured on every sample and shown as the median; `benchmark-results.json` keeps the samples with their range, and rows measured before Client Size was recorded show n/a. Approved CSS measurement corrections are limited to explicitly named `devup-ui`, `devup-ui-single` and `vinext-devup-ui` rows. `bun benchmark-check.js --write-css-baseline devup-ui` retains the per-file mode; `bun benchmark-check.js --write-css-baseline devup-ui-single,vinext-devup-ui` selects only the other two rows in one comma-separated argument. Only selected existing rows' CSS bytes change; unlike `--write-baseline`, this preserves sample count, build bytes, ratios, tolerances, all unselected rows and calibration history. Missing or unsupported names and conflicting writer modes are rejected.

Webpack:

| Library | Version | Median Build Time | Range | Build Size | CSS Size |
| --------------------------- | ------- | ----------------- | --------------- | ----------------- | ------------ |
| tailwindcss | 4.3.3 | 13.02s | 12.91s - 13.20s | 67,339,250 bytes | 5,852 bytes |
| styleX | 0.19.0 | 29.01s | 28.72s - 29.50s | 96,290,533 bytes | 425 bytes |
| vanilla-extract | 1.21.2 | 12.34s | 12.22s - 12.35s | 68,565,511 bytes | 294 bytes |
| kuma-ui | 1.6.4 | 13.55s | 13.49s - 13.57s | 75,627,429 bytes | 340 bytes |
| panda-css | 1.12.0 | 13.44s | 13.29s - 13.58s | 71,849,243 bytes | 15,615 bytes |
| chakra-ui | 3.37.0 | 20.29s | 19.95s - 21.06s | 207,425,028 bytes | 0 bytes |
| mui | 9.4.0 | 14.12s | 13.78s - 14.26s | 101,476,098 bytes | 0 bytes |
| **devup-ui (per-file CSS)** | 1.0.44 | **10.95s** | 10.87s - 11.09s | 67,427,695 bytes | 562 bytes |
| **devup-ui (single CSS)** | 1.0.44 | **11.04s** | 10.88s - 11.40s | 67,436,336 bytes | 790 bytes |
| Library | Version | Median Build Time | Range | Client Size | Build Size (incl. caches) | CSS Size |
| --------------------------- | ------- | ----------------- | --------------- | ----------- | ------------------------- | ------------ |
| tailwindcss | 4.3.5 | 10.23s | 10.16s - 11.07s | n/a | 69,807,397 bytes | 5,851 bytes |
| styleX | 0.19.2 | 24.33s | 24.04s - 25.58s | n/a | 99,500,267 bytes | 425 bytes |
| vanilla-extract | 1.21.3 | 10.14s | 9.56s - 10.29s | n/a | 70,957,513 bytes | 294 bytes |
| kuma-ui | 1.6.4 | 11.13s | 10.38s - 11.65s | n/a | 77,951,297 bytes | 340 bytes |
| panda-css | 2.5.0 | 11.04s | 10.14s - 11.22s | n/a | 75,860,686 bytes | 21,149 bytes |
| chakra-ui | 3.38.0 | 16.33s | 16.05s - 16.37s | n/a | 211,095,734 bytes | 0 bytes |
| mui | 9.5.0 | 11.33s | 10.65s - 12.03s | n/a | 104,631,987 bytes | 0 bytes |
| **devup-ui (per-file CSS)** | 1.0.44 | **9.11s** | 8.61s - 9.27s | n/a | 69,803,573 bytes | 494 bytes |
| **devup-ui (single CSS)** | 1.0.44 | **9.23s** | 8.65s - 9.31s | n/a | 69,788,823 bytes | 395 bytes |

Turbopack:

| Library | Version | Median Build Time | Range | Build Size | CSS Size |
| -------------------------------------- | ------- | ----------------- | ------------- | ---------------- | ----------- |
| tailwindcss | 4.3.3 | 5.68s | 5.51s - 5.73s | 38,440,013 bytes | 6,197 bytes |
| **devup-ui (direct APIs, single CSS)** | 1.0.44 | **5.79s** | 5.77s - 5.87s | 36,538,765 bytes | 327 bytes |
| **devup-ui (static `.css.ts`)** | 1.0.44 | **5.71s** | 5.67s - 5.99s | 36,580,207 bytes | 268 bytes |
| Library | Version | Median Build Time | Range | Client Size | Build Size (incl. caches) | CSS Size |
| -------------------------------------- | ------- | ----------------- | ------------- | ----------- | ------------------------- | ----------- |
| tailwindcss | 4.3.5 | 3.91s | 3.65s - 3.94s | n/a | 30,536,946 bytes | 6,196 bytes |
| **devup-ui (direct APIs, single CSS)** | 1.0.44 | **3.85s** | 3.60s - 4.02s | n/a | 28,905,488 bytes | 327 bytes |
| **devup-ui (static `.css.ts`)** | 1.0.44 | **3.91s** | 3.61s - 3.95s | n/a | 28,950,340 bytes | 268 bytes |

The Turbopack ranges do not overlap, so the direct-API median is 0.12s (2.1%) behind Tailwind on this fixture. The 3 cold samples were Tailwind `5.68s, 5.73s, 5.51s`, direct Devup UI `5.77s, 5.79s, 5.87s`, and static `.css.ts` `5.71s, 5.99s, 5.67s`. The fixtures have comparable app shapes, not pixel-identical styling: Tailwind styles the leading paragraph and button more heavily, while Devup UI exercises typed component/style props. Treat these as build-pipeline results rather than a per-rule microbenchmark. Every `.css.ts` module runs on the full Boa evaluator, which adds about 20 ms for the first module and under 1 ms for each further one. CI compares the Devup UI rows with a checked baseline (see `benchmark-gate.js` for the rule) and fails on a regression beyond the measured noise.
The Turbopack ranges overlap, so the direct-API median is 0.05s (1.3%) ahead of Tailwind on this fixture and the two are effectively at parity. The 3 cold samples were Tailwind `3.94s, 3.65s, 3.91s`, direct Devup UI `3.85s, 3.60s, 4.02s`, and static `.css.ts` `3.95s, 3.61s, 3.91s`. The fixtures have comparable app shapes, not pixel-identical styling: Tailwind styles the leading paragraph and button more heavily, while Devup UI exercises typed component/style props. Treat these as build-pipeline results rather than a per-rule microbenchmark. Every `.css.ts` module runs on the full Boa evaluator, which adds about 20 ms for the first module and under 1 ms for each further one. CI compares the Devup UI rows with a checked baseline (see `benchmark-gate.js` for the rule) and fails on a regression beyond the measured noise.
<!-- benchmark:end -->

## How it works
Expand Down
Loading
Loading