Repository navigation
Conversation
- react and bundler plugin READMEs: take Features and How it works from the root README (current base-37 output instead of fixed d0/d1 names), replace the stale 1.0.40 benchmark copy with a link to the generated root tables, and point the Korean link at the root README_ko.md - components and reset-css: replace the copied @devup-ui/react README with package-specific install, usage and API notes - eslint-plugin: note that preset rules can report the same code and fix a no-duplicate-value "correct" example that no-useless-tailing-nulls reports - AGENTS.md / libs/AGENTS.md: refresh commit, line counts and Oxc version Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restore the React, plugin, components and reset-css READMEs to the main branch structure and wording, then apply only the audit fixes: - Keep the benchmark section in every package README that had it. It sits between the benchmark markers, and render-benchmark-readme.js now writes it into those READMEs from benchmark-results.json together with the root README, so the existing CI check fails when a copy goes stale. The generator also leaves a blank line after the start marker so the READMEs stay prettier-clean. - Replace the fixed d0/d1 class names in the examples with the names the current build emits and add the note that generated names may change. - Point the Korean README link at the repository README_ko.md. - Components and reset-css: name their own package in the badges and install line and add a short usage section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a push to main, the new benchmark-refresh job takes the run the benchmark job measured. It rewrites benchmark-results.json, appends the run to benchmark/history.json (one entry per commit, with the gate verdict and the rows that regressed), regenerates the README sections and pushes the result to main as a [skip ci] commit made with GITHUB_TOKEN. It waits for the release job and runs no build, publish or changepacks step. A commit older than the published results is only added to the history. If pushing to main keeps failing, the job keeps one rolling pull request on chore/benchmark-refresh instead. The benchmark now measures client bytes (the JavaScript and CSS under .next/static or dist/client), keeps every size sample with its range, and records the runner, Next.js version, Devup UI version, the library version of each row and the date. The README tables gain a Client Size column and label Build Size as including caches. The checked results and the first history entry are seeded from the benchmark-run artifact of main run 43770976836 (commit c750d33, after the native engine), which predates client bytes, so that column shows n/a until the first refresh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The comparison cards now import the checked results instead of carrying numbers from run 33254030962, so the homepage shows the same run as the READMEs and moves with every refresh. The caption links the run and the commit it measured, and the crowns only show while Devup UI is the fastest or smallest card. Every benchmark value carries data-benchmark, and the Playwright screenshot config hides those elements through stylePath, so a refresh never changes a visual baseline. The baselines that include the bench section have to be regenerated once for this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #983, which records the styled identity wrapper in the extractor snapshots that made the publish job fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e2e/screenshot.css hides every [data-benchmark] element, so the bench section, its dark variant and the mobile component showcase no longer show the build times, sizes, versions or run line that the old baselines held. These are the actual images from the landing-next-e2e run 44011106735 (the same image on all three retries); they differ from the old baselines only in the hidden benchmark text and in the header's language switcher, which the old baselines predate and which the 1% ratio used to absorb. Only the Linux baselines are recorded here; CI runs the screenshots on Linux only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
Hiding the numbers with [data-benchmark] and a Playwright stylePath still left the screenshots tied to the shape of the live data. The e2e builds now render a frozen copy of benchmark-results.json instead, so a benchmark refresh on main can never change a screenshot. - e2e/fixtures/benchmark-results.json is the current results, frozen. - apps/landing/src/app/benchmark-data.ts picks the fixture when the build sets LANDING_BENCHMARK_DATA=fixture and the live file otherwise. Bench is a server component prerendered at build time, so the choice ships no client code, and it works the same under vinext and Next/Turbopack. The section carries data-benchmark-source="fixture|live". - CI sets the variable on every landing build that e2e serves (landing-next-e2e for the whole job; the two publish builds). On a push to main, publish rebuilds the singleCss site without it before uploading the Pages artifact and checks the deployed index.html says "live". - The local Playwright webServer sets the variable too. - e2e/landing-benchmark-fixture.spec.ts fails when a build under test reads the live file: it checks the source marker and that the section shows the fixture's run id, time and size. - e2e/screenshot.css, the stylePath and the data-benchmark markers are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bench section, its dark variant and the mobile component showcase now show the frozen e2e fixture (run 43770976836, commit c750d33, Devup UI 1.0.44) instead of hidden values. These are the actual images from the landing-next-e2e run 44029448584, identical on all three retries; the same run passed landing-benchmark-fixture.spec.ts, so they were rendered from the fixture. Only the Linux baselines are recorded: CI takes screenshots on Linux only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #911 (squash-merged as fc9a6f0), which this branch was stacked on. The README conflicts were all inside the generated benchmark sections; they were resolved by running `bun render-benchmark-readme.js`, and `--check` passes for every README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #963, which landed right after the previous merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The benchmark numbers in the READMEs and on the landing homepage now refresh themselves after every merge to main. Every run is also kept in a per-commit history, so you can see which change made a benchmark worse.
What happens on a push to main
A new
benchmark-refreshjob inpublish.ymlruns afterbenchmark. It takes thebenchmark-runartifact and does the following:benchmark-results.json: run id, full SHA and short commit, date, runner, Next.js and Devup UI versions, and each row's library version, samples, median, range, sizes, CSS size and client size.benchmark/history.json. There is one entry per commit, in commit order, capped at 500 entries (about 2 KB each, so about 1 MB). Each entry holds the gate verdict (pass/fail), the rows that regressed, and[median, min, max, bytes, cssBytes, clientBytes]for all 22 rows.README_ko.mdand the 7 package READMEs (added in docs: refresh package READMEs and agent metadata #911).chore(benchmark): record run <id> of <sha> [skip ci]and pushes them to main.Commit-back mechanism: a direct push to main, with a rolling PR as fallback
gh api …/branches/main/protectionreturns 404 (main has no branch protection). The only ruleset,main(id 9882973), hasconditions.ref_name.include: [], so it currently matches no branch. Default workflow permissions arewrite. SoGITHUB_TOKENcan push to main today.GITHUB_TOKENstarts no workflow. The commit also carries[skip ci]. So the refresh cannot recurse, and it cannot triggerpublish,releaseor changepacks.chore/benchmark-refreshand keeps a single PR open for it. A PR opened withGITHUB_TOKENgets no CI, and it only carries the latest run, so this path is a safety net, not the intended mode.Safety
needs: [benchmark, release]withif: !cancelled() && push to main. The job starts only afterrelease(changepacks) has finished, so the two never push at the same time. It runs no build, publish or changepacks step, and never toucheschangepacks/main(Update Versions #769).concurrency: benchmark-refresh-maingroup (cancel-in-progress: false) on top of the workflow's own per-ref group.benchmark-refresh.jsusesgit merge-base --is-ancestor.run.shais added to the history only and never overwrites newer results.Gate vs. display.
benchmark-gate.js,benchmark-baseline.jsonandbenchmark-check.jskeep their current behavior. The history is display and forensics only. Each entry's verdict is computed against the baseline checked in at that commit (git show <sha>:benchmark-baseline.json).How to trace a regression to a commit
bun benchmark-refresh.js --trend devup-ui-single-turbo. It prints one line per main commit: short SHA, commit time, gate verdict, median, ratio to the row's control in the same run, bytes, CSS bytes and client bytes.gateturnsfail, with the row listed inregressions) names the merge. Compare it with the line before.git log <previous>..<sha>lists the commits in between, which is usually a single squash merge.runid links to the full log and thebenchmark-runartifact. Workflow runs that a newer push cancelled while they were pending leave a gap. In that case the range covers more than one merge.Size metrics (per the size investigation)
.next/static/**/*.{js,css}, or vinextdist/client/**/*.{js,css}), uncompressed.sizes.{bytes,cssBytes,clientBytes}keep the samples with min and max.Seeded numbers.
benchmark-results.jsonand the first history entry come from thebenchmark-runartifact of main run 43770976836 (commitc750d33c, after NAPI), at full precision. These match themedianlines of job 132995530168. Library versions come frombun.lockat that commit. That run predates Client Size, so the column showsn/auntil the first refresh.Landing.
apps/landing/src/app/Bench.tsxreads its numbers throughbenchmark-data.ts(see Screenshots), which importsbenchmark-results.json. Bench is a server component, so no JSON ships to the client. The caption now links the run and the commit. Each crown shows only while Devup UI is the fastest or smallest card. Pages are deployed from the pushed commit, while the refresh lands right after it as a[skip ci]commit. So the homepage shows the newest numbers from the next deploy on, one merge behind the READMEs. Its run and commit links always match the numbers shown.Screenshots: the e2e builds render frozen fixture data. A benchmark refresh on main must never change a screenshot, so the e2e builds don't read the live numbers at all:
e2e/fixtures/benchmark-results.jsonis a frozen copy of the current results (run 43770976836, commitc750d33c, Devup UI 1.0.44).apps/landing/src/app/benchmark-data.tspicks the fixture when the build setsLANDING_BENCHMARK_DATA=fixture, and the livebenchmark-results.jsonotherwise. Bench is prerendered at build time, so the choice is made by the build and ships no client code. The same module works under vinext and Next/Turbopack, with no bundler alias. The section carriesdata-benchmark-source="fixture|live".landing-next-e2e, and on both landing builds inpublish. On a push to main,publishrebuilds the singleCss site without it before uploading the Pages artifact and checks that the deployedindex.htmlsayslive. The local PlaywrightwebServersets it too.e2e/landing-benchmark-fixture.spec.tsfails if a build under test reads the live file. It checks the source marker, and that the section shows the fixture's run id, time and size.e2e/screenshot.css,stylePath,data-benchmark) is removed.section-bench,dark-section-benchandcomponents-benchmark-section-mobile. The images were identical on all three retries, and that run passed the fixture spec. The other baselines that include the section (full-page-*,dark-full-page-*,components-benchmark-section-desktop,dark-components-benchmark-section-desktop) stay within the 1%maxDiffPixelRatio, so CI produced no actual images for them and they are unchanged.Owner decisions / required actions
*-win32.pngbaselines are never checked in CI and nothing in CI can regenerate them. They are produced on a Windows machine withbun run test:e2e:update, which now builds the landing with the fixture through the PlaywrightwebServer. Until then, a local Windows run shows diffs insection-bench,dark-section-benchandcomponents-benchmark-section-mobile.mainruleset targets no branch. If you fix the ruleset to targetmain(it requires thepublishandbenchmarkchecks with a strict policy), add a bypass for GitHub Actions or a GitHub App, or the job falls back to the rolling PR. A ruleset with strict required checks plus a bot commit after every merge would also mark every open PR out of date after each merge. Decide whether that is acceptable, or whether a path-based exception is preferable.Follow-ups (not in this PR)
next.config.tsvs.mjs.tsbuildinfodifference).docs/overview/page*.mdx) still shows its labeled-historical run 33254030962. It could read the same JSON.Test plan
bun test packages/plugin-utils/src/__tests__/benchmark-*.test.ts --coverage: 85 pass.benchmark-gate.js,benchmark-output.jsand the newbenchmark-record.jsare at 100% functions and lines. The CLI is tested end to end in a temporary git repository: newer commit, older commit, a commit with no baseline, a commit not on main,--trend, and error cases.benchmark-outputtests coverclientBytes.bun render-benchmark-readme.js --checkpasses for all 9 READMEs.formatJsonoutput is byte-identical to the checkedbenchmark-results.jsonandbenchmark/history.json(asserted in a test), andbunx prettier --single-quote --no-semi --checkpasses on every changed file.publish.ymlalready failed Prettier before this change.bun run lintis the final check.Bench.tsx,DevupUICard.tsxandOtherCard.tsxtype-check withtsc(TS 7), including the JSON import.actionlint1.7.12 onpublish.ymlreports only the existingwindows-11-vs2026-armrunner-label warnings.chore(benchmark): record run …, and the README Client Size column fills in.요약
main에 병합될 때마다 README와 랜딩 홈페이지의 벤치마크 수치가 자동으로 갱신됩니다. 모든 실행은 커밋별 이력에도 남기 때문에, 어떤 변경 때문에 벤치마크가 나빠졌는지 확인할 수 있습니다.
main push 시 동작
publish.yml에 새로 추가한benchmark-refreshjob은benchmark다음에 실행됩니다.benchmark-runartifact를 받아 다음 작업을 합니다.benchmark-results.json을 다시 씁니다: run id, 전체 SHA와 짧은 커밋, 날짜, runner, Next.js와 Devup UI 버전, 행마다 라이브러리 버전, 샘플, 중앙값, 범위, 사이즈, CSS 사이즈, 클라이언트 사이즈.benchmark/history.json에 실행을 추가합니다. 커밋마다 항목 하나를 커밋 순서대로 남기며 최대 500개까지 보관합니다(항목당 약 2KB, 전체 약 1MB). 항목마다 gate 판정(pass/fail), 회귀한 행, 22개 행 전체의[median, min, max, bytes, cssBytes, clientBytes]가 들어 있습니다.README_ko.md, 패키지 README 7개(#911에서 추가).chore(benchmark): record run <id> of <sha> [skip ci]로 커밋해 main에 push합니다.커밋 반영 방식: main에 직접 push하고, 실패하면 rolling PR로 대체
gh api …/branches/main/protection은 404를 반환합니다(main에 branch protection이 없음). 유일한 rulesetmain(id 9882973)은conditions.ref_name.include: []라서 현재 어떤 브랜치에도 적용되지 않습니다. 기본 워크플로 권한은write입니다. 따라서 지금은GITHUB_TOKEN으로 main에 push할 수 있습니다.GITHUB_TOKEN으로 한 push는 워크플로를 시작하지 않고, 커밋에[skip ci]도 붙어 있습니다. 그래서 재귀 실행이 생기지 않고publish,release, changepacks도 시작되지 않습니다.chore/benchmark-refresh에 force-push하고 PR 하나를 계속 열어 둡니다.GITHUB_TOKEN으로 연 PR에서는 CI가 돌지 않고 최신 실행만 담기 때문에, 이 경로는 비상용이지 기본 동작이 아닙니다.안전장치
needs: [benchmark, release]와if: !cancelled() && main push조건을 씁니다.release(changepacks)가 끝난 뒤에만 시작하므로 둘이 동시에 push하지 않습니다. 빌드, publish, changepacks 단계는 실행하지 않고changepacks/main(Update Versions #769)도 건드리지 않습니다.concurrency: benchmark-refresh-main그룹(cancel-in-progress: false)이 있습니다.benchmark-refresh.js가git merge-base --is-ancestor로 확인합니다.run.sha보다 오래된 커밋은 이력에만 추가하고 더 새로운 결과를 덮어쓰지 않습니다.gate와 표시.
benchmark-gate.js,benchmark-baseline.json,benchmark-check.js의 동작은 그대로입니다. 이력은 표시와 원인 추적에만 쓰입니다. 각 항목의 판정은 그 커밋에 체크인된 baseline(git show <sha>:benchmark-baseline.json)을 기준으로 계산합니다.회귀 원인 커밋 찾기
bun benchmark-refresh.js --trend devup-ui-single-turbo를 실행합니다. main 커밋마다 짧은 SHA, 커밋 시각, gate 판정, 중앙값, 같은 실행 안에서 control 행과의 비율, bytes, CSS bytes, client bytes를 한 줄씩 출력합니다.gate가fail이 되고regressions에 그 행이 나온 줄)이 원인 병합입니다. 바로 앞 줄과 비교해 보세요.git log <이전>..<sha>로 그 사이 커밋을 확인할 수 있으며, 보통 squash merge 하나입니다.runid로 전체 로그와benchmark-runartifact를 볼 수 있습니다. 대기 중에 더 새로운 push 때문에 취소된 워크플로 실행은 이력에 빈칸을 남깁니다. 이 경우 범위에 병합이 둘 이상 들어갈 수 있습니다.사이즈 지표 (사이즈 조사 결과 반영)
.next/static/**/*.{js,css}또는 vinextdist/client/**/*.{js,css})의 압축 전 크기입니다.sizes.{bytes,cssBytes,clientBytes}에 샘플과 min/max를 남깁니다.초기 수치.
benchmark-results.json과 첫 이력 항목은 main 실행 43770976836(커밋c750d33c, NAPI 이후)의benchmark-runartifact에서 전체 정밀도로 가져왔습니다. 이 값은 job 132995530168 로그의median줄과 같습니다. 라이브러리 버전은 그 커밋의bun.lock에서 가져왔습니다. 이 실행은 클라이언트 사이즈 측정 이전이라 첫 갱신 전까지 그 열은n/a로 표시됩니다.랜딩.
apps/landing/src/app/Bench.tsx는benchmark-data.ts(스크린샷 항목 참고)를 거쳐benchmark-results.json의 수치를 읽습니다. Bench는 서버 컴포넌트라 JSON이 클라이언트로 가지 않습니다. 캡션은 이제 실행과 커밋 링크를 보여 줍니다. 왕관은 Devup UI가 가장 빠르거나 가장 작을 때만 보입니다. Pages는 push된 커밋으로 배포되고, 갱신 커밋은 그 직후[skip ci]커밋으로 들어옵니다. 그래서 홈페이지는 다음 배포부터 새 수치를 보여 주며, README보다 병합 하나만큼 늦습니다. 실행/커밋 링크는 항상 보이는 수치와 일치합니다.스크린샷: e2e 빌드는 고정된 fixture 데이터를 렌더링합니다. main의 벤치마크 갱신이 스크린샷을 바꾸면 안 되므로, e2e 빌드는 실시간 수치를 아예 읽지 않습니다.
e2e/fixtures/benchmark-results.json은 현재 결과(실행 43770976836, 커밋c750d33c, Devup UI 1.0.44)를 고정한 사본입니다.apps/landing/src/app/benchmark-data.ts는 빌드에LANDING_BENCHMARK_DATA=fixture가 설정되면 fixture를, 아니면 실시간benchmark-results.json을 고릅니다. Bench는 빌드 시 prerender되므로 선택은 빌드가 하고, 클라이언트 코드는 추가되지 않습니다. 번들러 alias 없이 vinext와 Next/Turbopack에서 같은 모듈이 동작합니다. 섹션에는data-benchmark-source="fixture|live"가 붙습니다.landing-next-e2e는 job 전체,publish는 두 랜딩 빌드. main push에서는publish가 Pages artifact를 올리기 전에 이 변수 없이 singleCss 사이트를 다시 빌드하고, 배포될index.html이live인지 확인합니다. 로컬 PlaywrightwebServer도 이 변수를 설정합니다.e2e/landing-benchmark-fixture.spec.ts는 테스트 대상 빌드가 실시간 파일을 읽으면 실패합니다. source 표시와, 섹션에 fixture의 실행 id·시간·크기가 보이는지 확인합니다.e2e/screenshot.css,stylePath,data-benchmark)은 제거했습니다.section-bench,dark-section-bench,components-benchmark-section-mobile. 세 번의 재시도 모두 같은 이미지였고, 그 실행에서 fixture spec도 통과했습니다. 섹션이 들어간 나머지 baseline(full-page-*,dark-full-page-*,components-benchmark-section-desktop,dark-components-benchmark-section-desktop)은 1%maxDiffPixelRatio안에 들어서 CI가 actual 이미지를 만들지 않았으므로 그대로 두었습니다.소유자 결정 / 필요한 작업
*-win32.png는 CI에서 검사되지 않고, CI로 재생성할 수도 없습니다. Windows 머신에서bun run test:e2e:update로 만듭니다. 이 명령은 이제 PlaywrightwebServer를 통해 fixture로 랜딩을 빌드합니다. 그 전까지 로컬 Windows 실행에서는section-bench,dark-section-bench,components-benchmark-section-mobile에 차이가 납니다.mainruleset이 어떤 브랜치도 대상으로 하지 않아서 동작합니다. ruleset을main대상으로 고치면(publish,benchmark체크를 strict 정책으로 요구함) GitHub Actions나 GitHub App을 bypass에 추가해야 합니다. 그렇지 않으면 job이 rolling PR로 넘어갑니다. strict 필수 체크 ruleset에서는 병합 때마다 생기는 봇 커밋 때문에 열린 PR이 모두 out of date가 됩니다. 이를 받아들일지, 경로 기반 예외가 나을지 정해 주세요.후속 작업(이 PR 범위 밖)
next.config.ts와.mjs사이의.tsbuildinfo차이).docs/overview/page*.mdx)는 아직 과거 실행 33254030962를 표시합니다(과거 수치라고 표기됨). 같은 JSON을 읽게 할 수 있습니다.테스트 계획
bun test packages/plugin-utils/src/__tests__/benchmark-*.test.ts --coverage: 85개 통과.benchmark-gate.js,benchmark-output.js, 새benchmark-record.js모두 함수/라인 100%. CLI는 임시 git 저장소에서 처음부터 끝까지 검증했습니다: 새 커밋, 오래된 커밋, baseline이 없는 커밋, main에 없는 커밋,--trend, 오류 케이스.benchmark-output테스트가clientBytes를 다룹니다.bun render-benchmark-readme.js --check통과.formatJson출력이 체크인된benchmark-results.json,benchmark/history.json과 바이트 단위로 같습니다(테스트로 확인). 변경한 모든 파일에서bunx prettier --single-quote --no-semi --check통과.publish.yml은 이 변경 전부터 Prettier를 통과하지 못했습니다.bun run lint가 최종 확인입니다.Bench.tsx,DevupUICard.tsx,OtherCard.tsx가 JSON import를 포함해tsc(TS 7) 타입 검사 통과.publish.yml에 대한actionlint1.7.12 결과는 기존windows-11-vs2026-armrunner label 경고뿐입니다.chore(benchmark): record run …을 push하고, README의 클라이언트 사이즈 열이 채워지는지 확인.🤖 Generated with Claude Code