Skip to content

ci(build): Cache MSVC and VC6 compilation with ccache - #3608

Open
CryoTheRenegade wants to merge 9 commits into
TheSuperHackers:mainfrom
CryoTheRenegade:ci/cache-msvc-compilation
Open

CryoTheRenegade wants to merge 9 commits into
TheSuperHackers:mainfrom
CryoTheRenegade:ci/cache-msvc-compilation

Conversation

@CryoTheRenegade

@CryoTheRenegade CryoTheRenegade commented Oct 26, 2026 •

Copy link
Copy Markdown

Every push to a PR recompiles every file in CI. With an empty cache, the Zero Hour Release build step takes about 18 minutes, so each round of review fixes waits that long for results. This PR caches compiled objects so unchanged files are reused across pushes and commits.

Cache C and C++ compilation in the modern MSVC and VC6 CI jobs with ccache 4.14.2, which the workflow checksums on download. Each cache key includes the game, preset and compiler version. Restore prefixes let a new commit reuse objects from earlier commits.

 win32* and vc6 CI jobs (not vc6-weekly)
+ restore the compiler cache, capped at 500 MB per configuration
+ save it only outside pull requests, so caches update after merging to main
+ compile through ccache with PCH disabled
+ print per-build cache statistics, including after failures

ccache does not cache MSVC files that use a precompiled header unless its time-macro checks are relaxed. The cached CI jobs disable PCH instead, which keeps those checks and the existing source encoding. Path.cpp, PathfindLayer.cpp and W3DProfilerFrameCapture.cpp gain explicit includes for headers that PCH used to supply. Weekly VC6 releases compile with /Zi, which ccache cannot cache, so they keep their current setup. Local presets also stay unchanged.

Evidence

  • All 20 build jobs passed in the corrected run and the second run on hosted Windows runners. In the second run, all 12 modern configurations hit the cache 99.76 to 99.77% of the time. Build steps took 39 to 101 seconds and whole jobs took 103 to 272 seconds.
  • Five configurations restored a cache saved by an earlier commit, with 99.5% hits. Zero Hour Release's build step dropped from 18m 07s with an empty cache to 1m 03s after the include changes. That job took 2m 23s in total, including setup and cache transfer.
  • With a seeded cache, applying open PR #3605 missed only 7 of about 1,700 compilations per game. Compared with upstream's run of that PR on the same win32+t+e jobs, Generals' build step dropped from 6m 21s to 1m 03s and Zero Hour's from 4m 36s to 1m 03s. Whole jobs dropped from 7m 26s to 2m 41s and from 5m 30s to 2m 34s. Timings on hosted runners vary between runs.
  • All 8 VC6 builds passed with an empty cache and again with a restored cache. With the restored cache, hits were 99.7%, build steps took 20 to 34 seconds instead of 66 to 117 seconds, and jobs took 63 to 87 seconds. Each VC6 cache is about 35 MB.
  • In a run after splitting restore and save, all 20 build jobs restored the previous commit's cache, hit 99.7%, and saved a new cache.
  • Every configuration fit the 500 MB limit, and the largest cache was about 465 MB.
  • Full local Debug and Tracy Profile builds passed for both games and their tools.
  • Sampled Release and Debug objects matched their cached copies byte for byte. C and C++ smoke tests covered legacy source encoding and cache invalidation after a header change. The setup step rejected an archive with a wrong checksum. Actionlint, PowerShell parsing, preset gating and diff checks passed.

CryoTheRenegade and others added 4 commits October 26, 2026 10:16
Extend ccache to the VC6 CI presets. Weekly releases keep compiling
fresh because vc6-weekly uses /Zi, which ccache cannot cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pull requests restore the newest cache from main but no longer save
their own, so caches change only after changes are merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-26T20:19:42.555483Z
We couldn’t complete this review. Comment @codex review to try again.
06976ff PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The build workflows add conditional ccache setup, compiler-launcher configuration, statistics reporting, and cache management. The changes also grant cache-management permissions and update includes in two pathfinder source files and the profiler frame-capture source.

Changes

Conditional ccache workflow

Layer / File(s) Summary
Select presets and set up cache
.github/workflows/reusable-build-toolchain.yml, .github/workflows/ci.yml
The workflow enables ccache for win32 presets and vc6 presets other than vc6-weekly. It downloads and verifies ccache, configures it, restores a configuration-specific cache, and grants actions: write to the four build jobs.
Configure ccache compilation
.github/workflows/reusable-build-toolchain.yml
When ccache setup succeeds, CMake uses ccache for C and C++ compilation and disables precompiled headers. The workflow resets ccache statistics before the build.
Report and manage cache data
.github/workflows/reusable-build-toolchain.yml
The workflow reports ccache statistics and saves eligible caches. It deletes superseded caches only after confirming that the new cache exists.

Pathfinder include updates

Layer / File(s) Summary
Add pathfinder includes
Core/GameEngine/Source/GameLogic/AI/Pathfinder/Path.cpp, Core/GameEngine/Source/GameLogic/AI/Pathfinder/PathfindLayer.cpp
Both source files now include PreRTS.h before their existing headers.

Profiler include updates

Layer / File(s) Summary
Update profiler frame-capture includes
Core/GameEngineDevice/Source/W3DDevice/GameClient/W3DProfilerFrameCapture.cpp
The source adds profiler and Windows headers and uses the GameClient include path for the frame-capture header. The PROFILER_ENABLED guard remains.

Priority: Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant Workflow as GitHub Actions workflow
  participant ActionsCache as Actions cache
  participant CMake
  participant ccache
  Workflow->>ActionsCache: Restore configuration-specific cache
  CMake->>ccache: Use compiler launcher
  Workflow->>ccache: Report statistics
  Workflow->>ActionsCache: Save eligible cache
Loading

Possibly related PRs

Suggested reviewers: xezon, tintinhamans, skyaero42

Merge Risk: Low · previous review cbfdf

The CI caching change should work as described, but it grants Actions write permission to pull-request build jobs that do not need it. The exposure depends on a repository setting. Confirm that setting, or move cache cleanup to a push-only job, to keep pull-request builds read-only.

Pre-merge checks: Passed 4
Check name Status Explanation
Description check Passed The description clearly explains the ccache changes, affected CI jobs, configuration, exclusions, validation results, and intended performance improvement.
Title check Passed The title clearly and concisely identifies the main change: caching MSVC and VC6 compilation with ccache in CI.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The changes appear safe to merge.

Summary

Adds compiler caching for modern MSVC and non-weekly VC6 builds.

  • Supported Windows builds reuse compiled objects across commits.
  • CI removes older compiler caches when a newer one is ready.

No actionable issues were found in the changes since the previous review. No previous review threads were supplied.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Prepare ccache] --> B[Restore matching compiler cache]
  B --> C[Build without precompiled headers]
  C --> D[Print cache statistics]
  D --> E{Pull request?}
  E -->|Yes| F[Keep restored caches unchanged]
  E -->|No| G[Save updated cache]
  G --> H[Wait for build jobs]
  H --> I[Delete older caches with a current replacement]
Loading

Reviews (4) · Last reviewed commit: "ci(build): Delete superseded compiler ca..." · Reviewed by Greptile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@xezon

xezon commented Oct 27, 2026

Copy link
Copy Markdown

Every configuration fit the 500 MB limit, and the largest cache was about 465 MB.

What happens if that limit is reached?

Comment thread .github/workflows/reusable-build-toolchain.yml
Comment thread Core/GameEngine/Source/GameLogic/AI/Pathfinder/PathfindLayer.cpp Outdated
Comment thread Core/GameEngine/Source/GameLogic/AI/Pathfinder/Path.cpp Outdated
CryoTheRenegade and others added 2 commits October 27, 2026 09:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CryoTheRenegade

Copy link
Copy Markdown
Author

Every configuration fit the 500 MB limit, and the largest cache was about 465 MB.

What happens if that limit is reached?

ccache drops the least recently used entries once it passes 500 MB. The build keeps working, it just misses on older objects, and the saved cache never grows past 500 MB per configuration.

@Skyaero42

Copy link
Copy Markdown

How do we know this is not going to come back at us later? How can we be absolutely sure this works and keeps working? There has been a lot of tinkering on the CI pipeline lately and not always for the better.

@CryoTheRenegade

CryoTheRenegade commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

How do we know this is not going to come back at us later? How can we be absolutely sure this works and keeps working? There has been a lot of tinkering on the CI pipeline lately and not always for the better.

Honestly if this breaks, we have bigger issues. The main benefit is to speed up ci times by not having to recompile every file every time a pr is made. I agree that CI work has been a pain (or at least from what i have seen) and the hope is that this makes iterative changes to prs not take a million years to finish. The biggest issue i can see is if github has a mega stroke(which is often unfortunately) and doesnt let us download our toolkits, then all we have to do is turn the USE_CCACHE flag from 1 to 0 and it goes back to the way it was.


EDIT: Changed the build steps to continue on ccache download failure, it will just do a normal build.

@xezon xezon added Major Severity: Minor < Major < Critical < Blocker Performance Is a performance concern Build Anything related to building, compiling labels Oct 27, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d6ad605a-3a92-482b-bae8-0dd27f4abb5b
Commits

Reviewing files that changed from the base of the PR and between 1ddf5f1 and cbfdf9f.

Files selected for processing: 4
  • .github/workflows/ci.yml
  • .github/workflows/reusable-build-toolchain.yml
  • Core/GameEngine/Source/GameLogic/AI/Pathfinder/Path.cpp
  • Core/GameEngine/Source/GameLogic/AI/Pathfinder/PathfindLayer.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread .github/workflows/ci.yml Outdated
Pull request build jobs no longer need actions: write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build Anything related to building, compiling Major Severity: Minor < Major < Critical < Blocker Performance Is a performance concern

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants