Skip to content

Antalya 26.8: port the CAS PRs merged into antalya-26.6 after the PR 2540 cut - #2805

Open
filimonov wants to merge 22 commits into
antalya-26.8from
port/antalya-26.8/cas-from-26.6
Open

Antalya 26.8: port the CAS PRs merged into antalya-26.6 after the PR 2540 cut#2805
filimonov wants to merge 22 commits into
antalya-26.8from
port/antalya-26.8/cas-from-26.6

Conversation

@filimonov

@filimonov filimonov commented Nov 6, 2026

Copy link
Copy Markdown
Member

Port to antalya-26.8 of the eleven CAS PRs that landed on antalya-26.6 after the cut of the combined port #2540 (26.6 after PR 2437 plus 2509-2511). One commit per source PR, cherry-picked from the PR merge commit; conflict notes are in the commit messages. Four follow-up commits adapt the picks to 26.8 (ProfileEvents counter reads without .load(), a missing include, the order of the mount-lease sections in docs/en/antalya/cas/configuration.md).

Ported, in merge order (source category in brackets):

Resolved by hand against the 26.8-only code:

  • 2555: copyS3File.cpp routes both entry points through copyS3FileImpl on 26.8; the object-creation client is threaded through it, copyS3FileRange passes none. WriteBufferFromS3.cpp keeps the 26.8 transient-error branch. CasPartWriteTxn.cpp uses Cas::nowMs.
  • 2567: CasOrphanManifestSweep.cpp and CasDecommission.cpp against Fix early manifest deletion in CAS GC #2730 (BuildFloor sentinel and the renamed eligibility helper kept).
  • 2516: configuration.md lease paragraphs next to the Packed section; the new gtest_ca_wiring.cpp test after the owner-Move tests.

Not ported on purpose: the no-cas-storage tags of the EXPORT PART stateless tests, which 26.8 removed in 5a1ffb2 ("Allow every partition command, EXPORT PART included, on a CAS disk").

Local check: release unit_tests_dbms --gtest_filter='CAS*' 2904 passed, 0 failed.

Related: #2644
Related: #2540

Changelog category (leave one):

  • Critical Bug Fix (crash, data loss, RBAC)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

CAS (content-addressed storage): port the antalya-26.6 fixes and improvements merged after the combined port: a blob is adopted only when its marker reads Clean; an object-creating S3 request is never resent; a mount slot is claimed only with a strictly greater epoch and after the holder's declared lease; lease renewal continues until a definitive answer; renewal progress is shown in system.cas_mounts; nested server_root_id values are refused; decommission keeps a member's control objects and a decommissioned server root rejoins on start; custom disks are created outside storage_policies_mutex.

Documentation entry for user-facing changes

  • Documentation written: docs/en/antalya/cas/ carried over from the source PRs.

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

filimonov and others added 15 commits November 6, 2026 03:02
@filimonov @claude
Accept only a strictly greater epoch when claiming a CAS mount slot
bb13294 
Port of #2512 (antalya-26.6) to antalya-26.8.

(cherry picked from commit 2a034cd)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Adopt a CAS blob only when its marker reads Clean
ba448a8 
Port of #2552 (antalya-26.6) to antalya-26.8.

Conflict: `src/Disks/tests/gtest_cas_part_write.cpp`, counter reads without `.load()` as on antalya-26.8.

(cherry picked from commit fef96d5)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Show CAS mount renewal progress in system.cas_mounts
f4dc9e4 
Port of #2513 (antalya-26.6) to antalya-26.8.

(cherry picked from commit c147de3)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Wait for the lease a CAS mount holder declares before fencing or recl…
d361b97 
…aiming its slot

Port of #2516 (antalya-26.6) to antalya-26.8.

Conflicts: `docs/en/antalya/cas/configuration.md` (the lease-liveness paragraph replaces the old
"All servers sharing a pool" one and sits before the antalya-26.8 `Packed` section),
`src/Disks/tests/gtest_ca_wiring.cpp` (new test appended after the owner-Move tests).

(cherry picked from commit 21b7170)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Read ProfileEvents counters without .load() in the ported CAS part-wr…
78de57b 
…ite test

On antalya-26.8 `ProfileEvents::Counters::operator[]` returns the value, so the two reads
that came with the port of #2552 did not compile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Never resend a CAS blob's object-creating S3 request
a872187 
Port of #2555 (antalya-26.6) to antalya-26.8.

Conflicts: `src/IO/S3/copyS3File.cpp` (antalya-26.8 routes `copyS3File` and `copyS3FileRange` through
`copyS3FileImpl`; the object-creation client is threaded through it, the ranged copy passes none),
`src/IO/WriteBufferFromS3.cpp` (the 26.8 transient-error branch is kept and says "will retry" only
when a retry follows), `Pool/CasPartWriteTxn.cpp` (no local `nowMs`, `CasClock.h` has it),
`docs/en/antalya/cas/configuration.md` (settings table rows).

(cherry picked from commit 8fefc27)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Renew the CAS mount lease until a definitive answer; an expired lease…
eec06c4 
… is a pause

Port of #2517 (antalya-26.6) to antalya-26.8.

(cherry picked from commit 9e54034)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
CAS: refuse new nested server_root_id, delete S3 staging on abort, ac…
43502bc 
…tionable member refusals

Port of #2566 (antalya-26.6) to antalya-26.8.

(cherry picked from commit af662dc)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
CAS: decommission keeps a member's control objects; owner life and ep…
0554ec7 
…och nonce

Port of #2567 (antalya-26.6) to antalya-26.8.

Conflicts with the PR 2730 port already on antalya-26.8: `Gc/CasOrphanManifestSweep.cpp` (keeps the
`BuildFloor` sentinel test and the renamed eligibility helper; `sweepNamespace` admits through
`mountRequests` as this PR does), `Tools/CasDecommission.cpp` (PR 2730 comment kept; the old slot-removal
tail goes as in this PR).

(cherry picked from commit 9b348cf)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Log the first failed CAS blob publication attempt below Warning
a330030 
Port of #2594 (antalya-26.6) to antalya-26.8.

(cherry picked from commit 6a1085e)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Rejoin a decommissioned CAS server root on server start
1f16a37 
Port of #2574 (antalya-26.6) to antalya-26.8.

(cherry picked from commit 11ab9a4)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Create custom disks outside storage_policies_mutex
db92143 
Port of #2573 (antalya-26.6) to antalya-26.8.

(cherry picked from commit 52c95e2)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Keep the mount-lease paragraphs together in the CAS configuration doc
4e5fc3c 
The `Packed` parts subsection sat between the settings table and the lease-liveness prose; it now
follows that prose, so the three lease paragraphs read in the antalya-26.6 order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Read ProfileEvents counters without .load() in the remaining ported C…
c798655 
…AS tests

Same cause as 78de57b: on antalya-26.8 `ProfileEvents::Counters::operator[]` returns the value.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Include WriteBufferFromFileBase.h in the ported S3 object-creation test
b8dfb39 
On antalya-26.8 no other header of the test pulls it in transitively.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@github-actions

github-actions Bot commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown

Workflow [PR], commit [ad04f95]

@filimonov filimonov mentioned this pull request Nov 6, 2026
Open
11 of 31 tasks
k-morozov
k-morozov previously approved these changes Nov 6, 2026
View reviewed changes
@filimonov @claude
Call copyS3File without a source offset in the ported object-creation…
a2147e5 
… death test

On antalya-26.8 the ranged copy is `copyS3FileRange`; `copyS3File` has no
`src_offset` parameter. The call sits under `DEBUG_OR_SANITIZER_BUILD`,
which the release build does not compile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
k-morozov
k-morozov previously approved these changes Nov 6, 2026
View reviewed changes
@filimonov @claude
Serialize IDisk::checkAccess: disks created in parallel share the pro…
086efc9 
…be file

Every disk probes its access with the same file name, clickhouse_access_check_<server uuid>.
Since custom disks are created outside storage_policies_mutex
(#2573), two tables whose custom disks sit on one
backing disk are started in parallel at server start, and their probes overwrite and unlink
each other's file: the tables fail to attach with FILE_DOESNT_EXIST "While checking access
for disk ...". Seen in test_filesystem_cache on
#2805 (tables test and test_size_in_name, cache
disks over hdd_blob). A process-wide mutex around the probe; config disks are created one
after another anyway.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov filimonov added the CAS label Nov 6, 2026
@filimonov filimonov mentioned this pull request Nov 6, 2026
Open
11 of 31 tasks
filimonov and others added 2 commits November 6, 2026 18:03
@filimonov
Merge remote-tracking branch 'altinity/antalya-26.8' into port/antaly…
0a4515c 
…a-26.8/cas-from-26.6
@filimonov @claude
Tag 03800 and 03927 no-random-settings, as antalya-26.6 does
8c26e87 
Under randomized settings with `enable_filesystem_cache=0` these stateful tests read
`test.hits` from the web disk in one HTTP stream at about 1.5 MB/s: 150-175 s per query,
03927 killed at 1200 s. antalya-26.6 carries the tag since
#2013; antalya-26.8 does not.

CI: https://altinity-build-artifacts.s3.amazonaws.com/PRs/2805/a2147e556977e195b2d117c61b61d1274e080436/stateless_tests_amd_binary_cas_s3_storage_parallel/job.log
PR: #2805

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov filimonov added antalya antalya-26.8 Session label (releasy session config) labels Nov 6, 2026
@filimonov @claude
Say whether a failed CAS ref-log commit can still take effect
8f5bc11 
#2577, merged into antalya-26.6 as ae6e749.
Conflict in src/Disks/tests/gtest_cas_ref_wedge_every_attempt.cpp: keep the 26.8 ProfileEvents
counter reads without .load(), take the new expectThrowsCodeWithMessage assertions.

(cherry picked from commit ae6e749)

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
filimonov and others added 2 commits November 6, 2026 23:13
@filimonov @claude
Derive the merged part name in 05387_cas_publish_counts_replicated
4810e68 
The test counted `build_publish` and `ref_move` events for the literal `all_0_1_1`.
When the second insert's block number is skipped (a retried insert), the merge is
`all_0_2_1` and the counts read 0 although the merge was published once under its final name.
Take the active level>0 part name from system.parts instead.

CI: https://altinity-build-artifacts.s3.amazonaws.com/PRs/2805/8c26e8792eb4d675d6cfa59b06f8e31b38471008/result_pr.json
(Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2), part_log: all_0_0_0, all_2_2_0, merge all_0_2_1)
PR: #2805

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
@filimonov @claude
Skip the unknown-config check on the upstream image in test_custom_s3…
ad04f95 
…_disk_remote_url_allow_hosts

Both instances start from clickhouse/clickhouse-server:26.8.26.12 with the fork's config.xml,
and since ClickHouse#100332 an upstream server refuses it:
`Unknown elements 'cas_log', 'cas_gc_log' found in config /etc/clickhouse-server/config.xml
(UNKNOWN_ELEMENT_IN_CONFIG)`. Set skip_check_for_incorrect_settings for these two instances only.

CI: https://altinity-build-artifacts.s3.amazonaws.com/PRs/2805/8c26e8792eb4d675d6cfa59b06f8e31b38471008/integration_tests_amd_asan_ubsan_targeted/logs.tar.gz
PR: #2805

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@k-morozov k-morozov k-morozov left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

antalya antalya-26.8 Session label (releasy session config) CAS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@filimonov @k-morozov