Skip to content

CLDSRV-910: Rewrite the data of a self-copy when its encryption changes - #6346

Open
dvasilas wants to merge 6 commits into
development/9.3from
bugfix/CLDSRV-910-reencrypt-data-on-self-copy
Open

dvasilas wants to merge 6 commits into
development/9.3from
bugfix/CLDSRV-910-reencrypt-data-on-self-copy

Conversation

@dvasilas

Copy link
Copy Markdown
Contributor

A CopyObject of an object onto itself (same bucket and key, same location, versioning disabled) didn't rewrite the data, even when the copy changed the encryption.

This is the recommended way to encrypt an unencrypted object or to re-encrypt under another master key

  • Using it with an unencrypted object left the data in plaintext while the metadata and responses said it was encrypted.
  • Using it to re-encrypt recorded the new key in the metadata but kept the data key wrapped under the old one. So GETs decrypted with the wrong data key and returned wrong data.

With this PR, a self-copy rewrites the data when the destination is encrypted, and the source data is either unencrypted or encrypted under another master key. A self-copy under the same master key stays metadata-only.

The PR cherry-picks CLDSRV-908, so that a self-copy that rewrites the data deletes the old data.

Note: lib/kms/wrapper.js is reformatted with prettier in a separate commit.

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Hello dvasilas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval ⭐
/bypass_build_status Bypass the build and test status ⭐
/bypass_commit_size Bypass the check on the size of the changeset TBA ⭐
/bypass_incompatible_branch Bypass the check on the source branch prefix ⭐
/bypass_jira_check Bypass the Jira issue check ⭐
/bypass_peer_approval Bypass the pull request peers' approval ⭐
/bypass_leader_approval Bypass the pull request leaders' approval ⭐
/bypass_source_branch_lineage Bypass the cross-branch contamination check ⭐
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Incorrect fix version

The Fix Version/s in issue CLDSRV-910 contains:

  • 9.3.26

  • 9.4.11

  • 9.6.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 9.3.26

  • 9.4.11

  • 9.5.3

  • 9.6.0

Please check the Fix Version/s of CLDSRV-910, or the target
branch of this pull request.

Comment thread lib/kms/wrapper.js Dismissed
Comment thread lib/kms/wrapper.js Dismissed
Comment thread lib/kms/wrapper.js Dismissed
Comment thread lib/kms/wrapper.js Dismissed
Comment thread lib/kms/wrapper.js Dismissed
Comment thread lib/kms/wrapper.js Dismissed
@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@dvasilas

Copy link
Copy Markdown
Contributor Author

/create_integration_branches

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

The following options are set: create_integration_branches

@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.61905% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.39%. Comparing base (25e6830) to head (f454274).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
lib/kms/wrapper.js 65.15% 23 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/api/objectCopy.js 92.53% <100.00%> (+1.19%) ⬆️
lib/kms/wrapper.js 76.38% <65.15%> (+1.98%) ⬆️
@@                 Coverage Diff                 @@
##           development/9.3    #6346      +/-   ##
===================================================
+ Coverage            85.34%   85.39%   +0.05%     
===================================================
  Files                  207      207              
  Lines                13466    13485      +19     
===================================================
+ Hits                 11492    11516      +24     
+ Misses                1974     1969       -5     
Flag Coverage Δ
file-ft-tests 68.48% <69.04%> (+0.14%) ⬆️
file-ft-tests-null-compat 68.99% <69.04%> (+0.19%) ⬆️
kmip-ft-tests 28.37% <36.90%> (+0.02%) ⬆️
mongo-v0-ft-tests 69.69% <69.04%> (+0.19%) ⬆️
mongo-v1-ft-tests 69.62% <69.04%> (+0.14%) ⬆️
multiple-backend 36.84% <46.42%> (-0.03%) ⬇️
s3c-ft-tests-v0 64.04% <69.04%> (+0.04%) ⬆️
s3c-ft-tests-v0-null-compat 64.10% <69.04%> (+0.04%) ⬆️
s3c-ft-tests-v1 64.02% <69.04%> (+0.04%) ⬆️
sur-tests 36.68% <3.57%> (+0.73%) ⬆️
sur-tests-inflights 37.72% <3.57%> (-0.07%) ⬇️
unit 71.77% <72.61%> (+0.18%) ⬆️
utapi-v2-tests 34.65% <15.47%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread lib/api/objectCopy.js
Only when the destination is encrypted and the data is unencrypted or under another master key; a same-key self-copy stays metadata-only.
@dvasilas
dvasilas force-pushed the bugfix/CLDSRV-910-reencrypt-data-on-self-copy branch from 19797ca to f454274 Compare October 28, 2026 15:17
@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

History mismatch

Merge commit #b754602cdec1f8112d03bb5a3d69865f654808ce on the integration branch
w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy is merging a branch which is neither the current
branch bugfix/CLDSRV-910-reencrypt-data-on-self-copy nor the development branch
development/9.4.

It is likely due to a rebase of the branch bugfix/CLDSRV-910-reencrypt-data-on-self-copy and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

The following options are set: create_integration_branches

@dvasilas

Copy link
Copy Markdown
Contributor Author

/force_reset

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Reset complete

I have successfully deleted this pull request's integration branches.

The following options are set: create_integration_branches

@scality scality deleted a comment from bert-e Oct 28, 2026
@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Conflict

A conflict has been raised during the creation of
integration branch w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy with contents from bugfix/CLDSRV-910-reencrypt-data-on-self-copy
and development/9.4.

I have not created the integration branch.

Here are the steps to resolve this conflict:

 git fetch
 git checkout -B w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy origin/development/9.4
 git merge origin/bugfix/CLDSRV-910-reencrypt-data-on-self-copy
 # <intense conflict resolution>
 git commit
 git push -u origin w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy

The following options are set: create_integration_branches

@scality scality deleted a comment from bert-e Oct 28, 2026
@scality scality deleted a comment from bert-e Oct 28, 2026
@scality scality deleted a comment from bert-e Oct 28, 2026
@scality scality deleted a comment from bert-e Oct 28, 2026
@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Conflict

A conflict has been raised during the creation of
integration branch w/9.5/bugfix/CLDSRV-910-reencrypt-data-on-self-copy with contents from w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy
and development/9.5.

I have not created the integration branch.

Here are the steps to resolve this conflict:

 git fetch
 git checkout -B w/9.5/bugfix/CLDSRV-910-reencrypt-data-on-self-copy origin/development/9.5
 git merge origin/w/9.4/bugfix/CLDSRV-910-reencrypt-data-on-self-copy
 # <intense conflict resolution>
 git commit
 git push -u origin w/9.5/bugfix/CLDSRV-910-reencrypt-data-on-self-copy

The following options are set: create_integration_branches

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.8
  • development/9.0
  • development/9.1
  • development/9.2

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

@bert-e

bert-e commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

The following options are set: create_integration_branches

@dvasilas
dvasilas requested a review from leif-scality October 30, 2026 08:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants