Skip to content

[flink] Advance bucket baselines after successful Undo Recovery - #4955

Merged
platinumhamburg merged 1 commit into
apache:mainfrom
platinumhamburg:fix-undo-recovery-baseline
Nov 5, 2026
Merged

platinumhamburg merged 1 commit into
apache:mainfrom
platinumhamburg:fix-undo-recovery-baseline

Conversation

@platinumhamburg

Copy link
Copy Markdown
Contributor

Purpose

Closes #4952.

After successful Undo Recovery, an idle bucket keeps its original checkpoint baseline even though recovery has appended compensating writes. Subsequent checkpoints can therefore continue to depend on old changelog history and fail to restore after that history expires.

Brief change log

  • Return acknowledged log end offsets from Undo writes and retain the maximum per bucket after all writes complete.
  • Use the scanned recovery boundary when no compensating writes are required.
  • Persist these recovered baselines in subsequent checkpoints without requiring new sink records.

This does not extend the recoverability of older checkpoints beyond the available changelog history.

Tests

  • Reproduced the idle-bucket regression on main: the saved baseline remained 1 after Undo advanced the log end to 5.
  • Local clean build: 57 tests passed across UndoRecoveryOperatorStateTest, UndoRecoveryExecutorTest, UndoComputerTest, RecoveryOffsetManagerTest, UndoRecoveryOperatorFactoryTest, and UndoRecoveryManagerITCase.
  • Coverage includes a checkpoint and second restore without new sink writes, out-of-order write acknowledgements, and recovery ranges containing only empty batches.

API and Format

No changes to connector options or checkpoint serialization.

Documentation

No documentation changes required.

@platinumhamburg
platinumhamburg marked this pull request as ready for review October 30, 2026 08:00

@wuchong wuchong left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Please resolve the merge conflicts.

@platinumhamburg
platinumhamburg force-pushed the fix-undo-recovery-baseline branch from 34bec85 to 4cdadeb Compare November 4, 2026 14:23
@platinumhamburg
platinumhamburg merged commit 427988d into apache:main Nov 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flink] Advance idle bucket baselines after successful Undo Recovery

2 participants