[Mohit20057]

What is the purpose of the change

Sub-task of umbrella issue #8994.
Removes the deprecated legacy ZooKeeper backpressure cluster state methods in IStormClusterState and Config.TOPOLOGY_BACKPRESSURE_ENABLE (marked forRemoval = true since 2.0.0), and decouples dependent Nimbus topology cleanup, submission, and test helpers.

Upgrade notes

Removal of Config.TOPOLOGY_BACKPRESSURE_ENABLE

The public constant Config.TOPOLOGY_BACKPRESSURE_ENABLE ("topology.backpressure.enable"), originally marked for removal in 2.0.0, has been removed.

Worker backpressure in modern Storm versions is handled automatically, in-process, through internal worker queues and buffering. The legacy ZooKeeper-coordinated backpressure toggle is obsolete. Topologies that submit this key now get a log warning.

ZooKeeper /backpressure znodes are no longer cleaned up

The legacy cluster-state methods (backpressure(), setupBackpressure(), removeBackpressureFlag(), etc.) and the associated cleanup routine in Nimbus have been removed.

Notice for cluster operators: clusters upgraded from legacy 1.x installations, or with topologies that previously used ZooKeeper-based backpressure, will no longer have lingering /backpressure znodes purged automatically by Nimbus. Operators can safely delete any orphaned historical znodes directly in ZooKeeper with zkCli:

• rmr /storm/backpressure
• deleteall /storm/backpressure

How was this tested

• Built and compiled storm-client and storm-server
• Passed TestConfigValidate (38/38 tests passed)
• Passed NimbusClojurePortTest cleanup tests (5/5 tests passed)
• Passed NimbusClojurePortTest#testKillStorm (1/1 test passed)
• Checkstyle passed with 0 violations on both storm-client and storm-server

[reiabreu]

This review was generated with the help of an LLM (Claude) and reviewed by me before posting.

Two notes on the removals, inline.

[rzo1]

@Mohit20057 friendly ping: could you have a look at @reiabreu's review? The removal of the public constant needs a release/upgrade note (as asked in #8994), ideally also mentioning the /backpressure znodes that are no longer cleaned up. Thanks!

[Mohit20057]

Hi @rzo1 @reiabreu,

Thanks for the review! I have added a LOG.warn(...) in Nimbus when a submitted topology specifies topology.backpressure.enable. Here is the Release & Upgrade Note for this PR:

Release & Upgrade Note: Removal of Legacy ZooKeeper Backpressure State

1. Removal of Config.TOPOLOGY_BACKPRESSURE_ENABLE:

   • The public constant Config.TOPOLOGY_BACKPRESSURE_ENABLE ("topology.backpressure.enable"), originally marked for removal in 2.0.0, has been removed.
   • Worker backpressure in modern Storm versions is automatically handled in-process via internal worker queues/buffering; the legacy ZooKeeper-coordinated backpressure toggle is obsolete. Topologies submitting this key will now receive a log warning.

2. ZooKeeper /backpressure Znodes Cleanup Notice:

   • The legacy cluster-state methods (backpressure(), setupBackpressure(), removeBackpressureFlag(), etc.) and the associated cleanup routine in Nimbus have been removed.
   • Notice for Cluster Operators: Clusters upgraded from legacy 1.x installations (or topologies that previously used ZooKeeper-based backpressure) will no longer have lingering /backpressure znodes automatically purged by Nimbus. Operators can safely delete any orphaned historical znodes directly in ZooKeeper using the zkCli tool (rmr /storm/backpressure or deleteall /storm/backpressure).

[reiabreu]

Hi @rzo1 @reiabreu,

Thanks for the review! I have added a LOG.warn(...) in Nimbus when a submitted topology specifies topology.backpressure.enable. Here is the Release & Upgrade Note for this PR:

Release & Upgrade Note: Removal of Legacy ZooKeeper Backpressure State

1. Removal of Config.TOPOLOGY_BACKPRESSURE_ENABLE:

   • The public constant Config.TOPOLOGY_BACKPRESSURE_ENABLE ("topology.backpressure.enable"), originally marked for removal in 2.0.0, has been removed.
   • Worker backpressure in modern Storm versions is automatically handled in-process via internal worker queues/buffering; the legacy ZooKeeper-coordinated backpressure toggle is obsolete. Topologies submitting this key will now receive a log warning.

2. ZooKeeper /backpressure Znodes Cleanup Notice:

   • The legacy cluster-state methods (backpressure(), setupBackpressure(), removeBackpressureFlag(), etc.) and the associated cleanup routine in Nimbus have been removed.
   • Notice for Cluster Operators: Clusters upgraded from legacy 1.x installations (or topologies that previously used ZooKeeper-based backpressure) will no longer have lingering /backpressure znodes automatically purged by Nimbus. Operators can safely delete any orphaned historical znodes directly in ZooKeeper using the zkCli tool (rmr /storm/backpressure or deleteall /storm/backpressure).

thanks @Mohit20057 ! Do you mind resolving the conflicts?
A couple of issues come to mind:

• the umbrella issues mentions that API removal should be done in a major. Next release, however, will be 3.2.0. We do have a precedent of doing something similar and putting in master before a major release: Remove deprecated NIMBUS_SUPERVISOR_TIMEOUT_SECS and defaults entry (#8994) #9170. Will need some guidance here @rzo1
• Our current process of generating release notes still contains a good bit of manual steps by the person cutting the release, so this kind of detailed "release & upgrade notes" will get lost unless manually moved to the notes during the release, since it's only a comment in a PR @rzo1 @GGraziadei . Will try to think of something to improve our process

[reiabreu]

Hi @rzo1 @reiabreu,
Thanks for the review! I have added a LOG.warn(...) in Nimbus when a submitted topology specifies topology.backpressure.enable. Here is the Release & Upgrade Note for this PR:

Release & Upgrade Note: Removal of Legacy ZooKeeper Backpressure State

1. Removal of Config.TOPOLOGY_BACKPRESSURE_ENABLE:

   • The public constant Config.TOPOLOGY_BACKPRESSURE_ENABLE ("topology.backpressure.enable"), originally marked for removal in 2.0.0, has been removed.
   • Worker backpressure in modern Storm versions is automatically handled in-process via internal worker queues/buffering; the legacy ZooKeeper-coordinated backpressure toggle is obsolete. Topologies submitting this key will now receive a log warning.

2. ZooKeeper /backpressure Znodes Cleanup Notice:

   • The legacy cluster-state methods (backpressure(), setupBackpressure(), removeBackpressureFlag(), etc.) and the associated cleanup routine in Nimbus have been removed.
   • Notice for Cluster Operators: Clusters upgraded from legacy 1.x installations (or topologies that previously used ZooKeeper-based backpressure) will no longer have lingering /backpressure znodes automatically purged by Nimbus. Operators can safely delete any orphaned historical znodes directly in ZooKeeper using the zkCli tool (rmr /storm/backpressure or deleteall /storm/backpressure).

thanks @Mohit20057 ! Do you mind resolving the conflicts? A couple of issues come to mind:

• the umbrella issues mentions that API removal should be done in a major. Next release, however, will be 3.2.0. We do have a precedent of doing something similar and putting in master before a major release: Remove deprecated NIMBUS_SUPERVISOR_TIMEOUT_SECS and defaults entry (#8994) #9170
• Our current process of generating release notes still contains a good bit of manual steps by the person cutting the release, so this kind of detailed "release & upgrade notes" will get lost unless manually moved to the notes during the release, since it's only a comment in a PR @rzo1 @GGraziadei . Will try to think of something to improve our process

Asked Claude to quickly change the release notes script to capture "Upgrade notes" section from the PRs.
#9189

[Mohit20057]

@reiabreu Thanks! I have rebased the branch onto the latest origin/master, resolved the merge conflicts in Config.java and StormClusterStateImpl.java, and verified that mvn test-compile -pl storm-client,storm-server passes cleanly under JDK 23.

[rzo1]

Thanks for adding the upgrade notes and the warning for topology.backpressure.enable.

A few remarks on the notes:

1. They use ### sub-headings. With Print PR upgrade notes in the generated release notes #9189 as currently written, everything after the first heading is cut off, so this note would come out empty. Bullets or bold text instead of sub-headings would avoid that.
2. Some method names do not exist (backpressure(), removeBackpressureFlag()). The removed ones are backpressureTopologies, topologyBackpressure, setupBackpressure, removeBackpressure and removeWorkerBackpressure.
3. Nimbus still deletes dead-topology children under the backpressure subtree when ACL check/fixup is enabled (AclEnforcement), so "no longer cleaned up" is only true without it. Also, rmr is gone since ZK 3.5, deleteall is enough, and the path is <storm.zookeeper.root>/backpressure.

Nit: there is a double blank line in IStormClusterState.