Repository navigation
fix(autoscaler): leave a NotReady node out of the memory check - #158
Merged
Merged
Conversation
`kubectl top nodes` prints `<unknown>` for a node whose kubelet has stopped reporting, so one NotReady node inside the eligibility selector tripped the fail-safe on every poll and held every runner set at the floor while jobs queued, although the node places no runner pod and the ceiling already ignores it. The poll now reads the nodes once and both the ceiling and the memory check take their idea of a Ready node from the same jq definition (`scripts/autoscaler-node.jq`). A Ready node whose figure cannot be parsed still fails safe, and so does a node list that cannot be read.
Mearman
marked this pull request as ready for review
November 1, 2026 18:57
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
🎉 This PR is included in version 2.12.2 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #157
kubectl top nodesprints<unknown>for a node whose kubelet has stopped reporting. A NotReady node inside the eligibility selector therefore tripped the fail-safe on every poll and held every runner set at the floor, although the node places no runner pod and the ceiling already leaves it out.The poll now reads the nodes once, and both the ceiling and the memory check decide which nodes are Ready from the same jq definition (
scripts/autoscaler-node.jq). A figure counts only for a Ready node. A Ready node whose figure cannot be parsed still fails safe, and so does a node list that cannot be read. The ceiling used to read the nodes itself, so a failed node list now fails safe instead of keeping the last ceiling.The unit tests cover a False and an Unknown Ready condition with and without a selector, a NotReady node reporting a stale figure, and a Ready node with an unreadable figure beside a NotReady one. Breaking either the skip or the shared definition makes them fail. A new image test checks that every jq module a program includes is copied into the autoscaler image. I did not run
tests/autoscaler_ceiling/run.sh(it needs kind and Docker); it covers the ceiling on one Ready node and is unaffected by the memory check.