Skip to content

9.0 (fix): Track the asynchronous failure Tomcat reports three times, and note what Spring Framework 7.1.1 makes stale - #17101

Open
codeconsole wants to merge 3 commits into
apache:9.0.xfrom
codeconsole:fix/async-observation-followups-9.0.x
Open

codeconsole wants to merge 3 commits into
apache:9.0.xfrom
codeconsole:fix/async-observation-followups-9.0.x

Conversation

@codeconsole

Copy link
Copy Markdown
Contributor

Follow-up to #17088.

The Tomcat case that is still reported three times

On Tomcat, an asynchronous dispatch can fail and then be answered by an error controller that fails as well, with a server error that Spring MVC resolves (a 503 ResponseStatusException). That failure is still reported three times, and its observation stops twice. Tomcat tells the listeners of any asynchronous dispatch that ends in sendError that the processing completed, then that it failed, then that it completed again, with or without Grails. That is Tomcat bug 70362, with a fix in apache/tomcat#1264. Grails can't add a listener during that dispatch to correct it.

The case is now a row of FailedActionObservationSpec and its two copies:

  • On Jetty and Undertow, it is a row of the "fails with a server error Spring MVC resolves" feature, and it passes.
  • On Tomcat, it is a separate feature with @PendingFeature(reason = 'https://bz.apache.org/bugzilla/show_bug.cgi?id=70362'). A Tomcat upgrade that fixes it turns the feature into a failure that says so.

Notes in ServerErrorObservation for the Spring Framework 7.1.1 bump

spring-projects/spring-framework@088fb71f (7.1.1, for spring-projects/spring-framework#37731) changes ObservationAsyncListener.onComplete to read only what recordHandledException recorded, not jakarta.servlet.error.exception. Each case is still reported once with it, but two things go stale. Both now point at #37731:

  • request.removeAttribute(RequestDispatcher.ERROR_EXCEPTION) in FirstFailureListener.onComplete is only needed on 7.1.0.
  • The Javadoc of recordServerError(request, failure, errorHandler) says the filter reads the failure off the request once the exchange completes. From 7.1.1 that no longer holds for a request that started asynchronous processing.

The comment on recordHandledException(request, null) now says the call steps outside Spring's contract. org.springframework.web.filter is @NullMarked, and error isn't @Nullable. Spring has no public way to clear the attribute, so the call stays. ServerErrorObservationSpec fails if Spring starts rejecting null.

The class Javadoc line that ran to 168 characters is rewrapped at 120.

The Undertow completion fix from #17088 goes to 8.0.x separately, in #17100.

…ror controller fails

An asynchronous dispatch that fails, answered by an error controller that fails with a server
error Spring MVC resolves, is reported once on Jetty and Undertow, and three times on Tomcat.
Tomcat tells the listeners of an asynchronous dispatch that ends in an error it sent that the
processing completed, then that it failed, then that it completed again (Tomcat bug 70362), and
Grails cannot add a listener during that dispatch to correct it. The case is a row of the Jetty
and Undertow specs, and a pending feature on Tomcat, so the Tomcat upgrade that fixes it says so.
From 7.1.1 (spring-projects/spring-framework#37731), the observation filter reads only what was
recorded as handled once asynchronous processing completes, so removing the container's error
attribute there is only needed on 7.1.0, and the Javadoc that says the filter reads the failure
off the request no longer holds for an asynchronous request. Also say that clearing the handled
failure with null steps outside Spring's contract, and wrap the class Javadoc at 120 characters.
@codecov

codecov Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.6431%. Comparing base (733397b) to head (0d700e2).

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##                9.0.x     #17101   +/-   ##
=============================================
  Coverage     65.6431%   65.6431%           
  Complexity      33022      33022           
=============================================
  Files            2446       2446           
  Lines          121408     121408           
  Branches        22524      22524           
=============================================
  Hits            79696      79696           
+ Misses          31647      31642    -5     
- Partials        10065      10070    +5     
Files with missing lines Coverage Δ
...rg/grails/web/errors/ServerErrorObservation.groovy 81.2500% <ø> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. One asynchronous failure is still reported three times.

matrei
matrei previously approved these changes Nov 21, 2026

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 5c80974 (base 9.0.x). It merges cleanly with today's 9.0.x (733397b). Each change is what the description says, and 9.0.x is the right branch: ServerErrorObservation and these rows exist only there.

What I ran:

  • Unit tests of grails-web-common at the head: 434 tests, all pass.

  • FailedActionObservationSpec, JettyFailedActionObservationSpec and UndertowFailedActionObservationSpec at the head:

    • Results: 31 tests on Tomcat (one skipped as pending), 27 on Jetty and 27 on Undertow.
    • The new row passes on Jetty and Undertow.
    • On Tomcat, the pending feature fails on the NOTIFIED assertion (FailedActionObservationSpec.groovy:146), as intended. The 503 and the STOPPED assertions before it pass.
  • Two stand-ins for a Spring Framework change to recordHandledException(request, null) (ServerErrorObservation.groovy:147), to check the new comment's claim:

    • When the call throws, as Spring rejecting null would make it, three rows of ServerErrorObservationSpec fail: those where the container reports the failure.
    • When the call is dropped, as Spring ignoring null would amount to, the row "…when the container reports it and a failure Spring MVC records as handled follows" fails.

    So the claim holds either way.

On James's point

James asks to report the failure once rather than leave the test pending. That isn't something Grails can do here, and the pending feature is what shows the gap:

  • The three notifications and the second stop come from Tomcat calling the listeners of the processing three times: onComplete, onError, onComplete. That's bz 70362.
  • On apache/tomcat#1264, a Tomcat committer reproduced it on Tomcat's main branch with the plain servlet from the bug, [onComplete, onError(null), onComplete], and got [onComplete] with that PR.
  • Spring's listener stops the observation on every onComplete it hears. During that asynchronous dispatch, Grails can't add a listener, so it can't take any of the extra calls back.
  • The feature runs on every build and fails as pending. Once a Tomcat release with the fix arrives, it fails the build instead, and the annotation comes off.

I'd keep it as it is.

Notes

  1. If #17100 takes the Javadoc fix for AsyncErrorCompletionHandler (Tomcat doesn't complete the processing, it leaves it to the asynchronous timeout; only Jetty completes it), I'd make the same edit here. The 9.0.x copy says the same, and identical additions on both branches leave the merge-up of #17100 nothing to resolve in that file.

  2. Nit: in the Javadoc of the new feature (FailedActionObservationSpec.groovy:127-129), "an error it sent" reads as if Tomcat sent the error, and "with or without Grails" ends up describing the observation:

         * Tomcat tells the listeners of an asynchronous dispatch that ends in {@code sendError} that the processing
         * completed, then that it failed, then that it completed again, with or without Grails. So the observation reports
         * the failure three times and stops twice, and Grails can't add a listener during that dispatch to correct it.
    

Verified as correct

  • The two #37731 comments say what goes with Spring Framework 7.1.1 and what stays:

    • The removal of jakarta.servlet.error.exception (ServerErrorObservation.groovy:141-143) is only needed while completion reads that attribute, which it does on 7.1.0 only.
    • The recordHandledException(request, null) call stays after the bump: 7.1.1 still reads the handled attribute once the processing completes.
  • The recordServerError Javadoc (:71-72) now says that, from 7.1.1, the filter stops reading the failure off the request once asynchronous processing completes.

  • The rewrapped FirstFailureListener Javadoc fits in 120 columns. The new feature's labels are single-quoted, and its name matches the unrolled name of the Jetty and Undertow row.

  • The #17088 description now names the Tomcat case under "Limitation".

  • CI fails only where nothing in this PR reaches:

    • The Groovy Snapshot Canary fails as it did on #17088.
    • Build Grails Forge (Java 25, indy=false) failed resolving dependencies (Could not find groovy-bom-4.0.33.pom from repo.grails.org).
    • Hibernate5 Functional Tests (Java 21, indy=false) was cancelled.

    A rerun of the last two should do.

… who reports it

The Javadoc read as if Tomcat sent the error, and put "with or without Grails" on the
observation rather than on what Tomcat tells the listeners.
@codeconsole

Copy link
Copy Markdown
Contributor Author

@matrei Thanks.

@github-actions
github-actions Bot dismissed matrei’s stale review November 21, 2026 20:19

Dismissed automatically: the pull request changed after this approval, from 5c80974 to 0d700e2.

@testlens-app

testlens-app Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🚨 All tests passed but jobs failed 🚨

Failed Jobs without Test Failures

❌ Groovy Snapshot Canary Build / Build Grails (shard 0)
❌ Groovy Snapshot Canary Build / Build Grails (shard 1)

🏷️ Commit: 0d700e2
▶️ Tests: 172682 executed
⚪️ Checks: 100/100 completed

Rerun Controls

Click the checkbox to trigger a rerun:

  • Rerun jobs

Learn more about TestLens at testlens.app/docs.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The items this review found have been addressed. The repeated reports come from the server and Spring, not from this change, and the pending test records them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants