Repository navigation
8.0 (fix): Complete asynchronous processing on Undertow once an error its request dispatch sent is answered - #17100
Conversation
…t dispatch sent is answered A request dispatch that started asynchronous processing and then sent an error, as Spring MVC does for a server error it resolves, left that processing open forever on Undertow. Undertow sends the error, through the error page that maps to it, once the dispatch has returned, and ends the exchange with it. The processing then never completed and never timed out, so its listeners were never told: the observation of the request never stopped, and a DeferredResult was never resolved. A handler now completes the processing once the exchange has ended, as Tomcat and Jetty do.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 8.0.x #17100 +/- ##
==================================================
+ Coverage 62.5742% 62.5842% +0.0100%
- Complexity 27543 27552 +9
==================================================
Files 2249 2250 +1
Lines 111725 111747 +22
Branches 20486 20490 +4
==================================================
+ Hits 69911 69936 +25
+ Misses 32455 32448 -7
- Partials 9359 9363 +4
🚀 New features to boost your workflow:
|
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. Completion runs before the error response is sent.
| } | ||
| this.next.handleRequest(exchange); | ||
| AsyncContextImpl asyncContext = context.getOriginalRequest().getAsyncContextInternal(); | ||
| if (context.getErrorCode() > 0 && asyncContext != null && !asyncContext.isCompletedBeforeInitialRequestDone()) { |
There was a problem hiding this comment.
A request starts asynchronous processing, sends an error, and then completes. Completion runs before the error response is sent, because this skips the listener once completion has already been asked for. Run completion after the error response.
There was a problem hiding this comment.
Fixed in 41589e7. The order came from Undertow, not from this check: ServletInitialHandler.handleFirstRequest tells the listeners as soon as the handler chain returns, and only then sends the error. Without the check, the listeners hear onComplete twice. When the dispatch has completed the processing itself, the handler now sends the error through doErrorDispatch before it returns, then clears the error code so Undertow doesn't send it again. onComplete now comes after the error page has answered and the response has ended. New test: asynchronousProcessingThatADispatchCompletesAfterSendingAnErrorCompletesOnceTheErrorPageHasAnsweredIt. It got ["onComplete", "errorPage"] before the fix.
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 3c13e38 (base 8.0.x). It's on today's 8.0.x head (96a4ea4), so it merges cleanly.
AsyncErrorCompletionHandler is byte-identical to the copy #17088 put on 9.0.x, and so are the factory line and the three completion tests. The dispatcher-type test stays on 9.0.x with ForwardDispatcherTypeHandler, as it should.
The fix is right, and 8.0.x is the right branch. 7.0.x to 7.2.x have no grails-undertow: there, Undertow support comes from Spring Boot 3 itself, and Grails has no factory to register a handler in. The fix ends processing that otherwise never ends, so it needs no upgrade note on a patch branch.
What I ran:
-
Unit tests of
grails-undertow-spring-bootat the head: 74 tests, all pass. -
The three completion tests, 40 times each: 120 runs, all pass.
-
With the
!asyncContext.isCompletedBeforeInitialRequestDone()check (AsyncErrorCompletionHandler.java:52) removed, the third test fails with["onComplete", "onComplete"]. So it guards what the description says it does. -
The servlet from the description (start asynchronous processing, send a 503), with and without an error page, waiting 40 s, on:
- Undertow, with and without the handler
- Tomcat 11.0.27
- Jetty 12.1.15
See 1 and 2.
-
The merge-ups into 8.1.x and 9.0.x. See 3.
Before merge
1. Tomcat doesn't complete the processing either. Only Jetty does. The Javadoc (AsyncErrorCompletionHandler.java:29-33) says the handler completes it "as Tomcat and Jetty do". The description says a plain servlet hears onComplete on both. This is what the servlet from the description got:
| With an error page | Without one | |
|---|---|---|
| Jetty 12.1.15 | error page, onComplete, response |
onComplete, response |
| Tomcat 11.0.27 | error page, response, then onTimeout and onComplete 30 s later |
onTimeout, onComplete and the response, all 30 s later |
| Undertow, without the handler | error page, response, then nothing in 45 s | response, then onTimeout and onComplete 30 s later |
| Undertow, with the handler | error page, response, onComplete |
response, onComplete |
So Tomcat leaves the processing to its asynchronous timeout. Undertow does the same when no error page answers the error. Undertow never ends the processing only when an error page answers. Spring Boot registers an error page for a Spring MVC application, so the fix still covers the case that matters.
The functional specs pass on Tomcat for another reason. Grails leaves the failure on the request, and Tomcat reports it to the listeners and then completes the processing (CoyoteAdapter.java:366-372).
I'd have the Javadoc say what each container does:
* Completes the asynchronous processing a request dispatch started once the error that dispatch sent has been
* answered, as Jetty does. Undertow sends the error, through the error page that maps to it, once the dispatch has
* returned, and ends the exchange with it. The asynchronous processing is then never completed and never times out,
* so its listeners are never told it completed: {@code ServerHttpObservationFilter} never stops its observation of
* the request, and a {@code DeferredResult} is never resolved. Tomcat leaves such processing to time out, and so does
* Undertow when no error page maps to the error.
The description's servlet example needs the same correction. One more claim in the description doesn't hold: an async request timeout isn't such an error. Spring MVC answers a timeout from the asynchronous dispatch that follows it. WebAsyncManager sets an AsyncRequestTimeoutException as the result and dispatches (lines 332-338 in 7.0.9 and 7.1.0). The handler acts on the request dispatch only.
The 9.0.x copy has the same Javadoc. If #17101 makes the same edit, both branches add identical files, and the merge-up has nothing to resolve in the handler. Otherwise, take this side for the handler in the merge-up.
Notes
2. On James's point: the order is Undertow's own, and this handler can't change it. Take a dispatch that sends an error and then completes the processing itself. With an error page that records when it runs:
- Undertow:
onComplete, error page, response. That's with or without the handler. - Tomcat 11.0.27 and Jetty 12.1.15: error page,
onComplete, response.
Undertow completes that processing in ServletInitialHandler.handleFirstRequest, once the handler chain has returned. It calls handleCompletedBeforeInitialRequestDone() (:279-281), and only after that does it send the error (:284). The inner handler chain wrapper has returned by then. It can't move the completion behind the error page, short of doing Undertow's error dispatch itself.
The PR leaves this case as it was, and the third test pins that the processing still completes once. I'd keep the order out of this PR.
Both behaviours, this order and the processing that never completes, belong in an Undertow issue. A link from the Javadoc, like the #37731 notes in #17101, would tell whoever bumps Undertow when the handler can go. I found no existing issue for it.
3. Merge-up. It merges cleanly into 8.1.x. Into 9.0.x, git reports conflicts in two files:
- The factory.
UndertowServletWebServerFactoryTests: three hunks, the imports and the dispatcher-type test.
The description only names the factory line. In both files, taking the 9.0.x side gives exactly the 9.0.x files. The handler is an identical addition, unless 1 changes it on this branch only.
Verified as correct
git diffof the handler against 9.0.x is empty. The three tests match 9.0.x's, and the factory line sits where the 9.0.x one does.- Without the handler, a request whose error an error page answers never hears
onComplete, even after 45 s. With the handler, it hears it once the response is out. That case is the second test. - The handler acts on the request dispatch only. The error dispatch passes through it with its own dispatcher type. The trace shows
REQUESTfor/sendsError, thenERRORfor the error page. - On the request thread,
complete()only marks the processing as completed (AsyncContextImpl.java:264).handleCompletedBeforeInitialRequestDone()is what tells the listeners, so the listener calls both. - CI is green.
Only Jetty completes the asynchronous processing a request dispatch started once the error it sent has been answered. Tomcat leaves it to its asynchronous timeout, and so does Undertow when no error page maps to the error. Undertow never ends it only when an error page answers.
…has been answered on Undertow A request dispatch that started asynchronous processing, sent an error and then completed the processing itself had its listeners told before the error page ran. Undertow completes such processing once the dispatch has returned, and only then sends the error. When the dispatch has completed the processing, the handler now sends the error itself before it returns and clears it, so the listeners are told once the error has been answered, and Undertow does not send it again.
|
@matrei Thanks.
|
✅ All tests passed ✅🏷️ Commit: 41589e7 Learn more about TestLens at testlens.app/docs. |
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. On a second dispatch, completion still runs before the error response.
|
|
||
| @Override | ||
| public void handleRequest(HttpServerExchange exchange) throws Exception { | ||
| ServletRequestContext context = exchange.getAttachment(ServletRequestContext.ATTACHMENT_KEY); |
There was a problem hiding this comment.
A servlet dispatches to /second. /second starts asynchronous processing again, sends an error, and completes. onComplete still runs before the error page, because this only handles the first dispatch. Run completion after the error response there too.
On Undertow, a request dispatch that starts asynchronous processing and then sends an error, which an error page answers, leaves that processing open forever. Spring MVC sends such an error for a server error it resolves, for example a 503
ResponseStatusException, and Spring Boot registers an error page for a Spring MVC application.Undertow sends the error, through the error page that maps to it, after the dispatch has returned, and ends the exchange there. The task that would call
initialRequestDonenever runs. So the processing never completes and never times out, and itsAsyncListeners never hearonComplete.ServerHttpObservationFilternever stops its observation of the request, and the completion callbacks of aDeferredResultnever run. A plain servlet reproduces it:AsyncErrorCompletionHandler, an inner handler chain wrapper that the factory registers, fixes this. After aREQUESTdispatch that sent an error and left its asynchronous processing open, it adds an exchange-complete listener that completes the processing. On the thread of the request dispatch,AsyncContextImpl.complete()only marks the processing as completed, so the listener also callshandleCompletedBeforeInitialRequestDone(). That call is what tells the async listeners.A dispatch that completes the processing itself after sending the error is completed by Undertow once the dispatch has returned. Undertow then sends the error, so its listeners were told before the error page ran. In that case the handler sends the error itself through
doErrorDispatchbefore it returns, then clears it so Undertow doesn't send it again. The listeners are told once the error has been answered.This doesn't depend on Spring Framework 7.1. On 9.0.x the completion shipped in #17088, and the review there asked for it on 8.0.x too (the
grails-undertowservlet package is identical on both branches). The merge-up into 9.0.x conflicts in three files:ForwardDispatcherTypeHandler: keep both lines.UndertowServletWebServerFactoryTests: keep both sides, 9.0.x's imports and dispatcher-type test plus the tests here.AsyncErrorCompletionHandler, which both branches add: take this side.Tests, in
UndertowServletWebServerFactoryTests:onCompleteonce. This one passes without the handler too, and guards against a second completion.onCompleteafter the error page has run, once only. Without the change they heard it before.