Repository navigation
Don't block MainApplication.worker when a download fails - #54
Open
erickdeoliveiraleal wants to merge 1 commit into
Open
erickdeoliveiraleal wants to merge 1 commit into
erickdeoliveiraleal wants to merge 1 commit into
Conversation
When a source failed (timeout, 504, invalid data), BoundingBoxMapWithAIDownloader scheduled a retry on MainApplication.worker that slept 10 seconds and then re-downloaded *all* sources through GetDataRunnable. Since the worker is single threaded, validation, upload and other downloads were queued behind it. The retried data was also lost, since it was written to a dataset that had already been returned (and merged) by the caller. The retry now runs once, only for the failed source, on the MapWithAI fork join pool (after a 10 second delay), and the result is merged into the MapWithAI layer. In addition, the Esri paging loop in getAdditionalEsriData always requested the same offset (start + size) and only stopped when the progress monitor reached the expected object count. If that page was empty or failed, it looped forever, blocking the worker indefinitely. The offset now advances by a page per iteration and the loop stops on an empty page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After "Download data" with MapWithAI, the data sometimes appears, but JOSM keeps doing something in the background: validation, upload and other actions that run on
MainApplication.workersit in the queue and take a long time (or never start).MainApplication.workeris single threaded, so anything the plugin leaves running there blocks the rest of JOSM.Two code paths in
BoundingBoxMapWithAIDownloaderdo this.1. Retry on failure runs on
MainApplication.workerWhen a source fails (socket timeout, 504,
IllegalDataException),parseOsmreturns an empty dataset and schedules a retry onMainApplication.worker. The retry sleeps 10 seconds and then runs aGetDataRunnable, which re-downloads all configured sources for the area, not just the failed one (getDataRealignores theinfoset withsetMapWithAIInfo). Each of those downloads can fail and schedule another retry on the worker.The retried data was also lost: it was written into the
DataSetthat had already been returned to (and merged by) the caller.Fix: retry once, only for the failed source, on the MapWithAI fork join pool after a 10 second delay (
CompletableFuture.delayedExecutor), and merge the result into the MapWithAI layer, the same wayMapWithAIDataUtils.getMapWithAIDatadoes.2. Esri paging can loop forever
In
getAdditionalEsriData, the loop always requested the same offset (this.start + size, and both stay the same) and only exited when the progress monitor reached the expected object count. If that page came back empty (failed request, or fewer objects thanreturnCountOnlyreported), the loop never ended and the worker was blocked until JOSM was closed.Fix: advance the offset by one page per iteration, stop at the reported count, and stop on an empty page. Paged requests don't schedule background retries.
Testing
mvn packagebuilds.BoundingBoxMapWithAIDownloaderTestandGetDataRunnableTestpass.DownloadMapWithAITaskTest.testDownloadOsmServerReaderDownloadParamsBoundsProgressMonitorfails locally withIndexOutOfBoundsExceptionboth with and without this change. That is probably a Windows checkout problem:src/test/resources/__files/josmfile?page=...can't be created on Windows.spotless:check, because the formatter config comes from../00_core_tools.Not included: the manifest fix, which is in #53.
🤖 Generated with Claude Code