Skip to content

fix: apply workflow error policies to failed downloads - #102

Open
Shubham-Padkonde wants to merge 1 commit into
dbpedia:mainfrom
Shubham-Padkonde:fix/workflow-download-failures
Open

Shubham-Padkonde wants to merge 1 commit into
dbpedia:mainfrom
Shubham-Padkonde:fix/workflow-download-failures

Conversation

@Shubham-Padkonde

Copy link
Copy Markdown

Description

A workflow download step reports success when a file returns HTTP 404. The download API records the failed file and returns, but DownloadStep only reads successful entries. As a result, on_error: fail does not stop the workflow, continue reports success, and retry never retries.

Check the current step's newly recorded entries and raise when any download failed. The workflow engine can then apply the existing error policy. Standalone download behavior is unchanged.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Housekeeping

Validation

Six regression cases exercise the real download and workflow code with HTTP responses mocked at the request boundary: fail, continue, and retry, each with and without a workflow manifest. All six fail on the base branch and pass with this change. The retry case also verifies the downloaded file contents.

  • python -m pytest -q: 189 passed, 3 skipped (Windows, Python 3.13).
  • ruff check .: passed with Ruff 0.5.5.
  • git diff --check: passed.

Checklist

  • Code follows the project's Ruff configuration.
  • I have performed a self-review of my own code.
  • Added tests that reproduce the failure and exercise the fix.
  • New and existing unit tests pass locally.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e7532d41-d5e7-419a-94a5-8a7def748026
Commits

Reviewing files that changed from the base of the PR and between 3701c23 and 418411a.

Files selected for processing: 2
  • databusclient/workflow/steps.py
  • tests/test_workflow_download_failures.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant