Skip to content

build: bump pyo3 to 0.29 and add a cargo audit config - #7327

Open
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:cargo-audit-ignore-and-pyo3-0.29
Open

andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:cargo-audit-ignore-and-pyo3-0.29

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

No tracking issue. Found by running cargo audit on native/Cargo.lock.

Rationale for this change

cargo audit on main reports 10 vulnerabilities and 13 warnings. Only the pyo3 ones can be fixed from Comet's own manifests. The rest are in third-party crates pinned by DataFusion, arrow, or object_store, or are dev-only, or do not apply to how Comet uses the crate. Recording why in a checked-in config makes cargo audit --deny warnings a usable signal instead of a list everyone has to re-triage.

What changes are included in this PR?

Two commits:

  1. Bump pyo3 from 0.28 to 0.29 (resolves to 0.29.3). Fixes RUSTSEC-2026-0176 and RUSTSEC-2026-0177, which affect the optional python-udf feature. No source changes were needed.
  2. Add native/.cargo/audit.toml, which ignores the remaining advisories by ID, each with the reason and what unblocks removing it:
    • quick-xml (RUSTSEC-2026-0194 and RUSTSEC-2026-0195): fixed in quick-xml 0.41, which object_store 0.14 requires. Blocked on DataFusion moving to object_store 0.14. The 0.26 copy comes from pprof and is dev-only.
    • rsa (RUSTSEC-2023-0071): no fixed release exists. Reached only through reqsign credential signing.
    • zstd and zstd-safe (RUSTSEC-2026-0391, -0398, -0403 and the nine unsound warnings): fixed in zstd 0.14.3 and zstd-safe 8.2, but arrow-ipc 59 pins zstd ^0.13. Blocked on the arrow 60 upgrade. None applies to Comet's use. ShuffleCodecContext already resets the session before every frame, which is the upstream fix for RUSTSEC-2026-0391. Only Decoder::with_buffer (the read path) is used. Multithreaded compression, dictionaries, try_clone and the seekable API are unused.
    • Unmaintained async-std (criterion, dev-only), paste (proc-macro) and unsafe-libyaml (log4rs).

The ignores are by advisory ID rather than by crate, so a new advisory against the same crate still fails the audit.

How are these changes tested?

  • cargo audit --deny warnings exits 0 from native/.
  • cargo clippy --all-targets --features python-udf -- -D warnings passes on pyo3 0.29.3.
  • Of the python_udf unit tests, the 4 that do not need pyarrow pass. The other 11 need pyarrow, which could not be installed in my environment, so they are left to the pyarrow_udf_test workflow.

@github-actions github-actions Bot added build Build environment enhancement New feature or request labels Oct 29, 2026
@andygrove andygrove added the run-pyarrow-udf-tests Run the PyArrow UDF tests on this pull request instead of waiting for the merge queue label Oct 29, 2026

@sunchao sunchao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2fce9b2 across five independent scopes. I found no actionable code issue in the PyO3 migration or advisory-specific suppressions. The zstd explanations match Comet's reset/read paths and Arrow/Parquet's bulk codec use. Both changes are intended fixes relative to branch-1.2.

The advisory audit passed with the checked-in configuration against the current RustSec database using cargo audit --no-yanked --deny warnings (yanked-package checks excluded). The regular Rust CI action installs PyArrow and runs the feature-enabled Python UDF tests, so the separately skipped PyArrow workflow does not remove that unit coverage. That Rust job and the native build are still queued at this head. Holding approval until the current-head validation completes; local runtime tests were blocked by unavailable locked dependencies in my configured registry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Build environment enhancement New feature or request run-pyarrow-udf-tests Run the PyArrow UDF tests on this pull request instead of waiting for the merge queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants