[dependabot]

Bumps datasets from 4.1.1 to 5.0.1.

Release notes

Sourced from datasets's releases.

5.0.1

Bug fixes

• Fix version string in init.py by @​qgallouedec in huggingface/datasets#8244
• fix conda build by @​lhoestq in huggingface/datasets#8250
• Fix JSON loader schema inference for files starting with a UTF-8 BOM (#8241) by @​archievi in huggingface/datasets#8243
• Support hermes traces by @​lhoestq in huggingface/datasets#8255
• Fix batch(by_column=...) crashing after shard/shuffle/split by @​pkooij in huggingface/datasets#8259
• fix traces streaming by @​lhoestq in huggingface/datasets#8277
• support droid agent traces by @​cfahlgren1 in huggingface/datasets#8263
• Fix CI: commit operation equality (hfh 1.20.0) and pytest parametrize collection error by @​Wauplin in huggingface/datasets#8283
• Fix lance auth by @​lhoestq in huggingface/datasets#8301
• Fix symlink-following arbitrary file write in archive extraction by @​AAtomical in huggingface/datasets#8303
• Bug Fix: Resuming Twice Resets the Dataloader by @​francesco-bertolotti in huggingface/datasets#8295
• Bump fsspec and simpler wds compr by @​lhoestq in huggingface/datasets#8337
• fix: validate Arrow IPC record batches by @​XciD in huggingface/datasets#8350
• Make the dataset fingerprint independent of Arrow chunking by @​SuryanshSS1011 in huggingface/datasets#8339
• Fix casting a nullable LargeList to a different inner type by @​vineethsaivs in huggingface/datasets#8346
• docs: replace AutoFeatureExtractor with AutoImageProcessor in image preprocessing docs by @​gautamkishore in huggingface/datasets#8326
• Fix column drop in Arrow path of axis=1 concatenation by @​ebarkhordar in huggingface/datasets#8342
• Raise on length mismatch in batched IterableDataset.map by @​sohumt123 in huggingface/datasets#8332
• Fix require_storage_embed recursing into require_storage_cast by @​vineethsaivs in huggingface/datasets#8349
• Fix path traversal via metadata file_name in folder-based builders by @​Kaif10 in huggingface/datasets#8325
• Support batched=True in Dataset.to_dict by @​vineethsaivs in huggingface/datasets#8333
• Fix hdf5 external files by @​lhoestq in huggingface/datasets#8355
• remove bad require_storage test by @​lhoestq in huggingface/datasets#8357
• ensure fiels are in repo by @​lhoestq in huggingface/datasets#8356
• Keep flat numeric columns with nulls numeric in numpy format by @​ebarkhordar in huggingface/datasets#8352
• Fix bucket dataset card handling and push metadata accounting by @​pjh4993 in huggingface/datasets#8354
• Fix CSV loader dropping on_bad_lines/encoding_errors on pandas 2.0-2.2 by @​ebarkhordar in huggingface/datasets#8358
• Keep integers on the python read path for fixed-shape ArrayXD columns with nulls by @​ebarkhordar in huggingface/datasets#8363
• Rebatch arrow source before formatting in IterableDataset.filter to fix resume data loss by @​ebarkhordar in huggingface/datasets#8360
• Decode Json() columns in Dataset.to_pandas() by @​ebarkhordar in huggingface/datasets#8344
• fix buckets on windows by @​lhoestq in huggingface/datasets#8369
• Fix DatasetDict.push_to_hub leaving removed splits in the dataset card by @​pjh4993 in huggingface/datasets#8367
• Preserve nullable integer columns in to_json/to_csv/to_sql by @​ebarkhordar in huggingface/datasets#8366

Docs

• docs: fix duplicate "to" in IterableDataset push-to-hub example by @​DaoyuanLi2816 in huggingface/datasets#8252
• Clarify dataset creation vs loading workflows in create_dataset tutorial by @​zanvari in huggingface/datasets#8235

New Contributors

• @​DaoyuanLi2816 made their first contribution in huggingface/datasets#8252
• @​zanvari made their first contribution in huggingface/datasets#8235
• @​archievi made their first contribution in huggingface/datasets#8243
• @​pkooij made their first contribution in huggingface/datasets#8259
• @​AAtomical made their first contribution in huggingface/datasets#8303
• @​francesco-bertolotti made their first contribution in huggingface/datasets#8295
• @​XciD made their first contribution in huggingface/datasets#8350
• @​SuryanshSS1011 made their first contribution in huggingface/datasets#8339
• @​vineethsaivs made their first contribution in huggingface/datasets#8346

... (truncated)

Commits

• 921c2a7 release: 5.0.1 (#8370)
• c6fc5cd Preserve nullable integer columns in to_json/to_csv/to_sql (#8366)
• 6747b87 Fix DatasetDict.push_to_hub leaving removed splits in the dataset card (#8367)
• b305031 fix buckets on windows (#8369)
• 030a3e5 Decode Json() columns in Dataset.to_pandas() (#8344)
• 0f207a0 Rebatch arrow source before formatting in IterableDataset.filter to fix resum...
• adad35d Keep integers on the python read path for fixed-shape ArrayXD columns with nu...
• 8966746 Fix CSV loader dropping on_bad_lines/encoding_errors on pandas 2.0-2.2 (#8358)
• b8e861a Fix bucket dataset card handling and push metadata accounting (#8354)
• 521a590 Keep flat numeric columns with nulls numeric in numpy format (#8352)
• Additional commits viewable in compare view

[broly-code-security-scanner]

Broly Security Scan

Warning

Latest baseline snapshot is stale. Broly is running in PR-only fallback mode until the next scheduled baseline refreshes. This does not block the PR.

Warning

Summary

13 actionable finding(s) in this PR

• 🔴 3 critical
• 🟠 5 high
• 🟡 5 medium

Main table: 13 shown, 0 omitted of 13 actionable finding(s) above the reporting threshold. See the repository Security tab for the full set.

Severity	Scanner	Issue	Location	Dismiss	Verdict
🔴 CRITICAL	SCA	black@25.9.0 — 3 vulnerabilities (worst: PYSEC-2026-2120)	poetry.lock:1	d14	🔺 TRUE_POSITIVE · Confidence: HIGH
🔴 CRITICAL	SCA	transformers@4.56.2 — 15 vulnerabilities (worst: PYSEC-2026-2290)	poetry.lock:1	d16	🔺 TRUE_POSITIVE · Confidence: HIGH
🔴 CRITICAL	SCA	pillow@11.3.0 — 45 vulnerabilities (worst: GHSA-8g5g-8vmr-9782)	poetry.lock:1	d15	🔺 TRUE_POSITIVE · Confidence: HIGH
🟠 HIGH	SCA	urllib3@2.4.0 — 16 vulnerabilities (worst: GHSA-2xpw-w6gg-jr37)	poetry.lock:1	d12	🔺 TRUE_POSITIVE · Confidence: HIGH
🟠 HIGH	SCA	fsspec@2025.3.0 — 2 vulnerabilities (worst: GHSA-27vj-qcqg-25rc) → fixed in 2026.6.0	poetry.lock:1	d4	🔺 TRUE_POSITIVE · Confidence: HIGH
🟠 HIGH	SCA	aiohttp@3.12.15 — 72 vulnerabilities (worst: GHSA-2g87-pp6c-29x5)	poetry.lock:1	d17	🔺 TRUE_POSITIVE · Confidence: HIGH
🟠 HIGH	SCA	virtualenv@20.34.0 — 14 vulnerabilities (worst: GHSA-5vjq-rrrf-7h2q)	poetry.lock:1	d22	🔺 TRUE_POSITIVE · Confidence: HIGH
🟠 HIGH	SCA	pyarrow@21.0.0 — 2 vulnerabilities (worst: GHSA-rgxp-2hwp-jwgg) → fixed in 23.0.1	poetry.lock:1	d7	🔺 TRUE_POSITIVE · Confidence: HIGH
🟡 MEDIUM	SCA	idna@3.10 — 2 vulnerabilities (worst: GHSA-65pc-fj4g-8rjx) → fixed in 3.15	poetry.lock:1	d5	🔺 TRUE_POSITIVE · Confidence: HIGH
🟡 MEDIUM	SCA	pytest@8.4.2 — 2 vulnerabilities (worst: GHSA-6w46-j5rx-g56g) → fixed in 9.0.3	poetry.lock:1	d9	🔺 TRUE_POSITIVE · Confidence: HIGH
🟡 MEDIUM	SCA	filelock@3.19.1 — 4 vulnerabilities (worst: GHSA-w853-jp5j-5j7f)	poetry.lock:1	d19	🔺 TRUE_POSITIVE · Confidence: HIGH
🟡 MEDIUM	SCA	requests@2.32.5 — 2 vulnerabilities (worst: PYSEC-2026-2275) → fixed in 2.33.0	poetry.lock:1	d21	🔺 TRUE_POSITIVE · Confidence: HIGH
🟡 MEDIUM	SCA	pygments@2.19.1 — 2 vulnerabilities (worst: PYSEC-2026-2987) → fixed in 2.20.0	poetry.lock:1	d20	🔺 TRUE_POSITIVE · Confidence: HIGH

---

Fix Suggestions

black@25.9.0 — 3 vulnerabilities (worst: PYSEC-2026-2120) — poetry.lock:1

Pin black to a patched version once available (e.g., poetry add black@latest --group dev) or, as a workaround, remove/upgrade black and run the formatter only in an isolated, untrusted-input-free CI environment; monitor PYSEC-2026-2120, PYSEC-2026-2121, and GHSA-3936-cmfr-pm3m for fixed releases.

transformers@4.56.2 — 15 vulnerabilities (worst: PYSEC-2026-2290) — poetry.lock:1

Upgrade to the latest transformers release that addresses the individual advisories (e.g., poetry add transformers@latest or poetry update transformers), and since no single fixed version covers all 15 advisories, review each advisory individually and pin to the newest available version while monitoring for patched releases; as an interim mitigation, avoid loading untrusted models/checkpoints and disable unsafe deserialization paths.

pillow@11.3.0 — 45 vulnerabilities (worst: GHSA-8g5g-8vmr-9782) — poetry.lock:1

Upgrade to the latest available Pillow release (e.g., poetry add pillow@latest or poetry update pillow) and monitor for patched releases addressing GHSA-8g5g-8vmr-9782; as an interim workaround, restrict or sandbox processing of untrusted image inputs.

---

Dismiss false positives

Tick a box to dismiss the finding; untick it to bring the finding back. That is the same as replying /broly dismiss d1 and /broly undismiss d1. To record why it is a false positive, reply with /broly dismiss d1: your reason instead — Broly reuses those reasons to triage similar findings across the org.

• d14 · 🔴 CRITICAL · poetry.lock:1 · black@25.9.0 — 3 vulnerabilities (worst: PYSEC-2026-2120)
• d15 · 🔴 CRITICAL · poetry.lock:1 · pillow@11.3.0 — 45 vulnerabilities (worst: GHSA-8g5g-8vmr-9782)
• d16 · 🔴 CRITICAL · poetry.lock:1 · transformers@4.56.2 — 15 vulnerabilities (worst: PYSEC-2026-2290)
• d17 · 🟠 HIGH     · poetry.lock:1 · aiohttp@3.12.15 — 72 vulnerabilities (worst: GHSA-2g87-pp6c-29x5)
• d4 · 🟠 HIGH     · poetry.lock:1 · fsspec@2025.3.0 — 2 vulnerabilities (worst: GHSA-27vj-qcqg-25rc)
• d7 · 🟠 HIGH     · poetry.lock:1 · pyarrow@21.0.0 — 2 vulnerabilities (worst: GHSA-rgxp-2hwp-jwgg)
• d12 · 🟠 HIGH     · poetry.lock:1 · urllib3@2.4.0 — 16 vulnerabilities (worst: GHSA-2xpw-w6gg-jr37)
• d22 · 🟠 HIGH     · poetry.lock:1 · virtualenv@20.34.0 — 14 vulnerabilities (worst: GHSA-5vjq-rrrf-7h2q)
• d19 · 🟡 MEDIUM   · poetry.lock:1 · filelock@3.19.1 — 4 vulnerabilities (worst: GHSA-w853-jp5j-5j7f)
• d5 · 🟡 MEDIUM   · poetry.lock:1 · idna@3.10 — 2 vulnerabilities (worst: GHSA-65pc-fj4g-8rjx)
• d20 · 🟡 MEDIUM   · poetry.lock:1 · pygments@2.19.1 — 2 vulnerabilities (worst: PYSEC-2026-2987)
• d9 · 🟡 MEDIUM   · poetry.lock:1 · pytest@8.4.2 — 2 vulnerabilities (worst: GHSA-6w46-j5rx-g56g)
• d21 · 🟡 MEDIUM   · poetry.lock:1 · requests@2.32.5 — 2 vulnerabilities (worst: PYSEC-2026-2275)

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (GLM-5.3-Flash) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Attack Hypotheses · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.