Skip to content

Upgrade cibuildwheel to v4.3.3 and test wheels on Python 3.15 - #147

Merged
kesmit13 merged 5 commits into
singlestore-labs:mainfrom
mbUSC:cibuildwheel-v4
Nov 12, 2026
Merged

kesmit13 merged 5 commits into
singlestore-labs:mainfrom
mbUSC:cibuildwheel-v4

Conversation

@mbUSC

@mbUSC mbUSC commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

cibuildwheel v2.16.5 predates Python 3.15 and can't build or test on it. Upgrade to v4.3.3 and add cp315 to CIBW_BUILD. The wheel is abi3, so it is still built only once, on cp39; cibuildwheel reuses it for cp315 and only runs the tests there.

Side effects of going from v2 to v4, from cibuildwheel's changelog:

  • Linux wheels are built on manylinux_2_28 instead of manylinux2014, so they need glibc 2.28 or newer. CentOS 7 and Amazon Linux 2 users would fall back to the sdist or the pure-Python wheel. Setting CIBW_MANYLINUX_{X86_64,AARCH64}_IMAGE to manylinux2014 keeps the old baseline.
  • Each abi3 wheel is checked with abi3audit after repair, and a failure fails the build.
  • Windows wheels are repaired with delvewheel, which bundles any DLLs the extension links against.
  • Wheels are built with build instead of pip wheel, in environments without setuptools and wheel preinstalled. Both come from pyproject.toml's build-system.requires instead.

Note

Medium Risk
Changes the release wheel pipeline (cibuildwheel major upgrade, Python 3.15 test matrix, and manylinux image pinning), which affects binary compatibility and publish reliability for all platforms.

Overview
Upgrades the publish workflow from pypa/cibuildwheel@v2.16.5 to v4.3.3 so wheels can be exercised on Python 3.15. CIBW_BUILD now includes cp315-* alongside cp39-*; because the extension wheel is abi3, cibuildwheel still builds once on 3.9 and reuses that wheel for 3.15 while running tests on both ends of the supported range.

To offset cibuildwheel v3+ defaulting Linux builds to manylinux_2_28, the workflow pins CIBW_MANYLINUX_X86_64_IMAGE and CIBW_MANYLINUX_AARCH64_IMAGE to manylinux2014, preserving the older glibc baseline so older distros still install the compiled wheel instead of silently falling back to pure Python.

In accel.c, the local UTF-8 helper _PyUnicode_AsUTF8 is marked static to keep it file-local (avoids clashing with Python’s internal symbol naming during limited-API / abi3 wheel checks).

Reviewed by Cursor Bugbot for commit 8ca9937. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features
    • Added package wheel builds for Python 3.15, with the ABI-stable wheel tested on that version.
  • Compatibility
    • Linux wheels for x86_64 and aarch64 now target manylinux2014 for broader compatibility.

cibuildwheel v2.16.5 predates Python 3.15 and can't build or test on
it. Upgrade to v4.3.3 and add cp315 to CIBW_BUILD. The wheel is abi3,
so it is still built only once, on cp39; cibuildwheel reuses it for
cp315 and only runs the tests there.

Side effects of going from v2 to v4, from cibuildwheel's changelog:
- Linux wheels are built on manylinux_2_28 instead of manylinux2014,
  so they need glibc 2.28 or newer. CentOS 7 and Amazon Linux 2 users
  would fall back to the sdist or the pure-Python wheel. Setting
  CIBW_MANYLINUX_{X86_64,AARCH64}_IMAGE to manylinux2014 keeps the old
  baseline.
- Each abi3 wheel is checked with abi3audit after repair, and a
  failure fails the build.
- Windows wheels are repaired with delvewheel, which bundles any DLLs
  the extension links against.
- Wheels are built with `build` instead of `pip wheel`, in environments
  without setuptools and wheel preinstalled. Both come from
  pyproject.toml's build-system.requires instead.
@mbUSC mbUSC mentioned this pull request Oct 23, 2026
mbUSC pushed a commit to mbUSC/singlestoredb-python that referenced this pull request Oct 28, 2026
Python 3.15 removed zipimport.zipimporter.load_module(), which the
external function server used to load a zipped environment file. Load
it with find_spec()/exec_module() instead. Those methods only exist
from Python 3.10, so 3.9 keeps using load_module(); that branch can be
removed when 3.9 support is dropped.

Use asyncio.get_running_loop() in AwaitableUvicornServer rather than
get_event_loop(). Both callers already create it inside a running
event loop, so behaviour is unchanged.

Add 3.15 to the smoke-test and pre-commit matrices. The cibuildwheel
upgrade needed to build and test wheels on 3.15 is in singlestore-labs#147.

Tested locally on 3.15.0 against SingleStore 9.1.1 (management tests
excluded): 967 passed, 19 skipped, 1 failed, the same results as 3.14.
The failure, TestServerProcessMode::test_process_mode_worker_handles_request,
also fails on 3.14 on macOS and is unrelated to this change.
@kesmit13

Copy link
Copy Markdown
Collaborator

I ran this branch locally with cibuildwheel v4.3.3 (Linux x86_64, cp39-* cp315-*). It needs two changes before it can merge.

1. accel.c: make _PyUnicode_AsUTF8 static

As it stands the build fails at the new abi3audit step:

_singlestoredb_accel.abi3.so: "is_abi3": false, "non_abi3_symbols": ["_PyUnicode_AsUTF8"]
cibuildwheel: error: Audit command failed: abi3audit --strict --report ...

_PyUnicode_AsUTF8 isn't a CPython function. It's our own helper at accel.c:374, and it's declared without static, so it's exported from the .so. abi3audit sees an exported _Py* symbol and treats it as a call outside the stable ABI. v2.16.5 never ran abi3audit, which is why this hasn't shown up before. abi3audit runs on every platform's abi3 wheel, so the Windows and macOS builds would fail the same way.

The helper is only used inside accel.c, so the fix is one word:

-char *_PyUnicode_AsUTF8(PyObject *unicode) {
+static char *_PyUnicode_AsUTF8(PyObject *unicode) {

The name can stay. It's there so the helper can be swapped for CPython's own function once the limited-API baseline reaches 3.10. Note that the 3.10 limited-API function is PyUnicode_AsUTF8AndSize(obj, NULL), not PyUnicode_AsUTF8, and that it returns a borrowed buffer cached on the string object rather than a copy we own. The swap would have to drop the matching free() calls and make sure each source object outlives the stored pointers.

2. publish.yml: keep the glibc 2.17 baseline

cibuildwheel v3+ builds Linux wheels on manylinux_2_28 by default, which raises the minimum glibc from 2.17 to 2.28. Customers on CentOS/RHEL 7, Amazon Linux 2 or Ubuntu 18.04 wouldn't get an error: pip would skip the binary wheel and install our pure-Python py3-none-any wheel, so they'd lose the C accelerator without any warning. The extension is plain C with no need for a newer glibc or toolchain, so manylinux_2_28 gains us nothing. Pin the old image in the "Build and test wheels" env: block:

          # cibuildwheel v3+ builds on manylinux_2_28 (glibc 2.28) by default.
          # Pin manylinux2014 (glibc 2.17) so CentOS/RHEL 7, Amazon Linux 2 and
          # Ubuntu 18.04 still get the C extension instead of falling back to
          # the pure-Python wheel without any warning.
          CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
          CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014

i686 is already skipped by CIBW_SKIP, and musllinux isn't built. Using the manylinux2014 name means cibuildwheel supplies the image version it has pinned, so it keeps getting updated as cibuildwheel is upgraded. v4.3.3 pins the 2026.10.21 build, which includes CPython 3.15.

manylinux2014 is based on CentOS 7, which is end-of-life. We should plan to move to manylinux_2_28 once pypa stops rebuilding the image, and call it out in the release notes when we do.

Local result with both changes

singlestoredb-1.18.1-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
abi3audit: passed
ACCEL-OK 3.9.25  ...
Found previously built wheel ... compatible with cp315-manylinux_x86_64. Skipping build step...
ACCEL-OK 3.15.0 ... [GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]

The test command here only imports _singlestoredb_accel; it doesn't run test_basics against a database. Not covered locally: aarch64, Windows, macOS, and the database tests. CI will cover those, and will also confirm that the aarch64 manylinux2014 image has cp315.

To reproduce:

CIBW_BUILD="cp39-* cp315-*" CIBW_ARCHS_LINUX=x86_64 \
CIBW_SKIP="pp* *-musllinux* *-manylinux_i686" \
CIBW_MANYLINUX_X86_64_IMAGE=manylinux2014 \
CIBW_TEST_COMMAND='python -c "import sys, _singlestoredb_accel; print(sys.version)"' \
uvx cibuildwheel==4.3.3 --platform linux --output-dir wheelhouse

Your Name and others added 2 commits October 29, 2026 00:54
abi3audit, which cibuildwheel v4 runs on abi3 wheels, rejected the
extension because the non-static helper exported a _Py* symbol.

cibuildwheel v3+ defaults to manylinux_2_28; pin manylinux2014 to keep
the glibc 2.17 baseline so older distros still get the C extension.
@mbUSC

mbUSC commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

@kesmit13 good catch, thank you!

  1. _PyUnicode_AsUTF8 is now static, so it's no longer exported and abi3audit should pass.
  2. Pinned CIBW_MANYLINUX_X86_64_IMAGE / CIBW_MANYLINUX_AARCH64_IMAGE to manylinux2014 to keep the glibc 2.17 baseline.

kesmit13 added a commit that referenced this pull request Nov 11, 2026
* Add Python 3.15 support

Python 3.15 removed zipimport.zipimporter.load_module(), which the
external function server used to load a zipped environment file. Load
it with find_spec()/exec_module() instead. Those methods only exist
from Python 3.10, so 3.9 keeps using load_module(); that branch can be
removed when 3.9 support is dropped.

Use asyncio.get_running_loop() in AwaitableUvicornServer rather than
get_event_loop(). Both callers already create it inside a running
event loop, so behaviour is unchanged.

Add 3.15 to the smoke-test and pre-commit matrices. The cibuildwheel
upgrade needed to build and test wheels on 3.15 is in #147.

Tested locally on 3.15.0 against SingleStore 9.1.1 (management tests
excluded): 967 passed, 19 skipped, 1 failed, the same results as 3.14.
The failure, TestServerProcessMode::test_process_mode_worker_handles_request,
also fails on 3.14 on macOS and is unrelated to this change.

* Clean up failed environment imports and test the zip loader

If the module in a zipped environment file raised while importing,
main() left the half-initialized module in sys.modules, and a later
import in the same process would return it without retrying. Remove
the entry and re-raise, as load_module() did.

Add test_asgi_env_file, which runs a zip environment through main()
with uvicorn mocked out and checks that its functions are found, and
that a failed import leaves nothing in sys.modules. The smoke-test job
runs it on every Python version, 3.9 through 3.15, since the loader
takes a different path on 3.9 than on 3.10+.

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Kevin D Smith <ksmith@singlestore.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The publish workflow upgrades cibuildwheel, adds cp315 to the build selection, and pins Linux builds to manylinux2014. In accel.c, _PyUnicode_AsUTF8 becomes file-local.

Changes: Wheel publishing compatibility

Layer / File(s) Summary
C helper linkage
accel.c
_PyUnicode_AsUTF8 is now file-local. Its allocation and conversion behavior is unchanged.
Wheel build settings
.github/workflows/publish.yml
The workflow upgrades cibuildwheel to v4.3.3, selects cp39 and cp315, and pins x86_64 and aarch64 Linux builds to manylinux2014. Comments describe the abi3 build and test selection.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related PRs

Merge Risk: Minimal · up to 8ca99

The change updates wheel publishing to test on Python 3.15 and keeps the older Linux compatibility baseline. It also makes an internal C helper file-local so the new abi3 audit can pass. No actionable merge-blocking risk is evident; CI should still confirm the aarch64, Windows, and macOS builds.

Architecture risk: Low · up to 8ca99

The change affects 1 system.

Changed systems: accel.c

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — accel.c (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in accel.c: _PyUnicode_AsUTF8 is now file-local rather than externally linked.
  • observed — Modified behavior in .github/workflows/publish.yml: The wheel build now uses cibuildwheel v4.3.3 instead of v2.16.5, selects cp39 and cp315 builds instead of cp39 alone, and pins both x86_64 and aarch64 Linux images to manylinux2014. The added comments describe the abi3 build/test selection and the manylinux compatibility rationale.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the two main changes: the cibuildwheel upgrade to v4.3.3 and the addition of Python 3.15 wheel testing. The raw summary and objectives confirm both. The title does not mention the stat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: the cibuildwheel upgrade to v4.3.3 and the addition of Python 3.15 wheel testing. The raw summary and objectives confirm both. The title does not mention the static helper change or the manylinux2014 pinning, but those are secondary to the stated PR goal.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4aa2a43c-243d-4017-8120-6791fa31e015
Commits

Reviewing files that changed from the base of the PR and between 3266f62 and 8ca9937.

Files selected for processing: 2

  • .github/workflows/publish.yml
  • accel.c

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@kesmit13
kesmit13 merged commit 16f506b into singlestore-labs:main Nov 12, 2026
15 checks passed

This branch was previously deployed

1 inactive deployment
Base — 8ca9937c Deployed Nov 12, 2026 by kesmit13 via test-coverage #642
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants