Repository navigation
Host generic files and share one push path across hosted ecosystems - #537
Merged
Merged
Conversation
simonchrz
marked this pull request as ready for review
October 20, 2026 18:30
Hosted ecosystems are now entries in one table: each defines its package
names, how the stored filename is chosen, how the manifest is read and which
namespaces are served from pushes alone. npm and Composer move there
unchanged, and generic is added as a third entry.
Generic files are pushed to /api/hosted/generic/{name}/{version}/{file}
and served without a token from /generic/hosted/{name}/{version}/{file},
with the sha256 as ETag. They have no manifest, a version can hold several
files, and they never come from an upstream. "hosted" is reserved as an
upstream.generic name. DELETE with a file removes only that file.
For all hosted ecosystems:
- A second push of the same file is refused with 409. Clients pin pushed
files by version and often by hash, so a published file is no longer
silently replaced. Delete it first to push it again.
- PUT takes an optional ?sha256= and refuses bytes that don't match.
proxy hosted push always sends it.
Versions of generic files carry no version-level integrity, since it could
only describe one of their files; each file is checked against its own
sha256.
Refs git-pkgs#133, git-pkgs#134, git-pkgs#535
- Record the package, version and artifact of a push in one transaction that only inserts the artifact if no pushed copy exists. Of two concurrent pushes of the same file exactly one wins, and the loser's version integrity is rolled back with it. - Check for an existing file before reading the request body, and send Expect: 100-continue from proxy hosted push, so a refused re-push isn't uploaded first. - Leave an existing version record alone for multi-file ecosystems, so a generic push doesn't reset a version cached from upstream.generic. - Reserve the upstream.generic name "hosted" only while hosting is enabled, and only intercept /generic/hosted/ then. - Accept a file argument in proxy hosted delete for generic only. - Mark multi-file ecosystems explicitly, and give parse a min/max range.
The handler-level test runs on SQLite only. This one also runs against Postgres when PROXY_DATABASE_URL is set.
- Record a fetched artifact with its package and version in one transaction that refuses to replace a pushed artifact, so a fetch that finishes after a push keeps both the pushed file and its version's integrity. Pushes and cache fills share this write, including the compare-and-swap on the storage path that queues every replaced object. - Don't open the stored file for ecosystems without a manifest. - Derive multi-file versions from the file being named in the path. - Validate DELETE paths like PUT paths, answering 400 for malformed ones. - Parse ?sha256= with integrity.ParseHex.
simonchrz
force-pushed
the
feat/hosted-adapters
branch
from
October 21, 2026 10:25
26ed83f to
48916bc
Compare
andrew
requested changes
Oct 21, 2026
andrew
left a comment
andrew
left a comment
Contributor
There was a problem hiding this comment.
The new duplicate check can be bypassed with Composer name casing. An API test pushed acme/private@1.0.0, then acme/Private@1.0.0 with different bytes. Both returned 201. Their stored filenames differ, but they share the version integrity record, so downloading the original archive then failed with EOF.
For ecosystems allowing one file per version, please reject an existing pushed version atomically by canonical identity, including case aliases, and add an API regression test that verifies the first archive remains downloadable.
Composer names are case-insensitive, so acme/private and acme/Private share the version record and its integrity but get different filenames. A second push under the other case was accepted and broke the integrity check of the first archive. For ecosystems with one file per version, any pushed file of the version now refuses the push, both before the upload and atomically when recording it.
Contributor
Author
|
Done: for npm and Composer a push is now refused if the version already holds a pushed file under any filename, both before the upload and atomically when recording it (the version row is locked first). Added an API test for |
andrew
approved these changes
Oct 22, 2026
andrew
left a comment
andrew
left a comment
Contributor
There was a problem hiding this comment.
Case-alias re-push is refused before upload and atomically on record, and the regression test checks the first archive still downloads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #535 and pulls #133, #134 and #535 onto one path, so a new hosted ecosystem is a table entry rather than new switches.
What changes
PUT /api/hosted/generic/{name}/{version}/{file}, served without a token fromGET/HEAD /generic/hosted/{name}/{version}/{file}with the sha256 asETag. No manifest, several files per version, never fetched upstream. While hosting is enabled,hostedis reserved as anupstream.genericname.DELETEwith a file removes only that file; without one, the whole version.proxy hosted push generic tools 1.2.0 ./tool.tar.gznames the file after the local one.Expect: 100-continueclients don't upload a refused file.?sha256=on PUT, checked against the stored bytes; a mismatch is 400.proxy hosted pushalways sends it.Generic versions don't get a version-level integrity, since it could only describe one of their files, and an existing version record (for example one cached from
upstream.generic) is left alone. Each file is still checked against its own sha256 when served.Testing
Unit and API tests for generic push, download, list and single-file delete, 409 on re-push and after delete, concurrent pushes of one file, refusal before the body is read, sha256 match and mismatch, the reserved upstream name with hosting on and off, and the transactional record on SQLite. CI runs the database tests on SQLite only; the full suite, including concurrent pushes at the database level, also passes against Postgres 17 locally. One test runs against the full server. golangci-lint v2.13.1 is clean and the swagger docs are regenerated.