[arpitjain099]

Description of the changes being introduced by the pull request:

This adds an end-to-end test that runs the repository, uploader and client examples against each other, which is the part of #2228 that was not covered by the tuf.repository interface tests already merged in #2651.

The new tests/test_examples_e2e.py drives the three examples the way a user runs them from the command line (see their READMEs):

• examples/repository/repo is started as a live HTTP server on a free port
• examples/uploader/uploader does Trust-On-First-Use, claims a delegation, and uploads target files to that server
• examples/client/client does Trust-On-First-Use and downloads the uploaded targets

Because the client downloads through ngclient.Updater, a successful download means the metadata chain and the target hash both verified, so the test asserts the downloaded content matches what the uploader published.

It also covers the "client that has metadata version N can successfully update to N+M" case from the issue: after the client is initialized and has downloaded a first target, the uploader adds a second target (which publishes new targets, snapshot and timestamp metadata). The test checks that the snapshot version advanced and that the already-initialized client can then refresh and download the new target.

A few notes on how it stays deterministic and isolated:

• the example scripts import their private helper modules by name (from _simplerepo import SimpleRepository), so each example dir is put on PYTHONPATH
• the client and uploader store trusted metadata and keys under Path.home(), so HOME is pointed at a temporary directory; nothing touches the real home dir and every run starts from a clean repository
• readiness is handled by polling, first for the port and then for 1.root.json, rather than sleeping

One change to the examples turned out to be needed: RepositoryServer now binds without the reverse DNS lookup that HTTPServer.server_bind() does to fill in server_name, because that lookup runs before the socket starts listening and blocked for over 30 seconds on the macOS runner. Nothing in the example reads server_name. Otherwise this only wires the existing examples together as a test. Ran locally with python -m unittest tests.test_examples_e2e (also green under the project's surrounding example tests), and ruff check / ruff format / mypy are clean on the new file.

I kept this scoped to the examples e2e slice. Whether the remaining idea in the issue (unit-testing SimpleRepository directly) is still wanted, and whether #2228 should stay open for it, is up to you.

Addresses #2228

Please verify and check that the pull request fulfills the following
requirements:

• The code follows the Code Style Guidelines
• Tests have been added for the bug fix or new feature
• Docs have been added for the bug fix or new feature

[jku]

I'd really like to have this -- without it the examples are likely to break.

From CI:

TimeoutError: Could not connect to the repository example on port 39201

It's been so long since I touched tests that actually setup http servers that I don't have any useful advice but that issue does reproduce locally for me.

[arpitjain099]

Thanks, glad it's wanted. The CI failure is the examples repository server not being ready when the client connects (the TimeoutError on the port). I'll make the test wait until the server is actually reachable before the client step so it's deterministic, rather than relying on a fixed sleep. Updating the branch shortly.

[jku]

I'll mark this request changes (on account of the CI failure) so it's visible on the PR list -- I don't actually know what is needed here

[arpitjain099]

@jku found it, and it was mine rather than anything flaky. Fixed in 82a8f4b.

setUp was doing this:

self.env["PYTHONPATH"] = os.pathsep.join(
    str(self.examples_dir / name) for name in ("repository", "uploader", "client")
)

That replaces PYTHONPATH rather than adding to it. Where the test runner is using PYTHONPATH to make tuf importable, the repository subprocess then cannot import tuf at all. It surfaced as the unhelpful "repository example exited early with code 1", with the real cause only visible in the captured child output:

from tuf.api.exceptions import RepositoryError
ModuleNotFoundError: No module named 'tuf'

The example directories are now prepended to any existing PYTHONPATH instead of overwriting it.

I reproduced the failure rather than guessing at it: uninstall tuf, put the checkout on PYTHONPATH, and the test fails with exactly that error before the change and passes after. Full suite locally is 206 passed, 193 subtests passed.

Worth noting the log that put me on to it had already expired, so I updated the branch from develop to re-run CI and read the fresh one.

Separately, test-with-sslib-main was failing for the same reason, so that should clear too, though I see it is marked as not a merge blocker.

[arpitjain099]

Found it. The CI log has the real error under the timeout:

File "examples/repository/_simplerepo.py", line 16, in <module>
    from tuf.api.exceptions import RepositoryError
ModuleNotFoundError: No module named 'tuf'

CI runs the tests from the checkout without installing tuf, so tuf is importable only through the current directory. I start the server with cwd=test_dir, which loses that, and the subprocess dies before binding the port. Locally it passed because my environment had tuf installed.

Fixed by adding the repo root to the subprocess PYTHONPATH. Reproduced both ways: in a venv without tuf it failed with that ModuleNotFoundError and now passes, and it still passes where tuf is installed.

[arpitjain099]

The failure was the example server, not tuf. HTTPServer.server_bind() fills in server_name with a reverse DNS lookup of the bind address, and that runs before the socket starts listening. On the macOS runner the lookup blocked for more than thirty seconds, so the port never opened and the test timed out with nothing in the captured output to show why. I confirmed it by aborting the child process and dumping its stack, which sat in socket.getfqdn under server_bind.

The example now binds without that lookup, since nothing there reads server_name. I also split the readiness wait, so a failure now says whether the server never bound or bound and never answered. macos-latest and windows-latest both pass.