Repository navigation
ci: fix python-nightly marker expression for pytest 9.2 - #1271
Closed
Maxim Semenov (maximsemenov80) wants to merge 1 commit into
Closed
Maxim Semenov (maximsemenov80) wants to merge 1 commit into
Maxim Semenov (maximsemenov80) wants to merge 1 commit into
Conversation
The nightly passes -m "not x" so that it runs every test, including the stress tests that tests/e2e/iothub_e2e/pytest.ini deselects by default (addopts -m "not stress"); a command-line -m replaces that default. "x" is not a registered marker, and pytest 9.2.0 (2026-10-27) now validates marker names in -m expressions when --strict-markers is on, so every test leg fails with "Unknown marker(s) in '-m' expression: x" before running a test. Use "stress or not stress" instead: it selects every test and only references a registered marker. Simply dropping -m would silently stop running the stress tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Maxim Semenov (maximsemenov80)
requested review from
Carter Tinney (cartertinney) and
Ewerton Scaboro da Silva (ewertons)
and
a balanced review from Copilot
November 16, 2026 17:31
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the nightly pytest marker-selection update.
No open issues 🎉
What changed in this PR
Fixes nightly E2E test selection for pytest 9.2’s strict-marker validation while preserving selection of stress-marked tests and existing skip behavior.
Changes:
- Replaces
-m "not x"with-m "stress or not stress", using only a registered marker. - Updates comments to explain why nightly must override the default stress-test exclusion.
Full nightly recovery also depends on the separate IoT Hub hostname fix in #1270.
| File | Description |
|---|---|
vsts/python-nightly.yaml |
Updates the pytest marker expression and explains the nightly selection override. |
Review effort: Balanced
Carter Tinney (cartertinney)
approved these changes
Nov 16, 2026
Contributor
Author
Maxim Semenov (maximsemenov80)
deleted the
maximsemenov/nightly-fix-marker-expression
branch
November 17, 2026 16:42
Carter Tinney (cartertinney)
pushed a commit
that referenced
this pull request
Nov 17, 2026
…e2e pipelines (#1270) ## Problem The `DPS e2e` (def 331) and `Python E2E` (def 426) pipelines fail on `main` since 2026-11-10 with `HttpOperationError: ... 'Bad Request'` on IoT Hub registry calls (`get_device`, `create_device_with_sas`). The same commit (`3c779496`) passed on 2026-10-27. ## Root cause These pipelines install the `azure-iot` Azure CLI extension on demand (`The command requires the extension azure-iot. It will be installed first`) and get the latest stable; nothing pins it, and the Azure CLI itself comes from the hosted image. `azure-iot` **1.0.0** (released 2026-11-05) lists a breaking change in its [release history](https://github.com/Azure/azure-iot-cli-extension/blob/dev/HISTORY.rst): *"Hub connection-string/SAS and DPS linked-Hub creation commands default to automatic hostname selection."* So `az iot hub connection-string show` now returns `HostName=<hub>.device.azure-devices.net` (the device endpoint, TLS 1.3 preview) instead of `<hub>.azure-devices.net`. The test helpers and the `azure-iot-hub` registry manager take the host from that connection string and send registry (service) calls to the device endpoint, which answers 400. In the failing DPS run the earlier asserts pass (registration is `assigned` to the right hub); only the registry lookup fails. ## Fix Pass `--hostname-type classic` (supported by `azure-iot` 1.0.0; confirmed in `--help` for both commands): - `az iot hub connection-string show ... --pn iothubowner --hostname-type classic` in `dps-e2e.yaml`, `python-e2e.yaml` and `python-nightly.yaml`. - `az iot dps linked-hub create ... --hostname-type classic` in `dps-e2e.yaml`, so DPS links the classic host and the assigned hub it reports matches the test's linked hub. The event hub connection string (`--eh`) is an `sb://` endpoint and is left unchanged. (An earlier revision of this PR rewrote the host with a bash substitution; replaced by the supported option.) ## Verification Queued on this branch (commit `085be58e`): | Pipeline | Build | Result | |---|---|---| | DPS e2e (def 331) | 166255 | 20 passed, 4 skipped (was 20 failed on `main`, build 166137); the setup log shows only `<hub>.azure-devices.net` | | Python E2E (def 426) | 166256 | all jobs succeeded | | Python nightly (def 520) | 166257 | hub and DPS setup succeeded; all 11 test legs failed before running any test with `ERROR: Unknown marker(s) in '-m' expression: x`, which is unrelated to this PR (see below) | ## Not covered - **Nightly test legs were not exercised.** `python-nightly.yaml` runs `pytest ... -m "not x"`. pytest 9.2.0 (2026-10-27) now validates marker names in `-m` expressions when `strict_markers` is on, so this fails on `main` too, independent of this change. Removing `-m "not x"` (it selects everything) is a separate fix. - The horton e2e gates use templates from `Azure/iot-sdks-e2e-fx`; they may need the same change there. - This does not test the Python device SDK against a `.device.` endpoint, which real users may now get from `az iot hub connection-string show` and the portal. ## Also included: nightly marker fix (was #1271) `python-nightly` is a gate on this PR and was failing with `Unknown marker(s) in '-m' expression: x`: pytest 9.2.0 validates marker names in `-m` under `--strict-markers`. `-m "not x"` is changed to `-m "stress or not stress"`, which still runs every test, including the stress tests that `pytest.ini` deselects by default. Dropping `-m` would silently skip them. It is combined here because neither change can pass the PR gates alone: the hostname fix needs the nightly gate green, and the marker fix needs the hub setup fixed. Verified together on `python-nightly` build 166372 (all 11 legs passed, 112 passed / 20 skipped each). --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
python-nightlyfails on every test leg withUnknown marker(s) in '-m' expression: x.The nightly passes
-m "not x"so it runs every test, including the stress tests thattests/e2e/iothub_e2e/pytest.inideselects by default (addopts = ... -m "not stress"). A command-line-mreplaces that default.xis not a registered marker, and pytest 9.2.0 (released 2026-10-27) now validates marker names in-mexpressions when--strict-markersis on.Fix
Use
-m "stress or not stress". It selects every test and only references a registered marker.Dropping
-mentirely is not an option: theaddoptsdefault-m "not stress"would apply and silently stop running the three stress test files.-m "not x"-m-m "stress or not stress"Verification
python-nightlybuild 166372 on a temporary branch containing this change plus #1270 (--hostname-type classic, which the nightly's hub setup also needs): all 11 legs passed (112 passed, 20 skipped per leg).This PR is independent of #1270, but the nightly will stay red until both are merged.