Skip to content

ci: fix python-nightly marker expression for pytest 9.2 - #1271

Closed
Maxim Semenov (maximsemenov80) wants to merge 1 commit into
mainfrom
maximsemenov/nightly-fix-marker-expression
Closed

Maxim Semenov (maximsemenov80) wants to merge 1 commit into
mainfrom
maximsemenov/nightly-fix-marker-expression

Conversation

@maximsemenov80

Copy link
Copy Markdown
Contributor

Problem

python-nightly fails on every test leg with Unknown marker(s) in '-m' expression: x.

The nightly passes -m "not x" so it runs every test, including the stress tests that tests/e2e/iothub_e2e/pytest.ini deselects by default (addopts = ... -m "not stress"). A command-line -m replaces that default. x is not a registered marker, and pytest 9.2.0 (released 2026-10-27) now validates marker names in -m expressions when --strict-markers is on.

Fix

Use -m "stress or not stress". It selects every test and only references a registered marker.

Dropping -m entirely is not an option: the addopts default -m "not stress" would apply and silently stop running the three stress test files.

Option (pytest 9.2.0, local) Result
-m "not x" Errors
no -m Stress tests deselected
-m "stress or not stress" Everything runs

Verification

python-nightly build 166372 on a temporary branch containing this change plus #1270 (--hostname-type classic, which the nightly's hub setup also needs): all 11 legs passed (112 passed, 20 skipped per leg).

This PR is independent of #1270, but the nightly will stay red until both are merged.

The nightly passes -m "not x" so that it runs every test, including the stress
tests that tests/e2e/iothub_e2e/pytest.ini deselects by default (addopts -m "not
stress"); a command-line -m replaces that default. "x" is not a registered
marker, and pytest 9.2.0 (2026-10-27) now validates marker names in -m
expressions when --strict-markers is on, so every test leg fails with "Unknown
marker(s) in '-m' expression: x" before running a test.

Use "stress or not stress" instead: it selects every test and only references a
registered marker. Simply dropping -m would silently stop running the stress
tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the nightly pytest marker-selection update.

No open issues 🎉

What changed in this PR

Fixes nightly E2E test selection for pytest 9.2’s strict-marker validation while preserving selection of stress-marked tests and existing skip behavior.

Changes:

  • Replaces -m "not x" with -m "stress or not stress", using only a registered marker.
  • Updates comments to explain why nightly must override the default stress-test exclusion.

Full nightly recovery also depends on the separate IoT Hub hostname fix in #1270.

File Description
vsts/​python-nightly.yaml Updates the pytest marker expression and explains the nightly selection override.

Review effort: Balanced

@maximsemenov80

Copy link
Copy Markdown
Contributor Author

Folded into #1270. Neither change can pass the PR gates on its own (this one needs the hostname fix, #1270 needs the nightly marker fix), so they are combined there.

@maximsemenov80
Maxim Semenov (maximsemenov80) deleted the maximsemenov/nightly-fix-marker-expression branch November 17, 2026 16:42
Carter Tinney (cartertinney) pushed a commit that referenced this pull request Nov 17, 2026
…e2e pipelines (#1270)

## Problem

The `DPS e2e` (def 331) and `Python E2E` (def 426) pipelines fail on
`main` since 2026-11-10 with `HttpOperationError: ... 'Bad Request'` on
IoT Hub registry calls (`get_device`, `create_device_with_sas`). The
same commit (`3c779496`) passed on 2026-10-27.

## Root cause

These pipelines install the `azure-iot` Azure CLI extension on demand
(`The command requires the extension azure-iot. It will be installed
first`) and get the latest stable; nothing pins it, and the Azure CLI
itself comes from the hosted image. `azure-iot` **1.0.0** (released
2026-11-05) lists a breaking change in its [release
history](https://github.com/Azure/azure-iot-cli-extension/blob/dev/HISTORY.rst):
*"Hub connection-string/SAS and DPS linked-Hub creation commands default
to automatic hostname selection."*

So `az iot hub connection-string show` now returns
`HostName=<hub>.device.azure-devices.net` (the device endpoint, TLS 1.3
preview) instead of `<hub>.azure-devices.net`. The test helpers and the
`azure-iot-hub` registry manager take the host from that connection
string and send registry (service) calls to the device endpoint, which
answers 400. In the failing DPS run the earlier asserts pass
(registration is `assigned` to the right hub); only the registry lookup
fails.

## Fix

Pass `--hostname-type classic` (supported by `azure-iot` 1.0.0;
confirmed in `--help` for both commands):

- `az iot hub connection-string show ... --pn iothubowner
--hostname-type classic` in `dps-e2e.yaml`, `python-e2e.yaml` and
`python-nightly.yaml`.
- `az iot dps linked-hub create ... --hostname-type classic` in
`dps-e2e.yaml`, so DPS links the classic host and the assigned hub it
reports matches the test's linked hub.

The event hub connection string (`--eh`) is an `sb://` endpoint and is
left unchanged. (An earlier revision of this PR rewrote the host with a
bash substitution; replaced by the supported option.)

## Verification

Queued on this branch (commit `085be58e`):

| Pipeline | Build | Result |
|---|---|---|
| DPS e2e (def 331) | 166255 | 20 passed, 4 skipped (was 20 failed on
`main`, build 166137); the setup log shows only
`<hub>.azure-devices.net` |
| Python E2E (def 426) | 166256 | all jobs succeeded |
| Python nightly (def 520) | 166257 | hub and DPS setup succeeded; all
11 test legs failed before running any test with `ERROR: Unknown
marker(s) in '-m' expression: x`, which is unrelated to this PR (see
below) |

## Not covered

- **Nightly test legs were not exercised.** `python-nightly.yaml` runs
`pytest ... -m "not x"`. pytest 9.2.0 (2026-10-27) now validates marker
names in `-m` expressions when `strict_markers` is on, so this fails on
`main` too, independent of this change. Removing `-m "not x"` (it
selects everything) is a separate fix.
- The horton e2e gates use templates from `Azure/iot-sdks-e2e-fx`; they
may need the same change there.
- This does not test the Python device SDK against a `.device.`
endpoint, which real users may now get from `az iot hub
connection-string show` and the portal.



## Also included: nightly marker fix (was #1271)

`python-nightly` is a gate on this PR and was failing with `Unknown
marker(s) in '-m' expression: x`: pytest 9.2.0 validates marker names in
`-m` under `--strict-markers`. `-m "not x"` is changed to `-m "stress or
not stress"`, which still runs every test, including the stress tests
that `pytest.ini` deselects by default. Dropping `-m` would silently
skip them.

It is combined here because neither change can pass the PR gates alone:
the hostname fix needs the nightly gate green, and the marker fix needs
the hub setup fixed. Verified together on `python-nightly` build 166372
(all 11 legs passed, 112 passed / 20 skipped each).

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants