[ben-malbeclabs]

Summary of Changes

• With --cloudflare-zone-id (CLOUDFLARE_ZONE_ID) and --aop-hostnames (AOP_HOSTNAMES), the allowlist sync reads each hostname's per-hostname Authenticated Origin Pulls setting from Cloudflare after every pass. It exports doublezero_telemetry_allowlist_sync_aop_certificate_expiry_timestamp_seconds and doublezero_telemetry_allowlist_sync_aop_active per hostname. This is the AOP certificate expiry alert of the gNMI lockdown spec (malbeclabs/infra docs/plans/2026-10-29-gnmi-restart-lockdown-design.md, section 8). When the client certificate expires, or AOP is off for gnmic's hostname, every device drops.
• aop_active is 1 only when AOP is on for the hostname and both its association and its certificate are active. A hostname with no setting (Cloudflare's 404 with code 1553) reads 0 and has no expiry. Any other error, a wrong zone's 404 included, is a failed read.
• A hostname whose setting cannot be read loses both series, so the alert sees "not reported", never a stale expiry. errors_total{stage="aop"} counts the failures, and each is logged at ERROR.
• The read runs after the pass, so a failed read never stops a pass. A slow one delays the next pass by at most the 20-second report timeout. It is skipped after shutdown.
• The two flags are set together or not at all. Without them nothing changes. The token then also needs "Zone > SSL and Certificates > Read", which returns certificates and never keys.
• call is split into cfCall, and a Cloudflare error is now a typed cfAPIError, so the AOP reader can tell "no setting" from a failed read. Its message is unchanged.

The alert rule and the testnet overlay values are a malbeclabs/infra PR that pins the build of this commit.

Testing Verification

• TestCloudflareAOP_ReadsTheSetting decodes the fields as Cloudflare returned them for gnmic-testnet.doublezero.xyz on 2026-11-14, certificate left out.
• Tests pin the "no setting" answer, a wrong-zone 404, a 403 and a bad expires_on. They also pin each hostname's series: set, removed on a failed read and back on the next, and dropped when the setting is removed. They check that a failed read leaves the pass's writes and last_success intact, and that a pass cut short by shutdown reads nothing.
• Mutation checks, each with a failing test: no series delete on a failed read, no delete on a missing expiry, no shutdown guard, any 404 read as "no setting", and either status check dropped from Active.
• go test -race, go vet and golangci-lint are clean on telemetry/allowlist-sync/....

[Copilot]

🟡 Changes recommended

The failed-read test discards logs, so it cannot detect removal or severity downgrading of the expected ERROR message.

1 open issue

• Failed-read test does not assert the ERROR log · New

What changed in this PR

This PR adds per-hostname Authenticated Origin Pulls (AOP) monitoring to the telemetry allowlist sync for certificate expiry alerts.

Changes:

• The sync exports certificate expiry and active-state metrics when operators configure both new flags.
• The sync reads settings after each pass and removes both metric series when a read fails.
• The Cloudflare client distinguishes missing settings from API failures through typed errors.

File	Description
telemetry/​allowlist-sync/​internal/​allowlist/​sync.go	Adds optional AOP configuration and reads after each pass.
telemetry/​allowlist-sync/​internal/​allowlist/​sync_test.go	Tests metric updates, read failures, and shutdown behavior.
telemetry/​allowlist-sync/​internal/​allowlist/​metrics.go	Registers per-hostname expiry and active-state metrics.
telemetry/​allowlist-sync/​internal/​allowlist/​http_test.go	Tests response decoding, missing settings, and API errors.
telemetry/​allowlist-sync/​internal/​allowlist/​cloudflare.go	Extracts shared request handling and adds typed API errors.
telemetry/​allowlist-sync/​internal/​allowlist/​aop.go	Reads AOP settings and updates metrics within a timeout.
telemetry/​allowlist-sync/​cmd/​allowlist-sync/​main.go	Adds paired flags and connects the AOP reader.
CHANGELOG.md	Documents the metrics, configuration, and token permission.

_{Review effort: Balanced}