[sbokatuk]

Problem

The simulator checks can fail on a run where every check passed. DatadogNet PR 26 fixed this in DatadogNet, and this repo's runner has the same exposure: it greps the simctl launch --console-pty capture for FFMPEGKIT_E2E_DONE PASS, and the app exits straight after printing that line. simctl can stop relaying with the last lines undelivered, which is what happened in run 41110146674.

It's the same code path as the DatadogNet failure. I didn't search this repo's run history for a past occurrence.

Fix

Only .github/scripts/run-simulator-tests.sh changes, with the same logic as DatadogNet PR 26.

• The console stream stays the first source. The verdict is read as PASS, FAIL or nothing, instead of only looking for PASS.
• With no verdict on the console, the script reads the app's PASS, FAIL and FFMPEGKIT_E2E_DONE lines from the unified log, up to three times, and appends them to simulator-tests.log. Only then can it fail. A recovered verdict raises a ::warning::, so the race stays visible.
• A real FFMPEGKIT_E2E_DONE FAIL still fails, from either source.
• Every log query starts at this launch (--start @<unix time>) rather than --last 2m, so a PASS left by an earlier local run can't stand in for this one.

Test report

This repo's real verdict code, from echo "==> running" to the end of the script, ran against recorded console and log fixtures through a fake xcrun that honours the time bound, process name and BEGINSWITH prefixes. Marker FFMPEGKIT_E2E_DONE, process FFmpegKit.iOS.DeviceTests, the latter matching the device test project's name.

Scenario	Expected	Result
Current script, console lost the verdict	fails (the bug)	exit 1
Intact console, passing app (CRLF, as from the real pty)	passes, log not used	exit 0
Console lost the verdict, passing app	passes, warns	exit 0, one log query, ::warning::
Same, but the log only has the line on the second query	passes	exit 0, two queries
Intact console, real failure, stale PASS in the log	fails	exit 1
Console lost the verdict, real failure	fails, names the check	exit 1, FAIL line in the uploaded log
App reported nothing, stale PASS from an earlier launch	fails	exit 1, after three queries
PASS from another process	fails	exit 1
Another repo's marker	fails	exit 1
NUL byte in the console stream	passes	exit 0 (grep -a)

The harness catches a broken port: on SFMCSDK.Net, dropping -a, using --last 2m, reading only PASS and using a wrong process name each failed the matrix.

The same logic ran end to end on real iOS 26.5 and 26.0 simulators in DatadogNet PR 26, where a real app printed real verdicts through a truncated console.

Not covered

• This repo's app wasn't run on a simulator with this change.
• This PR's CI covers the normal path only, because the fallback runs only when the race fires.
• The race itself has not been reproduced on demand. The evidence that it happens is run 41110146674.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.