[tests] Ignore the AppSize tests for now. - #25480
Conversation
It seems that the results depend on the macOS version of the machine executing the tests. That's unfortunate when all my macs are at 26.5 and the bots are at 26.2, because I can't re-generate the expected app sizes locally. So ignore these tests until I can come up with a better solution.
There was a problem hiding this comment.
Pull request overview
Disables the tests/dotnet app size regression tests by marking the entire AppSizeTest fixture as ignored, due to expected size outputs varying by the macOS version on the machine running the tests.
Changes:
- Mark the
AppSizeTestNUnit fixture as ignored with a message explaining macOS-version-dependent results.
✅ [PR Build #1a9c44b] Build passed (Detect API changes) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
✅ [PR Build #1a9c44b] Build passed (Build packages) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [PR Build #1a9c44b] Build passed (Build macOS tests) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build #1a9c44b] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 183 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
It seems that the results depend on the macOS version of the machine executing the tests. That's unfortunate when all my macs are at 26.5 and the bots are at 26.2, because I can't re-generate the expected app sizes locally. So ignore these tests until I can come up with a better solution. Backport of #25480.
It seems that the results depend on the macOS version of the machine executing
the tests.
That's unfortunate when all my macs are at 26.5 and the bots are at 26.2,
because I can't re-generate the expected app sizes locally.
So ignore these tests until I can come up with a better solution.