You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today we use swift-snapshot-testing to make assertions on UI rendering of views.
We have Fastlane running these tests and it work as expected.
We have our internal GitLab CI runner, running the same Fastlane lanes, and it works like expected.
However in GitHub Actions some tests do not pass, even if they pass with the other environment, without further details.
It seems the provided environment with GitHub runners is unstable making some test fail (even if they pass on our GitLab CI and local computers), e.g. testAllTypographies*, testAllSizes*, testAllElevations* don't pass in GitHub Actions level but pass in our internal levels.
We do not know if there are hardware resources issues, shadow tomeouts or other configurations trobules witht he Mac runners or the in-use simulators.
Thus we will have checks failed for UI tests and must disable requirement son them to preven pull request to be merged.
This is a bit sad and we should find a workaround or a fix for this issue. This is not mandatory as we plan to move the demo app, with the UI tests, in out internal GitLab software forge in the future.
[16:40:24]: ▸ Test Suite OUDSTokensElevationUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:40:39]: ▸ ::error :: testAllElevationsInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (15.131 seconds)
[16:40:40]: ▸ ::error :: testAllElevationsInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.325 seconds)
[16:40:54]: ▸ ::error :: testAllElevationsOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.344 seconds)
[16:41:28]: ▸ ::error :: testAllElevationsOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (13.717 seconds)
[16:41:28]: ▸ Test Suite OUDSTokensOpacityUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:41:33]: ▸ ✔ [OUDSTokensOpacityUITests] testAllOpacitiesInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.511 seconds)
[16:41:33]: ▸ ✔ [OUDSTokensOpacityUITests] testAllOpacitiesInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.119 seconds)
[16:41:33]: ▸ ✔ [OUDSTokensOpacityUITests] testAllOpacitiesOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.139 seconds)
[16:41:34]: ▸ ✔ [OUDSTokensOpacityUITests] testAllOpacitiesOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.135 seconds)
[16:41:34]: ▸ Test Suite OUDSTokensBorderUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:41:34]: ▸ ✔ [OUDSTokensBorderUITests] testAllBordersInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.386 seconds)
[16:41:35]: ▸ ✔ [OUDSTokensBorderUITests] testAllBordersInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.291 seconds)
[16:41:38]: ▸ ✔ [OUDSTokensBorderUITests] testAllBordersOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.370 seconds)
[16:41:38]: ▸ ✔ [OUDSTokensBorderUITests] testAllBordersOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (0.253 seconds)
[16:41:39]: ▸ Test Suite OUDSTokensSpaceUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:41:43]: ▸ ✔ [OUDSTokensSpaceUITests] testAllSpaceInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.751 seconds)
[16:41:43]: ▸ ✔ [OUDSTokensSpaceUITests] testAllSpaceInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.348 seconds)
[16:41:44]: ▸ ✔ [OUDSTokensSpaceUITests] testAllSpacesOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.428 seconds)
[16:42:42]: ▸ ✔ [OUDSTokensSpaceUITests] testAllSpacesOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.564 seconds)
[16:42:43]: ▸ Test Suite OUDSTokensTypographyUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:44:09]: ▸ ::error :: testAllTypographiesInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (104.090 seconds)
[16:44:18]: ▸ ::error :: testAllTypographiesInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (42.247 seconds)
[16:44:23]: ▸ ::error :: testAllTypographiesOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (8.834 seconds)
[16:44:41]: ▸ ::error :: testAllTypographiesOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (4.303 seconds)
[16:44:42]: ▸ Test Suite OUDSTokensColorUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:44:43]: ▸ ✔ [OUDSTokensColorUITests] testAllColorsInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (4.323 seconds)
[16:44:43]: ▸ ✔ [OUDSTokensColorUITests] testAllColorsInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (4.951 seconds)
[16:44:44]: ▸ ✔ [OUDSTokensColorUITests] testAllColorsOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (4.311 seconds)
[16:44:44]: ▸ ✔ [OUDSTokensColorUITests] testAllColorsOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (4.065 seconds)
[16:44:44]: ▸ Test Suite OUDSTokensSizeUITests started on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)'
[16:44:47]: ▸ ::error :: testAllSizeInverseThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (2.872 seconds)
[16:44:50]: ▸ ::error :: testAllSizesInverseThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (3.435 seconds)
[16:44:52]: ▸ ::error :: testAllSizesOrangeThemeDark on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (2.221 seconds)
[16:44:54]: ▸ ::error :: testAllSizesOrangeThemeLight on 'Clone 1 of iPhone 16 Pro - OUDS Showcase (11989)' (1.749 seconds)
Idea of @ludovic35: check the l10n of the views in the used simulators. Maybe the simulator is not in the same locale as our, and the rendring of the view with the displayed token text are not the same.
Description
Today we use swift-snapshot-testing to make assertions on UI rendering of views.
We have Fastlane running these tests and it work as expected.
We have our internal GitLab CI runner, running the same Fastlane lanes, and it works like expected.
However in GitHub Actions some tests do not pass, even if they pass with the other environment, without further details.
It seems the provided environment with GitHub runners is unstable making some test fail (even if they pass on our GitLab CI and local computers), e.g.
testAllTypographies*
,testAllSizes*
,testAllElevations*
don't pass in GitHub Actions level but pass in our internal levels.We do not know if there are hardware resources issues, shadow tomeouts or other configurations trobules witht he Mac runners or the in-use simulators.
Thus we will have checks failed for UI tests and must disable requirement son them to preven pull request to be merged.
This is a bit sad and we should find a workaround or a fix for this issue.
This is not mandatory as we plan to move the demo app, with the UI tests, in out internal GitLab software forge in the future.
But could be nice to have a fix befoire.
Details
Below are the outputs of our last job:
In attachments the logs.
logs_31306604165.zip
job-logs.txt
The text was updated successfully, but these errors were encountered: