Skip linker-output-non-utf8 test on Apple #47289
Conversation
This test fails on APFS filesystems with the following error: mkdir: /Users/ryan/Code/rust/build/x86_64-apple-darwin/test/run-make/linker-output-non-utf8.stage2-x86_64-apple-darwin/zzz�: Illegal byte sequence This is due to APFS now requiring that all paths are valid UTF-8. As APFS will be the default filesystem for all new Darwin-based systems the most straightforward fix is to skip this test on Darwin as well as Windows.
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @aturon (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
|
||
| else | ||
| # This does not work in its current form on Windows or Apple APFS due | ||
| # to their filesystems requiring paths to be valid Unicode. |
There was a problem hiding this comment.
Are you sure Windows require paths to be valid Unicode? If not, please retain the original comment about Windows, and open a new sentence about the valid Unicode requirement with APFS.
There was a problem hiding this comment.
Yes, it needs to parse the path as Unicode to perform normalisation and case folding on NTFS and ReFS. MSDN covers this briefly
There was a problem hiding this comment.
This is not true - when MSDN refers to "unicode" it means un-validated UCS-2 - case mapping ignores invalid sequences. @retep998 probably knows more details.
There was a problem hiding this comment.
Windows does not perform unicode normalization on paths. Case folding is also done via a highly simplistic u16 to u16 mapping which will handle lone surrogates just fine. Windows uses WTF-16 for paths, which is a superset of UTF-16.
There was a problem hiding this comment.
Thanks for the correction. I'll switch the comment back.
Don't want to be nitpicky but maybe this is the case for Darwin based operating systems from Apple but the APFS driver is not included in the Darwin kernel itself and Apple won't publish its source code, so most likely non-Apple Darwin distributions (like PureDarwin) won't get any APFS support soon, let alone APFS being the default filesystem. In fact it seems that Apple has removed the HFS driver from its kernel releases and moved it to its own project. |
kennytm
added
the
S-waiting-on-review
|
@est31 That's fair. I was using Darwin to refer to the macOS/iOS/tvOS/watchOS family of operating systems but you're correct for other Darwin forks. The fact that this test runs on Linux should be enough to ensure that the compiler is dealing with non-UTF-8 paths correctly internally. I don't think it's a major loss to not run this test on platforms that could potentially support it, especially considering the complexity of determining the target filesystem type. |
The Windows situation is more complicated than I realised
|
@kennytm Restored the original comment |
|
|
||
| else | ||
| # This also does not work on Apple APFS due to the filesystem requiring | ||
| # valid UTF-u paths. |
|
Thank you! @bors r+ rollup |
|
📌 Commit b69c320 has been approved by |
kennytm
added
S-waiting-on-bors
…-test-on-apple, r=kennytm Skip linker-output-non-utf8 test on Apple This test fails on APFS filesystems with the following error: ```shell mkdir: /Users/ryan/Code/rust/build/x86_64-apple-darwin/test/run-make/linker-output-non-utf8.stage2-x86_64-apple-darwin/zzz�: Illegal byte sequence ``` The mkdir does succeed on an HFS+ volume mounted on the same system: ```shell $ mkdir zzz$$'\xff' $ ls zzz47432\xff ``` This is due to APFS now requiring that all paths are valid UTF-8. As APFS will be the default filesystem for all new Darwin-based systems the most straightforward fix is to skip this test on Darwin as well as Windows.
…-test-on-apple, r=kennytm Skip linker-output-non-utf8 test on Apple This test fails on APFS filesystems with the following error: ```shell mkdir: /Users/ryan/Code/rust/build/x86_64-apple-darwin/test/run-make/linker-output-non-utf8.stage2-x86_64-apple-darwin/zzz�: Illegal byte sequence ``` The mkdir does succeed on an HFS+ volume mounted on the same system: ```shell $ mkdir zzz$$'\xff' $ ls zzz47432\xff ``` This is due to APFS now requiring that all paths are valid UTF-8. As APFS will be the default filesystem for all new Darwin-based systems the most straightforward fix is to skip this test on Darwin as well as Windows.
…-test-on-apple, r=kennytm Skip linker-output-non-utf8 test on Apple This test fails on APFS filesystems with the following error: ```shell mkdir: /Users/ryan/Code/rust/build/x86_64-apple-darwin/test/run-make/linker-output-non-utf8.stage2-x86_64-apple-darwin/zzz�: Illegal byte sequence ``` The mkdir does succeed on an HFS+ volume mounted on the same system: ```shell $ mkdir zzz$$'\xff' $ ls zzz47432\xff ``` This is due to APFS now requiring that all paths are valid UTF-8. As APFS will be the default filesystem for all new Darwin-based systems the most straightforward fix is to skip this test on Darwin as well as Windows.
This test fails on APFS filesystems with the following error:
The mkdir does succeed on an HFS+ volume mounted on the same system:
This is due to APFS now requiring that all paths are valid UTF-8. As APFS will be the default filesystem for all new Darwin-based systems the most straightforward fix is to skip this test on Darwin as well as Windows.