[release/8.0-staging] Manually depad RSAES-PKCS1 on Apple OSes

[github-actions]

Backport of #97738 to release/8.0-staging

/cc @bartonjs

Customer Impact

• Customer reported
• Found internally

Apple macOS 14.3 and Apple iOS/iPadOS 17.3 have applied a variant of the OpenSSL 3.2 "implicit rejection" feature where rather than reporting a decryption failure for RSAEncryptionPadding.Pkcs1 they report success and return random data. As with the similar change on Linux, this change breaks routines which depend on the failure for correctness.

Regression

• Yes
• No
• OS Behavior Change

Apple macOS 14.3 and Apple iOS/iPadOS 17.3 have made this change intentionally, but this manifests as a functional regression in .NET applications depending on the current behavior.

Testing

This OS regression was caught by existing tests. New tests have also been added to verify the full reliability of the fix.

Risk

Low. The fix is in an area that we already have test coverage.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Backport of #97738 to release/8.0-staging

/cc @bartonjs

Customer Impact

• Customer reported
• Found internally

[Select one or both of the boxes. Describe how this issue impacts customers, citing the expected and actual behaviors and scope of the issue. If customer-reported, provide the issue number.]

Regression

• Yes
• No

[If yes, specify when the regression was introduced. Provide the PR or commit if known.]

Testing

[How was the fix verified? How was the issue missed previously? What tests were added?]

Risk

[High/Medium/Low. Justify the indication by mentioning how risks were measured and addressed.]

IMPORTANT: If this backport is for a servicing release, please verify that:

• The PR target branch is release/X.0-staging, not release/X.0.

• If the change touches code that ships in a NuGet package, you have added the necessary package authoring and gotten it explicitly reviewed.

Author:	github-actions[bot]
Assignees:	-
Labels:	area-System.Security
Milestone:	-