[mono][perf] iOS Mono HelloWorld startup regression

[matouskozak]

Description

Mono regressed in startup times after updating OSX version to osx.13 (4dffd80).

Update: at the same time, xcode and iOS were updated to versions 15 and 17, respectively.

Date	Diff	Change	Slowdown (ms)	Slowdown (%)
16 Dec 2023	2284495...4dffd80	4dffd80	8.3	18.8

Affected scenarios:

• iOS Mono
• iOS nativeAOT
• MAUI (TBC - perf measurements not working)

---

cc: @ivanpovazan @LoopedBard3
fyi: @SamMonoRT @akoeplinger @kotlarmilos

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Mono regressed in startup times after updating OSX to osx.13 (4dffd80)

Date	Diff	Change	Slowdown (ms)	Slowdown (%)
16 Dec 2023	2284495...4dffd80	4dffd80	8.3	18.8

Affected scenarios:

• iOS Mono
• iOS nativeAOT
• MAUI (TBC - perf measurements not working)

---

cc: @ivanpovazan @LoopedBard3
fyi: @SamMonoRT @akoeplinger @kotlarmilos

Author:	matouskozak
Assignees:	-
Labels:	area-Infrastructure-mono, perf-pipeline
Milestone:	-

[SamMonoRT]

cc @rolfbjarne fyi - this seems like an OS upgrade causing the regression.

[ghost]

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger, @kotlarmilos
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Mono regressed in startup times after updating OSX version to osx.13 (4dffd80)

Date	Diff	Change	Slowdown (ms)	Slowdown (%)
16 Dec 2023	2284495...4dffd80	4dffd80	8.3	18.8

Affected scenarios:

• iOS Mono
• iOS nativeAOT
• MAUI (TBC - perf measurements not working)

---

cc: @ivanpovazan @LoopedBard3
fyi: @SamMonoRT @akoeplinger @kotlarmilos

Author:	matouskozak
Assignees:	-
Labels:	area-Infrastructure-mono, os-ios, perf-pipeline
Milestone:	9.0.0

[rolfbjarne]

Is this testing on an actual device? Because it would be really weird for the macOS version used to build the app would affect the launch performance of the app on the device.

[matouskozak]

Is this testing on an actual device? Because it would be really weird for the macOS version used to build the app would affect the launch performance of the app on the device.

I believe the test are running on real iPhones in the lab (@LoopedBard3 correct me if I am wrong).

[LoopedBard3]

Yes, these tests are running on the devices themselves. At the same time we updated to OSX.13 and to xcode 15, we also updated to the iOS devices to run iOS 17.

[rolfbjarne]

Not sure if there's anything for us to do here, since this seems to be a regression in iOS itself.

[matouskozak]

@LoopedBard3 Not sure what happened, but the measurements in the lab started to get quite noisy for these devices

[matouskozak]

Based on the latest information, the regression is still present and most likely caused by the OS version updated. The previous graphs were showing measurements of two devices with different configurations combined. After studying the measurements individually, we can see that the root cause is most likely the OS version change. The devices in the graph below are using the following configuration:
Perfios-01/Mac044: 17.0.2 build 21A351
Perfios-02/Mac045: 14.2.1 build 18B121