Deprecated the old python infrastructure in favor of the new C# Infra #3281

mrsharm · 2023-08-17T23:49:37Z

Deprecates all python code, documentation and other misc tooling related to the python infrastructure.
GCPerfSim and the Environment based scripts are moved out of the python infrastructure.

markples · 2023-08-18T00:18:41Z

@mrsharm Are there any parts that you think deserve a closer look?

From a quick look,

Is the C# code (managed-lib[-executable]) fully a utility of the Python code?
Do the .md files (and associated graphics) have concepts that are still needed when using the C# infrastructure?

mrsharm · 2023-08-18T01:09:48Z

@markples:

Based on our testing so far, we should be 1 to 1 with the python infrastructure. There were some aspects such as the weighted scores that weren't considered to be important to move to the C# infra.
The md files are completely redundant as they were based on the old infra.

Since you most recently ran the infrastructure, please feel free to start creating issues highlight and suggestions for documentation.

* Update dependencies from https://github.com/dotnet/runtime build 20230817.12 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23417.4 -> To Version 8.0.0-rc.2.23417.12 * Update dependencies from https://github.com/dotnet/runtime build 20230817.23 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23417.4 -> To Version 8.0.0-rc.2.23417.23 --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

#3285) * Added more documentation for running notebooks and message that we'll be deprecating the python infra * Update src/benchmarks/gc/GC.Infrastructure/README.md Co-authored-by: Mark Plesko <markples@microsoft.com> * Fixed another linting issue --------- Co-authored-by: Mark Plesko <markples@microsoft.com>

…0818.14 (#3286) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23417.23 -> To Version 8.0.0-rc.2.23418.14 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…817.3 (#3283) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23415.4 -> To Version 8.0.0-beta.23417.3 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

Added WasmInterpreter and WasmAOT to the local testing script. This additionally included breaking the run BDN and associated steps into multi-run and single-run setups to support not being able to pass in multiple Wasm workloads to run simultaneously. Also, some formatting was better standardized for cleaner code.

…0823.11 (#3294) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23418.14 -> To Version 8.0.0-rc.2.23423.11 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…822.1 (#3291) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23419.1 -> To Version 8.0.0-beta.23422.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* [wasm] Don't strip native symbols * precommands.py: Fail if 'dotnet workload install' fails * RunCommand: Fix retry being off by 1

* Update dependencies from https://github.com/dotnet/runtime build 20230823.5 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23423.11 -> To Version 8.0.0-rc.2.23423.5 * Update dependencies from https://github.com/dotnet/runtime build 20230825.9 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23423.11 -> To Version 8.0.0-rc.2.23425.9 --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* ComWrappers RCW parallel access * Retain the RCW as a member. * Apply suggestions from code review Co-authored-by: Jan Kotas <jkotas@microsoft.com> * Review feedback. * Fix style * Review feedback * Disable for mono --------- Co-authored-by: Jan Kotas <jkotas@microsoft.com>

…825.2 (#3299) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23422.1 -> To Version 8.0.0-beta.23425.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…0826.4 (#3300) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23425.9 -> To Version 8.0.0-rc.2.23426.4 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

markples

Since anyone using the python infra may end up as this PR, I wanted to point out some of the new documentation locations in the repo: src\benchmarks\gc\GC.Infrastructure\README.md has a lot of information, and then src\benchmarks\gc\GC.Infrastructure\GC.Analysis.API\README.md, src\benchmarks\gc\GC.Infrastructure\Notebooks\README.md, and src\benchmarks\micro\README.md have more.

I haven't done a big scan, but there are probably a few more references to the python infra (like "Jupyter" in GC.Analysis.API\README.md), but I also don't think this needs to be 100% to merge.

…828.2 (#3303) [main] Update dependencies from dotnet/arcade

* Update dependencies from https://github.com/dotnet/runtime build 20230829.12 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23426.4 -> To Version 8.0.0-rc.2.23429.12 * Update dependencies from https://github.com/dotnet/runtime build 20230829.20 Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23426.4 -> To Version 8.0.0-rc.2.23429.20 --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…829.1 (#3306) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23428.2 -> To Version 8.0.0-beta.23429.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…0830.5 (#3308) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23429.20 -> To Version 8.0.0-rc.2.23430.5 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…ts them via own LTTng tracepoint provider that works against the updated ABI (#3170)

* Fixed the reporting of the GCPerfSim runs * Fixed computation of Gen2 Blocking and BGCs

…runtime js files (#3309)

…0831.9 (#3310) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23430.5 -> To Version 8.0.0-rc.2.23431.9 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…831.2 (#3312) Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23429.1 -> To Version 8.0.0-beta.23431.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

#3311) * [wasm] Fix build broken by recent update to tfm=net9.0 on runtime Essentially, override the packs as part of the build so that the correct ones are used. This uses `WasmOverridePacks.targets` from runtime's Wasm.Build.Tests . Fixes the error where 9.0 wasm targets are used, but with 8.0 runtime pack from the sdk: ``` emcc : error : /home/helixbot/work/C6E60A3B/w/BEDB0A92/u/artifacts/packages/microsoft.netcore.app.runtime.mono.browser-wasm/8.0.0-rc.1.23414.4/runtimes/browser-wasm/native/libmono-wasm-simd.a: No such file or directory ("/home/helixbot/work/C6E60A3B/w/BEDB0A92/u/artifacts/packages/microsoft.netcore.app.runtime.mono.browser-wasm/8.0.0-rc.1.23414.4/runtimes/browser-wasm/native/libmono-wasm-simd.a" was expected to be an input file, based on the commandline arguments provided) [/home/helixbot/work/C6E60A3B/p/performance/src/scenarios/blazoraot/app/emptyblazorwasmtemplate.csproj] ``` * [wasm] Override packs for blazor scenarios * [wasm] Move WasmOverridePacks.targets to build-common, and update blazor targets * [wasm] Override packs only when using workloads

…0906.8 (#3324) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23431.9 -> To Version 8.0.0-rc.2.23456.8 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Update dependencies from https://github.com/dotnet/arcade build 20230901.1 Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23431.2 -> To Version 8.0.0-beta.23451.1 * Update dependencies from https://github.com/dotnet/arcade build 20230901.1 Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk From Version 8.0.0-beta.23431.2 -> To Version 8.0.0-beta.23451.1 --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

Add Xamarin android default app run to maui_android runs. Clarify naming with .NET Android rather than Xamarin Android.

Maoni0 · 2023-09-08T05:12:44Z

@cincuranet, the test failures aren't related to the GC changes, could you please take a look? this means I cannot merge it (obviously we'll resolve the conflicts first)

cincuranet · 2023-09-08T06:36:40Z

Yes, unrelated. I'll merge on red after the conflicts are resolved.

…0907.7 (#3327) Microsoft.Extensions.Logging , Microsoft.NET.ILLink , Microsoft.NET.ILLink.Tasks , System.Threading.Channels From Version 8.0.0-rc.2.23456.8 -> To Version 8.0.0-rc.2.23457.7 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

…performance into deprecating_python_infra

mrsharm · 2023-09-08T18:20:46Z

I think I messed something up here by rebasing to the latest changes in order to resolve merge conflicts.

Deprecated the old python infrastructure in favor of the new C# Infra

4b0a200

mrsharm requested review from cshung, markples and Maoni0 August 17, 2023 23:55

dotnet-maestro bot and others added 14 commits August 18, 2023 09:03

Code formatting. (#3289)

907c9c8

[wasm] Don't strip native symbols (#3288)

726f271

* [wasm] Don't strip native symbols * precommands.py: Fail if 'dotnet workload install' fails * RunCommand: Fix retry being off by 1

Update build container to 22.04 (#3301)

ac39cb5

markples previously approved these changes Aug 28, 2023

View reviewed changes

dotnet-maestro bot and others added 9 commits August 29, 2023 14:06

Update dependencies from https://github.com/dotnet/arcade build 20230…

3095a50

…828.2 (#3303) [main] Update dependencies from dotnet/arcade

EventListener that listens for the events we care about and then emi…

bbce3d0

…ts them via own LTTng tracepoint provider that works against the updated ABI (#3170)

Fixed the reporting of the GCPerfSim runs in the Repro Steps (#3305)

3ad0477

* Fixed the reporting of the GCPerfSim runs * Fixed computation of Gen2 Blocking and BGCs

SizeOnDisk: fix up version replacement for dotnet.native, and dotnet.…

0a0cc95

…runtime js files (#3309)

adamsitnik and others added 7 commits September 1, 2023 16:03

Frozen collection benchmarks (#3089)

a908ec8

Fix TargetsWindowsParam on non-Windows. (#3315)

24bfc1a

Add missing documentation lines for command-line arguments (#3323)

efc21ba

Add Xamarin android default app run to maui_android runs. (#3302)

cc05d37

Add Xamarin android default app run to maui_android runs. Clarify naming with .NET Android rather than Xamarin Android.

dotnet-maestro bot and others added 2 commits September 8, 2023 18:41

Fixed merge conflicts

27d51c7

mrsharm dismissed markples’s stale review via 27d51c7 September 8, 2023 17:39

mrsharm added 3 commits September 8, 2023 10:43

Deprecated the old python infrastructure in favor of the new C# Infra

098bddd

Fixed merge conflicts

281752d

Merge branch 'deprecating_python_infra' of https://github.com/dotnet/…

0610605

…performance into deprecating_python_infra

mrsharm closed this Sep 8, 2023

mrsharm mentioned this pull request Sep 8, 2023

Deprecated the old python infrastructure in favor of the new C# Infra #3330

Merged

LoopedBard3 deleted the deprecating_python_infra branch March 11, 2024 20:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecated the old python infrastructure in favor of the new C# Infra #3281

Deprecated the old python infrastructure in favor of the new C# Infra #3281

mrsharm commented Aug 17, 2023

markples commented Aug 18, 2023

mrsharm commented Aug 18, 2023

markples left a comment

Maoni0 commented Sep 8, 2023

cincuranet commented Sep 8, 2023

mrsharm commented Sep 8, 2023

Deprecated the old python infrastructure in favor of the new C# Infra #3281

Deprecated the old python infrastructure in favor of the new C# Infra #3281

Conversation

mrsharm commented Aug 17, 2023

markples commented Aug 18, 2023

mrsharm commented Aug 18, 2023

markples left a comment

Choose a reason for hiding this comment

Maoni0 commented Sep 8, 2023

cincuranet commented Sep 8, 2023

mrsharm commented Sep 8, 2023