Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 16 Regressions on 9/29/2022 7:42:19 AM #8834

Closed
performanceautofiler bot opened this issue Oct 4, 2022 · 2 comments
Closed

[Perf] Linux/x64: 16 Regressions on 9/29/2022 7:42:19 AM #8834

performanceautofiler bot opened this issue Oct 4, 2022 · 2 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue Look Again mono-aot os-linux perf-regression runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 70b33f8f69128eeb1463691e24646987b54a0e3d
Compare 561b0974793ba91936f151eab07875938e853694
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 9.33 μs 11.03 μs 1.18 0.02 False
BuildProvider - Duration of single invocation 2.88 μs 6.35 μs 2.21 0.02 False
BuildProvider - Duration of single invocation 2.94 μs 6.30 μs 2.14 0.04 False
Scoped - Duration of single invocation 10.27 μs 12.08 μs 1.18 0.02 False
Singleton - Duration of single invocation 9.55 μs 11.07 μs 1.16 0.03 False
Transient - Duration of single invocation 9.27 μs 10.93 μs 1.18 0.02 False
Singleton - Duration of single invocation 9.47 μs 11.19 μs 1.18 0.02 False
BuildProvider - Duration of single invocation 2.90 μs 6.34 μs 2.18 0.02 False
Scoped - Duration of single invocation 10.08 μs 12.16 μs 1.21 0.02 False
Singleton - Duration of single invocation 9.54 μs 11.17 μs 1.17 0.01 False
Singleton - Duration of single invocation 9.38 μs 11.21 μs 1.20 0.02 False
Scoped - Duration of single invocation 9.93 μs 12.20 μs 1.23 0.04 False
Scoped - Duration of single invocation 10.06 μs 11.63 μs 1.16 0.03 False
BuildProvider - Duration of single invocation 2.89 μs 6.32 μs 2.18 0.05 False
Transient - Duration of single invocation 9.32 μs 11.00 μs 1.18 0.02 False
Transient - Duration of single invocation 9.56 μs 11.03 μs 1.15 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.028203907674772 > 9.750907226190371.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -43.39242794180458 (T) = (0 -11074.65086254494) / Math.Sqrt((17925.291632025237 / (37)) + (30964.369193820112 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -0.1880045573688928 = (9322.060924641806 - 11074.65086254494) / 9322.060924641806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.35141907905675 > 3.0797143570432564.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -112.78992653028956 (T) = (0 -6367.801813720458) / Math.Sqrt((749.1328639940182 / (37)) + (24714.516098949825 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -1.1823741381631412 = (2917.832328731729 - 6367.801813720458) / 2917.832328731729 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.3044804520587014 > 3.0801392965271126.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -149.712777088843 (T) = (0 -6368.212465220779) / Math.Sqrt((1480.4843306974449 / (37)) + (13355.686762426954 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -1.191084707939603 = (2906.4200220762614 - 6368.212465220779) / 2906.4200220762614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.08143350945017 > 10.723652288548049.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -53.33283145852039 (T) = (0 -11877.387871024705) / Math.Sqrt((16867.41704192701 / (37)) + (15537.673721541803 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.17060741758886772 = (10146.345984624706 - 11877.387871024705) / 10146.345984624706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.069895998594628 > 10.019967383762015.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -50.80182768242689 (T) = (0 -11201.508408837648) / Math.Sqrt((13343.512027992007 / (37)) + (20609.359816950415 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -0.17930943728218238 = (9498.362393039493 - 11201.508408837648) / 9498.362393039493 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.927934357881922 > 9.849021889791292.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -58.070718991622414 (T) = (0 -11022.482242927377) / Math.Sqrt((20232.915223374508 / (37)) + (8594.246752639197 / (28))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (28) - 2, .025) and -0.18194803460804573 = (9325.69107962739 - 11022.482242927377) / 9325.69107962739 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.193770886919697 > 9.948830410763932.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -44.853374167209545 (T) = (0 -11162.345775474038) / Math.Sqrt((21943.352563647004 / (37)) + (21636.69117934942 / (29))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (29) - 2, .025) and -0.17239713481142085 = (9520.959616870347 - 11162.345775474038) / 9520.959616870347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.339758519757892 > 3.0600407647458554.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -165.01379133647632 (T) = (0 -6358.50117818002) / Math.Sqrt((1425.8675047627685 / (37)) + (11516.806297947629 / (29))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (29) - 2, .025) and -1.1818853486400844 = (2914.223326236238 - 6358.50117818002) / 2914.223326236238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.160394564933934 > 10.726494324010662.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -48.91134465120908 (T) = (0 -11873.7603315614) / Math.Sqrt((18652.67086041883 / (37)) + (20347.05136510833 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.1733802456244281 = (10119.277511138462 - 11873.7603315614) / 10119.277511138462 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.165471290309705 > 10.011231517649584.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -49.475447251577705 (T) = (0 -11183.547406738411) / Math.Sqrt((16842.01405609768 / (37)) + (18120.72431449636 / (28))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (28) - 2, .025) and -0.17217216339229607 = (9540.874417605124 - 11183.547406738411) / 9540.874417605124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.20705226512739 > 9.912926952016134.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -60.67783874205658 (T) = (0 -11165.218533403367) / Math.Sqrt((12750.681786378342 / (37)) + (11443.272635346028 / (29))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (29) - 2, .025) and -0.1733734759827295 = (9515.48570164518 - 11165.218533403367) / 9515.48570164518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.199675126130828 > 10.481116258786713.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -56.464453740572814 (T) = (0 -11924.193059021803) / Math.Sqrt((14892.065134953536 / (37)) + (16044.223131023578 / (25))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (25) - 2, .025) and -0.18066624219752705 = (10099.546029898998 - 11924.193059021803) / 10099.546029898998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.629587986880702 > 10.602972321114764.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -41.25373736411041 (T) = (0 -11881.155386706117) / Math.Sqrt((24658.250644318636 / (37)) + (30716.967870207 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -0.1729941123977417 = (10128.913061992784 - 11881.155386706117) / 10128.913061992784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.315148192414845 > 3.0299267710059588.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -142.81023939347713 (T) = (0 -6359.317985745836) / Math.Sqrt((2265.721541479155 / (37)) + (13578.669454248009 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -1.1855930627665472 = (2909.6532625776836 - 6359.317985745836) / 2909.6532625776836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.996084392393321 > 9.846998313908243.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -49.370921166294245 (T) = (0 -11054.444884595405) / Math.Sqrt((17026.645457268212 / (37)) + (20665.21819697938 / (28))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (28) - 2, .025) and -0.18286884975225481 = (9345.452699097365 - 11054.444884595405) / 9345.452699097365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.033207551601972 > 9.872134977265034.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -42.576251162814 (T) = (0 -11052.26370130748) / Math.Sqrt((16545.904746826516 / (37)) + (28752.825130359186 / (25))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (25) - 2, .025) and -0.18197794077123264 = (9350.651412407877 - 11052.26370130748) / 9350.651412407877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 70b33f8f69128eeb1463691e24646987b54a0e3d
Compare 561b0974793ba91936f151eab07875938e853694
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BuildProvider - Duration of single invocation 3.12 μs 6.51 μs 2.08 0.05 False
Scoped - Duration of single invocation 11.61 μs 13.13 μs 1.13 0.04 False
Scoped - Duration of single invocation 11.54 μs 12.77 μs 1.11 0.06 False
Scoped - Duration of single invocation 11.35 μs 13.00 μs 1.14 0.08 False
Transient - Duration of single invocation 10.43 μs 12.04 μs 1.15 0.07 False
Singleton - Duration of single invocation 10.87 μs 12.05 μs 1.11 0.05 False
Transient - Duration of single invocation 10.37 μs 11.81 μs 1.14 0.06 False
BuildProvider - Duration of single invocation 3.06 μs 6.55 μs 2.14 0.05 False
Singleton - Duration of single invocation 10.86 μs 11.80 μs 1.09 0.03 False
BuildProvider - Duration of single invocation 3.05 μs 6.45 μs 2.11 0.03 False
Singleton - Duration of single invocation 10.66 μs 12.09 μs 1.13 0.04 False
Scoped - Duration of single invocation 11.13 μs 12.65 μs 1.14 0.06 False
BuildProvider - Duration of single invocation 3.08 μs 6.58 μs 2.14 0.03 False
Singleton - Duration of single invocation 10.62 μs 12.20 μs 1.15 0.05 False
Transient - Duration of single invocation 10.54 μs 11.96 μs 1.13 0.04 False
Transient - Duration of single invocation 10.27 μs 11.99 μs 1.17 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.509905153179189 > 3.2757807195811486.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -249.39595965426696 (T) = (0 -6351.695801257914) / Math.Sqrt((2637.2386860287475 / (37)) + (2549.889905068532 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -1.0448454521493513 = (3106.198463351645 - 6351.695801257914) / 3106.198463351645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.12738732467779 > 11.912810113754512.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -26.461619672575395 (T) = (0 -12932.163783003663) / Math.Sqrt((38286.79656807663 / (37)) + (61536.95077401679 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.135511931084416 = (11388.840072031144 - 12932.163783003663) / 11388.840072031144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.765364440837603 > 12.127289168241767.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -22.972770161778453 (T) = (0 -12812.627383104033) / Math.Sqrt((87632.14042470738 / (37)) + (25632.56768338595 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.11587565382037456 = (11482.128263340097 - 12812.627383104033) / 11482.128263340097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.99552962258902 > 12.080838989598606.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -23.58295636848544 (T) = (0 -12833.026509228805) / Math.Sqrt((60843.05626698476 / (37)) + (52018.29166336349 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.12479522435073669 = (11409.211411469485 - 12833.026509228805) / 11409.211411469485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.040242600034203 > 11.06005571510255.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -34.948073860303246 (T) = (0 -11925.439253994022) / Math.Sqrt((31775.436249140315 / (37)) + (21711.39537680261 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -0.1357243070360871 = (10500.294111971574 - 11925.439253994022) / 10500.294111971574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.054110849622774 > 11.3231830515768.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -26.77380910885867 (T) = (0 -11977.711376594469) / Math.Sqrt((59471.975217297804 / (37)) + (20753.52742860024 / (25))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (25) - 2, .025) and -0.12404895972931972 = (10655.86269434278 - 11977.711376594469) / 10655.86269434278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.809771795248489 > 10.967614428775782.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -32.57147267165157 (T) = (0 -11939.478173609306) / Math.Sqrt((46811.15044504699 / (37)) + (20196.159863619654 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.14060803352201 = (10467.643417118641 - 11939.478173609306) / 10467.643417118641 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.549315905577187 > 3.285525732219332.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -196.9598324983278 (T) = (0 -6365.32673436439) / Math.Sqrt((6623.497922114704 / (37)) + (2444.6827909099225 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -1.0462159887939322 = (3110.779492108358 - 6365.32673436439) / 3110.779492108358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.803578997766381 > 11.157691911757956.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -36.188734966131506 (T) = (0 -11974.522920977897) / Math.Sqrt((30288.5932231201 / (37)) + (18835.003505143744 / (28))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (28) - 2, .025) and -0.1321269002689658 = (10577.0147481992 - 11974.522920977897) / 10577.0147481992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.45162713707786 > 3.23107249669175.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -277.3904822552114 (T) = (0 -6355.3391501103315) / Math.Sqrt((2289.110831473815 / (37)) + (2062.6874385641927 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -1.0543327035012091 = (3093.6270153704395 - 6355.3391501103315) / 3093.6270153704395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.092008445712308 > 11.222682650888368.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -31.8761440807824 (T) = (0 -12052.373718801655) / Math.Sqrt((30663.155713158652 / (36)) + (31256.852258194853 / (27))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (27) - 2, .025) and -0.13450361316075857 = (10623.477597593019 - 12052.373718801655) / 10623.477597593019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.645678048944166 > 12.020442477491123.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -22.31545399656128 (T) = (0 -12859.763945662417) / Math.Sqrt((52168.90924323884 / (37)) + (72924.55387565734 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.12696026258698934 = (11411.018092281474 - 12859.763945662417) / 11411.018092281474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.576856740310529 > 3.2656253073771953.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -233.4987650083632 (T) = (0 -6360.478139447621) / Math.Sqrt((2931.684208687332 / (37)) + (3221.6331967195824 / (28))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (28) - 2, .025) and -1.0479610152956222 = (3105.7613362476477 - 6360.478139447621) / 3105.7613362476477 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.196433932618548 > 11.168739095774503.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -29.026393688814398 (T) = (0 -12013.879810351293) / Math.Sqrt((48835.14715379031 / (37)) + (20717.328124535838 / (27))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (27) - 2, .025) and -0.12407515224627332 = (10687.790568400693 - 12013.879810351293) / 10687.790568400693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.957796745675033 > 11.042549732252633.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -30.892060232469 (T) = (0 -11944.508377292108) / Math.Sqrt((26158.584369037304 / (37)) + (38878.9507186637 / (25))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (25) - 2, .025) and -0.14026332931260424 = (10475.219250006689 - 11944.508377292108) / 10475.219250006689 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.990102222364822 > 10.87620069562255.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -45.88932840899749 (T) = (0 -11906.672661760964) / Math.Sqrt((25042.381542672112 / (37)) + (8714.306107717377 / (26))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (26) - 2, .025) and -0.13973770822584106 = (10446.853320572627 - 11906.672661760964) / 10446.853320572627 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@naricc naricc removed the untriaged label Oct 12, 2022
@naricc naricc mentioned this issue Oct 12, 2022
Closed
@naricc
Copy link

naricc commented Oct 12, 2022

It looks like this one: dotnet/runtime@4287453

@naricc naricc mentioned this issue Oct 26, 2022
Closed
@naricc naricc closed this as completed Jan 31, 2023
@naricc naricc added the LinkedFromMaster Is this issue linked from the master perf issue label Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue Look Again mono-aot os-linux perf-regression runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@naricc