Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize path mapping lookups #59048

Merged
merged 14 commits into from
Aug 29, 2024
Merged

Optimize path mapping lookups #59048

merged 14 commits into from
Aug 29, 2024
+63 −22

Conversation

DanielRosenwasser
Copy link
Member

@DanielRosenwasser DanielRosenwasser commented Jun 27, 2024 •
edited
Loading

When checking if a specifier satisfies a pattern (e.g. for a path mapping entry in paths), it seems like TypeScript needs to check every single available pattern to see if the specifier is matched.

Here, I've slightly modified the code to create a Set of exact specifiers, and an array of all remaining Patterns. The Set becomes a fast look-up, and the fallback becomes a walk across the array. A new difference is that once we've found a match, we don't bother checking any other patterns unless they have a longer prefix than the current match. That can help quickly dismiss irrelevant matches.

This also adds a caches so that we don't end up re-creating these objects over and over.

In earlier iterations of this PR, the array of patterns was sorted and lookups on patterns could optimistically be done via a binary search; however, there's some nuance there with respect to how paths differs from exports, and we don't have enough test coverage there (nothing fails if you only sort by prefix specifier length!). So for now, I've stuck with a linear search.

@DanielRosenwasser
Use a Set and a binary search to find longest-prefix matches more q…
9de8ff3 
…uickly.

This also adds a naive cache for each call, though it is not resilient to being passed the same array of patterns twice with different contents, and could be defeated if the patterns frequently differ.
@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 27, 2024
@DanielRosenwasser
Copy link
Member Author

I don't know if our perf suites have repos with lots of path mappings, but...

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@DanielRosenwasser DanielRosenwasser changed the title Use a Set and a binary search to find longest-prefix matches more quickly Use a Set and a binary search to search for patterns on paths Jun 27, 2024
@DanielRosenwasser
Copy link
Member Author

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162456/artifacts?artifactName=tgz&fileId=E2F93981315A9DC4E4D25A7C90DF0240EEFB23C524E61C1D39685D480869765C02&fileName=/typescript-5.6.0-insiders.20240627.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-59048-4".;

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,759k (± 0.76%) 192,730k (± 0.74%) ~ 192,087k 195,648k p=0.575 n=6
Parse Time 1.97s (± 0.52%) 1.96s (± 0.53%) ~ 1.95s 1.98s p=0.134 n=6
Bind Time 1.05s (± 1.11%) 1.06s (± 1.19%) ~ 1.04s 1.07s p=0.281 n=6
Check Time 13.80s (± 0.47%) 13.87s (± 0.46%) ~ 13.76s 13.94s p=0.092 n=6
Emit Time 4.04s (± 0.88%) 4.03s (± 0.35%) ~ 4.01s 4.05s p=0.375 n=6
Total Time 20.86s (± 0.41%) 20.92s (± 0.28%) ~ 20.81s 20.97s p=0.422 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,376k (± 0.00%) 1,218,327k (± 0.00%) -49k (- 0.00%) 1,218,299k 1,218,372k p=0.037 n=6
Parse Time 6.67s (± 0.92%) 6.69s (± 0.35%) ~ 6.66s 6.72s p=0.747 n=6
Bind Time 1.86s (± 0.22%) 1.88s (± 1.19%) ~ 1.86s 1.92s p=0.073 n=6
Check Time 30.63s (± 0.35%) 30.60s (± 0.24%) ~ 30.47s 30.67s p=0.630 n=6
Emit Time 13.62s (± 0.19%) 13.56s (± 0.27%) -0.05s (- 0.40%) 13.50s 13.61s p=0.016 n=6
Total Time 52.78s (± 0.22%) 52.73s (± 0.16%) ~ 52.61s 52.84s p=0.335 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,132,386 2,132,386 ~ ~ ~ p=1.000 n=6
Types 926,170 926,170 ~ ~ ~ p=1.000 n=6
Memory used 2,114,898k (± 0.01%) 2,114,937k (± 0.01%) ~ 2,114,683k 2,115,049k p=0.630 n=6
Parse Time 7.89s (± 0.47%) 7.89s (± 0.19%) ~ 7.88s 7.91s p=1.000 n=6
Bind Time 2.74s (± 0.64%) 2.74s (± 0.46%) ~ 2.73s 2.76s p=0.731 n=6
Check Time 83.69s (± 0.49%) 83.47s (± 0.39%) ~ 83.18s 83.92s p=0.575 n=6
Emit Time 0.16s (± 4.99%) 0.16s (± 5.58%) ~ 0.15s 0.17s p=0.550 n=6
Total Time 94.49s (± 0.40%) 94.27s (± 0.35%) ~ 93.95s 94.71s p=0.378 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,231,673 +33 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,187 261,192 +5 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,348,423k (± 0.05%) 2,348,122k (± 0.04%) ~ 2,347,054k 2,349,167k p=0.689 n=6
Parse Time 6.00s (± 0.93%) 5.99s (± 0.87%) ~ 5.91s 6.04s p=0.748 n=6
Bind Time 2.27s (± 0.51%) 2.28s (± 1.09%) ~ 2.26s 2.33s p=0.244 n=6
Check Time 40.13s (± 0.28%) 40.08s (± 0.34%) ~ 39.89s 40.22s p=0.810 n=6
Emit Time 3.21s (± 2.66%) 3.21s (± 2.41%) ~ 3.11s 3.34s p=0.936 n=6
Total Time 51.63s (± 0.19%) 51.58s (± 0.23%) ~ 51.41s 51.74s p=0.470 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,231,673 +33 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,187 261,192 +5 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,424,816k (± 0.05%) 2,424,237k (± 0.03%) ~ 2,423,155k 2,425,368k p=0.689 n=6
Parse Time 6.22s (± 0.38%) 6.26s (± 1.00%) ~ 6.19s 6.37s p=0.261 n=6
Bind Time 2.05s (± 0.91%) 2.06s (± 1.14%) ~ 2.03s 2.09s p=1.000 n=6
Check Time 40.56s (± 0.11%) 40.52s (± 0.34%) ~ 40.33s 40.75s p=0.336 n=6
Emit Time 3.22s (± 2.09%) 3.20s (± 3.83%) ~ 3.10s 3.42s p=0.378 n=6
Total Time 52.06s (± 0.15%) 52.05s (± 0.15%) ~ 51.95s 52.18s p=1.000 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,818 258,840 +22 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,842 104,847 +5 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 428,229k (± 0.01%) 428,298k (± 0.01%) +69k (+ 0.02%) 428,256k 428,345k p=0.020 n=6
Parse Time 3.34s (± 0.57%) 3.31s (± 0.89%) ~ 3.26s 3.34s p=0.167 n=6
Bind Time 1.30s (± 1.84%) 1.32s (± 1.56%) ~ 1.29s 1.34s p=0.120 n=6
Check Time 17.80s (± 0.39%) 17.73s (± 0.39%) ~ 17.65s 17.81s p=0.227 n=6
Emit Time 1.37s (± 1.07%) 1.37s (± 0.55%) ~ 1.36s 1.38s p=0.934 n=6
Total Time 23.82s (± 0.21%) 23.73s (± 0.36%) ~ 23.61s 23.85s p=0.092 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,533k (± 0.03%) 369,460k (± 0.01%) ~ 369,392k 369,550k p=0.378 n=6
Parse Time 2.77s (± 1.28%) 2.76s (± 0.70%) ~ 2.73s 2.78s p=1.000 n=6
Bind Time 1.59s (± 0.65%) 1.60s (± 1.28%) ~ 1.58s 1.62s p=0.511 n=6
Check Time 15.49s (± 0.19%) 15.52s (± 0.32%) ~ 15.44s 15.58s p=0.294 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.27%) 19.88s (± 0.31%) ~ 19.77s 19.94s p=0.296 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,579 2,878,579 ~ ~ ~ p=1.000 n=6
Types 975,166 975,166 ~ ~ ~ p=1.000 n=6
Memory used 3,042,072k (± 0.00%) 3,042,036k (± 0.00%) ~ 3,041,971k 3,042,083k p=0.575 n=6
Parse Time 13.54s (± 0.39%) 13.55s (± 0.46%) ~ 13.47s 13.62s p=0.810 n=6
Bind Time 4.19s (± 0.18%) 4.19s (± 0.36%) ~ 4.18s 4.22s p=0.503 n=6
Check Time 73.36s (± 0.40%) 73.42s (± 0.26%) ~ 73.19s 73.66s p=0.575 n=6
Emit Time 23.91s (± 1.68%) 24.17s (± 0.94%) ~ 23.94s 24.55s p=0.128 n=6
Total Time 115.00s (± 0.32%) 115.32s (± 0.31%) ~ 114.92s 115.94s p=0.128 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,649k (± 0.01%) 411,579k (± 0.02%) ~ 411,443k 411,669k p=0.173 n=6
Parse Time 4.74s (± 0.56%) 4.70s (± 0.49%) -0.04s (- 0.77%) 4.67s 4.74s p=0.035 n=6
Bind Time 2.09s (± 0.68%) 2.08s (± 1.11%) ~ 2.05s 2.10s p=0.514 n=6
Check Time 20.78s (± 0.37%) 20.79s (± 0.54%) ~ 20.67s 20.98s p=0.747 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.61s (± 0.31%) 27.56s (± 0.41%) ~ 27.46s 27.76s p=0.470 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 463,064k (± 0.08%) 462,963k (± 0.06%) ~ 462,426k 463,213k p=0.173 n=6
Parse Time 2.65s (± 0.51%) 2.65s (± 0.50%) ~ 2.63s 2.67s p=0.801 n=6
Bind Time 0.98s (± 0.56%) 0.98s ~ ~ ~ p=0.071 n=6
Check Time 15.21s (± 0.48%) 15.17s (± 0.44%) ~ 15.10s 15.25s p=0.470 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.85s (± 0.39%) 18.81s (± 0.33%) ~ 18.73s 18.87s p=0.571 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,274ms (± 1.44%) 2,300ms (± 0.75%) ~ 2,272ms 2,314ms p=0.109 n=6
Req 2 - geterr 5,071ms (± 0.48%) 5,069ms (± 0.54%) ~ 5,042ms 5,109ms p=0.810 n=6
Req 3 - references 264ms (± 0.29%) 263ms (± 0.44%) ~ 262ms 265ms p=0.279 n=6
Req 4 - navto 226ms (± 1.33%) 228ms (± 0.18%) ~ 228ms 229ms p=0.390 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 88ms (± 6.70%) 83ms (± 7.76%) ~ 77ms 90ms p=0.157 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,417ms (± 0.54%) 2,428ms (± 1.26%) ~ 2,377ms 2,456ms p=0.297 n=6
Req 2 - geterr 3,792ms (± 0.16%) 3,798ms (± 0.26%) ~ 3,786ms 3,809ms p=0.297 n=6
Req 3 - references 278ms (± 0.30%) 278ms (± 0.29%) ~ 277ms 279ms p=0.718 n=6
Req 4 - navto 226ms (± 0.52%) 226ms (± 0.37%) ~ 225ms 227ms p=0.652 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 75ms (± 6.28%) 73ms (± 0.56%) ~ 73ms 74ms p=0.445 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,217ms (± 0.67%) 6,220ms (± 0.28%) ~ 6,200ms 6,242ms p=0.689 n=6
Req 2 - geterr 1,532ms (±11.81%) 1,698ms (± 0.84%) ~ 1,682ms 1,715ms p=0.173 n=6
Req 3 - references 109ms (± 9.33%) 113ms (± 3.81%) ~ 111ms 122ms p=0.171 n=6
Req 4 - navto 576ms (± 6.04%) 587ms (± 2.41%) ~ 574ms 611ms p=0.936 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,250ms (± 2.22%) 1,238ms (± 2.07%) ~ 1,212ms 1,287ms p=0.173 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 156.84ms (± 0.17%) 156.94ms (± 0.17%) +0.10ms (+ 0.06%) 155.80ms 160.68ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 283.97ms (± 0.28%) 284.00ms (± 0.29%) ~ 276.60ms 291.87ms p=0.420 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 227.89ms (± 0.14%) 228.08ms (± 0.16%) +0.18ms (+ 0.08%) 226.61ms 233.88ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 227.42ms (± 0.14%) 227.29ms (± 0.16%) -0.14ms (- 0.06%) 225.70ms 232.62ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

DanielRosenwasser commented Jun 27, 2024 •
edited
Loading

In some experiments on internal codebases, this seems to shave off maybe 200-300ms depending on the project (they are using pretty big lists of path mappings), but it looks like it ends up in a situation where the function gets called with two different arrays in an interleaving way. So instead of a single-slot cache, I added a WeakMap.

@typescript-bot test top400
@typescript-bot perf test this
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
perf test this ✅ Started 👀 Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162471/artifacts?artifactName=tgz&fileId=E81886FA4EB3F7B8BEB31252C9121DC5B98039962996BF8EF2C70B005CCCAED902&fileName=/typescript-5.6.0-insiders.20240627.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-59048-8".;

dmichon-msft
dmichon-msft reviewed Jun 27, 2024
View reviewed changes
src/compiler/core.ts Outdated
let matchedValue: T | undefined;
// use length of prefix as betterness criteria
let longestMatchPrefixLength = -1;

for (let i = 0; i < values.length; i++) {
for (let i = 0; i < endIndex; i++) {
const v = values[i];
const pattern = getPattern(v);
if (isPatternMatch(pattern, candidate) && pattern.prefix.length > longestMatchPrefixLength) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to run these conditions in the reverse order? No point in running the cost of pattern matching if the pattern to be matched is a worse fit than the current best.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice, what will probably happen is we'll have an empty prefix, that'll match, and then everything after will still need to be checked. It probably makes sense to walk the list backwards looking for a match in case a long match is found, and then bail out as soon as an entry with a shorter prefix length is encountered.

src/compiler/utilities.ts Outdated
return undefined;
}

let index = binarySearchKey(sortedPatterns, candidate, getPatternPrefix, compareStringsCaseSensitive);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Rush we have to efficiently match a large array of Git file paths to project prefixes as part of hashing our input states; for that we use this class which you might find useful for this application: https://github.com/microsoft/rushstack/blob/main/libraries/rush-lib/src/logic/LookupByPath.ts

Copy link
Contributor

@dmichon-msft dmichon-msft Jun 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some numbers, that tool matches 142936 unique file paths to 1279 unique path prefixes in 62 milliseconds on my codespace machine (single threaded). This is without taking any advantage of that the list of paths returned by Git are technically sorted.

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,340k (± 0.97%) 193,966k (± 1.02%) ~ 192,152k 195,829k p=0.173 n=6
Parse Time 1.57s (± 0.94%) 1.58s (± 0.26%) ~ 1.58s 1.59s p=0.210 n=6
Bind Time 0.86s (± 1.23%) 0.86s (± 1.04%) ~ 0.85s 0.87s p=0.452 n=6
Check Time 11.19s (± 0.28%) 11.23s (± 0.26%) +0.05s (+ 0.42%) 11.20s 11.28s p=0.030 n=6
Emit Time 3.27s (± 0.76%) 3.28s (± 0.57%) ~ 3.26s 3.31s p=0.292 n=6
Total Time 16.89s (± 0.23%) 16.96s (± 0.23%) +0.07s (+ 0.43%) 16.92s 17.02s p=0.024 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,353k (± 0.00%) 1,218,354k (± 0.00%) ~ 1,218,307k 1,218,402k p=0.688 n=6
Parse Time 7.95s (± 0.59%) 7.98s (± 0.77%) ~ 7.90s 8.09s p=0.332 n=6
Bind Time 2.22s (± 0.44%) 2.24s (± 0.34%) +0.02s (+ 0.90%) 2.23s 2.25s p=0.010 n=6
Check Time 35.75s (± 0.36%) 35.71s (± 0.31%) ~ 35.59s 35.86s p=0.630 n=6
Emit Time 16.19s (± 0.43%) 16.12s (± 0.98%) ~ 15.93s 16.40s p=0.148 n=6
Total Time 62.10s (± 0.27%) 62.05s (± 0.39%) ~ 61.76s 62.44s p=0.575 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,050 2,133,050 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,115,431k (± 0.00%) 2,115,624k (± 0.00%) +193k (+ 0.01%) 2,115,565k 2,115,656k p=0.005 n=6
Parse Time 6.64s (± 0.42%) 6.64s (± 0.21%) ~ 6.63s 6.66s p=0.935 n=6
Bind Time 2.33s (± 0.83%) 2.29s (± 0.24%) -0.04s (- 1.57%) 2.29s 2.30s p=0.008 n=6
Check Time 70.64s (± 0.39%) 70.88s (± 0.25%) ~ 70.66s 71.10s p=0.125 n=6
Emit Time 0.14s (± 2.95%) 0.14s (± 4.05%) ~ 0.13s 0.14s p=0.282 n=6
Total Time 79.75s (± 0.36%) 79.95s (± 0.24%) ~ 79.71s 80.19s p=0.229 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,231,694 +54 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,187 261,201 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,348,149k (± 0.06%) 2,348,480k (± 0.08%) ~ 2,346,242k 2,350,405k p=0.936 n=6
Parse Time 6.07s (± 0.96%) 6.01s (± 1.17%) ~ 5.91s 6.13s p=0.199 n=6
Bind Time 2.27s (± 0.56%) 2.31s (± 0.65%) +0.04s (+ 1.62%) 2.29s 2.33s p=0.005 n=6
Check Time 40.17s (± 0.48%) 40.12s (± 0.45%) ~ 39.90s 40.35s p=0.936 n=6
Emit Time 3.25s (± 4.29%) 3.24s (± 5.51%) ~ 3.05s 3.51s p=0.936 n=6
Total Time 51.77s (± 0.41%) 51.70s (± 0.72%) ~ 51.29s 52.32s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,231,694 +54 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,187 261,201 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,424,382k (± 0.04%) 2,424,553k (± 0.03%) ~ 2,423,438k 2,425,674k p=0.689 n=6
Parse Time 6.25s (± 0.78%) 6.24s (± 0.44%) ~ 6.20s 6.27s p=0.872 n=6
Bind Time 2.05s (± 0.82%) 2.05s (± 0.65%) ~ 2.03s 2.06s p=1.000 n=6
Check Time 40.62s (± 0.20%) 40.61s (± 0.27%) ~ 40.45s 40.78s p=0.748 n=6
Emit Time 3.21s (± 2.21%) 3.21s (± 1.79%) ~ 3.14s 3.31s p=0.936 n=6
Total Time 52.13s (± 0.25%) 52.12s (± 0.18%) ~ 51.99s 52.25s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,818 258,861 +43 (+ 0.02%) ~ ~ p=0.001 n=6
Types 104,842 104,856 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 428,208k (± 0.01%) 428,324k (± 0.01%) +115k (+ 0.03%) 428,243k 428,408k p=0.008 n=6
Parse Time 3.31s (± 0.53%) 3.32s (± 0.44%) ~ 3.30s 3.34s p=0.417 n=6
Bind Time 1.32s (± 1.34%) 1.31s (± 1.28%) ~ 1.29s 1.34s p=0.607 n=6
Check Time 17.80s (± 0.29%) 17.73s (± 0.34%) ~ 17.65s 17.80s p=0.107 n=6
Emit Time 1.37s (± 1.30%) 1.38s (± 1.16%) ~ 1.36s 1.40s p=0.251 n=6
Total Time 23.80s (± 0.17%) 23.75s (± 0.30%) ~ 23.65s 23.82s p=0.332 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,609k (± 0.02%) 369,461k (± 0.02%) -148k (- 0.04%) 369,362k 369,590k p=0.031 n=6
Parse Time 2.76s (± 1.22%) 2.78s (± 0.78%) ~ 2.74s 2.80s p=0.372 n=6
Bind Time 1.58s (± 1.01%) 1.58s (± 1.37%) ~ 1.56s 1.62s p=0.870 n=6
Check Time 15.44s (± 0.38%) 15.46s (± 0.29%) ~ 15.42s 15.54s p=0.329 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.78s (± 0.43%) 19.83s (± 0.34%) ~ 19.74s 19.92s p=0.261 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,629 2,878,629 ~ ~ ~ p=1.000 n=6
Types 975,177 975,177 ~ ~ ~ p=1.000 n=6
Memory used 3,042,136k (± 0.00%) 3,042,112k (± 0.00%) ~ 3,042,043k 3,042,169k p=0.574 n=6
Parse Time 13.56s (± 0.30%) 13.57s (± 0.55%) ~ 13.47s 13.70s p=0.630 n=6
Bind Time 4.18s (± 0.36%) 4.18s (± 0.43%) ~ 4.15s 4.20s p=0.803 n=6
Check Time 73.39s (± 0.58%) 73.47s (± 0.20%) ~ 73.30s 73.64s p=0.810 n=6
Emit Time 23.98s (± 0.53%) 23.98s (± 0.50%) ~ 23.77s 24.09s p=0.936 n=6
Total Time 115.12s (± 0.34%) 115.20s (± 0.12%) ~ 115.02s 115.40s p=0.575 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,554k (± 0.01%) 411,590k (± 0.02%) ~ 411,523k 411,708k p=0.173 n=6
Parse Time 3.82s (± 0.61%) 3.83s (± 0.60%) ~ 3.80s 3.86s p=0.468 n=6
Bind Time 1.68s (± 0.32%) 1.69s (± 0.44%) ~ 1.68s 1.70s p=0.137 n=6
Check Time 16.66s (± 0.31%) 16.72s (± 0.43%) ~ 16.63s 16.84s p=0.173 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.17s (± 0.19%) 22.24s (± 0.39%) ~ 22.15s 22.38s p=0.102 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,848k (± 0.08%) 462,785k (± 0.10%) ~ 462,361k 463,326k p=0.810 n=6
Parse Time 2.65s (± 0.69%) 2.64s (± 0.41%) ~ 2.62s 2.65s p=0.161 n=6
Bind Time 0.98s (± 0.42%) 0.98s (± 0.42%) ~ 0.97s 0.98s p=1.000 n=6
Check Time 15.14s (± 0.27%) 15.20s (± 0.42%) ~ 15.12s 15.28s p=0.196 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.77s (± 0.28%) 18.82s (± 0.34%) ~ 18.75s 18.90s p=0.261 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,752ms (± 0.56%) 2,767ms (± 0.52%) ~ 2,749ms 2,787ms p=0.173 n=6
Req 2 - geterr 6,910ms (± 9.53%) 6,700ms (±10.31%) ~ 6,031ms 7,366ms p=0.689 n=6
Req 3 - references 370ms (±10.88%) 360ms (±10.56%) ~ 318ms 395ms p=0.374 n=6
Req 4 - navto 328ms (± 7.02%) 330ms (± 7.42%) ~ 280ms 341ms p=1.000 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 122ms (±14.58%) 123ms (±10.39%) ~ 106ms 132ms p=0.868 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,612ms (± 1.12%) 3,581ms (± 0.51%) ~ 3,559ms 3,611ms p=0.128 n=6
Req 2 - geterr 5,540ms (± 0.41%) 5,552ms (± 0.81%) ~ 5,497ms 5,605ms p=0.936 n=6
Req 3 - references 417ms (± 0.24%) 416ms (± 0.25%) ~ 415ms 418ms p=0.203 n=6
Req 4 - navto 340ms (± 1.81%) 337ms (± 0.32%) ~ 335ms 338ms p=0.622 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 110ms (± 0.96%) 109ms (± 0.94%) ~ 108ms 110ms p=0.931 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,610ms (± 6.12%) 6,218ms (± 0.57%) ~ 6,191ms 6,287ms p=0.054 n=6
Req 2 - geterr 1,571ms (± 9.77%) 1,624ms (± 7.88%) ~ 1,365ms 1,697ms p=0.470 n=6
Req 3 - references 111ms (± 0.47%) 112ms (± 2.55%) ~ 111ms 118ms p=0.114 n=6
Req 4 - navto 596ms (± 2.51%) 603ms (± 1.42%) ~ 592ms 615ms p=0.470 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,237ms (± 1.44%) 1,238ms (± 1.11%) ~ 1,225ms 1,260ms p=0.936 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.34ms (± 0.15%) 157.40ms (± 0.17%) ~ 156.33ms 160.83ms p=0.113 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.29ms (± 0.13%) 232.38ms (± 0.16%) +0.09ms (+ 0.04%) 231.00ms 236.02ms p=0.042 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 340.27ms (± 0.31%) 340.61ms (± 0.31%) +0.34ms (+ 0.10%) 332.17ms 346.22ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 227.38ms (± 0.15%) 227.50ms (± 0.16%) +0.11ms (+ 0.05%) 226.07ms 231.05ms p=0.009 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

DanielRosenwasser commented Jun 27, 2024 •
edited
Loading

I think the latest change helped a bit; however, it seems like frequently this function ends up with an (effectively) identical list every single time with different reference equality. So I've tried adding an earlier cache. I'll see if that helps.

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162476/artifacts?artifactName=tgz&fileId=9683B5E6F6215D023679DE14CBF28D3038EEB938A0C5B3E05A39FA272049907B02&fileName=/typescript-5.6.0-insiders.20240627.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-59048-12".;

@DanielRosenwasser
Copy link
Member Author

For whatever reason, I'm still seeing at least one occurrence of an identical path array being passed in; but I think this is pretty ready for review.

@DanielRosenwasser DanielRosenwasser marked this pull request as ready for review June 27, 2024 19:43
@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this
@typescript-bot test tsserver top400
@typescript-bot test top400

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results
test tsserver top400 ✅ Started 👀 Results
test top400 ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,752k (± 0.75%) 193,395k (± 0.97%) ~ 192,132k 195,828k p=0.230 n=6
Parse Time 1.58s (± 0.62%) 1.58s (± 0.26%) ~ 1.58s 1.59s p=0.487 n=6
Bind Time 0.85s (± 0.88%) 0.86s (± 1.55%) ~ 0.84s 0.88s p=0.351 n=6
Check Time 11.18s (± 0.32%) 11.21s (± 0.53%) ~ 11.14s 11.31s p=0.332 n=6
Emit Time 3.28s (± 0.39%) 3.29s (± 0.92%) ~ 3.24s 3.33s p=0.329 n=6
Total Time 16.89s (± 0.25%) 16.94s (± 0.45%) ~ 16.83s 17.06s p=0.197 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,327k (± 0.00%) 1,218,321k (± 0.00%) ~ 1,218,258k 1,218,350k p=0.810 n=6
Parse Time 8.02s (± 0.59%) 7.96s (± 0.18%) -0.06s (- 0.71%) 7.94s 7.98s p=0.024 n=6
Bind Time 2.23s (± 0.25%) 2.23s (± 0.18%) ~ 2.22s 2.23s p=0.282 n=6
Check Time 35.79s (± 0.26%) 35.78s (± 0.19%) ~ 35.69s 35.85s p=0.936 n=6
Emit Time 16.21s (± 0.44%) 16.17s (± 0.46%) ~ 16.07s 16.25s p=0.377 n=6
Total Time 62.25s (± 0.24%) 62.15s (± 0.12%) ~ 62.05s 62.25s p=0.230 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,050 2,133,050 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,115,385k (± 0.01%) 2,115,452k (± 0.01%) ~ 2,115,246k 2,115,686k p=0.575 n=6
Parse Time 9.74s (± 0.28%) 9.70s (± 0.16%) -0.04s (- 0.43%) 9.68s 9.72s p=0.012 n=6
Bind Time 3.38s (± 0.55%) 3.37s (± 0.54%) ~ 3.34s 3.39s p=0.451 n=6
Check Time 101.73s (± 1.39%) 102.15s (± 0.40%) ~ 101.67s 102.77s p=1.000 n=6
Emit Time 0.20s (± 3.16%) 0.20s (± 4.47%) ~ 0.19s 0.21s p=1.000 n=6
Total Time 115.04s (± 1.22%) 115.42s (± 0.35%) ~ 114.94s 116.03s p=0.873 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,372 +41 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,007 260,989 -18 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,348,865k (± 1.03%) 2,339,403k (± 0.05%) ~ 2,337,509k 2,340,602k p=0.936 n=6
Parse Time 6.00s (± 1.04%) 6.00s (± 0.71%) ~ 5.95s 6.07s p=1.000 n=6
Bind Time 2.22s (± 0.53%) 2.24s (± 0.83%) +0.02s (+ 1.13%) 2.21s 2.26s p=0.030 n=6
Check Time 40.00s (± 0.33%) 39.95s (± 0.45%) ~ 39.68s 40.11s p=0.810 n=6
Emit Time 3.14s (± 3.31%) 3.19s (± 3.99%) ~ 3.04s 3.34s p=0.575 n=6
Total Time 51.37s (± 0.35%) 51.41s (± 0.43%) ~ 51.02s 51.66s p=0.630 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,372 +41 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,007 260,989 -18 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,412,674k (± 0.03%) 2,412,639k (± 0.02%) ~ 2,411,813k 2,413,139k p=0.936 n=6
Parse Time 5.22s (± 0.59%) 5.20s (± 0.95%) ~ 5.13s 5.25s p=0.298 n=6
Bind Time 1.69s (± 0.65%) 1.69s (± 0.69%) ~ 1.68s 1.71s p=0.729 n=6
Check Time 34.65s (± 0.12%) 34.60s (± 0.20%) ~ 34.50s 34.70s p=0.298 n=6
Emit Time 2.74s (± 2.60%) 2.70s (± 1.08%) ~ 2.67s 2.75s p=0.336 n=6
Total Time 44.30s (± 0.10%) 44.19s (± 0.30%) ~ 44.06s 44.36s p=0.230 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,166 258,152 -14 (- 0.01%) ~ ~ p=0.001 n=6
Types 104,689 104,671 -18 (- 0.02%) ~ ~ p=0.001 n=6
Memory used 427,352k (± 0.01%) 427,413k (± 0.02%) ~ 427,333k 427,530k p=0.093 n=6
Parse Time 3.29s (± 1.03%) 3.30s (± 0.63%) ~ 3.27s 3.33s p=0.366 n=6
Bind Time 1.33s (± 0.39%) 1.33s (± 0.61%) ~ 1.32s 1.34s p=0.929 n=6
Check Time 17.84s (± 0.29%) 17.82s (± 0.26%) ~ 17.73s 17.86s p=0.688 n=6
Emit Time 1.41s (± 0.90%) 1.40s (± 0.78%) ~ 1.39s 1.42s p=0.145 n=6
Total Time 23.87s (± 0.24%) 23.84s (± 0.17%) ~ 23.79s 23.91s p=0.421 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,532k (± 0.02%) 369,492k (± 0.02%) ~ 369,410k 369,605k p=0.378 n=6
Parse Time 2.31s (± 0.55%) 2.30s (± 0.45%) ~ 2.29s 2.32s p=0.456 n=6
Bind Time 1.34s (± 1.39%) 1.32s (± 0.31%) -0.02s (- 1.37%) 1.31s 1.32s p=0.025 n=6
Check Time 13.17s (± 0.47%) 13.16s (± 0.30%) ~ 13.10s 13.21s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 16.82s (± 0.43%) 16.79s (± 0.24%) ~ 16.73s 16.84s p=0.573 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,631 2,878,631 ~ ~ ~ p=1.000 n=6
Types 975,177 975,177 ~ ~ ~ p=1.000 n=6
Memory used 3,042,107k (± 0.00%) 3,042,146k (± 0.00%) +39k (+ 0.00%) 3,042,117k 3,042,168k p=0.016 n=6
Parse Time 13.58s (± 0.37%) 13.52s (± 0.33%) ~ 13.46s 13.58s p=0.076 n=6
Bind Time 4.18s (± 0.48%) 4.19s (± 0.26%) ~ 4.17s 4.20s p=0.503 n=6
Check Time 73.28s (± 0.16%) 73.19s (± 0.27%) ~ 72.86s 73.37s p=0.575 n=6
Emit Time 24.02s (± 0.75%) 24.02s (± 0.78%) ~ 23.73s 24.22s p=0.936 n=6
Total Time 115.06s (± 0.21%) 114.91s (± 0.35%) ~ 114.22s 115.30s p=0.936 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,550k (± 0.02%) 411,550k (± 0.01%) ~ 411,483k 411,610k p=0.936 n=6
Parse Time 3.82s (± 0.68%) 3.82s (± 0.73%) ~ 3.80s 3.87s p=1.000 n=6
Bind Time 1.69s (± 0.44%) 1.69s (± 0.32%) ~ 1.69s 1.70s p=0.476 n=6
Check Time 16.67s (± 0.35%) 16.73s (± 0.21%) ~ 16.67s 16.76s p=0.106 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.18s (± 0.19%) 22.25s (± 0.22%) +0.07s (+ 0.30%) 22.17s 22.32s p=0.024 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,763k (± 0.09%) 462,794k (± 0.09%) ~ 462,398k 463,190k p=0.936 n=6
Parse Time 2.65s (± 0.57%) 2.65s (± 0.37%) ~ 2.64s 2.66s p=0.869 n=6
Bind Time 0.98s (± 0.53%) 0.98s ~ ~ ~ p=0.174 n=6
Check Time 15.17s (± 0.30%) 15.16s (± 0.06%) ~ 15.15s 15.18s p=0.739 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.80s (± 0.29%) 18.79s (± 0.06%) ~ 18.78s 18.81s p=0.627 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,306ms (± 0.22%) 2,298ms (± 0.47%) ~ 2,281ms 2,311ms p=0.172 n=6
Req 2 - geterr 5,085ms (± 0.23%) 5,072ms (± 0.51%) ~ 5,029ms 5,104ms p=0.261 n=6
Req 3 - references 264ms (± 0.41%) 264ms (± 0.24%) ~ 263ms 265ms p=0.787 n=6
Req 4 - navto 228ms (± 1.03%) 227ms (± 1.11%) ~ 222ms 228ms p=0.391 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 87ms (± 6.17%) 89ms (± 3.14%) ~ 88ms 95ms p=1.000 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,425ms (± 0.62%) 2,419ms (± 0.38%) ~ 2,410ms 2,433ms p=0.689 n=6
Req 2 - geterr 3,807ms (± 0.34%) 3,797ms (± 0.34%) ~ 3,778ms 3,814ms p=0.261 n=6
Req 3 - references 278ms (± 0.30%) 279ms (± 0.53%) ~ 277ms 281ms p=0.055 n=6
Req 4 - navto 225ms (± 0.36%) 227ms (± 0.23%) +2ms (+ 0.89%) 227ms 228ms p=0.007 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 75ms (± 6.67%) 82ms (± 7.93%) ~ 69ms 85ms p=0.116 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 7,717ms (± 0.41%) 7,727ms (± 0.49%) ~ 7,655ms 7,765ms p=0.378 n=6
Req 2 - geterr 1,687ms (± 1.10%) 1,699ms (± 1.20%) ~ 1,662ms 1,719ms p=0.173 n=6
Req 3 - references 112ms (± 3.99%) 118ms (± 4.29%) ~ 111ms 121ms p=0.067 n=6
Req 4 - navto 594ms (± 1.96%) 598ms (± 2.10%) ~ 584ms 621ms p=0.748 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,243ms (± 2.01%) 1,262ms (± 3.43%) ~ 1,197ms 1,292ms p=0.471 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 187.23ms (± 0.17%) 187.29ms (± 0.19%) +0.06ms (+ 0.03%) 185.11ms 194.05ms p=0.030 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 231.36ms (± 0.15%) 231.54ms (± 0.14%) +0.19ms (+ 0.08%) 230.14ms 234.13ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 276.63ms (± 0.31%) 276.45ms (± 0.30%) -0.17ms (- 0.06%) 268.90ms 286.99ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 276.31ms (± 0.30%) 276.17ms (± 0.30%) -0.13ms (- 0.05%) 269.04ms 281.84ms p=0.032 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

@typescript-bot pack this
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 28, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162492/artifacts?artifactName=tgz&fileId=9A43BB1F5015C8E1240596441929304490B5CF692A75D0C591BEAA8E193450A502&fileName=/typescript-5.6.0-insiders.20240628.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-59048-28".;

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,254k (± 0.10%) 192,794k (± 0.75%) ~ 192,070k 195,722k p=0.873 n=6
Parse Time 1.31s (± 0.62%) 1.32s (± 0.92%) ~ 1.31s 1.34s p=0.115 n=6
Bind Time 0.71s 0.71s (± 0.89%) ~ 0.70s 0.72s p=1.000 n=6
Check Time 9.43s (± 0.33%) 9.45s (± 0.40%) ~ 9.40s 9.50s p=0.335 n=6
Emit Time 2.73s (± 0.31%) 2.74s (± 0.20%) ~ 2.74s 2.75s p=0.052 n=6
Total Time 14.18s (± 0.18%) 14.22s (± 0.34%) ~ 14.17s 14.28s p=0.227 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,319k (± 0.00%) 1,218,300k (± 0.00%) ~ 1,218,259k 1,218,326k p=0.297 n=6
Parse Time 7.96s (± 0.50%) 7.98s (± 0.57%) ~ 7.93s 8.04s p=0.518 n=6
Bind Time 2.22s (± 0.46%) 2.23s (± 0.38%) ~ 2.21s 2.23s p=0.672 n=6
Check Time 35.78s (± 0.48%) 35.73s (± 0.45%) ~ 35.52s 35.88s p=1.000 n=6
Emit Time 16.20s (± 0.49%) 16.12s (± 0.50%) ~ 15.99s 16.21s p=0.172 n=6
Total Time 62.16s (± 0.30%) 62.05s (± 0.38%) ~ 61.72s 62.31s p=0.520 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,050 2,133,050 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,115,331k (± 0.00%) 2,115,377k (± 0.00%) ~ 2,115,273k 2,115,544k p=0.298 n=6
Parse Time 9.72s (± 0.22%) 9.71s (± 0.49%) ~ 9.63s 9.76s p=0.747 n=6
Bind Time 3.38s (± 0.70%) 3.37s (± 0.95%) ~ 3.33s 3.40s p=0.375 n=6
Check Time 101.94s (± 0.43%) 102.17s (± 0.57%) ~ 101.29s 103.00s p=0.630 n=6
Emit Time 0.20s (± 4.95%) 0.20s (± 4.47%) ~ 0.19s 0.21s p=0.798 n=6
Total Time 115.24s (± 0.38%) 115.45s (± 0.49%) ~ 114.59s 116.24s p=0.810 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,413 +82 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,007 260,990 -17 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,339,417k (± 0.03%) 2,340,110k (± 0.04%) ~ 2,339,197k 2,341,110k p=0.173 n=6
Parse Time 6.03s (± 0.64%) 5.99s (± 0.73%) ~ 5.94s 6.05s p=0.128 n=6
Bind Time 2.26s (± 0.46%) 2.25s (± 0.71%) ~ 2.24s 2.28s p=0.191 n=6
Check Time 40.07s (± 0.24%) 40.10s (± 0.31%) ~ 39.91s 40.21s p=0.521 n=6
Emit Time 3.17s (± 2.10%) 3.12s (± 1.49%) ~ 3.07s 3.19s p=0.173 n=6
Total Time 51.53s (± 0.24%) 51.46s (± 0.24%) ~ 51.27s 51.61s p=0.230 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,413 +82 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,007 260,990 -17 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,412,890k (± 0.03%) 2,423,234k (± 1.01%) ~ 2,411,661k 2,473,314k p=0.471 n=6
Parse Time 6.30s (± 0.75%) 6.24s (± 1.08%) ~ 6.14s 6.34s p=0.108 n=6
Bind Time 2.02s (± 0.81%) 2.04s (± 0.97%) ~ 2.01s 2.07s p=0.140 n=6
Check Time 40.76s (± 0.49%) 40.74s (± 0.31%) ~ 40.60s 40.96s p=1.000 n=6
Emit Time 3.27s (± 1.90%) 3.23s (± 2.42%) ~ 3.14s 3.36s p=0.378 n=6
Total Time 52.35s (± 0.35%) 52.25s (± 0.36%) ~ 52.03s 52.53s p=0.378 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,166 258,160 -6 (- 0.00%) ~ ~ p=0.001 n=6
Types 104,689 104,672 -17 (- 0.02%) ~ ~ p=0.001 n=6
Memory used 427,337k (± 0.01%) 427,387k (± 0.01%) ~ 427,345k 427,473k p=0.173 n=6
Parse Time 3.29s (± 0.54%) 3.29s (± 1.17%) ~ 3.24s 3.33s p=1.000 n=6
Bind Time 1.31s (± 0.92%) 1.31s (± 0.92%) ~ 1.29s 1.32s p=1.000 n=6
Check Time 17.77s (± 0.41%) 17.77s (± 0.50%) ~ 17.67s 17.89s p=1.000 n=6
Emit Time 1.40s (± 1.11%) 1.40s (± 1.40%) ~ 1.38s 1.42s p=0.935 n=6
Total Time 23.78s (± 0.31%) 23.78s (± 0.32%) ~ 23.71s 23.92s p=1.000 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,473k (± 0.01%) 369,467k (± 0.02%) ~ 369,368k 369,576k p=0.936 n=6
Parse Time 2.77s (± 0.50%) 2.77s (± 1.01%) ~ 2.75s 2.81s p=0.935 n=6
Bind Time 1.59s (± 0.40%) 1.60s (± 1.02%) ~ 1.58s 1.62s p=0.615 n=6
Check Time 15.47s (± 0.20%) 15.48s (± 0.36%) ~ 15.41s 15.56s p=0.748 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.82s (± 0.20%) 19.85s (± 0.31%) ~ 19.74s 19.92s p=0.332 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,631 2,878,631 ~ ~ ~ p=1.000 n=6
Types 975,177 975,177 ~ ~ ~ p=1.000 n=6
Memory used 3,042,169k (± 0.00%) 3,042,135k (± 0.00%) ~ 3,042,089k 3,042,172k p=0.575 n=6
Parse Time 13.58s (± 0.37%) 13.52s (± 0.34%) ~ 13.48s 13.59s p=0.077 n=6
Bind Time 4.23s (± 1.87%) 4.19s (± 0.90%) ~ 4.16s 4.26s p=0.061 n=6
Check Time 73.34s (± 0.25%) 73.07s (± 0.19%) -0.27s (- 0.37%) 72.87s 73.21s p=0.020 n=6
Emit Time 24.00s (± 0.92%) 23.93s (± 0.37%) ~ 23.79s 24.04s p=0.873 n=6
Total Time 115.15s (± 0.33%) 114.71s (± 0.17%) -0.44s (- 0.38%) 114.45s 114.89s p=0.031 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,567k (± 0.02%) 411,545k (± 0.01%) ~ 411,504k 411,593k p=1.000 n=6
Parse Time 3.19s (± 0.34%) 3.17s (± 0.85%) ~ 3.14s 3.20s p=0.289 n=6
Bind Time 1.41s (± 0.53%) 1.42s (± 0.36%) ~ 1.41s 1.42s p=0.069 n=6
Check Time 14.23s (± 0.45%) 14.22s (± 0.33%) ~ 14.15s 14.29s p=0.748 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.83s (± 0.36%) 18.81s (± 0.31%) ~ 18.71s 18.88s p=0.470 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 463,025k (± 0.07%) 462,890k (± 0.08%) ~ 462,387k 463,286k p=0.575 n=6
Parse Time 2.66s (± 0.37%) 2.66s (± 0.39%) ~ 2.64s 2.67s p=0.315 n=6
Bind Time 0.98s (± 0.42%) 0.98s (± 0.42%) ~ 0.97s 0.98s p=1.000 n=6
Check Time 15.17s (± 0.42%) 15.18s (± 0.40%) ~ 15.10s 15.28s p=0.684 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.81s (± 0.33%) 18.82s (± 0.30%) ~ 18.72s 18.89s p=0.809 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,301ms (± 0.18%) 2,299ms (± 0.47%) ~ 2,281ms 2,313ms p=0.872 n=6
Req 2 - geterr 5,072ms (± 0.52%) 5,062ms (± 0.53%) ~ 5,018ms 5,095ms p=0.810 n=6
Req 3 - references 264ms (± 0.15%) 265ms (± 0.21%) ~ 264ms 265ms p=0.054 n=6
Req 4 - navto 227ms (± 0.95%) 227ms (± 0.90%) ~ 223ms 228ms p=0.673 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 90ms (± 2.62%) 89ms (± 2.17%) ~ 88ms 93ms p=0.858 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,573ms (± 0.39%) 3,568ms (± 0.91%) ~ 3,514ms 3,598ms p=1.000 n=6
Req 2 - geterr 5,547ms (± 0.35%) 5,534ms (± 0.34%) ~ 5,501ms 5,559ms p=0.127 n=6
Req 3 - references 417ms (± 0.63%) 417ms (± 0.40%) ~ 415ms 419ms p=0.742 n=6
Req 4 - navto 342ms (± 2.09%) 337ms (± 1.01%) ~ 331ms 341ms p=0.462 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 109ms (± 2.59%) 114ms (± 6.74%) ~ 108ms 125ms p=0.413 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,193ms (± 0.65%) 5,198ms (± 0.33%) ~ 5,165ms 5,213ms p=1.000 n=6
Req 2 - geterr 1,122ms (± 2.25%) 1,128ms (± 0.94%) ~ 1,120ms 1,148ms p=0.630 n=6
Req 3 - references 77ms (± 3.98%) 77ms (± 3.56%) ~ 75ms 82ms p=0.866 n=6
Req 4 - navto 458ms (± 0.74%) 459ms (± 0.81%) ~ 456ms 466ms p=0.720 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 835ms (± 1.49%) 833ms (± 1.43%) ~ 813ms 844ms p=0.873 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 228.75ms (± 0.17%) 228.75ms (± 0.15%) ~ 226.86ms 232.08ms p=0.294 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 231.66ms (± 0.15%) 231.54ms (± 0.13%) -0.12ms (- 0.05%) 229.99ms 235.13ms p=0.001 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.85ms (± 0.16%) 226.72ms (± 0.14%) -0.13ms (- 0.06%) 225.57ms 231.37ms p=0.001 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.49ms (± 0.15%) 226.30ms (± 0.15%) -0.19ms (- 0.09%) 225.10ms 230.73ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser DanielRosenwasser changed the title Use a Set and a binary search to search for patterns on paths Optimize path mapping lookups Jun 28, 2024
@DanielRosenwasser
Copy link
Member Author

@typescript-bot pack this
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 28, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162513/artifacts?artifactName=tgz&fileId=FE69A9E299527A889B20AB1EFDCCC750B25244355428604098F9E59D6BA8F0E402&fileName=/typescript-5.6.0-insiders.20240628.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-59048-32".;

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,322k (± 0.96%) 192,829k (± 0.75%) ~ 192,145k 195,769k p=0.230 n=6
Parse Time 1.58s (± 0.78%) 1.58s (± 0.35%) ~ 1.57s 1.58s p=0.863 n=6
Bind Time 0.87s (± 0.63%) 0.85s (± 0.61%) -0.02s (- 2.12%) 0.84s 0.85s p=0.004 n=6
Check Time 11.21s (± 0.59%) 11.22s (± 0.49%) ~ 11.15s 11.30s p=0.873 n=6
Emit Time 3.27s (± 0.72%) 3.28s (± 0.71%) ~ 3.26s 3.32s p=0.418 n=6
Total Time 16.92s (± 0.39%) 16.93s (± 0.45%) ~ 16.83s 17.05s p=0.936 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,349k (± 0.00%) 1,218,347k (± 0.00%) ~ 1,218,280k 1,218,392k p=0.810 n=6
Parse Time 6.68s (± 0.58%) 6.65s (± 1.06%) ~ 6.58s 6.75s p=0.422 n=6
Bind Time 1.87s (± 0.40%) 1.87s (± 0.55%) ~ 1.85s 1.88s p=0.437 n=6
Check Time 30.64s (± 0.35%) 30.64s (± 0.44%) ~ 30.45s 30.77s p=0.936 n=6
Emit Time 13.52s (± 0.57%) 13.56s (± 0.27%) ~ 13.51s 13.61s p=0.521 n=6
Total Time 52.71s (± 0.35%) 52.71s (± 0.27%) ~ 52.55s 52.91s p=1.000 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,039 2,133,039 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,115,360k (± 0.01%) 2,115,311k (± 0.00%) ~ 2,115,215k 2,115,355k p=0.630 n=6
Parse Time 7.91s (± 0.54%) 7.90s (± 0.37%) ~ 7.85s 7.94s p=0.571 n=6
Bind Time 2.78s (± 0.53%) 2.76s (± 0.63%) ~ 2.73s 2.78s p=0.070 n=6
Check Time 84.01s (± 0.50%) 83.71s (± 0.47%) ~ 83.18s 84.27s p=0.261 n=6
Emit Time 0.15s (± 3.53%) 0.16s (± 5.21%) ~ 0.15s 0.17s p=0.859 n=6
Total Time 94.85s (± 0.42%) 94.52s (± 0.41%) ~ 94.05s 95.10s p=0.173 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,347 +16 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,007 260,987 -20 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,339,541k (± 0.03%) 2,339,992k (± 0.04%) ~ 2,338,492k 2,340,932k p=0.471 n=6
Parse Time 6.02s (± 0.93%) 5.93s (± 0.68%) -0.09s (- 1.47%) 5.89s 5.99s p=0.025 n=6
Bind Time 2.23s (± 0.80%) 2.25s (± 0.44%) +0.02s (+ 0.97%) 2.24s 2.26s p=0.028 n=6
Check Time 40.04s (± 0.18%) 40.03s (± 0.24%) ~ 39.94s 40.18s p=0.687 n=6
Emit Time 3.13s (± 2.42%) 3.14s (± 3.00%) ~ 3.06s 3.30s p=0.689 n=6
Total Time 51.43s (± 0.20%) 51.37s (± 0.26%) ~ 51.24s 51.61s p=0.298 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,224,347 +16 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,007 260,987 -20 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,412,725k (± 0.02%) 2,412,957k (± 0.03%) ~ 2,412,029k 2,413,996k p=0.810 n=6
Parse Time 6.25s (± 0.56%) 6.24s (± 1.08%) ~ 6.15s 6.34s p=0.936 n=6
Bind Time 2.01s (± 1.21%) 2.01s (± 1.16%) ~ 1.99s 2.05s p=0.570 n=6
Check Time 40.58s (± 0.26%) 40.61s (± 0.33%) ~ 40.37s 40.73s p=0.471 n=6
Emit Time 3.22s (± 1.55%) 3.22s (± 2.20%) ~ 3.13s 3.31s p=0.748 n=6
Total Time 52.06s (± 0.23%) 52.10s (± 0.27%) ~ 51.94s 52.36s p=0.873 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,166 258,138 -28 (- 0.01%) ~ ~ p=0.001 n=6
Types 104,689 104,669 -20 (- 0.02%) ~ ~ p=0.001 n=6
Memory used 427,355k (± 0.01%) 427,351k (± 0.01%) ~ 427,303k 427,411k p=0.575 n=6
Parse Time 2.72s (± 1.06%) 2.75s (± 1.00%) ~ 2.72s 2.78s p=0.126 n=6
Bind Time 1.11s (± 0.47%) 1.10s (± 1.10%) ~ 1.08s 1.11s p=0.923 n=6
Check Time 15.13s (± 0.19%) 15.08s (± 0.25%) -0.05s (- 0.36%) 15.03s 15.14s p=0.034 n=6
Emit Time 1.19s (± 0.92%) 1.19s (± 0.69%) ~ 1.18s 1.20s p=0.662 n=6
Total Time 20.15s (± 0.23%) 20.12s (± 0.18%) ~ 20.07s 20.16s p=0.169 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,582k (± 0.04%) 369,567k (± 0.04%) ~ 369,406k 369,734k p=0.936 n=6
Parse Time 2.77s (± 0.70%) 2.77s (± 0.44%) ~ 2.76s 2.79s p=0.514 n=6
Bind Time 1.59s (± 1.08%) 1.58s (± 0.74%) ~ 1.57s 1.60s p=0.557 n=6
Check Time 15.46s (± 0.25%) 15.51s (± 0.21%) ~ 15.45s 15.54s p=0.053 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.81s (± 0.19%) 19.87s (± 0.15%) +0.05s (+ 0.27%) 19.81s 19.89s p=0.044 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,940 2,878,940 ~ ~ ~ p=1.000 n=6
Types 975,074 975,074 ~ ~ ~ p=1.000 n=6
Memory used 3,041,804k (± 0.00%) 3,041,773k (± 0.00%) ~ 3,041,705k 3,041,831k p=0.378 n=6
Parse Time 13.70s (± 0.20%) 13.68s (± 0.24%) ~ 13.63s 13.71s p=0.197 n=6
Bind Time 4.26s (± 2.91%) 4.19s (± 0.56%) ~ 4.16s 4.22s p=0.169 n=6
Check Time 74.73s (± 2.00%) 74.31s (± 1.86%) ~ 73.56s 77.13s p=0.423 n=6
Emit Time 23.19s (± 6.73%) 23.66s (± 6.61%) ~ 20.47s 24.40s p=0.378 n=6
Total Time 115.88s (± 0.20%) 115.84s (± 0.25%) ~ 115.45s 116.19s p=0.810 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,562k (± 0.01%) 411,511k (± 0.01%) ~ 411,443k 411,581k p=0.128 n=6
Parse Time 3.81s (± 0.46%) 3.83s (± 0.75%) ~ 3.80s 3.87s p=0.183 n=6
Bind Time 1.69s (± 0.44%) 1.70s (± 0.58%) ~ 1.69s 1.71s p=0.300 n=6
Check Time 16.77s (± 0.22%) 16.77s (± 0.17%) ~ 16.74s 16.82s p=0.936 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.26s (± 0.21%) 22.29s (± 0.15%) ~ 22.25s 22.34s p=0.332 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,961k (± 0.10%) 463,167k (± 0.02%) ~ 463,108k 463,338k p=0.810 n=6
Parse Time 2.64s (± 0.39%) 2.65s (± 0.48%) ~ 2.64s 2.67s p=0.391 n=6
Bind Time 0.98s (± 0.42%) 0.98s (± 0.64%) ~ 0.97s 0.99s p=0.673 n=6
Check Time 15.19s (± 0.51%) 15.15s (± 0.14%) ~ 15.12s 15.17s p=0.375 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.81s (± 0.38%) 18.78s (± 0.08%) ~ 18.76s 18.79s p=0.463 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,863ms (± 9.64%) 2,768ms (± 0.98%) ~ 2,728ms 2,800ms p=1.000 n=6
Req 2 - geterr 6,291ms (± 8.40%) 6,252ms (± 8.12%) ~ 6,006ms 7,288ms p=0.230 n=6
Req 3 - references 357ms (±11.34%) 386ms (± 5.20%) ~ 345ms 396ms p=0.170 n=6
Req 4 - navto 330ms (± 8.01%) 338ms (± 1.30%) ~ 331ms 343ms p=0.935 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 128ms (± 7.11%) 136ms (± 3.27%) 🔻+9ms (+ 6.93%) 132ms 144ms p=0.005 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,426ms (± 0.65%) 2,412ms (± 0.78%) ~ 2,392ms 2,446ms p=0.230 n=6
Req 2 - geterr 3,800ms (± 0.43%) 3,798ms (± 0.30%) ~ 3,781ms 3,813ms p=0.873 n=6
Req 3 - references 278ms (± 0.37%) 277ms (± 1.23%) ~ 271ms 280ms p=0.870 n=6
Req 4 - navto 226ms (± 0.46%) 229ms (± 2.36%) ~ 226ms 240ms p=0.176 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 74ms (± 7.44%) 77ms (±11.73%) ~ 67ms 85ms p=0.808 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,218ms (± 0.31%) 6,241ms (± 0.19%) +23ms (+ 0.36%) 6,223ms 6,256ms p=0.030 n=6
Req 2 - geterr 1,637ms (± 8.14%) 1,626ms (± 7.79%) ~ 1,371ms 1,706ms p=0.873 n=6
Req 3 - references 111ms (± 6.30%) 116ms (± 5.05%) ~ 110ms 122ms p=0.343 n=6
Req 4 - navto 589ms (± 3.37%) 601ms (± 3.42%) ~ 581ms 627ms p=0.289 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,223ms (± 7.32%) 1,264ms (± 2.25%) ~ 1,233ms 1,299ms p=0.689 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.06ms (± 0.16%) 157.23ms (± 0.19%) +0.17ms (+ 0.11%) 156.18ms 162.30ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 231.36ms (± 0.14%) 231.32ms (± 0.13%) ~ 229.89ms 233.93ms p=0.316 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 338.10ms (± 0.30%) 338.08ms (± 0.30%) ~ 329.94ms 344.07ms p=0.530 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.35ms (± 0.16%) 226.43ms (± 0.19%) ~ 224.97ms 232.33ms p=0.349 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

The current state of this just always caches the objects and divides the entries into exact specifiers (as a Set) and an array of patterns as written. This still helps the internal codebase in question since about half of the entries in the huge list are non-pattern specifiers.

I'm open to re-introducing the binary search. It probably still makes sense to sort the array and iterate backwards to find a match and bail earlier when possible. But apparently we don't seem to have a test where suffix length matters in the case of exports, and I'd need someone to make sure I get the details there right.

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 11, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@DanielRosenwasser DanielRosenwasser mentioned this pull request Jul 11, 2024
Open
@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,366k (± 0.95%) 193,416k (± 0.93%) ~ 192,147k 195,749k p=0.810 n=6
Parse Time 1.57s (± 1.51%) 1.57s (± 0.94%) ~ 1.55s 1.58s p=0.676 n=6
Bind Time 0.86s (± 0.88%) 0.86s (± 0.88%) ~ 0.85s 0.87s p=1.000 n=6
Check Time 11.19s (± 0.13%) 11.17s (± 0.68%) ~ 11.06s 11.24s p=0.872 n=6
Emit Time 3.27s (± 0.65%) 3.25s (± 0.72%) ~ 3.22s 3.28s p=0.089 n=6
Total Time 16.89s (± 0.20%) 16.84s (± 0.55%) ~ 16.73s 16.97s p=0.470 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,358k (± 0.00%) 1,218,384k (± 0.00%) ~ 1,218,350k 1,218,428k p=0.199 n=6
Parse Time 7.99s (± 0.93%) 7.98s (± 0.37%) ~ 7.95s 8.03s p=0.936 n=6
Bind Time 2.23s (± 0.52%) 2.23s (± 0.34%) ~ 2.22s 2.24s p=0.396 n=6
Check Time 35.82s (± 0.41%) 35.80s (± 0.33%) ~ 35.60s 35.90s p=0.629 n=6
Emit Time 16.25s (± 0.59%) 16.18s (± 0.41%) ~ 16.12s 16.27s p=0.146 n=6
Total Time 62.30s (± 0.11%) 62.18s (± 0.25%) ~ 61.94s 62.38s p=0.128 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,130,296 2,130,296 ~ ~ ~ p=1.000 n=6
Types 927,656 927,656 ~ ~ ~ p=1.000 n=6
Memory used 2,107,127k (± 0.01%) 2,107,082k (± 0.01%) ~ 2,106,817k 2,107,348k p=0.689 n=6
Parse Time 9.65s (± 0.38%) 9.65s (± 0.65%) ~ 9.59s 9.77s p=0.462 n=6
Bind Time 3.37s (± 0.83%) 3.38s (± 1.10%) ~ 3.32s 3.43s p=0.422 n=6
Check Time 102.50s (± 0.33%) 101.60s (± 1.52%) ~ 99.61s 103.19s p=0.471 n=6
Emit Time 0.19s (± 2.81%) 0.20s (± 5.89%) ~ 0.19s 0.22s p=0.859 n=6
Total Time 115.72s (± 0.31%) 114.82s (± 1.29%) ~ 112.87s 116.39s p=0.423 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,226,005 1,226,001 -4 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,615 261,591 -24 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,338,926k (± 0.01%) 2,339,355k (± 0.01%) ~ 2,339,004k 2,339,828k p=0.066 n=6
Parse Time 7.60s (± 0.58%) 7.57s (± 0.77%) ~ 7.46s 7.62s p=0.689 n=6
Bind Time 2.81s (± 0.52%) 2.81s (± 1.24%) ~ 2.76s 2.86s p=0.572 n=6
Check Time 49.48s (± 0.34%) 49.30s (± 0.35%) ~ 49.11s 49.62s p=0.173 n=6
Emit Time 4.81s (± 2.11%) 4.80s (± 0.71%) ~ 4.75s 4.84s p=1.000 n=6
Total Time 64.71s (± 0.36%) 64.50s (± 0.29%) ~ 64.25s 64.79s p=0.173 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,226,005 1,226,001 -4 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,615 261,591 -24 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,413,255k (± 0.02%) 2,423,592k (± 1.03%) ~ 2,413,254k 2,474,457k p=0.378 n=6
Parse Time 7.87s (± 1.20%) 7.82s (± 1.04%) ~ 7.71s 7.95s p=0.336 n=6
Bind Time 2.50s (± 1.54%) 2.52s (± 1.03%) ~ 2.48s 2.55s p=0.377 n=6
Check Time 50.20s (± 0.40%) 50.07s (± 0.38%) ~ 49.88s 50.44s p=0.298 n=6
Emit Time 4.81s (± 0.97%) 4.84s (± 0.80%) ~ 4.79s 4.88s p=0.333 n=6
Total Time 65.40s (± 0.36%) 65.28s (± 0.18%) ~ 65.22s 65.52s p=0.173 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,315 258,278 -37 (- 0.01%) ~ ~ p=0.001 n=6
Types 104,817 104,793 -24 (- 0.02%) ~ ~ p=0.001 n=6
Memory used 427,193k (± 0.01%) 427,242k (± 0.05%) ~ 427,056k 427,542k p=0.689 n=6
Parse Time 3.30s (± 1.09%) 3.30s (± 0.63%) ~ 3.28s 3.34s p=0.808 n=6
Bind Time 1.32s (± 0.93%) 1.31s (± 1.28%) ~ 1.28s 1.32s p=0.599 n=6
Check Time 17.84s (± 0.43%) 17.82s (± 0.29%) ~ 17.74s 17.89s p=0.420 n=6
Emit Time 1.64s (± 1.13%) 1.65s (± 1.85%) ~ 1.61s 1.70s p=0.935 n=6
Total Time 24.10s (± 0.39%) 24.09s (± 0.18%) ~ 24.02s 24.14s p=0.873 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,481k (± 0.02%) 369,552k (± 0.03%) ~ 369,449k 369,656k p=0.378 n=6
Parse Time 2.77s (± 0.84%) 2.77s (± 0.56%) ~ 2.74s 2.78s p=0.870 n=6
Bind Time 1.58s (± 0.40%) 1.59s (± 1.08%) ~ 1.57s 1.61s p=0.557 n=6
Check Time 15.52s (± 0.33%) 15.52s (± 0.34%) ~ 15.42s 15.56s p=0.688 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.87s (± 0.29%) 19.87s (± 0.27%) ~ 19.77s 19.91s p=0.421 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,890,434 2,890,434 ~ ~ ~ p=1.000 n=6
Types 977,956 977,956 ~ ~ ~ p=1.000 n=6
Memory used 3,054,070k (± 0.00%) 3,054,103k (± 0.00%) ~ 3,053,956k 3,054,220k p=0.378 n=6
Parse Time 16.88s (± 0.44%) 16.92s (± 0.47%) ~ 16.84s 17.05s p=0.421 n=6
Bind Time 5.13s (± 0.20%) 5.11s (± 0.37%) -0.02s (- 0.45%) 5.09s 5.14s p=0.042 n=6
Check Time 89.90s (± 0.46%) 90.66s (± 2.31%) ~ 89.51s 94.93s p=0.810 n=6
Emit Time 29.19s (± 0.91%) 28.36s (± 6.66%) ~ 24.51s 29.32s p=0.423 n=6
Total Time 141.11s (± 0.36%) 141.05s (± 0.17%) ~ 140.86s 141.50s p=0.936 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,239 267,239 ~ ~ ~ p=1.000 n=6
Types 108,815 108,815 ~ ~ ~ p=1.000 n=6
Memory used 411,855k (± 0.02%) 411,818k (± 0.01%) ~ 411,754k 411,930k p=0.471 n=6
Parse Time 3.82s (± 0.43%) 3.82s (± 0.78%) ~ 3.79s 3.86s p=0.675 n=6
Bind Time 1.68s (± 0.70%) 1.68s (± 0.97%) ~ 1.66s 1.70s p=0.870 n=6
Check Time 16.72s (± 0.22%) 16.68s (± 0.37%) ~ 16.61s 16.77s p=0.228 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.22s (± 0.22%) 22.17s (± 0.30%) ~ 22.08s 22.24s p=0.334 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 510,475 510,475 ~ ~ ~ p=1.000 n=6
Types 161,600 161,600 ~ ~ ~ p=1.000 n=6
Memory used 447,981k (± 0.08%) 447,998k (± 0.08%) ~ 447,554k 448,455k p=0.936 n=6
Parse Time 2.62s (± 0.71%) 2.62s (± 0.67%) ~ 2.60s 2.65s p=0.935 n=6
Bind Time 0.98s (± 0.84%) 0.97s (± 0.53%) ~ 0.97s 0.98s p=0.523 n=6
Check Time 14.40s (± 0.48%) 14.46s (± 0.54%) ~ 14.38s 14.59s p=0.199 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.00s (± 0.45%) 18.06s (± 0.44%) ~ 17.98s 18.19s p=0.296 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,294ms (± 0.42%) 2,293ms (± 0.30%) ~ 2,284ms 2,305ms p=0.573 n=6
Req 2 - geterr 5,066ms (± 0.61%) 5,060ms (± 0.63%) ~ 4,999ms 5,081ms p=1.000 n=6
Req 3 - references 265ms (± 0.19%) 266ms (± 0.19%) ~ 265ms 266ms p=0.311 n=6
Req 4 - navto 228ms (± 0.18%) 228ms (± 0.28%) ~ 227ms 229ms p=0.673 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 93ms (± 3.19%) 93ms (± 2.44%) ~ 91ms 97ms p=0.652 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,424ms (± 1.20%) 2,420ms (± 0.96%) ~ 2,392ms 2,450ms p=0.810 n=6
Req 2 - geterr 3,792ms (± 0.40%) 3,797ms (± 0.29%) ~ 3,778ms 3,809ms p=0.575 n=6
Req 3 - references 276ms (± 0.42%) 277ms (± 0.38%) ~ 275ms 278ms p=0.619 n=6
Req 4 - navto 226ms (± 0.28%) 226ms (± 0.33%) ~ 225ms 227ms p=0.718 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 74ms (± 6.97%) 74ms (± 7.51%) ~ 69ms 84ms p=0.934 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,192ms (± 0.57%) 5,185ms (± 0.44%) ~ 5,163ms 5,227ms p=0.810 n=6
Req 2 - geterr 1,125ms (± 0.86%) 1,121ms (± 2.09%) ~ 1,095ms 1,165ms p=0.295 n=6
Req 3 - references 76ms 76ms ~ ~ ~ p=1.000 n=6
Req 4 - navto 457ms (± 0.23%) 457ms (± 0.16%) ~ 456ms 458ms p=0.931 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 828ms (± 1.36%) 832ms (± 1.39%) ~ 814ms 846ms p=0.689 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.22ms (± 0.16%) 157.21ms (± 0.18%) ~ 156.16ms 161.26ms p=0.427 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.55ms (± 0.15%) 230.57ms (± 0.18%) ~ 228.98ms 237.04ms p=0.729 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.73ms (± 0.19%) 226.67ms (± 0.14%) ~ 225.20ms 230.08ms p=0.258 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.02ms (± 0.16%) 226.06ms (± 0.15%) ~ 224.36ms 228.17ms p=0.207 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

weswigham
weswigham approved these changes Jul 23, 2024
View reviewed changes
src/compiler/utilities.ts Outdated
return candidate;
}
export function matchPatternOrExact(patternOrStrings: ParsedPatterns, candidate: string): string | Pattern | undefined {
const { matchableStringSet, patterns } = patternOrStrings;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const { matchableStringSet, patterns } = patternOrStrings;
const { matchableStringSet, patterns } = parsedPatterns;

just because, well, it's not an array of pattern or string anymore. The comment above this function also needs updating.

@DanielRosenwasser DanielRosenwasser merged commit e6f3baf into main Aug 29, 2024
33 checks passed
@DanielRosenwasser DanielRosenwasser deleted the binarySearchAndSetsForPatterns branch August 29, 2024 16:17
@DanielRosenwasser DanielRosenwasser mentioned this pull request Aug 30, 2024
Closed
1 task
@DanielRosenwasser DanielRosenwasser mentioned this pull request Sep 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmichon-msft dmichon-msft dmichon-msft left review comments

@jakebailey jakebailey jakebailey left review comments

@weswigham weswigham weswigham approved these changes

@andrewbranch andrewbranch Awaiting requested review from andrewbranch

@sheetalkamat sheetalkamat Awaiting requested review from sheetalkamat

Assignees

@DanielRosenwasser DanielRosenwasser

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@DanielRosenwasser @typescript-bot @weswigham @jakebailey @dmichon-msft