Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid exponential blowup in size of Builder type #591

Merged

Conversation

sapphire-arches
Copy link
Contributor

See the comment added to builder.rs for the juicy details, but the tl;dr is that by not calling a shared internal function in the various ElementBuilder entrypoints, we avoid an exponential explosion in the size of the resulting Builder object type and thereby dodge limits on mangled symbol size.

An alternative solution would be recommending using v0 name mangling (RUSTFLAGS=-Csymbol-mangling-version=v0) but that seems less convenient for users and anyway we should probably avoid generating exponential types in case the compiler takes a peek and gets very sad and slow.

This is kind of hard to add an automatic (stress) test for, as testing requires poking the generated .wasm file to see if any of the mangled names are looking exponential-ish.

See the comment added to builder.rs for the juicy details, but the
tl;dr is that by *not* calling a shared internal function in the various
`ElementBuilder` entrypoints, we avoid an exponential explosion in the
size of the resulting Builder object type and thereby dodge limits on
mangled symbol size.
@codecov
Copy link

codecov bot commented Mar 20, 2023

Codecov Report

Patch coverage: 23.15% and project coverage change: -0.30 ⚠️

Comparison is base (4eb5852) 60.82% compared to head (067f336) 60.52%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #591      +/-   ##
==========================================
- Coverage   60.82%   60.52%   -0.30%     
==========================================
  Files          56       56              
  Lines        9340     9396      +56     
==========================================
+ Hits         5681     5687       +6     
- Misses       3659     3709      +50     
Impacted Files Coverage Δ
packages/sycamore/src/builder.rs 22.88% <23.15%> (-1.64%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Collaborator

@lukechu10 lukechu10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for the very detailed explanation!

@lukechu10 lukechu10 merged commit fedc881 into sycamore-rs:master Mar 25, 2023
@alecmocatta
Copy link

This commit took our build time from 11 mins to 3.5. Appreciate it!

Details
time:   0.002; rss:   38MB ->   38MB (   +0MB)        parse_crate
time:   0.000; rss:   38MB ->   38MB (   +0MB)        attributes_injection
time:   0.000; rss:   39MB ->   39MB (   +0MB)        plugin_loading
time:   0.000; rss:   39MB ->   40MB (   +0MB)        setup_global_ctxt
time:   0.001; rss:   40MB ->   41MB (   +0MB)        crate_injection
time:   0.397; rss:   41MB ->   98MB (  +57MB)        expand_crate
time:   0.397; rss:   41MB ->   98MB (  +57MB)        macro_expand_crate
time:   0.001; rss:   98MB ->   98MB (   +0MB)        maybe_building_test_harness
time:   0.000; rss:   98MB ->   98MB (   +0MB)        AST_validation
time:   0.000; rss:   98MB ->   98MB (   +0MB)        finalize_imports
time:   0.001; rss:   98MB ->   98MB (   +0MB)        compute_effective_visibilities
time:   0.001; rss:   98MB ->   99MB (   +0MB)        finalize_macro_resolutions
time:   0.029; rss:   99MB ->  108MB (  +10MB)        late_resolve_crate
time:   0.000; rss:  108MB ->  108MB (   +0MB)        resolve_postprocess
time:   0.031; rss:   98MB ->  108MB (  +10MB)  resolve_crate
time:   0.000; rss:  108MB ->  108MB (   +0MB)        complete_gated_feature_checking
time:   0.001; rss:  108MB ->  109MB (   +0MB)        early_lint_checks
time:   0.000; rss:  109MB ->  109MB (   +0MB)        prepare_outputs
time:   0.018; rss:  109MB ->  113MB (   +4MB)        looking_for_entry_point
time:   0.000; rss:  113MB ->  113MB (   +0MB)  looking_for_derive_registrar
time:   0.000; rss:  114MB ->  115MB (   +0MB)        unused_lib_feature_checking
time:   0.022; rss:  109MB ->  115MB (   +6MB)        misc_checking_1
time:   0.012; rss:  115MB ->  120MB (   +5MB)        type_collecting
time:   0.000; rss:  120MB ->  120MB (   +0MB)        impl_wf_inference
time:   0.000; rss:  162MB ->  162MB (   +0MB)        orphan_checking
time:   0.405; rss:  120MB ->  189MB (  +69MB)        coherence_checking
time:   0.021; rss:  189MB ->  195MB (   +5MB)        wf_checking
time:   0.147; rss:  195MB ->  226MB (  +31MB)        item_types_checking
time:   0.263; rss:  226MB ->  262MB (  +36MB)        item_bodies_checking
time:   0.849; rss:  115MB ->  262MB ( +147MB)        type_check_crate
time:   0.004; rss:  262MB ->  262MB (   +0MB)        match_checking
time:   0.003; rss:  262MB ->  262MB (   +0MB)        liveness_checking
time:   0.007; rss:  262MB ->  262MB (   +0MB)  misc_checking_2
time:   0.208; rss:  262MB ->  286MB (  +24MB)        MIR_borrow_checking
time:   0.000; rss:  286MB ->  286MB (   +1MB)        MIR_effect_checking
time:   0.006; rss:  287MB ->  288MB (   +2MB)        crate_lints
time:   0.008; rss:  288MB ->  289MB (   +1MB)        module_lints
time:   0.014; rss:  287MB ->  289MB (   +3MB)  lint_checking
time:   0.007; rss:  289MB ->  289MB (   +0MB)        privacy_checking_modules
time:   0.000; rss:  289MB ->  289MB (   +0MB)  check_lint_expectations
time:   0.023; rss:  286MB ->  289MB (   +3MB)  misc_checking_3
time:   0.001; rss:  289MB ->  290MB (   +0MB)        monomorphization_collector_root_collections
time: 111.013; rss:  290MB ->  199MB (  -91MB)        monomorphization_collector_graph_walk
time:   0.345; rss:  199MB ->  271MB (  +72MB)        partition_and_assert_distinct_symbols
time:   0.027; rss:  281MB ->  283MB (   +2MB)        write_allocator_module
time: 265.081; rss:  283MB ->  127MB ( -156MB)        codegen_to_LLVM_IR
time: 376.530; rss:  289MB ->  128MB ( -162MB)        codegen_crate
time:   0.000; rss:  131MB ->  132MB (   +0MB)        serialize_dep_graph
time:   0.197; rss:  132MB ->  244MB ( +112MB)        free_global_ctxt
time:   0.767; rss:  461MB ->  508MB (  +47MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.1)
time:   1.738; rss:  485MB ->  411MB (  -74MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.8)
time:   2.403; rss:  473MB ->  415MB (  -58MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.13)
time:   2.807; rss:  471MB ->  353MB ( -118MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.10)
time:   3.048; rss:  471MB ->  323MB ( -148MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.5)
time:   3.236; rss:  470MB ->  364MB ( -107MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.3)
time:   3.462; rss:  473MB ->  421MB (  -52MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.7)
time:   2.523; rss:  424MB ->  479MB (  +55MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.4)
time:   1.059; rss:  449MB ->  552MB ( +103MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.0)
time:   4.814; rss:  506MB ->  497MB (  -10MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.15)
time:   0.181; rss:  501MB ->  533MB (  +32MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.6)
time:   1.132; rss:  489MB ->  448MB (  -41MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.11)
time:   1.482; rss:  478MB ->  484MB (   +7MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.9)
time:   0.180; rss:  435MB ->  490MB (  +55MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.2)
time:   0.406; rss:  502MB ->  305MB ( -196MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.12)
time:   0.734; rss:  452MB ->  325MB ( -127MB)        LLVM_lto_optimize(notebook.e7b41ac9-cgu.14)
time: 274.643; rss:  396MB ->  351MB (  -46MB)        LLVM_passes
time:   0.001; rss:  143MB ->  116MB (  -27MB)        join_worker_thread
time:  10.600; rss:  244MB ->  111MB ( -132MB)        finish_ongoing_codegen
time:   0.000; rss:   92MB ->   81MB (  -11MB)        serialize_work_products
time:   0.547; rss:   63MB ->   57MB (   -6MB)        run_linker
time:   0.000; rss:   57MB ->   57MB (   +0MB)        link_binary_remove_temps
time:   0.557; rss:   60MB ->   57MB (   -2MB)        link_binary
time:   0.558; rss:   74MB ->   57MB (  -17MB)        link_crate
time:  11.159; rss:  244MB ->   57MB ( -186MB)  link
time: 389.504; rss:   20MB ->   57MB (  +37MB)        total
    Finished release [optimized + debuginfo] target(s) in 10m 57s
time:   0.000; rss:   38MB ->   38MB (   +1MB)        parse_crate
time:   0.000; rss:   38MB ->   38MB (   +0MB)        attributes_injection
time:   0.000; rss:   39MB ->   39MB (   +0MB)  plugin_loading
time:   0.000; rss:   40MB ->   40MB (   +0MB)        setup_global_ctxt
time:   0.000; rss:   41MB ->   41MB (   +0MB)        crate_injection
time:   0.125; rss:   41MB ->   76MB (  +35MB)        expand_crate
time:   0.125; rss:   41MB ->   76MB (  +35MB)  macro_expand_crate
time:   0.000; rss:   76MB ->   76MB (   +0MB)  maybe_building_test_harness
time:   0.000; rss:   76MB ->   76MB (   +0MB)  AST_validation
time:   0.000; rss:   76MB ->   76MB (   +0MB)  maybe_create_a_macro_crate
time:   0.000; rss:   76MB ->   76MB (   +0MB)        finalize_imports
time:   0.000; rss:   76MB ->   76MB (   +0MB)  compute_effective_visibilities
time:   0.002; rss:   76MB ->   79MB (   +3MB)        late_resolve_crate
time:   0.000; rss:   79MB ->   79MB (   +0MB)        resolve_postprocess
time:   0.002; rss:   76MB ->   79MB (   +3MB)  resolve_crate
time:   0.000; rss:   79MB ->   79MB (   +0MB)  complete_gated_feature_checking
time:   0.000; rss:   79MB ->   79MB (   +0MB)        early_lint_checks
time:   0.000; rss:   79MB ->   80MB (   +0MB)        prepare_outputs
time:   0.001; rss:   80MB ->   81MB (   +2MB)        looking_for_entry_point
time:   0.000; rss:   81MB ->   81MB (   +0MB)        looking_for_derive_registrar
time:   0.000; rss:   82MB ->   82MB (   +0MB)        unused_lib_feature_checking
time:   0.002; rss:   80MB ->   82MB (   +3MB)  misc_checking_1
time:   0.001; rss:   82MB ->   85MB (   +3MB)        type_collecting
time:   0.000; rss:   85MB ->   85MB (   +0MB)        impl_wf_inference
time:   0.000; rss:   89MB ->   89MB (   +0MB)        orphan_checking
time:   0.008; rss:   85MB ->  113MB (  +28MB)        coherence_checking
time:   0.001; rss:  113MB ->  116MB (   +3MB)        wf_checking
time:   0.002; rss:  116MB ->  121MB (   +6MB)        item_types_checking
time:   0.010; rss:  121MB ->  135MB (  +13MB)        item_bodies_checking
time:   0.022; rss:   82MB ->  135MB (  +52MB)        type_check_crate
time:   0.000; rss:  135MB ->  135MB (   +0MB)        match_checking
time:   0.000; rss:  135MB ->  135MB (   +0MB)        liveness_checking
time:   0.000; rss:  135MB ->  135MB (   +1MB)        misc_checking_2
time:   0.006; rss:  135MB ->  149MB (  +13MB)        MIR_borrow_checking
time:   0.000; rss:  149MB ->  149MB (   +1MB)        MIR_effect_checking
time:   0.000; rss:  149MB ->  150MB (   +0MB)        crate_lints
time:   0.000; rss:  150MB ->  150MB (   +1MB)        module_lints
time:   0.000; rss:  149MB ->  150MB (   +1MB)        lint_checking
time:   0.000; rss:  150MB ->  150MB (   +0MB)        privacy_checking_modules
time:   0.000; rss:  150MB ->  150MB (   +0MB)        check_lint_expectations
time:   0.001; rss:  149MB ->  150MB (   +1MB)        misc_checking_3
time:   0.000; rss:  150MB ->  151MB (   +0MB)        monomorphization_collector_root_collections
time:   0.474; rss:  151MB ->  317MB ( +166MB)        monomorphization_collector_graph_walk
time:   0.100; rss:  317MB ->  338MB (  +21MB)        partition_and_assert_distinct_symbols
time:   0.000; rss:  340MB ->  341MB (   +1MB)        write_allocator_module
time:   2.957; rss:  341MB ->  622MB ( +281MB)        codegen_to_LLVM_IR
time:   3.567; rss:  150MB ->  622MB ( +471MB)  codegen_crate
time:   0.000; rss:  622MB ->  622MB (   +0MB)  serialize_dep_graph
time:   0.038; rss:  622MB ->  457MB ( -165MB)        free_global_ctxt
time:   0.034; rss:  378MB ->  398MB (  +20MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.0)
time:   0.034; rss:  382MB ->  387MB (   +5MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.15)
time:   0.034; rss:  390MB ->  384MB (   -7MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.5)
time:   0.043; rss:  395MB ->  389MB (   -5MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.6)
time:   0.054; rss:  393MB ->  379MB (  -14MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.4)
time:   0.018; rss:  377MB ->  388MB (  +11MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.1)
time:   0.019; rss:  386MB ->  388MB (   +2MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.2)
time:   0.049; rss:  387MB ->  396MB (   +9MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.7)
time:   0.117; rss:  380MB ->  398MB (  +18MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.9)
time:   0.006; rss:  397MB ->  398MB (   +1MB)  LLVM_lto_optimize(notebook2.21c99ed8-cgu.13)
time:   0.144; rss:  393MB ->  398MB (   +5MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.14)
time:   0.054; rss:  384MB ->  399MB (  +15MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.12)
time:   0.111; rss:  391MB ->  339MB (  -51MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.10)
time:   0.038; rss:  395MB ->  318MB (  -76MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.11)
time:   0.007; rss:  340MB ->  319MB (  -21MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.3)
time:   0.022; rss:  377MB ->  302MB (  -75MB)        LLVM_lto_optimize(notebook2.21c99ed8-cgu.8)
time:   4.385; rss:  369MB ->  302MB (  -67MB)        LLVM_passes
time:   0.000; rss:  292MB ->  287MB (   -5MB)        join_worker_thread
time:   1.635; rss:  457MB ->  287MB ( -170MB)        finish_ongoing_codegen
time:   0.000; rss:  287MB ->  287MB (   +0MB)  serialize_work_products
time:   0.424; rss:  289MB ->  238MB (  -51MB)        run_linker
time:   0.428; rss:  287MB ->  238MB (  -48MB)        link_binary
time:   0.428; rss:  287MB ->  239MB (  -48MB)        link_crate
time:   2.063; rss:  457MB ->  239MB ( -218MB)        link
time:   5.871; rss:   20MB ->   97MB (  +77MB)        total
    Finished release [optimized + debuginfo] target(s) in 3m 34s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants