Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix sccache for CI #2473

Merged
merged 1 commit into from
Jan 11, 2022
Merged

Fix sccache for CI #2473

merged 1 commit into from
Jan 11, 2022

Conversation

tiziano88
Copy link
Collaborator

@tiziano88 tiziano88 commented Jan 10, 2022 •
edited
Loading

Fix sccache for CI

Run locally only instead of using GCS, and then use GitHub Actions cache
to cache the sccache directory and most other Rust target directories.

Caching is only enabled for some steps currently, as there is not enough
space available for the cache entries otherwise. We should look into how
to unify some of our crates separately, and then enable caching for more
steps.

Ref #2474
Fix #2014

@tiziano88 tiziano88 force-pushed the tzn_clippy_cache branch 6 times, most recently from 51159fb to a4367fd Compare January 11, 2022 00:42
@tiziano88 tiziano88 changed the title Cache for Cargo Clippy Fix sccache for CI Jan 11, 2022
@tiziano88 tiziano88 marked this pull request as ready for review January 11, 2022 00:43
@tiziano88 tiziano88 mentioned this pull request Jan 11, 2022
Closed
mariaschett
mariaschett reviewed Jan 11, 2022
View reviewed changes
Copy link
Contributor

@mariaschett mariaschett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Very much looking forward to caching!

Dockerfile

# TODO(#2014): Re-enable sccache once runner runs backend binaries directly rather than using `cargo run`.
# ENV RUSTC_WRAPPER sccache
ENV RUSTC_WRAPPER sccache
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to link #2014 ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

.github/workflows/ci.yaml Outdated
@@ -84,16 +84,33 @@ jobs:
# See https://doc.rust-lang.org/nightly/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci
- name: Cache Rust artifacts
uses: actions/cache@v2
if: |
${{ matrix.cmd == 'run-cargo-clippy' || matrix.cmd == 'run-functions-examples --application-variant=rust' }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did caching work? Because if I understand the last CI run correctly, run-cargo-clippy took still 36 min (https://github.com/project-oak/oak/runs/4769340244?check_suite_focus=true). We'd expecting this to be less, no? But it seems we just kicked off a run again!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got a run with cargo clippy running in 4 mins instead of 40 mins (with no changes to the code), but now it's hard to reproduce because the cache keeps being evicted by other jobs. I think we'll have to submit this and see how it goes to have more useful data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to submit and observe!

But should we maybe note somewhere (e.g., the title), that the sccache is only working for run-cargo-clippy and Rust run-function-examples? We still may want to look into run-cargo-tests, if this works well, no?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG, I think the next thing to explore would be the single workspace though, there is just not enough space available to enable it for another step, otherwise I would do that already.

mariaschett
mariaschett reviewed Jan 11, 2022
View reviewed changes
Copy link
Contributor

@mariaschett mariaschett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

mariaschett
mariaschett approved these changes Jan 11, 2022
View reviewed changes
Copy link
Contributor

@mariaschett mariaschett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@tiziano88
Fix sccache for CI
a749bf0 
Run locally only instead of using GCS, and then use GitHub Actions cache
to cache the sccache directory and most other Rust target directories.

Caching is only enabled for some steps currently, as there is not enough
space available for the cache entries otherwise. We should look into how
to unify some of our crates separately, and then enable caching for more
steps.

Ref project-oak#2474
Fix project-oak#2014
@tiziano88 tiziano88 merged commit 8253435 into project-oak:main Jan 11, 2022
@tiziano88 tiziano88 deleted the tzn_clippy_cache branch January 11, 2022 14:31
@github-actions
Copy link

Reproducibility Index:

a73ba4b220013f3f27dd70de57cc70f908b28eee169840db997be75d9174087a  ./examples/bin/abitest_linear_handles.wasm
75fe1860ac10f39227a4399edceedfe6277b816820fdc69fe498d309e9f1ef71  ./examples/bin/aggregator.wasm
e1f213131e5a3c2e0d5d0f088c1814e2be09cf5f9c121642fff69f4290df73e9  ./examples/bin/chat.wasm
2f8ccbff92c7ac8386bdf58d12ba9866cd20f970a17425b468649c9bfba65464  ./examples/bin/hello_world.wasm
bc7d7f79c2d15bd49c4725f32df3bfa56bdd1acde1cdb702d23b5c9205c5ab97  ./examples/bin/http_server.wasm
64927b21aa5545698cb7e825353fa6dedf5f2ade16ac41b1979c2645c3821876  ./examples/bin/injection.wasm
ef64965f26f1f16ad498c8f7d48487c3768c43d45cd153af95fd33279d6415b3  ./examples/bin/private_set_intersection.wasm
515c45553a8336f300c8b0a54c41a795c9014073b0b6f3aecba880f22c3cfa25  ./examples/bin/private_set_intersection_handler.wasm
bddbed34a63294ef20ce9565a27e3613a72d437c0113b745c13aff00eaa63a1c  ./examples/bin/proxy_attestation_example.wasm
1de856aa0584ef8c04b098479854c7004f30e14655224b8eadc2511146205d5d  ./examples/bin/translator.wasm
f991ec4ec67472d6ca2897f627856babe8fa1893276985a3050567cee50cd153  ./examples/bin/trusted_database.wasm
a929f56a0b8d479fcffb31a777d26643ec88614babddc659cba0d0a375d0958e  ./oak_loader/bin/oak_loader
8315368538d3137a76942e424ca3c56a5157879f2c9c6b9454403fbff4d6d456  ./oak_functions/loader/bin/oak_functions_loader

Reproducibility Index diff:


      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    
















  
  

    

      

  




    


    

      

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



      

    

  




  
          




      

  

    
    

    
  

    Reviewers
  


    


        

          

  
      @mariaschett
  
    mariaschett



          
            
              
            
          
          mariaschett approved these changes

        


      






    

  


      
  

    Assignees
  



        

    No one assigned







      


  


  

    Labels
  



    

    None yet







      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    
      

        
    

    Development
  



          


Successfully merging this pull request may close these issues.


  
    
            
        Re-enable sccache


  


    
  




        
      

    

  
      

  

    

      2 participants
    

    

        
          @tiziano88 
        
          @mariaschett